Evaluation¶

The Lumbar SPIDER Challenge evaluation is based on the Dice similarity coefficient (DICE) score, which is a widely used metric for evaluating segmentation performance. The DICE score measures the overlap between the predicted and reference segmentations and ranges between 0 (no overlap) and 1 (perfect overlap). For this challenge, the DICE score will be calculated for each of the three anatomical structures separately: vertebrae, intervertebral discs (IVDs), and spinal canal. The final ranking will be determined based on the mean DICE score of all structures.

To ensure that all structures are equally important, the ranking will also take into account the mean DICE score for each individual structure. In addition, the ranking will consider the mean DICE score for the vertebrae, intervertebral discs, and spinal canal separately.To determine the final ranking, the mean of the relative ranks of each score will be used. This approach ensures that all scores are weighted equally and that participants who perform well on all structures are rewarded.

Participants will be able to see their results on a leaderboard, which will be updated automatically.