sadya
02-17-2006, 02:52 AM
[source:http://www.sciencedaily.com/releases/2006/02/060215230842.htm]
John W. Emerson, assistant professor of Statistics at Yale, using information found on the web for an exercise in his classroom, examined the results of the recent European Women's Figure Skating Competition and identified a potentially serious flaw in the system for selecting the judging panel.
(......)
The Computer: A Phantom Figure Skating Judge?
John W. Emerson
Assistant Professor of Statistics
Yale University
(.......)
Random elimination of three judges results in 220 possible combinations of nine-judge panels. However, only one panel actually determines the outcome. An examination of the Ladies' 2006 European Figure Skating Championships illustrates the problem.
The Short Program was a close competition between four of the top five skaters: Irina Slutskaya (66.43), Elena Sokolova (60.88), Sarah Meier (60.87), Elena Gedevanishvili (60.19), and Carolina Kostner (60.04). The scores were calculated after a computer randomly excluded judges 4, 6, and 11, whose identities and nationalities are unknown.
Only 50 of the 220 possible panels would have resulted in the same ranking of the skaters following the Short Program. Scores calculated using all of the twelve judges would have resulted in the same ranking, but with slightly different numerical scores.
Random elimination of a different set of judges could have radically changed these standings. Only Slutskaya's standing was secure; each of the other skaters could have placed as high as 2nd or as low as 5th in the Short Program. If the scores had been similarly close following the Free Skate (they were not, fortunately), the medal standings would have been determined by the random selection of the panels of judges.
The following graphs show the distribution of Short Skate rankings for each of the top 5 finishers, based on 220 possible panels of judges. Each of these panels awarded the highest score to Slutskaya. Meier was particularly lucky: while she placed 3rd, more than half of the possible panels would have placed her in 4th or 5th position. Conversely, Gedevanishvili, who placed 4th, was particularly unlucky -- more than half of the possible panels would scored her in 2nd or 3rd position. Even Kostner, in 5th place, would have been ranked 2nd or 3rd by about one-third of the panels.
Imagine a similarly close competition for the Olympic medals in Torino, Italy.
I hope I never have to hear a 4th or 5th place finisher give the following interview: "I did my best, and I would have won Bronze if all twelve judges' scores had been included. And if a different panel of 9 judges had been selected, I might have won Gold."
We can only hope that the podium in Torino on February 23 will be determined by the judging of the skaters on the ice. Not by a computer.
Statistics: http://www.stat.yale.edu
John W. Emerson: http://www.stat.yale.edu/people/jayemerson.html
Supplementary information available: http://www.stat.yale.edu/~jay/
John W. Emerson, assistant professor of Statistics at Yale, using information found on the web for an exercise in his classroom, examined the results of the recent European Women's Figure Skating Competition and identified a potentially serious flaw in the system for selecting the judging panel.
(......)
The Computer: A Phantom Figure Skating Judge?
John W. Emerson
Assistant Professor of Statistics
Yale University
(.......)
Random elimination of three judges results in 220 possible combinations of nine-judge panels. However, only one panel actually determines the outcome. An examination of the Ladies' 2006 European Figure Skating Championships illustrates the problem.
The Short Program was a close competition between four of the top five skaters: Irina Slutskaya (66.43), Elena Sokolova (60.88), Sarah Meier (60.87), Elena Gedevanishvili (60.19), and Carolina Kostner (60.04). The scores were calculated after a computer randomly excluded judges 4, 6, and 11, whose identities and nationalities are unknown.
Only 50 of the 220 possible panels would have resulted in the same ranking of the skaters following the Short Program. Scores calculated using all of the twelve judges would have resulted in the same ranking, but with slightly different numerical scores.
Random elimination of a different set of judges could have radically changed these standings. Only Slutskaya's standing was secure; each of the other skaters could have placed as high as 2nd or as low as 5th in the Short Program. If the scores had been similarly close following the Free Skate (they were not, fortunately), the medal standings would have been determined by the random selection of the panels of judges.
The following graphs show the distribution of Short Skate rankings for each of the top 5 finishers, based on 220 possible panels of judges. Each of these panels awarded the highest score to Slutskaya. Meier was particularly lucky: while she placed 3rd, more than half of the possible panels would have placed her in 4th or 5th position. Conversely, Gedevanishvili, who placed 4th, was particularly unlucky -- more than half of the possible panels would scored her in 2nd or 3rd position. Even Kostner, in 5th place, would have been ranked 2nd or 3rd by about one-third of the panels.
Imagine a similarly close competition for the Olympic medals in Torino, Italy.
I hope I never have to hear a 4th or 5th place finisher give the following interview: "I did my best, and I would have won Bronze if all twelve judges' scores had been included. And if a different panel of 9 judges had been selected, I might have won Gold."
We can only hope that the podium in Torino on February 23 will be determined by the judging of the skaters on the ice. Not by a computer.
Statistics: http://www.stat.yale.edu
John W. Emerson: http://www.stat.yale.edu/people/jayemerson.html
Supplementary information available: http://www.stat.yale.edu/~jay/