Scores change, but the rate of improvement stays the same

[crest99]

In a moment of extraordinary boredom today, I compared the World Class scores from San Antonio and Allentown. I frankly have been a little frustrated with my old corps' stalled score, and I wanted to see how its number over the past couple of weeks compared with other units. What I found more or less proved my personal grumbling baseless. In fact, what surprised me was how little variation there was in the increase in each corps' score, especially between the top and bottom corps.

To explain a little more, the average increase in score for 22 WC corps between San Antonio and Allentown (not including Surf, which was not in TX) was 4.06 points. The high was 5.15 (well done Mandarins) and the low was 3.05 (Pioneer). Obviously 2 points is a pretty big spread. But the weird thing (at least in my mind) is that the average increase in score among the top 7 (4.16) is virtually the same as the bottom 7 (4.11). The same is true when you compare the current top 12 (4.02) with the bottom 10 (4.12).

I know judging in this activity is not only subjective but also relative, which is to say that scores generally have more to do with the differences between the corps before and after you than a nominal figure. Still, intuitively one would think that the rate of improvement would decrease - even just a little bit - as you move towards the top. In plain English, I would have expected to see the improvement among the top corps be smaller than that of the bottom corps, especially as the top units start to max out the sheets. I was under the impression that it's harder to go from a 94 to a 95 than from a 74 to a 75.

A few caveats here: I know Allentown was 2 nights. But it was back to back and I would guess that even if you used different data the trend would hold up. Also, I should add that I think the judges are good and honest people, and I'm not trying to beat the slotting horse or imply any kind of conspiracy or cabal. Please take unfounded allegations elsewhere.

S.A. Allentown Increase

Blue Devils 90.95 95.25 4.3

Cadets 90.4 95.2 4.8

Cavaliers 91.05 94.85 3.8

Crown 89.55 93.7 4.15

Phantom 87.4 91.5 4.1

SCV 86.15 90.6 4.45

Bluecoats 86.6 90.15 3.55

Boston 84.55 88.25 3.7

Madison 83.8 87.2 3.4

BK 82.9 86.7 3.8

Blue Stars 82.9 86.1 3.2

Spirit 79.05 84 4.95

Glassmen 78.6 82.4 3.8

Troopers 78.55 83.05 4.5

Academy 77.75 81.8 4.05

Colts 76.55 80.65 4.1

Crossmen 75.8 79.85 4.05

PC 73.3 76.95 3.65

Mandarins 70.45 75.6 5.15

Teal 70.65 74.9 4.25

Cascades 68.45 73 4.55

Pioneer 66 69.05 3.05

Edited August 10, 2011 by crest99

[ouooga]

Alright, I'm a stats nerd and I find that really interesting. I would think that if you had more room to improve, you'd improve more than the group who had less room to improve.

So if that's the case, do corps on the bottom end even have a shot? I mean, if you come out of the box somewhere between 13-17, is top 6 just completely out of the question? Or is it that all corps just work (improve) about equally as hard, and if a corps really did want to step it up they could?

[Peel Paint]

True this year and true more often than not in most recent years. Movement in overall placement after mid-July is more the exception than the rule. It doesn't mean changes in placement won't happen this week, especially with tightly clustered corps, but it's rare to see it happen breaking out well above or below a tight cluster with more than a few corps.

It's not a judging conspiracy. Corps polish their shows at about the same rate, IMO.

Edited August 10, 2011 by Peel Paint

[crest99]

True this year and true more often than not in most recent years. Movement in overall placement after mid-July is more the exception than the rule. It doesn't mean changes in placement won't happen this week, especially with tightly clustered corps, but it's rare to see it happen breaking out well above or below a tight cluster with more than a few corps.

It's not a judging conspiracy. Corps polish their shows at about the same rate, IMO.

Even if you stipulated that corps are roughly equally good at cleaning, one would expect that in general as a corps gets better (especially among the very top groups, late in the season) there is less to clean, the remaining issues are smaller in scope, and that consequently they would receive less credit for these improvements. So you'd think a corps outside the top 17 that was still fixing big things would see bigger jumps in their score down the home stretch than a top unit that's basically polishing a finished product.

Change in placement (or the lack thereof) is an interesting, but slightly different, issue.

[emc2]

In a moment of extraordinary boredom today, I compared the World Class scores from San Antonio and Allentown. I frankly have been a little frustrated with my old corps' stalled score, and I wanted to see how its number over the past couple of weeks compared with other units. What I found more or less proved my personal grumbling baseless. In fact, what surprised me was how little variation there was in the increase in each corps' score, especially between the top and bottom corps.

To explain a little more, the average increase in score for 22 WC corps between San Antonio and Allentown (not including Surf, which was not in TX) was 4.06 points. The high was 5.15 (well done Mandarins) and the low was 3.05 (Pioneer). Obviously 2 points is a pretty big spread. But the weird thing (at least in my mind) is that the average increase in score among the top 7 (4.16) is virtually the same as the bottom 7 (4.11). The same is true when you compare the current top 12 (4.02) with the bottom 10 (4.12).

I know judging in this activity is not only subjective but also relative, which is to say that scores generally have more to do with the differences between the corps before and after you than a nominal figure. Still, intuitively one would think that the rate of improvement would decrease - even just a little bit - as you move towards the top. In plain English, I would have expected to see the improvement among the top corps be smaller than that of the bottom corps, especially as the top units start to max out the sheets. I was under the impression that it's harder to go from a 94 to a 95 than from a 74 to a 75.

A few caveats here: I know Allentown was 2 nights. But it was back to back and I would guess that even if you used different data the trend would hold up. Also, I should add that I think the judges are good and honest people, and I'm not trying to beat the slotting horse or imply any kind of conspiracy or cabal. Please take unfounded allegations elsewhere.

S.A. Allentown Increase

Blue Devils 90.95 95.25 4.3

Cadets 90.4 95.2 4.8

Cavaliers 91.05 94.85 3.8

Crown 89.55 93.7 4.15

Phantom 87.4 91.5 4.1

SCV 86.15 90.6 4.45

Bluecoats 86.6 90.15 3.55

Boston 84.55 88.25 3.7

Madison 83.8 87.2 3.4

BK 82.9 86.7 3.8

Blue Stars 82.9 86.1 3.2

Spirit 79.05 84 4.95

Glassmen 78.6 82.4 3.8

Troopers 78.55 83.05 4.5

Academy 77.75 81.8 4.05

Colts 76.55 80.65 4.1

Crossmen 75.8 79.85 4.05

PC 73.3 76.95 3.65

Mandarins 70.45 75.6 5.15

Teal 70.65 74.9 4.25

Cascades 68.45 73 4.55

Pioneer 66 69.05 3.05

Now look at the past to present unweighted score (2003 to 2010) get the avg total points for each corp from the 1st show to the last show in each year and compare placement. Have fun

[soccerguy315]

So if that's the case, do corps on the bottom end even have a shot? I mean, if you come out of the box somewhere between 13-17, is top 6 just completely out of the question? Or is it that all corps just work (improve) about equally as hard, and if a corps really did want to step it up they could?

this is why it is important to be slotted well to start the season.

[Tez]

While the better corps have less to clean, they generally add more to their shows near the end of the year.... This can increase GE and/or add in more demand thus increasing repertoire scores faster than the lower corps.

[audiodb]

Even if you stipulated that corps are roughly equally good at cleaning, one would expect that in general as a corps gets better (especially among the very top groups, late in the season) there is less to clean, the remaining issues are smaller in scope, and that consequently they would receive less credit for these improvements. So you'd think a corps outside the top 17 that was still fixing big things would see bigger jumps in their score down the home stretch than a top unit that's basically polishing a finished product.

No....that can't happen. Scores are not a linear measure of quality. They are just numbers that judges use to rank and rate the corps. Since there aren't many numbers to work with, judges must deviate from a strict linear "rate" in order to ensure they "rank" correctly. The process is known as "numbers management".

[skywhopper]

Even if you stipulated that corps are roughly equally good at cleaning, one would expect that in general as a corps gets better (especially among the very top groups, late in the season) there is less to clean, the remaining issues are smaller in scope, and that consequently they would receive less credit for these improvements. So you'd think a corps outside the top 17 that was still fixing big things would see bigger jumps in their score down the home stretch than a top unit that's basically polishing a finished product.

I think the flaw in this argument is the idea that the numerical values of the scores actually have meaning. They don't.

[Kamarag]

No....that can't happen. Scores are not a linear measure of quality. They are just numbers that judges use to rank and rate the corps. Since there aren't many numbers to work with, judges must deviate from a strict linear "rate" in order to ensure they "rank" correctly. The process is known as "numbers management".

Exactly. Also, tolerances change as the season goes on. A 75 in late June and a 75 in early August aren't the same. The 75 in August is judged much more harshly, which is why the corps at the bottom end tend to stagnate in score, despite major improvements late in the season. It's a shame that happens, but it does. In most cases, the lower end corps improve to a much greater degree than the corps at the top over the course of the season, it's just not reflected in the numbers.

Edited August 11, 2011 by Kamarag