ShortAndFast Posted July 23, 2008 Share Posted July 23, 2008 I would be very interested in knowing whether anyone has ever thought to put together statistical information on each judge which in retrospect might be able to point to an individuals tendencies and biases (or lack of them) Specifically comparing how one marks a corps compared to their peers. Supposing that you had such a data set, what pattern would you look for to establish "bias"? How would you distinguish between an honest judge scoring a corps lower because they had a bad performance and a biased judge scoring a corps lower unfairly? If a judge always agrees with her peers, is that evidence of honesty or of slotting? Statistics can be used to detect bias and fraud in large datasets - the book Freakonomics has a number of good examples - but it can be very hard to find a good analytical approach to reveal it. Seems to me judges must be completely independent. Who do you propose to pay judges to keep them independent of DCI and the corps who comprise it? If the answer is a decentralized group like "the fans", how do you propose that they would define and maintain judging standards and consistency? Quote Link to comment Share on other sites More sharing options...
Scerpella Posted July 23, 2008 Author Share Posted July 23, 2008 (edited) Supposing that you had such a data set, what pattern would you look for to establish "bias"? How would you distinguish between an honest judge scoring a corps lower because they had a bad performance and a biased judge scoring a corps lower unfairly? If a judge always agrees with her peers, is that evidence of honesty or of slotting? Well if a judge is usually scoring identically to their peers it leaves only two possiblities, that either all of them are accurate or all of them are not. It may in fact be slotting but thats another study. So for purposes of this study we ignore it when judges fall into line. But for example if a judge was found to score certain groups higher or lower than others that might be a place to look closely. Arrive at a standard deviation and check to see who exceeds it and in what circumstances on a regular basis. Obviously these things could only be measured every two years or so because it is certainly possible and plausible that one judge may have an uncharacteristic high opinion of one group that others do not. But if you measure across seasons it becomes a bit easier to determine which may be bias and which is a corps just ringing someones bell. Statistics can be used to detect bias and fraud in large datasets - the book Freakonomics has a number of good examples - but it can be very hard to find a good analytical approach to reveal it. Agreed Who do you propose to pay judges to keep them independent of DCI and the corps who comprise it? If the answer is a decentralized group like "the fans", how do you propose that they would define and maintain judging standards and consistency? By independent I mean subject to their own hiring firing and disciplinary actions. Subject to their own review. Still paid for by DCI and contest promoters but not able to be swayed or controlled in any way by DCI. Edited July 23, 2008 by Scerpella Quote Link to comment Share on other sites More sharing options...
Top Cat Posted July 23, 2008 Share Posted July 23, 2008 There's a whole lot of insights I can share on this thread: No, there is no "certification" type program for judges, especially DCI. Past experience (other pagentry activities: BOA, WGI), academic credentials (like music degree where applicable) and who you know gets you in the door to judge for DCI. The current judge administration system is as "independent" as we will ever see. DCI judges are hired by DCI, just like MLB hires umpires, NFL hire referees, etc. There is no "formalized" feedback mechanism for the corps (staffs) to advise DCI judge Administrator John Phillips on what their evaluation is for specific judges. Informally there are constant phone calls, messages, etc. from some staffs about some judges. Bottomline is those that yell the most will get their way, but not all of the time. To review assignments off the recaps to make determinations if someone has been "dropped" is fruitless. Most DCI judges will come and go at their request. Also, there has been and ussually still occurrs a system of rotation, that after a judge has been assigned to the "heavy" shows for awhile (like several years) they are rotated back to ease the pressure on their schedules. If you do want to take the time to plotb out all the - lets say - DCI quarters, semi-finals and finals panels for the last 5-10 years you will see a repeat of certain individuals - those that the corps trust the most, or the administration is most comfortable that they will not "screw" up; some come and are gone - they fill a judging void based on THEIR schedule; one timers - they had their shot and are out or dropped. Quote Link to comment Share on other sites More sharing options...
Bob984 Posted July 23, 2008 Share Posted July 23, 2008 I.E. the same 3 corps winning almost every title for the last.............what........20 years? Oops - Phantom and Madison had one in there. Even then - PR only tied - must have been a glitch.Have those 3 ALWAYS had the top show, with no one else that maybe should have had it? (89 and 06 come to mind..........) It is very clear that if the judges aren't generous to BD, Cavies and Cadets, they are pretty much put out to dry - specifically BD and Cavies. It seems they can get away with dissing the Cadets every now and then. I.E. 2006........notice this year Caders weren't scoring so hot - all of a sudden they are neck and neck with Cavies. Looks like same Top 3 again. Notice 06 GE Visual Judge from finals isn't judging much. Yes Joe Allison is still judging some - even after putting Cavaliers ahead of PR in Music GE. I will even admit - Cavaliers should have won Visual GE, but no way should they have won Music GE in 2006. Not a chance. I was there, and I have a Music Ed degree - so I'm qualified to evaluate that. Wow............Since 2003 - I have a hard time saying any Cavaliers music caption should have been ahead of anything PR or SCV or Crown does. Yay machine sound effects! I couldn't agree more......I'm another music educator.......for anyone to say Cavies have more music effect happening than Crown or Phantom this year; perhaps they need new batteries for THEIR HEARING AID!!!!! While I am at it...check out last night....Crown beats BD in music....guard is BD, now by 1.1 !!???!! It was TWO TENTHS on Saturday!!! I was also there in 2006.....sat there at semis with hall of famers who were shaking their heads when Phantom's 3rd place score was announced....check out Cavies visual numbers at finals in 06......SUDDENLY 19.8 and 19.9......I think the "insurance policy" was "called in" well before showtime. Perhaps if Phantom or Crown let the pit take the melody half of the show, and then spent another minute or two "bending a note" then they could win, too....lmao......... GB Quote Link to comment Share on other sites More sharing options...
silvertrombone Posted July 23, 2008 Share Posted July 23, 2008 Great thread, OP. I have thought about doing some kind of research/comparison but was anticipating waiting until off-season. I'm busy anyways, but at least then I've got this season's numbers to compare as well! However, any comparison would require AT LEAST 5 years worth of comparisons. And since we have many years' worth of data available on Corps Reps, I'll need some help. Anyone game? There will have to be a whole lot of data input--heck, Blue Knights web site may have that stuff able to be up/downloaded to save time. Thoughts on how to set it up? Quote Link to comment Share on other sites More sharing options...
wdlover Posted July 23, 2008 Share Posted July 23, 2008 I have a Music Ed degree - so I'm qualified to evaluate that. HAHAHAHAHAHAHAHAHAHA... 8 out 10 people that hold a music ed degree are barely qualified to teach music, let alone judge music effect. :tongue: Quote Link to comment Share on other sites More sharing options...
Wadep66 Posted July 23, 2008 Share Posted July 23, 2008 (edited) Everyone has biases-- they are impossible to not have, but if there is awareness of them, they can be overcome if there is a desire to. If you have been subjected to diversity training or testing, you know what I'm talking about. In a cursory look at scoring, judges tend to sometimes change the ordinals and the spread between corps. When corps have a down performance, the scores seem to pick up on that). The spread is so tight at the top because of the performance level the top corps acheive by finals. It could easily be argued many years that any of the top 2 or 3 could have deserved victory. If it is not possible to break the top corps, then SCV would never have dropped as low as they have and even Madison has won some championships. When I was in HS Band, there was something about our style that judges either loved or hated, sometimes to the point that 3 of 5 judges would have us in the top three and the others near last place. Who was right? From what I see, the biases on this board are far stronger than the biases of the judges will ever be. If I had all kinds of time on my hand, I'd collect the data and use a statistical model to determine if the data supports this argument, but alas, I don't have the many hours or patience that would take. I just settle for the outcome of championships and gripe about it for the next few years like everyone else here. Edited July 23, 2008 by Wadep66 Quote Link to comment Share on other sites More sharing options...
corps fan Posted July 23, 2008 Share Posted July 23, 2008 perfect...and cheap...way to fix all the judging woes: we get rid of all judges and have an applause-o-meter as the determiner of winners. you only have to pay for the machines, electricity to run them, and someone to drive them around...or make the show coordinator responsible for that. gives fans overall control of who wins, and actually gives merit to what "effect" is supposed to be...how the show relates to the crowd. Quote Link to comment Share on other sites More sharing options...
Medeabrass Posted July 23, 2008 Share Posted July 23, 2008 www.fromthepressbox.comSeems to me judges must be completely independent. This is such a great observation!!! You put into words what I think has been bothering me in the back of my mind. How do we think the NBA or (any sports league for that matter) would function if the top echelon of teams had the ability to determine the fate of the refs and how they call games? I think there would be a danger that the top teams would get more favorable calls over the lower tier teams. At the very least, it wouldn't look good. Quote Link to comment Share on other sites More sharing options...
Medeabrass Posted July 23, 2008 Share Posted July 23, 2008 HAHAHAHAHAHAHAHAHAHA... 8 out 10 people that hold a music ed degree are barely qualified to teach music, let alone judge music effect. :tongue: I've known hospital administrators who've made great brass caption heads so there's some truth to this statement. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.