Enough Judging Conspiracy Theories

[Jeff Ream]

14 hours ago, BDCorno said:

Fart, you're making my head hurt. I've judged both dog shows and drum corps, so try and follow along here. Dog breeds do have "manuals" (breed standards) that physically describe the structure of a dog. However, it also talks about temperament, carriage, attitude in the ring and interaction with other dogs, and most importantly, TYPE. This isn't really a whole lot different than the process that you would go through in judging drum corps (or band, or horses, or cats)...The written standards (sheets, criteria, whatever it might be called in what you're judging) are only adequate in that they give you a general and technical idea of what is desirable. Sadly, this is where many people stop...and it's where the "SCV marched and played for 56 seconds at 208 bpm, and BD only marched and played for 44 seconds at 192...how are they still winning" mentality comes from. Judging experience, background and training should yield an evaluator with the ability to look beyond the written standard/sheets/criteria. Most any yokel can say '"well, they checked all the boxes" and still put up the wrong dog/corps/horse/cat. It's the ability to detect, appreciate, and reward QUALITY of design and achievement. Clearly, some of the design quality, depth and less obvious demands that top corps are employing these days goes right over the head of many spectators/fans, as well as the individual performance quality/training in some instances. THAT is where I think you're missing the boat in understanding what's going on here. 

As I think I've pointed out, a .4 difference in spread isn't an overly significant or out of line scoring variance show to show, even among similarly proficient units with similar content. Corps #1 is up by .2 on Friday, say with a 98/98 to corps #2's 97/97. Next night, corps #2 does a lights out show and go 98/98 to corps #1 who does a flat performance and a 97/97. Judges have to RANK first of all, and .1 is a typical margin. If they decide that there is a more decisive advantage in one subcaption, there is your .5. DON"T FORGET that unless it's GE, the score is halved in the final total, so your .4 or .6 variance is effectively only .2 or .3. It happens. Corps have good and bad nights, judges may or may not have the ability to detect and reward the depth or quality of design or performance (due to background and/or training), or the timing of major flaws impacted the effectiveness of the program (analysis captions). It's hardly simplistic. I find it interesting that you seem to allege preference or bias, yet you clamor for complete consistency of scoring. The complexity of a modern drum corps show, and trying to get 150 to perform as one, does not lend itself to total consistency, though the better groups are usually more consistent night to night than less competent groups. Asking for consistency in scoring night to night, performance be ######, is the ultimate bias and preference. Do you see where this is a contradiction? 

As a breeder, exhibitor and judge of dogs; and as a musician, teacher and judge of drum corps, winter guards and bands; I subscribe to the proposition that judges need to start and end their evaluation process with type/quality as the main consideration. Are the technical requirements/standards important? Certainly, but not in a vacuum, and definitely not as overriding factors that cancel out or ignore type/quality as the aim of the process. I've seen dogs that "check all the boxes" when you look at their structure in a static position. However, on the move, the movement doesn't validate the structure for a variety of reasons. Conditioning, attitude, poor handling, and other factors can turn a diamond of a canine specimen into a lump of coal. Same with corps. OK, they march and play a lot, do some cool drill moves, high notes, neat drum solo, color guard has nice impact points...but if the show isn't structured well, performers don't have the quality of movement or musicianship (though marching and playing things technically clean), then the check boxes don't validate the quality/type that you're looking for. It's the box 4/box 5 conundrum, and as with dog judges, DCI judges can fall into the pedantic approach, rather than to appreciate intrinsic performance quality as the overriding aim of the process. Can't say I saw any evidence of that during DCI week, at least among the top half of the finals lineup. Seem pretty obvious that you did, but I think it's more a matter of perspective than some type of conspiracy or bias.

Oh, and by the way, bull riding is more than staying on for the appointed time. You do realize that the quality of the ride is scored too? Gee whillikers.

i heart you platonically

[Jeff Ream]

8 hours ago, rpbobcat said:

After reading through all of the posts on this thread i have a a couple of questions and a comment:

The questions:

1.Is there some way to see the "resumes" of DCI's judges ?

2.Is there any place to see what DCI's requirements  are to be a judge ?

3.Is there any place to see what DCI's training/testing of judges  consists of.

Comments:

1.I'm an engineer and land surveyor.

I had obtain several years of acceptable experience,then take  licensing exams to practice.

I don't know if DCI actually has a "test" for judges.

My suggestion,have a several  judges do "sheets" for the same show,

If they are properly trained,and "objective",their sheets should all be about the same for each corps.

2.I keep hearing about people becoming judges.

Other then people in education,or possibly retirees, I don't know many people,including myself, who could work their full time job and judge contests all summer.

 

 

  

 

knowing someone who applied to judge and recently passed, there is in place a formal training program. They do several webinars a year as well as in person updates during the season, and any prospective judge has to attend several shows and do recordings and then submit those and numbers to go along with them. I doubt you can find the specifics online, but you can contact the judging coordinator and they may be able to give you a lot of info.

 

but no, you will not see sheets all be the same. Some may score high. Some may score low. Not all will see and hear every exact thing the same way leading to their numerical rankoing and rating.

 

Just as an umpires strike zone is different or some NFl refs call holding different, a judge in DCI will not read every show exactly the same way another judge did.

[ContraFart]

1 hour ago, Stu said:

With this posting of yours, I am now laughing so hard I might bust my spleen concerning your vehement OMG .5 variance complaint!!!!

That was just an example, try and understand the bigger picture I am trying to paint.

[Jeff Ream]

6 hours ago, barigirl78 said:

Did you feel the way last year when they decided not to release recaps?  We have no skin in the game, why do we need to see them?

i cared because I like to see how the meat is ground up. I use DCi, and even DCA recaps as a learning tool. When I watch a show, based on personal tastes, i may not always get why corps A scored this and corps B scored this. But then I go look at the recaps, and the picture becomes clear. Unlike many i dont focus on the total caption score...the real story is told in the sub boxes. if someone in July is 86/84 and someone else is 85/81 on a sheet, the rationale speaks volumes. Close in content/rep, but it's the performance thats opening up the gap.

 

thats why I cared. 97% of the time i dont even look at the names on the recap of the judges, i just look at the math. And to be honest, after the second week of shows, it was the fun of chasing them just to see how many people in DCI were willing to hand them out on the sly despite the edict to hide them, because really, the way DCi rolled it out in their initial PR statement was nothing more than a "#### you" to the credibility of the judges, which IMO, was horribly uncalled for. 

[Jeff Ream]

6 hours ago, ContraFart said:

But my argument is that there is far less objective criteria in DCI than in any other subjective scoring sport. There is no perfect model to compare a drum corps show to, in other sports there are textbook definitions of how things are supposed to be done. A triple axle is a triple axle, there is an objective definition. A trained eye can better assess the launch angle, the rotation speed, the foot position, etc, but the naked eye of a layman can at least know it was a triple axle or a triple lutz and know that its a required part of their show. Drum corps fans cannot agree on difficulty content or physical visual demand.

For example BD and Crown use almost fundamentally different brass techniques. There is a different sound, a different balance, a different clarity of tone, different technical aspects and neither one is right or wrong, its a matter of preference. This is why I say the difference between the top corps is only a matter of degrees. Where on the sheets will it definitively say that Crowns Brass was better than BDs? I think there can be reasonable cited arguments that can defend either. This is why I keep going back to the fact the the judge assigned to the caption is more important than the performance on the field.

and even in ice skating, you have subjective calls made. like, oh, every week despite text book stuff

[Jeff Ream]

6 hours ago, ContraFart said:

Every once in a while there is art that moves people in a way that its imperfections are dismissed. 2009 Crown is that show for me.

but yet you acknowledge imperfections, thereby undercutting your own argument

[Jeff Ream]

4 hours ago, BDCorno said:

Yes. You hit on what I'm trying to get across...there is no way to eliminate the subjective from the judging system. Objective criteria are a somewhat clunky way of describing aspects of something artistic, "beautiful", or with a high degree of complexity/thought inherent in design. The simple presence of the objective criteria is no assurance of a competitive, successful program. Your average guy might sit down with a sketch artist and describe your ideal woman in terms of looks, and after giving all the particulars, I doubt what you end up with will be what you'd expect. Being able to detect depth/quality of design, training/quality of movement and musical from the performers, and whether the program has clear intent that is successfully communicated/structured, are all aspects that can't be described by objective criteria. I know it's maddening for those that can't see past the objective criteria, which causes endless discussion and complaints. While I don't care for the fact that DCI sort of hides the sheets, making them public will simply feed more of the criticism and nattering over descriptive elements that are less important than an overall perspective. It's sort of a no-win spot for DCI. Hey, I'm sure you can see from my moniker where I'm from, and I will rightly criticize my compatriots of similar background when they go off a cliff at times over a JK or MR giving them (or a competitor) a score that they don't like. I'm a bit more pragmatic about it, and I'm not going to indulge conspiracy theories unless really blatant...which is rare. Politics in DCI judging? Yeah, there is a bit, but it's not really related to what's on the field. Do judges have biases? Sure, everyone does, and a good judge overrides/deals with their biases in the evaluation and training processes. I will say that there are some judges that are better able to appreciate depth of design and quality than others. I surely hope that the training leads them all to a higher place where they can deal with that issue...that's about all you can ask.

here they showed the GE sheet when the new system launched:

 

http://www.dci.org/ViewArticle.dbml?DB_LANG=C&DB_OEM_ID=33500&ATCLID=209936977&SPID=166025&SPSID=965782

 

oh here's all of them

 

http://www.dci.org/ViewArticle.dbml?DB_LANG=C&DB_OEM_ID=33500&ATCLID=209937964&SPID=166025&SPSID=965782

[Jeff Ream]

4 hours ago, ContraFart said:

Now you are just being an as*hole

nah, that's tame for him.

[Jeff Ream]

3 hours ago, BDCorno said:

OK, I'll just finish out my comments by relating a judges training story from a seminar with Mike Rubino years ago:

All the brass judges we had at the seminar were shown about a half dozen shows, and the shows reflected everything from the finest groups of the day to beginner-level. We were to listen to the brass performances and give a rank/score to each one. At the conclusion, we made a big chart on the blackboard and filled in our scores. It was quickly apparent that there is a clear phenomenon in play...the better the group, the less variance in score, the less disagreement in terms of rank and rate. At the top, errors pop right out at you off of a spotless table of performance. The beginning level groups had a multiplicity of issues, from intonation to tone quality to accuracy, many times layered one upon the other. Of course, the better the group, the more consistent the performance. Lesson learned here? There is less disagreement at the top (aka "slotting" to the conspiracy theorist), while the lesser groups have to endure big swings in score and sometimes rank, due to their many shortcomings. It's not as easy now, as the quality of performers has risen at the top tier to ridiculous levels. A single clipped note, late release or other minor defect not detectable from the stands might make the difference between a caption win and a loss. It's an interesting dynamic...just thought I'd relate this as a useful dynamic to consider.

that is indeed the way it goes.

[Jeff Ream]

3 hours ago, Bobby L. Collins said:

It's like watching a blind and deaf cat chase its  tail.  You want to step in and calm the cat down, but then you realize that this is really all the cat has to do in life to entertain itself, besides grooming......

did you get your tail yet?