Judging/Scoring Tutorial?

[IllianaLancerContra]

4 hours ago, DawsonBurnes said:

I'm just confused on how judges quantify a performance. Is one note crack a loss of half a point, or something like that? And are they judged like AP tests (points added to the score based on their achievement), or do they start at the top score for the caption and take away points? 

In the old days (prior to 1984) yes, a mistake resulted to a loss of 0.1 (or more depending on how many made the mistake) from the relevant execution caption (brass, percussion, marching).  the 0.1 point was called a "tic", and the judging system from then is often referred to as the 'tic era'.  The general effect & program difficulty captions were build-up, much as today.  

One result, unintended or not, is that back under the tic system, the casual fan could tell whether a corps was having a good run or not.  Today, it is all in the hands of the judges, and they are somehow expected to know more than the casual fan what is good or not.

Edited June 2, 2017 by IllianaLancerContra
more better English...

[Stu]

2 hours ago, IllianaLancerContra said:

In the old days (prior to 1984) yes, a mistake resulted to a loss of 0.1 (or more depending on how many made the mistake) from the relevant execution caption (brass, percussion, marching).  the 0.1 point was called a "tic", and the judging system from then is often referred to as the 'tic era'.  The general effect & program difficulty captions were build-up, much as today.  

One result, unintended or not, is that back under the tic system, the casual fan could tell whether a corps was having a good run or not.  Today, it is all in the hands of the judges, and they are somehow expected to know more than the casual fan what is good or not.

You do realize that even a ‘tic’ system is still a 100% fuzzy logic subjective adjudicating process. By that I mean that no matter how well defined the sheets, there is absolutely no objective situation at all, ever, in which a human can define when a note, a foot placement, a horn angle, an implement spin/toss, an interval between marchers, crosses over from being executed correctly to becoming a tic.  And while there are some subjective situations like that in objective scored sports, the winner in say Down Hill Skiing is defined by completely objective means apart from human opinion (the one who crosses the finish line in the fastest time wins).  Whereas in competitions like DCI, Ice Dancing, Cheer Leading, et al, the contests that only have adjudicators ranking scores based on shear observation, the ranking and scoring outcome is purely and 100% based on the opinion of adjudicators (ie the winner is completely subjective in nature).  This 100% subjective aspect exists in DCI no matter how well defined the scoring sheets, no matter a tear-down (tic) system or a build-up (current sheets) system, because it boils down to shear, albeit educated, but still shear human opinion.

[ShortAndFast]

6 hours ago, dcsnare93 said:

It's not as daunting as it may seem. Not all 40 groups are in the same scoring range. There's still some numbers management when it comes to ranking within boxes, but the judge will have a good impression from the performance alone of what rating they're going to assign.

I would think the challenge is more along the lines of: "It's 2:30 and I'm watching SCVC and BDB. I know what box/subbox to put them in and what spread they deserve, but I also know in 3 hours I'll be seeing World Class groups who are likely at a very comparable level. So I need to leave enough space to assign the correct ranking after I see those groups. And of course I need to accurately remember what I saw 3 hours ago so that I can correctly rank all the similar groups."

But I certainly buy that the judges who get asked to do Prelims are pros at this, and that many of the scores in Prelims aren't that tough to assign.

[xandandl]

4 minutes ago, ShortAndFast said:

I would think the challenge is more along the lines of: "It's 2:30 and I'm watching SCVC and BDB. I know what box/subbox to put them in and what spread they deserve, but I also know in 3 hours I'll be seeing World Class groups who are likely at a very comparable level. So I need to leave enough space to assign the correct ranking after I see those groups. And of course I need to accurately remember what I saw 3 hours ago so that I can correctly rank all the similar groups."

But I certainly buy that the judges who get asked to do Prelims are pros at this, and that many of the scores in Prelims aren't that tough to assign.

Thank God for tote sheets.

[dcsnare93]

2 hours ago, ShortAndFast said:

I would think the challenge is more along the lines of: "It's 2:30 and I'm watching SCVC and BDB. I know what box/subbox to put them in and what spread they deserve, but I also know in 3 hours I'll be seeing World Class groups who are likely at a very comparable level. So I need to leave enough space to assign the correct ranking after I see those groups. And of course I need to accurately remember what I saw 3 hours ago so that I can correctly rank all the similar groups."

But I certainly buy that the judges who get asked to do Prelims are pros at this, and that many of the scores in Prelims aren't that tough to assign.

Yes, this happens, but is also where tote sheets come in handy. I'd argue its harder in a random draw situation, than say prelims though.

I once judged a random draw contest with 42 units, and the highest number I gave out was toward the middle of the contest. It gets a bit more tricky towards the latter part of the contest where groups are more similar but you want to leave room for what follows. Managing subs is a learned skill, one judges' adjudicating DCI regionals and finals are quite adept at doing.

Edited June 3, 2017 by dcsnare93

[Brian Porter-Szucs]

The original post is a great question, and I don't think it has been answered yet. In a sport like gymnastics, a good TV broadcast will have an expert who will offer insight on why a particular routine got scored higher or lower than another.  We might learn, for example, to watch how a foot gets placed on the bar, or whether a knee bends more than a certain amount on a particular move.  Viewers might be told that a particular flip was done with significantly more elevation than another flip, thus making it more impressive.  It would be magnificent to have some insight of this nature about what constitutes "excellence" in the various DCI captions.  I know that judging is subjective, but it isn't random.  The judges are clearly doing more than saying to themselves "that was cool!"  Some of the stuff they are looking for is obvious to even a casual fan: a snare line that isn't quite in sync, a dropped flag, a musician who plays a wrong note, a marching line that goes ragged. But very little of that obvious stuff is evident among the really good corps.  We sometimes read comments on this forum about a show being "dirty" or a particular night's performance being unusually excellent.  Clearly the people who say these things have specific observations in mind.  I realize that are probably a thousand small things, so maybe someone knows of a resource online (or even in print--yes, I'm a dinosaur who still sometimes turns to books) that can offer some help. I guess I"m looking for the sorts of specifics that go beyond vague comments like "corps X had a higher level of difficulty" or "corps Z had more innovative design" or "corps Y has a much better sound."  

Or maybe I'm wrong, and it really does all come down to some judges saying "that was cool!" :)

[mjoakes]

Just now, Brian Porter-Szucs said:

The original post is a great question, and I don't think it has been answered yet. In a sport like gymnastics, a good TV broadcast will have an expert who will offer insight on why a particular routine got scored higher or lower than another.  We might learn, for example, to watch how a foot gets placed on the bar, or whether a knee bends more than a certain amount on a particular move.  Viewers might be told that a particular flip was done with significantly more elevation than another flip, thus making it more impressive.  It would be magnificent to have some insight of this nature about what constitutes "excellence" in the various DCI captions.  I know that judging is subjective, but it isn't random.  The judges are clearly doing more than saying to themselves "that was cool!"  Some of the stuff they are looking for is obvious to even a casual fan: a snare line that isn't quite in sync, a dropped flag, a musician who plays a wrong note, a marching line that goes ragged. But very little of that obvious stuff is evident among the really good corps.  We sometimes read comments on this forum about a show being "dirty" or a particular night's performance being unusually excellent.  Clearly the people who say these things have specific observations in mind.  I realize that are probably a thousand small things, so maybe someone knows of a resource online (or even in print--yes, I'm a dinosaur who still sometimes turns to books) that can offer some help. I guess I"m looking for the sorts of specifics that go beyond vague comments like "corps X had a higher level of difficulty" or "corps Z had more innovative design" or "corps Y has a much better sound."  

Or maybe I'm wrong, and it really does all come down to some judges saying "that was cool!" :)

As I mentioned in an earlier reply, I think I learned some things so far. But the gymnastics analogy described here is quite good. Watching a show, what kinds of things might I look for, or listen for, that would improve - even if a little - my ability to understand a good performance from a medium from a not so good?

[Stu]

3 hours ago, Brian Porter-Szucs said:

... In a sport like gymnastics, a good TV broadcast will have an expert who will offer insight on why a particular routine got scored higher or lower than another.  We might learn, for example, to watch how a foot gets placed on the bar, or whether a knee bends more than a certain amount on a particular move.  Viewers might be told that a particular flip was done with significantly more elevation than another flip, thus making it more impressive.  It would be magnificent to have some insight of this nature about what constitutes "excellence" in the various DCI captions...

 

2 hours ago, mjoakes said:

As I mentioned in an earlier reply, I think I learned some things so far. But the gymnastics analogy described here is quite good. Watching a show, what kinds of things might I look for, or listen for, that would improve - even if a little - my ability to understand a good performance from a medium from a not so good?

Back in the late seventies or early eighties, when DCI Finals were broadcasted on PBS, one year there was an ‘expert’ who provided commentary to help the audience understand the show complexities. It was received worse than a lead balloon; mainly because he talked over performances in order to point out what the judges were looking for in show design and execution. And for years after, I mean years and years afterward, the commentators of the subsequent broadcasts were told to shut the heck up and let the performances speak for themselves!!

[IllianaLancerContra]

On 6/2/2017 at 8:56 PM, Stu said:

You do realize that even a ‘tic’ system is still a 100% fuzzy logic subjective adjudicating process. By that I mean that no matter how well defined the sheets, there is absolutely no objective situation at all, ever, in which a human can define when a note, a foot placement, a horn angle, an implement spin/toss, an interval between marchers, crosses over from being executed correctly to becoming a tic.  And while there are some subjective situations like that in objective scored sports, the winner in say Down Hill Skiing is defined by completely objective means apart from human opinion (the one who crosses the finish line in the fastest time wins).  Whereas in competitions like DCI, Ice Dancing, Cheer Leading, et al, the contests that only have adjudicators ranking scores based on shear observation, the ranking and scoring outcome is purely and 100% based on the opinion of adjudicators (ie the winner is completely subjective in nature).  This 100% subjective aspect exists in DCI no matter how well defined the scoring sheets, no matter a tear-down (tic) system or a build-up (current sheets) system, because it boils down to shear, albeit educated, but still shear human opinion.

 

Yes, it is subjective that individual judges have to determine what they feel constitutes enough of a mistake to warrant a tic.  But at least that can relate to something fans can directly observe - corps A had more undressed lines than than Corps B, for their M&M score is lower.  Rather than "I am Expert Judge & I say Corps A gets this score and Corps B gets this score.  I am able to tell because I am Expert Judge"

Edited June 5, 2017 by IllianaLancerContra

[Schnitzel]

On 6/2/2017 at 4:52 PM, mjoakes said:

Are the judges paid fees? Decent fees for what they do? Put up in JW Marriott suites with room service picked up by DCI?

Fees, yes.  Very poor ones.  (About 30% of a decent Marching Band contest in Fall.)

Most DCI Judges take a net loss on taxes as an Independent Contractor.  No deluxe hotels or room service, either.  Usually lowest-fare flights, which means getting to airports by 6am, (after 3-4 hours of sleep) cattle-car seats (in steerage) and connections to the next small-market airport, where maybe there will be a show sponsor there to give a ride to a "discount" hotel, where there might, but not always, be a place to eat before going to the next stadium.  Rinse, lather, repeat.  A decidedly non-glamourous experience, only for those truly dedicated to contributing to a unique and wonderful youth activity.

Edited June 4, 2017 by Schnitzel