Let's Play A Game - You Can Only Go Forward

[Jeff Ream]

5 hours ago, karuna said:

Meh.  I understand we all enjoy the competition and agree to "play the game" but the reality is the "numbers" are really just very subjective opinions about what the judges like better.

On the one hand we have "criteria" which (on the face of it) appears to be an objective way to derive a number and assign it to a narrowly defined caption.  Add those numbers together and BOOM! the score is what the score is.  Right?

On the other hand we have "rank and then rate" where we say "well it's really impossible to put down an objective number" so let's get the spreads correct.  "I don't know how good the Blue Devils are but they are 3 tenths behind SCV in my caption".  Because yeah we've defined precisely what a tenth means.  In fact even the best trained judge is reacting to what he likes.  Of course they will state clearly "what I like is excellence!"  But that excellence can mean entirely different things to different people.  

On the gripping hand,  ALL of this comes down to a judge's internal preferences.  Sure we like to say we're objective and that it doesn't matter how we feel about this program or that,  but it's all very much wrapped up in subjective feelings.  And those feelings extend well beyond what's going on with performers on the field that night.  How do I feel about this designer?  How convincing is their pre-season description of their concept?  How does this person feel about me?   What do my peers expect from this program?    It goes ON and ON and ON.   Every sound, spin and movement is being processed through a myriad of subconscious filters shaped by a lifetime of emotional experience.   We aren't objective observers.  Never were.  Never can be.

The judges truly believe in their system and in their training.  They believe they can be objective evaluators.  But they can't, they aren't and we shouldn't really expect them to be.  They believe they can objectively measure percussion or brass or guard or movement.  And that's all it takes to get the ball rolling:  FAITH! 

It's all very confusing because at the end of all this emotional opining, we end up with a recap. NUMBERS! And we instinctively have a deep respect for numbers.  Numbers are real.  Math doesn't lie.  SCIENCE!

But in the end, it truly is what the judges like.

We hope and pray that judges like what we like.  

like? Um no. I have rated high numbers to shows I never want to see again. The group performed the hell out of what they were given, which also matched what the sheet called for, but i didn't actually "like" the shows.

[Jeff Ream]

4 hours ago, Stu said:

I am a judge. Experienced in two associations including UIL. I have been on crews, and have also put crews together. Been that way since the mid 1990's. So let me tell you a true story (which I think I have also told in another thread a while back).

A few years ago a rather prominent scholastic competition, one looked at as a major Statewide draw, who had used the same judges pool with certified qualified trained judges for decades, wanted fresh eyes and ears. So they contracted judges with extensive experience, but just not within that State.

We were given the sheets in advanced which were Music GE , Music Field, (music judges traded places for finals), Visual GE, Visual Field, (visual judges traded places for finals), Percussion (box prelims and field finals), and Guard (box both prelims and finals), and we were told to apply our own training and experience in order to interpret their sheets. At some point in the past we all had worked with each other, so we were familiar with each other.

By the end of the night we judges were pretty much consistent in scores and rankings with each other. Which was also consistent with our experience and training. But OMG most of the perennial top bands were livid!!!!

We had placed bands in ranking positions never before seen in that competition. While the bands we gave awards to were overjoyed, only a few had ever received top caption awards before; and our top three overall had never been in the top three.

We were not invited back; and the next year the show went back to using the same judges as before.

So, you tell me. With our trained and experienced crew, did we provide bad feedback, bad scoring, and bad ranking? Or was it that our trained and experinced evaluations were just different than what they were accustomed to having for all of those previous years?

apparently so if you weren't invited back.

 

and odds are, like DCI, the bands make the rules...and have a say in who judges.

Edited June 11, 2019 by Jeff Ream

[karuna]

1 hour ago, Jeff Ream said:

like? Um no. I have rated high numbers to shows I never want to see again. The group performed the hell out of what they were given, which also matched what the sheet called for, but i didn't actually "like" the shows.

See?   FAITH!

[cixelsyd]

16 hours ago, Jeff Ream said:

says scores go way out of whack for a show? aren't even curious? Hell a 3 tenth drop in some circles on here creates a frenzy. if someone in July has been high box 4, and suddenly one night they high 5 or even high 3....yeah that's to be looked at, even if the corps didn't raise a concern....and odds are corps will raise a concern if the score has a big drop ( who complains if it makes a big jump?)

 

A good caption head is in touch with their team all the time, getting feedback, impressions, seeing if they see trends developing. If a judge for lack of a better sophisticated term, ####s up, why should they get treated with kid gloves? did the tape suck? Not linking what and how? Yeah they should get pulled and get some additional training.  I used to do some training for our circuit....and the key was I didn't care about if you used all the right buzzwords, or where you r numbers ended up....did your commentary match the number you assigned? Was commentary good feedback, not just saying #### for the sake of saying ####?  If you gave good educational feedback, and your numbers matched that, then go to it....and notice what numbers they gave in terms of the actual score mattered....did it match the commentary.

 

Boom, all you need.

 

judge some time...administrate judges some time....you'd understand

Oh, I understand completely.

You do not judge in DCI, either... but your responses are so full of the slotting mentality, it must be imprinted in your DNA by now.  Trends, trends, trends.  And if anything should ever buck the trend... RED ALERT!  Any wonder that this breeds conformity?

The powers-that-be seem aware that these behaviors cause slotting.  That is why they keep trying to fix it the only way they know how, by hiding the "trends" - secret semifinals scores of 1988, withholding opening day scores in 2011, the great recap caper of 2016.

[cixelsyd]

11 hours ago, Stu said:

I am a judge. Experienced in two associations including UIL. I have been on crews, and have also put crews together. Been that way since the mid 1990's. So let me tell you a true story (which I think I have also told in another thread a while back).

A few years ago a rather prominent scholastic competition, one looked at as a major Statewide draw, who had used the same judges pool with certified qualified trained judges for decades, wanted fresh eyes and ears. So they contracted judges with extensive experience, but just not within that State.

We were given the sheets in advanced which were Music GE , Music Field, (music judges traded places for finals), Visual GE, Visual Field, (visual judges traded places for finals), Percussion (box prelims and field finals), and Guard (box both prelims and finals), and we were told to apply our own training and experience in order to interpret their sheets. At some point in the past we all had worked with each other, so we were familiar with each other.

By the end of the night we judges were pretty much consistent in scores and rankings with each other. Which was also consistent with our experience and training. But OMG most of the perennial top bands were livid!!!!

We had placed bands in ranking positions never before seen in that competition. While the bands we gave awards to were overjoyed, only a few had ever received top caption awards before; and our top three overall had never been in the top three.

We were not invited back; and the next year the show went back to using the same judges as before.

So, you tell me. With our trained and experienced crew, did we provide bad feedback, bad scoring, and bad ranking? Or was it that our trained and experinced evaluations were just different than what they were accustomed to having for all of those previous years?

You were not "trained" on the local trends.

[Stu]

11 hours ago, Jeff Ream said:

apparently so if you weren't invited back.

 

and odds are, like DCI, the bands make the rules...and have a say in who judges.

The real reason we were not invited back was actually because we followed their directive. We applied our professional trained experience to their sheets. But instead of reinforcing their ingrained belief of what was correct artistic interpritation, we upset their apple cart. The past judges were well trained and we held them with high regard. But our professional eyes and ears did not fit their preconceived set of opinions, because our professional views did not reinforce the opinions of the establishment over the past decades. It was not due to our incompetence, but due to our artistic subjective evaluation not reinforcing the artistic subjective evaluation of those before us.

Edited June 11, 2019 by Stu

[Stu]

12 minutes ago, cixelsyd said:

You were not "trained" on the local trends.

Ding ding ding. They were hoping our professional eyes and ears would automatticly reinfore their established trends. And it backfired on them.

[Stu]

10 hours ago, Jeff Ream said:

like? Um no. I have rated high numbers to shows I never want to see again. The group performed the hell out of what they were given, which also matched what the sheet called for, but i didn't actually "like" the shows.

In your opnion they performed the hell out of it. In your opinion it matched what the sheet interprition should be. And with other groups in your opinion they did not perform the he## out of it. Yes you pulled that from your training and experience; and that should be respected. But it was still your subjective opinion nonetheless.

I know professionally trained musicians, and also trained in a judging system, who would also likely have a vastly different opinion than yours. Does that make your opinion correct, and theirs wrong? Does that make their opinion correct, and yours wrong? Nope to both questions. It is all a matter of whose training and whose experience is evaluating at that particular moment.

I proposed moving forward with something new as per request of the original poster. You pretty much called it crap because what I suggested did not fit into your notion, your opinion, that the current established training and experience is the only viable system.

You knocked Wynton, a pro with vast symphonic, concert, jazz, blues, experience as not being competent enough to evaluate music GE without your same training on proper sheet interpretation. Think about that. It is a 'this is my apple cart' type opinion where he should be trained to think and evaluate just like you. Well I say reverse it. He and other pros should interp the sheets and train you to think and evaluate like him. Have the judges conform to a more professional opinion of music GE as opposed to academic. That change in subjective opinion of the sheets is what really has your shorts in a wad.

Edited June 11, 2019 by Stu

[Jeff Ream]

5 hours ago, cixelsyd said:

Oh, I understand completely.

You do not judge in DCI, either... but your responses are so full of the slotting mentality, it must be imprinted in your DNA by now.  Trends, trends, trends.  And if anything should ever buck the trend... RED ALERT!  Any wonder that this breeds conformity?

The powers-that-be seem aware that these behaviors cause slotting.  That is why they keep trying to fix it the only way they know how, by hiding the "trends" - secret semifinals scores of 1988, withholding opening day scores in 2011, the great recap caper of 2016.

no I don't judge there. I do however know people there, and I'm not afraid to reach out and ask questions. I ask DCA folks questions. I ask WGI folks questions. Administrators and working judges alike. Even people with corps. 

 

and everything told me to me has been the same from all segments and lines up with what I've typed.

And with the advent of the most recent sheets plus the Cesario stuff so many mock, we've never seen this much shifting, top to bottom, in decades as we have since probably about 2013.

 

and in 16, they realized...hiding recaps did nothing to affect the outcomes. It did however make great sport in acquiring them.

[Jeff Ream]

5 hours ago, Stu said:

The real reason we were not invited back was actually because we followed their directive. We applied our professional trained experience to their sheets. But instead of reinforcing their ingrained belief of what was correct artistic interpritation, we upset their apple cart. The past judges were well trained and we held them with high regard. But our professional eyes and ears did not fit their preconceived set of opinions, because our professional views did not reinforce the opinions of the establishment over the past decades. It was not due to our incompetence, but due to our artistic subjective evaluation not reinforcing the artistic subjective evaluation of those before us.

sure. I'll let you go with that.