Slotting 2013

[drilltech1]

Of course, any shape of a physical object could be penalized as being "excessively band-y". Phantom, with a book, a castle, and a sword? One point deduction each.

Hmm. Didn't Angels and Demons win 2011 and have a number of representational images (angels' wings, demon's horn and tail, zipper, etc.)? I remember that before they became the purple pants band, that the Cream team made a guitar in their Rock Star show, plus the annual appearence(s) of the Crown. Mark down the Fleur de Lis six points? surely, you jest!

[13strokeroll]

Appeal to Authority and obfuscation.

None of the metrics in the pages you cited discussed the determination of which of several biases is responsible for the demonstrated error. Every page was about simply determining that an error has occurred, typically because two judgments diverge. That's it.

The closest the author comes is discussing the forces that might cause a medical intern to attempt to match the assessments of a mentor. He recommends (not surprisingly) keeping their assessments secret until they are complete.

In no way does he discuss (in those pages) how to distinguish between different types of human bias, such as memory-induced bias (bias toward previous results), rank-order bias (bias in favor of the order of performance), patriotism bias (bias based on the judges own region), and reputation bias (bias based on, well, reputation).

Now, I would suggest that what most of us are calling slotting is really the memory induced bias. Essentially biasing in favor of the judges expectations from prior shows.

However, the OP (tupac) seemed to be talking about the reputation bias in his clarification. He says that Cavies will be moved ahead of Madison (presumably because that's where they "belong"). That's not based on recent scores, but on their history.

Interestingly, the bias you first discussed would be some kind of clustering bias in which the corps are grouped artificially. I couldn't find that bias in the articles that I can find, but I would think it's out there somewhere. Is this what you mean by slotting? Maybe we've been talking about different things.

If so, we can't conclude it is happening, because we don't know what forces could cause performing arts groups to actually cluster in real performance levels.

I said I had completed discussing this, but I can see there is still significant misunderstanding. So, one last round to clear the confusion. I'll try to order them so they are clearer to follow.

1. No appeals here. The question arose about my career and I answered it. I'm mildly amused that the more I tried to clarify and explain, the more hostility came out, even more so when I was essentially agreeing with those that became so hostile.

2. Even though subjective data brings about wider ranges of possible single outcomes (scores on a given night), once you understand what those ranges are, they are actually very easy to assess and are more consistent over time (think of my definition as a tunnel of a certain width of possible scores rather than what I think most people are thinking is a series of specific spots from show to show to show). I am using data only to show that the judging process is fairly consistent and measureable over time - which I feel you agree since you have referenced the BK charts pretty frequently. p charts (your BK charts are a variation) do not identify root causes, only that they occur. It takes the user to arrive at root causes. Single charts might not identify anything concrete, but the series of charts - your comment that the variation was universally greater in the past group of BK charts set off alarms in my head. Contrary to intuition, variation is good. The lack of variation observed more lately is a signifier of "stratification". Stratification indicates one of 2 things: the measuring is not stringent enough OR "tampering." Again, the jargonized "tampering" does not have the negative connotations that many might feel compelled to unconsciously add to it - it only indicates an artificial adjustment of assessments. The clustering of 4 medians that was observed also appeared more regularly as the variation began to reduce across SEASONS (the earlier seasons with more variation had far less clumping about distinct medians).

3. The author (Deming was the single person responsible for taking post-war Japan and turning it into the manufacturing juggernaut it was until perhaps 1997 in most part basing their measurements mot on specific objective values, but customers subjective perceptions) was describing that he was finding significant, multiple instances in which assessors (plural) were consistently allowing their assessments to be influenced. The type of bias observed was not important to the observation. We must have had different editions of Deming - mine (ca 1986) had a large section devoted to Walter Shewhart , Deming's mentor and creator of much of the study of the analysis of variance as well as multiple exceprts from his other book Statistical Adjustment of Data. In fact, specific causes that you feel prevent conclusions were unimportant to Deming and Shewhart. The whole point was that identifying consistent trends was the most critical. His other work builds on this, but that's another story.

4. Yes, we have been talking about different definitions of slotting. I had try to clarify that in a few posts, but the more I tried to clarify, the more trolling into former posts and negative ratings on comments about hard working kids in a semis calibre corps started showing up. Given my observations - stratification, lack of similar stratification in previous seasons, and clustering - when viewed TOGETHER - led to what is the only logical conclusion. As I have repeatedly said, my definition of slotting does not accuse judges of intentionally placing corps in specific ranks. My definition of slotting is more concerned with spaces BETWEEN ranks and the repeated clusters supports that. It's not WHICH corps which are grouped, but that they ARE that is the evidence of this. Yes, like groups attract like staff and talent - but not as neatly as this. My best analogy - the clusters are similar to opening a box of puzzle pieces and sorting them into smaller groups by edge pieces, color, etc and then assembling them within their smaller groups before putting the whole thing together. I would classify it as a subset of an order bias - but not based on a bias towards WHAT is being ordered, but to that things need to be organized so they become easier to order.

Now that I've clarified, I'll leave the discussion again. :thumbup:/>

[13strokeroll]

For whatever it's worth, the only time we can test this is at regionals and finals in the GE categories when you have two judges rating the same caption. Comparing scores between performances is impossible. Someone somewhere suggested double panels for every caption, and I think that would be a good idea for finals week, or at least for finals. If you double-paneled all the shows all year and rotated judges carefully, you might start approaching enough comparable data to start making assessments about slotting etc, but only if you assume all the judges aren't in on the deal. Unfortunately, that'd be outrageously expensive. Also, can you imagine having three additional field judges scurrying around?

Agreed. I wouldn't even be upset if scores were assigned at Regionals and Finals with the remainder of the tour consisting only of performances, perhaps with corps still getting recaps as a basis for improvement. But, I think we are well past a point where that would be feasible.

Edited July 26, 2013 by 13strokeroll

[N.E. Brigand]

2. Even though subjective data brings about wider ranges of possible single outcomes (scores on a given night), once you understand what those ranges are, they are actually very easy to assess and are more consistent over time (think of my definition as a tunnel of a certain width of possible scores, rather than what I think most people are thinking is a series of specific spots from show to show to show). I am using data only to show that the judging process is fairly consistent and measurable over time--which I feel you agree with, since you have referenced the BK charts pretty frequently. p charts (your BK charts are a variation) do not identify root causes, only that they occur. It takes the user to arrive at root causes. Single charts might not identify anything concrete, but the series of charts--your comment that the variation was universally greater in the past group of BK charts set off alarms in my head. Contrary to intuition, variation is good. The lack of variation observed more lately is a signifier of "stratification". Stratification indicates one of 2 things: the measuring is not stringent enough OR "tampering." Again, the jargonized "tampering" does not have the negative connotations that many might feel compelled to unconsciously add to it - it only indicates an artificial adjustment of assessments. The clustering of 4 medians that was observed also appeared more regularly as the variation began to reduce across SEASONS (the earlier seasons with more variation had far less clumping about distinct medians).

Just for clarification: I thought that Pete said that the graphs he posted to this discussion were of random numbers, not any actual corps numbers taken from the Blue Knights' page. Am I wrong about that? And if I'm right, and random numbers can generate trends that you find significant, what does that mean?

[13strokeroll]

Just for clarification: I thought that Pete said that the graphs he posted to this discussion were of random numbers, not any actual corps numbers taken from the Blue Knights' page. Am I wrong about that? And if I'm right, and random numbers can generate trends that you find significant, what does that mean?

No, they were actual results from previous seasons. I wasn't clear about the randomness jab. I interpreted it as a misunderstanding about one of my comments. No shots aimed at anyone, but I can see how newbies shy from DCP discussions.

[Pete Freedman]

No, they were actual results from previous seasons. I wasn't clear about the randomness jab. I interpreted it as a misunderstanding about one of my comments. No shots aimed at anyone, but I can see how newbies shy from DCP discussions.

The BK chart is the BK chart. It is of course real drum corps scores (I would hope!)

The other graphs I posted were plots of random numbers generated on my computer running a program. They show fake corps at different score levels but progressing in "real" ability at an equal and fixed level. Then judging "error" is added (one chart shows only a random error for each score, the other adds to that a random per-show bump up or down for all corps.

However, it is clear that 13strokeroll has been talking about the grouping or clustering of corps as "slotting", not the apparently fixed underlying (hidden) progression of corps as slotting. I suppose we should really go with the OP's definition, which is still different.

[Pete Freedman]

Partially correct. I manage the managers of inspectors that assess aesthetic criteria on large ticket items. Perhaps I should have also referenced Pirsig's notes on the study of the inherent combination of science and art to transcend preconceived notions of quality?

Appeal to Authority and obfuscation.

1. No appeals here. The question arose about my career and I answered it.

Introducing technical terminology that you believe is over the heads of the other people is what I meant. You do enjoy doing that.

2. Even though subjective data brings about wider ranges of possible single outcomes (scores on a given night), once you understand what those ranges are, they are actually very easy to assess and are more consistent over time (think of my definition as a tunnel of a certain width of possible scores rather than what I think most people are thinking is a series of specific spots from show to show to show). I am using data only to show that the judging process is fairly consistent and measureable over time - which I feel you agree since you have referenced the BK charts pretty frequently. p charts (your BK charts are a variation) do not identify root causes, only that they occur. It takes the user to arrive at root causes. Single charts might not identify anything concrete, but the series of charts - your comment that the variation was universally greater in the past group of BK charts set off alarms in my head. Contrary to intuition, variation is good. The lack of variation observed more lately is a signifier of "stratification". Stratification indicates one of 2 things: the measuring is not stringent enough OR "tampering." Again, the jargonized "tampering" does not have the negative connotations that many might feel compelled to unconsciously add to it - it only indicates an artificial adjustment of assessments. The clustering of 4 medians that was observed also appeared more regularly as the variation began to reduce across SEASONS (the earlier seasons with more variation had far less clumping about distinct medians).

I see what you're saying, but again, we don't know the forces that could cause the corps to actually stratify. So even if the data shows non-random stratification (as opposed to the appearance of stratification in random data), which has not been demonstrated, we still need to show that the corps are not actually stratified. I can think of several forces that could hypothetically cluster corps:

- Financial similarities

- Reputation similarities that affect recruitment (for example, whether a corps is expected to make finals)

- The G7

- Any number of other factors I can't think of, but would need to discover and disprove.

We have to rule these things out before we can rule in judging bias.

But first of course, we have to establish that non-random stratification is happening.

3. The author (Deming was the single person responsible for taking post-war Japan and turning it into the manufacturing juggernaut it was until perhaps 1997 in most part basing their measurements mot on specific objective values, but customers subjective perceptions) was describing that he was finding significant, multiple instances in which assessors (plural) were consistently allowing their assessments to be influenced. The type of bias observed was not important to the observation. We must have had different editions of Deming - mine (ca 1986) had a large section devoted to Walter Shewhart , Deming's mentor and creator of much of the study of the analysis of variance as well as multiple exceprts from his other book Statistical Adjustment of Data. In fact, specific causes that you feel prevent conclusions were unimportant to Deming and Shewhart. The whole point was that identifying consistent trends was the most critical. His other work builds on this, but that's another story.

Yes that was not in the section book I saw, but I'll be able to check it out again in a week or so. It's on Amazon and interestingly a large subset of its pages are readable in the preview. However, certain pages are removed, including some in the relevant section. But I was able to briefly see an actual copy of the book.

4. Yes, we have been talking about different definitions of slotting. I had try to clarify that in a few posts, but the more I tried to clarify, the more trolling into former posts and negative ratings on comments about hard working kids in a semis calibre corps started showing up. Given my observations - stratification, lack of similar stratification in previous seasons, and clustering - when viewed TOGETHER - led to what is the only logical conclusion. As I have repeatedly said, my definition of slotting does not accuse judges of intentionally placing corps in specific ranks. My definition of slotting is more concerned with spaces BETWEEN ranks and the repeated clusters supports that. It's not WHICH corps which are grouped, but that they ARE that is the evidence of this. Yes, like groups attract like staff and talent - but not as neatly as this.

Evidence of this would be necessary to really establish it as a judging bias.

My best analogy - the clusters are similar to opening a box of puzzle pieces and sorting them into smaller groups by edge pieces, color, etc and then assembling them within their smaller groups before putting the whole thing together. I would classify it as a subset of an order bias - but not based on a bias towards WHAT is being ordered, but to that things need to be organized so they become easier to order.

That could very well happen, judges saying in effect, "Ah, X corps. They're essentially competing with Y corps and Z corps so I'll only really consider them relative to each other, and put that group where it belongs."

Now that I've clarified, I'll leave the discussion again. :thumbup:style="font-size: 9pt;">/>

Yeah, right.

[emc2]

I said I had completed discussing this, but I can see there is still significant misunderstanding. So, one last round to clear the confusion. I'll try to order them so they are clearer to follow.

1. No appeals here. The question arose about my career and I answered it. I'm mildly amused that the more I tried to clarify and explain, the more hostility came out, even more so when I was essentially agreeing with those that became so hostile.

2. Even though subjective data brings about wider ranges of possible single outcomes (scores on a given night), once you understand what those ranges are, they are actually very easy to assess and are more consistent over time (think of my definition as a tunnel of a certain width of possible scores rather than what I think most people are thinking is a series of specific spots from show to show to show). I am using data only to show that the judging process is fairly consistent and measureable over time - which I feel you agree since you have referenced the BK charts pretty frequently. p charts (your BK charts are a variation) do not identify root causes, only that they occur. It takes the user to arrive at root causes. Single charts might not identify anything concrete, but the series of charts - your comment that the variation was universally greater in the past group of BK charts set off alarms in my head. Contrary to intuition, variation is good. The lack of variation observed more lately is a signifier of "stratification". Stratification indicates one of 2 things: the measuring is not stringent enough OR "tampering." Again, the jargonized "tampering" does not have the negative connotations that many might feel compelled to unconsciously add to it - it only indicates an artificial adjustment of assessments. The clustering of 4 medians that was observed also appeared more regularly as the variation began to reduce across SEASONS (the earlier seasons with more variation had far less clumping about distinct medians).

3. The author (Deming was the single person responsible for taking post-war Japan and turning it into the manufacturing juggernaut it was until perhaps 1997 in most part basing their measurements mot on specific objective values, but customers subjective perceptions) was describing that he was finding significant, multiple instances in which assessors (plural) were consistently allowing their assessments to be influenced. The type of bias observed was not important to the observation. We must have had different editions of Deming - mine (ca 1986) had a large section devoted to Walter Shewhart , Deming's mentor and creator of much of the study of the analysis of variance as well as multiple exceprts from his other book Statistical Adjustment of Data. In fact, specific causes that you feel prevent conclusions were unimportant to Deming and Shewhart. The whole point was that identifying consistent trends was the most critical. His other work builds on this, but that's another story.

4. Yes, we have been talking about different definitions of slotting. I had try to clarify that in a few posts, but the more I tried to clarify, the more trolling into former posts and negative ratings on comments about hard working kids in a semis calibre corps started showing up. Given my observations - stratification, lack of similar stratification in previous seasons, and clustering - when viewed TOGETHER - led to what is the only logical conclusion. As I have repeatedly said, my definition of slotting does not accuse judges of intentionally placing corps in specific ranks. My definition of slotting is more concerned with spaces BETWEEN ranks and the repeated clusters supports that. It's not WHICH corps which are grouped, but that they ARE that is the evidence of this. Yes, like groups attract like staff and talent - but not as neatly as this. My best analogy - the clusters are similar to opening a box of puzzle pieces and sorting them into smaller groups by edge pieces, color, etc and then assembling them within their smaller groups before putting the whole thing together. I would classify it as a subset of an order bias - but not based on a bias towards WHAT is being ordered, but to that things need to be organized so they become easier to order.

Now that I've clarified, I'll leave the discussion again. />

Your a smart person and i totally agree. Used his methods of continuous improvement through out my business career in (tec). Study with Demings and Bill Conway, a under study of Demings. Have a great day.

Edited July 27, 2013 by emc2

[HockeyDad]

Blah Blah Blah. BD's been slotted in First Place, yet again. Gee, how exciting....

[Magbo]

I guess I'm just wondering how the Cavies are gaining on Madison again this year. I sure that the G-7 powers to be will do what ever they can to keep the top 7 from last year the same.

I'm thinking the same thing. The recaps from Atlanta for the Scouts are really questionable...Is DCI getting ready to make their G7 move??? C'mon Scouts...you can do it!