How many Grand Prix do you need to judge before you have enough experience to become a Level 3 Judge? Ten? Twenty?
How many times do you need to be a Team Lead before you are good at it?
When is good enough, well, good enough?
Based on the title of this blog and my penchant for rhetorical questions (What's the point of all these rhetorical questions?) you might have guessed that I am coming down against the idea of "experience" being a useful measure of someone's judging ability. Clearly, if you try to reduce this to black and white, experience does matter; someone with ten years of experience judging is going to be much better at it than someone with one year of experience. However, what I am arguing against is the mentality that there is a linear progression between those two points, and simply piling on repeated "experience" is enough to become a better judge.
My own history as a judge colors my position on this subject. Before making Level 3, I judged 6 Grand Prix, 3 Pro Tours, and a US Nationals. PTs were larger and crazier than they are today, and GPs were smaller (800 was considered large), while US Nationals was somewhere between the two. I think it's reasonable to average out those events and call the aggregate an equivalent of 10 GPs worth of experience by today's standards. Before getting into a comparison with today, I want to digress a bit.
Seven years ago, Jeff Morrow wrote an article about the growth mindset. To sum up his article, there are two different mindsets, the fixed and the growth mindset, which he distinguishes as such:
"A person with the fixed mindset believes that characteristics like intelligence, talent, and aptitude at a particular activity are essentially fixed qualities. A person with the growth mindset, on the other hand, believes that aptitude, talent, and even basic intelligence can be improved with effort and practice."
As Jeff points out, the Judge Program is fundamentally built around the idea of the growth mindset. We are truly a cult of self-improvement. Jeff specifically calls out improvement as a result of effort and practice, whereas I think the current focus on experience incorrectly leans more heavily on existence and luck.
Experience and existence are remarkably similar words, and they can look similar in application, but they aren't the same thing, and the differences matter quite a bit. Earlier I pointed out that I had 10 "GPs" worth of experience before testing for L3. These days, that's a drop in the bucket for a road warrior L2 in the United States, but that's partly because there are so many more GPs in this country, as well as the StarCityGames.com Tour, which while not the same as GPs today, do overlap somewhat with GPs from "back in the day."
In fact, with the schedules of the GP and Open Series being what they are, judges in the US can easily get twice the experience that I did in a single year. But there I go using that word, "experience," in a place where I don't think it applies. What I mean to say is that a judge can exist at 20 GPs and SCG Opens in a single year, but that their experience doesn't increase in anywhere close to a linear fashion to match that.
Abe Corson hit on this in his recent interview for his 100th Review Milestone. When I asked him why he had written so many reviews in 2012, he cited that there were fewer events that year and "...the only reason I had any hope of keeping up with this pace was that I just wasn’t as active as I am now. 2012 was before the increase in number of US GPs, so there weren’t as many entire-weekend-consuming things for me to get sucked into. There was at the same time fewer experiences about which to write and more leftover time for me to do it."
I mentioned luck being a factor that people lean on in terms of their experience calculation. What I mean by luck is that judges exist at events and hope that something happens that tests them or teaches them. This is similar to the request to "give me feedback" in the hopes that someone else sees and comments on something you've done. It's a very passive approach to learning, and it's one that plays into a more fixed mindset because you aren't exerting your own effort and practice to grow; you are waiting for someone else or something else to come along and teach it to you.
Existing at fewer events meant more time to think about those events, internalize lessons, write reviews, and yes, gain experience. It's counter-intuitive but you should go to fewer events to become a better judge.
Thursday, January 28, 2016
Friday, January 22, 2016
Feedback as a Scorekeeper
Matt Braddock recently asked me (and Jennifer Dery) for some tips on giving feedback from the tournament role of the Scorekeeper, a role that Jenn and I have both filled at StarCityGames.com Opens and Grand Prix. Matt was primarily asking about SKing local tournaments like PPTQs, but I think the concepts translate well because the types of interactions that you have don't change that much in scale compared to other aspects of tournaments.
The most striking difference of the scorekeeping seat is that it is just that, a seat. Depending on the size of the tournament, you may be stuck in your chair for a majority of the round. Even if you aren't actively entering results, it behooves you to stay in the area to field questions like "How do I drop?" and those rare times that you do get away from the chair are better spent getting fresh air, coffee, or lunch.
This means that you are going to be limited in the interactions that you have with judges in the event. You are rarely going to have the opportunity to shadow someone on a judge call or appeal. When you think about it, this is the lifeblood of judging: the intersection of rules and policy knowledge with customer service skills. What can a Scorekeeper provide feedback on without these things?
One of the people that an SK can help the most is the Head Judge of the event. Your seat, while not ideally situated for floor interactions, is in a good place to see a lot of the things that the HJ does in the tournament, chief among them the HJ opening announcements. During these announcements, Floor Judges are often busy with other tasks like distributing player rewards, collecting decklists, or distributing match result slips (if the announcements happen at the beginning of Round 1). Unless there are some late registrants, the SK is generally free to listen to the announcements and provide some much needed feedback on them.
Some things to consider when listening to announcements:
* Are they too long? You can only hold players' attention for so long. Stick to the essentials. Consider what announcements could wait until later.
* The use of humor in announcements can be controversial. I personally like them. Whatever your preference, if it is used, that's ripe for commentary.
* Did the HJ sound good? This usually encompasses two things. First, do they have a good announcement voice? Generally, it is better to speak in the lower register when addressing a group, especially when not using a microphone. Second, speed is an important thing to monitor. People tend to talk faster when they are nervous, and this can be to the detriment of announcements.
* On a related note to sounding good, were they prepared?
If your event is using match result slips to record penalties that's a great route to interacting with judges about IPG policy. Especially as we transition into a new era of Hidden Card Error (HCE), we will all be exploring the nuances of this infraction. As the Scorekeeper for an event, you are the only person who sees all of the infractions that are issued, so you can spot trends and especially common mistakes that judges make. My scorekeeping and blogging associate Jennifer Dery makes it a habit of writing about such common issues here.
End of Round procedure is another point of interaction between judges and scorekeepers. Smaller tournaments might not have much interaction here besides a quick check like "Table 10 and 14 are the last two playing." At larger events, there is definitely a lot more going on, making it ideal fodder for discussion and feedback.
Those are just a few ways that you can think about feedback from the scorekeeper's seat. There are certainly other ways. For example, if you can negotiate one round to spend some time on the floor, it's a great mental break in addition to an opportunity to interact with judges in a more traditional manner. The point is that being a scorekeeper doesn't have to be the death knell to you giving feedback to others.
The most striking difference of the scorekeeping seat is that it is just that, a seat. Depending on the size of the tournament, you may be stuck in your chair for a majority of the round. Even if you aren't actively entering results, it behooves you to stay in the area to field questions like "How do I drop?" and those rare times that you do get away from the chair are better spent getting fresh air, coffee, or lunch.
This means that you are going to be limited in the interactions that you have with judges in the event. You are rarely going to have the opportunity to shadow someone on a judge call or appeal. When you think about it, this is the lifeblood of judging: the intersection of rules and policy knowledge with customer service skills. What can a Scorekeeper provide feedback on without these things?
One of the people that an SK can help the most is the Head Judge of the event. Your seat, while not ideally situated for floor interactions, is in a good place to see a lot of the things that the HJ does in the tournament, chief among them the HJ opening announcements. During these announcements, Floor Judges are often busy with other tasks like distributing player rewards, collecting decklists, or distributing match result slips (if the announcements happen at the beginning of Round 1). Unless there are some late registrants, the SK is generally free to listen to the announcements and provide some much needed feedback on them.
Some things to consider when listening to announcements:
* Are they too long? You can only hold players' attention for so long. Stick to the essentials. Consider what announcements could wait until later.
* The use of humor in announcements can be controversial. I personally like them. Whatever your preference, if it is used, that's ripe for commentary.
* Did the HJ sound good? This usually encompasses two things. First, do they have a good announcement voice? Generally, it is better to speak in the lower register when addressing a group, especially when not using a microphone. Second, speed is an important thing to monitor. People tend to talk faster when they are nervous, and this can be to the detriment of announcements.
* On a related note to sounding good, were they prepared?
If your event is using match result slips to record penalties that's a great route to interacting with judges about IPG policy. Especially as we transition into a new era of Hidden Card Error (HCE), we will all be exploring the nuances of this infraction. As the Scorekeeper for an event, you are the only person who sees all of the infractions that are issued, so you can spot trends and especially common mistakes that judges make. My scorekeeping and blogging associate Jennifer Dery makes it a habit of writing about such common issues here.
End of Round procedure is another point of interaction between judges and scorekeepers. Smaller tournaments might not have much interaction here besides a quick check like "Table 10 and 14 are the last two playing." At larger events, there is definitely a lot more going on, making it ideal fodder for discussion and feedback.
Those are just a few ways that you can think about feedback from the scorekeeper's seat. There are certainly other ways. For example, if you can negotiate one round to spend some time on the floor, it's a great mental break in addition to an opportunity to interact with judges in a more traditional manner. The point is that being a scorekeeper doesn't have to be the death knell to you giving feedback to others.
Wednesday, January 6, 2016
Flash Feedback
It’s been said that the Judge Program is a Cult of Self-Improvement. It seems that within that there is a sub-Cult of Long Windedness. Reviews today are just longer than they were 7 years ago. But does that mean we are better off? (This blog was going to include stats to back this up, but I... didn't want this to get too long. Maybe I will write a follow up.)
There’s no question that the average review today is more substantive and full of meaningful feedback. It’s a natural consequence of our evolution as judges and our use of this feedback tool. But as a result, fewer reviews are being written. (The entire Judge Program is writing more reviews, but the average per judge has gone down.)
That’s feedback that is being left off the table. Hopefully, you’re still getting a chance to provide that feedback face-to-face, but if it isn’t making into the written form of a review, there’s still some lost momentum there.
In our modern Cult of Long-Windedness, there is a popular lament as to why a judge hasn’t written a review for another judge: “I didn’t have enough material.” Translation: “The review isn’t long enough.”
I’m here to call BS and lead a movement to remove this excuse from our vernacular.
I get it. I really do. More feels like “better.” I am probably the inadvertent leader of the Long-Winded Cult, and I know for a fact that receiving reviews from me has injected people with the false notion of what a review (or an “L3 review”) should look like. Just look how long this stupid blog post is!
Feedback ungiven just disappears into nothing. It’s a waste.So writing anything has got to better than sitting on an unfinished review that is “too short.” So with that, I introduce to you the titular concept of Flash Feedback.
Just write that review. Don’t mind how short it is. In fact, work on making it shorter. Using Twitter has been a wonderful learning experience for me in terms of word economy, and I would like to set an analogous bar for Flash Feedback: 140 words (instead of characters). Challenge yourself to enter a review with 140 words or less.
Note that isn’t for all reviews. Some reviews should be longer and more in-depth. Ultimately, the goal is to be able to recognize what feedback should be long and what feedback should be short. Flash Feedback is about taking away the stigma of the “too short” review, and getting in the habit of entering your feedback, no matter how short you think the content is. In fact, you’ll probably find that you need to do some creative editing in order to get your Flash Feedback under the limit. That’s a great practice to get into in general, and will serve you well even when you write your long reviews, because the best writing should always be trimmed a bit.
Finally, here's an example of a Flash Feedback review that I wrote. It is exactly 140 words, but that includes that first introduction line, and doesn't include a link to this blog itself. I'm on the fence right now about whether to include those into my word count in the future, although I will definitely continue to mention that it is Flash Feedback. I've also omitted some biographical details about the event itself, which serve only to elaborate on the details that the drop down fields do not cover, and aren't really a part of the feedback.
-----------
This is a Flash Feedback review.
You got appealed twice to me (I was Support Judge). 1st was a typical Kolaghan's Command vs. Spellskite. 2nd was Ugin’s -X exiling colored artifacts. You were right and I upheld both times. We talked about how to explain rules to players by using examples that are related, namely Electrolyze vs. Spellskite.
I also showed you the system for tracking Warning numbers on slips. This is an underrated way you can help yourself remember to ask if a player has prior Warnings, and it can help the Scorekeeper.
Other than this, we only had short interactions, but I am left with the overwhelming impression that you are a friendly, hard-working judge. I’ve been told this by others, so it was good to confirm it for myself, and to put a face to the name.
-----------
There’s no question that the average review today is more substantive and full of meaningful feedback. It’s a natural consequence of our evolution as judges and our use of this feedback tool. But as a result, fewer reviews are being written. (The entire Judge Program is writing more reviews, but the average per judge has gone down.)
That’s feedback that is being left off the table. Hopefully, you’re still getting a chance to provide that feedback face-to-face, but if it isn’t making into the written form of a review, there’s still some lost momentum there.
In our modern Cult of Long-Windedness, there is a popular lament as to why a judge hasn’t written a review for another judge: “I didn’t have enough material.” Translation: “The review isn’t long enough.”
I’m here to call BS and lead a movement to remove this excuse from our vernacular.
I get it. I really do. More feels like “better.” I am probably the inadvertent leader of the Long-Winded Cult, and I know for a fact that receiving reviews from me has injected people with the false notion of what a review (or an “L3 review”) should look like. Just look how long this stupid blog post is!
Feedback ungiven just disappears into nothing. It’s a waste.So writing anything has got to better than sitting on an unfinished review that is “too short.” So with that, I introduce to you the titular concept of Flash Feedback.
Just write that review. Don’t mind how short it is. In fact, work on making it shorter. Using Twitter has been a wonderful learning experience for me in terms of word economy, and I would like to set an analogous bar for Flash Feedback: 140 words (instead of characters). Challenge yourself to enter a review with 140 words or less.
Note that isn’t for all reviews. Some reviews should be longer and more in-depth. Ultimately, the goal is to be able to recognize what feedback should be long and what feedback should be short. Flash Feedback is about taking away the stigma of the “too short” review, and getting in the habit of entering your feedback, no matter how short you think the content is. In fact, you’ll probably find that you need to do some creative editing in order to get your Flash Feedback under the limit. That’s a great practice to get into in general, and will serve you well even when you write your long reviews, because the best writing should always be trimmed a bit.
Finally, here's an example of a Flash Feedback review that I wrote. It is exactly 140 words, but that includes that first introduction line, and doesn't include a link to this blog itself. I'm on the fence right now about whether to include those into my word count in the future, although I will definitely continue to mention that it is Flash Feedback. I've also omitted some biographical details about the event itself, which serve only to elaborate on the details that the drop down fields do not cover, and aren't really a part of the feedback.
-----------
This is a Flash Feedback review.
You got appealed twice to me (I was Support Judge). 1st was a typical Kolaghan's Command vs. Spellskite. 2nd was Ugin’s -X exiling colored artifacts. You were right and I upheld both times. We talked about how to explain rules to players by using examples that are related, namely Electrolyze vs. Spellskite.
I also showed you the system for tracking Warning numbers on slips. This is an underrated way you can help yourself remember to ask if a player has prior Warnings, and it can help the Scorekeeper.
Other than this, we only had short interactions, but I am left with the overwhelming impression that you are a friendly, hard-working judge. I’ve been told this by others, so it was good to confirm it for myself, and to put a face to the name.
-----------
Subscribe to:
Posts (Atom)