The Fallacy of Experience

How many Grand Prix do you need to judge before you have enough experience to become a Level 3 Judge? Ten? Twenty?

How many times do you need to be a Team Lead before you are good at it?

When is good enough, well, good enough?

Based on the title of this blog and my penchant for rhetorical questions (What's the point of all these rhetorical questions?) you might have guessed that I am coming down against the idea of "experience" being a useful measure of someone's judging ability. Clearly, if you try to reduce this to black and white, experience does matter; someone with ten years of experience judging is going to be much better at it than someone with one year of experience. However, what I am arguing against is the mentality that there is a linear progression between those two points, and simply piling on repeated "experience" is enough to become a better judge.

My own history as a judge colors my position on this subject. Before making Level 3, I judged 6 Grand Prix, 3 Pro Tours, and a US Nationals. PTs were larger and crazier than they are today, and GPs were smaller (800 was considered large), while US Nationals was somewhere between the two. I think it's reasonable to average out those events and call the aggregate an equivalent of 10 GPs worth of experience by today's standards. Before getting into a comparison with today, I want to digress a bit.

Seven years ago, Jeff Morrow wrote an article about the growth mindset. To sum up his article, there are two different mindsets, the fixed and the growth mindset, which he distinguishes as such:

"A person with the fixed mindset believes that characteristics like intelligence, talent, and aptitude at a particular activity are essentially fixed qualities. A person with the growth mindset, on the other hand, believes that aptitude, talent, and even basic intelligence can be improved with effort and practice."

As Jeff points out, the Judge Program is fundamentally built around the idea of the growth mindset. We are truly a cult of self-improvement. Jeff specifically calls out improvement as a result of effort and practice, whereas I think the current focus on experience incorrectly leans more heavily on existence and luck.

Experience and existence are remarkably similar words, and they can look similar in application, but they aren't the same thing, and the differences matter quite a bit. Earlier I pointed out that I had 10 "GPs" worth of experience before testing for L3. These days, that's a drop in the bucket for a road warrior L2 in the United States, but that's partly because there are so many more GPs in this country, as well as the StarCityGames.com Tour, which while not the same as GPs today, do overlap somewhat with GPs from "back in the day."

In fact, with the schedules of the GP and Open Series being what they are, judges in the US can easily get twice the experience that I did in a single year. But there I go using that word, "experience," in a place where I don't think it applies. What I mean to say is that a judge can exist at 20 GPs and SCG Opens in a single year, but that their experience doesn't increase in anywhere close to a linear fashion to match that.

Abe Corson hit on this in his recent interview for his 100th Review Milestone. When I asked him why he had written so many reviews in 2012, he cited that there were fewer events that year and "...the only reason I had any hope of keeping up with this pace was that I just wasn’t as active as I am now. 2012 was before the increase in number of US GPs, so there weren’t as many entire-weekend-consuming things for me to get sucked into. There was at the same time fewer experiences about which to write and more leftover time for me to do it."

I mentioned luck being a factor that people lean on in terms of their experience calculation. What I mean by luck is that judges exist at events and hope that something happens that tests them or teaches them. This is similar to the request to "give me feedback" in the hopes that someone else sees and comments on something you've done. It's a very passive approach to learning, and it's one that plays into a more fixed mindset because you aren't exerting your own effort and practice to grow; you are waiting for someone else or something else to come along and teach it to you.

Existing at fewer events meant more time to think about those events, internalize lessons, write reviews, and yes, gain experience. It's counter-intuitive but you should go to fewer events to become a better judge.