Thursday, March 8, 2012

WoW Survey Results: behind the scenes of "crossplay"

I tend to round to two decimal places for all the stuff I do, just so you know. I like two decimal places because it's the same level of accuracy that you get with money; so it feels specific without being overly so.

During the writing of my post on crossplay, I encountered a problem. When I was looking at the difference in crossplay between the young and the old group, I wanted a way to talk about whether a particular increase for a race or class was more or less than the average increase. I've included a small sample of one of those tables below.

Crossplay by age for race
You'll notice that the last column has been removed also. That's because that's exactly what we're here to talk about. My first instinct for that last column, which I'll just refer to as the "change" column, was to just to take the difference. This would give us a difference of 10.23 for the humans and 9.33 for the trolls. I didn't like that, though, because I felt that the trolls' gain was more significant since they started off much lower than the humans.

My next instinct was to try ratios (division) for the last column. This would give the humans a result of 1.33, a ~33% increase, and the trolls a result of 1.56, a ~56% increase. While this looks good for these two numbers, I actually have conceptual problems with this as well. Suppose I included a third race here for which the young group had 90% female characters and 100% female characters for the older group. This would give us a result of 1.11 (repeating, of course). I feel that's too low though; going from having 10% male characters to zero is a pretty significant thing. So I wanted something that will scale well at the low end and at the high end. Just as gaining 10 percentage points is a big deal when you have very little, such as the trolls, it's also a big deal when there's not much left to pick from, such as for this hypothetical race.

So it occurred to me that just considering how many female characters there were would be insufficient, and that I would also need to look at how many male character there were. This is when I turned to looking at the odds. In short (if you don't want to read that link), the odds of something happening is the probability that it will happen divided by the probability it won't happen. So if the odds of something happening are 10:1, it's 10 times more likely to happen than to not happen. So I decided to look at the ratio of the odds for each age group, or as I summarized in the last post
where p is the percentage of female characters for that group. When I did all the calculations and thought about how I felt about the results, everything just looked right. The values that I felt should be bigger than others were all that way. 

So if you're wondering how this stuff get's done; that's how. It's trial, error, and intuition.