For those of you who don't know, a day ago I sent out surveys to three groups: Those on the Mini-Normal Queue, those on the Mini-Theme Queue, and then those who had posted in the "Redefining Normal" MD thread which I also opened up to the public. I got a pretty good response. The MD survey probably has some crossover with the other surveys, though judging by the differences, probably not a ton.
Before results and analysis, a disclaimer: although I did some things to preserve the integrity of the surveys, they are in no way scientific. They're certainly not random. The questions could have been phrased differently (for example, I asked, "which roles do you consider normal" rather than "which roles do you think should or shouldn't be normal"). Also, I only included a finite list of roles (40 to be exact). Last, I am not a statistician nor am I great with spreadsheets. There is every chance that I have made a mistake either with the statistical stuff or with implementing the statistical stuff. Let me know if this is the case.
To make things simpler, MD=Public survey, MN=MiniNormal Survey, MT=Mini Theme Survey
First, some demographic information about each group.
For each n=
MD: 28
MN: 24
MT: 21
So now for some results. I've broken these down into 6 categories for viewing. You can see the raw data by visiting google documents here.. There's more data available, but that link takes you to the compiled stuff.
To understand the rating, understand how I scored each question. Each person had the ability to rank a role "Absolutely Normal" "Probably Normal" "Borderline Normal" "Probably Not Normal" "Under no circumstances Normal." Each response had a point value, 4 for Absolutely, 3 for probably, etc.
The categories I'm listing are:
1. The point average for Mini-Normal mods, those who implement the roles into setups.
1. Average among all three surveys
2. Normalized Average among all three surveys (normalized around 0. This is useful for actually ranking the level of normalness each has, but does not answer what is "normal." In other words, this average takes into account the fact that different people work on different scales, but since this scale isn't an arbitrary 0-4 rating but rather people telling me an actual answer, it's not the full story. Anything between -1 and 1 is within 1 std. deviation of the mean.
3. Average Percent across all three surveys rated the role either Absolutely or Probably Normal.
4. Difference between MD and MN score for each role
5. Difference between MT and MN scores.
Last edited by zoraster on Tue Dec 22, 2009 5:46 pm, edited 4 times in total.
Some thoughts:
1. Mini-Normal mods are very, very conservative with what they think is normal compared to other mods but especially compared to the those participating in mafia discussion.
This is interesting because these are the people who are implementing the roles into setups. Perhaps it's good that they're conservative, but I think they have a fairly skewed view for some reason. Or maybe because they're the ones actually implementing stuff, we're the ones with a skewed view.
2. I'd suggest that anything above a 3 is absolutely normal, anything above a 2.5 is almost certainly normal, and anything above a 2 is probably normal. But others don't have to agree with those arbitrary lines. Perhaps the real line should be anything within 1 standard deviation of the norm on the low end is the cutoff. Or perhaps anything that gets above a %50 for Absolutely Normal or Probably Normal should be considered normal as at least half the game would be fine having those roles in a normal game without any concerns.
Anyway, there's lots to read into this data, but I thought I'd start us off with a couple of points. What do others think?
Actually, the way the survey was worded, 2 was "borderline normal", so anything that was as 2 or better was a role that the majority considered to be at least borderline normal.
On a side note, you should have at least included "Mason (unconfirmed alignment)" in the survey. I consider that a more normal role then either confirmed mason or neighbor, personally.
I want us to win just for Yos' inevitable rant alone. -CrashTextDummie
2.5 is a conservative estimate. With the standard deviation of 1, the game could contain roles people don't find normal. With a 2.5, that would limit it to 1.5. Besides, no one complains when a normal role is NOT used in a game, just when a nonnormal role is used in a game.
1. Mini-Normal mods are very, very conservative with what they think is normal compared to other mods but especially compared to the those participating in mafia discussion.
they also have almost zero modding experience (avg games modded is .2, lol)
Yosarian2 wrote:Actually, the way the survey was worded, 2 was "borderline normal", so anything that was as 2 or better was a role that the majority considered to be at least borderline normal.
On a side note, you should have at least included "Mason (unconfirmed alignment)" in the survey. I consider that a more normal role then either confirmed mason or neighbor, personally.
What's the difference between a neighbor and an unconfirmed mason?
Yosarian2 wrote:Actually, the way the survey was worded, 2 was "borderline normal", so anything that was as 2 or better was a role that the majority considered to be at least borderline normal.
On a side note, you should have at least included "Mason (unconfirmed alignment)" in the survey. I consider that a more normal role then either confirmed mason or neighbor, personally.
What's the difference between a neighbor and an unconfirmed mason?
"neighbor" was developed to communicate "mod confirmed as alignment unconfirmed masons" more easily.
nhammen wrote:When a Neighbor flips, everyone knows they were unconfirmed. If a Mason flips, nobody knows either way, except the people in the Mason group.
I've never heard that argument before. Huh.
I know I consider (mod confirmed) mason more normal than (mod unconfirmed) mason, but that's because scum mason is the only role that justifies (mod unconfirmed) mason, and scum mason likely would have gotten the lowest score on the poll.
I replaced into Chess Mafia for 6 months, and all I got was a win and this lousy sig.
Anything above two basically has to be considered 'Normal', and probably as low as 1.5 or so would be borderline.
(11:04:10 PM) senspizzaline: That's actually my bold prediction for the year
(11:04:19 PM) senspizzaline: Miami finishes 2nd in the AFCE.
(11:05:35 PM) jhawk01b: my bold prediction for the year is that whoever wins the NFC West will have a winning record
SensFan wrote:Anything above two basically has to be considered 'Normal', and probably as low as 1.5 or so would be borderline.
For once I'm a lot closer to agreeing with SensFan. Calling something "borderline" is acknowledging someone else could use it and it'd be fair game.
Although I think Inventor is really a questionable role for normals, not because of how hard it is to find in games but how it's really a different dynamic. Most of the time in a normal once you've done the setup, the actions you take as a mod shouldn't affect the game... you've written the game and it's done. But the inventor requires a mod to make it up as they go, which I think isn't a normal dynamic.
Actually, that's not the way I have seen Inventor used. From what I've seen, they are given a list of what items they can invent. Maybe I have Rocissi Summer in my mind too much.
I think that this begs the question, though - is this a serious enough problem to merit putting the drastic restrictions that we are considering? If the MN mods are
more
conservative than the players complaining about the roles that the MN mods are implementing, that implies that the problems are rather limited.
My posts are best read in the calm, reassuring voice of Morgan Freeman. I don't sound anything like him, but they sound best that way.
Temporarily retiring following the end of my current obligations.
To add to Neto's point, I still say you'd achieve much better results by just saying "You must warn people before including alignment-changing effects in a Normal." There. Problem solved.
(11:04:10 PM) senspizzaline: That's actually my bold prediction for the year
(11:04:19 PM) senspizzaline: Miami finishes 2nd in the AFCE.
(11:05:35 PM) jhawk01b: my bold prediction for the year is that whoever wins the NFC West will have a winning record
Netopalis wrote:I think that this begs the question, though - is this a serious enough problem to merit putting the drastic restrictions that we are considering? If the MN mods are
more
conservative than the players complaining about the roles that the MN mods are implementing, that implies that the problems are rather limited.
I think the problem may actually be reverse. Normal mods are cautious in general because they don't know the parameters of the roles. Because they don't like being yelled at, they err on the side of caution. This is understandable. But it's also something we could circumvent.
And then there's the outlier problem. There's the person who reads the rules and realizes that they don't lay out what is and isn't a normal and so they put in truly weird things and then people get hissy, yet they're following the rules. If only 1 out of 5 games is like this, it's still going to have 12 people who are at risk of being put off by a setup that they did not approve of.
So yes, I think it's a problem. We don't necessarily want to force our new mods to become truly constrained. By putting in new rules we actually expand their options should they want to. Because I promise if we put in new rules that explicitly say that X, Y, and Z roles are considered normal that mods will start to use X,Y, and Z roles... which I think is generally a good thing especially if it gets them away from the typical Cop-Doc-Roleblocker setup.
If you want, we can use this survey as a measure of normality: if you give vanilla townie/mafia goon a score of 4.00 (since they are obviously normal), then I'd suggest that a 12 person mininormal should have a normality score of at least 46.00, where a normality score is obtained by associating to each role its score on the chart and adding up those numbers for all 12 roles.
(Obviously, 46.00 is a rough guesstimate, but that seems close to a right value for me.)
I replaced into Chess Mafia for 6 months, and all I got was a win and this lousy sig.
If you want games that have a list of roles explicitly allowed, play Semi-Opens.
If you want games that don't have Cults or NKImmuneMillerVigCops, play Normals.
(11:04:10 PM) senspizzaline: That's actually my bold prediction for the year
(11:04:19 PM) senspizzaline: Miami finishes 2nd in the AFCE.
(11:05:35 PM) jhawk01b: my bold prediction for the year is that whoever wins the NFC West will have a winning record
I don't know, i feel like requiring mods to do math with decimals before their game is asking for trouble. Also, 46 is pretty high. If you did a gunsmith and a watcher, you'd be unable to balance with an appropriate mafia power.
SensFan wrote:If you want games that have a list of roles explicitly allowed, play Semi-Opens.
If you want games that don't have Cults or NKImmuneMillerVigCops, play Normals.
Once again, semi-opens aren't something that exist as something that regularly occurs. Quit it.
SensFan wrote:If you want games that have a list of roles explicitly allowed, play Semi-Opens.
If you want games that don't have Cults or NKImmuneMillerVigCops, play Normals.
Once again, semi-opens aren't something that exist as something that regularly occurs. Quit it.
If there are as few semi-opens as you claim there are, then why is the solution to turn an entire fucking Queue into semi-open games? If people wanted semi-open games, there'd be more of them!
Now, if I want a closed game with no flavour, under your proposal I need to go find another site to play.
(11:04:10 PM) senspizzaline: That's actually my bold prediction for the year
(11:04:19 PM) senspizzaline: Miami finishes 2nd in the AFCE.
(11:05:35 PM) jhawk01b: my bold prediction for the year is that whoever wins the NFC West will have a winning record
SensFan wrote:If you want games that have a list of roles explicitly allowed, play Semi-Opens.
If you want games that don't have Cults or NKImmuneMillerVigCops, play Normals.
Once again, semi-opens aren't something that exist as something that regularly occurs. Quit it.
If there are as few semi-opens as you claim there are, then why is the solution to turn an entire fucking Queue into semi-open games? If people wanted semi-open games, there'd be more of them!
Now, if I want a closed game with no flavour, under your proposal I need to go find another site to play.
Under my proposal you get exactly what you would have had before only you'd probably get more of it! Did you even read the proposal? Have you even read the current rules?
If you're arguing for the status quo, you're arguing that we should use the term "standard roles" but not define that so that we have no idea what are and aren't standard roles but rather should guess what they are.
But to return: the rules allow for closed games in exactly the same manner as the current rules except the "standard" roles part is actually defined and probably increases the number of roles that are considered standard thus actually increasing the number of actually plausibly occurring roles -- it makes it LESS open, not the other way around. This should be something you stand behind rather than feed your obstructionist, contrarian appetite.
I want it to be clear: I respect yos and mr. flay and others who have expressed concerns about my proposal. They've shown they have real issues that need to considered and responded to. I mean in no way to mitigate their contribution to the debate. I think their concerns all have answers and perhaps we will end up agreeing to disagree.
Last edited by zoraster on Tue Dec 22, 2009 4:22 pm, edited 2 times in total.
You are talking about a list of explicitly allowed roles. That makes a game semi-open.
Right now, I LIKE the fact that Normal games aren't cookie-cutter. I don't see why you think there is a need to remove open Normals.
(11:04:10 PM) senspizzaline: That's actually my bold prediction for the year
(11:04:19 PM) senspizzaline: Miami finishes 2nd in the AFCE.
(11:05:35 PM) jhawk01b: my bold prediction for the year is that whoever wins the NFC West will have a winning record
zoraster wrote:I don't know, i feel like requiring mods to do math with decimals before their game is asking for trouble. Also, 46 is pretty high. If you did a gunsmith and a watcher, you'd be unable to balance with an appropriate mafia power.
A Gunsmith is a flavored cop. Making a mini normal mod use a sane cop rather than a Gunsmith (and tossing in a Godfather and/or miller to replicate the effects you want from a Gunsmith) seems exactly like what I'd want from a test for normality.
(Also as I said, 46 was a rough guesstimate; other people may have personal preference for a lower or higher number.)
Last edited by Thok on Tue Dec 22, 2009 4:24 pm, edited 1 time in total.
I replaced into Chess Mafia for 6 months, and all I got was a win and this lousy sig.
