The Newbie Matrix6 stats thread (complete)

[goodmorning]

Well, TBF I think the original spirit was meant as "7v1 at the start of a Day" which has definitely happened in Newbies given the Town 7.4% perfect winrate.

[Toomai]

I kinda missed what exactly was proposed here. So you'd like to see stats like "When the game reaches 5:2, town mislynches X% of the time"? Or "When the game reaches 5:2, town's win rate is Y%"? Or something else?

I mean, the idea seems doable (if not as trivial as some of the other stats), but I'd like something a bit more concrete.

[Bicephalous Bob]

The first one, but also: "When the game reaches x:y, what is the scum lynch rate divided by y/(x+y)"?

y/(x+y) being the chance of hitting scum in a random lynch.

[Toomai]

Okay so kind of an extrapolation of the Day 1 lynch accuracy stats. That should be doable.

[Toomai]

Took a bit longer than expected but I have the stats. I plan to do justice to them in an update later today or tomorrow, but here's some things that jumped out at me:

• Half of all no-lynches occur in 4:2.
  
• Town is far better than random in 6:1, scoring a scum lynch rate of 43.8% (expected 14.3%). 7:1 is similar (46.2% instead of 12.5%) but has only 13 samples so far.
  
• Scum gets lynched in 33.8% of 3:2 LyLos (expected 40%) and 56.1% of 2:1 LyLos (expected 1/3).
  
• 3:2 is the only ratio where town does worse than random in "don't lynch town". But it's also the ratio where random has the best chance of succeding, so that's not saying much.

Note that, due to the way in which I store the stats, if a game has a no-lynch followed by a no-kill (two chances to lynch with the same ratio), the first one is ignored and the second one counts. As a result there's probably a slightly higher number of no-lynches than is being reported.

[Toomai]

Did the update. Have a look.

[reinoe]

Out of 82 town wins, 19 wins (23.2%) involved only scum lynches.
13 wins (15.9%) were semi-perfect (no town lynches).
6 wins (7.3%) were perfect (no town deaths).

Out of 86 scum wins, 54 wins (62.8%) were perfect (no scum deaths).

Those stats are crazy, especially for the number of perfect scum wins.

[Quilford]

These are really interesting stats. You're doing God's work here.

[llll]

If mafia have higher average lifespans than town, that could explain some of the discrepancy between the replacement rates.

[Toomai]

Jailkeepers live longer than Roleblockers but replace out at the same rate (per slot) as Goons. I don't think "average lifespan" would be a telling stat in that regard.

[Toomai]

I found an error in the spreadsheets that resulted in reversing the lifespan of Jailkeepers and Bulletproofs in setup A, meaning JKs were living too long and BPs were dying night 1. I have corrected it.

[Toomai]

BREAKING NEWS

(yes, this deserves such formatting)

Town is 54-76 (41.5%) in games with 5 newbies, 3 SEs, and 1 IC.
Town is 40-27 (59.7%) in games with 6 newbies, 2 SEs, and 1 IC.

The p-value for this is 0.01552 (Z-score -2.418).

There is a statistically significant difference in town winrate between 5-3-1 games and 6-2-1 games.

Currently, the listmod chooses whether a game will be 5-3-1 or 6-2-1 based simply on how many people happen to be queued up. Maybe this might need to change.

[TierShift]

Have you corrected for multiple comparisons? I'm not quite sure that that is statistically significant.

If there is some sort of relation between newbie slots and winrate, I do not understand it. Whatever I can think of, does not nearly explain the large difference.

[Mina]

This has been brought to my attention, and confuses me, because it seems like the exact opposite result of what I'd expect intuitively. I'd have thought more experienced players = a better town overall. (Unless this is due to replacements, or to something like people giving less of a shit about newbie games when the queue has fewer active newbies.)

Assuming this pattern isn't caused by other factors or random chance, wouldn't sticking to one or the other actually make newbie set-ups MORE unbalanced? You'd have a point if either the 2-SE or 3-SE set-up was significantly closer to 50% than the other. But apparently, 2 SEs mean scum is disadvantaged, while 3 SEs mean town is disadvantaged. Actually, that means if list mods hadn't been changing it up for years, we'd have had much more lopsided results if these two equally unbalanced distributions weren't averaging each other out!

(It seems obviously better for the site if half of newbie games are rigged for scum to win 60% of the time and the other half for town to win 60% of the time than for town to only win 40%. Well, "obviously better" assuming someone even noticed the change--like I've said before in Matrix 6 discussions, I personally prioritize giving newcomers a more positive first experience over stats.

Here's where people should nag me to do newbie surveys already!

)

[hitogoroshi]

I have a hunch this is because the most important replacements for town are replacements in flaked slots and games with fewer SE's are more likely to have/need to have an SE replace in to the flaked slot.

Can you run the numbers by final distribution of newbie/SE/IC instead of starting?

[Espeonage]

Could also correlate with having just one experienced player in the scum team increases the rate, thus the higher rate of one experienced player in a scum slot increasing scum's win rate.

[Mina]

hito, since only the original slot's "rank" is recorded and not the replacement's experience, I don't think it's feasible unless you force poor Toomai to look up the entire game history of every single replacement in the past two hundred games to see if he/she has played three games with one outside The Road to Rome. I'm kind of curious to know how win rates correlate to the ratio of SEs/ICs to newbies on each faction. It's probably a combination of hito's and Espeonage's theories.

[prawneater]

Isn't the sample size too small to draw conclusions?

[Toomai]

Can you run the numbers by final distribution of newbie/SE/IC instead of starting?

No unfortunately, for both technical and practical reasons.

I'm kind of curious to know how win rates correlate to the ratio of SEs/ICs to newbies on each faction.

I had this in v1 of the project. Don't remember why it's not in v2. I'll see if I can revive it.

Isn't the sample size too small to draw conclusions?

The typical yardstick of significance is 30 samples, and we have 130 of one and 67 of the other.

[Toomai]

Okay stat acquired.

"421-110" is "town has 4Nb, 2SE, 1IC; scum has 1Nb, 1SE, 0IC". And so on.

This isn't statistically significant overall at the moment, but the pattern is evident.

[Mina]

Thanks, Toomai! TBH, it doesn't look like there's much of a pattern (well, I guess someone could make the "two scum newbies means they'll flake and get replaced by awesome people!" argument )

[Zachrulez]

Could also correlate with having just one experienced player in the scum team increases the rate, thus the higher rate of one experienced player in a scum slot increasing scum's win rate.

This is what I think is happening. More experienced players being in a game make it more likely that one or both draw scum and those games probably take a more pro scum direction than one that has newbie scum. There are exceptions... but that's what I think is happening.

Counterintuitively I think cutting back on the amount of experienced players in the newbie queue would be a better answer than finding the right amount of SEs. Something like 2 ICs or 1 IC and 1 SE or even 1 IC period. I think newbie scum having to contend with experienced players that happen to be town for longer in a game make it more likely for newbie scum to ultimately be found and can be a unbalancing factor.

I don't think there's an easy answer though, the above is just my hunch on what a solution should be.

Edit: Ultimately the question is what you want to have more influence on the newbie games overall? The experienced players or the newbies themselves? I would argue that newbies should be determining the course of their games for better or worse as often as that can be achieved.

[TierShift]

Newbie scum have the highest winrate, so that doesn't work. It also doesn't explain why there's such a large difference in winrate.

[Zachrulez]

Newbie scum have the highest winrate, so that doesn't work. It also doesn't explain why there's such a large difference in winrate.

I thought about that, but when you follow the newbie scum line over to the 2SE 1IC setup, they don't anymore so I'm not calling that statistically significant. (In regards to it smashing my theory anyway.) I think 3SE and 1IC is a system where the experienced players being town actually manages to destroy itself because statistically you would expect to find at least one scum in the group most of the time. It's an explanation I find easier to buy because I don't think newbie scum outplaying experience is the norm even though I wouldn't say it's impossible for it to happen.

Edit: Actually over on the 3 SE 1 IC side the winrate is higher when one of the SEs is scum. Missed that when I was replying to this post.

[TierShift]

I'm more often fooled by newbscum than by any other type of scum, anywhere. Maybe that's just me.