Mini Normal Stats Update

[callforjudgement]

Hoopla, when doing this analysis, did you get much of a sense of what proportion of the game played "last scum PR alive can use action/kill", and/or how many scum wins were achieved with one living PR scum?

As a current NRG member: we make mods decide on "all scum can action+kill" or "no scum can action+kill", and encourage them to specify which is the case in the ruleset. (Most mods outright place a rule in the ruleset saying how many abilities can be used per night.)

It may be interesting to compare the game results to the length of the review thread, too. Some reviewers go over games much more thoroughly than others, and some games are almost impossible to get into a runnable state. (Also, note that historically not all Mini Normals were checked for balance; there are a few that were allowed to run with a "this game is not checked for balance" warning, and in most of those cases the reviewers believed that the games were highly scumsided.)

[callforjudgement]

One problem is that there's lots of things to consider when you're reviewing a setup than just town:scum win balance. For example, excessive swing is less fun than an imbalance, often: if you make a setup where the scum can be effectively defeated by D2 and it actually happens, the scumteam will be very upset. Likewise, you have to ensure that the setup won't be decided by role assignment (if there's a role that badly needs to not be NKed, you don't want it to go to a player who always gets NKed N1, but there's no way to ensure that when randomizing roles correctly, and thus you have to get rid of the role altogether). You want to try to preserve the mod's idea behind the setup (unless it's utterly unworkable), so that it isn't just a case of the NRG picking out setups for mods. And of course, you need the setup to be one that'll actually be fun to play.

In general, this may mean that being mildly (but only mildly!) scumsided might actually be more fun than being balanced. The easiest way to "balance" a scumsided setup is to increase the chance of the town completely locking scum out early, but they won't enjoy it if that happens. In an extreme case, imagine a meta in which scum are generally very good and town are pretty bad by comparison (not saying that necessarily describes the meta here; it's a thought experiment). Pretty much the only way to get a 50:50 win rate will be to give town a chance of winning the game at random despite a lack of scumhunting ability, and the scum will feel highly cheated if it actually happens.

From another point of view, imagine a game that ends in 2:1 lylo. Then imagine a game which ends in 2:1 lylo but with a confirmed townie. Having seen these situations repeatedly from a number of viewpoints, the former feels more balanced to the players (regardless of alignment), but the latter is closer to balanced theoretically, and most likely in practice as well (it's very hard to read players in 3p lylo and looking back over the history of the game isn't as helpful as you'd think). Now consider what sort of games lead to these endings. Hitting 2:1 with no confirmed players is pretty easy. Hitting 2:1 with a confirmed townie, though, means (barring combinations involving bulletproof players who can either confirm players themselves or be confirmed by other players) there were two confirmed townies at the start of the night before. Two! Tweak the numbers a bit, and suddenly the setup-with-two-confirmed-townies-that-became-a-balanced-endgame is a setup-with-so-many-confirmed-townies-that-scumhunting-is-pointless,-town-win-easily-and-the-scum-are-upset.

And from another another point of view, 10:2 and 11:2 mountainous are both disastrously scumsided (in addition to their other known problems), and yet it's common for both factions to feel like they have a chance through much of the game. (I suspect 9:2 or maybe even 7:2 mountainous would actually be

more

enjoyable, despite being even more scumsided.) This probably works better if the players don't know it's scumsided, though.

And from yet another point of view, I once replaced into a game for which there was only one set of actions that gave my faction even a theoretical chance to win, all the actions in question would need to have been taken by other players, and they had no reason to do so in any case. I enjoyed myself, right up until the end when I realised it was impossible to win (but then the game was almost over so I just played it out). What was important was simply that I didn't know I had no chance to win, because it let me play to my win condition, scumhunt, drive lynches, and all the other things that are part of the game.

Anyway, after all that arguing to say that games being scumsided is not necessarily indicative of a problem, I will say that I'm not convinced that there isn't, in fact, a problem. Some of the most scumsided games could probably have been made more balanced or even townsided with changes that wouldn't have fundamentally changed the nature of the setup nor allowed for a blowout, and doing so would be a good idea if possible.

I think the most useful thing to do here would be to find specific examples of scumsided setups that have been allowed through the Normal queue, that a) were reviewed for balance; b) were perceived as scumsided postgame; c) (not required but would be helpful) have a public review thread. Then we can debate how it went wrong and what should have happened instead.

(As a counterpoint: did any excessively townsided setups slip through the review net recently? I know you sometimes get cases where players claim the setup was townsided, but oddly those setups are often won by scum, which means that the players may well be incorrect in their assessment.)

[callforjudgement]

I'm very leery of letting roleblockers into games I review. Preventing swing isn't a good reason to add a scum roleblocker (town roleblockers, sure, but not really scum roleblockers). The main reason to add a scum roleblocker into a game is as a method of preventing the game being broken by massclaim, and even then you need to put in a lot of safeguards to prevent the roleblocker taking over.

I was talking in general terms about the problems that go into reviewing, rather than about any particular problem. As I said, I feel it's quite possible that games are being misreviewed. (When I'm on a review team, it's not always the case, but sometimes I'm doing the lion's share of the reviewing and the other reviewers just do an /approve at the end. This leads to the possibility that some reviews I'm not on have nobody putting in effort, although it might just be that when one reviewer starts taking charge the others sit back, but everyone's willing to step up to the plate if nobody else has yet. I'm going to need to read some reviews for completed that didn't involve me at some point and see what I think of them.)

[callforjudgement]

ps: 11:2 mountainous isn't "disastrously" scum sided; it's just like 60-40

In actual games, it's 100-0. That could just be a small sample size, but it's kind-of too large a winrate to ignore.

[callforjudgement]

Hmm, so let's look at some review topics for completed Mini Normals. I'm going to focus on ones I wasn't involved in (both so that I can give an "outside opinion" and to avoid any risk accidentally spoiling an ongoing game due to not realising it's ongoing). Other reviewers and people who chip in, feel free to review mine! It's always worth improving as a reviewer.

---

This is the most recent such topic (unless I've missed one). The original, mod-suggested setup was Tracker, Doctor, 1-shot BP, 7×VT vs. 1-shot Strongman, 2×Goon. The three reviewers rated that original setup as townsided, highly scumsided, and marginally scumsided but balanced enough to run.

This kind-of shocks me; to me, the setup is scumsided and highly so. (The easiest way to see this is to replace the Tracker with an unlimited Cop; I'd still be far from certain that the result is even townsided, let alone broken in town's favour which is what you'd expect after making that change in a hypothetical balanced setup.) So I agree with one of the reviewers but disagree with the other two. The final version of the setup was Cop, Jailkeeper, 1-shot BP, 7×VT vs. JOAT (watch/track/roleblock), 1-shot Strongman, Goon; this passed with basically no discussion. I'd be staring at that setup for hours trying to work out how balanced it was and probably writing paragraphs of text, asking the other reviewers to check my reasoning; it's not an easy setup to review and I'm glad I didn't have to review it, so I'm surprised at the relative speed of the review.

As it happens, scum won the actual game. Postgame discussion makes it seem as though it's unlikely the setup was responsible, or at least nobody in the game seemed to have problems with the setup (unless I missed something).

---

Next up is this review. The originally posted setup was Hated JK, Doctor, 7×VT versus 3×Weak Goon, Traitor (as an Open). I assess the setup in question as highly scumsided (being an Open gives town a lot of power but it isn't enough to balance the huge advantages scum have), and the reviewers in the thread agreed with me. So at least there's agreement there :-D The final setup was Weak Bodyguard, 3×Bodyguard, 6×VT vs. Ascetic Traitor, 2×Goon (as an Open). There was a lot of discussion about the setup before the reviewers decided it was balanced, and this is a setup that needs a lot of discussion and that I doubt I can come to a balance conclusion on without spending a lot of time (which probably isn't worth it for writing this post). The game ended in a town win; most of the effort the reviewers went to was ensuring that the game wasn't breakable for town (always a risk in an Open). After the game, one of the reviewers changed their mind about the setup and said that it was too swingy to run (and probably also townsided), and the scum were fairly depressed, although given how unusual the setup is I can't really blame the reviewers for missing the issue. (I'd probably have tried to persuade the mod to get the game reviewed by the community at large, though. There's no reason to keep an Open review secret.)

[callforjudgement]

For what it's worth, even though I love White Flag as a setup and think we should keep running them, I

also

think it's scumsided in practice (although possibly not as scumsided as the results on the wiki page make it look).

But then, I also feel that (in practice; obviously not in theory) 2:9 is more townsided than 2:11 (because in 2:11 scum have more time to NK competent scumhunters before the game gets into the low numbers where scum are easier to identify). So I guess my balance opinion on vanilla games is quite a way off where the public opinion is.

[callforjudgement]

2 wins out of 7 is not anywhere close to a statistically significant result on the "actual" balance (never mind that only 5 are actually listed on the page, and one of those was 11p).

Right, I was trying to point out that they probably don't tell the full story (and I know they don't include every game). So I'm agreed with you there.

[callforjudgement]

Now I'm really curious as to whether town win rates went up after Encryptor became explicitly normal. Giving scum power when there's no reason to is a pretty common mistake, but Encryptors serve as a nice relief valve for the designs of newbie mods (it's one of the most commonly seen roles in setups designed by first-time mods). Note that it's actually negative utility compared to just giving scum daytalk.

On the subject of scum power roles, by the way, it's worth noting that you can give scum power roles that don't make their team stronger. In one of my Normals I gave scum a Tracker, and the main purpose of this was to try to steer them into fixing potential odd/even issues of the setup for me; given that many of the town's power roles didn't target, it didn't meaningfully help them hurt the Town (it might have improved their fakeclaims slightly but that's about it). Likewise, it's possible to give scum a power role that doesn't actually do anything in the setup, to mislead them about which of their players is the most valuable.

I definitely agree with the odd/even points made here. When I'm reviewing a setup, I count the power of a 1-shot vig as being based

entirely

on the ability for town to get back to odds (and the power of a 2+-shot vig is based on the chances of it hitting twice and/or the town saving a player). This is counteracted by the risk of killing a townie who could otherwise save themself by claiming, and so a 1-shot vig can often be negative utility. (The main upside for the role is that the vig can at least normally confirm themself by claiming the kill.) I should mention that saving roles like Doctor have exactly the same issue as killing roles like Vigilante (actually, I'd be willing to consider Bodyguard

more

useful for town than Doctor in some setups, because town may have to no-lynch anyway after a save and there's less ambiguity in the night results with a Bodyguard than with a Doctor). That said, this isn't a mistake I see mods commonly make nowadays; people are fairly aware of it. So it may be that analysing old setups doesn't make much sense to learn about current balance issues. (I'm much more interested in whether normals are being misbalanced right now, than if they've historically been messed up. What are the stats for Mini Normals since the Micro queue was established? That'd allow for a fairer 10:3 to 7:2 comparison.)

Incidentally, I was actually in one of the games you mentioned above (Mini 1304), and agree with you that it was highly scumsided. (That said, I actually figured out the scum the morning after the night I got nightkilled, which basically just shows that scum made a good nightkill choice. They were trying to be too cute, though, and could have won easily with a simpler strategy.)

Meanwhile, I disagree with the approach of assessing balance by counting roles. As an extreme case, 3×Cops + 7×VT versus 3×Goon is highly townsided even if you allow for the town potentially not believing the setup, whereas 3×Doctor + 7×VT versus 3×Goon is probably somewhat scumsided in practice even if you allow for the possibility of the Doctors identifying each other (e.g. at massclaim) and setting up a protection loop. My preferred method of assessing balance in a setup that I don't have a similar setup to compare it with is to count the number of players that are likely to get their alignment confirmed overNight over the course of the game (my current estimate for this when balancing a 10:3 is "slightly more than 3"), plus the number of players likely to be mostly confirmed via what makes sense in the setup at massclaim (ideally around 1).

An example: Looking at Mini 1445, we have Cop, JK, 8×VT vs. Tracker, 2×Goon; the Tracker effectively counts as a delayed vanillaise for the purpose of balance (because scum will kill a suspected power role the night after), so we can give the power roles a 11/13 chance of being alive at the end of N1, 8/13 at the end of N2, 5/13 at the end of N3. Let's call this 80%, 60%, 40% for the purpose of approximation (as the numbers aren't perfectly accurate anyway), so we can assume that on average the Cop and JK each get a little under 2 shots off (slightly less for the Cop due to the chance of an accidental block, slightly more for the JK as their role can save themself from dying as it confirms a scum). The JK is almost blank in this setup; if they stop a kill early and aim at unlikely kill targets they may guess that their target is scum, but unlikely kill targets rarely perform kills, and thus the JK's only useful late (when they probably won't be alive). Any positive effect they have is therefore mostly a rounding error. I'll happily allocate 1 and a half or even two confirmations to the Cop (it's a powerful role), and if I were being generous I'd add one massclaim confirmation to the town total (not two as scum can cause confusion by claiming power, with only two town power roles). We're still somewhat short, though. You could add something like an Innocent Child to this game and it'd be much closer to balanced (especially if they were revealed at game start to give some amount of JK WIFOM that'll likely work mostly in town's favour). In this case, my method comes out with much the same conclusion as Hoopla's. If the town roles were different, though (say Cop+Watcher, which are both very powerful and also synergise well with each other), I'd be willing to let the game run with just the two power roles, at least from the point of view of win:loss balance (and the setup's actually relatively low-swing for a two-power-role setup, with the main swing risk being the Watcher being shot or forced to claim early).

[callforjudgement]

What are some examples of town sided normal games that have happened? My gut was that mini normal 1775 was town sided, and I remember complaints on mini normal 1782 (though I disagreed there). Any that stand out from those who know a lot on those things?

1782 doesn't look obviously townsided to me (although I haven't analysed it in detail), although I'd have been very worried that it's excessively swingy (with most of the town roles getting better as scum's numbers dwindle, an early scum lynch would be very hard to come back from; and the JK result that gives town more information also leaves more townies alive). In the actual game, it looked somewhat like town snowballed after blocking a kill, which would have left the game feeling townsided. It'd have quite possibly felt equally scumsided if more town PRs had died early. Swingy games have an unfortunate habit of leaving someone upset pretty much whatever happens.

[callforjudgement]

the funny thing about guessing what roles make sense as town in a setup is that, other than the fact that most people can't do this (see: how they balance their own setups), it requires the mod thinking about it in the same way you do

You can sometimes do it if you're sufficiently in tune with the modmeta of the mod in question, but probably not otherwise.

neighborhoods aren't power neutral when a neighbor is a power role.

Neighbourhoods can sometimes be non-power-neutral even in the absence of that. Most notably, they reduce the target pool for town rolecops, as the fact that they're neighbours normally becomes evident after a while and thus there's not much point in role-investigating them; Micro 597 used this technique pretty well.

[callforjudgement]

Last time I was in a Normal where two people claimed Cop, they were both scum.

[callforjudgement]

We've been arguing that in the setup thread. Everyone agrees that the scum roles were powerful, but there's a lot of debate about how powerful the town roles are. (For the record, I was one of the reviewers, and thought that town might be

slightly

weak but couldn't be strengthened without making the setup dangerously close to being broken by early claims. I also thought that the setup was easily balanced enough to run.)

It should be noted that in the game itself, town would probably have won trivially if not for the Mafia Doctor, who was pretty much the only thing holding the setup together. I think the reviewers acknowledged that a dead Doctor early would leave scum with very little chance because the town had so much power.

[callforjudgement]

For summary information, there's the List of Mini Normals, which has results, but no information about the setups other than faction balance (it's also slightly outdated but not by much, and only contains faction information on more recent games).

For full information, you want the Mini Normal Archives; the format probably isn't what you'd want but can likely be parsed by computer.

[callforjudgement]

Bodyguard is a little weaker than 1-shot Cop, and (in most setups) stronger than Doctor. (If it stops a kill, you effectively confirmed yourself via "swapping places with" the best nightkill target, but the nightkill target doesn't become any less confirmed in the process). It's possible that I'm overestimating how good scum are at choosing nightkill targets, though, and thus overestimating Bodyguard as a result.

I suspect your sample size is too small to get really meaningful information about how much roles are being overvalued/undervalued (e.g. 1472 was won by town despite being clearly scumsided; 1590 is a very similar setup that's almost as scumsided (you can do Follow the Tracker in it to get a slight edge) but also was won by town). (There's also the issue of role interactions, e.g. one good reason to put Neighbours in a setup is to make any Rolecops in the setup more powerful by reducing their target pool, as the Neighbours tend to be exposed fairly quickly and a Rolecop rarely has reason to scan them.)

The results for 7 VT games are very interesting, mostly because 7 VT = 3 town power roles is the easiest point on the scale to balance. (It wouldn't surprise me if a major part of the reason more complex setups tend to be scumsided is that the harder a setup is to balance, the lower the bar for allowing it to run seems to be, because eventually nobody can be sure what effect a change would have.)

Finally, the perception of balance affects balance. If players believe a setup is unbalanced, that makes it more scumsided (because it causes them to draw incorrect assumptions from the lylo massclaim). This could well be why we seem to be stuck in a balance rut at 30-40%; it could be that this effect is forcing 50% games down to 40% no matter what we do. (I note that most new mods in the Mini Normal queue submit obviously scumsided setups and have to be persuaded to make them more townsided; it's likely that the site as a whole has similar opinions on balance to the new moderators.)

Anyway, I have a suggestion: we decide upon an intentionally "townsided" balance standard (along the lines of a 1-shot Cop or actiated Innocent Child more townsided than a typical setup), and allow moderators to opt into the more townsided standard in reviews, then advertise it during signups (the previous standard would also be available). After we've had a few games we can compare to see if it's a) making win rates more balanced, b) more fun. (It seems plausible that the two goals will be in conflict; I can imagine that making games unusually townsided might increase the chance of a town blowout, which tend to be unfun. Or to view things through a simpler lens, games that go through 3:2 tend to be more fun than games that go through 4:1.)

[callforjudgement]

Successfully BG'ing an IC isn't zero value, but is close to it. Successfully Doc'ing an IC is MUCH higher value for obvious reasons (it stops a night kill, someone on town might see you do it, and if it takes it evens to odds, then you just gave town a HUGE advantage).

Let's take the simplest possible example going into night: Innocent Child, Bodyguard/Doctor, VT, Goon. Assume it's a closed setup, players haven't claimed yet, and scum kill the Innocent Child. With a Bodyguard, we're looking at a guaranteed 50% EV the following night. With a Doctor, scum can claim any role that could stop the kill and doesn't directly counterclaim a Doctor (e.g. Bulletproof), and now town has only a 33% EV (assuming that scum will no-kill if town no-lynch). The Bodyguard removes themself from the lynch pool; the Doctor doesn't.

This isn't a contrived situation, either; it frequently comes up in actual games. You obviously remember this incident, given how much we argued about it. Your slot was huge lynchbait at that point in the game (admittedly, you'd only just replaced in, but I'm assuming that you wouldn't have saved the slot just by your presence). If you'd Doctored the confirmed townie and then been lynched, town would likely have been worse off than if you'd Bodyguarded them

even though

it would have taken town back to odds! (Obviously, it would have been even more townsided if town were on odds at the time and there wasn't a vig messing up the odd/even analysis.)

Successfully Doc'ing an IC is MUCH higher value for obvious reasons (it stops a night kill, someone on town might see you do it, and if it takes it evens to odds, then you just gave town a HUGE advantage).

"Someone on town might see you do it" is a negative. How are you going to prove you're the doctor, rather than the

scum who tried to make the nightkill

?