ScumBait

This forum is for discussion related to the game.
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

ScumBait

Post Post #0 (ISO) » Thu May 08, 2008 9:05 am

Post by weilawei »

I noticed on the wiki that there were a couple tools to aid mafia players. Well, I hacked together the very first version of a new tool aimed at providing vote counts, logs, and stats for Mafia players and mods alike. This is the 0.2.1 omg-kills-still-dont-work-but-post-ids-do release.

Features
  • Vote, unvote, and FoS tallies
  • Action and ignored action logs
  • Player activity percentages
  • Per player vote, unvote, FoS logs
  • Output in BBCode, JSON, and plain text
  • Save game support
Coming soon!
  • Kill support
  • Player replacements
  • Player activity rankings
  • Web/GUI interface?
  • All around more stats
Requested Features
  • Citations/quote records
  • Charts and graphs
  • Overall player history
Bugs

If it breaks trying to refresh a saved game, delete the save file and retry. This will be fixed for 0.3.
  • Settings are re-imported as dicts, not Settings objects
  • Abbreviations are seen as new players
  • Junk votes are counted
Usage Notes

This is a command line tool. I may make a GUI or web wrapper if I have time. You can specify the moderator, the game, and the start/end points where ScumBait will scan a thread.

ScumBait will always load a saved game (threadnumber.game) if there is one. To create a saved game, run with the --json option and no other reports. Save the output in threadid.game. If you don't wish to use saved data, specify --refresh to start anew or specify --start to continue from a given post. Note that you need to specify these
after
the game option. (I know this all sucks, but I'm working on it.)

Kills are currently not accounted for. This makes it harder to use for entire games, thus I'd love suggestions on what a reasonable format to expect for a kill is. I've just joined the fora, after all.

Installation

ScumBait is written in Python and you'll need to have BeautifulSoup and SimpleJSON. I've included them for convenience. It should run on any OS supporting Python.



Thanks to Kison for contributing code for post IDs. Also, thanks to Surye for offering hosting. The next release minor release (0.3) will be hosted at scumbait.datamachine.net.

From The Help


Code: Select all

ScumBait 0.2

Copyright 2008 Rob Crowther <weilawei@gmail.com>
This software is Open Source Software.
Please see the LICENSE for more details.

Available options are:
--help or -h: Display this help message
--version or -v: Display version notice
--refresh or -r: Use new data instead of saved game cache
--game or -g: Specify the thread id of the game
--bbcode or -b: Display output in bbcode
--tally or -t: Tally votes
--fos or -f: Tally FoS's
--log or -l: Log of player actions
--ignored or -i: Log of ignored/unrecognized actions
--json or -j: Dump JSON data.
--mod or -m: Specify the game moderator
--start or -s: The post to begin on (defaults to 0)
--end or -e: The post to end on (defaults to end of thread)
--player or -p: Print a report on a player

Sample usage: ./scumbait.py -b -g 10000 -m weilawei -t
This will display a bbcode tally of votes from thread 10000, run by weilawei.

If there is a file named with the game id and extension '.game',
it will be used to load settings and vote data.


Examples


To grab vote tally, FoS tally, action log, and stats for all players in game 1000, with moderator Mod:

Code: Select all

python scumbait.py -g 1000 -m Mod -tflp all


To save game 1000 data for quick report generation:

Code: Select all

python scumbait.py -g 1000 -m Mod -rj > 1000.game


To continue game 1000 from post 136:

Code: Select all

python scumbait.py -g 1000 -m Mod -s 136 -tflp all


To grab fresh data for game 1000:

Code: Select all

python scumbait.py -g 1000 -m Mod -rtflp all


To grab game 2000 being moderated by Mr. Mod:

Code: Select all

python scumbait.py -g 2000 -m "Mr. Mod" -tflp all
Last edited by weilawei on Sun May 11, 2008 12:58 pm, edited 14 times in total.
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
shaft.ed
shaft.ed
dem.agogue
User avatar
User avatar
shaft.ed
dem.agogue
dem.agogue
Posts: 4998
Joined: August 15, 2007
Location: St. Louis

Post Post #1 (ISO) » Thu May 08, 2008 10:03 am

Post by shaft.ed »

Thanks for doing this weilawei. Gemelli had a similar program but couldn't continue hosting it.

Just a tip, you might want to remove the output of the ongoing game and use a finished game for an example.
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #2 (ISO) » Thu May 08, 2008 1:20 pm

Post by Adel »

thanks weilawei!

now, before I download it and run it, could someone with the proper skill set (and who isn't a "townsperson") please vet the source for security? I don't feel like getting pwned today.
User avatar
Elmo
Elmo
Mafia Scum
User avatar
User avatar
Elmo
Mafia Scum
Mafia Scum
Posts: 3047
Joined: September 7, 2007
Location: happy

Post Post #3 (ISO) » Thu May 08, 2008 6:03 pm

Post by Elmo »

Sounds nice! Image
Succinctness is pro-town.

Passion is inversely proportional to the amount of real information available. ~ Gregory Benford
User avatar
Khelvaster
Khelvaster
Mafia Scum
User avatar
User avatar
Khelvaster
Mafia Scum
Mafia Scum
Posts: 1265
Joined: May 5, 2007

Post Post #4 (ISO) » Fri May 09, 2008 6:53 am

Post by Khelvaster »

Adel wrote:thanks weilawei!

now, before I download it and run it, could someone with the proper skill set (and who isn't a "townsperson") please vet the source for security? I don't feel like getting pwned today.
Lag actually does exist in real life. For proof, look no further than Jesus: When he died, it took him three *days* to respawn.
User avatar
Surye
Surye
Mafia Scum
User avatar
User avatar
Surye
Mafia Scum
Mafia Scum
Posts: 2330
Joined: December 23, 2007
Location: San Diego, CA
Contact:

Post Post #5 (ISO) » Fri May 09, 2008 12:03 pm

Post by Surye »

If you want some hosting for this, shoot me a PM. I've been working on a similar tool in Python and PyQt. What's the license on your code? We might be able to help eachother out.
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #6 (ISO) » Fri May 09, 2008 1:06 pm

Post by weilawei »

The 0.2 release is upon us! Go check out the first post in this thread for details.


New features include a per player activity stat and logs and working savegames.
Surye wrote:If you want some hosting for this, shoot me a PM. I've been working on a similar tool in Python and PyQt. What's the license on your code? We might be able to help each other out.
I'd definitely be interested in better hosting than using free upload services. It's licensed under the MIT license, which I feel is less restrictive than the GPL.

Also, the a GUI sounds really great. I'm sure that with the JSON output, you'd be able to incorporate data from ScumBait in no time.

Before I go back to refactoring and writing code, I've got two issues currently. Votes, unvotes, and FoS are always put in bold and are roughly written the same. That's great. What's the definitive way to say how many players must be lynched and the way to say a player has been killed?
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
Elmo
Elmo
Mafia Scum
User avatar
User avatar
Elmo
Mafia Scum
Mafia Scum
Posts: 3047
Joined: September 7, 2007
Location: happy

Post Post #7 (ISO) » Fri May 09, 2008 2:14 pm

Post by Elmo »

I never got very far with my tool-that-is-similar-to-this, but I thought about some of this. I don't believe there is a single definitive way to say who's dead. One can take a guess based on the fact that the person who starts the thread must be the mod, and the mod will usually have both posted a 'morning, alice is dead' post and maintain a list of players in the first post - most of that is probably fairly parseable if you made it resilient to occasional weirdness. 'Lynched Day [1-9]' is fairly easy, weird killing methods (f.e. first post of Fritz Fest) could probably be found by looking for 'Bob ... night 3'. An interesting direction is simply to figure out who is still alive based on who posts (more than once) after night (or one could detect 'bah!' posts :)). You could certainly use that for doublechecking.

p.s. Adel, Khelv: I have looked at the source briefly, and I would be comfortable running it on my computer.
Succinctness is pro-town.

Passion is inversely proportional to the amount of real information available. ~ Gregory Benford
User avatar
ShadowLurker
ShadowLurker
9 years old
User avatar
User avatar
ShadowLurker
9 years old
9 years old
Posts: 3491
Joined: August 8, 2006
Location: hot cause he's fly

Post Post #8 (ISO) » Fri May 09, 2008 2:48 pm

Post by ShadowLurker »

I thought there was going to be childporn when I opened this topic.
:sadtorch Ken Hoang, A.D/Fuzzie, Cameron Ferris, Taj Johnson-George, Annie Duke, Patti Blagojevich, Maria/Tiffany :sadtorch
:torch Tammy/Victor, Dan/Jordan
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #9 (ISO) » Fri May 09, 2008 3:39 pm

Post by Adel »

check out this thread for some functions some players would like to see in this kind of tool:

http://www.mafiascum.net/forum/viewtopic.php?t=7863

I think trying to code a set-up balancing equation is a tar pit which is best avoided.

some highlights:
- Post count by player
-Word usage tallies by player
-Voting records/timeline
*how many times player x quoted player y
*how many words outside of quotes player x's posts average
*graph of activity level (words, posts, votes, citations by other players) by player, with post number serving as the unit of time

there could be a interface that would let you easily enter a pre-defined evaluation of each post. We could probably generate a pretty definitive list of possible casual relationship between player. It would be a trivial matter to display posts one at a time in a display that also features the radial button for defining those relations ships... each button could open a slider for inputting a % chance guesstimate.
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #10 (ISO) » Fri May 09, 2008 3:43 pm

Post by Adel »

I have python 2.5 installed under XP (C:\Python25).

Which folders do I place the ScumBait, BeautifulSoup and SimpleJSONin files in? What do I type into the command line for these files to load?
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #11 (ISO) » Fri May 09, 2008 3:51 pm

Post by weilawei »

Adel wrote:I have python 2.5 installed under XP (C:\Python25).

Which folders do I place the ScumBait, BeautifulSoup and SimpleJSONin files in? What do I type into the command line for these files to load?
Assuming you saved it to C:\Code\scumbait\ like I have it, you'd do something like:

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 8000 -m ModName -tfl -p all


Note that this window is going to close as soon as it finishes, so you might want to open a command line (cmd) and pipe it to a file like so:

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 8000 -m ModName -tfl -p all > c:\my_log.txt


Now, if you have c:\Python25 in your PATH environment variable, you won't need to type out the full path to python. Also, you -can- put BeautifulSoup and SimpleJSON in your c:\Python25\Lib\site-packages\ folder--but they should work where they are in the scumbait\ folder.

If you're on OS X or *nix, you just need to make sure it's executable (chmod +x scumbait.py) and do a command like:

Code: Select all

./scumbait.py -g 8000 -m ModName -tfl -p all
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #12 (ISO) » Fri May 09, 2008 3:54 pm

Post by Adel »

as for GUI stuff, this is the kind of visual products I produce by manually collecting information and then assembling via graphics software. A program that would automate this process could actually lead to some very strong mafia play based on quantifiable evidence:

http://www.mafiascum.net/wiki/images/f/ ... _whome.gif
http://www.mafiascum.net/wiki/images/d/ ... ravhen.gif
http://www.mafiascum.net/wiki/images/f/ ... 5vd2d1.gif
http://www.mafiascum.net/wiki/images/a/ ... player.gif
http://www.mafiascum.net/wiki/images/3/ ... player.gif
http://www.mafiascum.net/wiki/images/c/ ... yer515.gif

Doing these is so time consuming that I don't, even though I've had some great results from this approach (see the link in my sig for an example)
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #13 (ISO) » Fri May 09, 2008 4:09 pm

Post by weilawei »

Python has a vast range of available libraries, most of which I don't feel like re-implementing. I'm a professional+paid programmer, so I do these things in my spare time after work and don't have time to reinvent wheels.

NetworkX looks like a great graph library for Python, simple to get started with. I might consider support for graphs before the GUI, because the GUI is really just going to be a wrapper in many ways. Image output is a more core functionality.

Either way, I have a huge TODO list of things to do. (Oh yeah, and vim is the world's best editor. Sorry emacs guys.)
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
Macavenger
Macavenger
Goon
User avatar
User avatar
Macavenger
Goon
Goon
Posts: 768
Joined: March 10, 2008
Location: Oregon

Post Post #14 (ISO) » Fri May 09, 2008 4:12 pm

Post by Macavenger »

weilawei wrote: (Oh yeah, and vim is the world's best editor. Sorry emacs guys.)
Vote: weilawi
LAL
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #15 (ISO) » Fri May 09, 2008 4:12 pm

Post by Adel »

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 7270 -m Adel -tfl -p all > c:\my_log.txt

produced a text file named "my_log" only containing
Option 'p' is not recognized.
For more information see -h (--help).

what did I do wrong?

http://www.mafiascum.net/forum/viewtopic.php?t=7270 is the game I was trying to grab information from.
Last edited by Adel on Fri May 09, 2008 4:17 pm, edited 1 time in total.
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #16 (ISO) » Fri May 09, 2008 4:16 pm

Post by Adel »

weilawei wrote: NetworkX looks like a great graph library for Python, simple to get started with. I might consider support for graphs before the GUI, because the GUI is really just going to be a wrapper in many ways. Image output is a more core functionality.
NetworkX does look great, and I really appreciate the time you've put into it so far! I was just sharing ideas. I haven't written a line of code in over 14 years, so I certainly can't do what you can do.
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #17 (ISO) » Fri May 09, 2008 4:32 pm

Post by weilawei »

I really appreciate someone besides me testing this out. =D

I'd suggest re-downloading it and trying again. I downloaded a fresh copy and it worked. I copy and pasted the command line you used.

If it still doesn't work, I'll test it out on another computer in the morning. The 0.1 release didn't have support for the -p switch.
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #18 (ISO) » Fri May 09, 2008 4:32 pm

Post by Adel »

Image
is from the NetworkX gallery.

There are all kinds of variables I would love to be able to assign for the circle diameter, line width, circle color, and line color!
Last edited by Adel on Fri May 09, 2008 4:33 pm, edited 1 time in total.
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #19 (ISO) » Fri May 09, 2008 4:33 pm

Post by Adel »

weilawei wrote:The 0.1 release didn't have support for the -p switch.
damnit! I didn't DL the new version. I know better... sorry about that.

I downloaded the new version, and I got a good dump. I'll comb through it tomorrow for accuracy. Thanks for walking me through it this far.

You may have just saved me 10 hours a week.
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #20 (ISO) » Fri May 09, 2008 4:56 pm

Post by Adel »

issue 1:

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 7703 -m Mr. Flay -tfl
 -p all > c:\testaab.txt
bad dump

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 8222 -m Mr Stoofer -t
fl -p all > c:\testaac.txt
bad dump

Code: Select all

c:\python25\python.exe c:\code\scumbait\scumbait.py -g 8286 -m EmpTyger -tfl
 -p all > c:\testaaf.txt
good dump

I think the "Mr. Flay" and "Mr. Stoofer" fail as moderator names because of the space.

~~~

issue two: some player post stuff like "
vote:we end this day allready
which your script interpets as a vote for "we".

~~~

issue three: some players post "
vote:WW
" instead of "
vote:WildWest
"

~~~
general feedback:


in the log summary it would be useful if the post number was listed for each line.
example:

Code: Select all

elvis casts first vote for rabbit.  (post 4)
carrot unvotes for elf leaving 0 vote(s). (post 9)
carrot casts first vote for elf. (post 9)

instead of

Code: Select all

elvis casts first vote for rabbit. 
carrot unvotes for elf leaving 0 vote(s).
carrot casts first vote for elf.
Last edited by Adel on Fri May 09, 2008 5:08 pm, edited 1 time in total.
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #21 (ISO) » Fri May 09, 2008 4:59 pm

Post by weilawei »

I do have a problem with names. It's hard to figure out when a sentence is ending and when it's just part of someone's name. So far, it defaults to the shortest possible form of someone's name.

I think I may try to grab the player list at the beginning and use an algorithm for close forms (similar to predictive texting). Anything else will get tossed into the ignored actions list. (Which. incidentally, is what I use to see when I'm missing votes and other odd things happen.)

TODO >_<

EDIT: I've added the post number into the log format, but it's still just going to say 0 until tomorrow when I get around to putting more than "return 0" in the id_of_post() function. Also, I've fixed names somewhat; they now recognize periods and spaces in the name. As long as the -entire- vote is in bold. Moderator names haven't been fixed yet as that's in a different part of the code. Short forms still don't add up to a vote for the full name.
Last edited by weilawei on Fri May 09, 2008 5:55 pm, edited 3 times in total.
[size=67]you will birth a god and build an eternal paradise[/size]
User avatar
Adel
Adel
Crystalline Logick
User avatar
User avatar
Adel
Crystalline Logick
Crystalline Logick
Posts: 6743
Joined: May 23, 2007
Location: Central Oregon / High Desert

Post Post #22 (ISO) » Fri May 09, 2008 5:14 pm

Post by Adel »

Kills are currently not accounted for. This makes it harder to use for entire games, thus I'd love suggestions on what a reasonable format to expect for a kill is
well, when >50% of living player vote for someone that results in a lynch, but some games have deadlines with other factors that come into play.

Players who die during the night are often listed in the second post by the moderator after a player got enough votes to be lynched.
User avatar
shaft.ed
shaft.ed
dem.agogue
User avatar
User avatar
shaft.ed
dem.agogue
dem.agogue
Posts: 4998
Joined: August 15, 2007
Location: St. Louis

Post Post #23 (ISO) » Fri May 09, 2008 5:25 pm

Post by shaft.ed »

I now have to vote entirely in code so that the robots cannot take my brain away.

Side note, why no Asian chess champs? I thought they were smart or sumtin.
User avatar
weilawei
weilawei
Townie
User avatar
User avatar
weilawei
Townie
Townie
Posts: 95
Joined: April 27, 2008
Location: Taxachusetts

Post Post #24 (ISO) » Fri May 09, 2008 5:35 pm

Post by weilawei »

shaft.ed wrote:I now have to vote entirely in code so that the robots cannot take my brain away.
You do that and I'll sic Bruce Schneier on you.
[size=67]you will birth a god and build an eternal paradise[/size]
Post Reply

Return to “Mafia Discussion”