Page 1 of 1
Posted: Wed Jul 28, 2004 9:13 am
by PoohBear
I have several directories of hundreds and hundreds of game films and I find managing them to be very time consuming and prone to error.
I know that a Myth "film" is a collection of all the keystrokes and results in a game, that you can't rewind it, and that "viewing" a film basically amounts to replaying it in fast or slow motion.
As a concept, is a "film parser" even possible outside of the Myth application? I've heard of "chat extractors." Can basic game info be similarly extracted independent of the Myth engine? The info I'm looking for would be statistics commonly useful to tournament officials: player name, login, teams, map name, version, and so on.
Thanks in advance.
Posted: Wed Jul 28, 2004 9:24 am
by Doobie
I believe the answer is yes, and I think someone made just such a tool, but I can't remember who it was or where it is now...
Who is it that's a big film freak... damn, i should know this. Maybe it will come to me later.
Posted: Wed Jul 28, 2004 12:58 pm
by Orlando the Axe
can you figure out logins from films?
Posted: Wed Jul 28, 2004 1:33 pm
by Baak
Who is it that's a big film freak... damn, i should know this. Maybe it will come to me later.
That'd be me! 
I actually have managed to get just about everything I can out of the films, but there are several "support files" needed for the really obscure stuff that I gather.
And alas, some things are only available by using Myth code (such as determining outcomes of random teaming, etc.) and things like "who won", etc.
The app I wrote is for the PC and I no longer have a public version due to the complexitites of some of the support files (Perl scripts) and the fact that I don't have time right now to support it. I began work on a completely-Perl version of it as well and am most of the way through it.
In addition as part of the Perl version I built in a way to extract all the game type vars for maps like how many flags, balls, for each game including custom flags/balls (this was REALLY involved!). Ultimately I plan to put this in some kind of mysql/php db on-line so it will be searchable (I've processed about 1,000 maps so far).
Right now you can see the results of my Film Renamer & Cataloger - here's a link to my order's film catalog and you can see all the data that gets extracted (there are almost 3,000 films cataloged in there so it takes a bit to load but it'll give you the idea of what it extracts in the first few lines):
OoH Game Catalog
(if you want it all in an XL spreadsheet that is also available on the Hall of Records page as a zip file):
OoH Hall of Records
I also tie the output of my Film Renamer & Cataloger to another Perl script that outputs a nice Summary of all the films (this one is formatting correctly):
OoH Game Catalog Summary
And my pride and joy these days is my Ultimate Chat Extraction that dumps into an HTML file including all those wonderful uber mac chars that I convert via some serious magick into HTML - check this out - go to our Film Paks page and click on any of the "View Chat" links to view the extracted chat - it's sweet (color coded and everything *AND* I figured out how to timestamp each line including showing Planning Time, Sudden Death, and Post Game chat!!) - here's an example from last week's OoH Match:
Ulltimate Chat Extraction Example
I want to someday have a hosted version of all this run so you can post a zip of films and then get the results back accordingly. Alas, Work, Life & Other Stuffâ„¢ get in the way. 
Cheers!!
~Baak
P.S. Hey Doobie! Could you nudge iron to glance at my last two posts here - one is a tiny tweak request for 1.5.1 (wasn't sure where I was supposed to post this?) - the other is a hard crash I encountered and wasn't sure where to post either - been a bit "out-of-the-loop" lately 
Thanks!
Edited By Baak on 1091049650
Posted: Wed Jul 28, 2004 3:04 pm
by Baak
Heh - so I *had* to figure out why the current OoH Catalog was not printing properly in plain text...
So I've fixed it and have uploaded the new files. The Catalog now spaces correctly - but note there is no "header" line showing what the columns represent. Many appear blank because somewhere in the entire catalog they are filled (there are almost 3,000 films now!). Fun ones (to me) are sudden death lengths.
The first sections in the catalog are parts of the renamed film file that I use for sorting, etc. - there are only 31 chars allowed for a film name so I made best use of them. An example film name is:
040725 0027 BNQT.Redsand_v^ StB
[Date] [Time] [Firstlettersoffirstfourplayernames] [.=5+players] [Mapname] [^=extrapluginon] [StB=Steal the Bacon]
All details beyond that are in the Catalog listing for that film. 
Seeya,
~Baak
Edited By Baak on 1091049501
Posted: Thu Jul 29, 2004 11:46 am
by Baak
Hey PoohBear,
I can't promise to do this for anyone and everyone, but if you can .zip your films into one zip file and post it somewhere for me to download, I'll run it through the FR&C and zip up all the results (renamed films + catalogs, etc.).
Sound good?
Posted: Thu Jul 29, 2004 12:17 pm
by El Tainted Blissarino
I understand the whole replaying the keystrokes and such...but how does it keep track of duds in that case? Are they not totally random, or are the random numbers from the original game stored as well?
Posted: Thu Jul 29, 2004 12:38 pm
by Baak
I don't understand everything about how all the information is stored in the films (it still amazes me), but I do know this:
The random numbers are generated like many random numbers on a computer, using "pseudo-random" generators - they are not "purely" random. The random number generator that spits out these pseudo-random numbers one after the other has to start somewhere, and that somewhere is a "seed". It uses the seed to combine in a mathematical way to generate the next number and continue from there.
Typically the seed is the current time (in seconds from 1/1/1970 or whatever), but it can be anything.
The trick is - and here's the neat part - starting from a given seed (let's say it's 79952369), *all* the pseudo-random numbers generated starting with that seed will follow the same exact sequence once started with that seed.
So... all you have to do is save *one number* - the seed - and you will be able to generate every single random number after that in the same exact sequence!
My guess is: it's a combination of the seed, timestamps, etc. that can recreate the game in exact detail.
This is how duds remain duds in films, etc. The sequence of random numbers is going to be the same each time the film is played.
Long ago when I was SERIOUSLY hacking apart the headers of the films I found the seed and fiddled with it. Lo and behold, it wreaked complete havoc on the film - all the starts/team combos were different, and the whole game unfolded differently.
So the seed is the big key.
But most of the rest of it remains a mystery to me. Other than extracting the chat from the guts of the film, and every single bit of useful info from the header I can think of...
I really wish you could figure out winner/loser via the film. Not sure why they didn't just store that info at the very end. It really wouldn't have been difficult.
Posted: Tue Aug 03, 2004 1:08 am
by CyberBob
Posted: Fri Aug 06, 2004 11:22 am
by PoohBear
Baak, I'm blown away by your generous offer. Thanks, I'll get going on bundling all my loose films into tidier directories.
I clicked your links and, again, wow! The Game Catalog Summary is the one that's most intriguing -- it reminds me of those scary health factoids, like "the average American drinks 35 gallons of nondairy creamer per year" or somesuch.
Have you considered running your utilities on the current crop of MWC tourney films? I have a pretty thorough (and renamed) collection of films I can send you. Suppose all the chat of the games were online: Contemplating the ability to cull a complete collection of team in-fighting makes me tingle.
Posted: Fri Aug 06, 2004 11:43 am
by Baak
lol, yes! It is just about the most detail I could glean from the films!
(oh how I wish the win/loss/perm info was at the end though!)
I'm leaving on a vacation this weekend, so you might want to hurry to post the zip file of your films for me to process. A big zip that I can get to via the web is the best way.
If there are third-party maps that I don't have info for (I have a *lot* (about 1,000) but obscure ones I sometimes miss), then we can get those after I return.
I'll check into the MWC films when I return as well.
If I get a chance this Holiday Season I may just dust off the Perl version of the Renamer/Cataloger and see if I can't get it working on-line. My ultimate goal would be to have something where you could upload zips of films and then a zip of the processed films would be waiting for you 24 hours later.
Baak's 24-Hour Myth Film Processing! :laugh:
When I left off with the Perl version I had actually got it to work for Myth TFL (this was harder for third-party plugins than Myth II), Myth II, *and* Myth III. Would be great to have it do them all automatically, wouldn't it?
:: Goes off in search of the 36-hour Day ... ::