Page 1 of 2
Posted: Fri May 07, 2004 9:32 pm
by qwerty2
If you've got films from 1.4.3, 1.5b1, 1.5b2 please upload them to hl.udogs.net. I want a heap of normal gameplay films so I can put this dwarf/pus ghols dudding issue put to rest.
Posted: Sat May 08, 2004 1:15 am
by Woden
qwerty2 wrote:If you've got films from 1.4.3, 1.5b1, 1.5b2 please upload them to hl.udogs.net. I want a heap of normal gameplay films so I can put this dwarf/pus ghols dudding issue put to rest.
Not a bad idea, but if people are intentionally sending you just films where dudding occurs in above average amounts you aren't going to get a sample of films that can be analyzed to find accurate trends.
Ultimately if you don't like the idea of 'artificial' tests (i.e. firing up a solo desert game and having a duff throw 100 or so bottles) then you are going to need to look at a random sample of films to get an accurate picture of what is going on.
Posted: Sat May 08, 2004 3:26 am
by qwerty2
Woden wrote:Ultimately if you don't like the idea of 'artificial' tests (i.e. firing up a solo desert game and having a duff throw 100 or so bottles) then you are going to need to look at a random sample of films to get an accurate picture of what is going on.
ya I know, I wont be using maps that can skew the datasets either like winter or drowned empire.
Posted: Sat May 08, 2004 2:16 pm
by Orlando the Axe
I'm making a couple test films hold on.
Posted: Sat May 08, 2004 2:36 pm
by Orlando the Axe
Ok I just made two films of six dwarves firing away for a little over 5 minutes each time. One in 1.3 and one in 1.5b2 I didn't take any statistics on them, some throws bounce and explode into other throws so those will have to not be counted but there should be a fairly large amount of data here. I can do it longer if needed, hopefully it won't be. I'm uploading it to my dropbox on hl.udogs.net
Posted: Sat May 08, 2004 2:39 pm
by Orlando the Axe
by the way the file is called OtA dud test.zip
Posted: Sun May 09, 2004 5:13 am
by qwerty2
I don't need or want those, I want actual games. The boring throwing bottles over and over again doesn't prove anything. It didn't find anything with the drawf hard death either.
Posted: Sun May 09, 2004 8:08 am
by lank
if anything raw throwing tests in each version should give you a control set against which you can compare actual game performance. worth having them i would say...
the hard death test did show if nothing else that the random probability of game events (generation of projectiles from projectile groups, in that scenario) was not skewed in any particular direction by a significant amount between 1.3 and 1.4. more specifically the engine was following the instructions as given by the tags themselves, even if the tags were set "incorrectly" for 1.3-traditional play.
Edited By lank on 1084108205
Posted: Sun May 09, 2004 8:48 am
by Archer
I've got a film where I go eight or so pus throws without a single detonation… damn that was annoying.
How do you want it uploaded? Anonymous FTP?
~J
Posted: Sun May 09, 2004 3:09 pm
by Orlando the Axe
The point of the throws is that it is much fairer and less biased than game samples. For one you get many more throws, secondly as Archer has just shown people are more likely to send in a game where you get a lot of duds... and not as likely to send in game where say there were no duds at all. It's more of a controlled experiment.
Posted: Sun May 09, 2004 6:37 pm
by lank
yes, there's no reason the random number generator should behave differently in a controlled test than in an actual game.
also, in a game you can have bottles throwing everywhere on the map, and a controlled test you can do them all in one section at a time and check. or you can set up a system to record actual duds (bounces may not be possible except by direct observation) versus detonations.
Posted: Mon May 10, 2004 5:41 am
by Woden
qwerty2 wrote:The boring throwing bottles over and over again doesn't prove anything. It didn't find anything with the drawf hard death either.
Actually it did, I just didn't realize what I was looking at.
Posted: Mon May 10, 2004 7:11 am
by qwerty2
I know all about controlled tests. I will explain again, I do not want controlled tests, I do not want films soley because you dudded 8 times. What I want are films from normal games.
The reason I dont want controlled tests is because it doesn't prove much, lank is right that I can use the controlled test as a control but thats all I am going to use it for.
For example if dwarf hard death was tested soley on a controlled test of a duff dying then it wouldn't have shown that they drop more bottles. I tested this myself, shot duffs with archers over and over again but couldn't find anything.
Posted: Mon May 10, 2004 8:32 am
by lank
it required a specifically set up controlled test, which showed that the engine was behaving perfectly and inspired me to make sure the prgr was actually ok.
if you're looking for an anomoly in the random number generator as it pertains to causing duds a controlled test can't be beaten in terms of number of data points or quality. you're also able to set controlled tests up so that you can test each variable separately, as i did with the dwarf hard deaths (forced them to hd at very low damages). similar tests can be set up to find out exactly how many duds there were and exactly how many detonations (although it's harder because of bouncing, but it shouldn't be hard to do the maths to figure out what you should see anyway).
if you can make sure that you have an even (i.e. random/blind) sampling of films of the right type then you're going to get results, but in a much harder and more round-about fashion.
i'm not trying to dissuade you from doing these tests, but i understand less and less well exactly what facet of the code you're trying to test here.
admittedly i'm getting more and more interested in setting up a rigourous set of control tests anyway. if you're testing the statistical nature of the engine, again, controlled tests can't be beaten. if you're looking for anomolous behaviour compared to 1.3 then once again the same, and you'll do better by comparing values in any tags that have been edited between 1.3 and 1.5.
anyway, i'll try and shut up about the value of controlled tests. i've certainly made my point.
once again, qwerty, good luck. 
Posted: Mon May 10, 2004 8:46 am
by Brainbug
qwerty2 wrote:I know all about controlled tests. I will explain again, I do not want controlled tests, I do not want films soley because you dudded 8 times. What I want are films from normal games.
change the duffs from "dwarf riot" to normal M2 duffs, and then make some ppl download and play 10 full FFA games, i guess that will be enough to check things out...