Simulation accuracy : Analysis : TheSilphRoad

Simulation accuracy : Analysis : TheSilphRoad

I hesitated a looooong time earlier than posting this evaluation, however….

Well, right here we’re !

So at the moment, I wanna speak about Simulator accuracy for Pokémon GO and the way they don’t seem to be… nicely, correct.

To reveal this, I took the freedom of downloading and analyzing this raid video by KaitoNolan. I took particularly this video because it comprises many of the issues I wanna speak about (Mostly Deaths and Quick/Charged assaults.), It’s additionally a clear run because it comprises no rubberbending of any type and no transfered charged assault upon loss of life.

Quite the barbaric time period I used right here. Basically, if considered one of your pokemon begins a charged assault and dies earlier than the transfer hits the opponent, then the remainder of that transfer’s timer is carried on the following pokemon, which will not have the ability to act till that timer is completed So let’s get on with the evaluation.

  • The video was analyzed utilizing a video encoder software program referred to as “Yua”. It shows a timestamp aswell as a Frame counter (which is able to turn out to be useful later on this put up).

  • The place to begin of the Raid I select is when the Run away and the change button seems (You can begin attacking). The ending level is when these buttons disappear (Attacks are now not accepted by the sport). This ends in a 3816 frames lengthy battle (or 127.327 seconds RTA). Note that the video runs at 29.97 FPS

  • Attacks have been counted by checking if the transfer triggered the Boss’s HP bar to flash. I additionally double checked if the vitality bar was rising accordingly. Time begins on the primary body the vitality is incremented and ends X seconds later in accordance with the GAME_MASTER knowledge.

  • Damage Data comes from and strikes knowledge comes from the GAME_MASTER file.

5 Pokémons have been concerned within the raid struggle, the sixth went un-used.

Let’s introduce our fighters :

  1. Gengar, Hex/Shadow Ball, 15/15/15 IVs, Lv. 40

  2. Alakazam, Psycho Cut/Shadow Ball, 15/15/15 IVs, Lv. 40

  3. Bannete, Shadow Claw/Shadow Ball, 15/15/15 IVs, Lv. 40

  4. Absol, Snarl/Dark Pulse, 15/13/15 IVs, Lv.40

  5. Cacturne, Sucker Punch/Dark Pulse, 15/15/15 IVs, Lv. 40

This first knowledge will depend the variety of profitable assaults throughout the raids, aswell because the dodges. The ensuing complete harm can also be displayed.

I splitted the info into “waves”. Each wave begins with a Fast assault and finish with both a loss of life or a Charged assault.

The harm appears to checks out with our information of Tier three bosses having 3k HP, nevertheless you would possibly discover that I highlighted 2 numbers :

What occurred there’s that within the 4th Psycho Cut of Alakazam’s Wave four hit Gengar on the similar time it is Focus Blast landed on Alakazam, killing him. I am unsure if the hit truly counts in that case however I could not discover any rubberbending, so I left it for transparency.

For Catcturne’s Wave, the 4th Sucker Punch landed on the identical body because the struggle ended. As it’s the server who decides when a struggle ends, I would be stunned if this assault was truly accepted by the server and despatched again to the consumer in lower than a body as there’s at all times some latency. So once more, unsure if that hit counted, however I left it for transparency.

However, this doesn’t change a factor. One of these hits are legitimate for positive. (I would personally say the Psycho Cut is the legitimate one.)

If the Psycho Cut is the one invalid, the harm depend would fall to 3008. If the Sucker Punch is the one invalid, the harm depend falls to 3007. Either manner, it is nonetheless above 3k.

Now that we have now these datas, it is time for the half that pursuits me : The timing.

So on my facet, the struggle lasts for 127.327 seconds *in actual time*, so I am gonna take Pokébattler’s timing knowledge which incorporates Quick/Charged delay and Death time, apply it to each wave and see if it checks out.

But earlier than I do, to maintain issues truthful, I am gonna observe the wasted time first. That means the time the place Kaitonolan was inactive or the time wasted/gained by dying throughout a transfer.

Those timings are as comply with :

  • Gengar wasted time : Frames 801-822 (Zero.701 seconds).

  • Alakazam wasted time : Frames 1102-1118 (Zero.533 seconds); supposedly dies Zero.23 seconds early. We’ll go with 7 frames (Zero.233 seconds), so Zero.three seconds wasted.

  • Banette wasted time : Dies 6 frames early (-Zero.2 seconds)

  • Absol wasted time : Frames 3033-3056 (Zero.767 seconds)

  • Cacturne wasted time : Frames 3599-3621 (Zero.734 seconds); Recovers theoritically Zero.four on the finish (Fight ends at the start of the short assault harm home windows, which is Zero.3s out of the Zero.7s cooldown). Total of Zero.334 seconds misplaced.

Alright, now we have now some truthful knowledge to work with, they are going to be added within the type of “Wasted time” within the spreadsheet. Regarding the sport knowledge, we’re working with :

And now we apply Pokébattler’s knowledge :

We run the outcomes :

And as you may clearly see, there’s a huge discrepency of round 6.5 seconds in these calculations. It does not appear to be a lot right here as a result of the struggle was cleared by a 49 seconds margin. But it makes fairly the distinction when simulating tighter fights.

So why such a discrepency ?

Let’s run 2 Hypothesis.

Hypothesis N°1 : Lag isn’t correct :

Pokébattler’s calculation of lag is merely a mean of the anticipated lag inside a struggle. What if KaitoNolan had manner much less lag than what you’d usually anticipate ?

With that in thoughts, we’re gonna run the identical timing simulations however with extra optimism, as in no lag in any way.

three seconds off. We’re nearer, however that is nonetheless an enormous discrepency.

Hypothesis N°2 : The clock is mendacity (and the sport runs barely sooner within the video than meant) :

So perhaps our values are appropriate however the in-game timer is mistaken.

To test that, let’s rewind the video again to Frame Zero. The clock begins at 177 seconds. However, it solely stays as such for six frames (Zero.2 seconds) earlier than ticking all the way down to 176, which might indicate the clock begins round 177.2 seconds.

Going ahead in time, the raid stops 5 frames after the clocked displayed 49 seconds, which might implyed there’s nonetheless round 49.833 seconds on the clock when the raid stops.

Making the substraction, 177.200-49.833 = 127.367. Merely Zero.04 slower than the RTA, so I would say there isn’t any points with the clock.

So, why are we getting such a distinction between Pokébattler’s time and the in-game time ?
To discover out, let’s go again to the start of the video.

However this time, not solely will we depend the variety of assaults, however we will even notice down their timestamp, or the time within the video the place the transfer landed.

So how was lag calculated right here ?

For the primary hit of the raid, I merely substracted the primary window harm of Hex in accordance with the GAME_MASTER.

For each consecutive Quick assaults, I substracted the distinction in timing aswell because the cooldown for the transfer. If there was any dodge or wasted time throughout the assault, I additionally substracted that aswell, since they don’t seem to be a part of the lag.

For each charged assault, I substracted the remaining cooldown of the earlier Quick Attack transfer, the primary harm window for the charged transfer, and once more, any potential wasted time.

For a Fast assault following a Charged assault, I substracted the remaining cooldown of the charged assault and the primary harm window of the short assault.

The common lag obtained is round 24ms. This is shut from what Pokébattler makes use of for his or her simulations.

However, you may need observed one thing : I disregarded the lag between a loss of life of a pokemon and the following pokemon’s Fast Attack. If you’re observant, you may additionally discover that they’re far sooner than the supposed 1.9 second loss of life time delay that Pokébattler has calculated for his or her knowledge.

You could have additionally observed that I made no distinction between Fast assaults lag and Charged assaults lag. That isn’t essential however I believed I would point out it.

So what if we stand on the truth that the Death Time is 1.9 second and that the sport is simply glitchy ?

If we do this, we encounter 2 points :

There could be 2 invalid Psycho Cuts, 2 invalids Shadow Claws, 1 invalid Snarl and a couple of invalids Sucker Punch.

This makes a discrepency within the vitality depend, since there should not be sufficient Quick assaults in each Wave 1 (Except Gengar) to throw their respective charged transfer.

But for the sake of argument, let’s assume that it is one other glitch from the sport that provides us free vitality. We nonetheless have one other situation.

Mainly, the harm counter is all the way down to 2941 ! 59 HP quick from defeating the boss. This would not be such an enormous situation on this case, as a result of there’s nonetheless nearly 50 seconds on the timer, however think about on tighter fights, this is able to be an enormous deal !

So it appears like there’s a huge situation on our arms concerning loss of life time. This is without doubt one of the two major inaccuracy of simulators that I wished to level out. So let’s set the report straight and assessment the mechanics of the battle.

Death time.

To be truthful, I am unsure I understood how Pokébattler made their calculations. If I bought this accurately, they calculated between the second of loss of life and when the fight log confirmed “super effective” for the primary time.

However, Combat log solely applies when the pokémon is bodily current on the sphere (When it is identify and CP seem). But whereas the fight log does not lie, the sport does.

There is at all times a discrepency between what’s proven on display screen and what’s taking place within the recreation. It’s largely noticeable on Android the place you may click on buttons earlier than they really seem on display screen.

GoBattleSim has taken the method of setting 1 second as their loss of life time in accordance to the GAME_MASTER “SWAP_DURATION_MS” worth. I can not blame them for that, however that worth is definitely a lure.

So how does it work then ?

First, it is essential to notice that the sport treats swapping and deaths like 2 distincts strategies of switching.

When a pokémon is manually swapped out, the sport applies the “SWAP_DURATION_MS” delay of 1 second. Here is the difficult half : When the pokémon arrives on the sphere, it doesn’t have a correct goal assigned. This is the place one other worth of the GAME_MASTER comes into play : “RETARGET_SECONDS”. This worth applies an additional delay of Zero.5 second, after which the pokemon will get it is goal assigned and enter is permitted for the participant.

However, a loss of life isn’t handled as a swap by the sport. Instead, upon a pokémon’s loss of life, the following one is straight away out on the sphere. But but once more, it has not goal assigned, so, the sport calls upon the “RETARGET_SECONDS” worth and add an additional Zero.5 second delay, after which the pokemon will get it is goal assigned and enter is permitted for the participant.

So, Death time would seem like round Zero.5 seconds, however this isn’t precisely true. It could possibly be decrease or greater relying the cellphone that you’ve.

Since more often than not, you are gonna die from a Charged assault, there’s gonna be numerous particles displayed on the sphere, mixed with the loss of life animation, this will trigger a whole lot of lags on gadgets. So the worth could possibly be decrease if the sport compensate for lag an excessive amount of, or greater if it doesn’t compensate sufficient.

I reveal this behaviour within the following video :

But then, why the delay ?

Even if the Pokémon’s HP reaches Zero on the consumer facet, the loss of life animation and the swap will not occur till the consumer receives an acknowledgment that the pokémon is lifeless from the server.

I imagine this Zero.5 seconds is to account for many latency on the client-server communication and stop an unfairness throughout battles for many who have greater latency. This manner, I personally recommend the server sends out a packet containing the timestamp of the Pokémon’s loss of life, after which the consumer would swap the pokémon and permit enter on the timestamp + Zero.5 seconds (and never from when the swap truly occur), this is able to account for community lag, however not gadget lag nevertheless (In this situation, the consumer may permit firing an assault earlier or later than meant if earlier lags has offset the in-game battle timer)

Attacks lags.

” Lag is outlined as unintentional delay attributable to processing time, programmer bugs, web delays and different unwanted side effects of being a multi-player recreation. Quick Attack Lag is the additional delay between the tip of a fast assault and the beginning of the following assault. Quick assaults are queued on the server facet, and customarily have very quick delays.

” Special Attack Lag is very similar to Quick Attack Lag. It is defined as the delay between the end of a special attack and the beginning of the next quick attack. For unknown reasons, Special Attack Lag is higher than Quick Attack Lag. “

Source :

The definition of a lag is appropriate, however “Quick attacks are queued on the server side” appears to be a bit complicated to me. As this appears open for interpretation on how the entire system works, right here is my speculation :

To forestall delays, the sport can “store” assaults. For instance, for those who double faucet your display screen quickly, 2 fast assaults will probably be saved within the recreation, and they are going to be each executed consecutively on display screen with out additional enter from the coach.

The recreation then handles domestically the cooldown of assaults, aswell because the harm to the defender and your vitality.

The defender nevertheless, works in a “predictive mode”. That means the server sends the sign to the consumer to make the defender make both a Quick or charged assault. However, if a quick communication error have been to happen, the consumer would attempt to predict as much as 1 assault from the defender till receiving correction from the server.

So how does the server suits in all of this ?

My private speculation could be that the consumer despatched to the server a knowledge within the type of “I damaged the defender at X seconds remaining on the clock”, one after the opposite, and the server would solely test for the validity of the transfer, and would then replace the battle on the server facet.
But if the server receives an invalid transfer from the consumer, then the server would resend each battle knowledge that it has to the consumer, and the consumer will updates accordingly.
Lastly, if the server occurs to lose communication for a sure time period, then the server would cease the defender from appearing any additional whereas awaiting reconnection. The timer nevertheless would proceed to depend all the way down to Zero.

This speculation was made by working 2 experiments.

The first experiment I made was to go on a battle and deliberately lower the communication with the server, after which I might begin attacking time and again with solely fast assaults to see if the sport would permit me to take action.

Now you are in all probability questioning how do I forestall the sport from speaking with the server with out triggering the “No Internet connection” error ?

Well, the way in which the sport checks for connection is by merely trying if there’s an energetic connection or not reasonably than wether or not it has entry to Internet. So by establishing a hotspot on a second cellphone that has no SIM card nor Wi-Fi and connecting to it, we will successfully trick the sport into considering that there’s a connection whereas there is not.

The anticipated outcomes to verify the assault was queued server facet is that if :

However, as seen within the video under, I used to be in a position to assault the defender “just fine” on the consumer facet.

As for the delay, it’s nonetheless current :

The second experiment was to go on a battle, quickly faucet on the display screen to retailer 2 fast assaults in reminiscence and rapidly lower the connection earlier than the second assault will get an opportunity to show on display screen.

This was made by switching the display screen’s cellphone off by urgent the facility button after the two faucets on the display screen.

The anticipated outcomes to verify the assaults have been queued server facet is that if :

As seen within the video under, neither of these outcomes have been achieved. I attempted many occasions with completely different timings, however I couldn’t get it to not roll again the second assault.

You can discover my experiments on these movies :

Remember that these experiments merely represent my proof for the case, not undisputable proof, so do not take it as absolute fact. My understanding/logic of the entire thing CAN be flawed. Unless somebody goes forward and checks the total communication methodology between the consumer and the server in a struggle, how a battle work will probably be left to debate.

So I recommend the lag is fully client-sided, however now you could be questioning why ?

Remember at the start of this put up, I advised you that KaitoNolan’s video was working at 29.97 FPS. In addition, my 2 experiments movies have been encoded at a full 30 FPS, however it’s kind of to go away it at that for the anlysis.

I am unable to communicate for KaitoNolan, however as for myself, the RAW footage that I started working with was not fixed. It was oscillating between 25 and 29.12 FPS at finest in a standard gymnasium battle.

For reference, my first exepriment initially runned at a mean of 28.082191 FPS in accordance with VLC, though AZ Recorder was set to report at 30 FPS.

So if the recording does not attain a continuing 30 FPS, I extremely doubt the sport does. And even when it did, the framerate between the unique and the encoded footage would nonetheless be completely different, inflicting inconsistencies when analyzing it.

So now, you could be questioning why I did not analyze the RAW footage then ?

I could not. Vegas would not open the file and Yua was distorting the video, extending it and creating lag the place there was any. Furthermore, the footage from KaitoNolan’s video was instantly taken from YouTube, which implies it was already NOT the unique footage anymore, because it was already edited.

So now, does it imply that the lag is merely a results of the framerate distinction and doesn’t truly exist in-game ?

Well, no. As we noticed within the outcomes, there are unfavorable and optimistic lag, so it is proof that the sport continuously tries to appropriate itself. However, there’s nonetheless an quantity of round 2.6 seconds of lag, I am nearly sure such a diffrence cannot be attributable to the framerate except you have been to utterly decelerate the video.

But how far does it makes my evaluation off ?. Well let’s try !

Side notice : It does make form of a distinction that the lag is consumer based mostly as a substitute of server based mostly, becausetas lag is not equal for each gadget, it could be pointless to simulate battles with a preset lag that is not correct to your gadget. Just wanna throw that on the market.

So with all the pieces we have discovered up to now, it is time to make one other calculation on KaitoNolan’s raid.

But earlier than that, we have to fill within the final of the lags with our new loss of life delay of Zero.5 seconds. So I am gonna substract that aswell as the primary timing for the harm window of the Quick Attack.

Quick rundown :

  • Alakazam first assault : Zero.264 seconds of lag

  • Banette first assault : -Zero.050 seconds of lag

  • Absol first assault : -Zero.148 seconds of lag

  • Cacturne first assault : -Zero.099 seconds of lag

With that, our new complete lag quantities to 2.570 seconds.

So, as well as of our lagless time and new loss of life time, 2.570 seconds will probably be added to the full end result.

Only Zero.Zero55 second discrepency. That’s not good, but it surely’s extraordinarily nearer than our 2 earlier outcomes !

Now the place may these final 55ms of inaccuracy ?

Well, I’ve 2 speculation :

I am solely human after. Interpreting what’s a wasted body based mostly upon the place I feel a Fast assault ended is fairly subjective.

But it would not clarify all the pieces. After all, if I missed 1 wasted body, the inaccuracy would go all the way down to Zero.022 and if I missed 2, the inaccuracy would go all the way down to -Zero.Zero12. Either manner, it isn’t Zero.

Again, that is so far as video evaluation can carry us.

You see, once I calculated lag for a charged assault for instance, I would take the remaining cooldown of the present Fast assault and add to it the window of the Charged assault after which substract the entire on the timing distinction.

Let’s take for instance Banette. When I calculate his lag on Shadow Ball, I take the remaining cooldown of the earlier Shadow claw, which is theoritically Zero.45 seconds, add to that the two.four seconds needed for Shadow ball to hit, and substract the entire timing to the distinction between timestamps.

However, this sum provides as much as 2.85 seconds, which is unattainable to get on a 30 FPS video.

The closest we will get is both 85 frames (2.833 seconds) or 86 frames (2.866 seconds), so there’s certain to be some lag on my spreadsheet. In this case, it occurs to be a Zero.019 second lag. The similar factor will also be famous when calculating Absol’s lag.

As I am not precisely positive how I may appropriate this inaccuracy, I favor to go away it as-is.

So, does that imply that simulators are to not be trusted ?

Well, within the case of GoBattleSim, for instance, the one inaccuracy are these concerning loss of life time (and doubtlessly lag for a few of you), so the one consequence could be that the Time To Win displayed could be greater than what it is purported to show.

Futhermore, for those who occur to make use of dodging throughout your runs, then the simulators can by no means be correct to your playstyle, since the preferred simulators in the marketplace both dodge nothing or dodge all charged/all assaults.

The actual situation is that a greater loss of life time mess with the rankings of Glass cannons, that are clearly liable to dying rather a lot.

So as a basic rule of thumb, at all times take a simulator end result with a grain of salt.

If a simulator deems a struggle as do-able however you may’t do it, perhaps it does not simulate sufficient lag in comparison with what your gadget is outputting and vice versa.

Unfortunately, this does not maintain true for each simulator…

As I stated within the put up, Death delay and assaults lags have been the two issues that bothered me concerning simulation accuracy.

For the aim of this put up, I analyzed just a few random battle logs from Tier four/5 bosses and got here throughout one other inaccuracy that appears to be particular to Pokébattler (a.ok.a I did not discover this on GoBattleSim)

Normally, when a Raid Boss fires a 1-bar charged assault. There is a “buffer” the place the boss should then throw a minimum of 2 Fast assaults minimal earlier than being allowed to throw one other charged transfer. I suppose on that half, Niantic determined that 1 bar strikes have been compulsory far more highly effective than multi-bar charged assaults and they also put that in place to forestall the raid boss from decimating groups in 30 seconds.

However, it looks like Pokébattler does not take that into consideration. Analyzing three battle logs (It was actually tough, for some purpose, my browser saved crashing when loading the total log. Those logs have been from Ho-oh battles with Extrasensory/Steel Wing and Solar Beam aswell as Steel Wing/Fire Blast), I came upon that the raid boss was allowed within the simulation to fireplace off charged transfer with as little as 1 fast assault between every charged. Sometimes, they have been even fired again to again !

This is an enormous situation because it closely penalize as soon as once more Glass cannon, but it surely additionally favors extra tanky Pokémons to get greater into the leaderboard (More harm = More vitality, so extra charged assaults/harm)

This situation does not happen when simulating as solo makes an attempt because the boss does not get the prospect to achieve 100 vitality having firing off 2 or three fast assaults (In basic ? There could be some instances the place you may pull that off.) So tier three and under on the whole are spared from this inaccuracy (in addition to solo makes an attempt in Tier four and above, however they’re solely correct as solo try, not multiplayer makes an attempt).

At the state the place issues presently stand, I personally drew off 2 conclusions by way of this complete evaluation :

It has what you anticipate from a simulator. You can customise from gamers to the content material of your events. It feels very user-friendly and does not even require signing up, so it is accessible to everybody from the get go.

It has it is share of inaccuracies as said above, BUT, and for this reason I like this simulator, you may even change the battle parameters used for the simulations ! So meaning for instance you could change your lag to no matter worth you are feeling is true out of your gameplay expertise and get a way more correct simulation than with the common preset. To edit these settings, Click on “Customise” and within the sub-menu, click on “Battle Settings”, you may change them from there. The solely draw back is that this window isn’t precisely user-friendly, though it isn’t that onerous to learn. It appears like a greater presentation of the factor could be a lot welcome.

I am unsure how excessive of an influence the second situation talked about has on Pokébattler’s simulator, It could also be quite a bit, it could not, but it surely positively have an effect. As such, I am unable to regard Pokébattler’s leaderboard of counters as correct (Although it should get many of the order proper a whole lot of time, as a result of Pokémon GO is such an unbalanced recreation.).

Again, this solely stands as my private opinion

And with that, I conclude my evaluation concerning simulation accuracy.

Feel free to debate my factors within the feedback and proper me for those who see something mistaken with my evaluation.

And lastly, thanks for studying this far (for those who did not skip right here). This was actually an enormous mission, and being alone, an exhausting quantity of labor.

See on Reddit

Leave a Reply