Front page stats post

[HamsterIV]

This is in relation to the front page announcement:
http://gunsoficarus.com/blog/chewing-on-some-stats/#more-7147

Raw stats strait from the server are interesting. I suspected the Pyra was the most popular ship but not by the margin the numbers indicate.

What I am really excited about is the changes to the crew form. It benefits my play style of gathering 3 random strangers under my balloon and going off to cause as much trouble as my play time allows. However for more social/clan focused players I fear this change would be seen as a negative.

The underdog feature sounds interesting too but I wonder if it is going to drive off newbies when they see they are outclassed. The level hiding thing helped keep people in lobbies when they were ignorant of what they were facing.

[Squidslinger Gilder]

Really...really....Muse is using usage stats to decide when to nerf or buff ships? Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted. Course another question comes up, if they've had these numbers for years, why haven't they done anything about it? Clearly they look at the numbers then instantly decide to nerf the hell out of the higher numbers hoping that makes the crappier ships more viable.

Sigh...the fail is strong with this...

So underdog means literally you reward people for sucking and being stacked. So this is turning into 5yr old soccer in the sky.  I...I know someone is going to exploit this and make Muse look all kinds of stupid but I just don't want to even bother giving it more thought right now.

[Byron Cavendish]

Sadly statistics cant actually teach devs what is really happening in their own game. Spire may be the most used by the masses, but any of us can still take a lumberjack and a merc and shut down the whole ship.

[Koali]

I have a better idea: instead of nerfing the strong ships, buff the weak ones?

[Byron Cavendish]

Dude, I've brought this up in multiple firesides. The answer is something you can guess.

[Koali]

"We're working on Co-op." --> "No."

[Hoja Lateralus]

Really...really....Muse is using usage stats to decide when to nerf or buff ships? Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted. Course another question comes up, if they've had these numbers for years, why haven't they done anything about it? Clearly they look at the numbers then instantly decide to nerf the hell out of the higher numbers hoping that makes the crappier ships more viable.

Sigh...the fail is strong with this...

So underdog means literally you reward people for sucking and being stacked. So this is turning into 5yr old soccer in the sky.  I...I know someone is going to exploit this and make Muse look all kinds of stupid but I just don't want to even bother giving it more thought right now.

Pretty much this. Underdog makes every stomp match a win-win situation for worse players, which means they will have better MMR, which will make them face the vets more often, which... you get the idea. Also this is even more stupid since vets often take more casual/fun builds on order to make those matches at least 'a bit' fun, so those players will get MMR bonus for winning with our glorious flak spire, banshee mobula or munker. Brilliant.

Also, should we expect mobula buff since it's least used ship?
HUH, OKAY, LET'S THROW ALL THE BALANCE OUT OF THE WINDOW AND HIT IT WITH A FRIDGE IF WE'RE AT IT.
But to be honest I think they looked over the usage only in pyramidion case (we don't know the stats but I think before the nerf it could easily have got over 30% of usage) because it was so dominant. Notice that three most used ships are the easiest to fly/crew 'and' are the ones that are locked in the novice lobbies (so novice captains pilot them right after they leave novice to keep themselves competetive). Moreover, probably all of us take easier ships (such as those three) when flying with randoms on pubs and fly those harder ships mostly knowing they have trustworthy crew - friends/vets.

I like the voting thing very much. Finally Devs perhaps admitted that they actually can give people *gasp* what they really want? *thunder in the distance*
Voting mechanics is GOOD. Like, in 90% of the cases it makes the game better. That's why we've been asking for votekick for soooooo long. But okay, assume no votekick, still make scramble/non-scramble vote and perhaps change map voting availible (both are pretty common in CS:GO #csgomastergame).

"We're working on Co-op." --> "No."

You know what? I call bullshit. Devs use it as an excuse not to work on some things. If this is true - why are they working on Stamina for skirmnish? Why changing flamer mechanics? Why HUD changes? Why they gave us new gun instead of bugfixes or (I think) much more desired maps? They could've focused on minor stuff (like indicator of party/team/lobby voicechat) and just wait for the Co-Op. But they clearly aren't.

Oh, and speaking of statistics a guy called Thorin, CS:GO, LoL and Hearthstone commenter said this:
"Making games is not a science. Making games is also not an art. Making games is a combination of those two - a craft"

[Squidslinger Gilder]

They are working on Stamina because it is going to be huge to COOP. Trust me I flew COOP mode and Stamina is going to be a necessity if they don't buff a lot of ships. You can't keep up with the ships in it without Stamina.

[Byron Cavendish]

So just add it to COOP. I know difference of opinion on that, but still, the devs do have a real attention deficit disorder when it comes to projects.

[Omniraptor]

I really don't  get the hate for underdog mechanic. It only rewards the noobs if they win a stacked game, remember? And they don't usually win stacked games, that's the whole point of stacking. I agree though that muse don't need to provide another mmr safety net, the safety net is built into the glicko2 system already. Even so, the rewards mentioned in blog post are cosmetic! It explicitly says achievements and badges for winning as underdog.

Re: using match data to check balance of ships. Howard said they will now start correlating ship usage and mmr to get a more detailed picture of what vets vs noobs use.

Other amazing future improvements-

No more novice pilots in regular lobbies! No more second pilots! WOOT

Unimited rematches! No more being stuck with useless ally in crew form after a match! FINALLY WOOT

Custom game now has all matches! RETURN OF MATCHLIST WOOT

Now the only thing missing is named custom lobbies (duck pond)

[Indreams]

Goldfish: 12322 matches (22.5%), 5975 wins (48.5% win rate)
Junker: 7969 matches (14.5%), 3819 wins (47.9%)
Squid: 4892 matches (8.9%), 2496 wins (51.0%)
Galleon: 5135 matches (9.4%), 2412 wins (47.0%)
Spire: 6216 matches (11.3%), 3634 wins (58.5%)
Pyramidion: 14510 matches (26.5%), 7057 wins (48.6%)
Mobula: 3765 matches (6.9%), 2014 wins (53.5%)

Taking a look at this part, since I'm a big numbers nerd.

I would argue that usage rate is fine. Muse tells us that Goldfish and Pyramidion are "overused"; Squid, Spire, Galleon, and Mobula  are "underused". But if we think about it, aren't Goldfish and Pyramidion supposed to be well-rounded, relatively easy, and commonly used ships? Aren't Squid, Spire, Galleon, and Mobula supposed to be specific-talent, harder, and less common ships? We really shouldn't balance ships based on usage. That would be like buffing Medics in TF2 because you almost never see more than 1 per team.

Looking at Win Rates. Squid and Mobula's win rates looks about right because it is almost always veteran, competent players that take those ships. Pyra win rate is where it should be, since inexperienced players usually take this ship. Same goes for the Junker and the Goldfish.
However, Muse should take a look at the Spire. That win-rate is statistically significant. I ran the stats with the null hypothesis that the real proportion of Spire win rate is .5. alternative hypothesis that the real proportion is greater than .5, used the proportion p1 = 3634/6216 = .585, and p2 = 3108/6216 = .5. The resulting pvalue was 1.436*10^-21. Since pvalue < alpha, I reject the null hypothesis. There is sufficient evidence that the real proportion of Spire win rate is greater than .5.
For non-statisticians (and great-statisticians that got confused by my novice write up), this means that Spire is almost definitely having more wins than it should. I think the hull health buff it received last patch needs to be toned down a bit.

So, these are the things I can see from the numbers. Spire needs a fix. Other ships are fine where they are (for the moment). Muse should not base balance on ship usage. Balance and ship usage are unrelated things. Squid isn't being flown frequently and that's how it should be.

Muse, try to base your balance on player feedback, controlled tests, and mathematics (which the forum does for you once in a while).

And Muse, although you guys sometimes do things I don't like, I still appreciate the hard work, and I love Guns of Icarus Online.

[Crafeksterty]

We’ll always be diligent in listening to everyone’s feedback as well as in finding ways to use data to improve the game.  While we do, we also have a mission to continue to create and to innovate on the game.  This could at times affect a person’s use case, and as a result, we’ll need to do our best to understand it and account for it.  Over the past 2.5  years, we’ve been constantly innovating on the game. We’ve added in numerous features and systems, introduced new weapons and skills, and carved out better specialization for each ship, weapon, and skill,  all to make the game a better, more diverse, and more interesting experience than it was when we first launched.  Innovation, however, is a 2-way street.  While we never want to be complacent, we also ask our great players to not be complacent either, and to continue to partner with us on this wild and incredible ride. We could not have gotten this far without your amazing support and feedback.

And then

Sigh...the fail is strong with this...

I have a better idea: instead of nerfing the strong ships, buff the weak ones?

I like the voting thing very much. Finally Devs perhaps admitted that they actually can give people *gasp* what they really want? *thunder in the distance*

You know what? I call bullshit. Devs use it as an excuse not to work on some things. If this is true - why are they working on Stamina for skirmnish? Why changing flamer mechanics? Why HUD changes? Why they gave us new gun instead of bugfixes or (I think) much more desired maps? They could've focused on minor stuff (like indicator of party/team/lobby voicechat) and just wait for the Co-Op. But they clearly aren't.

Also, should we expect mobula buff since it's least used ship?
HUH, OKAY, LET'S THROW ALL THE BALANCE OUT OF THE WINDOW AND HIT IT WITH A FRIDGE IF WE'RE AT IT.

(When they clearly said that the next ship they will be looking at is the GALLEON)

Im quite sick of seeing really untoughtfull, asshole posts come up so quickly after minutes or a day of news coming in. Understandable to a degree because we are players whos dealt with many changes, but when they give us what we want (Which in this case the insight of how they can look at the game). Even worse comments arrive.

I feel like it would be healthier for them to not give us any good info just for their own mentality, lol.

I mean the underdog system sounds alot like the  "wanted" system. Alot of us commented on the fact that we would be harrased if we were wanted for the whole week. Some people even complained, but similarly i can see it being the same case for the underdog where if noobs are going to fight the big guys, theyl sky rocket down to the ground. But is it going to last a week? In a way it is suppoused to encourage newbies in taking it serious. What we dont know is exactly how much will this new system read our stats of ranking correctly. Is someone with a lvl 45 pilot and lvl 10s crew worth being targeted by an underdog team? I sure dont think so but you get the point.

"The system will be abused" let them abuse it, and youl get quicker games. We dont know how it will really work or if it will be considered in the very end, and no statements can confirm how it is going to play out.

Other amazing future improvements-

No more novice pilots in regular lobbies! No more second pilots! WOOT

Unimited rematches! No more being stuck with useless ally in crew form after a match! FINALLY WOOT

Custom game now has all matches! RETURN OF MATCHLIST WOOT

And weve all complained of these things, and now they are fixing it! YAY! Lets celebrate god damnit!

Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted.

Ive complained how the spire was big and easy to hit, yett more fragile than a junker whos tanky and hard to hit. Made even easier where the spire was slow.
I argued that the spire could get a health buff and or a speed buff in terms of acceleration. Muse brought forth an odd skew of what i wanted where both health and speed were the case but not as much as how i wanted. And it prooved to be a more balanced balance than what i had. The spire buff essentially was what I wanted for quite some time.

Spire needs a fix

I agree, but where. Maybe health down to 850 (from current 950)? Currently the spire is at a good spot but i cant help thinking that the majority of matches versus the spire dont know how easy it really is to kill a spire. Ive seen too many Gat, hades... Gat, flak... Mercury lightflak... Goldfishes despite being firepowerless, are tanky as hell in pub matches. Not many (at all) pub-players know proper explosive shot timing.

[nanoduckling]

Hi Indreams, fellow stats nerd here. The underlying win fraction for all of these ships except the squid is different from 0.5 (or rather to be more insufferably technical, with an alpha of 0.05 I reject the null in all of these cases using a normal approximation to the binomial test, with bonferroni post hoc corrections). What matters here isn't statistical significance (we were assured we would have it for all interesting effects just from the sample size), but the effect size.

However, even looking at effect size we have a problem, lack of control. The spire has a much higher win fraction than the other ships, 0.17 difference. That is pretty large. The stats don't tell us why though. If Muse are balancing the game based on just these aggregate numbers when they have the MMR then this is my disappointed face. I strongly suspect that the best statistical approach here would be not the straight foward binomial analysis myself and Indreams are looking at, but a generalized linear model which included ship and MMR as predictor variables, perhaps even with some other factors thrown in (were the participants in a novice match? Did one of the ships fit the bill as a nerf build? Other factors I haven't thought of?). Even then I think the model needs to be non-linear, and it is basically doing what the matchmaker does, predicting matches.

Without controls for things like player skill these aggregated numbers don't mean much. If they haven't already, Muse should probably hire a professional statistician if they are serious about balance, because the numbers the present here are largely useless if the are looking to justify recent balance changes. There is a big human element here so such an effort combines expertise in both the humanities and technical disciplines. To do this properly and completely you have to interview the target population to inform and enable the construction of a suitable operationalisation of the potential contributory factors, then combine that domain specific knowledge with a suitable statistical model including those factors, then validate the model with some test data. Only then are you really in a position to start meaningfully interpreting the statistics.

[Indreams]

@Nanoduckling

You are a better statistician than I. Salute!
I agree that these numbers don't really mean much. (that's always a problem with statistics, numbers almost never mean anything)
But its always fun to look at them (hence number nerd).

If they haven't already, Muse should probably hire a professional statistician if they are serious about balance,

That'd be super cool! Imagine having a fireside with a GOIO Statistician!


I get to apply boring school learning to exciting SteamPunk Airship Combat! I love it every time we have this kind of discussion on the forums! 

[Keyvias]

Guys, whoa, slow it down.
Balance is not decided by ship usage numbers, that would be horrible.

The numbers serve a couple of factors:
First and foremost, they're pretty cool to look at. As some of you guys are numbers nerds, you know, raw stats are interesting.
Second, noticing trends is a good way to find new ways to look at problems. Our experiences, the number crunching we do, and feedback from awesome vets, does not define every experience in the game.  It is important that we look at the game from every angle.
Third, you never know where the next source of inspiration comes from.  Does it come from looking at the numbers, trying an all harpoon ship, or playing some random $2 steam game. Information and experiences can all be great starting points.

We like to share this behind the scenes stuff because we find it interesting. We knew there would be backlash, but I think getting to show some cool numbers is a lot of fun.