Guns Of Icarus Online
Main => Gameplay => Topic started by: HamsterIV on April 08, 2015, 07:23:39 pm
-
This is in relation to the front page announcement:
http://gunsoficarus.com/blog/chewing-on-some-stats/#more-7147
Raw stats strait from the server are interesting. I suspected the Pyra was the most popular ship but not by the margin the numbers indicate.
What I am really excited about is the changes to the crew form. It benefits my play style of gathering 3 random strangers under my balloon and going off to cause as much trouble as my play time allows. However for more social/clan focused players I fear this change would be seen as a negative.
The underdog feature sounds interesting too but I wonder if it is going to drive off newbies when they see they are outclassed. The level hiding thing helped keep people in lobbies when they were ignorant of what they were facing.
-
Really...really....Muse is using usage stats to decide when to nerf or buff ships? Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted. Course another question comes up, if they've had these numbers for years, why haven't they done anything about it? Clearly they look at the numbers then instantly decide to nerf the hell out of the higher numbers hoping that makes the crappier ships more viable.
Sigh...the fail is strong with this...
So underdog means literally you reward people for sucking and being stacked. So this is turning into 5yr old soccer in the sky. I...I know someone is going to exploit this and make Muse look all kinds of stupid but I just don't want to even bother giving it more thought right now.
-
Sadly statistics cant actually teach devs what is really happening in their own game. Spire may be the most used by the masses, but any of us can still take a lumberjack and a merc and shut down the whole ship.
-
I have a better idea: instead of nerfing the strong ships, buff the weak ones?
-
Dude, I've brought this up in multiple firesides. The answer is something you can guess.
-
"We're working on Co-op." --> "No."
-
Really...really....Muse is using usage stats to decide when to nerf or buff ships? Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted. Course another question comes up, if they've had these numbers for years, why haven't they done anything about it? Clearly they look at the numbers then instantly decide to nerf the hell out of the higher numbers hoping that makes the crappier ships more viable.
Sigh...the fail is strong with this...
So underdog means literally you reward people for sucking and being stacked. So this is turning into 5yr old soccer in the sky. I...I know someone is going to exploit this and make Muse look all kinds of stupid but I just don't want to even bother giving it more thought right now.
Pretty much this. Underdog makes every stomp match a win-win situation for worse players, which means they will have better MMR, which will make them face the vets more often, which... you get the idea. Also this is even more stupid since vets often take more casual/fun builds on order to make those matches at least 'a bit' fun, so those players will get MMR bonus for winning with our glorious flak spire, banshee mobula or munker. Brilliant.
Also, should we expect mobula buff since it's least used ship? :D
HUH, OKAY, LET'S THROW ALL THE BALANCE OUT OF THE WINDOW AND HIT IT WITH A FRIDGE IF WE'RE AT IT.
But to be honest I think they looked over the usage only in pyramidion case (we don't know the stats but I think before the nerf it could easily have got over 30% of usage) because it was so dominant. Notice that three most used ships are the easiest to fly/crew 'and' are the ones that are locked in the novice lobbies (so novice captains pilot them right after they leave novice to keep themselves competetive). Moreover, probably all of us take easier ships (such as those three) when flying with randoms on pubs and fly those harder ships mostly knowing they have trustworthy crew - friends/vets.
I like the voting thing very much. Finally Devs perhaps admitted that they actually can give people *gasp* what they really want? *thunder in the distance*
Voting mechanics is GOOD. Like, in 90% of the cases it makes the game better. That's why we've been asking for votekick for soooooo long. But okay, assume no votekick, still make scramble/non-scramble vote and perhaps change map voting availible (both are pretty common in CS:GO #csgomastergame).
"We're working on Co-op." --> "No."
You know what? I call bullshit. Devs use it as an excuse not to work on some things. If this is true - why are they working on Stamina for skirmnish? Why changing flamer mechanics? Why HUD changes? Why they gave us new gun instead of bugfixes or (I think) much more desired maps? They could've focused on minor stuff (like indicator of party/team/lobby voicechat) and just wait for the Co-Op. But they clearly aren't.
Oh, and speaking of statistics a guy called Thorin, CS:GO, LoL and Hearthstone commenter said this:
"Making games is not a science. Making games is also not an art. Making games is a combination of those two - a craft"
-
They are working on Stamina because it is going to be huge to COOP. Trust me I flew COOP mode and Stamina is going to be a necessity if they don't buff a lot of ships. You can't keep up with the ships in it without Stamina.
-
So just add it to COOP. I know difference of opinion on that, but still, the devs do have a real attention deficit disorder when it comes to projects.
-
I really don't get the hate for underdog mechanic. It only rewards the noobs if they win a stacked game, remember? And they don't usually win stacked games, that's the whole point of stacking. I agree though that muse don't need to provide another mmr safety net, the safety net is built into the glicko2 system already. Even so, the rewards mentioned in blog post are cosmetic! It explicitly says achievements and badges for winning as underdog.
Re: using match data to check balance of ships. Howard said they will now start correlating ship usage and mmr to get a more detailed picture of what vets vs noobs use.
Other amazing future improvements-
No more novice pilots in regular lobbies! No more second pilots! WOOT
Unimited rematches! No more being stuck with useless ally in crew form after a match! FINALLY WOOT
Custom game now has all matches! RETURN OF MATCHLIST WOOT
Now the only thing missing is named custom lobbies (duck pond) :D
-
Goldfish: 12322 matches (22.5%), 5975 wins (48.5% win rate)
Junker: 7969 matches (14.5%), 3819 wins (47.9%)
Squid: 4892 matches (8.9%), 2496 wins (51.0%)
Galleon: 5135 matches (9.4%), 2412 wins (47.0%)
Spire: 6216 matches (11.3%), 3634 wins (58.5%)
Pyramidion: 14510 matches (26.5%), 7057 wins (48.6%)
Mobula: 3765 matches (6.9%), 2014 wins (53.5%)
Taking a look at this part, since I'm a big numbers nerd.
I would argue that usage rate is fine. Muse tells us that Goldfish and Pyramidion are "overused"; Squid, Spire, Galleon, and Mobula are "underused". But if we think about it, aren't Goldfish and Pyramidion supposed to be well-rounded, relatively easy, and commonly used ships? Aren't Squid, Spire, Galleon, and Mobula supposed to be specific-talent, harder, and less common ships? We really shouldn't balance ships based on usage. That would be like buffing Medics in TF2 because you almost never see more than 1 per team.
Looking at Win Rates. Squid and Mobula's win rates looks about right because it is almost always veteran, competent players that take those ships. Pyra win rate is where it should be, since inexperienced players usually take this ship. Same goes for the Junker and the Goldfish.
However, Muse should take a look at the Spire. That win-rate is statistically significant. I ran the stats with the null hypothesis that the real proportion of Spire win rate is .5. alternative hypothesis that the real proportion is greater than .5, used the proportion p1 = 3634/6216 = .585, and p2 = 3108/6216 = .5. The resulting pvalue was 1.436*10^-21. Since pvalue < alpha, I reject the null hypothesis. There is sufficient evidence that the real proportion of Spire win rate is greater than .5.
For non-statisticians (and great-statisticians that got confused by my novice write up), this means that Spire is almost definitely having more wins than it should. I think the hull health buff it received last patch needs to be toned down a bit.
So, these are the things I can see from the numbers. Spire needs a fix. Other ships are fine where they are (for the moment). Muse should not base balance on ship usage. Balance and ship usage are unrelated things. Squid isn't being flown frequently and that's how it should be.
Muse, try to base your balance on player feedback, controlled tests, and mathematics (which the forum does for you once in a while).
And Muse, although you guys sometimes do things I don't like, I still appreciate the hard work, and I love Guns of Icarus Online.
-
We’ll always be diligent in listening to everyone’s feedback as well as in finding ways to use data to improve the game. While we do, we also have a mission to continue to create and to innovate on the game. This could at times affect a person’s use case, and as a result, we’ll need to do our best to understand it and account for it. Over the past 2.5 years, we’ve been constantly innovating on the game. We’ve added in numerous features and systems, introduced new weapons and skills, and carved out better specialization for each ship, weapon, and skill, all to make the game a better, more diverse, and more interesting experience than it was when we first launched. Innovation, however, is a 2-way street. While we never want to be complacent, we also ask our great players to not be complacent either, and to continue to partner with us on this wild and incredible ride. We could not have gotten this far without your amazing support and feedback.
And then
Sigh...the fail is strong with this...
I have a better idea: instead of nerfing the strong ships, buff the weak ones?
I like the voting thing very much. Finally Devs perhaps admitted that they actually can give people *gasp* what they really want? *thunder in the distance*
You know what? I call bullshit. Devs use it as an excuse not to work on some things. If this is true - why are they working on Stamina for skirmnish? Why changing flamer mechanics? Why HUD changes? Why they gave us new gun instead of bugfixes or (I think) much more desired maps? They could've focused on minor stuff (like indicator of party/team/lobby voicechat) and just wait for the Co-Op. But they clearly aren't.
Also, should we expect mobula buff since it's least used ship? :D
HUH, OKAY, LET'S THROW ALL THE BALANCE OUT OF THE WINDOW AND HIT IT WITH A FRIDGE IF WE'RE AT IT.
(When they clearly said that the next ship they will be looking at is the GALLEON)
Im quite sick of seeing really untoughtfull, asshole posts come up so quickly after minutes or a day of news coming in. Understandable to a degree because we are players whos dealt with many changes, but when they give us what we want (Which in this case the insight of how they can look at the game). Even worse comments arrive.
I feel like it would be healthier for them to not give us any good info just for their own mentality, lol.
I mean the underdog system sounds alot like the "wanted" system. Alot of us commented on the fact that we would be harrased if we were wanted for the whole week. Some people even complained, but similarly i can see it being the same case for the underdog where if noobs are going to fight the big guys, theyl sky rocket down to the ground. But is it going to last a week? In a way it is suppoused to encourage newbies in taking it serious. What we dont know is exactly how much will this new system read our stats of ranking correctly. Is someone with a lvl 45 pilot and lvl 10s crew worth being targeted by an underdog team? I sure dont think so but you get the point.
"The system will be abused" let them abuse it, and youl get quicker games. We dont know how it will really work or if it will be considered in the very end, and no statements can confirm how it is going to play out.
Other amazing future improvements-
No more novice pilots in regular lobbies! No more second pilots! WOOT
Unimited rematches! No more being stuck with useless ally in crew form after a match! FINALLY WOOT
Custom game now has all matches! RETURN OF MATCHLIST WOOT
And weve all complained of these things, and now they are fixing it! YAY! Lets celebrate god damnit!
Okay you know what that means everyone, this forum and every second spent on discussing the horrifically broken ships have been time wasted.
Ive complained how the spire was big and easy to hit, yett more fragile than a junker whos tanky and hard to hit. Made even easier where the spire was slow.
I argued that the spire could get a health buff and or a speed buff in terms of acceleration. Muse brought forth an odd skew of what i wanted where both health and speed were the case but not as much as how i wanted. And it prooved to be a more balanced balance than what i had. The spire buff essentially was what I wanted for quite some time.
Spire needs a fix
I agree, but where. Maybe health down to 850 (from current 950)? Currently the spire is at a good spot but i cant help thinking that the majority of matches versus the spire dont know how easy it really is to kill a spire. Ive seen too many Gat, hades... Gat, flak... Mercury lightflak... Goldfishes despite being firepowerless, are tanky as hell in pub matches. Not many (at all) pub-players know proper explosive shot timing.
-
Hi Indreams, fellow stats nerd here. The underlying win fraction for all of these ships except the squid is different from 0.5 (or rather to be more insufferably technical, with an alpha of 0.05 I reject the null in all of these cases using a normal approximation to the binomial test, with bonferroni post hoc corrections). What matters here isn't statistical significance (we were assured we would have it for all interesting effects just from the sample size), but the effect size.
However, even looking at effect size we have a problem, lack of control. The spire has a much higher win fraction than the other ships, 0.17 difference. That is pretty large. The stats don't tell us why though. If Muse are balancing the game based on just these aggregate numbers when they have the MMR then this is my disappointed face. I strongly suspect that the best statistical approach here would be not the straight foward binomial analysis myself and Indreams are looking at, but a generalized linear model which included ship and MMR as predictor variables, perhaps even with some other factors thrown in (were the participants in a novice match? Did one of the ships fit the bill as a nerf build? Other factors I haven't thought of?). Even then I think the model needs to be non-linear, and it is basically doing what the matchmaker does, predicting matches.
Without controls for things like player skill these aggregated numbers don't mean much. If they haven't already, Muse should probably hire a professional statistician if they are serious about balance, because the numbers the present here are largely useless if the are looking to justify recent balance changes. There is a big human element here so such an effort combines expertise in both the humanities and technical disciplines. To do this properly and completely you have to interview the target population to inform and enable the construction of a suitable operationalisation of the potential contributory factors, then combine that domain specific knowledge with a suitable statistical model including those factors, then validate the model with some test data. Only then are you really in a position to start meaningfully interpreting the statistics.
-
@Nanoduckling
You are a better statistician than I. Salute!
I agree that these numbers don't really mean much. (that's always a problem with statistics, numbers almost never mean anything)
But its always fun to look at them (hence number nerd).
If they haven't already, Muse should probably hire a professional statistician if they are serious about balance,
That'd be super cool! Imagine having a fireside with a GOIO Statistician!
I get to apply boring school learning to exciting SteamPunk Airship Combat! I love it every time we have this kind of discussion on the forums! :D
-
Guys, whoa, slow it down.
Balance is not decided by ship usage numbers, that would be horrible.
The numbers serve a couple of factors:
First and foremost, they're pretty cool to look at. As some of you guys are numbers nerds, you know, raw stats are interesting.
Second, noticing trends is a good way to find new ways to look at problems. Our experiences, the number crunching we do, and feedback from awesome vets, does not define every experience in the game. It is important that we look at the game from every angle.
Third, you never know where the next source of inspiration comes from. Does it come from looking at the numbers, trying an all harpoon ship, or playing some random $2 steam game. Information and experiences can all be great starting points.
We like to share this behind the scenes stuff because we find it interesting. We knew there would be backlash, but I think getting to show some cool numbers is a lot of fun.
-
Speaking of cool looking stats, do you guys keep a heat map of kills and deaths on all the levels?
(http://www.rockpapershotgun.com/images/dec07/2forts.jpg)
-
We really should, I'll bring it up with the team. Thanks for the idea Hamster!
-
no more multiple pilot ships?
=(((
-
Just wanted to say one thing about matchmaking.
I started to look for a match.
I wore my leather jacket.
Locked the door to my place.
Went to the shop for some sweets.
Bought them. Paid for them with credit card, cause I'm broke with ready cash.
Came back.
The matchmaker had not found a match yet.
#tru-story
-
Really i never wait longer than 2 minutes legit, am not sure if this is just a thing now, or maybe an issue. because i never get long wait times like people are saying,
Are all the othions open and ticked for 2vs2 3vs3 4vs4 and servers ticked, or have you restricted them?
-
Come queue with my TB team next Saturday, you will see, oh you'll see...
-
Hmm so all the match options are ticked? along with server choices? and people are waiting excess 15 minutes?
-
On average we wait 25 minutes for every 5 minute game, some waits have been in excess of 45 minutes to an hour. We do sometimes choose 2v2 but rarely nowadays as it makes it even harder, and it'll eventually just loosen the preferences anyway and start a dam 4v4. That was when we queued, since the long waits we have been making lobbies.
-
Ok il relay that and see what can be tweaked, because we knew people were complaining about matchmaking, but 45 minutes might be a problem.
-
Really i never wait longer than 2 minutes legit, am not sure if this is just a thing now, or maybe an issue. because i never get long wait times like people are saying,
Are all the othions open and ticked for 2vs2 3vs3 4vs4 and servers ticked, or have you restricted them?
I restrict always only the region - European servers only. America servers tend to serve rather high latency (150ms+ if I'm lucky), for unfortunately I use very unstable and poor remote connection (not sure how this is called in english - the modem's work is based on a SIM card given by operator, and very dependant on the mobile phone network) - I can't afford anything else.
Well, for me Matchmaker serves 2 flavors, while searching solo.
1) Find a match in seconds, only to find it's a lobby you left 5 times, because you have encountered assholes there,
2) Wait up to 10 minutes:
a) It's a different lobby; you are so tired of waiting you are staying here
b) ...it's the same damn lobby.
-
Ok il relay that and see what can be tweaked, because we knew people were complaining about matchmaking, but 45 minutes might be a problem.
Ok, I can't quote you in edit, screw it. i'm too tired for that.
That's quite common when number of people drops below damned 200 people to wait for such time, it's been mentioned multiple times from the times of after-sale-downfall, and no, another sale is not the solution, as you will only lose more people.
-
Yeah if you have high MMR I suspect you get shafted by the system. Friend and I have been playing again recently. We always make a custom lobby and fill from MM. It is faster and even then sometimes we wait 15 mins. On slow nights it can be 30+ mins.
I have noticed one thing tho...when I crew for him, it fills quicker vs me piloting.
Either way matches are always 100% one sided. 5-0 over and over. Only had fun when we suddenly found a 3v3 last night. Just...can't fill custom 3v3s without 8 people so till then, infinite crappy 2v2s.
-
Yeah so i have been talking to bubbles, and their is things in the works to fix the long waiting times with prompts to change things up, if their is no match for your mmr to make it balanced, also until somthing is solid in place, it might be easier making a custom game then open up to queue that way it is a little more social and you can recruit from global, just like old school.
I know alot of people don't agree with matchmaking, but custom game is essentially the same before it was implemented, obviously the system is in place for good reasons, but has its flaws, when it comes to certain situations,
we are also monitoring long wait times to see the circumstances, so if you are in queue for over i dunno 20 mins, send us an email with time and date, and jerry will get on finding a reason why so we can continue to improve,
thank you for the patience and continued support, i understand it being frustrating but together am sure the problems will be worked out,
Thanks Grey
-
Upon further investigation of quite a few specific cases of long queue times, a pattern emerged. In the majority of the long queue wait time cases we investigated, they invariably had to do with crew form, and more specially crew form of multiple ships and crews. In these cases, especially for higher skilled players, the match maker would have a harder time finding balanced opponents because there weren’t enough players of similar skill levels available in queue at the time.
it might be easier making a custom game then open up to queue that way it is a little more social and you can recruit from global, just like old school.
This. Sitting in stacked crew forms means you'll wait... what do you expect?
When playing with pubs there is no wait problem. I've played thousands of matches in MM and the system works until there's very low pop. I can only attest for playing on US servers.
I've played more MM than anyone and I don't have problems. I spent the bulk of my time pre MM and that was much slower. Keep up the improvements!
-
The problem with that mentality is that it implies that it is okay to punish 'stacked crews'. Call em stacked crews, I call em playing with my friends. Were not stacking to win, were stacking to play together, because were friends. I want to play with my friends, not pugs. I don't think it's fair to believe punishing a certain playstyle is okay.
-
Also, should we expect mobula buff since it's least used ship? :D
HUH, OKAY, LET'S THROW ALL THE BALANCE OUT OF THE WINDOW AND HIT IT WITH A FRIDGE IF WE'RE AT IT.
(When they clearly said that the next ship they will be looking at is the GALLEON)
Im quite sick of seeing really untoughtfull, asshole posts come up so quickly after minutes or a day of news coming in. Understandable to a degree because we are players whos dealt with many changes, but when they give us what we want (Which in this case the insight of how they can look at the game). Even worse comments arrive.
I may have been overdramatic here, but I was making a point that low usage =/= need of buff, and that pyramidion was a special case because it had had such a huge majority of usage.
Other amazing future improvements-
No more novice pilots in regular lobbies! No more second pilots! WOOT
Unimited rematches! No more being stuck with useless ally in crew form after a match! FINALLY WOOT
Custom game now has all matches! RETURN OF MATCHLIST WOOT
And weve all complained of these things, and now they are fixing it! YAY! Lets celebrate god damnit!
I agree, although I have my complaints surely (we always do) this is 'the' patch I am looking forward to see.
@Keyvias
Certainly, although we kind of overlooked it, it's really nice to finally see some stats, especially remembering that on the last Fireside you (devs) said that you won't post it. We're all of course really happy to see them.
-----------------------------
I've written to Howard to tell my fears about the underdog system, and he wrote:
"The not counting losses is only in player stats and progression, so it's really cosmetics. Players' MMRs are would be reflected and adjusted as normal. So people's MMR will change. In other words, we're not touching MMRs with this construct." - underlining and bolding mine
The problem with that mentality is that it implies that it is okay to punish 'stacked crews'. Call em stacked crews, I call em playing with my friends. Were not stacking to win, were stacking to play together, because were friends. I want to play with my friends, not pugs. I don't think it's fair to believe punishing a certain playstyle is okay.
I think BlackenedPies didn't mean it that way, I believed the word "stacked" is simply technical term. In the game where cooperation is OP the ship of four vet friends can be really powerful in pubs filled with randoms. I believe it's "no one's fault" in a way that you must wait long time, it's just that with low population it's hard for system to find someone of your level, especially if you are in a crew form of several ships.
On the other hand I fully agree, in this 'highly cooperational game' people naturally want to play together, especially when even one stupid person on the ship can ruin your whole game (yes, it can). You say "well you play together so you have a problem", but people are going to play together (at least crewing full ship), so some may say "well, the game is meant to be played together, so it's the problem of the system".
-
(https://i.imgflip.com/jz7b9.jpg)
-
The idea of using stats for balance purposes is completely legitimate. I'd rather that then having Muse listen to every idea on the forums. The stats they do show aren't the most useful, but they do give some interesting information. This mostly has to do with ship picks. More informative stats would show pick % on different maps, or even pick and win rate on certain maps with certain paired ships. You could even split it up by MMR to look at different skill level preferences and success rates.
So realistically, this is some very simple information that shows what ships in general players like to pick in all types of matches. As we've all probably noticed, the easiest ships to fly are picked most often, and the tougher ships to fly are picked less often. Sounds about right. The top 3 picked ships also happen to be the novice ships, and although they're not looking at novice matches, are they excluding novice players who don't know how to unlock their ships? You can also consider that players who graduate from novice are most comfortable with these three ships, and will try and stick with them more often for a while.
As for the underdog system, I have no idea why people have their panties in a twist. It seems like a really nice compromise. Stomps/stacks do in fact happen, and this system just helps the underdog in that situation without punishing the stack in any way. I'd recommend actually reading the blog post to better understand the idea that's still being tossed around. Currently it sounds like the system will look at a match as it starts and evaluate both teams. If there's a significant difference in skill level based upon their stats, one team will be considered the underdog. Win or lose, the underdog's stats aren't affected and they get a badge for playing to the end (ie: not quitting midmatch). It seems more a solution to the leavers problem than anything else.
Then finally the match making system has always been interesting, and I still find it weird when people complain about the time it takes for their 8+ person team to find a match. If you've ever played in the old lobby system, finding room for a whole crew was always a lot of effort; it's very rare for a single side in a lobby to be completely empty. You and your team would still jump from lobby to lobby trying to find enough open spots and pull everyone in. Or find one that's mostly empty, and beg players to swap with you so you could all play together. The match making system doesn't shift players around who are already in the lobby, so that's the only real difference between the new and old system. The best and fastest solution is to start your own damn lobby and open it up to queue. It's like you only bother trying to queue up so you can complain about the system. If you really want them to improve the queue times, their best solution would be to drastically limit the number of people you can queue up with; which is probably something you don't want.