Author Topic: Goals and Ramifications of Matchmaking  (Read 80202 times)

Offline Imagine

  • Member
  • Salutes: 59
    • [MM]
    • 19 
    • 33
    • 22 
    • View Profile
    • Twitch Stream
Re: Goals and Ramifications of Matchmaking
« Reply #15 on: February 21, 2014, 09:48:40 pm »
So, if I'm reading this correctly, even with Matchmaking you'll be put into a lobby first so that you can adjust player loadouts to whatever the captain wants?

Other than that I do think that it'd be nice if the captain/pilot can determine what they want their crew makeup to be when they join through matchmaking, whether that's 2engie or 3engie setup.

Offline Squidslinger Gilder

  • Member
  • Salutes: 287
    • [TBB]
    • 31 
    • 34
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #16 on: February 21, 2014, 09:54:08 pm »
Designated slots by the captain along with matchmaking would work. However we'd likely see lots of high level engineers and very few gunners as people will realize they cannot get into matches as gunners. This is a common issue in mmos where dps get stuck waiting all day for groups as tanks and healers are more in demand.

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #17 on: February 21, 2014, 09:54:33 pm »
@velvet, firstly, it is a fair point about having statistically significant numbers in testing, if that is what you're referring to.  And with the changes that we've implemented in the past, it is reasonable to say that we at time don't see the full ramifications until things are live.  I am well aware of what is adequate and statistically significant testing.  To get to there is a challenge, but the goal is of course to get there. 

I think there is a bit of confusion to the point I am making.  By saying that people wait excessively long before match start is not saying that we are removing strategy or giving people time to strategize.  The issue here is that match lobby wait extends beyond strategy and tactic.  The system nearly completely depends on the player to ready for example.  It is thereby a manual system that really doesn't actively facilitate.  It is dependent upon player mediation, and therefore isn't designed to optimize the efficiency of match readiness.  We can debate this more in depth, but I do think that the limitation of the current system is apparent.  The aim of the new system is in part to try to allow people to make strategic and tactical decisions and choices, but improve efficiency of match start.  If through testing, we find that the system is not optimal, we will then need to improve upon it. 

With the point about offering people no choice to form crew, this is actually incorrect.  People will explicitly have the option to form crew.  And the aim is to make that effective.  If it isn't in testing, then we'll again need to find ways to improve it. 

On teaching.  You are correct.  When veteran players teach new players, it is beautiful and it is something that we want to foster.  The current Novice system I agree has flaws, but when people whose skill levels are disparate are paired together, the experience can be actually excruciating for everyone.  This I am not conjecturing.  This is based on what we have seen before Novice was even implemented.  And this is based on studies in other games as well.  Teaching does happen a lot, and we have a great community, but sadly it doesn't happen often enough, because it is often not aligned with the interests of the players involved who are at very different stages of the learning curve. 

The point about choice is valid, but herein lies the paradox in this point.  You are correct that often people do not want to teach, and it is more often than not that new players find themselves not on the ship of someone willing to teach, but on a side that a well organized team destroys repeatedly.  This is of course anecdotal.  But really what it comes down to is that, can we facilitate learning by experience sharing between players of different levels.  This is your main concern here correct?  I think we are conscious of this, and it is something we want, but at the same time do a better job of ensuring balance.  Are we there yet?  We shall test and find out. 

Ok, with "team stacking," let me just make this point as clear as I can.  We DO NOT have issues with team stacking, if that term means people who know each other want to form up and play together.  It is not a derogatory term to me.  The issue here is that, can we find reasonable opponents for this team to play with.  In the current system, the answer is systematically no.  The evidence is pretty clear here.  And you are likely correct in assuming that a crew formed clan team can win against a match made experienced team, and I think that fair.  I don't think that proves the merit of either system, as at worst, we can do as well as what we do now. 

I think there is also a conception here that Match making is somehow anti clan, or that we are somehow anti clan.  If somehow we gave off that impression, I definitely apologize.  We are not anti clan.  In fact, I would say that based on track record, we have been pretty far removed from that.  We are not trying to do things to diminish clans.  I would say that we have been really active in facilitating clans and the competitive scene.  In fact, the community system that I have been promising about for the last 2-3 months is being worked on right now as we speak.  As we have finally gone from concept and specifications to implementation.  And we took more time with that to make sure that we took in feedback from you guys.  With the new system in place, the aim is to actually have a place in game for clans to form and recruit, and for new players to sign up. 

I think the point about featured match flexibility and accessibility is a valid concern, and it can potentially take away flexibility.  And this is the exact kind of feedback we want.  If in testing, you feel that this takes away the benefit of the current system that is vital.  Then we should think of ways to remedy this.  And things like this really helps, and this is the point I wanted to make. 

Actually, with scramble, what I am telling you is that, statistically speaking, it definitely helped retention.  This was not a matter of my opinion.  It really did happen.  I'm not making this up hahaha. 

I think if I am correct, I don't see a lot of cases of games with match making systems also offering match list, and have them be in juxtaposition.  Eric did more of the benchmark research, so he can speak better.  But I definitely understand your point and concern here. 

I think the more productive thought process is, if you like, you can list out briefly what you think are both the pros and cons of the current system.  By doing this, the issues that we are tackling should be clearer, and we can then test systematically against the concerns of the new system to make sure we retain the pros of the current system. 

For now, I would once again still encourage people to participate and try and test.  Otherwise, whatever data and feedback we collect here would be biased (statistically speaking). 

Lastly, I think we have proven that we are not without reason, and that a lot of the decisions we made turned out to be right.  And I think the way we made good decisions was to listen to you guys and look at whatever data we can collect.  We will continue to do the same. 

Thanks!  Howard
« Last Edit: February 21, 2014, 10:07:09 pm by Dev Bubbles »

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #18 on: February 21, 2014, 10:01:39 pm »
@gilder thank you!  I can't tell you how much what you said means to me.  I really appreciate it.  And I just want to say again that we really really respect you guys' opinions and feedback.  You guys make really astute and well thought out arguments, and are the driving force for us to continue to improve.  We are absolutely not saying that what we have in dev app is perfect.  We are actually anticipating to make adjustments and redesigns etc.  And that is why we are trying our hardest to test this out.  If we didn't value your feedback, we would have just gone ahead and implemented.  But we are not.  We know how much testing would be a challenge, but we gotta try.  Hope I'm at least sounding reasonable and rational on a Friday night :D 

@tropo, please don't take what I said the wrong way.  I absolutely value your opinion, but I am just making a point that it is your free will and choice, and I can't force you to do anything or coerce you into thinking something that you don't agree with.  I would love for you to test, but you really don't have to.   And if you think we F up, you deserve to and have the right to not support us.  That's the point I was trying to make.   Hope I didn't upset or offend you.  Thanks a lot. 

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #19 on: February 21, 2014, 10:04:13 pm »
@Imagine, unless I'm being a total idiot, which I can be at times, you are correct. 

You and Gilder's points are spot on.  If the new system doesn't do that effectively, we'll need to find a way to improve. 

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #20 on: February 21, 2014, 10:22:01 pm »
@everyone, I know that there is angst, and that you guys are only speaking up because you care and love the game.  I cannot tell you how much it means to us.  With every patch, we are taking a risk, because change is never easy.  But I think we as devs need to have the courage to want to change and strive to do better, and ultimately, you should demand us to change and improve.  We took all the risks and devoted close to 4 years of our lives to make this game, and you guys took a risk to invest time in this game that is not like other games.  The last thing we want is to disrespect anyone and screw things up.  We made plenty of mistakes, but we could never let that stop us from keep trying.  We built this community together with you guys through hurricane, exhaustion, and countless hours of communication, (and for me personally a lot of weird diseases and a concussion).  I think you should be safe to assume that we have the best interest of you guys at heart.  Thanks!

Offline Imagine

  • Member
  • Salutes: 59
    • [MM]
    • 19 
    • 33
    • 22 
    • View Profile
    • Twitch Stream
Re: Goals and Ramifications of Matchmaking
« Reply #21 on: February 21, 2014, 10:27:19 pm »
Designated slots by the captain along with matchmaking would work. However we'd likely see lots of high level engineers and very few gunners as people will realize they cannot get into matches as gunners. This is a common issue in mmos where dps get stuck waiting all day for groups as tanks and healers are more in demand.
I don't think that'd be the case honestly. I'd still say more captains take 2/1 over 3/0, but if we're locking spots into 2/1 then it takes away a lot of the customization that captains like on their ship. Gunners wouldn't have it harder than they do now, I feel.

Anyways, as for the our angst, just know that we rage and complain and brood and celebrate because we care.
 
Like, a lot. Sometimes, perhaps, too much ;)

Offline Tropo

  • Member
  • Salutes: 31
    • [SAC]
    • 45 
    • 45
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #22 on: February 21, 2014, 10:32:02 pm »
@bubbles all good
keep the match list and look at community and social there important parts of online gaming
also showing new player what to do or how to do it

i feel a lot of players are leaving this game because high level people shout at them and complain that there doing this or that with out any reason or explanation as to how to do it properly

most classic one is yelling at some one to take some load out and never explaining why they should use x rounds in x gun

i could see match making make this worst

and i feel you could improve the details around the ammo guns and types ways to kill enemy ships before letting all this lose

have you eva seen the movie Idiocracy this is whats going thought my head when i think of match making

Offline Byron Cavendish

  • Member
  • Salutes: 89
    • [TB]
    • 21 
    • 31
    • 45 
    • View Profile
    • The Brotherhood
Re: Goals and Ramifications of Matchmaking
« Reply #23 on: February 21, 2014, 10:45:52 pm »
Ya I'm with ya on that Gilder. I voiced my concerns and I guess I'll let this play out, and see what happens. But this may not end well.

Offline Byron Cavendish

  • Member
  • Salutes: 89
    • [TB]
    • 21 
    • 31
    • 45 
    • View Profile
    • The Brotherhood
Re: Goals and Ramifications of Matchmaking
« Reply #24 on: February 21, 2014, 10:52:40 pm »
Quote
The issue here is that, can we find reasonable opponents for this team to play with.  In the current system, the answer is systematically no.  The evidence is pretty clear here.  And you are likely correct in assuming that a crew formed clan team can win against a match made experienced team, and I think that fair.  I don't think that proves the merit of either system, as at worst, we can do as well as what we do now. 

I have to firmly disagree with you there Howard. I am fairly confident saying that most of the high level, top tier players and clans know each other. We are a small enough community that this is easily possible. If we are on at the same time, we find each other. If I make a "Tea Time", it will be filled with Cakes, SIR's, and Sacs in no time. In many ways we have created our own matchmaking system. I don't ever see you on when this happens (which is perfectly okay), but I find it funny that you are so sure we need this, when we really don't. I can understand the new players might need this, but the really the only threat to them is the mid tier players; the level 5-10's, really first breaking out of the newbie phase, trying to prove themselves to the big fish.

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #25 on: February 21, 2014, 11:02:25 pm »
@tropo, thanks a lot. 
About the issue of higher level and lower level player interaction.  I look at this as at leafs in part a system issue.  In a intense and highly team oriented environment, conflicts between people can and do arise, and because this game mirrors how teams actually interact pretty well, people can play and end in conflict.  But I think that the system in place has a lot to do with this as well.  Let me digress and look at NYC subway for example.  People at rush hour in the subway are generally really curt and rude and mean.  You can argue that New Yorkers are just like that.  But if you analyze this enough, you'll realize that most of this is system driven.  In the absence of a train schedule, when trains arrive and leave stations almost on whims, and when you don't know when the next train is coming, and you don't have any markings as to where to line up, where the car door is doing to open, what you get is a free for all.  You have no choice but to push your way into the train, otherwise, you miss work, and you get fired (ok, maybe not that extreme, but you get the point). 

What does that have to do with us.  Well, as much as it pains me to say, the current match list system not only doesn't do anything to facilitate better player interaction, it is in a number of ways conducive to less than ideal player interaction.  Don't get me wrong, players a lot of times do have great interactions in the match lobby for example, but I would almost argue that they achieve this in spite of the system, and I don't think that's optimal. 

On better gun to component indication, working on it.  This is something Nathan is working on this week. 

I haven't see the movie, but I just looked up what it's about.  LOL.  I really don't know.  I actually think that a match making system would allow players at similar level to make and execute better tactical decisions and to be more challenged and put to the test. 

I think one of the concerns for you guys would be how easy and expedient it is to form crew, because in reality, that is what you do when in a match in match list in current system, but just through coordination on your part.  And I think that is something we should test and take feedback on as well. 

Thanks! 

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #26 on: February 21, 2014, 11:10:06 pm »
@Byron, actually I think you missed my point, and actually agreed with my point :D  My point is precisely that it is not system facilitated or driven.  The mere fact that you have to coordinate amongst yourself and the system doesn't do anything to facilitate finding of people with similar experience to play with means that the system is not optimal.  When you say "we," you mean clans that are familiar with one another, but we (devs) have to design a system for all players.  On another note, if currently you can find each other and want to play with each other, you can do that in the new system as well.  If after testing, you feel like we can better facilitate your use case, we can definitely work on it to improve it. 

Offline Urz

  • Member
  • Salutes: 75
    • [MM]
    • 45 
    • 45
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #27 on: February 21, 2014, 11:15:24 pm »
I'm on the same side as many who have posted already; I have no interest or desire to ever use matchmaking. My recommendation to those with concerns is to friend everyone you enjoy playing with, and bypass matchmaking completely via the friends list. That is my plan, and I worry for those who haven't been playing long enough to build up a list.

When I'm looking for a lobby, I am evaluating them. Here is a list of things that I take into consideration:

Loadouts of the players
Ship builds
Levels/matches played of players
How many friends are in the lobby
If there is anyone blocked in the lobby
If I want to play on this map
Whether players are communicating in the lobby or not
Having a particular achievement I want to work on
How long I'm willing to wait

Each of these factors contributes to whether or not I determine the lobby I want to play in. On some days, certain factors will be more important than others. Sometimes I'm fine taking captain spot on an inexperienced crew and playing professor. Sometimes I just want to hop on a strong ship and be a good engineer. These are not determinations your algorithm will be able to account for.

Forcing matchmaking onto us instead of presenting it as the default option takes away this agency from the players. Removing the ability for me as a player to evaluate which lobby I will enjoy the most will create a less enjoyable gameplay experience. If your sole motivation is to increase your "retention" and in that endeavor you're willing to sacrifice the ability of the players to make their own decisions, well that is pretty disappointing. That you bring up scrambled lobbies as a "success" only makes me care less about your analytics.

Ultimately I can only voice my opinion, and as I have a fairly large stake in the Guns of Icarus community, I'm forced to wait and see. I really do hope you make the correct decision, whatever it may be.
« Last Edit: February 21, 2014, 11:17:00 pm by Urz »

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #28 on: February 21, 2014, 11:15:52 pm »
@Byron, let me add something real quick.  If we are only offering match making, I would say you are correct.  But this is not the case.  When you create a match now, you are in a sense creating a custom match, and in your case, you actually want this match to be filled with people who are competitive enough to play with you.  This case we support in the new system.  Granted, you won't be able to create a whackier match names for all to see, but you can still let people you know to join you.  That medium of communication is not severed.  If you find the new system to not work as well as you like in this regard, we'll try to figure out the pain points and improve.
« Last Edit: February 21, 2014, 11:33:35 pm by Dev Bubbles »

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #29 on: February 21, 2014, 11:31:27 pm »
@Urz, first of all, with scramble.   You can not care about statistics in general or my ability to analyze, and you are entitled to your opinion.  But unfortunately, I do have to rely on statistics and scientific method to make judgments.  If somehow that disappoints you, I'm not sure what to make of it. 

Now, about potentially evaluating things before you join a lobby.  You are assuming that this evaluation is done statically, and that the lobby dynamic doesn't change.  Well, if that works for you, sure.  I'm not sure where I said that I'm sacrificing the ability for players to make their own decisions, but you would like to make unsubstantiated claims without bothering to try the new system, you are welcome to.  Like you said, it is your opinion.  I do find the use case you raise interesting, and I'll consider that. 

If there is a method in the new system that works for you, sure, by all means do so. 
I am also taking risk backing you in the competitive structure, and I hope that I am making the correct decision as well. 
Have a good night.