Author Topic: Goals and Ramifications of Matchmaking  (Read 81076 times)

Offline Urz

  • Member
  • Salutes: 75
    • [MM]
    • 45 
    • 45
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #30 on: February 21, 2014, 11:45:45 pm »
@Urz, first of all, with scramble.   You can not care about statistics in general or my ability to analyze, and you are entitled to your opinion.  But unfortunately, I do have to rely on statistics and scientific method to make judgments.  If somehow that disappoints you, I'm not sure what to make of it.   

I'm not doubting your ability to analyze statistics. The addition of scramble lobbies has affected my own gameplay experience negatively, and you just touted the feature as an example of "I was right you were wrong because of numbers". I'm sure there are a lot of things you could do to increase new player retention that would negatively affect your regular players (such as removing the match list).

Quote
I'm not sure where I said that I'm sacrificing the ability for players to make their own decisions, but you would like to make unsubstantiated claims without bothering to try the new system, you are welcome to.

Removing the match list is limiting the player's choice in which lobby they wish to play in.
« Last Edit: February 21, 2014, 11:48:07 pm by Urz »

Offline Keyvias

  • Member
  • Salutes: 83
    • [Muse]
    • 12 
    • 27
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #31 on: February 22, 2014, 12:01:12 am »
Hey Urz,

You're right that removal of the match list is in fact a removal, but our goal is to build and test a system that puts players into the best lobbies as quickly as possible.
That's why we're so gun-ho about testing this.
This isn't to say players like you don't have an exact formula that will bring you into the match you want, but most of the time this is not the case.  Most of the time players are not finding good matches and they're taking a lot of time to find these matches.

This will be a different style of match creation and yes it will be different.  Many elements we've included since inception have been different though and we have lost players for our choices from Ui to balance there will always be people who say the old is better than the new. Overall though I like to believe that the game experience we've created has improved with the time we've put in.
It comes down to trying to do the best thing for all of our players, old and new.  The current lobby system has many MANY flaws.  We've been asked by multiple people on multiple occasions to make a second game inside the lobby because the wait times are so bad.  We've had requests for liar's dice, checkers, and a couple others.
Gathering 16 players should not have that difficulty and it means we have done something wrong.

As far as scramble goes, it did help our retention and we received no negative emails after its implementation (Other than the fact buried non-scrambled matches, which was our mistake, but we fixed that.) so that's why Howard (and I) consider it a success.

There is no doubt that this is a controversial and literately game changing choice we're looking at, but while you feel your game is at stake. This is our life. This hasn't been a hasty nor light decision.  Let's be honest if we bunk this up you guys lose a game and our team goes to ramen.
We will do everything we can to make the game great and if you have a way that solves the issues that players have experienced or solutions to the problems you see in matchmaking we'd love to hear it.  Making this game better is everyone's goal here.

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #32 on: February 22, 2014, 12:02:11 am »
That is not what I am saying Urz.  If that's how you took it, fine, I apologize.  What I was trying to say was that if we are just looking at numbers overall, we can see an improvement of retention.  And because we implemented the feature specifically to be measured against attrition and player levels, I would consider the feature implementation a success, because it achieved what we set out to do.  It's true that we didn't exclusively release scramble to isolate just the effects of scramble, so I am making a bit of an assumption here still.  But I think that assumption is pretty safe.

Ok, if you mean by removing match list, your ability to see a match list and pick matches to join is limited, that's fair.  And f as a result, you don't use match making, I think that's perfectly ok.  I don't think for a second that match making is perfect.  If you have a valid use case, then the question becomes can we support your use case.  And how easy we accommodate your use case.  I think that we can try and find out.  If ended up we don't support how you play the game at all, then I think that's definitely cause for concern. 

Offline Queso

  • Muse Games
  • Salutes: 126
    • [Muse]
    • 13
    • 25 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #33 on: February 22, 2014, 12:38:20 am »
You guys pretty much covered what I was concerned about, as I figured you would. It's hard to be okay with something till you have specifics. I'm looking forward to it now. It really should up player retention without sacrificing a lot of the excellent social side of Icarus.

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #34 on: February 22, 2014, 01:02:05 am »
It's good to have concerns, and it just means that we need to test, consider, and address them. 
We definitely have in mind the social aspect of the current system as well, and want to make sure we preserve that.  That is one strength in the current system that we recognize for sure. 


Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #35 on: February 22, 2014, 01:11:42 am »
Let me just make another point.  Before scramble was implemented, I had a debate with Qwerty, and he astutely asked me about match list performance.  For everyone who's gone through the TGS event days when we had to manually force start matches know that match list is a performance bottleneck.  We have since then done a ton of work to drastically optimize the match list performance, but as of today, it is the single biggest performance bottleneck, and it is the one system that's least scalable and most ready to fail at higher concurrents.  If we want to grow bigger as a community, the match list is the least sustainable of a system performance wise.  While I was right in that the match list can handle increased concurrents, I am also wrong at the time because it is not future proof performance wise.  And that factored into our decisions as well. 

Offline Thomas

  • Member
  • Salutes: 80
    • [SPQR]
    • 20 
    • 44
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #36 on: February 22, 2014, 01:48:08 am »
Ultimately I think matchmaking is a good idea, for a number of reasons. The big disadvantage is that we do lose some choice in the matter. However, part of the matchmaking goal is to put you into fair matches, something most players try to look for anyways. You might lose the ability to randomly create some 'for fun' matches like all mine battles and flamethrower wars, but there's a custom match for that.

The biggest issues I come across in the match list system are long wait times to start a match, and imbalanced matches. The scramble option helps relieve some of the imbalance, because it keeps shifting around.


Wait times prior to a match is caused by a number of factors. First you need all captains slots full, then you need all captains to ready. Captains only ready when they're ready, and they often wait for everyone to be ready because we're full of happy go lucky friendly people that don't like force starting a match.

For a single captain to be ready, they have to be prepared for the match. This can range from a player just being happy they're in a captain slot, to a captain making sure his crew and his ally ship meet his expectations of loadouts and experience. How long this takes depends on the crew's knowledge and experience.

Most captains also want to have a full crew.


Match imbalance happens for a number of reasons, and most of them aren't intentional. Players want to be on the 'winning' team, they'll flock to the higher ranks, or they'll want to play with their friends/clans. Newer players tend to leave more often, letting more experienced players stick around and start to group more.



You toss in matchmaking, and those issues start to go away. Matches get automatically filled with players around the same experience/skill. Higher ranking captains tend to have higher standards, and these are more rapidly met by higher ranking players who know to do what their captain expects. Lower ranking players don't have that many expectations, and they'll often get a crew who's happy to do whatever.

Matches will also be fair more often. The higher ranking players will be congregated together, but they'll also be put against other high ranking players. This leads to much more interesting matches. This also allows the newer players a less steep learning curve. As they get better, they'll be put against players that are better, and slowly learn the game rather than being tossed into the deep end of the pool under the system we have now.


I've also been informed that this reduces the load on the server by creating the optimal number of matches instead of having 50 matches waiting to fill up when they each have 60% of their slots full. (where 30 matches would be the optimal number).

It might also improve the appearance of different maps (such as 3v3 and 4v4 maps). In my ideal matchmaking system, you'd just put in whether you prefer a capture point match or death match, instead of choosing a specific match you want. The increased variety would appeal to a lot of players instead of doing dunes over and over because someone needs the achievement.



Obviously it's not going to work 100% of the time. There's going to be times when teams aren't balanced, or it takes a long time to find a match. That's just life. But it certainly would improve upon what we have now (in terms of wait time and balance).


You can still group with your friends, and start your custom matches by inviting players; so I don't really see too many downsides to the system. This has been thought about a lot, even up to ways to keep the socializing factor still around. Overall I think it'll be a solid system that improves player retention and general enjoyment of the game.

Offline Squidslinger Gilder

  • Member
  • Salutes: 287
    • [TBB]
    • 31 
    • 34
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #37 on: February 22, 2014, 02:28:43 am »
Well, the worst that can happen is we don't like it, raise a fuss and then Muse responds, hopefully fixes it accordingly. Which they did for Scramble. We raised concerns, you listened and Scramble turned out to be really a good move.

I keep trying to remind myself this isn't EA, Blizzard, or Bioware. When I've tested for other game companies, you can raise more fuss that you ever could have thought possible but they'll never listen. They just put up the build, if any big bugs are found they're fixed, but as for your input on the dev, you have no input. Even if its the worst idea in the history of gaming (ESO beta, Hawken Alpha), they'll never listen. Its just talking to a wall. 

Offline Puppy Fur

  • CA Mod
  • Salutes: 10
    • [OVW]
    • 12 
    • 35
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #38 on: February 22, 2014, 05:56:34 am »
Possibly having two different areas. One for opinions. One for bugs. As not to mix them. That way we can still post our opinions without interrupting testing results (bugs and such).

Offline Velvet

  • Member
  • Salutes: 45
    • [Gent]
    • 19 
    • 22
    • 41 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #39 on: February 22, 2014, 07:31:12 am »
Thank you for taking the time and effort to provide such detailed responses. The effort you put into improving this game and interacting with its community is commendable.

As before, I maintain that removing matchlist damages a part of the game experience. As Byron mentioned, spontaneous accumulation of experienced players in one lobby isn't a particularly rare occurrence and is always pretty interesting when it happens. I think the chances of this happening through matchmaking are unlikely. If you have streamlined the lobby process enough, most people will spend the majority of their time in game rather than searching for matches. Considering the fairly low population on the servers at certain times of day I think even if there are skilled enough players to take on whichever formed crew is looking for an opponent, they'll probably be in-game and unavailable.

As for underdogs beating clan teams, yes it will happen. But I don't think it will happen enough to make the experience enjoyable for these underdogs unless clan teams intentionally handicap themselves by ships/compositions/loadouts, which seems far less interesting than agreeing in lobby to manually scramble.

I still have essentially the same question: Why is there a hurry to remove matchlist? Yes, it may be for the best for it go eventually, but is there a compelling reason to remove it simultaneously with the addition of matchmaking? I'd have thought the safest option would be a two stage implementation and deimplementation to ensure that if matchmaking puts a large number of people off the game you don't lose major chunks of the active playerbase. This would also feel less sudden and aggressive a change which may help to gradually transition players onto the new system.

And one more question. To what extent will crewing on a less skilled player's ship (who dies and loses frequently) negatively affect the "skill rating" of an experienced crew? I'm thinking from the perspective of flying with clan or friends again - in pub matches almost everyone gets a shot at piloting at some point and it would be a shame if matchmaking somehow discouraged this. That leads on to the broader question of whether a good player can get a high skill rating independently of a good team.

Offline Crafeksterty

  • Member
  • Salutes: 73
    • [GwTh]
    • 17 
    • 28
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #40 on: February 22, 2014, 09:30:04 am »
Yes, that is Custom Match.  You will be able to invite your friends into it or have friends join on you (we will also offer password support if we really need to, the user flow for the specifics of join on friend is still being ironed out ).  And yes, Custom Match bypasses matchmaking completely.

This is what i was saying i feared, If the only way we can join is thru FRIENDS then it is flawed. Mostly for us competetive folk. We are going to have to friend almost each and every single person that are in competetive gameplay. Im not talking tournaments, but scrimmages. Scrimmages are essentialy 8 people, apart from the other 8 duking it out. To join, we have to join thru friends. By the end where a competive player or an orginized group plays orginized games, they will have to have everyone as friends on the friends list where it is hard to filter out your current online friends and constant friend coming online messages popping up etc etc.

That is unless you do something with the friendslist, where that is more orginized and whatnot.



Uh, i may sound against the matchmaking concept, im not. I actualy love the idea, i saw every thing good about it. I just want an addition that does not take away orginized play or making us hassle with the friend/friendslist function by too much. Allready some of us have so many friends on friendslist we cant find eachother no more :|


Addittion Edit:

Holy crap, alot of people are against matchmaking, to various degrees. Im not a good person at explaining things. But what matchmaking does is kill the game that is Lobbies of Icarus. Should be apparent since it is so high up in the thread. And makes having people, and playing the game alot quicker.

To me, that is the MAIN dish of the new system. Make you play the game. They even added in with filtering what you dont want and want. Which again, is a secondary dish where you no longer end up on Labyrinth if you just wanna play deathmatch games.

There is not many problems with the stated matchmaking features. In comparison to todays lobbies.


My only concern is 8 people who arent really friended with the other 8 people who want to duke it out. Now, imagine when there are ALOT more competetive players who want to play orginized play. Your friendslist will have to be reserved only for people who, are just contacts than actual FRIENDSLIST. Other than that, if there is an answer to that, im happy. Atleast i will become happy :3
« Last Edit: February 22, 2014, 09:46:51 am by Crafeksterty »

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #41 on: February 22, 2014, 10:05:40 am »
@Gilder, thanks a lot, really appreciate it.  Yeah this is what motivates us to open this system up for testing as much as possible.  I think there might be another confusion about how much testing we intend to do, or how long this is staying in dev app before we release.  Let me just clarify, we are not releasing match making in a week (people have come to expect things staying in dev app for a week before release, and with match making, it is not the case), or even 2 weeks, or 3 weeks, etc.  We intend to test and iterate for a while.  If anyone feels like, oh my god, this thing is on dev app, Howard is going to release it next week, please do not stress.  In dev app, with matching making, you may even say the UI is not even polished yet, and as you will see from the test document, that is totally by design.  We wanted you guys to help test the usage flow and functionality first before we even polish the final UI.  We think that it's smarter to go back to our alpha roots for a system this big, and to get you involved at the earlier stages.  It's risky, because we are relying on you guys to not be biased by work in progress UI for example, but we think it'll turn out well. 

The reason why this thread exists, and that the match making is in dev app testing, is precisely because we want to work with everyone.  And if there are issues, or if anyone finds inadequacies in supporting the use cases of organized teams and clans, we'll definitely need to take a look and improve. 

Overall, I think this is great.  This thread alone helps us to understand more precisely what your concerns are.  It actually really useful for us. 

Have a great weekend everyone!!  Howard

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #42 on: February 22, 2014, 10:26:50 am »
@Crafeksterty, yeah I hear you.  You concern is actually shared by the others who commented, and it's valid.  And it's something we definitely have to take into consideration.  Thanks a lot!

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #43 on: February 22, 2014, 10:34:23 am »
@Velvet, thanks a lot.  We're not being hasty actually.  If I was really being hasty, I would have called for design and implementation a few months back.  So rest assured that we're not trying to be hasty.  I think for now, we really want to focus on testing the system first.  With rollout, I don't think having incongruous systems in juxtaposition is a good idea based on experience, but I really think we can debate that at a later time.  For now, I think the goal for us should be to test the system, and find the inadequacies in both supporting use cases as well as in the things that it is designed to do (ex. wait time, match balance, etc.).  As far as skill rating, we should be accounting for this, but this is something we definitely want to test for effectiveness as well. 

Offline Dev Bubbles

  • Muse Games
  • Salutes: 55
    • [Muse]
    • 16 
    • 23
    • 45 
    • View Profile
Re: Goals and Ramifications of Matchmaking
« Reply #44 on: February 22, 2014, 10:37:54 am »
@Puppy Fur, yeah you're right.  But I think it's ok for now.  We'll need to have a different way to collect, organize, and filter test results.  Internally, we might use Trello this time because of the scale, and we'll need to make sure the pipeline for feedback isn't messy. 
Thanks!