Hey, it’s your resident game designer at Muse Games, Eric. As of late, I’ve been taking on a lot of marketing responsibilities like managing our Facebook page for Guns of Icarus Online to bantering on Twitter. Muse is a small outfit so if any of us have free time or the skillset, we’ll try to fill in additional roles. To some extent, I’m a jack-of-all-trades and maybe only a master at one (game design, duh). I think that’s enough to preface the story I’m about to tell, a huge fiasco but also an unexpected win for us at Muse Games.
If something can go wrong… it will definitely go wrong if the internet is involved.
More after the jump…
What we wanted to do was drive the number of Likes on our Facebook page. Previously, we ran promotions through our email list like “the difference in likes from now till the end of the competition is the number of beta keys I will be randomly distributing.” This definitely drove traffic to our page, the first email list promotion earned us about 500 likes, but this method was not only random but people quickly lost interest when we ran it two more times (300 likes, 170 likes). We needed to step up our game. We reached the point where we need to ramp up our player base very heavily so after we distributed keys to all of our patient email listers, we decided to try a Facebook Like Application.
What these apps do is hide information if you haven’t liked a particular Facebook page. These are fairly common nowadays so we wanted to trade one Facebook Like for a beta key to our Guns of Icarus Online. Here’s a summary of what happened:
- FB Like App created with 6500 (7/312012, evening)
- App quickly circumvented, beta keys pool bled dry, obvious usage of bots
- Keys appear on a variety of websites with misleading information
- Attempting to put out all the fires
- New FB Like App with more security created with 800 keys
- More shenanigans force our Gmail account to lock itself for 24 hour (Google spam safeguards)
- Personal email server used, also crashed quickly
- Unused keys reclaimed, App taken down (8/1/2012, afternoon)
- Writing this article (8/2/2012)
I suppose we’re pretty humble at Muse, naive to some extent, but we always thought our game was small and not that many people would be using the App so let’s just say our first round of security measures weren’t stellar (just filling out a simple form, pop-up with the key on submit). Our first mistake. We knew you could grab extra keys but what really shocked us what was people writing bots to automate the process for them. Through some cyber sleuthing, we figured out that one person was able to grab 1827 of the 6500 beta keys we had associated with our app. If you take a handful for your friends, that’s cool! BUt what are you going to do with 2000? Very quickly, the pool dried up.
A person with access to closed beta is another person with closed beta access. That’s a good thing! So what we started to worry about was the loss of messaging as the keys were siphoned off and re-posted on opposite ends of the internet wormhole. Many ended up on Steam Gifts and with Steam Gifts each giveaway needs to be associated with an existing Steam game page. We didn’t have Guns of Icarus Online’s page live yet so people were linking to Guns of Icarus (Classic). This was an obvious problem, the keys and the associated game weren’t even the same. To make things worse, the Steam page for beta says ‘retail’ when in fact the keys are just for beta—this very clear for all our old beta players. New players didn’t know what the keys were for. Was it retail? Beta? Both? Not only does this create a buckshot of expectations issues—logging into a buggy beta server to play what they think is retail—we soon realized that it was possible for people to start trading our beta keys for full retail keys of other games! This was a huge potential for all sorts of people to get screwed over, not just us. The customer—whoever, wherever, whenever, and however they came to be one—is always the most important party. We want everyone to have a good experience when they touch our product, even if it’s to return it to us because they didn’t like it.
Without having to say, there were many fires to put out and I spent the entire day wishing I had the FDNY trailing behind, giving me a helping hand. Like some twisted ARG, I followed the breadcrumb trails left behind in Facebook comments, discussion threads, and Twitter to find all of the places where our keys ended up at to see what the damage was. There were even reports that of beta keys on sale for cold hard cash. My heart skipped a beat. As I raced through the series of tubes, I soon ended up on Steam Gifts and found many erroneously linked giveaways. I created my Steam Gifts account and began spreading the good word to ensure the correct messaging got out. Much to my amazement though was the fact that many SG’ers were already reporting the misleading promotions and even copy-pasted my own damage-control comments from our FB page.
As I alt-tabbed between our FB threads and those on Steam Gifts, I realized that our sympathizers were the FDNY volunteers I was hoping for—our real fans. What I realized as BPM started lowering again was that the people who exploited our poorly secured application have very little impact on us in the long run. Sure, some people might be confused. We can’t exactly help that. However, our new fans that were attracted by these free keys (some who even started forfeiting their back alley beta keys) started digging into our game. We saw a spike of 7000 unique visits the day this was happening from our paltry 400 for the past several months (this has now dropped back down to 1000 but we’re hoping this sticks).
All kinds of comments were flooding in from supporting our indie spirit to those asking for keys since we had disabled our app. As I posted update after update of what we were doing—tracking down the fires, revamping the app—I was just shocked at the amount of participation that was happening. We’ve never had anything like this before. Our second attempt at our app was also quickly sabotaged even though we added additional security that required checking your email. Whether by mistake or not, we were getting too many invalid email accounts that caused our Gmail account to lock itself down due to its spam safeguards. Even when transferring to a private email server, it was bombarded and crashed. We eventually just took the entire app down from our page. Strangely enough, we noticed additional traffic on our servers after the takedown which points to more bot usage that even had the ability to spoof emails.
What I ended up doing was grabbing the emails from the database as soon as the app was taken down. I sifted through the list and took out, as best as I could, all the emails I that I thought were from bots. I then sent out keys to these 100 or so players their keys. Remember, the customer is the most important. We fessed up to our idiocy, apologized, and everyone was cool with it. We tried and we were applauded for doing so.
That ended our little fiasco. The app was taken down forever. Fires were put out in several places. It was a long day. Our developer in Taiwan, up all night, programming in a language he doesn’t know, all while a typhoon swept by. It was a “romantic” evening, he said.
Even though this is just days after the entire debacle, we’re noticing a heavy increase in traffic on both our website and Facebook page. We were able to gain over 1000 FB Likes when the app was running, but more interesting is the rate of downtime Likes we’re getting now. Previously we’d only get a few a day. Now seeing several every hour. I doubt this rate will keep up, but I’ll keep monitoring it regardless. It’s satisfying to see more and more people liking and commenting on each of our individual posts.
The main takeaway from the whole experience is to be honest and not be afraid about talking to customers. The internet is a big place where a small handful can affect the majority (for better or for worse), but always remember that those are the few and oftentimes least vocal. Sometimes they don’t think they’re doing anything wrong so I wouldn’t vilify them right off the bat either. Even throughout the breakdowns and mishaps we experienced, we were still trying to fulfill beta key requests through personal Facebook messages. These gestures, even though we could never satisfy all the requests (there were tons! We’re a popular game now! woo!) the number of thank yous in public comments are extremely valuable. A small amount of goodwill seems to go a long way.
Furthermore, this sets an example—the bar. Another thing that we noticed was suddenly a few people emailing us out of the blue asking to be a part of our Community Ambassadors program. Originally we created the CA Program for our most dedicated fans who were already evangelizing the game for us. I think now, after seeing how we operate and act towards our community, other players aspire to be a part of that. Definitely very strong reciprocation we’re seeing here.
There are still a lot of unknowns right now. We’re only a few days post-disaster so long term results are still to be seen.
UPDATE 8/7/12:
Tons of people are asking us for beta keys on our FB page. Kind of scary having people just throw their emails out into the open (we delete those ASAP). Our feedback email address is getting a lot of activity too. Have we reached the tipping point? Yikes!
