Lessons Learned Running a Private Beta

On the web, building a successful business depends on moving quickly, testing often and talking to your customers. We're big believers in launching your MVP (minimum viable product) because it accelerates the learning process.

However, what if your product simply isn't ready yet? It might be close, but still has bugs and UI issues to figure out. That's where we were about 6 weeks before launching Help Scout, when we decided to do a private beta. I've never been a fan of the "beta" tag, but it felt right in this case. We were desperate for feedback.

If you decide to do a beta, planning is critical so that you can make the most of your participants' time and willingness to put up with your product in it's early stages. Here are some of the lessons we came away with:

1. Qualify the Participants

A private beta is for true early adopters, with patience and a genuine willingness to help. Make sure you seek out participants that fit this profile, have the pain point you are solving and are able to spend time using the product.

We announced our private beta on this blog and stopped taking applications after the first 100 signups. We then reached out to every single person with a series of questions. Such as:

  • What does your company do? How many people are there?
  • What's appealing to you about Help Scout? How would it fit into your business?
  • How many email inquiries do you get on average per day?
  • What do you currently use to solve this problem?
  • What features are most important to you in an email ticketing system?

Most of our participants answered the questions. If I had to do it over again, I'd make everyone fill them out. If people aren't willing to answer a few questions from you, they aren't going to provide good feedback about your product. We ended up with about 30 companies that fit the profile and we gave them access to the application.

2. Talk to Participants

Most of your participants will not be pro-active in communicating with you. You must go out of your way several times during the process to reach out for feedback. This feedback is absolute gold, but you have to ask for it. The more rapport you can get with your participants, the better feedback you can expect.

Looking back, the best feedback we got by far was over the phone. Nothing really does the trick like talking with someone in real-time about their problem and your product. We still listen back to some of our calls when we envision how our customers would want a certain feature to work.

3. Keep Participants in the Loop

It can be really frustrating to use a product that doesn't always work properly. With that in mind, you must be overly transparent with participants about bugs, ongoing issues and so forth.

In Help Scout, we created an "outstanding issues" page that outlined everything we knew about. We also listed forthcoming features and items that were recently knocked off the list. This cut down on the number of repeated inquiries we got and kept our participants in the know. Win-Win!

4. Reward Your Customers

At Brightwurks we're huge fans of saying thanks. We rewarded people based on their level of participation during the program. 10-15 of the most active companies got free accounts with unlimited users for life, provided they agree to keep giving feedback and let us reach out every once in a while to get their opinion.

For other participants, we gave them a free month or year based on how active/helpful they were. For EVERY company, we said thanks. That's really important.

5. Don't Draw it Out

When the product is ready, launch it! Our private beta was less than 6 weeks, just long enough for us to knock out some bugs and build all of the account management/billing tools. Launching the MVP is still crucial; private beta just gives you a short time to bridge the gap. Don't lose momentum and stay focused on launching.

Running a private beta is something that requires a LOT of time and thought if you want to get a lot out of it. Put a plan on paper and free up your schedule to talk with customers every day. Follow these steps and it will be incredibly helpful, putting you on the right path to launching a solid product.