Hacking Growth: the growth experiment process

What does the book argue?

That growth is a loop, run by a small cross-functional group, at a fixed rhythm, forever. Four steps: analyze the data, generate ideas, prioritize them, run the tests. Then start again next week. The authors' claim is that the rhythm produces more than the ideas do, because a team that ships 100 small tests a year will find things that no amount of strategy discussion would have found.

The second half walks the funnel: acquisition, activation, retention, monetization, with the same loop applied at each stage. The activation chapters are the strongest, which is why this teardown sits in the activation stage of the library.

Is the product a must-have yet?

The book refuses to start with marketing, and this is the most useful thing in it.

The test is one survey question, asked of active users rather than signups: how would you feel if you could no longer use this product? Very disappointed, somewhat disappointed, not disappointed. Ellis benchmarked the threshold across nearly 100 startups and found that companies which struggled to grow almost always had fewer than 40% answering "very disappointed", while companies with strong traction almost always exceeded it (First Round Review, November 2018).

The best public account of using it is Superhuman's. Rahul Vohra's team started at 22%, reached 33% after they segmented the answers to find out who loved it most, and got to 58% within three quarters by building only for that group (First Round Review, November 2018).

Two practical notes the book leaves out. Ask only people who have used the product recently and more than once, or you are surveying tourists. And read the free-text answers from the "somewhat disappointed" group before anything else, because that is where the next feature is written down.

What does the growth experiment process actually look like?

Here is the running version, with the meeting, because the meeting is the process.

One scored list, visible to everyone. Every idea goes in one place. Each gets an ICE score: impact, confidence and ease, each 1 to 10, averaged. The book is clear that the score is rough on purpose. Its job is to stop the loudest person in the room from deciding, not to be accurate.

The 45 minute weekly meeting. Same day, same time, no agenda negotiation:

• 5 minutes, the numbers. The same four numbers every week. For activation that is visit to signup, signup to activating action, median time to activation, and 7-day second-session rate. Nobody presents. Everyone reads.

• 10 minutes, last week's results. Each test called out loud as won, lost or inconclusive, with the number. No debate about why. Losses get the same 30 seconds as wins, which is the thing that makes people willing to run risky tests.

• 10 minutes, what runs next. Pick off the top of the scored list. Every pick gets an owner and a date in the meeting, not after it.

• 15 minutes, one thing examined properly. One piece of research, one customer call recording, one funnel someone has pulled apart. This is the slot that keeps the list from going stale.

• 5 minutes, write it down. Who owns what, by when.

The experiment record, eight fields. Before a test runs, it gets written down:

1. The hypothesis, as "if we do X, then Y will change, because Z".

2. The one metric that decides it, with its denominator.

3. The baseline, this month.

4. The minimum effect worth detecting, and the traffic needed to see it.

5. The planned duration, decided before the start.

6. The prediction, as a number, from the person who proposed it.

7. The result.

8. The decision: keep, kill, or run again.

Field 6 is the one I would add to the book. Writing a predicted number down before the test is the only honest calibration there is, and it fixes the weakest part of ICE, which is that confidence is where people lie to themselves. After a quarter you can see whose predictions are worth anything, including your own. Mine were badly over-optimistic for about the first year I did this.

What has dated, and what it costs a small team?

The scale of the examples. Almost every story comes from a consumer internet company with millions of users. The loop as written assumes you can call a test in a week.

The spare engineers. The book's growth team has a product manager, engineers, a designer, an analyst and a marketer. Most companies reading it have one person doing growth part time. The roles matter more than the headcount, and the honest version at seed stage is two people with a standing 45 minute meeting and someone who can ship a page.

The arithmetic of a small team. This is the part nobody says out loud. At Microsoft, one third of experiments proved effective, one third were neutral and one third were negative. At Google and Bing, only about 10% to 20% produced positive results (Kohavi and Thomke, Harvard Business Review, September-October 2017). A small team that runs 2 tests a month runs 24 a year, and at those hit rates that is somewhere between 2 and 8 winners in a year.

Which changes what a test should be. If you only get a handful of winners, each attempt has to be big enough to matter when it works. Four weeks on a rebuilt onboarding flow beats eight two-day tests on email subject lines, even though the second feels more like the book.

What counts as a test. The book is loose about this, and I would be stricter. Five customer interviews that change what you build next is a successful experiment. So is a day spent making the activation event visible in the CRM. Teams that only count A/B tests end up with a loop that cannot run when traffic is low, and then they abandon the loop rather than widen the definition.

The word itself. "Growth hacking" has aged into something closer to a marketing title than a method. The method is fine. The word tells a founder nothing about what the person will do on Monday.

Who should read it?

Anyone about to start running growth as a job rather than as a set of campaigns. Read part one properly, use the must-have survey in the first week, and treat the second half as examples rather than instructions. If you have read it already, the thing worth going back for is the activation chapter, which is better than its reputation.