Product-led Growth

SaaS onboarding: iterative testing and ongoing investment

Published December 15th, 2021

As far as product-led growth strategies are concerned, there are many different ideas and projects to explore. But iterative testing is always a constant.

Elise Bentley

Sr. Director of Marketing at Tiny

Of late (perhaps due to the busyness of our lives) it seems to be that we’re taking a once-and-done approach to many projects – thereby dropping the idea of iteration and testing.

However, that can’t be the case when it comes to product-led organizations – your onboarding is a crucial part of your product, and your product experience. You should therefore be continually investing your resources into progressively iterating and improving it – to ensure your users enjoy the best possible user experience.

This three-part series on SaaS user onboarding, covers how it can be used as a tool to facilitate product-led growth:

Part 1 – The who, what, when, where, why
Part 2 – The commercialization of onboarding
Part 3 – Iterative testing and ongoing investment

What does iterative testing mean?

Often when you search testing onboarding processes, most articles delve deeply into the world of UX design. That kind of testing is absolutely needed and invaluable when developing and creating your onboarding flow (even when you’re iterating it), but the area of testing that often gets overlooked is iterative testing.

So, what does iterative testing mean?

Differing from UX iterative testing, the iteration process for your SaaS onboarding, it’s probably best defined as Growth Iterative Testing (GIT). It’s the process of learning, testing and experimenting, to see how you can improve conversions throughout the sign-up process.

For example, at any one time on the Tiny website, there’s five different experiments going on in various places. What we’re looking for with these experiments, are ways to decrease friction and increase satisfaction with the website (and by extension, our overall onboarding process).

However, most visitors to the site wouldn’t even notice when an experiment starts or stops – which is the biggest benefit of iterative testing.

to borrow this eloquent definition from convert-uplift

“Iterative testing is the process of basing tests on insights gleaned from previous experiments to make changes gradually and that are evidence based. It allows a site or app to evolve gradually over time with small steps rather than a large redesign.”

Who’s responsible for iterative testing?

Every company is different. But looking back at my answer to this in Part 1 - The who, what, when, where and why of the series, it’s the same story here. There’s a lot of people – each with a different stake or interest, who get involved in the iterative process.

At Tiny, iterative testing falls within the remit of our growth team, which cohesively combines different departments, working together on experimentation:

Product
Design
Marketing
Engineering

From experience, we’ve found that it's impossible to build (and expand) strong iterative processes without the cohesive support of all four teams. However, where the ownership of iterative testing resides is a purely personal decision, and there’s no right or wrong answer.

The business case behind iterative testing

Committing both time and resources across four departments can quickly become a costly exercise. So, what’s the business case and justification for continuing to invest in testing your onboarding process?

The faster and easier your customer is onboarded and happily using your product, the more likely they’ll stick around (assuming you have a good product) for the longer term.

For product-led companies, like Tiny, iterative testing gives you direct access to valuable feedback from users, customers and clients – while also reducing friction and delivering faster growth.

SaaS testing methodology

For simplicity sake, let’s put the methodologies into two large, broad baskets:

Conversion testing
Customer feedback testing

1. Conversion Testing

Conversion testing or Conversion Rate Optimization (CRO) is the practice of increasing the percentage of users who perform a desired action on a website. That action can be: purchasing a product, signing up for a service, filling in a form or clicking on a link.

A frequent point of conversation between Product and myself, is whether or not user flow testing falls into conversion testing. Take this with a grain of salt, but personally I include user flows, user paths or journeys on the website as well as onboarding, within the conversion bucket. Why? Because they’re all trying to solve three questions: Why aren’t people taking that action? Why are they abandoning the process? Why are they leaving the site?

Conversion rate testing isn’t about selling snake oil, it’s focused on removing friction and helping users find the solution to their problem as quickly and efficiently as possible.

2. Customer Feedback Testing

Customer Feedback Testing is a vital part of testing an onboarding campaign. It’s the domain where many Designers and UX Teams align themselves and can be put into two different camps:

Usability testing
Product Feedback

Within the larger sphere of testing, the UX team has it right – customers are integral in helping companies figure out what to build, and what customers are looking for. Therefore, usability testing is crucial for understanding and serving users in the most beneficial way possible.

But, when it comes to iterative testing on an onboarding program, I’d challenge that we need a slightly different approach.

When looking at iterative testing, the most value lies in the collaboration between Marketing, Product and Design in getting customer feedback. It is important we talk to our customers and understand what they are struggling with, why aren’t they adopting the product, what are the barriers. Through getting this sort of iterative testing and feedback we can improve not just the onboarding process, but the entire underlying product itself.

When to start testing

Building a testing program for the very first time can be daunting. It definitely takes a leap of faith – especially if you’re not 100% sold on the value, exactly what to test or how to go about it.

Our recommendation is to start simple. It doesn’t need to be a complex program that does multivariate testing, A/B testing or extreme personalization. Instead, take a building block approach and gradually create an increasingly sophisticated program.

Bring your testing group together and have regular sessions to work out the schedule of experiments. Get some executive direction on where improvements are needed (i.e. improve the number of initial sign-ups). Then brainstorm testing ideas – even crowdsource across your company – we do it regularly!

A simple vetting matrix helps you determine what may make the most impact, what can be tested simultaneously and you then have a strategy and testing roadmap for the next 6–12 months.

The key to starting is planning!

A Journey of a Thousand Miles Begins with a Single Step

- Lao Tzu

Why invest in continuous improvement?

We’ve covered iterative testing, so now let’s consider why it’s specifically important to the process of building a successful user onboarding flow.

First, think about what conversion rates need improvement. If, through iterative testing you can increase activation rates on your product by 10% (which Tiny's last major overhaul on onboarding did), what impact does that have on revenue?

Here's some quick maths on why it makes sense to continuously invest and iterate on onboarding. In this hypothetical onboarding scenario, you want to assess three core metrics:

Initial sign-up
Activation of product
Conversion to won

These three areas form the ‘funnel’ for your onboarding process (yes the onboarding process is a funnel within a funnel) and convert at different percentages to get to your ‘end’ conversion rate.

Step	Conversion rate
Website sessions > Initial sign-up	1.50%
Initial sign-up > Activation of product	25%
Activation of product > Conversion to won	15%
Initial sign-up > Conversion to won	3.74%

So, how do those rates break down into revenue and customers? Let’s have a hypothetical situation where in a year you get 500,000 website views and have an average deal size of $1,200.

Step	Total number
Website sessions	500,000
Initial sign-up	7,500
Activation of product	1,875
Conversion to won	281
Average deal size	$1,200
Revenue	$337,200

From there you undertake some rapid tests on your onboarding process and you’re able to make modest increases in one core area.

Step	Original Conversion Rate	New Conversion Rate
Website sessions > Initial sign-up	1.50%	1.50%
Initial sign-up > Activation of product	25%	30%
Activation of product >Conversion to won	15%	15%
Initial sign-up > Conversion to won	3.74%	4.50%

Without changing any of the numbers (except for working one step of the funnel at the very top) you end up with a different revenue profile and a very different conversion-to-won rate.

Step	Total number
Website sessions	500,000
Initial sign-up	7,500
Activation of product	2,250
Conversion to won	338
Average deal size	$1,200
Revenue	$405,600

This example shows that by iteratively working on just one metric within the funnel, it’s worth an additional $68,400 to this example business. I can say from the Tiny experience, that by running just one or two iterative experiments it’s possible to get these kinds of increases in conversions throughout the onboarding path.

In this particular model, every 1% improvement in the onboarding process and getting customers from one step to the next, is worth roughly an additional $90K to this example business.

If you pushed it to 1 million website views, it would be worth $180K – which highlights that improvements within an onboarding experience compound directly with the amount of traffic pushed into the top of the sign-up and onboarding funnel.

Test, test, test

User onboarding and the automated process is just one step in the wider customer journey and needs to be complemented not just with a beautiful web experience, but also strong emails, and the customer success team service.

The core to creating a truly great experience that creates loyalty and generates growth is to engage and work together with your wider cross-functional team to deliver the experience your customers deserve.

This is the final part of a three part series on user onboarding and how it can be used as a tool to grow your product-led business. Read the other parts of the series: