A/B Testing with Simple Analytics

A/B testing should be in every business “growth toolbox.” Understanding how users interact with different website versions will increase your conversion rate and increase signups and, eventually, sales.

You could use Google Analytics 4, which requires a complex and time-consuming setup process. There are simpler methods to approach this.

This article is an easy-to-follow use case on how to set up A/B tests that will effectively optimize your signup conversions by Daniel Nguyen of BoltAI.

Let’s dive in!

1. Hypothesis and Approach
2. Test Design
3. Execution and Tracking
4. Results and Reporting
5. Final Thoughts

Hypothesis and Approach

First of all, you need to create a hypothesis you want to test. In this case, changing elements like headlines or call-to-action (CTA) can impact conversion rates.

To test this, you need to:

1. Randomly show two versions of the website to different users with a different CTA or headline
2. Track the conversion of each version
3. Create a report dashboard to check the result

Test Design

We want to show each visitor a version of the website. One version is the "control" group or the version already in use. The second version changes a single element.

How these versions behave is up to you.

In this example, we’ll be running a headline test. Variant A (left) and variant B (right).

You can run other tests for calls to action text, pop-ups, featured images, etc., but the method is the same.

Daniel decided to store these variants in cookies and used conditional rendering to present the content of each version. Its good to note that this is not the only way to do this. At Simple Analytics, we don't use any cookies. We would test this by showing a different version of the website every other day. Use version A today and version B tomorrow and so on.

He used Next.js, but it also works for other frameworks.

Here is the code:

import Cookies from 'js-cookie'
import { useEffect, useState } from 'react'

function Page() {
  const [variant, setVariant] = useState(null)
  
  useEffect(() => {
    let cookieValue = Cookies.get('variant')
    if (!cookieValue) {
      cookieValue = Math.random() < 0.5 ? 'a' : 'b'
      Cookies.set('variant', cookieValue, {
        expires: 30,
      })
    }

    setVariant(cookieValue)
  }, [])

  return (
    <h1>
      {variant === 'a' && <span>Current headline</span>}
      {variant === 'b' && <span>New headline</span>}
      {variant === null && <span> </span>}
    </h1>
  )
}

Execution and Tracking

Now, we have the variant stored in a cookie. We need to track the conversion event of each version.

Tracking these conversion events with Simple Analytics is, well, simple. For BoltAI, Daniel wanted to track the number of downloads.

Here is the code used to create the event:

function Hero() {
  const handleBeforeDownload = () => {
    const variant = Cookies.get('variant') || null
    if (variant) {
      window.sa_event && window.sa_event('click_download', { variant })
    } else {
      window.sa_event && window.sa_event('click_download')
    }
  }

  return (
    <Button onClick={handleBeforeDownload}>
      <span className="mx-2">Download</span>
    </Button>
  )
}

Results and Reporting

In the Simple Analytics website dashboard, open the tab "Goals" and click "Add goal."

• Pick the conversion event name. It's “click_download” in this case
• Make sure to filter unique visits & with the right variant
• Set a name.

Now, we can view the conversion stats directly on his dashboard to review the results of the A/B test.

Final Thoughts

A/B testing can be very helpful to grow your business by optimizing your conversion rate. It doesn't have to be complicated. As shown in this case, it's relatively easy to set up an A/B test. It doesn't require a complex tool like Google Analytics 4.

Simple Analytics is an intuitive tool. It takes less time to set up, is easy to use, respects user privacy, and provides real-time insights.

Feel free to give it a spin and see for yourself.