A or B?

I mentioned we relaunched our website last month, and now I can report that we’re doing our first A/B test on the new site.

We’re testing two versions of our Request for Information button. Our traffic is currently split between these two pages:

  1. https://www.cscc.edu/admissions/index-1.shtml – RFI in the middle of the page
  2. https://www.cscc.edu/admissions/index-3.shtml – larger RFI at the bottom of the page

Stay tuned, we should have a winner in a few weeks.

A/B testing is something I want to do a lot more of. It offers the possibility of statistically valid customer insights without having to, y’know, talk to a bunch of people.

I subscribe to Behave.Org (formerly WhichTestWon), a semi-weekly newsletter of A/B tests. You guess which one you think is the best, and then see the winner. It’s totally humbling – I get about 60% of them right.

This is our first A/B test in about four years, since we started the web redesign process. We did about a half-dozen A/B tests on the old site, and I must admit we never implemented a change that had a statistically significant effect.

I’m not sure how to account for that. I reckon choosing a college is a pretty robust process, with a long “sales cycle.” So maybe it’s not susceptible to micro changes in UI. But I keep plugging away, keep looking for some way to move the needle.

I fancy myself a person with pretty decent taste. But, like, how do you know for sure? I feel like we spend a lot of time making decisions democratically, or by seniority. Hopefully this’ll give us  some data on how to make things better.

Leave a Reply

Your email address will not be published. Required fields are marked *