We are seriously into split-testing at Grockit. For all of our split-tests, we generate a standard report for each bucket of users that includes a whole suite of behaviors like these:
Once you’ve bolted-on visionary inspiration and beautiful design to achieve a high degree of product/market fit for your web application, it’s time to optimize the business and split-testing is one of the most powerful tools you have in your arsenal.
Our custom split-test reporting lets us really get a good handle on the lifetime behavior (and -value) of a particular group of users who have a specific and explicit Grockit experience. This goes above and beyond the more standard ‘did more people click through on this variation?’ flavor of split-test reporting.
But wait, there’s more:
Segmenting your users by source & acquisition channel is a widespread best practice, most often done by tacking on custom URL parameters to the end of, say, an Adwords destination or email newsletter URL. We do this too. In fact, you must be smart about segmenting your users when you get to the scaling step of the Startup Pyramid in order to develop your marketing channels.
At Grockit, we’ve connected this type of off-site channel segmentation with our on-site split-test reporting. When you land on Grockit as a new visitor, we create a guest account for you, which is then populated with all of your user data if you choose to signup, and continues to track your usage of Grockit, so we know how well we are serving you.
One of the bits of data that we store for each user is the original URL that you landed on the first time you came to Grockit. This lets us run a split-test comparison report on any group of different landing URLs/URL parameters. Using this tool, we can analyze both off-site variables like advertising channels, and also on-site factors like landing pages that position Grockit differently for different groups of users.
All this is, of course, in the service of ironing out the kinks in your scalable startup equation, and not just for the sake of metrics alone.
