Optimizely is a platform that provides useful tools to build and run A/B tests as well as personalization experiments for websites and mobile apps. Optimizely comes with so many excellent tools that the possibilities and ways of creating an experiment turns out to be endless. But the support it requires can be so overwhelming that utilizing it to its full potential becomes difficult.
When a high-end tool like this starts to throw issues and alerts at you out of nowhere, it can leave new and non-techy users of Optimizely confused. (That’s why we also highlight a few Optimizely A/B testing problems later in this article.) But first, let’s look at some tips which could prevent a few of those Optimizely issues from happening.
Best Tips for a Smooth Optimizely Experimenting Process
A few basic tips can elevate the performance of your experiments and ease the whole process in general. Here are a few best practices you can include in your experimentation process to enhance your A/B test building process on Optimizely.
1. Avoid Using HTML Tables for Page Layout
2. Use Unique IDs for Elements Wherever Possible
3. Track Goals on Landing Pages via Custom Events
While tracking user behavior in a landing page experiment on Optimizely, the tracking goals for every visit could consist of clicking on the CTA, clicking on Add to Cart, or page view of the confirmation page after submitting forms. This last one is applicable to the elements which do have a confirmation page, but for cases in which websites do not have a confirmation page after form submission, you can use the custom event goal on a form present on the page.
This is just an example — custom events can be used to track many visitor actions that are not directly associated with clicks and page views.
After the setup of a custom event goal in Optimizely, it gets triggered. It calls for Optimizely snippets and passes an event ID. That snippet takes the ID to generate an HTTP request to configure that event to your experiment reports. The elements which could use this are many — for example, pop-up views or tracking visitor actions in a single-page framework.
4. Launch and Monitor Your Experiment
After the hypothesis and test building, the launch is the step that makes an experiment count. It may look like the job is done, but it is better to keep a watch on it before the final results.
Launching and monitoring Optimizely experiments is advised. Check its progress to know what is working and what is not. The stats engine of Optimizely allows the user to peek into the running tests to know more, but the requirements and nature of the test determine what you need to check first. Per our understanding, we tend to follow Optimizely’s advice, which says to make sure:
These are some of the best tips which you can follow, and we hope they will help you in some way or another. The goal is to utilize the tool in the best way possible. Optimizely is a paid tool, so you’ll want to utilize it to its fullest to take advantage of all the perks it has to offer.
It becomes frustrating to have to call for Optimizely’s support for every other test, which makes it take more time than usual to do an experiment. Here, we’ve curated a list of common queries and issues for A/B testing with Optimizely, along with their answers.
Solutions to 7 A/B Testing Problems With Optimizely
Optimizely is a pretty steady tool to work with, but if you’re doing it all yourself, it’s natural to make some mistakes or encounter new errors. Optimizely does have a full source guide dedicated to troubleshooting solutions to the problems which arise due to testing complications.
We know this because some of our clients face these issues due to their lack of a dedicated A/B testing developer team. But those who are aware of common issues can easily figure out ways to overcome and prevent these issues from happening beforehand.
Here, we’ve selected issues our clients need us to solve most often. We’re asked for our assistance to resolve these issues because with our experience comes the know-how of resolving many kinds of errors.
1. No Traffic in an Experiment
This isn’t about not having enough traffic on your site; it’s about not having traffic on the variations even after having enough traffic on your website. If you see no visitors on the result page of your experiment, there could be a bug left behind that prevents traffic from being allocated evenly, or from showing up at all.
Here are the steps to confirm and fix this issue:
You can find a detailed explanation in a document Optimizely prepared for no traffic in an experiment.
2. How to Run a Test on a Single-Page Application
Because of that, the process of building an experiment here varies. The A/B test uses the loading time of a webpage to apply changes and make a variation. But the dynamic changes of the single page based on the actions of the users can sometimes cause overwriting of the changes made by the A/B test. This then affects the results of the whole experiment. This is why it’s important to understand Optimizely’s process of building and running a test on single-page applications.
Optimizely has a powerful kit of features that enables you to build experiments on dynamic pages and SPAs. To gain access to these features, you need to enable support for dynamic websites in your project.
Here’s how you can enable support for dynamic websites or SPAs in Optimizely:
Once you’ve enabled support for dynamic websites in your project, you’ll gain access to a few additional options in the page settings for activating Optimizely pages. The options are:
You can use these options to define the conditions under which the page will activate or deactivate. When the page activates, the experiment will run.
You can follow our detailed guide on this topic to know how to activate the pages, with additional tips regarding single-page application A/B tests.
3. Optimizely’s A/B Test Results in Google Analytics
Users can integrate their Google Analytics accounts with Optimizely to see real-time experiment reports within GA. It also simplifies the usage of GA goals to trigger events and better audience targeting.
Since Google Analytics already segments the audience on the basis of location, device, age platforms, and other factors, that could also be taken as a consideration while allocating the traffic on the same basis, if required. The integration could be done by two methods, both of which require a Google Tag Manager Account.
Optimizely’s Knowledge Base also has valuable tips for having a no-error GA and Optimizely integration.
4. Experiment Not Activating
You work for days on an experiment, and after all the work is done, you hit Publish, and something is not right. So devastating. In some scenarios, maybe it is not even activating, which results in having no visitors on the variation, or not being able to see a variation at all.
An A/B test takes some time to show some sort of activity in the experiment reports, which delays the identification of any issue that is causing the experiment to work improperly. To prevent this, you can follow a few simple steps after publishing a test to identify any issues as soon as possible and make the test run smoothly.
5. Flash of Original Content
Flashing of original content is not an Optimizely-specific problem — it can be seen in other tools, too. This is when the original content gets flickered in the variation while loading the test’s changes. This not only impacts conversions but manipulates the experiment results as well. For one reason or another, the browser you’re using to view your test is lagging behind the changes you’ve made within Optimizely.
How to troubleshoot flashing/flickering:
Flashing or flickering most often occur when you are changing the hero image on your site. The default hero image generally loads faster than the hero image injected by the testing tool or loaded from an external content delivery network (CDN). The first step would be to ask the developer to upload the images on their server. The image size is also a key factor here — compress the header images to the smallest file size you can without compromising their quality.
You can fix this by adjusting your CSS, too. Check out our post on A/B test flickering troubleshooting for more details.
6. How to Clear Your Browser Cache and Cookies in Chrome
Clearing the cache and cookies is suggested as the simplest solution to fix editor loading issues. This has a simple two-step process for major browsers. Here are the steps for Google Chrome specifically:
This is the same process we normally use to clear our history when it starts to load slowly. There is a wide variety of browsers, and you can check the process for other browsers if required.
7. Compatibility Mode Warning and Mixed Content in the Editor
Sometimes the Optimizely editor shows the warning, “You’re in Compatibility Mode.” This means the page is encountering issues with mixed content. It activates because your browser’s security blocks the sensitive or mixed content (HTTP) from loading into the Optimizely editor’s secure content (HTTPS).
This could be fixed in one of two ways: