Optimizely is a platform that provides useful tools to build and develop A/B tests as well as personalization experiments for websites and mobile apps. Optimizely comes with so many excellent tools that the possibilities and ways of creating an experiment turns out to be endless. But the support it requires can be so overwhelming that utilizing it to its full potential becomes difficult.
When a high-end tool like this starts to throw issues and alerts at you out of nowhere, it can leave new and non-techy users of Optimizely confused. (That’s why we also highlight a few Optimizely A/B testing problems later in this article.) But first, let’s look at some tips which could prevent a few of those Optimizely issues from happening.
Best Tips for a Smooth Optimizely Experimenting Process
A few basic tips can elevate the performance of your experiments and ease the whole process in general. Here are a few best practices you can include in your experimentation process to enhance your A/B test building process on Optimizely.
1. Avoid Using HTML Tables for Page Layout
HTML tables are hard to adjust with Optimizely and JavaScript in general and can affect performance drastically in browsers such as Internet Explorer. Optimizely suggests doing it with CSS if possible.
2. Use Unique IDs for Elements Wherever Possible
JavaScript is used in Optimizely to execute page loading to show the test’s variation. Using a unique element ID to present an element helps Optimizely identify them easily while executing JavaScript. This way, there are fewer chances for the page variation to look different in other browsers. This also makes up for clean variation code.
3. Track Goals on Landing Pages via Custom Events
While tracking user behavior in a landing page experiment on Optimizely, the tracking goals for every visit could consist of clicking on the CTA, clicking on Add to Cart, or page view of the confirmation page after submitting forms. This last one is applicable to the elements which do have a confirmation page, but for cases in which websites do not have a confirmation page after form submission, you can use the custom event goal on a form present on the page.
This is just an example — custom events can be used to track many visitor actions that are not directly associated with clicks and page views.
After the setup of a custom event goal in Optimizely, it gets triggered. It calls for Optimizely snippets and passes an event ID. That snippet takes the ID to generate an HTTP request to configure that event to your experiment reports. The elements which could use this are many — for example, pop-up views or tracking visitor actions in a single-page framework.
4. Launch and Monitor Your Experiment
After the hypothesis and test building, the launch is the step that makes an experiment count. It may look like the job is done, but it is better to keep a watch on it before the final results.
Launching and monitoring Optimizely experiments is advised. Check its progress to know what is working and what is not. The stats engine of Optimizely allows the user to peek into the running tests to know more, but the requirements and nature of the test determine what you need to check first. Per our understanding, we tend to follow Optimizely’s advice, which says to make sure:
- Goals are tracking properly
- Variations are performing in an expected range
- You can now consider segmenting your visitors based on browser, source, device, custom, etc.
- If you find anything which prompts you to stop and fix it, then do so. Optimizely allows the user to pause and relaunch the tests accordingly.
These are some of the best tips which you can follow, and we hope they will help you in some way or another. The goal is to utilize the tool in the best way possible. Optimizely is a paid tool, so you’ll want to utilize it to its fullest to take advantage of all the perks it has to offer.
It becomes frustrating to have to call for Optimizely’s support for every other test, which makes it take more time than usual to do an experiment. Here, we’ve curated a list of common queries and issues for A/B testing with Optimizely, along with their answers.
Solutions to 7 A/B Testing Problems With Optimizely
Optimizely is a pretty steady ab testing tool to work with, but if you’re doing it all yourself, it’s natural to make some mistakes or encounter new errors. Optimizely does have a full source guide dedicated to troubleshooting solutions to the problems which arise due to testing complications.
We know this because some of our clients face these issues due to their lack of a dedicated A/B testing developer team. But those who are aware of common issues can easily figure out ways to overcome and prevent these issues from happening beforehand.
Here, we’ve selected issues our clients need us to solve most often. We’re asked for our assistance to resolve these issues because with our experience comes the know-how of resolving many kinds of errors.
1. No Traffic in an Experiment
This isn’t about not having enough traffic on your site; it’s about not having traffic on the variations even after having enough traffic on your website. If you see no visitors on the result page of your experiment, there could be a bug left behind that prevents traffic from being allocated evenly, or from showing up at all.
Here are the steps to confirm and fix this issue:
- Check that the Optimizely snippet on your website is present in the <head> tag; there should also only be one Optimizely snippet.
- Check for the targeting issue:
- Open the Optimizely log
- Filter the log by your experiment ID
- Look for any errors, such as failing to match traffic allocation, audience, or URL targeting
- Fix any errors if detected, and then refresh the reports to check if the test has started to show traffic or not.
You can find a detailed explanation in a document Optimizely prepared for no traffic in an experiment.
2. How to Run a Test on a Single-Page Application
Single-page applications are like an aesthetically pleasing genre of websites. They are easy for a user to interact with. Single-page applications use JavaScript and other suitable frameworks to manipulate the elements on the existing page itself.
Because of that, the process of building an experiment here varies. The A/B test uses the loading time of a webpage to apply changes and make a variation. But the dynamic changes of the single page based on the actions of the users can sometimes cause overwriting of the changes made by the A/B test. This then affects the results of the whole experiment. This is why it’s important to understand Optimizely’s process of building and running a test on single-page applications.
Optimizely has a powerful kit of features that enables you to build experiments on dynamic pages and SPAs. To gain access to these features, you need to enable support for dynamic websites in your project.
Here’s how you can enable support for dynamic websites or SPAs in Optimizely:
- Go to Settings > Implementation.
- Under the Dynamic Websites section, check Enable Support for Dynamic Websites.
- Save to apply changes.
Once you’ve enabled support for dynamic websites in your project, you’ll gain access to a few additional options in the page settings for activating Optimizely pages. The options are:
- When the URL changes
- When the DOM changes.
You can use these options to define the conditions under which the page will activate or deactivate. When the page activates, the experiment will run.
You can follow our detailed guide on this topic to know how to activate the pages, with additional tips regarding single-page application A/B tests.
3. Optimizely’s A/B Test Results in Google Analytics
Users can integrate their Google Analytics accounts with Optimizely to see real-time experiment reports within GA. It also simplifies the usage of GA goals to trigger events and better audience targeting.
Since Google Analytics already segments the audience on the basis of location, device, age platforms, and other factors, that could also be taken as a consideration while allocating the traffic on the same basis, if required. The integration could be done by two methods, both of which require a Google Tag Manager Account.
- To automate the integration setup, you will need to use the Container Import functionality of Google Tag Manager. This is a comparatively shorter method than Option 2. You can follow this guide on Optimizely and Google Analytics via GTM analytics integration.
- The other method is to do this manually. For that, you can follow this Optimizely and Google Analytics integration checklist, which has a step-by-step guide for using GTM to have your Optimizely Experiments reports in your Google Analytics dashboard. That checklist also explains what to do when you have two or more options to choose from. It is not one click away, but with a tutorial in hand, it is relatively simple.
Optimizely’s Knowledge Base also has valuable tips for having a no-error GA and Optimizely integration.
4. Experiment Not Activating
You work for days on an experiment, and after all the work is done, you hit Publish, and something is not right. So devastating. In some scenarios, maybe it is not even activating, which results in having no visitors on the variation, or not being able to see a variation at all.
An A/B test takes some time to show some sort of activity in the experiment reports, which delays the identification of any issue that is causing the experiment to work improperly. To prevent this, you can follow a few simple steps after publishing a test to identify any issues as soon as possible and make the test run smoothly.
- Check the snippet on the page. It has to be correct and present in the <head> tag.
- To make sure the test is running:
- Check that the desired test has Running status on the dashboard.
- Go to the experiment page, and open the browser’s console.
- To see if you’ve been given a variation in Optimizely X, use the JavaScript APIs mentioned here.
- Type optimizely.get(‘state’).getVariationMap() into the console.
- Check if your experiment ID is present on that page.
Find these steps explained in detail via Optimizely troubleshooting.
- Targeting Issues
In case there is something left over there which is preventing the test from working properly:
- Use the Optimizely Log.
- Filter the log by your Experiment ID.
- Check whether the log reveals any Condition errors, if you failed to match any required allocations.
- If there is an error there, fix that and change them in Optimizely.
- Other issues require different methods:
- For cases in which you are using manual or conditional activation, follow this link.
- Is your site dynamic (for example, how we explained it above in the single-page application portion)? If so, you can use dynamic support to resolve various types of test queries.
- Is your jQuery defined or not? There is a process to use jQuery properly in your experiments. They should align within the Optimizely’s jQuery measures as well.
5. Flash of Original Content
Flashing of original content is not an Optimizely-specific problem — it can be seen in other tools, too. This is when the original content gets flickered in the variation while loading the test’s changes. This not only impacts conversions but manipulates the experiment results as well. For one reason or another, the browser you’re using to view your test is lagging behind the changes you’ve made within Optimizely.
How to troubleshoot flashing/flickering:
- Check to make sure you are using “pure” JavaScript (instead of jQuery).
- If you have too many UI changes in your variation code, consider moving them to Experiment CSS.
- Within the code, make the`body:{opacity:0;}`and make it`body:{opacity:1;}` when dependent elements load on the DOM.
Flashing or flickering most often occur when you are changing the hero image on your site. The default hero image generally loads faster than the hero image injected by the testing tool or loaded from an external content delivery network (CDN). The first step would be to ask the developer to upload the images on their server. The image size is also a key factor here — compress the header images to the smallest file size you can without compromising their quality.
You can fix this by adjusting your CSS, too. Check out our post on A/B test flickering troubleshooting for more details.
6. How to Clear Your Browser Cache and Cookies in Chrome
Clearing the cache and cookies is suggested as the simplest solution to fix editor loading issues. This has a simple two-step process for major browsers. Here are the steps for Google Chrome specifically:
- Type chrome://settings/clearBrowserData in the browser bar (the URL bar).
- From the dropdown menu, select a time range to clear your entire cache and cookies.
- Then, check:
- Browsing history
- Download history
- Cookies and other site and plug-in data
- Cached images and files.
- Click Clear browsing data.
- Refresh your browser, and re-open Chrome.
This is the same process we normally use to clear our history when it starts to load slowly. There is a wide variety of browsers, and you can check the process for other browsers if required.
7. Compatibility Mode Warning and Mixed Content in the Editor
Sometimes the Optimizely editor shows the warning, “You’re in Compatibility Mode.” This means the page is encountering issues with mixed content. It activates because your browser’s security blocks the sensitive or mixed content (HTTP) from loading into the Optimizely editor’s secure content (HTTPS).
This could be fixed in one of two ways:
- Disabling the browser’s security setting:
Google Chrome for Mac or Linux:
- Quit Chrome (if you have it open).
- Open Terminal.
- Type this command into Terminal to open a Chrome incognito window:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome –user-data-dir=/tmp/tempchrome –allow-running-insecure-content –incognito &
Google Chrome for Windows:
- Right-click the Chrome icon you want to modify.
- Select Properties.
- In the Target field, add the command line flag after ‘chrome.exe’ as follows:
chrome.exe –allow-running-insecure-content - Click OK to save.
There exist methods to disable browser security settings in other browsers, too.
- Load the page securely via HTTPS.
Compatibility mode might leave you confused, but it has a very simple reason behind it. It exists just to protect your live website from any kind of harmful activities which could result in a negative impact on the audience’s behavior and conversions. If it suspects any kind of mixed content, it enables compatibility mode. You can shut it down by using the two methods linked above.
Conclusion
We hope this article will help you prevent errors from happening in your experiments. Optimizely users get to explore many new aspects and perks of using it consistently, and our troubleshooting tips are just to make sure the growth and quality of your experiments continually increase with the fewest hurdles possible.
If you have more testing questions, problems, or situations you find hard to deal with — in Optimizely or any other tool — in general, feel free to ping Brillmark. We’ll make sure to assist you in resolving your testing issues quickly.