Banner – A/B test
Galaxy chocolate has created vegan chocolate with a few different flavors. They have 2 main product pictures that are used online in the form of banners or ads. The first impression of these images counts a lot, especially online where attention and engagement are the highest currency that lead to conversion.
The question was, which one of these two images is the best one to use for online marketing?
We tested these two images in a standard A/B test with our tool to see how people actually react.
Which one do you think is better?
The overall scores already paint a very clear picture comparing the two images. The Engagement of Version A (-1.55 %) is much worse than the Engagement of Version B (10.66 %). This already shows that Version B has a much stronger positive impact than Version A.
The Emotional Impact supports this as well. Version A evoked much less emotions than Version B. The emotion that was most present with Version A was contempt. This was the driving emotion for the negative Engagement score. Version B on the other hand had mostly happy as an emotion, which drove the positive Engagement score.
The HeatMap of Version A also shows that the main focus remains mostly in the center. Version B, on the other hand, creates much more visual engagement as testers were looking at all chocolate flavors.
All of this already shows a clear winner. Version B is perceived significantly better than Version A. Now, let’s dig a little deeper to see what exactly on these images creates these emotions.
To understand WHY Version B is better, we look at the Engagement heatmap.
Here, we combine eye tracking data and the emotion data. This way we know what exactly led to which emotion. Red is a negative Engagement Score, and green is a positive Engagement score. Of each, we also show the 5 strongest points.
Version A’s Engagement heatmap resembles the negative summary score of engagement. Interestingly, the individual flavors evoke a negative reaction and the surrounding of the chocolate bars also evokes this negative reaction. Version B’s Engagement heatmap also resembles the positive summary score of engagement. But here, the individual flavors evoke a strong positive reaction.
The context in which the chocolate is shown makes a huge difference
The overview shows the timeline coupled with the Heatmap recording (top left). This is the best visualization so you have side by side what testers saw and how they felt at exactly that moment.
Just looking at the emotions, it’s clear that Version B creates a constant positive emotion from start to finish. Version A on the other hand does not create any emotional engagement in the first 5 seconds and only then contempt appears.
The Heatmap recording (top left) also shows the general viewing pattern across all testers. Version A creates a more centered focus, whereas Version B creates a better scanning behavior. Testers look at the different flavors and visually engage rather than being stuck at a few locations like in Version A.
Version B creates positive engagement from the start and better viewing behavior
Individual eye movements
The last step to look at viewing behavior is the individual eye movements recording. The Heatflow video shows the overall pattern across all testers, whereas the individual eye movements recording shows the same data but now each tester is shown individually.
In both cases you can see that testers read the flavors, but in Version A they also scan the surroundings more than in Version B. This confirms that the context impacts the viewing and the emotions.
In Version B, testers read the flavors and are more structured in their scanning by looking at the brand and the vegan label. The individual eye movements recording shows that the structural scan path is highly influenced by the different contexts of Version A and Version B.
Version B’s context creates a structured scan path
These results make clear that in this A/B test, Version B is the clear winner. We also know now that it’s the context in which the chocolate is presented that makes this big difference. This can be used to guide future banner creation.
To create and increase a positive brand and product awareness, which also leads to more conversions, use Version B.
Cut your advertising costs by using only your best ads
Ads that your potential customers don't notice is a big wasted opportunity. With Focal Insights, harness the possibilities to the best advertising strategy.