Daniel James - How to Conduct Proper Video A/B Testing

I'm DanielJames- Content Marketing Manager at Rakuten My name is Daniel James. I am the Manager of Content Marketing at Rakuten Marketing. Our goal is really to make sure that people are staying ahead of trends, they're getting enough thought leadership content that can really drive whatever their business goals may be and help them accomplish things in their organization through marketing. So my session is focused on A-B testing. A-B testing is the idea of really, in my opinion, trying to understand what resonates with your audience and what gets them to respond. Your audience could be defined as customers who are part of your email subscription list. They could be followers on your social media page. They could be people who show up into a brick and mortar store. Or likewise, they could be people who read your blog post, people who love your videos on YouTube. Your audience could be any defined group of people who pay attention to your content in some way. The reason A-B testing is so important is because you want to understand what resonates with that audience and how they respond to certain things. One of the best ways to do that is to run tests that give them option A or option B without necessarily them knowing so you can get real feedback based on what their behaviors are. My goal in tomorrow's presentation is to talk about A-B testing from a strategic standpoint and get into that topic. But from the perspective of not every A-B test is going to be considered a success, sometimes you'll end up with an inconclusive result or a negative result. And in those cases, what can you do with that test and the data behind that test that can still be quantifiable in some way and still attribute to a business goal? I think there's a number of different ways you could A-B test with a video. I think that one of the best ways might be length of video. Now this is an example scenario. I think one way you might be able to test it, have two videos, one say a minute 30 and the other five minutes. Have two unique links, one for each. Send it to a focused audience who is engaged or subscribed or what have you with your video content and see who lasts longer duration-wise watching the video. Does the longer content get the same length of time percentage-wise? Are they watching 100%, 80%, 50% of the video versus the minute 30? Are they watching 100%? Are they watching 50%? And then think about how is it coming across? Are you getting the same message across? Are they able to take the same key concept away? I think length is a very powerful resource, especially when it comes to video. Like I said, we're all connected and because of that, we all have attention that goes in different directions. Capturing someone's attention for any length of time is a great thing. Being able to hold that attention for a certain amount of time takes a certain level of skill and also an understanding on what that audience might want. Being able to test something like that I think is really important. Then finally, you want to make sure you define that KPI that correlates to that goal. If your goal is to get someone to just watch your video through its entirety, you're going to want to know what percentage of the video was watched on average for each one. Being able to define a KPI like that is really important. Likewise, if your goal is a little bit different, your KPI will need to change. Another example, you have a video. Same thing, only this time, rather than measuring length, you're trying to measure what their behavior is after. Are they clicking a link embedded in the video at some point in the video? You'll want to measure the conversion rate of that, obviously, but you'll also want to look into things like did they even get to that link and consider how length plays a factor into that. There's a number of different ways you can really play with video and A-B testing. There's certainly a big case to do it. This is a huge format. A lot of people, like you said, are going to be investing in it. That means you need to make sure you understand what your audience wants from video. You're going to want to test different types of content, length, and understand what you want to get from that by putting that out there. I think a visual component is a powerful tool. I think something like video can be engaging on a multitude of levels. It's able to capture someone when it's done well, the same way that writing is or photography, but the difference is you can explore a medium and do so many different things with video all at once that you might not be able to accomplish in the same way as written content or visual content. That's not to say they're not important. They are, but with video content, you could create a tutorial video that could educate one person on one aspect of something, another person on another aspect, or get someone else to be thinking about something differently. It really is just such a powerful tool. I would say it's just such an engaging format.