You should always have a control group when you can because this will account for environmental changes that are hard to track. If you send out the email to only 80% of your users, then you can track how much content creation goes up in that group as opposed to the control group. There might be an unforeseen reason, outside of your company’s control or knowledge, that has led to a widespread decrease in content creation on your site. Without a control group, you might be led to think that the email reduced content creation, which would be far from the truth. Without a control group, you might believe that content creation went down by 10% (instead of up by 5%) for everyone, and you would attribute this decrease to the email you sent.

A/B tests can show you valuable data. You may think you know what the subject line of an email should be to ensure it’s opened, but an A/B test will tell you the truth. You may think you know what landing page the email should send them to so that they will start creating content, but an A/B will tell you the truth. There are very few tools that create such large gains overall.

info@digitaldiamondmedia.com ph: 415-496-6236