Michael Roebuck's Blog

The question of whether or not you should use the Hackersafe or ControlScan type security logos on your Yahoo! Store has always been a mystery to me that I've been fighting with for some time now. On one hand, they claim to make users feel safe and secure and thus increase sales. On the other hand, Yahoo Stores are already scanned and already secure. Why pay for a service when I can create my own logo and link to a page describing Yahoo! Store Security?

I've recently had the opportunity to meet two wonderful people. Cresta Pillsbury from Hackersafe and Savannah Finney from ControlScan. By "wonderful", I really mean "wonderful." You won't find two nicer, more professional people anywhere, who bend over backwards to help you. My life has improved just by knowing them. Yes, they are that great. Unfortunately, this is business and my clients expect me to make and save them money.

HackerSafe Test

Hackersafe allowed us to run an A/B test on one of our clients. Half of the visitors saw the Hackersafe logo, the other half didn't. I felt the Hackersafe A/B data was a little skewed as they tagged duplicates by the same IP address. I fixed this by manually removing repeats who didn't have an order number. What I found was Hackersafe counted 112 legitimate orders as repeats and thus weren't included in the results they provided. In actuality, these 112 orders were repeat customers and should have been included. I have 25 days of data comprised of 1831 orders.

To throw another wrench into the test, this client sends an email blast out about every 2 days. Apparently, email can skew the results because existing customers already trust us and the HackerSafe logo has less of an impact. Because of this, I went one step farther and looked at each of the 1831 orders. Yahoo! gives me the ability to track if they are new or repeat customers so I looked at the days where new customers placed more orders than repeat customers. Of those days, it was a fairly even split of orders placed whether they saw the HackerSafe logo or not.

In aggregate, in the 25 day test period, I noticed an increase of orders of 4% (as opposed to the 12% HackerSafe claims). Because of the increase, I'm still skeptical, but we are willing to continue the tests. We'll run the HackerSafe logo on a full time basis for a few months and see what happens. Until then, I'm afraid I still haven't answered the question of whether or not you should pay for the HackerSafe logo. You should though, test it and be sure to look at the data file provided and review that data yourself. If you need any help or find any flaws in my methodology, give me a call.

ControlScan Test

ControlScan was going to let us do an A/B test but they use a third party software package that isn't that compatible with Yahoo. Because of that, we had to go by sales alone. In a 3 week period, the sales after ControlScan was added were fairly level with the 3 weeks prior. We think we could squeeze a 1% increase to ControlScan. Certainly not enough to justify itself.

This was actually very surprising to me because ControlScan consisted of tall skyscraper banners as well as a banner on the top of the website and check-out pages. HackerSafe was only a tiny little banner on the top of the website. I really thought ControlScan would outperform HackerSafe. Needless to say we removed the ControlScan banners and will follow up with a HackerSafe A/B test.

Final Word

Test, Test, Test. Results will differ on everyone's website. The websites we added the banners too already had a link to Yahoo security. We were really testing whether or not the name recognition of the banners would increase sales. Everybody I spoke with at the client site hadn't even heard of HackerSafe or ControlScan and nobody cared or said they would click on the banner to see what it was. The Marketing Director recently went to a site and noticed the HackerSafe logo but only because she knew we were testing it with her site. Is a commercial logo really better than just writing that your order is secure on the website headers? Does a customer really recognize "HackerSafe" or "ControlScan" as better than, "Your order is Secure" or a homemade "Site Security" banner linking to a descriptive page on your own website?

I can't show you the client banners we created and I'm not sure if I can copy the HackerSafe logos, but what would mean more to you:

I guess I still have to fight with myself on whether or not we want to recommend this product or not. Certainly bigger companies are using it, but does that make it right? If anybody has any information on these or other site security services, please comment or let me know.

Labels: controlscan, hackersafe, security, yahoo