The Bing It On Challenge
At first glance, the latest Google/Bing spat seemed ho hum. Bing comes out and claims that nearly 2:1 of people prefer them over Google, the Internet rolled their eyes, nothing really changed, and that was that. But then, seemingly out of nowhere, came the Freakonomics professor, who rather viciously attacked the Bing It On study. In case you missed it, Bing has been staging a rather clever “Bing It On” challenge at both NFL games and airports, which had passerbys use both Google and Bing to search for terms. Of course, both search engines were unknown to the user, and only after they chose their favorite results were the search engines revealed. Bing won big time, by a nearly 2:1 ratio. So far, 5 million people have been sampled. The study that came under fire from the Freakonomics professor, however, was slightly smaller.
Bear in mind the implications of Microsoft declaring that people greatly preferred Bing to Google – within the scope of the current search breakdown:
Putting It Into Perspective
To say that Bing won out by a nearly 2:1 ratio is significant in the face of facts on the ground. Google is currently beating Bing by at least 2:1 (including Yahoo) in search market share, which certainly makes the Bing It On Challenge results rather interesting. For a while, it seemed that Google’s Bing It On PR seemed to float around neutrally in cyberspace for the past year (Bing’s search market share didn’t move more than a point), right up until October 1st, 2013, when Professor Ayres (of Yale University) came out and rather thoroughly debunked Bing’s experiment in his New York Times Freakonomics blog. And not only did Professor Ayres debunk the experiment, he went so far as to claim that Google could sue Microsoft for its Bing It On experiment:
…we think that Google has a colorable deceptive advertising claim against Microsoft. It could be worth a lot of money on lost ad revenue if the claims misled people into thinking that a substantial majority of people prefer Bing over Google.
Remember the fear that Microsoft invoked in the mid 90’s? I do. And I know that even though Bing is a daily reminder of Microsoft’s search share inferiority to Google, Microsoft will not take the threat or allegation of a lawsuit while lying down. And sure enough, they responded just 1 day later, via Microsoft behavioral scientist Matt Wallaert.
Quick Matt Cutts Cameo
Now, before we jump into the heart of the argument between Bing and the Freakonomics Professor, we need a Matt Cutts quote on the topic, and thankfully, he graciously alerted his Twitter followers to the Freakonomics post:
Which was followed up by a slick Google+ post:
The Fight
Essentially, the back and forth between all parties centered around the (small) sample size of 1000 participants, the lack of location/personalization, and the fact that Bing offered ‘suggested search terms’ to participants who couldn’t immediately think of a search term. Additionally, Professor Ayres (the Freakonomics guy) reconstructed the Bing VS. Google experiment in Yale University, and found that only 41% of participants preferred Bing results.
As a relative SEO newb, I have a few immediate reactions. The idea that 1000 participants isn’t enough does not necessarily hold weight. How many participants would have pleased Professor Ayres? Every Internet user in the world? This most recent study was conducted by a completely independent research company, and Microsoft behavioral scientist Matt Wallaert rightfully pointed out that a typical Gallup Presidential poll is 1500 people, and no one seems to complain about the validity of those. And even so, the nearly 2:1 Bing over Google results were so staggering that even a large margin of error, say 5%, would still give Bing a considerable advantage.
In what was probably the biggest eye-catcher in Bing’s experiment and pointed out by the Freakonomics professor, it was that Bing offered up specific search terms to participants who couldn’t come up with their own. It doesn’t take a conspiracy theorist to surmise that perhaps Bing calculatingly chose the specific search terms that they suggested to participants. In other words, Bing made sure that their suggestions performed better in Bing than in Google.
Wallaert fights this notion by saying he doesn’t know if Bing performed better in those suggested queries because, of course, Bing doesn’t track user data:
I have no idea if he is right. Because as noted in the previous answer, we don’t track the results from the Bing It On challenge. So I have no idea if people are more likely to select Bing when they use the suggested queries or not.
Followed by this:
Here is what I can tell you. We have the suggested queries because a blank search box, when you’re not actually trying to use it to find something, can be quite hard to fill. If you’ve ever watched anyone do the Bing It On challenge at a Seahawks game, there is a noted pause as people try to figure out what to search for. So we give them suggestions, which we source from topics that are trending now on Bing, on the assumption that trending topics are things that people are likely to have heard of and be able to evaluate results about.
I don’t know if I’m truly convinced. It’s very generous of Bing to offer up suggestions to deer-in-the-headlights participants, but I just can’t imagine that Bing would risk offering up search terms (truly random ones) that might perform better in Google, all on their own volition. It’s just too risky, and frankly, not even smart.
Is Bing the Darling of SEO?
There is one rather interesting caveat with the Freakonomics objection. Their reenactment of the Bing VS. Google challenge found that 53% of users preferred Google, and 41% chose Bing (6% said it was a tie). Consider that Bing currently holds an 18% market share with another 11% from Yahoo for a 29% total search market share. This highlights an interesting discrepancy between Bing’s overall 29% share and the discriminating Freakonomics experiment of 41%. This 12% difference seems to imply that Bing has the potential to jump from their current 29% (depending on how you interpret Yahoo’s role) to at a minimum, 41%. And if you add in the remaining 6% of undecideds from the Freakonomics experiment, Bing can climb as high as 47%! I think that a 47% Bing search market share is a number that would make Microsoft bigwigs chomp at the bit, and likely give the Google brass a great bit of pause. Also, the last time we heard about 47%, it sunk a Presidential contender.
Sure, this 53/47 percent split is far from likely, as Yahoo’s 11% share could be stripped from Bing at a moment’s notice, among other salient issues. But Google has to be a bit worried, right? It would seem that SEO folks ideally prefer Bing to Google, since Bing still provides us with organic search traffic query data, and Google no longer does. This has me wondering: SEO folks argue that the reason we need this traffic query data is to better improve the user’s experience. By knowing and understanding what users are looking for, website owners and builders can optimize their websites with better content that is more suited to what customers are truly searching for. This creates an all around better web and search experience for any Internet user. Now that Google is providing us with zero traffic query data, Bing has more appeal- yes with SEO people and website owners, but more importantly, this appeal will ultimately land with Internet users as well, since their experiences will be shaped by Bing’s provided query data.
The Rise & Fall of Google?
The question now seems to be whether Google’s astronomical 67% search market share is looking flimsy, and they are not sitting by idly. Maybe Google’s 100% not provided maneuver was not necessarily a reaction to the NSA scandal but rather in response to surging search engine DuckDuckGo’s clever & catchy marketing campaign? With Google’s increasingly clenched fists; 100% not provided, revoking API access for companies (MOZ, Raven) that were scraping its SERP’s for data, people are becoming more and more affected by the newest and latest restrictions. Sure, the hurt starts with site owners and SEO people. But I believe that it will have a reciprocal affect on the overall user experience for anyone using Google for search.
So, it seems to me that even though the Freakonomics study mostly debunked the Bing It On results, it may have inadvertently highlighted a bigger search market share for Bing in the process, and outlined a path for future migration away from Google. And with Google’s increasing restrictions, the public may begin to feel an effect, and ultimately, maybe, try a new search engine.