Thursday, September 20, 2012

I Took the Bing It On Challenge ... AND LIVED!!!!



Since you are currently on the World Wide Internet, as it is called, you are probably able to use what is called a Searching Engine to find the things you want, be they My Little Pony jpgs, or Gangnam Style parodies, or ... well, I don't know. Erotica? Anyway, since you are using a Search Engine, I can say, with some confidence, that that engine is named Google.

However, you may not be aware that Google is not the only Searching Engine (or search engine, as the technocrats call it). There are actually several of them. For example, Microsoft spent more money than an unaided human brain can comprehend to create one called Bing. However, its market share is still only around 15%, as it is not Google.


To increase Bing's user base, Microsoft created the Bing It On Challenge. This 21st century techno-version of the Pepsi Challenge has you use a computer to enter five search terms. Then it shows you the Bing and Google search results, side by side, and rearranged slightly so you can't instantly tell which is which. You pick which of the search results is better. Then the thing tells you if you preferred Google or Bing. And, if you preferred Bing, WHY WERE YOU USING GOOGLE ALL THIS TIME, YOU DUMB IDIOT!?!?

Here is a commercial for the Bing It On Challenge. Watch it, if you dare.

(A quick aside about this commercial. I find it fascinating. I am amazed at how abrasive and arrogant the Bing guy is. I want to never, ever, ever use Bing, just to make that guy sad. In the very unlikely event that a Microsoft marketing person reads this, a little unsolicited advice: When you are trying to sell things to technical people, remember that this is a demographic that responds very poorly to bullies.)

Well, I can never resist a good challenge, so I took it myself, and Google won. Not by a huge amount, but by enough. And the reason why was pretty interesting, which is why I am writing this. It doesn't mean Bing is a worse search engine than Google, or bad in some Universal Sense, but I think it says a lot about how search engines work and why making a good one is so difficult.

The thing about a challenge like this is that when you buttonhole someone and say, "Search for five things! Now!" this on-the-spot person will probably just grab five well-known proper nouns. Justin Bieber. My Little Pony. Muffin recipes. Stuff that is easy work for search engines. Every single one is going to be able to handle that stuff. But that is not how I (and, I suspect, many others) use Google.

The reason Google became so popular in the first place is because of its almost supernatural ability to guess what the user is thinking. A search engine must take a random clump of nouns, adjectives, prepositions, and so on, and then tease from those words the specific thing the user's brain desires. In other words, it's an artificial intelligence problem, and a very deep one.

The reason Google took over the world is that it knew you were thinking of Justin Bieber even when you didn't type in Justin Bieber. If you put in "annoying teen singer", the words "Justin Bieber" will appear somewhere on the results page. (And it does, on both Google and Bing.) Google isn't as good as it used to be, but it's still pretty darn good.

(By the way, all search results described are as of 5:30 PM, Pacific Time, Sunday, September 15, 2012, Seattle, U.S.A., Third Planet From Sol.)

The search results below are for me using a search engine the way I, personally, use a search engine. Which engine, Google vs. Bing, is best written to match my aging, dumb brain?

Let's find out!

1. For a first test, I picked a real life thing. A friend once told me that I had to check out this Penny Arcade webcomic. Next time I was at a computer, I tried to, but I couldn't remember the strip's name. I went to Google and searched for something much like this:

the cartoon about video games

I think most people would reasonably agree that, if you put this term into a search engine, Penny Arcade should show up on the first page.

Google - First hit is the Wikipedia page "List of television programs based on video games". Fourth hit is Penny Arcade. Pretty good.

Bing - The first two hits are, oddly, general pages for the Cartoon Network. Lots of cartoons, but none specifically about video games. Penny Arcade doesn't show up in the first five pages.

The Winner - Google.

2. I'm still in a nerdy mood, so I pick something a geek might want to hunt for: Classic 80s SF movie Blade Runner. What would you look for if someone told you about its awesomeness but you could only remember the rough description:

the movie where the hguy hunts androids in the future

I mistyped "guy", but I left it that way. Let's see how well the engines handle that.

Google - Wikipedia page for Blade Runner is entry five. As a bonus, the sixth entry is the Wiki page for "Do Androids Dream of Electric Sheep?" which is the story Blade Runner is based on. Nice work.

Bing - Blade Runner wasn't in the first five pages, but  "Do Androids Dream of Electric Sheep?" is at least on page 3.

The Winner - Google.

3. No more nerdiness. Let's pick a factoid from history. Something juicy. I think we can all agree that there's only one particularly well-known answer for:

the girl who killed people with an axe


Google - Wow. There are a lot of female axe killers out there. The Wikipedia page for Lizzie Borden is down at hit 8.

Bing - First entry is a Yahoo Answers page where someone answers the question. The second hit is the Wikipedia page for Lizzie Borden.

The Winner - Bing, narrowly.

4. Now I'm going to pick something serious. Something real. Something I bet a million worried teenagers search for every day:

how do you get aids

Google - First hit is a basically accurate kidshealth.org article titled "How do you get AIDS?" Second hit is a similar article from aids.gov.

Bing - Top three hits are the same as for Google.

The Winner - Tie. Thank goodness. It'd be worrying if either one messed this one up. If some shenanigans caused the first hit to be some bad information (such as that HIV doesn't cause AIDS, which some people actually believe), it could cost real, non-hypothetical lives.

5. For the fifth one, I decide to engage in a bit of whimsy. I wanted to know just how willing these search engines are to sacrifice their own interests in order to help me:

what is the best search engine

Google - First hit is an about.com article called "The 10 Best Search Engines of 2012," that describes the pros and cons of ten search engines. Google and Bing are in there, of course.

Bing - The first official hit is the same. But it's right below an ad link for, you guessed it, the Bing It On challenge. Because of how the page is laid out, the ads look an awful lot like real links. It's kind of shifty, and Google does a much better job of making ads look distinct from results. But hey, if I'm just evaluating quality of search results ...

The Winner - Tie.

What Have We Learned?

Not too much, really. I know that Google AI’s approximation of human thought matches the way my brain works better than Bing's. But who cares? Someone else might use the exact same search terms to fish for entirely different answers. And maybe Bing is better at predicting those.

Microsoft isn't trying to prove that Bing is the be all and end all of search engines. They're trying to get people to give it a shot and they’re hoping that a portion of them will stick with it.

It's a tough job, though. If someone is used to using Google, Bing merely being as good or a little bit better won't get anyone to switch. To get people to actually break a habit requires a big improvement. Very difficult. If that's your goal then getting your targets to actually try the new thing is an absolutely necessary first step. Hey, I gave Bing a try. Didn't work for me, but at least I looked at it. So, victory!

Pity about that commercial, though. MAN, but that guy is a jerk.

16 comments:

  1. I've been fooling with search engines for a long time and two things I always test besides me own name is "nurgle" and generally something to do with "calves". The first used to matter to me more, but the second is more illuminating when combined with an adjective. Search engines now want to know who you are and use your past search history and probably other data they've collected on you to filter your results. Google definitely uses my IP address as do advertisers. I kinda let Google customize my search results or personalize them, but I don't let Bing know as much about me, as I use it as a better approximation of what other's see when they search the Internet for say "fat calves" or some keyword phrase I care about. Does "fat calves" yield animal images, or rants about anatomy? I'm off to take the challenge and find out.

    ReplyDelete
  2. Well that was anti-climactic. My attempts to go to bingiton.com via your blog or by typing it in, result in me being redirected to bing.com, maybe Canadians don't matter...

    ReplyDelete
  3. Back when I first started using primarily Google, it wasn't the quality of search results for my reason to keep using it: It was the speed of the engine. Back then, a lot of the other search engines were covered in ads. Banner ads, animated ads, and back on dial-up, that really impacted how fast the site loaded. Google was pretty much always "bare-bones" looking and had mainly text ads, so it was always much faster to use.

    Back then, that's what got me to use Google over the others and what got me to stay. Nowadays? Eh, I've been using it for at least 12 years now, I see no reason to change. And it's pretty much ingrained nowadays, when I want to search I'm typing in google.com before I even think about it. It'd take some effort to retrain myself to type in bing.com. =p

    Yeah, yeah, so there's a search box next to the address bar, I never use that. Been on the internet too long, since before those fancy doodads came into existence.

    ReplyDelete
  4. When I did it bing barely eeked out Google. Barely. I used a lot of very nerdy queries too. (i.e. fan remake of tomb raider 2.) I don't know if I'll switch because of that but it was interesting.

    Now, I have a huge beef against search engines that I blame mostly on Google. A lot of times I have to go several pages deep into my search results to find what I want because people are so into search engine optimization and gaming the search results that usually the top ten pages are stupid ads to buy an ebook or harvest email addresses. Gah!

    Anyway, rant mode off. T'was an interesting challenge though.

    ReplyDelete
    Replies
    1. On a side note I was having a problem with the captcha here and decided to use the "listen to" option.

      What a mistake! Sounded like Dameon reciting the Satanic verses. Will never do that again!

      Delete
  5. In europe the challenge also fails, redirect towards bing.

    ReplyDelete
  6. Some things that makes me prefer Google too:

    - Google is still not as good at recognizing symbols but searching "y = sin x" returns the graph but not bing. However, not all equations returns graphs though.

    - One extremely powerful feature I love of Google is Auto-Complete. It's particularly useful to use as a spell-checker! And a laughs. :P

    ReplyDelete
  7. Bing is also using this to collect curated data sets of which queries they need to improve compared to Google. Sneaky really. By taking this challenge, you are contributing a data set to Bing which exactly tells them where they lose to Google, and where they win.

    ReplyDelete
  8. Your last point is right on. In the 90s, I rotated through search engines fairly frequently before finally settling on AltaVista (though I thought AskJeeves was awesome some of the time). I don't remember exactly why I switched to Google, but it was definitely because they were doing something much better than everything else. It would take something really special to pull me away, and Bing ain't it.

    ReplyDelete
  9. I remember in college, keeping a small stable of semi-reliable search engines... those days before the crafting of Google. My logic then was much the same as why I had multiple installed web browsers later on. Some stuff fails. When a thing I want is not available via what I am using, I switch. Then google came about... and there was no longer a need to switch. At this point, I can no longer remember what search engine I had the best luck with... info something?

    I recall surprising the heck out of my peer group for my accurate searches. Which would be when I had gleaned enough about the subject to determine at least one phrase that had to be in there. There's a difference in the type of page you get that states "Studies show" vs "one study, however". That skillset has somewhat atrophied... although now and again, it does help narrow the field.

    ReplyDelete
  10. /me searches for "binf it on" in bing, hitting enter without correcting spelling.
    ...
    >Including results for bind it on.
    >Do you want results only for binf it on?

    xD
    Pretty sure Bing fails the self-acknowledgement challenge. :P

    ReplyDelete
  11. you should try blekko 3 engine monte.

    http://blekko.com/ws/+/monte

    ReplyDelete
  12. Hi,
    I am from Virat Bharat I read This Blog. This Is very Informative to our Youth. Really Good If you Want to see our Site You can easily Visit

    http://www.viratbharat.com/

    Thanking You
    Puneet Kardam

    ReplyDelete
  13. The minute you think of giving up, think of the reason why you held on so long EQ2 Plat, no matter when you start, it's important that you don't stop after you starting EQ2 Platinum, no matter when you end, it's more important that you don't regret after you ended EQ2 Gold.

    Do not pray for tasks equal to your powers.Pray for powers equal to Buy rs gold your tasks.Then the doing of work shall be no miracle, but you shall be the miracle, Fear not that the life shall come to an end, but rather fear that Runescape Money it shall never have a beginning,. At twenty years of age Runescape2 Money., the will reigns; at thirty, the wit; and at forty, the judgment.

    ReplyDelete
  14. Well it seems that a search in Google for 'the girl who killed people with an axe' now gives Lizzie Borden as the top 4 results. It appears that Google has automatically adapted to the results of this experiment.

    The Bing results do not appear to have changed.

    ReplyDelete
  15. Wow, from the original article, I would say Google won hands down, not, "not by a huge amount." I will not be switching for anything now.

    BTW, I just discovered your blog, now I am going through your back articles. I especially like your game design articles - always very thoughtful ,even for a non-programmer like me.

    Rather, I am a PnP GM, and I find your insights equally useful in my medium.

    ReplyDelete