Why Yahoo Glue is a Bigger Deal than You Think

Yahoo's India team quietly launched Yahoo Glue out of the glare of the media circus around Yahoo these days.  Glue has been noticed by a few commentators (e.g., TechCrunch), who mostly see it as response to Google's Universal Search (or Ask's search interface). They might be missing the point. Glue is fundamentally different from Universal Search and represents a whole new way of thinking about search.

The fundamental distinction is: where do the results come from? Google's Universal Search searches across Google's properties: web search, image search, YouTube, Scholar, Google News, and so on.  Glue, on the other hand, includes not just Yahoo properties (web search, Yahoo! Answers, Flickr), but also pulls in results from WebMD, HowStuffWorks, and even (ironically) Google Blog Search and YouTube. For example, compare Universal Search (diabetes) with Glue (diabetes). In this respect, Glue bears more similarity to mashups such  as Addict-o-Matic than to Universal Search.

The Web today is a far different place from what it was when Google's search paradigm was invented. The web was then a collection of documents; it is now a collection of applications. Applications such as Facebook, MySpace, YouTube, and Yelp. Each application has its own deep collection of data, and we tend to think of them as being different information types rather than just "web pages". Yet the search model flattens each of these rich interactive services into a collection of web pages that can be indexed -- that's really putting very new wine in a very old bottle.

The one-index-fits-all model forces a linear ranking of incomparable types, such as images, videos, how-to content, facts, and opinion. That's really comparing apples to oranges. The correct way is to deal with each of these information sources as a first class citizen, with its own kind of data and interaction paradigm. Let them compete for real estate on a 2-dimensional search results page (as opposed to 1-dimensional list), and may the best ones win.

Of course, there are many technical challenges to be solved in getting there.  Sending each query to every web application is a recipe for disaster; and dealing with many different APIs is not conducive to scaling. To keep themselves relevant, however, search engines must evolve from indexes to intelligent routers of searches to third-party applications. 

Comments

[mailman]

I agree. Web is moving beyond 'reverse document index' based search but big search engines have not. It's great to see Yahoo out innovating Google in this regard. This is a great direction and complements their recent announcement of opening up their Search platform to third party data imports.

http://www.ysearchblog.com/archives/000534.html

[abhishek]

Nicely worded! Its refreshing to see Yahoo continue to innovate under pressure. However, they do have a far way to go. For a topic such as "Heart Pain" the result drops to search results. I will be interested to see how committed they are to this initiative and do they walk the last mile which is what differentiates an experiment from a consumer product.

[Trevor Lee]

Well put, you finally explained the issue in a way that made sense to me.

[Joe Lazarus]

I agree with a lot of your points regarding the need for innovation in search, but I'm not as enthusiastic about the Glue concept specifically.

It's important for search pages to clearly and efficiently guide the user towards the result that is likely to be most relevant. Glue offers more choices, but without highlighting certain results over others in a clear way, the user ends up having to scan a lot of information in order to build any confidence in the relative quality of the various results. It's just too much work. I applaud the effort to rethink search, but I don't see this as the right direction.

I hope Yahoo! and other companies continue to experiment. We're due for another major search innovation. Hasn't been one since Page Rank in my opinion.

[jack]

Very interesting article. A truly smart move by Yahoo, in my opinion.

I was surprised how powerful Glue is for general/vague queries (e.g. 'Mumbai', 'Tandoori chicken', 'Data mining'). The results have great diversity (e.g. a video lecture showed up) and are aesthetically presented using the whole page.

Strict ranking is somewhat pointless for vague queries. Image and product search engines know this and often display results in grid format opposed to list format.

[Eric Norlin]

http://gluecon.ipower.com/blog/?p=7

[Todd Levy]

Anand -

I'd really love to hear your feedback on my universal search mashup SearchQuilt...

http://searchquilt.com/
http://searchquilt.com/search/?q=diabetes

Here are a few thoughts about it based on the (thought-provoking) criteria in your post:

> where do the results come from?

It's a bit Google-heavy (at least for now) though also includes products from Amazon and auctions from eBay.

It'd be trivial to incorporate delicious links or Technorati results or [insert name of any index with an API or public RSS feeds] but I'm not so confident of the value in light of PageRank's (perceived?) superiority in producing relevant results.

> linear ranking of incomparable types

With my "quilted" layout, the rankings of each individual data type are preserved.

Plus it avoids the "a la carte problem" -- click the images tab for images, then the news tab for news, then groups, then blogs, etc -- allowing for a content-type agnostic overview.

> with its own kind of... interaction paradigm

I've not really expended much effort to create specialized layouts for the different data types, though I appreciate that this is a ripe area for exploration.

Look forward to your thoughts.

TL

P.S. When I read your comment about "very new wine in a very old bottle" it reminded me of John Barlow's wonderful article "The Economy of Ideas: Selling Wine Without Bottles on the Global Net"

http://homes.eff.org/~barlow/EconomyOfIdeas.html

It's not got much to do with this post, but 15 years later it's still my favorite article about the "new rules" for intellectual property. Strong recommendation if you've not read it before.

[Anand Rajaraman]

Joe Lazarus:

I agree. Yahoo Glue and others out there (e.g., SearchQuilt, which Todd Levy points out) address some of the issues raised in this post, but don't go far enough in terms of figuring out which information sources out there are best for a given search and highlighting them. IMO the Holy Grail here is a service that takes a search and determines which sources to query on the fly.

[Anand Rajaraman]

Todd:

I tried out SearchQuilt and it addresses some of the same issues as Yahoo Glue. Good stuff!

As Joe Lazarus points out and I agree, the next step is to expand the set of sources, and figure out which ones to surface for each search. That will require some interesting algorithms.

[Dave]

Regarding the comments about Yahoo's Glue not going far enough to point out what is 'truly relevant' for the query term - how do you know? User A and User B can use the same term to convey completely different intents. User A may search 'HDTV' wanting to research the technology while User B may search 'HDTV' looking to buy the cheapest set they can find.

IMHO - at least Glue gives you a rich visual indication of how you might need to refine your search, as opposed to having to click through 4 or 5 web search links before deciding you have to change the query. Glue is a quicker path to search success for the user in my opinion.

[Greg Linden]

Looks quite a bit like A9's old federated search to me. While Yahoo Glue does more than just show the different results from different sources in columns as A9 did, Yahoo Glue still mostly appears to punt on the hardest problems with federated search, query routing and relevance rank of the merged results.

By the way, you might be interested in Google's take on federated search, at least as voiced by Googler Alon Halevy and others in their paper, "Structured Data Meets the Web: A Few Observations".

http://www.cs.berkeley.edu/~jeffery/pubs/debull06-2.pdf

I have some excepts from and thoughts on that paper here if you are interested:

http://glinden.blogspot.com/2007/03/end-of-federated-search.html

Thanks, Anand.