Learning cluster labels from search logs

I much enjoyed a paper, "Learn from Web Search Logs to Organize Search Results" (PDF), that will be presented by Xuanhui Wang and ChengXiang Zhai at the upcoming SIGIR 2007 conference.

The idea is simple but clever. Search logs contain queries that people used to find the information they wanted. These queries might be better labels for groups of web documents than labels extracted from the content.

From the paper:

When the search results are diverse (e.g., due to ambiguity or multiple aspects of a topic) [or poor] ... it [may] be better to group the search results into clusters so that a user can easily navigate into a particular interesting group.

The general idea in virtually all the existing work is to perform clustering on a set of top-ranked search results to partition the results into natural clusters, which often correspond to different subtopics of the general query topic.

Clusters discovered in this way do not necessarily correspond to the interesting aspects of a topic from the user's perspective .... The cluster labels generated [also often] are not informative enough to allow a user to identify the right cluster.

For example, the ambiguous query "jaguar" may mean an animal or a car. A cluster may be labeled as "panthera onca." Although this is an accurate label for a cluster with the "animal" sense of "jaguar", if a user is not familiar with the phrase, the label would not be helpful.

Our idea of using search engine logs is to treat these logs as past history, learn users' interests using this history data automatically, and represent their interests by representative queries.

For example, in the search logs, a lot of queries are related to "car" and this reflects that a large number of users are interested in information about "car." Different users are probably interested in different aspects of "car." Some are looking for renting a car, thus may submit a query like "car rental"; some are more interested in buying a used car, and may submit a query like "used car"; and others may care more about buying a car accessory, so they may use a query like "car audio."

By mining all the queries which are related to the concept of "car", we can learn the aspects that are likely interesting from a user's perspective.

[Our] experiments show that our log-based method can consistently outperform cluster-based method and improve over the ranking baseline, especially when the queries are difficult or the search results are diverse. Furthermore, our log-based method can generate more meaningful aspect labels than the cluster labels generated based on search results when we cluster search results.

There are some compelling examples of the different labels you get with the two methods (search log-based vs. content-based) on page 7 in tables 5 and 6.

The paper also mentions that this kind of technique only appears to help when the query is ambiguous or difficult. The same is true of personalization, but I think that is to be expected. If we have already have good information about user intent -- the query is not ambiguous and the top results are already just dandy -- more information (from personalization, clustering, NLP, or anything else) will not make any difference.

In all, good stuff. I love techniques that look at what people are finding and doing to help other people. Buried in the search logs, there is all the hard work searchers do to discover what they need in a sea of information. By surfacing it, we can help searchers help each other.