FROM TMR TO TURTLE: PREDICTING RESULT RELEVANCE FROM MOUSE CURSOR INTERACTIONS IN WEB SEARCH

Abstract

The prime aspect of quality for search-driven web applications is to provide users with the best possible results for a given query. Thus, it is necessary to predict the relevance of re- sults a priori. Current solutions mostly engage clicks on results for respective predictions, but research has shown that it is highly benecial to also consider additional features of user interaction. Nowadays, such interactions are produced in steadily growing amounts by internet users. Processing these amounts calls for streaming-based approaches and incrementally updatable relevance models. We present StreamMyRelevance!|a novel streaming-based system for ensuring quality of ranking in search engines. Our approach provides a complete pipeline from collecting interactions in real-time to processing them incrementally on the server side. We conducted a large-scale evaluation with real-world data from the hotel search domain. Results show that our system yields predictions as good as those of competing state-of-the-art systems, but by design of the underlying framework at higher eciency, robustness, and scalability. Additionally, our system has been transferred into a real-world industry context. A modied solution called Turtle has been integrated into a new search engine for general web search. To obtain high-quality judgments for learning relevance models, it has been augmented with a novel crowdsourcing tool.