March 13, 2017: Seminar - Cheng Li: "User-in-the-loop learning systems for high-recall retrieval"


User-in-the-loop learning systems for high-recall retrieval

Cheng Li
University of Michigan
March 13, 2017
11:00 a.m., Room 1000 SEO


Unlike ad hoc Web search, many scenarios like literature review, social media analysis, medical search, legal search, and market research requires a high recall of results on top of high precision. In this talk, I will introduce our recent work on developing user-in-the-loop algorithms for high-recall retrieval problems. I'll describe and demonstrate the effectiveness of ReQ-ReC (ReQuery-ReClassify), a double-loop system that won the TREC microblog retrieval task. In the outer-loop, diverse queries are generated to retrieve documents related to certain aspects of the information need. In the inner-loop, an active classifier is trained to determine the relevance of documents.

In reality, rate limits apply to most search services, making high-recall retrieval and user interactive retrieval considerably challenging. To retrieve multiple aspects of an information need within limited number of searches, we augment ReQ-ReC with a new paradigm of retrieval where multiple queries are kept "active" simultaneously. Queries take turns to retrieve the next "page" of results, and the turns are assigned by a principled multi-armed bandit that exploits old but useful queries and explores new and promising queries.


Cheng Li is a Ph.D. candidate in School of Information at the University of Michigan. Her research interest includes information retrieval, data mining, machine learning (especially deep learning), and natural language processing, with applications to retrieving, managing, and analyzing large-scale data from Web, scientific literature, social networks, and various online communities. In particular, she is interested in developing algorithms for user-interactive retrieval and end-to-end learning for text and network data.

Copyright 2016 The Board of Trustees
of the University of
Helping Women Faculty Advance
Funded by NSF