Our very own Ariel Nitzav, together with his teammate Eilam Shapira, came in first place in the Information Retrieval Competition, held as part of the Information Retrieval course presented by Prof. Oren Kurland from the Industrial Engineering and Management faculty at the Technion. Their performance score was the highest in the competition in recent years.
The task of the competition is to design and fine-tune a search engine to be tested on a newswire collection (TREC ROBUST-04). The winning team in the competition would be the one to design a search engine which would achieve the best MAP score.
As a training set, the teams are given a collection of 50 queries (out of a total of 249). The articles in the collection are manually ranked according to their relevance to each of the queries beforehand, and their aim is to design a search engine which would match those rankings most accurately. The performance of the teams is evaluated by the course staff using the other 199 queries.
Ariel and Eilam’s search engine was constructed using the Indri toolkit and a Python script which fused rankings induced by optimized RM3 and Okapi-BM25 models and achieved a MAP score of 0.3168 on the test set. The method they used outperformed machine-based fusion models used by other competitors.
For further details we invite you to check out a short presentation which elaborates on their model’s mathematical background as well as two other models submitted by them for the competition.