Subject
16 entries
Information Retrieval
Bookmarks
TART: Task-Aware Retrieval with Instructions
TART (Task-Aware Retrieval with Instructions) introduces BERRI, a dataset of ~40 retrieval tasks annotated with human-written task instructions, and trains a multi-task retrieval system that adapts its behavior based on explicit instructions. TART outperforms much larger models on BEIR by understanding the user's intent rather than just matching query-document similarity.
ART: Questions Are All You Need to Train a Dense Passage Retriever
ART (Autoencoding-based Retriever Training) trains dense retrievers without labeled question-passage pairs — only questions and an unpaired document collection. This removes the main bottleneck for deploying dense retrieval in new domains where annotation is expensive.
Haystack: Question Answering at Scale
Haystack by deepset is an open-source NLP framework for building question-answering and search systems at scale — Retriever-Reader pipelines over large document corpora. An early entrant in what became the RAG ecosystem.
Metarank: ML-Powered Ranking Engine
Metarank is an open-source ML-powered ranking engine — takes user feedback signals (clicks, purchases, bookmarks) and trains a Learn-to-Rank model to personalize product listings and search results. Low-code alternative to building a custom LTR pipeline.
LambdaMART In Depth
An in-depth technical explainer on LambdaMART — the gradient boosted tree algorithm for learning-to-rank that underlies most production search and recommendation systems. Explains lambda values, pairwise swapping, and DCG optimization in a way that builds genuine intuition.
Internet Search Tips
Gwern Branwen's guide to effective internet search — covering advanced operators, specialized databases, tracking down primary sources, and the general epistemics of knowing when you've actually found what you're looking for. One of the most useful practical research guides online.
Using Machine Learning to Rank Search Results (Part 2)
A practical tutorial on applying machine learning to re-rank search results — part 2 of a series covering feature engineering, training data construction, and evaluation for LTR in production search. Hands-on complement to the more theoretical LTR literature.
Test Driving Elasticsearch Learning to Rank with a Linear Model
OpenSource Connections' hands-on tutorial for the Elasticsearch Learning to Rank plugin with a linear model — walks through feature logging, model training, and deployment. The entry point for adding ML-powered ranking to an existing Elasticsearch stack.
Solr-LTR: From Zero to Learning to Rank in Apache Solr
A practical guide and reference implementation for setting up learning-to-rank in Apache Solr from scratch — covers the Solr LTR plugin, feature extraction, model training, and deployment. Complement to the Elasticsearch LTR tutorials for Solr shops.
LTR with Bees: Learning to Rank in Apache Solr
Christine Poerschke's tutorial using a bee-related dataset to demonstrate learning-to-rank in Apache Solr — end-to-end example from feature extraction through model training to serving. Concrete and dataset-grounded, unlike most abstract LTR tutorials.
Learning to Rank with Vespa
Thiago Martins' tutorial on learning-to-rank with Vespa for text search — covers how Vespa's native ML integration makes LTR simpler than Elasticsearch/Solr plugins, with models evaluated inside the search engine. Vespa's approach to embedding LTR natively.
Learning to Rank 101: Linear Models
OpenSource Connections' foundational explainer on linear models for learning-to-rank — the first step before gradient boosted trees. Covers feature engineering and the intuition for why linear LTR models are both a useful starting point and a useful baseline.
Is Your Infrastructure Ready for Learning to Rank?
OpenSource Connections' guide to the infrastructure required before you can deploy learning-to-rank — covering the judgment lists, feature logging pipelines, training data management, and model versioning that most LTR tutorials skip. The operational reality of LTR in production.
Intuitive Explanation of Learning to Rank: RankNet, LambdaRank, and LambdaMART
Nikhil Dandekar's intuitive explainer on the evolution from pointwise to pairwise to listwise learning-to-rank, covering RankNet, LambdaRank, and LambdaMART. One of the best conceptual introductions to how modern search ranking algorithms actually work.
What is TF-IDF? The 10 Minute Guide
A 10-minute introduction to TF-IDF — the classic term-weighting scheme that balances how often a word appears in a document against how rare it is across the corpus. Still one of the most useful baselines in text analysis despite being decades old.
Recommender Systems
Wikipedia's overview of recommender systems — collaborative filtering, content-based filtering, and hybrid approaches. Saved in 2012 when Netflix Prize aftermath and the rise of personalization made recommendation algorithms a hot research area.