Subject
8 entries
Survey
Bookmarks
The Rise and Potential of LLM-Based Agents: A Survey
An 86-page survey paper by Zhiheng Xi et al. comprehensively mapping the architecture, capabilities, and applications of LLM-based agents — establishing the planning/memory/tool-use framework that became the standard way to think about agent components. The field's foundational survey document.
Pretrain, Prompt, Predict: NLP Survey
A comprehensive survey of the Pretrain, Prompt, Predict paradigm in NLP — covering prompt engineering, answer engineering, and in-context learning as a unified framework. The academic backbone behind what practitioners call 'prompt engineering'.
Transformer Models: An Introduction and Catalog
Xavier Amatriain's catalog of transformer models covers the full landscape of architecture variants and named models as of mid-2022 — encoder-only, decoder-only, encoder-decoder, and multimodal. A useful orientation map for the pre-ChatGPT era of rapid model proliferation.
Pre-Trained Models: Past, Present and Future
Comprehensive survey of large-scale pre-trained models (PTMs) tracing the evolution from BERT and GPT through four research frontiers: architecture, contextual use, efficiency, and interpretability. Required reading for understanding how self-supervised pre-training became the unified backbone of modern AI.
A Survey on Graph Representation Learning Methods
Khoshraftar and An (York University) provide a comprehensive survey of graph representation learning covering node embeddings, GNNs, and knowledge graph methods, with attention to both spectral and spatial approaches. It's the right starting point for anyone orienting to the GRL landscape — methodical coverage from DeepWalk through GAT.
A Practical Guide to Multi-Objective Reinforcement Learning and Planning
A Springer survey on multi-objective reinforcement learning and planning — covers scalarization, Pareto-based methods, and utility-based approaches for agents that must balance competing rewards. Useful reference for RL research where single-reward framing is inadequate.
What Are the Most Important Statistical Ideas of the Past 50 Years?
Andrew Gelman and Aki Vehtari's 2021 JASA paper enumerating 8 ideas that most changed statistics in the past 50 years — from counterfactual causal inference to bootstrapping to overparameterized models. A rare high-level synthesis by two of the field's most credible voices.
10 Categories of Deep Recommendation Systems
James Le's survey of 10 categories of deep learning-based recommendation systems — from MLP and autoencoder approaches through attention-based and graph neural network methods. A useful taxonomy for understanding how the field moved beyond matrix factorization.