LLM

Reframing Instructional Prompts to GPTk's Language

ACL 2022 Findings paper showing that manually reframing instructional prompts — decomposing tasks, itemizing steps, adding positive examples — yields 6–12% performance gains on GPT-2 and GPT-3. The key insight is that models respond better to concrete, step-by-step instructions than to long abstract descriptions.

PaLM: Scaling Language Modeling with Pathways

PaLM is Google's 540B parameter language model trained across 6144 TPU v4 chips using the Pathways distributed training system. It achieved breakthrough performance on multi-step reasoning and BIG-Bench, and documented discontinuous capability gains at scale — capabilities that emerged suddenly with more compute.

Editing Models with Task Arithmetic

Task Arithmetic shows that fine-tuning deltas (task vectors) can be added, subtracted, and combined arithmetically to edit model behavior — without any additional training. Negation forgets a task; addition combines capabilities; analogical transfer works across tasks.

Agent Super Spy: LLM observability for local agent development

Local LLM proxy stack that gives you full visibility into what your AI agents actually do. Dual-layer observability via OpenTelemetry traces and raw HTTP capture.

Remind: generalized memory consolidation for AI agents

Remind consolidates raw AI experiences into generalized concepts stored in a knowledge graph, rather than dumping everything into a vector database. The distinction matters: instead of retrieving raw transcripts, it surfaces patterns like 'user prefers statically typed languages.'

Exploitation-Validator: LLM sub-agent vulnerability analysis

exploitation-validator uses LLM sub-agents to find and validate code vulnerabilities through a five-stage pipeline that explicitly cross-checks findings against actual source code to eliminate hallucinations before generating PoC exploits.

Linear Representations and Superposition in LLMs

A clear explainer on how LLMs encode thousands of features in relatively small embedding spaces via superposition — the Johnson-Lindenstrauss lemma applied to neural representations. Good primer on the theoretical foundations behind SAE-based interpretability.

LLM Council: multi-model deliberation as a Claude Code plugin

LLM Council is a Claude Code plugin that runs a 3-phase multi-model deliberation on any query: parallel responses, cross-model ranking, chairman synthesis. Built by DAIR.AI on Fireworks AI inference.

LangExtract: Gemini-powered structured information extraction

LangExtract is Google's open-source Python library for structured information extraction from unstructured text using Gemini's controlled generation. Maps every extracted element back to exact character positions in source text.

Context Lens: see what's filling your context window

Context Lens is a local LLM API proxy that shows what's filling your context window — system prompts vs. tool definitions vs. conversation history vs. tool results. Answers the question every developer asks: why is this session so expensive?

Claude Code: connecting to local models when quota runs out

A practical guide to routing Claude Code to a local LLM when Anthropic quota runs out — using Ollama or similar tools as a fallback so work doesn't stop mid-session. Cheap-plan users will relate.

Sherlock: real-time LLM API traffic interceptor and dashboard

Sherlock intercepts LLM API traffic and shows token usage, costs, and context window consumption in a real-time terminal dashboard. Debug prompts and track costs across AI development sessions without adding SDK instrumentation.

Documentor: documentation aggregator for AI assistants

Documentor is a tool for aggregating external documentation into a format optimized for LLM consumption — crawling docs sites, building search indexes, and storing structured outputs that AI assistants can query during development work.

Dexto: intelligence layer for AI agents

Dexto is an open-source AI agent orchestration platform that adds stateful memory, tool management, and error recovery to LLMs — supporting 50+ models, 30+ MCP tools, and deployment across web UI, CLI, REST API, and Discord. A production-oriented agent infrastructure layer.

TOON: Token-Oriented Object Notation

TOON (Token-Oriented Object Notation) is a compact data format for LLMs — ~40% fewer tokens than JSON while remaining human-readable. Combines YAML indentation with CSV-style tabular layouts, with TypeScript SDKs and CLI tools.

Vibe-DSPy: Natural Language to DSPy Signatures

Vibe-DSPy converts natural language descriptions into DSPy signatures automatically, with iterative refinement from feedback. Takes the vibe-coding approach to generating structured LLM pipeline components.

GPU Calculator for LLM Inference

An interactive calculator for estimating GPU requirements for LLM inference — input model size, quantization, batch size, and context length to get memory and throughput estimates. Practical first tool when planning self-hosted or cloud inference deployments.

Hacker News Summary by betacat: AI-Powered HN Digest

An AI-powered Hacker News summarizer that uses ChatGPT to generate daily digests of top HN stories and discussions. Useful for keeping up with HN signal without spending time in the comment threads.

CodeGPT: BYOK AI Coding Agents

CodeGPT is a BYOK (Bring Your Own Key) AI coding assistant for VS Code and JetBrains IDEs — you supply your own API keys for Claude, Gemini, or others, keeping code under your control. Differentiates on transparency and model flexibility over GitHub Copilot.

Advanced AI Agents Course (DAIR.AI)

DAIR.AI's Advanced AI Agents course covers sophisticated agentic patterns — prompt chaining, routing, parallelization, multi-agent architectures, evaluator-optimizer patterns, and deployment on Google Cloud Run. 38 lessons, 4.5 hours, taught by Elvis Saravia.

The Future Of Reasoning

A YouTube video titled 'The Future Of Reasoning' — content unavailable for fetching, but likely covers the state and trajectory of machine reasoning capabilities in AI systems.

Agent-Honeypot: LLM vs. LLM Adversarial Testing

Agent-Honeypot is an automated red-teaming platform where an attacker LLM generates adversarial prompts against a defender LLM — testing alignment safeguards via authority claims, emergency framing, and incremental requests. A systematic way to stress-test LLM safety before deployment.

Reducing LLM Deception with Self-Other Overlap Fine-Tuning

Self-Other Overlap (SOO) fine-tuning reduces deceptive behavior in LLMs by aligning internal activations for self-referential and other-referential prompts — cutting deceptive responses from 73% to 17% on Mistral-7B with minimal capability loss. A representation-level approach rather than behavioral supervision.

Awesome LLM Judges: curated research on LLM evaluation systems

Curated collection of research on LLM-as-a-judge evaluation systems — covering ensemble methods, fine-tuned judge models, hallucination detection, generative reward models, and safety moderation. Maintained by Haize Labs alongside their Verdict library.

Activation Steering Notebook (concentration_notebooks)

A Jupyter notebook exploring activation steering in neural networks from the concentration_notebooks repo. Saved as a reference in the mechanistic interpretability and model steering space.

monitors4codegen — Monitor-Guided Decoding

Monitor-Guided Decoding uses LSP (Language Server Protocol) static analysis as a 'monitor' during code LM generation to enforce semantic validity — identifiers must exist, types must match. NeurIPS 2023 paper from Microsoft with the multispy Python library for building LSP-backed code gen applications.

Kiln Fine-Tuning LLM Models Guide

Kiln's practical fine-tuning guide walks through generating synthetic training data (the 'Ladder' strategy: 10 manual → 30 LLM → 1000 synthetic) and running training jobs on OpenAI, Fireworks, and Unsloth. Emphasizes data quality over cost during generation.

annotateai

annotateai is a Python package that uses LLMs to automatically annotate research papers — highlighting key claims, methodology sections, results, and limitations. Useful for quickly mapping what a paper is about before reading it in full.

Virtual Personas for Language Models via an Anthology of Backstories

BAIR research on giving LLMs stable virtual personas using an 'anthology of backstories' — a collection of diverse biographical narratives that, combined, produce consistent and controllable personality profiles. Relevant for social simulation, synthetic data generation, and studying opinion dynamics.

Autonomous Hacker: LLM-Powered Security Research Agent

The autonomous-hacker module in R3DRUN3's sploitcraft demonstrates using LLMs as autonomous security research agents — scanning, identifying vulnerabilities, and generating exploits with minimal human input. An educational proof of concept for LLM-assisted offensive security.

Use Prolog to Improve LLM Reasoning

A proposal for using Prolog as a symbolic reasoning layer alongside LLMs — offloading formal logical inference to Prolog while the LLM handles natural language parsing and generation. A neuro-symbolic hybrid that addresses LLMs' known weakness at strict logical deduction.

TensorZero

TensorZero is a self-hosted LLMOps platform that unifies gateway, observability, evaluation, optimization, and A/B testing in one Rust-built system. The feedback loop: production inference data flows back into prompt and model improvement automatically.

L1B3RT45: AI System Prompts Collection

L1B3RT45 is a GitHub repository collecting leaked and reverse-engineered system prompts from major AI assistants (Claude, ChatGPT, Mistral, Zamba2). With 18.5k stars, it documents the operational instructions guiding commercial AI models, useful for prompt engineering research and AI transparency.

Cohere Prompt Tuner: Automated Prompt Optimization

Cohere's Prompt Tuner automatically improves prompts for their models by testing variations and selecting what performs best — prompt optimization as a first-class product feature rather than a manual art.

ai-llm-agent-solver: Autonomous Gandalf Challenge Solver

An LLM-powered agent that autonomously solves the Gandalf AI challenge — a prompt injection security game where you try to extract a secret password from a guarded AI. Uses OpenAI API and agent-based reasoning.

DSPy Self-Discover Framework

A DSPy + Groq-powered framework that takes natural language task descriptions and generates executable Python code to solve them — automated problem-solving through AI-driven reasoning and code generation. Now archived.

TravelPlanner: A Benchmark for Real-World Planning with Language Agents

TravelPlanner is a benchmark for evaluating LLM planning capabilities in complex real-world scenarios — GPT-4 scored 0.6% on the full benchmark, revealing that even the best LLMs struggle with multi-constraint sequential planning. A sobering check on agentic AI ambitions.

jailbreak_llms: CCS'24 Jailbreak Prompt Dataset

A dataset of 15,140 ChatGPT prompts including 1,405 jailbreak prompts collected from Reddit, Discord, and open-source datasets — published at CCS 2024. The most comprehensive public collection of real-world jailbreak attempts against LLMs.

How I Studied for My Ham Radio General Exam

Simon Willison documents how he used LLMs to study for the ham radio General license exam — using Claude to explain questions, generate practice problems, and fill in knowledge gaps. A practical case study in LLM-assisted learning for a technical domain.

Inspectus: LLM Analytics and Visualization

Inspectus is labml.ai's tool for visualizing LLM internals — attention maps, token distributions, and model analytics rendered as interactive visualizations in Jupyter notebooks. Makes transformer attention patterns inspectable without custom code.

Eidolon: open-source agent service framework

Eidolon is an open-source agent service framework that treats AI agents as first-class services with defined APIs, not just Python scripts — enabling agents to communicate with each other and be composed into larger systems. Targets production deployment of multi-agent architectures.

Memoria: structured memory management for AI agents

Memoria is a memory manager for AI agents that routes information to appropriate memory types (episodic, semantic, procedural) to enable dynamic, context-aware responses. An attempt to implement a structured cognitive architecture for LLM memory.

iauto: Low-Code LLM + RPA Automation Engine

iauto is a low-code automation engine that combines LLMs with RPA tools (Appium, Playwright) in YAML-defined workflows. Bridges AI reasoning with traditional robotic process automation for adaptive task execution.

Diff History for Neural Language Agents

Diff history applies the Unix diff command to sequential agent observations, replacing full text states with change deltas. Dramatically reduces context length while preserving critical information — small models tuned with diff history matched SOTA on NetHack with 1800x fewer training examples.

Batch classification with Instructor

An Instructor example showing how to classify multiple items in a single batched LLM call using Pydantic schemas — more efficient than sequential one-at-a-time classification, and type-safe by construction.

DSPy: Compiling Declarative Language Model Calls (SBTB23 talk)

Omar Khattab's ScalaBytesConf 2023 talk introducing DSPy — the core presentation that launched the framework into wider attention. Covers the compile-don't-prompt philosophy and demonstrates self-improving LLM pipelines.

WhisperFusion: real-time voice conversations with AI

WhisperFusion is Collabora's pipeline for real-time voice conversations with AI — combining WhisperLive (real-time STT) with WhisperSpeech (TTS) and an LLM to create a fully local end-to-end spoken AI assistant.

Fine-tuning language models: a practical overview

A clear introductory overview of LLM fine-tuning from the GenAI Guidebook — covers why and when to fine-tune, the mechanics of weight updates, and the major techniques including LoRA and RLHF.

txtai: All-in-One Embeddings Database

txtai is an all-in-one open-source embeddings database combining semantic search, LLM orchestration, and language model workflows. Positions itself as the engine underneath an AI application rather than a standalone vector database.

AirLLM: 70B LLMs on a 4GB GPU

AirLLM runs 70B LLMs on a single 4GB GPU and 405B Llama3.1 on 8GB VRAM — without quantization, distillation, or pruning. It optimizes inference memory usage itself rather than compressing the model, preserving full model quality on consumer hardware.

TokenCost: LLM API Cost Calculator

TokenCost is a Python library from AgentOps that counts tokens and calculates USD costs for 400+ LLM models before making API calls. Keeps a live-updated pricing database so your cost estimates don't go stale when providers update pricing.

Prompt Lookup Decoding: Speculative Decoding without a Draft Model

Prompt lookup decoding replaces the draft model in speculative decoding with simple n-gram string matching against the prompt itself — achieving 2.4x speedup on summarization and QA with zero quality loss. Works whenever output heavily references the input, which is most practical LLM tasks.

Data Exfiltration from Writer.com via Indirect Prompt Injection

PromptArmor and Kai Greshake demonstrate data exfiltration from Writer.com via indirect prompt injection — where malicious content in a document the AI assistant processes causes it to leak user data. A concrete case study of the class of attack affecting all document-processing LLM applications.

Braintrust Data: LLM Evaluation Platform

Braintrust is an AI evaluation platform for logging, scoring, and comparing LLM outputs across experiments and prompt versions. Addresses the core LLMOps problem: how do you know if your prompt change made things better or worse?

Lobe Chat: Open-Source High-Performance Chat Framework

Lobe Chat is a high-performance open-source ChatGPT framework with speech synthesis, multimodal support, and an extensible plugin system — one-click self-deployable to Vercel. More production-ready and feature-complete than basic ChatGPT clones, with an active plugin ecosystem.

Comparing RAG Copilots: OpenAI, Anthropic, Perplexity, and More

Graphlit's comparison of RAG-powered copilots from OpenAI, Anthropic, Perplexity, and others against unstructured data retrieval tasks. Useful late-2023 benchmark of the RAG quality landscape across providers before the tooling matured.

Danswer: Natural Language Q&A Over Private Sources

Danswer (now Onyx) is an open-source enterprise Q&A system that connects to Slack, GitHub, Confluence, and other internal tools to answer natural language questions over private knowledge. Self-hosted with strong access-control enforcement — a production-ready alternative to building RAG from scratch.

ChunkViz: Visualize Text Splitting for RAG Pipelines

ChunkViz is a visual tool for comparing different text splitting strategies — see exactly how your documents get chunked with different chunk sizes, overlap settings, and splitting methods. Essential for debugging RAG pipelines where chunking quality directly affects retrieval quality.

Ava: All-in-One Desktop App for Running LLMs Locally

Ava is an all-in-one desktop app for running LLMs locally — chat, image generation, and model management in a single native application. An alternative to LM Studio targeting users who want everything bundled together without CLI setup.

CopilotKit: In-App AI Chatbots and AI Textareas for React

CopilotKit is a React library for embedding AI chatbots and AI-powered text areas into web apps — with the app's own context injected automatically. Makes it straightforward to add a context-aware AI assistant to an existing React app without building the RAG layer yourself.

SuperDuperDB: Bring AI to Your Database

SuperDuperDB integrates AI models and APIs directly with existing databases — train, manage, and query models where your data already lives rather than moving data to a separate vector database. A database-native alternative to building a separate AI data pipeline.

Bulktag: Batch Image Tagging with GPT-4 Vision

Bulktag uses GPT-4 Vision to batch-tag images automatically — send a folder of images, get back descriptive tags for each. An early practical application of GPT-4V for metadata generation and image library organization.

LLM Visualization: Interactive 3D Transformer Walkthrough

An interactive 3D visualization of how LLMs work — walking through the transformer architecture token by token, layer by layer, with actual weight animations. The clearest visual explanation of attention, embeddings, and feedforward layers available.

LLM Course: Roadmaps and Notebooks for Large Language Models

A comprehensive GitHub repository by Maxime Labonne with roadmaps, notebooks, and articles for learning about LLMs — from fundamentals through fine-tuning and deployment. One of the most starred LLM learning resources, covering the full practitioner pipeline.

The Architecture of Today's LLM Applications

GitHub's overview of LLM application architecture patterns as of late 2023 — the standard components (inference, context, orchestration, state, tools) and how they fit together. A useful snapshot of the consensus architecture before it fragmented into frameworks.

Outlines: Guided Text Generation

Outlines is a Python library for guided LLM text generation — constraining model outputs to match JSON schemas, regex patterns, or context-free grammars. It was one of the first production-quality structured generation libraries before OpenAI's own structured outputs feature.

OpenGPTs: Open-Source Custom AI Assistants

OpenGPTs is LangChain's open-source alternative to OpenAI's GPT builder — create custom AI assistants with configurable tools, instructions, and memory backends. Released the same week as OpenAI's GPT Store announcement.

LLM Verified with Monte Carlo Tree Search

A research project using Monte Carlo Tree Search to guide and verify LLM code generation — MCTS explores the tree of possible completions and selects branches where generated code actually passes verification checks. An early example of search-augmented LLM reasoning.

QLoRA Minimal: Fine-tuning Notebook

A minimal Jupyter notebook demonstrating QLoRA fine-tuning — strips away framework boilerplate to show the core mechanics of 4-bit quantized LoRA training. Good reference for understanding what QLoRA actually does at the implementation level.

The Next Era of AI: GPT-4

A HackerNoon overview of GPT-4's architectural advances over GPT-3, covering multimodal capabilities, improved reasoning, and the scaling leap that made it qualitatively different. Useful historical context for understanding what made GPT-4 a step-change rather than an incremental improvement.

RAG at Planet Scale

Arcus describes their multi-tiered RAG approach for handling massive external data corpora at planet scale — one of the largest RAG deployments of 2023. The key innovation is tiered retrieval that narrows the candidate pool progressively rather than searching the full index directly.

LLMs Confabulate, Not Hallucinate

A terminological argument that LLMs "confabulate" rather than "hallucinate" — the distinction matters because confabulation (confidently filling gaps with plausible-sounding fabrication) implies a specific mechanism, while hallucination implies randomness. Getting the mechanism right leads to better mitigations.

Contextually Splitting Documents — Neum AI

Neum AI introduces context-aware document splitting that improves RAG retrieval quality on structured documents like SEC filings and contracts — by splitting along semantic boundaries rather than fixed character counts. A practical improvement to the chunking step most RAG tutorials treat as an afterthought.

The Rise and Potential of LLM-Based Agents: A Survey

An 86-page survey paper by Zhiheng Xi et al. comprehensively mapping the architecture, capabilities, and applications of LLM-based agents — establishing the planning/memory/tool-use framework that became the standard way to think about agent components. The field's foundational survey document.

RAG Is More Than Just Embedding Search

Jason Liu's influential post arguing that RAG systems need more than vector similarity search — covering query understanding, routing, reranking, and structured extraction as the layers that separate working RAG from production-grade RAG. Written for the Instructor library blog.

PDFTriage: Question Answering over Long, Structured Documents

PDFTriage is a method for question answering over long, structured PDF documents that uses the document's structure (pages, sections, tables of contents) as a triage layer before retrieval — letting the LLM navigate the document intelligently rather than relying on flat embedding search.

LLM Agent Paper List

A curated paper list accompanying the 86-page 'Rise and Potential of LLM-Based Agents' survey — organized by agent component (planning, memory, tool use, action) and application domain. The most comprehensive catalog of LLM agent research from September 2023.

Generative AI's Act Two — Sequoia Capital

Sequoia's 'Generative AI Act Two' argues that one year after the ChatGPT moment, the real work begins: moving from demos and experiments to applications that deliver genuine business value. A VC thesis on where the durable opportunities in the generative AI wave actually lie.

Arxiv Paper 2309.08963

An arxiv paper from September 2023 saved alongside the LLM agent survey and RAG papers from the same day — likely in the LLM reasoning, agents, or retrieval space based on context. Content unavailable from the bookmark.

Optimal Chunk Size for Large Document Summarization

Vectify AI introduces a method to automatically determine the optimal chunk size for large document summarization with LLMs — rather than fixed-size chunking, the approach finds the chunk granularity that maximizes summary quality for a given document type.

Making Large Language Models Work For You

Simon Willison's WordCamp 2023 keynote on making LLMs work for you — a practical, grounded introduction covering what LLMs actually are, their real limitations, and the specific use cases where they're genuinely useful today. One of the better practitioner introductions from 2023.

Graft: Enterprise AI from Playground to Production

Graft is an enterprise AI company helping businesses move from LLM playground to production — included in Weekend Fund's Enterprise AI Stack. A signal of the 2023 market for companies building the deployment and operations layer on top of frontier models.

Graph of Thoughts: Solving Elaborate Problems with LLMs

Graph of Thoughts (GoT) extends chain-of-thought and tree-of-thought prompting to arbitrary graph structures — letting LLM reasoning steps combine, branch, and loop rather than just proceeding linearly. More expressive than CoT or ToT for complex multi-step problems.

Compute Watch: LLM Compute Costs and GPU Availability Tracker

Compute Watch is a tracker for LLM compute costs and GPU availability — benchmarking inference costs across providers and tracking H100/A100 spot availability. Useful for anyone making infrastructure decisions around model serving costs.

Avenging Polanyi's Revenge: LLM Approximate Omniscience in Planning

A talk titled 'Avenging Polanyi's Revenge' arguing that LLMs' approximate omniscience changes planning — they've absorbed tacit knowledge that previously required human experts, enabling a different category of automated planning than rule-based systems allowed.

Reasoning or Reciting? LLM Capabilities via Counterfactual Tasks

An arxiv paper testing whether LLMs are reasoning or reciting — using counterfactual tasks (modified versions of standard benchmarks) to distinguish genuine reasoning from pattern-matching on training data. Finds LLMs show sharp performance drops on counterfactual variants, suggesting heavy reliance on memorization.

Patterns for Building LLM-based Systems & Products

Eugene Yan's comprehensive guide to patterns for building LLM-based systems — covering evals, RAG, fine-tuning, caching, guardrails, defensive UX, and user feedback collection. One of the most referenced practical engineering posts of 2023.

llm-reasoners: Advanced LLM Reasoning Algorithms

llm-reasoners is a library for advanced LLM reasoning algorithms — implementing Tree of Thoughts, RAP (Reasoning via Planning), and other structured reasoning approaches over standard chain-of-thought. Useful for research into how to get LLMs to reason more reliably on complex tasks.

Autoblocks: Debug and Monitor Generative AI Features

Autoblocks is a developer-centric monitoring and debugging platform for LLM features — trace LLM calls, replay sessions, catch regressions, and improve model behavior with production data. Part of the 2023 LLMOps tooling wave.

Gorilla CLI: LLMs for Your CLI

Gorilla CLI is an LLM-powered tool that converts natural language commands into correct shell commands — type what you want to do, get the right command. Built by the team behind the Gorilla LLM model for API function calling.

Experimental Prompting DSL: Origin of Instructor

Jason Liu's experimental prompting DSL for structured LLM outputs via OpenAI function calling — an early version of what became the Instructor library. Shows the origin of the pydantic-based approach to reliable structured extraction from LLMs.

Build a Support Bot with Supabase, OpenAI, and Windmill

Windmill's tutorial on building a Discord/Slack support bot using Supabase for vector storage and OpenAI for embeddings and generation — a worked RAG example for team documentation Q&A. Shows how to wire up the standard stack (embed, store, retrieve, generate) for a specific product use case.

Beware of Unreliable Data in Model Evaluation

Cleanlab's case study showing that noisy test data leads to suboptimal prompt selection for LLMs — you can choose the wrong prompt because your evaluation data contains labeling errors. A practical warning about data quality in LLM evaluation pipelines.

LLM Powered Autonomous Agents

Lilian Weng's survey post on LLM-powered autonomous agents — covering the planning, memory, and tool use components that compose into agent architectures. One of the most cited and comprehensive overviews of the agent design space from mid-2023.

Mithril Security: Confidential AI Inference with BlindAI

Mithril Security builds BlindAI — an open-source AI inference solution using Trusted Execution Environments (TEEs) so you can run AI on sensitive data without exposing it to the model provider. Addresses the tension between AI utility and data privacy.

AnythingLLM: Documents to Chatbot

AnythingLLM is a full-stack open-source application for turning any document collection into a chatbot — with a polished UI, workspace management, and multi-model support. The self-hosted alternative to ChatGPT Enterprise for teams wanting document Q&A over their own files.

OpenChat: LLM Custom Chatbot Console

OpenChat is an open-source LLM chatbot console with document ingestion and custom bot creation — an early self-hosted alternative to ChatGPT for teams wanting to deploy domain-specific chatbots over their own data.

ChatGPT + Google Drive with LangChain

A tutorial for connecting ChatGPT to Google Drive using LangChain and Python in 30 lines of code — an early canonical example of the document Q&A pattern using LangChain's document loaders and retrieval chain.

Landmark Attention: Random-Access Infinite Context

Landmark Attention paper extending LLaMA 7B to 32k token context by using landmark tokens to represent blocks of input, enabling attention-based retrieval of relevant blocks rather than attending over all tokens. Achieves near-GPT-4 context length through architectural change rather than longer pretraining.

Democratizing AI with Open-Source Language Models

LWN's coverage of the democratizing AI discussion around open-source language models — examining the tension between open-source access enabling innovation and the risks of unguarded deployment. A useful 2023 snapshot of the open vs. closed AI debate from the Linux community perspective.

All the Hard Stuff Nobody Talks About When Building with LLMs

Honeycomb's post-mortem on building their LLM-powered Query Assistant — the engineering challenges they didn't expect, including output validation, latency at the tail, prompt brittleness, and user trust. Unusually honest practitioner account from a team that shipped an LLM feature to production.

Per Prompt: Weekly LLM and AI Digest

Per Prompt is a weekly digest of interesting news, tools, and developments in LLMs, GPT, and AI — a curation newsletter for staying current without following the firehose. Typical of the newsletter wave that emerged during 2023's rapid AI news cycle.

OpenLLaMA: Open Reproduction of LLaMA

OpenLLaMA is an open-source reproduction of Meta's LLaMA model trained on the RedPajama dataset, released under permissive Apache 2.0 licenses. It was one of the first serious attempts to produce a fully open LLaMA-quality model that anyone could use commercially.

Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings

Chatbot Arena is LMSYS's crowdsourced LLM benchmark where users rate anonymous head-to-head model comparisons using Elo ratings — producing rankings that reflect real user preferences rather than task-specific metrics. The Elo format became the dominant live benchmark for tracking which frontier model is currently best.

Timeline of Transformer Models and Large Language Models

A visual timeline of transformer models and large language models from 2017 through the present, mapping the lineage of architectures from the original Attention Is All You Need paper through GPT-4 and beyond. A useful reference for understanding model genealogy.

MLC-LLM: LLMs on Every Device

MLC-LLM enables running large language models natively on any device — laptops, phones, browsers — without a server, by compiling models through the Apache TVM compiler stack. One of the earliest frameworks to make on-device LLM inference practical across diverse hardware.

JARVIS / HuggingGPT: LLM as AI Model Orchestrator

Microsoft JARVIS (also published as HuggingGPT) uses ChatGPT as a task planner that routes subtasks to specialized Hugging Face models — an early demonstration that LLMs could orchestrate other AI models as tools. A prototype of the multi-model agent pattern.

Self-Refine: Iterative LLM Output Improvement via Self-Feedback

Self-Refine is a framework where LLMs generate feedback on their own outputs and iteratively refine them — no human feedback, no gradient updates. Shows that a single LLM can be its own critic and improve outputs across diverse tasks.

Horace: Self-Hosted LLM Chatbot with ChatGPT Plugin Support

Horace is an open-source LLM chatbot server that supports ChatGPT plugins — letting you run a self-hosted chatbot that can use the growing ecosystem of ChatGPT plugin tools. An early attempt to bring plugin-based tool use outside of OpenAI's walled garden.

ChatArena: Multi-Agent Language Game Environments for LLMs

ChatArena provides multi-agent game environments for LLMs — structured settings where multiple LLMs interact, debate, negotiate, or play games with defined rules and roles. A research framework for studying emergent multi-agent behavior.

Alpaca LoRA: Instruct-Tuning LLaMA on Consumer Hardware

Alpaca LoRA applies Low-Rank Adaptation to fine-tune LLaMA on consumer GPUs — making instruction-following fine-tuning accessible without data center hardware. Part of the March 2023 democratization wave that showed open LLM development didn't require massive compute.

Prompt Engineering: Lilian Weng's Comprehensive Survey

Lilian Weng's canonical reference on prompt engineering techniques — zero-shot, few-shot, chain-of-thought, self-consistency, tree-of-thoughts, and more — grounded in research papers. Still the most comprehensive single-author survey of the space.

gpt-repository-loader: Pack a Repo for GPT

gpt-repository-loader concatenates a git repository into a single text file formatted for GPT ingestion — solving the context-packing problem for code understanding before embeddings-based retrieval became standard.

LlamaIndex: Composable Indices and Query Decomposition

A LlamaIndex notebook demonstrating composable indices with query decomposition on city data — showing how to break complex queries into sub-queries across multiple document indices and synthesize the results. An early tutorial on the multi-hop retrieval patterns LlamaIndex specialized in.

Guardrails: Adding Structure and Validation to LLM Outputs

Guardrails adds structural and semantic validation to LLM outputs — defining schemas for what the model should return and automatically retrying or correcting when outputs don't conform. An early solution to LLM output reliability before JSON mode and structured outputs became standard.

Dalai: Run LLaMA Locally with One Command

Dalai is a one-command installer for running LLaMA models locally — npm install to set up, then query models via CLI or socket server. One of the first tools to make local LLM inference accessible to developers without ML expertise.

Generate Rather Than Retrieve: Large Language Models Are Strong Context Generators

Yu et al. (2022) show that prompting an LLM to generate its own background context before answering a question (GenRead) outperforms retrieval-based approaches on several knowledge-intensive NLP benchmarks. The result challenges the assumption that retrieval is necessary for grounding LLM outputs.

From Prompt Alchemy to Prompt Engineering: Analytic Augmentation

An essay arguing for 'analytic augmentation' — using structured logical reasoning and philosophical method to improve LLM prompts, moving from intuitive prompt tweaking (alchemy) to principled prompt design (engineering). Connects formal reasoning frameworks to practical prompt construction.

Colossal-AI: Open-Source ChatGPT Training Replication

Colossal-AI released an open-source implementation of the ChatGPT training process (SFT + RLHF) that runs on a single GPU with 1.6GB memory — 7.73x faster than naive implementations. Made the ChatGPT training pipeline accessible to researchers without multi-GPU clusters.

Llama Hub: LlamaIndex Data Connector Marketplace

Llama Hub is the LlamaIndex community marketplace for data loaders — connectors that pull data from Notion, Slack, GitHub, databases, APIs, and more into LlamaIndex for RAG pipelines. The npm registry equivalent for LLM data connectors.

Promptable.js: TypeScript Library for LLM Apps

Promptable.js is an early TypeScript library for building LLM apps with Prompt, Search, Chain, and Trace primitives — a TypeScript analogue to LangChain that appeared in February 2023 before LangChain's own TypeScript support matured. Historically significant as one of the first LLM frameworks for TypeScript developers.

Compute-Optimal LLMs: Chinchilla Scaling Calculator

howmanyparams.com is a calculator for compute-optimal LLM training based on Chinchilla scaling laws — given a compute budget, it tells you the optimal model size and token count. A practical tool for applying the Hoffmann et al. scaling law findings.

Cognosis AI Platform

Cognosis AI's open-source platform repository — an early AI agent platform from 2023. Content not available but saved as a reference to an early-stage AI infrastructure company working on agent platforms before the space became crowded.

Running Large Language Models in Production: Cohere's TIF

Cohere's post on their Transformer Inference Framework (TIF) — covering the systems challenges of serving large language models in production at scale. An early look at how LLM serving differs from traditional ML model serving and what optimizations matter.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis et al. (Facebook AI, 2020) introduce Retrieval-Augmented Generation, a hybrid architecture that combines dense passage retrieval with seq2seq generation to ground language model outputs in a non-parametric knowledge store. RAG defined the template that most production knowledge-grounded LLM systems follow today.

Large Transformer Model Inference Optimization

Lilian Weng's comprehensive survey of transformer inference optimization techniques — covering quantization, distillation, pruning, efficient attention, speculative decoding, and hardware-level optimizations. The definitive reference for the topic as it stood in early 2023.

ChatRWKV: ChatGPT Powered by an RNN

ChatRWKV is a ChatGPT-like chatbot built on the RWKV architecture — a language model that achieves transformer-level performance using an RNN design, enabling constant memory inference regardless of sequence length. A significant architectural alternative to attention-based transformers.

What Building "Copilot for X" Really Takes

An essay from the team behind Codeium on what actually goes into building a 'Copilot for X' product — inference scale, latency budgets, context window management, and the unglamorous infrastructure work. A 2022 reality check on what the AI coding assistant category requires.

Prompt Injection Attacks Against GPT-3

Simon Willison's September 2022 post naming and describing prompt injection attacks against GPT-3 — one of the first clear articulations of the attack class where malicious content in the environment overrides the developer's system prompt. The post that put the term 'prompt injection' into common use.

The Biggest Bottleneck for LLM Startups Is UX

The argument that UX and UI talent was the primary constraint on LLM startups in late 2022 — not model quality or API access, but the ability to build interfaces that made AI capabilities usable. A claim that proved prescient: the best early AI products were product/design wins as much as technical ones.

What Learning Algorithm Is In-Context Learning? Investigations with Linear Models

ICLR 2023 paper showing that transformers trained on in-context learning tasks implicitly implement gradient descent and ridge regression on linear problems, with layers encoding weight vectors and moment matrices. Foundational theoretical work explaining ICL as implicit algorithm execution rather than pure pattern matching.

Program of Thoughts Prompting: Disentangling Computation from Reasoning

Program of Thoughts separates reasoning from computation by having LLMs write executable Python programs rather than performing arithmetic inline, delegating number-crunching to an interpreter. It substantially outperforms chain-of-thought on numerical reasoning benchmarks by eliminating the arithmetic errors that plague prose reasoning chains.

Large Language Models are Human-Level [...] (Nov 2022)

A November 2022 paper claiming LLMs perform at human level on some capability, with a truncated filename that doesn't reveal the specific domain. Likely from the computational social science or NLP annotation literature; the save date coincides with ChatGPT's launch and peak interest in LLM capability claims.

Reward Model Ensembles Help Mitigate Overoptimization

Coste et al. (2022) show that ensembling multiple reward models substantially reduces reward overoptimization during RLHF, where a policy learns to exploit artifacts in a single reward model rather than truly improving. This is a practical mitigation for one of the central failure modes in aligning language models with human preferences.

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs

Sap et al. test LLMs on theory-of-mind tasks and find they rely on spurious correlations and dataset artifacts rather than genuine social reasoning. Models perform well on training distributions but fail on adversarial or decontextualized ToM tests — a gap that matters for social AI applications.

Automatic Prompt Engineer (APE)

Automatic Prompt Engineer (APE) is a research implementation that automatically generates and selects optimal prompts for LLMs using a generate-then-score approach. An early automated prompt optimization method predating DSPy.

Scaling Instruction-Finetuned Language Models

The FLAN-T5 and Flan-PaLM paper (2022) showing that instruction finetuning across 1.8K tasks improves performance on zero-shot, few-shot, and chain-of-thought prompting across multiple model families. Established instruction finetuning as a general-purpose method and released Flan-T5 checkpoints that shaped the open-source LLM ecosystem.

A Complete Introduction to Prompt Engineering

Mihail Eric's comprehensive introduction to prompt engineering for LLMs — covering few-shot prompting, chain-of-thought, instruction tuning, and evaluation. Published in late 2022 when prompt engineering was emerging as a recognized discipline.

Manifest: Prompt Programming with Foundation Models

Manifest is a Python library from Stanford's HazyResearch lab for prompt programming with foundation models — a unified interface across providers with caching, batching, and structured output support. An early formalization of LLM programming patterns before LangChain dominated.

NatBot: GPT-3-Driven Browser Automation

NatBot is an early LLM-driven browser automation bot by Nat Friedman — it takes a natural language task, drives a Chromium browser via Playwright, and completes multi-step web tasks. A 2022 proof-of-concept for what later became the AI browser agent category.

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

BetterBrain: Natural Language to SQL with Schema Awareness

BetterBrain converts natural language to SQL while correctly handling schema constraints — an early 2022 demo of LLM-powered text-to-SQL that understood the database schema rather than generating syntactically-valid but semantically-wrong queries.

Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models

A research paper presenting interactive and visual tools for prompt engineering that let users iteratively adapt large language models to ad-hoc tasks without fine-tuning. It addresses the core usability gap in prompt engineering: the feedback loop between prompt edits and output quality is opaque without tooling that makes the connection visible.

Large Language Models Can Self-Improve

Huang et al. (2022) show that LLMs can bootstrap their own reasoning ability by generating chain-of-thought rationales, filtering with self-consistency majority vote, and finetuning on the high-confidence outputs — no human labels needed. A clean demonstration that LLMs can improve themselves without supervised signal.

Large Language Models Are Human-Level Prompt Engineers

Large Language Models Are Human-Level Prompt Engineers (APE) introduces an automated method for generating and selecting optimal prompts using LLMs themselves, matching or beating human-crafted instructions on 19 of 24 NLP tasks. It reframes prompt engineering as a program search problem, making manual iteration unnecessary.

GPT-3 + Python Interpreter (gpt.py)

A 2022 Replit demo by Sergey Karayev showing GPT-3 armed with a Python interpreter — doing exact arithmetic, making API requests, and answering questions that pure text generation gets wrong. An early, concrete preview of what LLM tool use would look like.

GPT-3 and Arithmetic (MIT LINGO Lab)

MIT LINGO Lab's analysis of GPT-3's arithmetic abilities — probing how and when it succeeds or fails at basic math, and what this reveals about how language models represent numerical reasoning. Relevant to understanding the difference between pattern matching and genuine computation.

OpenAI Codex TodoMVC Challenge

A walkthrough of using OpenAI Codex to build a TodoMVC app from natural language descriptions. Captures the pre-Copilot moment when Codex-powered code generation was the frontier of AI-assisted coding.

Memorizing Transformers

ICLR 2022 spotlight proposing memory-augmented transformers that use approximate k-NN lookup into stored (key, value) pairs at inference time, without weight updates. Scales to 262K token memory with consistent perplexity improvements — an early approach to giving language models dynamic, test-time-updateable knowledge stores.

Modern NLP with Large Language Models (Sinan Ozdemir, Maven)

Sinan Ozdemir's Maven cohort course on modern NLP with GPT-3/4 and BERT — covering information retrieval, multi-task pipelines, and prompt engineering. One of the early structured courses teaching practitioners how to build with LLMs.

GPT-3 Complete-Program Synthesis (Riley Goodside)

Riley Goodside demonstrates complete-program synthesis using GPT-3 via the 'format trick' — combining instruction prompts with contextually informative templates to generate entire programs deterministically. An early exploration of LLMs for structured code generation beyond autocomplete.

Solving Quantitative Reasoning Problems with Language Models (Minerva)

Lewkowycz et al. at Google Research introduce Minerva, a language model pretrained on general text and further trained on technical content that achieves state-of-the-art on quantitative reasoning benchmarks without external tools. It correctly answers nearly a third of undergraduate-level science problems — an early proof that domain-specific pretraining unlocks STEM reasoning at scale.

Language Models Can Teach Themselves to Program Better

Haluptzok, Bowers, and Kalai show that language models can generate their own programming problems and solutions, verify correctness with a Python interpreter, then fine-tune on the verified examples — more than doubling test accuracy. A clean demonstration of self-improvement via external verification.

GPT-3 Long Instruction Following (Riley Goodside)

Riley Goodside demonstrates GPT-3 following a nearly 2,000-character instruction prompt precisely. An early illustration that LLMs can be reliable instruction followers with detailed context, countering the intuition that prompts should be kept short.

Rytr: AI Writing Assistant

Rytr is an AI writing assistant for generating marketing copy, blog posts, and emails using structured templates — one of the early consumer LLM writing tools that predated ChatGPT. The template-driven UX made it accessible to non-technical users before free-form prompting became normal.

How to Play with the GPT-3 Language Model

Simon Willison's practical guide to getting started with GPT-3 — written in June 2022 when most developers still hadn't touched it. One of the clearest early explainers on the OpenAI API and what the model could actually do.

FauxPilot: Open-Source GitHub Copilot Server

FauxPilot is an open-source, self-hosted alternative to GitHub Copilot that runs Salesforce's CodeGen models locally via Triton Inference Server. Built in response to privacy concerns about Copilot sending code to OpenAI's servers.

AlexaTM 20B: Amazon's Few-Shot Language Model

AlexaTM 20B is Amazon's 20B parameter seq2seq language model that outperforms PaLM 540B on one-shot summarization and sets state-of-the-art on multilingual translation — while training on one-fifth of GPT-3's carbon footprint. A strong argument for encoder-decoder architecture in few-shot settings.

DART: Differentiable Prompt Makes Pre-Trained Language Models Better Few-Shot Learners

DART (Differentiable pRompT) trains prompt templates end-to-end via backpropagation, treating prompts as learnable continuous vectors rather than fixed text. It makes small pre-trained language models competitive few-shot learners without scaling to GPT-3 sizes — an important stepping stone between hand-crafted prompts and full fine-tuning.

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Wang, Roberts, Scao et al. (BigScience Architecture Group) conduct a large-scale comparison of model architectures (causal decoder, non-causal decoder, encoder-decoder) and pretraining objectives (autoregressive, masked LM) for zero-shot generalization. The key finding: causal decoders + autoregressive LM win at zero-shot; non-causal decoders + masked LM + multitask fine-tuning win overall.

Multitask Prompted Training Enables Zero-Shot Task Generalization

T0 shows that training a language model on 2000+ diverse human-authored prompts across 170+ NLP tasks dramatically improves zero-shot generalization to unseen tasks. An 11B T0 model outperformed 175B GPT-3 zero-shot — proving prompt diversity during training matters more than raw scale for generalization.

Prompts.ai: Advanced GPT-3 Playground

Prompts.ai was a community-built advanced GPT-3 playground that extended OpenAI's official interface with chaining, templates, and variable injection — an early tool for the prompt engineering community before ChatGPT made LLMs mainstream.

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

Reynolds and McDonell (2021) argue that prompting large language models is better understood as programming than as few-shot learning — the few examples in a prompt aren't training data but rather code that specifies the desired computation. This reframing opens up principled prompt design strategies that beat naive templating.

Transformer Models: An Introduction and Catalog

Xavier Amatriain's catalog of transformer models covers the full landscape of architecture variants and named models as of mid-2022 — encoder-only, decoder-only, encoder-decoder, and multimodal. A useful orientation map for the pre-ChatGPT era of rapid model proliferation.

What Language Model to Train if You Have One Million GPU Hours?

An ablation study by the BigScience group comparing architectural choices and training setups for large multilingual language models targeting 100B+ parameters within a fixed 1M A100 GPU-hour budget. It shows that careful architecture and training setup decisions at the 1.3B scale transfer predictably to larger models, making principled design tractable even at extreme scale.

BLOOM: Open Multilingual Large Language Model

BLOOM is the first open, multilingual large language model trained transparently by a global coalition of AI researchers — 176B parameters, 46 languages, trained on the Jean Zay supercomputer in France. A direct counterpoint to GPT-3's closed access.

YaLM-100B: Yandex 100B Parameter Language Model

Yandex opensourced YaLM-100B, a 100-billion-parameter GPT-like language model trained on Russian and English text. One of the first non-Western frontier-scale language models released openly, notable both for its scale and its Russian-language capabilities.

OPT Training Chronicles

Meta's OPT-175B training logbook — a day-by-day record of hardware failures, instabilities, loss spikes, and interventions during a ~2 month distributed training run. Rare public documentation of what large language model training actually looks like at scale.

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

Switch Transformers (Fedus, Zoph, Shazeer 2021) scales language models to 1.6 trillion parameters using a simplified sparse Mixture of Experts architecture that routes each token to exactly one expert. It's the paper that made sparse MoE practical at scale and laid the architecture foundation for models like Mixtral and GPT-4.

TextSynth Playground

TextSynth is a web playground and API for running large language models efficiently — built by Fabrice Bellard (of ffmpeg and QuickJS fame) using his GGML-based inference engine. In 2022 it was one of the few accessible ways to experiment with GPT-J, GPT-NeoX, and similar open models.

GPT-J: Self-Hosted Open-Source LLM

A guide to running GPT-J, EleutherAI's open-source GPT-3 analog, in Docker — a 2022 snapshot of what self-hosting a large language model looked like before the Llama era democratized this further. Early example of the open-weight model movement.

GPT-Neo: Open Source GPT-3 Scale Language Models

GPT-Neo is EleutherAI's open-source implementation of GPT-style language models at GPT-3 scale — the first serious open attempt to replicate GPT-3's capabilities before open-weight models became common. Historically significant as the origin of the open LLM movement.

Why GPT-3 Matters

Leo Gao's early analysis of why GPT-3 is qualitatively different from prior language models — written in May 2020 when GPT-3 was first announced. One of the clearer technical takes on what the scaling breakthrough meant, from someone who later worked on it.