Subject
33 entries
Performance
Bookmarks
Running local LLMs with Claude Code via Unsloth
How to run local LLMs with Claude Code via llama.cpp and Unsloth's GGUF models. The critical fix: Claude Code added an attribution header that breaks KV cache and causes 90% slower inference — must be disabled in settings.json.
less_slow.cpp: Learning High-Performance C++
less_slow.cpp is a benchmarking and teaching repository by Ash Vardanian covering high-performance C++ from SIMD vectorization to CUDA and io_uring — designed to build intuition for performance-oriented design through measurements, not theory. A practical companion to reading architecture manuals.
Docker layer caching in GitHub Actions
Blacksmith's guide to Docker layer caching in GitHub Actions — covering the different cache backends (inline, registry, gha), when each works, and how to get 2x faster builds. Practical guide to a topic where bad defaults waste significant CI time.
Prompt Lookup Decoding: Speculative Decoding without a Draft Model
Prompt lookup decoding replaces the draft model in speculative decoding with simple n-gram string matching against the prompt itself — achieving 2.4x speedup on summarization and QA with zero quality loss. Works whenever output heavily references the input, which is most practical LLM tasks.
Large Transformer Model Inference Optimization
Lilian Weng's comprehensive survey of transformer inference optimization techniques — covering quantization, distillation, pruning, efficient attention, speculative decoding, and hardware-level optimizations. The definitive reference for the topic as it stood in early 2023.
How fast can we perform a forward pass?
A deep technical investigation into the theoretical and practical speed limits of a single transformer forward pass, accounting for compute, memory bandwidth, and hardware constraints. The answer matters for inference costs, latency SLAs, and understanding where efficiency gains are actually possible.
Transformer Inference Arithmetic
Carol Chen (kipply) works through the arithmetic of transformer inference — compute vs. memory bandwidth, KV cache sizing, and how batch size shapes throughput/latency tradeoffs. Essential reference for anyone reasoning about LLM serving costs.
Speeding Up the EVM (Flashbots Research)
Flashbots research on techniques to speed up the Ethereum Virtual Machine — covering parallelism, caching, and other optimizations to increase throughput beyond the EVM's sequential execution model. Relevant to scaling Ethereum L1 without moving to L2.
Scalene: High-Performance Python Profiler
Scalene is a high-performance Python profiler that measures CPU time, GPU time, and memory simultaneously with very low overhead — and attributes memory allocation and copy costs line-by-line. The best profiler for Python if you care about both speed and memory.
OrioleDB: Modern Cloud-Native PostgreSQL Storage Engine
OrioleDB is a PostgreSQL table access method (storage engine) built to solve PostgreSQL's wicked scalability problems — bloat, vacuum overhead, lock contention — by using a copy-on-write B-tree and row-level WAL. Aims to make PostgreSQL cloud-native without forking it.
CMU 15-721: Advanced Database Systems
CMU 15-721 is Andy Pavlo's advanced database systems course covering in-memory databases, query compilation, concurrency control, and storage engines — the internals that most engineers never see. Free lectures, reading list of seminal papers, and a reputation as one of the best systems courses available.
Cleaning Up Your Postgres Database
Crunchy Data's guide to PostgreSQL database maintenance — identifying bloat, reclaiming space with VACUUM, finding unused indexes, and cleaning up dead connections. Practical operations reference for keeping a Postgres database healthy.
django-seal: Queryset Sealing for Django
django-seal lets you mark a QuerySet as 'sealed' so that any lazy evaluation attempt (iterating after the context is closed, triggering N+1 queries) raises an exception. It enforces eager loading discipline at the queryset level, catching ORM performance mistakes in development.
benchm-ml: ML Algorithm Benchmark Comparison
A systematic benchmark of machine learning algorithms across platforms and implementations — comparing gradient boosting, random forests, neural networks, and others on speed and accuracy. One of the best empirical references for choosing between ML tools in 2015, when the xgboost vs sklearn debate was live.
Top Mistakes Developers Make When Using Python for Big Data Analytics
A practical rundown of the top Python performance mistakes for big data workloads — covering generator vs. list comprehension choices, pandas anti-patterns, and when to reach for NumPy. Still relevant since Python's core performance traps haven't changed.
MySQL INSERT Speed Optimization
MySQL 5.0 reference manual section on optimizing INSERT performance — covering bulk inserts, transaction batching, and buffer sizing. The reference for squeezing write throughput out of MySQL when ingestion speed matters.
PoWA — PostgreSQL Workload Analyzer
PoWA (PostgreSQL Workload Analyzer) is a real-time statistics and workload analysis tool for PostgreSQL — collects query statistics, visualizes slow queries, and helps identify index and configuration improvements. A targeted alternative to generic APM tools for Postgres-specific performance work.
MapGraph — GPU-Accelerated Graph Analytics
MapGraph is a high-performance graph analytics framework using GPU acceleration for large-scale graph algorithms. An early example of GPU-accelerated graph processing before NVIDIA's RAPIDS ecosystem made this mainstream.
Getting the Best Performance out of NumPy
Featured recipe from the IPython Cookbook on getting the best performance out of NumPy — covering vectorization, broadcasting, memory layout, and avoiding Python loops. The kind of practical optimization guide that separates slow scientific Python from production-grade numerical code.
Scaling SQL with Redis
David Cramer's post on using Redis to scale SQL databases — covering caching patterns, read replica offloading, and where Redis fits in a stack that can't abandon SQL entirely. Practical patterns from the Disqus/Sentry engineering blog at production scale.
Functional Performance with Core Data Structures — PyData SV 2014
Matthew Rocklin's PyData SV 2014 talk on functional performance with core data structures — showing how functional programming patterns and Python's built-in data structures enable high-performance computation without reaching for C extensions.
Advanced R — Hadley Wickham
Hadley Wickham's Advanced R — the definitive guide to R's unusual object systems, functional programming patterns, environments, and performance profiling. Essential reading for anyone who wants to move from R user to R programmer.
Parakeet: A Faster Python for a Better Tomorrow
Parakeet was a Python JIT compiler targeting NumPy array operations, promising to make numerical Python code run at near-native speeds without rewriting in C or Cython. An early attempt at the problem that Numba later solved more completely.
High Performance at Massive Scale: Lessons Learned at Facebook
Summary of a 2009 Facebook engineering talk on high-performance systems at massive scale — covering their memcached deployment, MySQL sharding, and the operational realities of running at hundreds of millions of users. An early public window into big-company distributed systems practice.
The Buzz and Fuzz on SSD in Hadoop
Hadoopsphere's analysis of SSD adoption in Hadoop clusters — the trade-off between SSD's dramatically faster random I/O and its higher cost per GB compared to spinning disk. An early look at how flash storage would eventually reshape big data infrastructure.
Is There Room for SSDs in the Hadoop Framework?
StorageTuning blog's technical assessment of where SSDs fit in Hadoop's storage hierarchy — evaluating specific bottlenecks (NameNode metadata, shuffle I/O, random reads) where flash storage provides measurable gains over spinning disk.
Apache Hadoop: Best Practices and Anti-Patterns
Yahoo's engineering blog guide on Hadoop best practices and anti-patterns from the team that ran the world's largest Hadoop clusters in 2010 — practical tuning advice from first-hand production experience at scale.
Virtualization Performance: Zones, KVM, Xen
Brendan Gregg's benchmark comparing the performance overhead of Solaris Zones, KVM, and Xen virtualization at Joyent. A rare empirical comparison from someone with the systems instrumentation expertise (DTrace) to measure what actually matters.
Speeding Up Your Python Code
Max Burstein's practical guide to Python performance — profiling-driven optimization covering list comprehensions, generators, local variable access, string concatenation, and C extension use. A good 2013-era reference for the pragmatic Python performance techniques.
RubyTune: Rails Performance Cheat Sheet
RubyTune's Rails performance cheat sheet — a condensed reference for profiling, benchmarking, and optimizing Rails applications. Saved when learning Rails performance optimization in early 2013.
Nick Cave: Soundsuit Sculptures
Nick Cave's Soundsuit sculptures — elaborate wearable costumes that produce sound when worn, combining visual spectacle, performance, and social commentary. The content was saved with a Burning Man comparison, though Cave's work is more serious than that framing suggests.
Ruby GC Internals
A guide to Ruby's garbage collector internals from 2012, covering the mark-and-sweep algorithm and the stop-the-world pauses that made Ruby 1.9's GC a known performance bottleneck. Ruby 2.0 and 2.1 later introduced generational and incremental GC to address these issues.
Benchmarking High Performance I/O with SSD for Cassandra on AWS
Netflix's 2012 benchmark of SSD-backed Cassandra on AWS EC2 — showing 3-4x latency improvements over spinning disk for their use case. An early real-world data point on SSD economics in cloud distributed databases.