Skip to main content
Ryan Orban

Ryan Orban

Subject
5 entries

Benchmarking

Bookmarks

  1. TravelPlanner: A Benchmark for Real-World Planning with Language Agents

    TravelPlanner is a benchmark for evaluating LLM planning capabilities in complex real-world scenarios — GPT-4 scored 0.6% on the full benchmark, revealing that even the best LLMs struggle with multi-constraint sequential planning. A sobering check on agentic AI ambitions.

  2. Revenue Watcher

    Revenue Watcher tracks and compares revenue growth benchmarks for SaaS and software companies. A useful reference for founders and investors who want to know what 'good' growth looks like at different ARR stages.

  3. benchm-ml: ML Algorithm Benchmark Comparison

    A systematic benchmark of machine learning algorithms across platforms and implementations — comparing gradient boosting, random forests, neural networks, and others on speed and accuracy. One of the best empirical references for choosing between ML tools in 2015, when the xgboost vs sklearn debate was live.

  4. TodoMVC

    TodoMVC is a project that implements the same Todo application across every major JavaScript framework, letting developers compare syntax and architecture side by side. It became the de facto benchmark for evaluating JS frontend frameworks.

  5. Benchmarking High Performance I/O with SSD for Cassandra on AWS

    Netflix's 2012 benchmark of SSD-backed Cassandra on AWS EC2 — showing 3-4x latency improvements over spinning disk for their use case. An early real-world data point on SSD economics in cloud distributed databases.

All bookmarks