Subject
44 entries
Mlops
Bookmarks
TensorZero
TensorZero is a self-hosted LLMOps platform that unifies gateway, observability, evaluation, optimization, and A/B testing in one Rust-built system. The feedback loop: production inference data flows back into prompt and model improvement automatically.
Graft: Enterprise AI from Playground to Production
Graft is an enterprise AI company helping businesses move from LLM playground to production — included in Weekend Fund's Enterprise AI Stack. A signal of the 2023 market for companies building the deployment and operations layer on top of frontier models.
Running Large Language Models in Production: Cohere's TIF
Cohere's post on their Transformer Inference Framework (TIF) — covering the systems challenges of serving large language models in production at scale. An early look at how LLM serving differs from traditional ML model serving and what optimizations matter.
Replicate Cog: Containers for Machine Learning
Cog is Replicate's open-source tool for packaging machine learning models into standardized Docker containers with automatic HTTP APIs. It's the packaging layer that powers Replicate's model hosting platform — define inputs/outputs in Python, get a container that serves the model.
Aporia — ML Observability
Aporia is a cloud-native ML observability platform for monitoring deployed models — drift detection, performance monitoring, and explainability tooling. Addresses the gap between training-time metrics and what models actually do in production.
SE-ML: Software Engineering Practices for Machine Learning
SE-ML is a catalog of software engineering practices adapted for machine learning systems — covering data management, model training, deployment, and monitoring. A structured reference for treating ML development with the same rigor as traditional software engineering.
MLOps: Machine Learning Operations
A comprehensive guide to MLOps — the practices, tools, and culture for deploying and maintaining machine learning models in production. Covers the full lifecycle from experiment tracking through model serving, monitoring, and retraining pipelines.
Made With ML: MLOps Curriculum
Made With ML is a free, project-based curriculum for learning ML engineering and MLOps — covering not just model training but the full production pipeline from data to deployment. One of the most practical and comprehensive self-study resources for applied ML.
Banana Serverless Template: Stable Diffusion
Banana.dev's serverless template for deploying Stable Diffusion as an API — package the model, push to Banana, get an API endpoint. One of the first practical paths to serving Stable Diffusion at scale without managing GPU infrastructure.
Abacus.AI
Abacus.AI is an enterprise ML platform providing AI capabilities to companies without large AI teams — real-time feature stores, automated training, and deployment. Bookmarked via the jobs page, suggesting career interest in AI platform infrastructure work.
Sematic: Python-First ML Pipeline Orchestration
Sematic is an open-source ML pipeline orchestration framework using Python decorators — no YAML, seamless local-to-cloud execution, built-in artifact tracking dashboard. Competes with MLflow and Kubeflow but stays in pure Python.
Weights & Biases — Fully Connected ML Blog
The Weights & Biases Fully Connected blog covers ML research, tutorials, and MLOps practices. Saved as a resource hub — W&B's writing quality is high, and their reports feature interactive experiment tracking visualizations unavailable elsewhere.
AI Infrastructure Landscape Map
An infographic map of the AI infrastructure ecosystem covering categories from data labeling to model deployment, produced by the AI Infrastructure Alliance. Useful as a reference snapshot of the MLOps tooling space circa 2021.
AI Infrastructure Ecosystem 2022
A mid-2022 industry report mapping the AI/ML infrastructure ecosystem across three build strategies (build-your-own, end-to-end platforms, best-of-breed composable stacks) and the full toolchain from orchestration through model serving. A useful time-capsule of the MLOps landscape just before the LLM era reshaped it.
A Hitchhiker's Guide to Distributed Training of Deep Neural Networks
Chahal, Grover, and Dey survey the algorithms and engineering techniques for distributed deep learning training, covering data parallelism, model parallelism, AllReduce strategies, gradient compression, and mixed precision. A practical reference for scaling training from single GPU to multi-node clusters.
MLOps Toys: Curated List of Machine Learning Projects
MLOps Toys is a curated directory of ML projects and tools — organized by category (data, model training, serving, monitoring, etc.). A snapshot of the MLOps ecosystem circa 2022, useful as a reference map of what tools existed before the category consolidated.
cleanlab 2.0: Automatically Find Errors in ML Datasets
cleanlab 2.0 is an open-source Python framework for automatically finding and fixing errors in ML datasets — mislabeled examples, out-of-distribution samples, near-duplicates. Built on the 'confident learning' statistical framework for label noise estimation.
Kubernetes for MLOps: Scaling Enterprise Machine Learning, Deep Learning, and AI
A book/transcript from the This Week in ML podcast by Sam Charrington on using Kubernetes as the operational backbone for enterprise ML workloads. Covers the full spectrum from containerized training jobs to model serving, making the case that Kubernetes is the de facto standard for scaling ML in production.
Effective Data Science Infrastructure
Ville Tuulos's Manning book on building productive data science infrastructure, with Metaflow as its centerpiece framework. The core argument—that infrastructure exists to make people productive, not to be technically clever—is a useful corrective to the endless tooling churn in ML engineering.
MLOps Maturity Models: Google and Microsoft Frameworks
ZenML's overview of MLOps maturity models from Google and Microsoft — frameworks for thinking about how ML organizations can systematically improve how they develop and deploy models. Useful if you're trying to level up a team's ML practices from ad-hoc to automated.
MLOps Notebooks Syllabus
A Jupyter notebook curriculum for MLOps — covering model deployment, monitoring, CI/CD for ML, feature stores, and data versioning. Practical operational coverage for the gap between training a model and running it reliably in production.
DAGsHub: GitHub for Data Science
DAGsHub is a GitHub-like collaboration platform for data science — adding data versioning (DVC), experiment tracking, and model registry to the familiar git workflow. Addresses the pain point that standard git doesn't handle large data files or ML experiment metadata.
The Unbundling of Airflow
fal.ai's analysis of why Apache Airflow is being unbundled by specialized tools — Prefect and Dagster for orchestration, dbt for transformation, Temporal for long-running workflows. The monolithic DAG orchestrator is fracturing along functional lines.
Full Stack Deep Learning (Spring 2021)
Full Stack Deep Learning is a free course bridging ML research and production deployment — covering the full pipeline from data management and model training to testing, monitoring, and team structures. The course for researchers who want to ship and engineers who want to understand ML.
Production Code for Data Science: Our Experience with Kedro
Beamery's engineering team shares their experience using Kedro to bring software engineering discipline to data science code in production — covering what worked, what required adaptation, and how the pipeline structure changed their team's workflows.
Ploomber: Data Pipelines from Dev to Production
Ploomber is a Python framework for building data pipelines that can develop in Jupyter notebooks and deploy to Kubernetes, Airflow, or AWS Batch without rewriting code. Solves the notebook-to-production gap by treating notebooks as first-class pipeline tasks.
Migrate Kedro Pipeline to Vertex AI
A walkthrough by Ivan Nardini on migrating a Kedro data science pipeline to run on Google Vertex AI Pipelines — covering the Kedro-Vertex plugin, pipeline conversion, and deployment. Shows how Kedro's portability story works in practice against a major cloud ML platform.
Kedro: Production-Ready Data Science Pipelines
Kedro is an open-source Python framework for building reproducible, maintainable, and modular data science pipelines — applying software engineering principles (catalogs, pipelines, project templates) to ML workflows. The answer to 'how do data science teams write production-grade code.'
Real-Time Machine Learning: Challenges and Solutions
Chip Huyen's definitive survey of real-time ML challenges — the engineering gap between batch ML pipelines and systems that must make predictions on live data with low latency. Covers online prediction, continual learning, and the infrastructure needed to bridge training and serving.
Lessons on ML Platforms from Netflix, DoorDash, Spotify, and More
Ernest Chan's Towards Data Science survey of ML platform design at Netflix, DoorDash, Spotify, and other tech companies — extracting common patterns and lessons from how production ML infrastructure evolved at scale. Practical systems thinking for ML platform builders.
Hidden Technical Debt in Machine Learning Systems
Sculley, Holt, Golovin et al. at Google extend the software engineering concept of technical debt to ML systems, identifying ML-specific forms that accumulate invisibly at the system level: boundary erosion, entanglement, hidden feedback loops, undeclared consumers, and data dependencies. The canonical paper explaining why ML systems are uniquely expensive to maintain.
Feature Stores — A Hierarchy of Needs
Eugene Yan's survey of feature stores organized as a hierarchy of needs — from basic feature reuse to online serving to real-time streaming. Practical taxonomy for understanding when you need a feature store and what tier of sophistication your use case requires.
ML Zoomcamp — Free Cohort Machine Learning Course
Alexey Grigorev's ML Zoomcamp — a free cohort-based machine learning course covering regression, classification, deployment, and MLOps fundamentals. A comprehensive practical curriculum from the author of Machine Learning Bookcamp.
Cookiecutter Data Science Template
Cookiecutter Data Science is a standardized, opinionated project template for data science work in Python — a sensible starting folder structure that makes projects reproducible and shareable. The de facto standard for organizing Python data science projects.
Stanford CS 329S — Machine Learning Systems Design
Stanford CS 329S Machine Learning Systems Design — Chip Huyen's course on building production ML systems. Covers the full lifecycle from problem framing through data, training, deployment, and monitoring with real-world case studies.
Operationalizing Machine Learning: Forrester Research Report
Forrester Research report on operationalizing machine learning in enterprise settings, covering the organizational and technical requirements for moving models from development into production reliably. Useful snapshot of where enterprise ML practice stood in late 2021 and the key gaps between experimentation and production.
Machine Learning Engineering for the Real World
A practical guide to ML engineering as a discipline — applying software engineering processes (agile, simplicity, iterative development) to ML projects from scoping through production. Makes the case that ML projects fail not from algorithmic complexity but from lack of engineering discipline around planning, experimentation, and deployment.
Inferrd: Managed GPU Model Inference
Inferrd was a managed ML model serving platform with GPU support — deploy models via API without managing infrastructure. A 2021-era MLOps tool in the model deployment category that eventually got acquired.
Made With ML
Made With ML is a free, project-oriented ML curriculum covering the full stack from data to deployment — written by Goku Mohandas. One of the most practical ML learning resources available, because it treats production concerns as first-class alongside model building.
Reproducible Deep Learning PhD Course
Simone Scardapane's PhD course on Reproducible Deep Learning — covering Git, Docker, DVC, experiment tracking, and CI/CD for ML research. Addresses the reproducibility crisis in deep learning with practical tooling.
ML in Production — Best Practices for Real-World ML Systems
ML in Production is a blog and newsletter focused on building and operating real-world ML systems — covering experimentation programs, deployment, monitoring, and the organizational practices that make ML succeed in production environments.
Effective Testing for Machine Learning Systems
Jeremy Jordan's framework for testing machine learning systems — pre-train checks, invariance tests, directional expectation tests, and minimum functionality tests. Organizes tests around model 'skills' rather than code structure.
Data Science Meets DevOps: MLOps with Jupyter, Git, and Kubernetes
Kubeflow's blog post on MLOps practices combining Jupyter notebooks, Git, and Kubernetes — the infrastructure side of making ML reproducible, collaborative, and deployable. Establishes the canonical MLOps stack for Kubernetes-based ML teams.
The Perilous World of ML: Pipeline Jungles and Hidden Feedback Loops
John Foreman on the hidden technical debt in ML systems — pipeline jungles and feedback loops that make production ML fragile in ways that pure model metrics never reveal. Anticipates the 'Hidden Technical Debt in Machine Learning Systems' Google paper by months.