Subject
9 entries
Explainability
Bookmarks
Mnemom: proving what AI agents are thinking
Mnemom.ai makes AI agent reasoning transparent — "Prove What Your AI Agents Are Thinking." An agent explainability tool for validating agent decision-making.
LLM Visualization: Interactive 3D Transformer Walkthrough
An interactive 3D visualization of how LLMs work — walking through the transformer architecture token by token, layer by layer, with actual weight animations. The clearest visual explanation of attention, embeddings, and feedforward layers available.
Explaining Transformer Model Predictions
A practical comparison of SHAP, Transformers Interpret, and Ferret for explaining Hugging Face transformer predictions. Key takeaway: different methods give different results for the same prediction — all require careful interpretation.
Gradient Explanations for HuggingFace BERT Classification
A tutorial by Victor Dibia on generating gradient-based explanations for HuggingFace BERT text classification models in TensorFlow 2.0 — visualizing which tokens most influenced the model's prediction. Explainability for transformer classifiers was a practical gap in 2022 since attention maps alone are insufficient.
Interpretable Machine Learning
Christoph Molnar's free online book covering the theory and practice of interpretable machine learning — from inherently interpretable models (decision trees, linear regression) to post-hoc methods (SHAP, LIME, counterfactuals). The standard reference for understanding and explaining ML model behavior.
Shapash: Making Machine Learning Models Transparent
Shapash is MAIF's Python library for making ML models interpretable to non-technical stakeholders — wrapping SHAP and LIME with better visualizations and business-friendly explanations. Targets the gap between data scientists and decision-makers.
FACET: Human-Explainable AI
FACET is BCG Gamma's Python library for human-explainable AI — extending SHAP with interaction effects and redundancy-aware feature importance, plus simulation tools for model-based what-if analysis. More sophisticated than vanilla SHAP for understanding feature relationships.
SHAP: SHapley Additive exPlanations
SHAP (SHapley Additive exPlanations) is the standard Python library for explaining individual predictions from any ML model using game-theoretic Shapley values. It works across tree models, deep neural networks, and linear models, and produces both local and global interpretability.
ELI5 — sklearn Explainability Module
ELI5's sklearn module provides model explanation tools for scikit-learn estimators — feature importance, prediction decomposition, and permutation-based importance across linear models, tree ensembles, and SVMs. The explainability companion for sklearn workflows.