Water Cooler Small Talk, Ep. 11: Overfitting in RAG evaluation
Why memorizing for the exam doesn’t mean you understand the subject The post Water Cooler Small Talk, Ep. 11: Overfitting…
Category: General
David Autor named head of the Department of Economics
David Autor, the Daniel (1972) and Gail Rubinfeld Professor in the MIT Department of Economics, has been named head of…
Amplify the Expert: A Philosophy for Building Enterprise RAG
Enterprise Document Intelligence [Vol.1 #M1] – The thesis behind every architectural choice in this series The post Amplify the Expert:…
LLMs help robots understand vague instructions and focus on key details
Imagine working at a warehouse or office sometime in the near future, and you’re asked to help a new trainee…
How to Ace Data and ML Behavioural Interviews
How to smash through data / ML behavioural interviews The post How to Ace Data and ML Behavioural Interviews appeared…
Vector RAG Isn’t Enough — I Built a Context Graph Layer for Multi-Agent Memory
I benchmarked raw chat history, vector-only RAG, and a context graph on the same multi-agent conversations. The results exposed a…
MIT in the media: Exploring how curiosity-driven science is an essential ingredient in America’s success
Over the past 80 years, America’s bold, sustained investment in scientific research, and the discoveries, ideas and innovations that flowed from…
The Hot Path Belongs to GBDTs, Agents Own the Cold Path: A Payment-Fraud Benchmark
A reproducible benchmark on latency, cost, and reproducibility, and where agents actually earn their keep. The post The Hot Path…
3 Agents. 3 LLMs. 1 Aging GPU: Engineering Parallel Inference on Bare Metal
Beat the 8GB VRAM limit. Learn how to run three different LLMs on a single 8GB GPU using C++ layer…
Letting an LLM Pick the Right RAG Page: The Arbiter Pattern at the End of Retrieval
Enterprise Document Intelligence [Vol.1 #7C] – One LLM call ranks the candidates with reasons. The output is one typed object…
One Month Into Learning Data Engineering in Public: Here’s What I Didn’t Write About
A reflection on the first month of learning data engineering in public, and what actually kept me going. The post…
Improving the speed and energy-efficiency of AI agents
Agentic workflows are artificial intelligence-powered software systems that chain together multiple models and external tools to tackle complicated tasks, like…
How to Build a Credit Scoring Grid From a Logistic Regression Model
Turning model coefficients into a 0–1000 score, with risk classes and stability checks The post How to Build a Credit…
Your First Task as a Data Engineer in a New Company? Make the ETL Pipeline Testable
A practical data engineering onboarding workflow for environment setup, automated testing, and AI-assisted development. The post Your First Task as…
A Three-Phase Factual Recall Circuit in Gemma-2B and Gemma-12B-IT
Activation patching reveals how facts are stored, routed, and read out across transformer layers, and why the residual stream does…
Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead
A practical walkthrough using text-to-SQL as the example The post Why I Stopped Using One Agent and Built a Multi-Agent…
Exploring the societal impacts of AI
At the recent AI and Society Forum at MIT, experts from across the Institute discussed the potential benefits and dangers…
How to Create Powerful Loops in Claude Code
Learn about the concept of loops to power your coding agents. The post How to Create Powerful Loops in Claude…
I Spent an Hour on a Data Preprocessing Task Before Asking Gemini
How Gemini solved my Pandas problem in seconds, and why data science fundamentals still matter to spot suboptimal solutions The…
Retrieval Is Filtering, Not Search: A Mental Model for Enterprise RAG
Enterprise Document Intelligence [Vol.1 #7A] – Stop searching strings. Filter line_df and toc_df. Pick anchors small, expand context large The…
Build Your Own Local AI Coding Agent with Gemma 4 and OpenCode
From installing Ollama to launching OpenCode with a local model, step by step. The post Build Your Own Local AI…
New chip could help tiny robots traverse complex environments
A new chip developed by MIT researchers could help tiny, low-power UAVs avoid obstacles as they zip around tight corners…
Encoding Categorical Data for Outlier Detection
Why one-hot encoding isn’t always the best approach, and alternative encodings The post Encoding Categorical Data for Outlier Detection appeared…
How to Use Claude Code in Your Browser
Learn how to apply coding agents to verify work in your browser. The post How to Use Claude Code in…
Neural Networks, Explained for Beginners: Start Here If They’ve Confused You
The intuition behind neural networks and why they need activation functions. The post Neural Networks, Explained for Beginners: Start Here…
Tool Calling, Explained: How AI Agents Decide What to Do Next
Understanding ow LLMs interact with the world around them, from returning data to taking action The post Tool Calling, Explained:…
Reconstructing the Table of Contents a PDF Forgot to Ship, So RAG Can Scope by Section
Enterprise Document Intelligence [Vol.1 #5septies] – When a PDF prints a contents page but exposes no outline, two ways to…
What Are the Possibilities to Build Date Tables in Self-Service Environments?
For years, I created date tables with DAX code whenever I didn’t have a way to create them upstream of…
7 Crucial Barriers Between Data Teams and Self-Healing Data Architecture
What data teams need to build with AI to make self-healing data architecture a practical reality The post 7 Crucial…
Making a PDF’s Images Searchable for RAG, Without Paying to Read Them All
Enterprise Document Intelligence [Vol.1 #5sexies] – image_df tells you where every picture is. Turning the few that matter into searchable…
Materialized Lake Views in Microsoft Fabric: When Your Medallion Fits in a SELECT Statement
Five surfaces collapsed into one declarative layer. Here’s the full story of Materialized Lake Views in Microsoft Fabric – from…
Python 3.14 and its New JIT Compiler
A technical overview and some benchmarks The post Python 3.14 and its New JIT Compiler appeared first on Towards Data Science.
A better way to model the behavior of metal alloys
Companies working at the frontier of aerospace, energy, and computing are constantly looking for new materials to improve performance. But…
Building a Custom GStreamer Plugin for NVIDIA DeepStream
Why Custom Inference in DeepStream? The post Building a Custom GStreamer Plugin for NVIDIA DeepStream appeared first on Towards Data…
Parse Scanned PDFs for RAG with EasyOCR: Free OCR Gives You Words, Not a Document
Enterprise Document Intelligence [Vol.1 #5quinquies] – Same 1974 scanned PDF, two engines. EasyOCR recovers text. Docling recovers text + sections…
GPU-Resident Top-K for Agentic RAG: I Built a CUDA Kernel So My Retrieval Step Would Stop Bouncing Off the GPU
The PCIe transfer latency is silently bottlenecking your agentic inference. Here is how building a custom device-resident vector search kernel…
Structured Outputs with LLMs: JSON Mode, Function Calling, and When to Use Each
Getting reliable, readable responses out of your LLM, and knowing which tool to reach for The post Structured Outputs with…
How Powerful is Claude Fable (Mythos) 5 for Coding?
Learn about the upsides and downsides of Claude Fable 5 The post How Powerful is Claude Fable (Mythos) 5 for…
Proteins: A Mosaic Pattern to Rule Them All?
For decades, the existence of the hydrophobic core, a region in the 3D structure of proteins where hydrophobic amino acids…
MIT in the media: For the future of tech, “Massachusetts can absolutely lead”
On June 9, The Boston Globe released its 2026 “Tech Power Players” list, recognizing 50 influential local leaders in technology and…
Dispatching the Parsed RAG Question: Chunk Strategy, Model Tier, Activations, Audit
Enterprise Document Intelligence [Vol.1 #6c] – The decisions the parser makes on top of the user string, using the document’s…
In game theory, generalists sometimes win out over specialists
Whether you’re playing poker against a single opponent or find yourself in a bidding war over a home purchase with…
Your Churn Threshold Is a Pricing Decision
How unit economics should set your classification cutoff, and why they rarely do. The post Your Churn Threshold Is a…
The Secret to Reproducible and Portable Optimization: ORPilot’s Intermediate Representation (IR)
Why production-level AI optimization modeling agent needs reproducibility and portability, and how IR helps achieve them The post The Secret…
You Probably Don’t Need an Agent Framework
Most LLM applications need a clear workflow, not an autonomous agent. Here’s how to build one in plain Python. The…
What the Question Parser Extracts from a User String: Keywords, Scope, Shape, Decomposition, Clarification
Enterprise Document Intelligence [Vol.1 #6b] – The five field families the parser reads straight from the user’s question, with the…
Could AI tell you where you left your keys?
An auto factory worker can remember the storage bin where she left a partly assembled component the night before, and…
MIT’s Initiative for New Manufacturing builds momentum
In May, the Initiative for New Manufacturing (INM) marked its first anniversary with MIT Manufacturing Week, four days of events…
Unlocking UK house-building with AI-accelerated planning
UK government partners with Google DeepMind to build a new AI-powered prototype aimed at faster housing decisions.