Towards Data Science

8/13/2026

Google's Open Knowledge Format (OKF) is a Markdown+YAML skeleton for sharing knowledge between humans and AI agents. This post reuses that skeleton for a very specific job — an agent-to-agent hand-off of pre-tokenized integer arrays between three Qwen2.5-Coder models (7B, 3B, 1.5B) — and shows the 28–37% TTFT reduction plus the one full-vocabulary equivalence check that keeps the whole thing safe. The post How to Utilize OKF Efficiently to Enable Knowledge Exchange Among LLMs appeared first on Towards Data Science.

8/13/2026

Enterprise Document Intelligence [Vol.1 #9ter] - The pipeline from Article 9 calls a model at several steps to be sure it is right. On easy questions that is needless latency. A per-question signal routes them past the model, about two seconds saved for a keyword match. The post Cut an Enterprise RAG Pipeline’s Latency and Cost by Calling the LLM Less, Not by Buying a Faster Model appeared first on Towards Data Science.

8/13/2026

Learn how to run OpenClaw bots for increased productivity The post How to Orchestrate a Fleet of OpenClaw Bots appeared first on Towards Data Science.

8/13/2026

A practical guide to choose the proper tool for your agentic workflows and systems The post LangChain vs LangGraph: 4 Key Differences and When to Use Each appeared first on Towards Data Science.

8/12/2026

Enterprise Document Intelligence [Vol.1 #5nonies] - Nature, plan, execute, synthesize: closing brick 1 with a dispatcher that reads each PDF’s nature and picks the method that fits, fitz, Docling, PaddleOCR, EasyOCR, MinerU or Surya, then folds the outputs into one corpus The post Before Full Agentic RAG: Know How You Decide, and the Parsing Methods You Pick From appeared first on Towards Data Science.

8/12/2026

From one gradient to every gradient The post Backpropagation Explained for Beginners (Part 3): How Backpropagation Really Works appeared first on Towards Data Science.

8/12/2026

Image inputs and structured outputs with Gemma 4 and Ollama The post Building Multimodal Workflows with a Local LLM appeared first on Towards Data Science.

8/12/2026

A reproducible Lagos case study with population data, transport access, and airspace constraints The post How to Place Vertiport Locations in Any City Using Geospatial Machine Learning appeared first on Towards Data Science.

8/11/2026

Checking an A/B test until it crosses p The post Stop Calling the First Significant Day a Win appeared first on Towards Data Science.

8/11/2026

Not all Python data libraries are created equal! The post Should AI Developers Make the Switch from Polars to Pandas? appeared first on Towards Data Science.

8/11/2026

How to diversify a budget without losing the shadow prices that explain the result The post The Budget Split That Explains Itself appeared first on Towards Data Science.

8/11/2026

I replayed the same 27 real production tasks through two local models, one hardware upgrade apart, to find out what it actually takes to replace Claude as the brain behind a 90-tool personal agent. The post Can a Local LLM Run My AI Assistant? appeared first on Towards Data Science.

8/10/2026

Learn how to optimize your CI/CD pipeline for coding agents The post How to Effectively Deploy Code With Claude Code appeared first on Towards Data Science.

8/10/2026

Giving an AI agent access to a data warehouse doesn't automatically make it agent-ready. The real challenge lies in teaching the agent what the data means and when it's reliable enough to use.  The post Building an Agent-Ready Data Warehouse: What Traditional Architectures Do Wrong appeared first on Towards Data Science.

8/10/2026

A clear, math-first walkthrough of how VAEs learn to generate new data The post Variational Autoencoders (VAEs) Explained: From Theory to ELBO and the Reparameterization Trick appeared first on Towards Data Science.

8/10/2026

Learn how Spatial Pyramid Pooling enables CNNs to handle any image size, with a from-scratch PyTorch implementation The post SPP-Net Paper Walkthrough: Breaking the Fixed-Size Constraint appeared first on Towards Data Science.

8/9/2026

Building my first dbt models and learning what “analysis-ready” data actually means The post I Thought Loading Data Was the Finish Line. It Was the Starting Point. appeared first on Towards Data Science.

8/9/2026

Why use it? How to implement it? What can we do when it fails? The post How to Implement Structured Output with Local LLMs appeared first on Towards Data Science.

8/8/2026

Many Transformer explainers start with the finished architecture. We ask why it looks the way it does. The post Before Q, K, and V: Reconstructing the Transformer appeared first on Towards Data Science.

8/8/2026

Building a production-ready web interface for a stateful LangGraph agent The post Building a Streamlit UI for My LangGraph AI Agent appeared first on Towards Data Science.

8/7/2026

From Static Plots to Interactive Data Exploration The post Matplotlib vs Plotly: Which Python Chart Tool Should You Choose? appeared first on Towards Data Science.

8/7/2026

Enterprise Document Intelligence [Vol.1 #12] - The category of question most RAG pipelines silently fail on, and the pipeline shape that handles them The post Loop Engineering for Listing Questions: When the Answer Is Every Passage, Not the Top One appeared first on Towards Data Science.

8/7/2026

Faster dataframe engines are nice, but they don't reduce the amount of syntax an analyst has to hold in their head. The post The Problem with pandas Isn’t Performance. It’s Cognitive Overhead. appeared first on Towards Data Science.

8/7/2026

How a single evaluation choice inflated my results by 25 points, and what rebuilding honestly taught me about ML systems people might depend on The post My Fall-Detection Model Scored 94%, and It Was Lying to Me appeared first on Towards Data Science.

8/6/2026

A step-by-step guide to building a data agent and conversational interface that let business users to explore data in natural language without SQL The post I Built an AI Data Agent Which Can Query Data and Answer Business Questions. Here’s How. appeared first on Towards Data Science.

8/6/2026

The downside of conference travel The post Last Month’s Machine Learning Lessons Learned appeared first on Towards Data Science.

8/6/2026

A minimal loop with real API calls, validation, compact outputs, and trace evidence before adding an agent framework The post I Built a Tool-Calling Agent in Python. Here’s How I Debugged It appeared first on Towards Data Science.

8/6/2026

Enterprise Document Intelligence [Vol.1 #11] - When the first answer points elsewhere in the document, the pipeline loops back to fetch the linked context The post Loop Engineering for Cross-References: When RAG Answers ‘see Section 7.2’ Instead of the Actual Answer appeared first on Towards Data Science.

8/5/2026

An open, 2.8-trillion-parameter model shipped with 47 pages of its own recipe. Reading it tells you what building a frontier model now involves, and how little of it is the model. The post How a Frontier Model Gets Built, Read from the Kimi K3 Report appeared first on Towards Data Science.

8/5/2026

A primer about Semi-Supervised Learning, the approaches taken with different algorithms and the limitations of using unlabelled data. The post Introduction to Semi-Supervised Learning appeared first on Towards Data Science.

8/5/2026

Research-backed cues to detect LLM-generated text along with the mathematical intuition as to 'why' The post Is This Slop? Detecting AI-Generated Content Without a Model appeared first on Towards Data Science.

8/5/2026

Enterprise Document Intelligence [Vol.1 #5octies] - Rules propose, LLM validates: six deterministic signals on span-level typography surface heading candidates, one bounded loop keeps the real ones, and the same toc_df drops back into the RAG pipeline The post Building Document Structure with Loop Engineering: Recovering a PDF’s Outline from Body Typography for RAG appeared first on Towards Data Science.

8/4/2026

A practical guide to Bronze, Silver and Gold, with a working Python and DuckDB example The post The Medallion Data Architecture: An Introduction appeared first on Towards Data Science.

8/4/2026

An online simulation and a novel method for increasing power The post How to Get More Statistical Power from Fewer Research Participants appeared first on Towards Data Science.

8/4/2026

Data Storytelling Series, Chapter 1 The post Are Home Teams Favoured by Referees in Football/Soccer? appeared first on Towards Data Science.

8/4/2026

Building manager–specialist workflows with the OpenAI Agents SDK The post Using Agents as Tools appeared first on Towards Data Science.

8/3/2026

Enterprise Document Intelligence [Vol.1 #M2] - Every RAG system is built in three engineering layers stacked on one LLM call: prompt (the call itself), context (what fills the model’s window), loop (when the next call fires and when it stops). Knowing which layer you are standing on is half of building and debugging RAG The post Prompt, Context, Loop: The Three Engineering Layers Every RAG System Is Built On appeared first on Towards Data Science.

8/3/2026

Create a local CLI Agent from scratch completely for free The post How to Build CLI Agents with Python & Ollama appeared first on Towards Data Science.

8/3/2026

What actually makes a Forward Deployed Engineer, told through one supply chain project. The post The AI Was the Easy Part: What Is a Forward-Deployed Engineer in a Supply Chain? appeared first on Towards Data Science.

8/3/2026

How use Claude to craft an outstanding resume that lands offers The post How Claude Help Me Build My $200k+ ML Resume appeared first on Towards Data Science.

8/2/2026

Perform non-programming tasks with coding agents The post How to Apply Coding Agents to Non-Programming Tasks appeared first on Towards Data Science.

8/2/2026

A step-by-step guide to building, running, and monitoring a stateful customer support agent using Python, LangGraph, and Langfuse. The post I Replaced a 15-Minute Booking Process with a LangGraph AI Agent appeared first on Towards Data Science.

8/1/2026

Most coding agents treat prompt construction like retrieval: gather more files, add more context, hope the model figures it out. But that approach breaks down fast. As context grows, irrelevant code competes for attention, and when the window fills, agents start compressing their own memory—often mid-task. What looks like “forgetting” is usually just degraded context. This article explores a different approach: treating prompt construction like a compiler that decides what to keep, what to reduce, and what to discard entirely. The post Coding Agents Don’t Need Bigger Context Windows — They Need a Context Compiler appeared first on Towards Data Science.

8/1/2026

A hybrid LLM application pattern that combines a predefined workflow with adaptive agent behavior The post Put the Agent Inside the Workflow appeared first on Towards Data Science.

7/31/2026

How a seemingly harmless move to a multi-agent architecture quietly tripled our LLM costs and what actually fixed it. The post The 3× Token Bill We Didn’t See Coming appeared first on Towards Data Science.

7/31/2026

In five to ten years, the sharpest manager in your company might not be human, might not sleep, and might exist entirely in shared GPU memory. This is the systems-level view of the algorithmic corporation — why middle management collapses into a protocol, what breaks in the current AI stack, and what has to be built for autonomous agents to actually run a business. The post When the Code Becomes the CEO: Why Your Next Manager Might Be a Decentralized Agentic Loop appeared first on Towards Data Science.

7/31/2026

A practical tutorial for recording model tool requests, real function results, patches, checks, screenshots, and a saved run log. The post How to Debug AI Coding Agents When They Change the Wrong Thing appeared first on Towards Data Science.

7/31/2026

A friendly introduction to one of the most powerfull optimization techniques using the uncapacitated facility location problem The post How Benders Decomposition Works Part I: Optimality Cuts appeared first on Towards Data Science.

7/30/2026

How one open-source ecosystem made state-of-the-art AI accessible The post The Python Ecosystem That Changed AI Development appeared first on Towards Data Science.

7/30/2026

Optimise how you interact with your coding agents The post How to Organize All of Your Coding Agent Tasks appeared first on Towards Data Science.

7/30/2026

What it actually takes to turn a company's scattered knowledge into something an LLM can reliably use — and why the demo is 5% of the work. The post How to Build a Context Layer and a Company Brain appeared first on Towards Data Science.

7/30/2026

The full conjecture is stated over abstract fields, but the counterexample is a concrete 3D function that we can explain and visualize using familiar geometric ideas and a little algebra. The post A Simplified View of the Jacobian Conjecture appeared first on Towards Data Science.

7/30/2026

How statistical physics explains the transition from deterministic predictions to generative AI. The post How to Decode the Temperature Parameter in LLMs appeared first on Towards Data Science.

7/29/2026

Prompt engineering helps you write better prompts—but it doesn’t help you change them safely. This article explores a common production failure where a simple variable rename breaks every live call, and introduces a lightweight static analysis tool that treats prompts like contracts, catching breaking changes before they ship. The post Prompt Engineering Is Solved—Prompt Management Isn’t appeared first on Towards Data Science.

7/29/2026

Why prediction-driven variable selection misses confounders and how Bayesian Adjustment for Confounding attempts to fix it. The post Why Your Best Predictive Model Gives the Wrong Treatment Effect appeared first on Towards Data Science.

7/29/2026

Building an ALNS heuristic in Python for vehicle routing, time windows, capacity constraints, and mandatory driver breaks. The post Los Movimientos, Part II: Solving Large Pickup-and-Delivery Problems with Adaptive Large Neighborhood Search appeared first on Towards Data Science.

7/29/2026

This is the opening piece of a four-part deep dive series, on building a high-frequency streaming pipeline against a live public API. The data source is openSenseMap, a citizen-science IoT network used for climate research, mostly in Germany. A live public API is what makes it useful: it produces data-quality problems and edge cases that clean sample datasets never show. This article focuses on step-1: Normalization, later pieces cover matching algorithms, adaptive polling and noise filtering, and a vendor-agnostic Apache Iceberg pipeline with Terraform that runs locally in Docker and moves to AWS or GCP with minimal change. The post Avoiding Entity Key Drift in a Data Lake: Step 1, Normalization appeared first on Towards Data Science.

7/28/2026

Five models, sustained generation, real wall-socket energy at $0.31/kWh — and the surprise the RTX-3090 numbers predicted, only bigger. The post How Much Does a Local LLM Actually Cost to Run? I Measured Every Watt on Apple Silicon appeared first on Towards Data Science.

7/28/2026

from custom integrations to a universal standard for tool access The post MCP Explained: How Modern AI Agents Connect to the Real World appeared first on Towards Data Science.

7/28/2026

You "vibe coded" the import. Understand Adam's optimization dynamics, why it fails spectacularly, and how to fix it. The post Don’t Just “Throw Adam at It”: Misunderstanding Adam Will Cost You appeared first on Towards Data Science.

7/28/2026

The idea that makes backpropagation possible. The post Backpropagation Explained for Beginners (Part 2): There Has to Be a Better Way appeared first on Towards Data Science.

7/27/2026

Using mathematical optimization to solve a pickup-and-delivery problem with time windows. The post “Los Movimientos”: The Routing Problem That Nearly Broke My Spirit appeared first on Towards Data Science.

7/27/2026

In a world where human time is expensive, learn how to use it only when really necessary The post Reducing Human Annotation with ML Active Learning appeared first on Towards Data Science.

7/27/2026

The mean keeps making its usefulness felt in all sorts of situations, often in truly non-obvious ways The post The Most Beautiful Statistic: The History and the Science of the Humble Mean appeared first on Towards Data Science.

7/27/2026

A practical reproduction of three retrieval baselines, including the crashes, fixes, and score checks that matter for RAG systems. The post How I Reproduced BM25, Dense Retrieval, and SPLADE on a 16GB MacBook appeared first on Towards Data Science.

7/26/2026

Maximize your efficiency with Claude Code The post How to Efficiently Prompt Claude Code appeared first on Towards Data Science.

7/26/2026

Building a browser-use agent with OpenAI Agents SDK and Playwright MCP The post How to Give an LLM Agent a Browser appeared first on Towards Data Science.

7/25/2026

Architecting cost-effective infrastructure by navigating the latency and storage trade-offs of HNSW, SPANN, and DiskANN The post How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes appeared first on Towards Data Science.

7/25/2026

I generated a Kármán vortex street without solving a single fluid equation. Here's how the Lattice Boltzmann Method gets there instead, derived from first principles, implemented in C++, and run on a supercomputer. The post The Fluid Simulator That Doesn’t Solve the Fluid Equations appeared first on Towards Data Science.

7/24/2026

Tabular foundation models predict the missing column of any spreadsheet zero-shot, the way an LLM completes text — and on the TabArena benchmark they now sit above fully tuned gradient-boosted trees. An introduction to how they work, an independent reproduction of the strongest open one, and a map of where XGBoost still wins. The post Tabular LLMs: An Introduction to the Foundation Models That Predict Your Spreadsheet appeared first on Towards Data Science.

7/24/2026

Automating the classification and extraction of PII from emails using AWS The post Build and Run an Intelligent Document Processing (IDP) System in the Cloud appeared first on Towards Data Science.

7/24/2026

Enterprise Document Intelligence [Vol.1 #8quater] - Two angles on the cascade, cost and a validation loop, backed by a real sweep of twenty local models against a hosted flagship The post Loop Engineering for RAG Generation: An LLM Cascade from a Cheap Local Model Up to a Hosted Flagship appeared first on Towards Data Science.

7/24/2026

Most AI memory systems keep the newest information—not the most important. Here's how I used the Ebbinghaus forgetting curve to build a better memory engine for LLMs. The post Context Windows Forget What Matters — I Built a Usage-Reinforced Decay Engine for AI Agent Memory appeared first on Towards Data Science.

7/23/2026

$8 million vs $5k + Potentially Going Viral The post When Data Science Makes Us Sad: The Story of an Overbooked Flight appeared first on Towards Data Science.

7/23/2026

Enterprise Document Intelligence [Vol.1 #8ter] - Naming the RAG error correctly matters: model reads the context, so a wrong answer is an extraction error, not a hallucination. Seven typed-contract patterns keep the generation brick honest, with a decomposition rule for small models The post Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Generation Contract appeared first on Towards Data Science.

7/23/2026

Patience, Optimism, Discipline, Projects, Teams The post Lessons Learned After 8.5 Years of ML appeared first on Towards Data Science.

7/23/2026

The hidden cost of asynchronous systems, how tiny CPU tasks quietly became our biggest bottleneck while scaling hundreds of LLM agents. The post Why Adding More AI Agents Made Our System Slower appeared first on Towards Data Science.

7/22/2026

Enterprise Document Intelligence [Vol.1 #8bis] - Two regimes for sending retrieved candidates to the generation brick, the sufficiency signal that picks between them, and the per-question type dispatch that makes it cheap The post Loop Engineering for RAG Generation: iterate top-k one at a time appeared first on Towards Data Science.

7/22/2026

If you have ever wanted to actually build an LLM inference runtime yourself — pack your own weights, own every barrier, capture your own CUDA graphs — this is what that journey looks like on an H100. A step-by-step tour of a small runtime called annotated-llm-runtime, and the three bugs that produced most of the annotations. The post How To Build Your Own LLM Runtime From Scratch appeared first on Towards Data Science.

7/22/2026

A hands-on walkthrough of code execution with the OpenAI Agents SDK and Docker The post Build an LLM Agent That Can Write and Run Code appeared first on Towards Data Science.

7/22/2026

Static analysis nailed the malicious skill and over-flagged the useful one. The gap between those results is where human judgement actually earns its keep. The post Detecting Vulnerabilities in Agent Skills with SkillSpector: From Green Checkmark to Real Security Judgment appeared first on Towards Data Science.

7/21/2026

Enterprise Document Intelligence [Vol.1 #9bis] - Your RAG isn’t hallucinating, it’s answering the wrong context faithfully. On real NIST and World Bank documents, watch each of the four bricks break, and the contract that closes it The post Prompt Engineering Isn’t Enough: How Four Bricks of Context Engineering Stop RAG Hallucinations appeared first on Towards Data Science.

7/21/2026

A reproducible 100-step LoRA fine-tuning run for OpenVLA, with dataset checks, Colab setup, training metrics, and W&B evidence. The post I Tried Fine-Tuning a Robot AI Model on Colab. Here Is What Worked appeared first on Towards Data Science.

7/21/2026

Exploring GPU acceleration with cuDF, cudf.pandas, and the Polars GPU Engine The post How Much of a Data Science Workflow Can Run on a GPU Today? Part 1: Accelerating Data Preparation appeared first on Towards Data Science.

7/21/2026

A hands-on guide to tracking experiments, logging models, and reproducing results with ML Flow. The post Are Your ML Experiments a Mess? Here’s the Fix appeared first on Towards Data Science.

7/20/2026

Apply long-running coding agents to become a more productive engineer The post How to Run Claude Code Agents for 24+ Hours appeared first on Towards Data Science.

7/20/2026

Enterprise Document Intelligence [Vol.1 #10B] - The LLM as last line of defence, then two real escalations walked end to end: a flat table to Azure, a figure to a vision model The post Loop Engineering with Adaptive Parsing in Action: Parsing Flat Tables with Azure and Figures with a Vision LLM appeared first on Towards Data Science.

7/20/2026

How do you make decisions when you can't trust anyone in the room? The post Water Cooler Small Talk, Ep. 12: Byzantine Fault Tolerance appeared first on Towards Data Science.

7/20/2026

Having categorized data is everything in reporting. Uncategorized data cannot be grouped and aggregated. But sometimes we must assign a category to uncategorized data according to certain rules. Let’s see how I solved this in a facility management project. The post Automatically Assign a Category to Uncategorized Rows in Power Query and DAX appeared first on Towards Data Science.

7/19/2026

Let's discover how neural networks learn, step by step The post Backpropagation Explained for Beginners (Part 1): Building the Intuition appeared first on Towards Data Science.

7/19/2026

Enterprise Document Intelligence [Vol.1 #6quinquies] - Prompt engineering, then context engineering, then loop engineering. On the question side, the loop is small by design: read the doc, ask what is missing, re-parse. The post Loop Engineering for RAG Question Parsing: The Small Loop That Runs Before Retrieval appeared first on Towards Data Science.

7/19/2026

An AI agent passed every metric in the eval harness I published, then the CFO killed it — its successful resolutions cost more than the humans it replaced. The post Your AI Agent Passed Every Eval. Finance Still Killed It. appeared first on Towards Data Science.

7/18/2026

A practical enterprise AI architecture with data agents, AI-powered QA, and AI governance. The post Many Companies Use AI. Few Know How to Build an AI-Native Enterprise Data Platform. appeared first on Towards Data Science.

7/18/2026

Enterprise Document Intelligence [Vol.1 #10A] - The escalation cascade and the free, deterministic checks that flag a failed parse before you pay for a deeper one The post Loop Engineering with Adaptive PDF Parsing: Start Cheap, Pay for a Heavier Parser Only When the Page Needs It appeared first on Towards Data Science.

7/18/2026

A practical guide to combining pre-churn scoring with uplift modelling for smarter retention. The post How to Improve Customer Retention in FinTech appeared first on Towards Data Science.

7/17/2026

Maximize the latest OpenAI model The post How to Work Effectively with GPT-5.6 appeared first on Towards Data Science.

7/17/2026

On the value of building on existing foundations The post Using Classical ML to Empower AI Agents appeared first on Towards Data Science.

7/17/2026

Everyone is talking about loop engineering, but most discussions assume an LLM sits at the center of the loop. I wanted to isolate the architecture itself. So I built a deterministic, zero-dependency Python benchmark that replaces the model with simple rules, allowing me to measure one question directly: can a goal-directed controller isolate failures better than a traditional linear pipeline? After validating the benchmark across 300 random seeds—and fixing a subtle bug that initially invalidated my own results—I found that the controller consistently completed independent branches that a linear executor never reached. This article walks through the architecture, the benchmark design, the debugging process, and the evidence behind a narrow but practical claim: failure isolation is a measurable property of control flow, independent of LLM reasoning. The post Context Engineering Isn’t Enough — A Loop Engineering Experiment With No LLM Inside the Loop appeared first on Towards Data Science.

7/17/2026

AI's energy crisis is reviving an old idea: computing with physics instead of digital logic. Here's how analog chips actually work, why noise nearly killed the idea once already, and what happens when you simulate that noise yourself. The post Analog AI Is Back, But Can It Survive Its Own Noise? appeared first on Towards Data Science.

7/17/2026

Enterprise Document Intelligence [Vol.1 #9B] - One call wires the four upgraded bricks together, run on a paper, a NIST standard, and a report with a broken TOC The post One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited appeared first on Towards Data Science.