Towards Data Science

9/30/2026

How the way we represent data can change what we think the data is saying The post How Many Stories Can Your Data Tell? appeared first on Towards Data Science.

9/30/2026

Coding agents give us more time for discovery and our review practices should follow the analysis The post Insight Is Still the Currency of Data Science appeared first on Towards Data Science.

9/30/2026

In DAX, we reuse existing measures all the time when writing new measures. What happens when we write a measure based on another measure and try to change a filter already set in the nested measure? The post How to Solve Issues When You Nest Measures While Overwriting the Same Filter appeared first on Towards Data Science.

9/30/2026

One run, and what it actually proves The post Towards Spec-Driven Test Automation: Part 2 appeared first on Towards Data Science.

9/29/2026

Benchmarking the impact of fewer, larger files across three SQL workloads The post I Compacted 1,000 Apache Iceberg Files Into 6. Here’s What Happened to Query Performance. appeared first on Towards Data Science.

9/29/2026

Not every decision needs a decoder, generation Is not always a decision The post When All You Have Are Decoders, Every Decision Looks Like Generation appeared first on Towards Data Science.

9/29/2026

The real shift is bigger than productivity: AI is reshaping ownership, judgment, and the career path of data scientists. The post AI Made Data Scientists Faster. Now It’s Expanding the Job. appeared first on Towards Data Science.

9/29/2026

Agent design patterns every data engineer must know The post How to Design Architectural Guardrails Around AI Agents appeared first on Towards Data Science.

9/29/2026

Here’s how to solve it exactly. The post Building Fair Evaluation Sets Is a Combinatorial Problem appeared first on Towards Data Science.

9/28/2026

A solution where the "goto" operator becomes irreplaceable. The post Guided Merge Sort : An Optimized Sorting that Picks the Best from Ordinary and Multi-Way Merge Sort Algorithms appeared first on Towards Data Science.

9/28/2026

Turn a small open-source Qwen LLM into a fast, single-pass text classifier by swapping its language-modeling head The post How to Make Your Own JEV Model from an Open LLM appeared first on Towards Data Science.

9/28/2026

A small, strange discovery in machine learning called grokking The post The AI That Learned to Understand Long After It Stopped Trying appeared first on Towards Data Science.

9/28/2026

Detect hidden shifts in feature relationships with adversarial validation and scikit-learn The post How to Catch Data Drift When Every Feature Looks Normal appeared first on Towards Data Science.

9/27/2026

How calibrated decision models can handle high-frequency graph decisions while LLMs remain focused on reasoning, synthesis, and open-ended generation. The post GraphRAG with TypeSafe Jev: A System One Approach to Scalable Knowledge Graphs appeared first on Towards Data Science.

9/27/2026

Every boundary you draw removes a signal your tooling was relying on. That is a structure problem, not a search problem. The post Good Architecture Deletes the Signals Your Agent Depends On appeared first on Towards Data Science.

9/26/2026

My AI detectors flagged many genuine reviews, and filtering them made the sentiment model less accurate. The post AI Slop Is Already in Your Training Dataset. I Tested Three Ways to Spot It. appeared first on Towards Data Science.

9/26/2026

Inside a transformer, token index is a coordinate. Paragraph structure is what turns it into a metric. The post Your LLM Has a Curved Space of Paragraphs appeared first on Towards Data Science.

9/25/2026

Part 1: Understanding the technologies shaping our future The post 10 Things I’m Learning Beyond AI to Become More Technologically Fluent appeared first on Towards Data Science.

9/25/2026

Autoregressive rollout and uncertainty propagation. Second in a series on probabilistic forecasting for physical signals. The post Your Model's MSE Is Lying to You: Part II appeared first on Towards Data Science.

9/25/2026

RAG retrieves. Agents act. I built both separately, connected them explicitly, and ran the same nine tasks through all three systems. The post RAG Isn't an Agent — I Built the Layer Between Retrieval and Action appeared first on Towards Data Science.

9/25/2026

I tested TypeSafe AI’s Jev on 3,080 classification tasks to see how its accuracy, latency, calibration, and confidence compare with LLMs — and whether it works as a practical decision layer for AI systems. The post Jev vs. LLMs: When AI Moves from Generation to Decision-Making appeared first on Towards Data Science.

9/24/2026

Get more out of your coding agent subscriptions The post How to Maximize Your Coding Agent Subscriptions appeared first on Towards Data Science.

9/24/2026

Retrieval is not evidence. How to build AI that proves its own claims. The post Beyond RAGs: Building Actually Truthful AI Harnesses appeared first on Towards Data Science.

9/24/2026

Why a green test suite can mean nothing The post Towards Spec-Driven Test Automation: Part 1 appeared first on Towards Data Science.

9/24/2026

The reliability mechanisms we add to LLM pipelines are often the ones that make them confidently wrong. The post When the Correct Answer Is Nothing, What Does Your Pipeline Return? appeared first on Towards Data Science.

9/23/2026

Reproducing Anthropic's "Toy Models of Superposition" from scratch in NumPy, with hand-derived gradients and no borrowed numbers. The post I Trained a Tiny Network to Compress Data. It Drew a Pentagon. appeared first on Towards Data Science.

9/23/2026

A Journey through TF-IDF, vector space, and text classification The post From Words to Vectors: What Happens in Between? appeared first on Towards Data Science.

9/23/2026

The mechanics behind local reasoning experiments with Unsloth and why the reward function matters as much as the model. The post How GRPO Trains Small Language Models with Verifiable Rewards appeared first on Towards Data Science.

9/23/2026

A beginner-friendly guide to building a world model in Python, letting it daydream its way through CartPole, and accurately measuring when the illusion collapses. The post How to Make Your First World Model from Scratch appeared first on Towards Data Science.

9/22/2026

A small adversarial test set that catches the retrieval failures your evaluation set never will The post Break Your Own RAG Pipeline Before Users Do appeared first on Towards Data Science.

9/22/2026

Learn how to effectively code up an internal tool using Claude code or Codex The post Build a Speaker-Recognition App with Claude Code appeared first on Towards Data Science.

9/22/2026

Finding citations, consolidating code, fact-checking and preparing for the defence The post 4 Ways to Use AI on a PhD Thesis appeared first on Towards Data Science.

9/22/2026

The AI that makes decisions instead of generating text The post An Introduction to Jev appeared first on Towards Data Science.

9/21/2026

Exploring TypeSafe's Jev and comparing it with OpenAI on intent classification The post A New Kind of Model for AI Decision-Making? appeared first on Towards Data Science.

9/21/2026

Why this matters less for what Astra can do and more for what every other model hasn't been tested for The post GPT-6 Astra Just Hit OpenAI's Highest Cybersecurity Risk Level appeared first on Towards Data Science.

9/21/2026

How to put a dollar value on your organization's operational data The post Google Offered $10M for a Dying Airline's Data. How Can You Value Yours? appeared first on Towards Data Science.

9/21/2026

Five scikit-learn defaults that deserve a closer look before your next model reaches production The post Your AI Assistant Wrote the Code. Who Checked the Defaults? appeared first on Towards Data Science.

9/20/2026

Beyond basic graph retrieval: six production-oriented architectures for combining semantic search, knowledge graphs, and LLM reasoning. The post GraphRAG: A Practitioner's Guide to 6 Advanced Architectural Patterns appeared first on Towards Data Science.

9/20/2026

Understanding and implementing CBAM (Convolutional Block Attention Module) from scratch with PyTorch The post CBAM Paper Walkthrough: The Double-Attention Mechanism appeared first on Towards Data Science.

9/19/2026

Deduplicating a 10,000-row supplier list in Python, where the hard part is deciding what a similarity score of 91 means The post One Vendor, Four Spellings: How Deterministic Stages Beat Similarity Scores appeared first on Towards Data Science.

9/19/2026

One near miss, four months of running agents, and the question almost nobody is asking: what are you supposed to do while the AI writes the code? The post AI Made Me 5x Faster. It Also Made Me 5x Worse at My Job. appeared first on Towards Data Science.

9/18/2026

How to verify your app aligns with your intent without ever reading a line of generated code. The post Coding Agents Keep Shipping Silent Failures — Here Is How to Catch Them appeared first on Towards Data Science.

9/18/2026

The recurring cost of production AI is not inference. It is re-qualification: the eval reruns, prompt retuning, and regression testing you owe every time a model changes under you. Here is what that tax actually covers, and how to budget for it before it surprises you. The post We Pinned Our Model Version to Stay Safe. The Provider Deprecated It Anyway. appeared first on Towards Data Science.

9/18/2026

How do you set yourself up for a career that will stand the test of time when things are changing so fast? The post Starting a Career in Data Science in the Age of AI appeared first on Towards Data Science.

9/18/2026

Multi-agent coding systems don't necessarily fail because agents can't communicate. They can fail because important commitments made in conversation have nowhere to live afterward. The post Multi-Agent Coding Isn’t Enough — Agents Need a Commitment Layer appeared first on Towards Data Science.

9/17/2026

Starting with a local Parquet file, then joining it to data stored in the cloud The post Building a Data Lakehouse with DuckDB and DuckLake appeared first on Towards Data Science.

9/17/2026

What it takes to turn counterfactual analysis into an AI product The post How I Built a Multi-Agent System for Interrupted Time Series Analysis (ITSA) appeared first on Towards Data Science.

9/17/2026

From a chocolate bar with no price tag to a marketing mix model in PyMC, and the 200-year-old integral that stood in between. The post Why You Think Like a Bayesian but Were Taught Like a Frequentist appeared first on Towards Data Science.

9/17/2026

I built four AI retrieval architectures on a laptop and benchmarked them against the same set of documents and questions. Here’s what the results taught me about the trade-offs between plain RAG, graph RAG, and simply putting everything into a frontier model’s context window. The post When Does Graph RAG Actually Add Value? A Hands-On Experiment appeared first on Towards Data Science.

9/16/2026

Robust Estimation Series: comparing classical and modern robust estimators through theory, code, and experiments The post How to Make Linear Regression Survive Outliers appeared first on Towards Data Science.

9/16/2026

PyTorch and TensorFlow tensor broadcasting: how silent shape errors cause difficult-to-debug machine learning bugs The post Silent Broadcasting Can Ruin Your Model appeared first on Towards Data Science.

9/16/2026

A VRAM budget formula for LLM serving, and three optimization strategies mapped to the traffic patterns that trigger the OOM. The post The KV Cache Tax: Why Inference Servers Run Out of Memory Before Compute appeared first on Towards Data Science.

9/16/2026

What ordering and not eating a large pizza tells us about ML memory management The post The N Squared Pizza Problem appeared first on Towards Data Science.

9/15/2026

How moving randomness outside the computation graph turns noisy gradient estimators into low-variance, differentiable ones The post Reparameterization Tricks: Variance Reduction by Smarter Gradients appeared first on Towards Data Science.

9/15/2026

Keep your apps looking professional with Claude Code design skills The post How to Build Consistent Designs with Claude Code appeared first on Towards Data Science.

9/15/2026

How statistical moments connect the mean, the variance, and higher powers of a distribution The post Seizing the Moment: The Hidden Silhouette of Data appeared first on Towards Data Science.

9/15/2026

Before reaching for an LLM API on every classification problem, it's worth knowing what a decades-old baseline can already do with the labeled data you have — and exactly how much more data buys you. The post How Many Labeled Examples Does a Text Classifier Actually Need? I Measured It. appeared first on Towards Data Science.

9/14/2026

First in a series on probabilistic forecasting for physical signals. Next: what happens when you roll the forecast forward more than one step. The post Your Model’s MSE Is Lying to You appeared first on Towards Data Science.

9/14/2026

When Codex is the right shape for the problem, when Claude Code is, and how I split 5 specialist agents between them on dense AI capacity work. The post When to Use One Model and When to Use a Team of Agents appeared first on Towards Data Science.

9/14/2026

A viral debate over loops versus graphs points to a bigger shift in how we build AI systems. Here’s what graph engineering actually means, how it differs from prompt, context, and loop engineering, and why it matters. The post Graph Engineering for AI Agents: From Prompts and Loops to Workflows appeared first on Towards Data Science.

9/14/2026

Why the skill-inflation panic is aimed at the wrong thing, and what it costs to make agent knowledge a build artifact instead of a file. The post From Static to Dynamic Skills: A Different Model for Agent Knowledge appeared first on Towards Data Science.

9/13/2026

Building a FastAPI endpoint for churn prediction, and everything that broke between "it runs" and "it's live The post Your Model Isn't Done Until Someone Else Can Call It appeared first on Towards Data Science.

9/13/2026

A practitioner's guide to estimating what an opt-in AI feature actually did, when nobody randomized it. The post Your AI Adoption Lift Is a Selection Effect appeared first on Towards Data Science.

9/12/2026

A real Weave project that regression-tests three OpenAI models against the exact reply format your app depends on. The post One Capital Letter Was Silently Breaking My AI Support Bot, and It Wasn't in the New Model appeared first on Towards Data Science.

9/12/2026

What large operators can teach us about turning alert fatigue into faster, safer service assurance The post Stop Managing Alarms: An Incident-First Blueprint for Telecom AIOps appeared first on Towards Data Science.

9/11/2026

I built a system that automatically discovers, verifies, and applies relevant requirements from earlier interactions without asking the user where they came from. The post Coding Agents Don't Need Longer History — They Need Intent Continuity appeared first on Towards Data Science.

9/11/2026

Why AI coding makes software design more important The post Software Design in the Age of AI appeared first on Towards Data Science.

9/11/2026

Frequentist confidence intervals and Bayesian credible intervals answer different questions, and confusing them can distort product decisions The post The 95% Illusion: Why Your Confidence Interval Isn't What You Think It Is appeared first on Towards Data Science.

9/11/2026

Clarifying the math behind Anthropic’s representation workspace The post Demystifying Anthropic's J-Space: A Mathematical Primer appeared first on Towards Data Science.

9/10/2026

Better understand the intent of your coding agents The post How to 5x Your Communication Effectiveness with Claude Code appeared first on Towards Data Science.

9/10/2026

Why autonomous agents expose a new explainability problem in fraud detection The post What SHAP Can't Explain About Agentic AI Fraud appeared first on Towards Data Science.

9/10/2026

Moving from static model assignment to intelligent, task-level LLM selection. The post Optimizing LLM Inference Costs in Multi-Agent Systems with Adaptive Model Routing appeared first on Towards Data Science.

9/10/2026

A model is only as reliable as the assumptions behind it The post Who Questions What Works: When Should We Retest Our Assumptions? appeared first on Towards Data Science.

9/9/2026

A practical guide to building, testing, and documenting SQL transformations The post Getting started with dbt appeared first on Towards Data Science.

9/9/2026

Understanding permutation symmetry in deep learning, and what it means for weight averaging and model merging The post The Symmetry That Breaks Neural Network Averaging appeared first on Towards Data Science.

9/9/2026

How we split a tightly coupled Python pipeline into independently deployable services The post When One Process Becomes Too Much: Splitting a Pipeline into MCP Services appeared first on Towards Data Science.

9/9/2026

Statistical thinking beyond formulas The post 10 Statistical Traps We Often Overlook appeared first on Towards Data Science.

9/8/2026

My first impressions of OpenAI's new frontier model The post How to Maximize GPT-6 Astra appeared first on Towards Data Science.

9/8/2026

How model validation standards are changing for LLM-based systems: what breaks, what carries over, and how to test output quality The post The Model Validation Playbook for GenAI: Lessons from Banking appeared first on Towards Data Science.

9/8/2026

Learn how to simulate reality with Python The post A Beginner’s Guide to World Models appeared first on Towards Data Science.

9/8/2026

Towards Data Science launches a video showcase for real-world AI work The post Introducing ShipAI appeared first on Towards Data Science.

9/8/2026

A context window can be technically complete and still describe a world that no longer exists. I built a deterministic benchmark to measure the cost of acting on stale context. The post Context Windows Don’t Know What’s Still True — I Built a Validity Layer That Does appeared first on Towards Data Science.

9/7/2026

I saved my day with a mere $52 bill. But you don't have to repeat my mistake. The post I Vibe-Coded an App in Just Two Hours (And Regretted It the Next Day) appeared first on Towards Data Science.

9/7/2026

How to catch a payload that looks correct but isn't, using a watchdog pattern with working Python. The post Why Most Multi-Agent Systems Fail Even When Evaluation Passes appeared first on Towards Data Science.

9/6/2026

AI companies quietly watermark billions of words a day. Here’s how to apply the same three families of techniques to your own writing—and what real experiments reveal about which watermarks survive copy-paste, editing, and paraphrasing. The post Text Watermarking in Python: Catch Whoever Copies Your Writing appeared first on Towards Data Science.

9/6/2026

Using LDA for dimensionality reduction in classification problems The post Linear Discriminant Analysis (LDA) in Real-Life: Dimensionality Reduction in a Real-Estate Dataset appeared first on Towards Data Science.

9/5/2026

From scalar observations to self-attention, and how positional information restores sequence order The post Why Transformers Need Positional Encoding For Time Series: A Visual Guide appeared first on Towards Data Science.

9/5/2026

Improving reinforcement learning for complex physics The post Dynamical System Transfer Learning with Reduced Order Models appeared first on Towards Data Science.

9/4/2026

Why the default 50/50 split is the wrong move when your treatment is more expensive than your control, and how cost-based sampling weights fix it The post Optimal Traffic Allocation Under Heterogeneous Variant Cost appeared first on Towards Data Science.

9/4/2026

Three conditions that must hold before splitting prefill from decode pays off, and why chunked prefill is the right default below that threshold. The post Disaggregation Is a Thousand-GPU Problem appeared first on Towards Data Science.

9/4/2026

Power BI Premium is gone. Microsoft Fabric took its place. Here's what actually changed for you, what didn't, and where to start — without the panic. The post The Power BI Developer's Survival Guide to Microsoft Fabric appeared first on Towards Data Science.

9/4/2026

Learn how to run a lot of parallel coding agents without expensive, powerful hardware at home The post How to Run 10+ Claude Code Sessions Without a Powerful Computer appeared first on Towards Data Science.

9/3/2026

Turning a trained churn classifier into a FastAPI service that other software can actually call. The post My Model Worked Perfectly. Then I Tried to Make It Useful. appeared first on Towards Data Science.

9/3/2026

Enterprise Document Intelligence [Vol.1 #B4] - A diagnostic and five composable operations, not a decision tree The post Tables in PDFs for RAG: Don’t Flatten the Grid appeared first on Towards Data Science.

9/3/2026

I built a prompt dependency graph that separates everything a component can reach from the smaller set that actually needs targeted evaluation. The post Changing One Prompt Can Affect 50 Others — I Built a Prompt Dependency Graph to Find What Needs Retesting appeared first on Towards Data Science.

9/3/2026

A practical framework for reducing uncertainty before agents accelerate implementation The post How to Solve the Right Problem in the Age of Agentic AI appeared first on Towards Data Science.

9/2/2026

I built a matcher meant to finish the cleanup that normalization left behind. Testing it against real data showed that no version of it could be made safe. What follows is the architecture that was left once the matcher was set aside. The post Avoiding Entity Key Drift in a Data Lake: Step 2, When Fuzzy Matching Stops Working appeared first on Towards Data Science.

9/2/2026

Enterprise Document Intelligence [Vol.1 #B3] - A confident wrong answer is a bug. A bare “no answer” with no justification is almost as bad. Each of the four bricks has one piece of evidence to show The post A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence appeared first on Towards Data Science.

9/2/2026

A visual guide to how graph neural networks work under the hood The post Graph Neural Networks: GCN, MPNN, and GAT, Explained Simply appeared first on Towards Data Science.

9/2/2026

Why the standard groupBy function isn’t enough The post A Practical Introduction to PySpark Window Functions appeared first on Towards Data Science.