🦀 Building AI Agents in Rust

Author(s): Enzo Lombardi Originally published on Towards AI. A provider with no socket Every provider Eugene has spoken to so far, going all the way back to Part 6, ends the same way: a URL, a header, a JSON body over HTTP. Anthropic’s Messages API, OpenAI’s Chat Completions, and even Ollama running on the same laptop as the agent all get the same treatment, because the Provider trait was built around one assumption: somewhere, there is a socket. Ollama already narrows the distance to zero latency-wise, but the shape of the call is still “make an HTTP request to localhost and wait.” This closing post asks what happens when you drop that assumption entirely and drive a local model the way you’d drive any other child process: stdin in, stdout out, no port to bind, nothing to curl while it’s still warming up. The article explains how to implement the existing Provider abstraction without any HTTP socket by keeping a local model process (DwarfStar’s ds4 REPL) alive and communicating via stdin/stdout. It contrasts the usual HTTP-style “one request per turn” approach with a pipe-based design that preserves session state and avoids re-sending the full transcript each turn, using a mutex-protected child process and logic to read until the REPL prompt reappears. It also clarifies a key boundary: this pipe integration outputs plain text only and doesn’t support tool calling, so the agent loop behaves correctly by taking the “no ToolCall emitted” path. Finally, it argues that ds4-server is preferable when tool calling and multiple clients are needed, while the no-socket pipe route fits narrower use cases like CI jobs, sandboxed evaluations, or fast local inference for “Fast-tier” tasks, and notes how the new adapter is added to Eugene’s provider workspace. Read the full blog for free on Medium. Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor. Published via Towards AI