Zubin Koticha · 03:23 "The moment humans can no longer monitor agents, the agents are already far beyond where we are."
Put an AI agent into production and you can't predict what will break. The input space is infinite, agents reach into the outside world through tools, and sessions run for hours. The premise of this 50-minute discussion is that extending the conventional eval (unit test) paradigm is no longer enough. The video presents the paradigm shift from "testing to monitoring," along with implementation methods across two tracks — implicit and explicit signals — paired with a live workshop driving an actual coding agent.
The speakers are two from Raindrop: Zubin Koticha (Raindrop's co-founder and CEO, a second-time founder whose previous company was acquired by Coinbase) and Danny Gollapalli (the company's backend engineer and SDK lead). Raindrop positions itself as "Sentry for AI agents" and is a monitoring infrastructure startup; it closed a $15M (roughly 22 billion yen) seed round led by Lightspeed Venture Partners in December 2025.
The problem framing is sharp. Agents (A) recursively call sub-agents, so the combinatorial space explodes; (B) run autonomously for hours without user input, so there is no fixed moment at which to observe; (C) are already deployed in healthcare, finance, and the military, where failures can be catastrophic. Even with a golden dataset of "input-output test sets," covering every edge case is impossible in principle.
The answer is signal design. Explicit signals (verifiably true or false — error rate, latency, user regeneration, cost) and implicit signals (semantic judgments — refusals, task failure, user dissatisfaction, NSFW, jailbreaks, and even "positive wins"). Three ways to implement implicit signals: regex, binary classifiers, and self-diagnosis (asking the agent to introspect). Collected signals flow into alerts and A/B experiments. This is the shape of "monitoring" in the agent era.
Key observations
"Humanity's last problem" — a reading of the era (03:14)
Prefacing it as a controversial framing, Zubin offers the thesis: "The moment humans can no longer monitor agents and find problems, the agents are already far beyond where we are." Observability is not merely an engineering technique but "the last line of defense for humans to keep up with AI's progression." A fresh framing that translates the safety conversation from "alignment is hard" into a concrete threshold: "if monitoring breaks, we're out."
The Claude Code source code leak contained regex (06:38)
About five weeks earlier (leak of March 31, 2026 — the npm package `@anthropic-ai/claude-code` v2.1.88 included .map files exposing all 513,000 lines of source code), Anthropic's Claude Code source was leaked. Inside was a long regex file called `userPromptKeywords.ts`, designed to catch signs that users are upset (phrases like "terrible," "worst"). "Even Anthropic uses regex for monitoring — so regex is a sufficiently strong signal." A skillful use of field evidence to back the speakers' rule of thumb with the implementation from an industry leader.
"Models aren't trained to criticize themselves" (22:39)
A pitfall when implementing self-diagnosis tools. LLMs have been RLHFed to polish their outputs heavily, so "frankly reporting their own mistakes" is something they're bad at. Without careful tool naming and descriptions, agents do not call self-diagnosis tools. In the live workshop, they plant a permission error in a write tool, demonstrate the agent attempting a bash HEREDOC bypass, and show the prompt and tool-description tweaks needed before the agent finally calls self-diagnosis. A moment where the gritty work of fine-tuning prompts and tool descriptions takes the foreground, rather than abstract argument.
Video outline
- (00:00) Introductions; explanation of Raindrop — "Sentry for AI agents"
- (00:50) How agent failures differ from conventional software failures — nondeterministic, infinite input/output space, tools reaching into the outside world
- (01:14) Why the problem will worsen — increasing complexity, long sessions, high stakes (healthcare, finance, military)
- (01:52) Limits of the eval paradigm — covering every edge case is infeasible
- (02:52) The paradigm shift from testing to monitoring
- (03:14) "Humanity's last problem" — once we can't monitor, agents have already gone ahead
- (03:46) Signal design — two tracks: implicit vs. explicit
- (04:02) Examples of explicit signals — error rate, latency, regenerations, cost
- (04:25) Three implementations of implicit signals — regex, classifiers, self-diagnosis
- (05:11) Signals Raindrop provides by default — refusal, task failure, dissatisfaction, NSFW, jailbreak, "win"
- (06:38) Lessons from the Claude Code leak (March 31, 2026) — even Anthropic monitors with regex
- (07:30) From signals to A/B experiments — partially shipping prompt changes or new tools and measuring effect
- (20:16) Second half: the live workshop begins — adding self-diagnosis to a coding agent in a public repository
- (22:39) Models don't want to criticize themselves; crafting tool descriptions
- (48:57) Q&A — whether open data is offered, compliance constraints
- (50:00) Closing
Sources
Everything You Need To Know About Agent Observability — Raindrop (AI Engineer)
Raindrop official: raindrop.ai · $15M seed (December 2025): PR Newswire
Note: the official YouTube title lists "Danny Gollapalli and Ben Hylak," but in the video's opening introduction we hear Zubin (CEO and co-founder) and Danny. Zubin Koticha is Raindrop's current CEO and co-founder. This article treats the SRT as primary information and records the talk as a joint appearance by Zubin and Danny.