Agentic AI & Platform Engineering — Field Notes by Ashutosh Upadhyay

Field Notes RSS

Agentic AI &
Platform Engineering

Production notes from the intersection of drug discovery, cloud infrastructure, and autonomous AI — what actually ships.


Ashutosh Upadhyay

Ashutosh Upadhyay

Principal Engineer — Agentic AI & Cloud Platform Johnson & Johnson Innovative Medicine Antwerpen, Belgium

I build the platforms that let scientists focus on curing disease instead of fighting infrastructure. 12 years in enterprise cloud, specializing in Agentic AI and production-grade platform engineering for pharmaceutical R&D on AWS — embedding AI agents and LLM-assisted workflows into drug discovery pipelines.

My edge: I don't just build platforms. I make them disappear so the science can move faster.

AWS Kubernetes / EKS GitOps Agentic AI Bioinformatics / Omics Imaging Multi-LLM Systems
Connect on LinkedIn View Full Resume
12+
Years in Enterprise Cloud
46
Published Technical Posts
40+
Tools Built for AI Agents
All AI Dev Agentic AI Multi-Agent FinOps LLM Eval Security Platform Data Eng Release Domain AI Observability Memory Learning MLOps Architecture GitOps
Posts 46 entries
🔍

No posts found

Try a different search term or clear your filters.

01

Building AI With AI on Claude Code

How I use Claude Code's agent system to build AI systems — with a mandatory Advisor that catches production bugs, persistent memory that compounds across sessions, and a full EKS deployment architecture.

Read →
02

From Chatbot to Autonomous Research System

How I built a 35-tool AI agent that autonomously queries databases, builds knowledge graphs, and generates research reports — running on Kubernetes.

Read →
03

Multi-Agent AI Collaboration

Why one agent isn't enough — and how to build a team of AI specialists that research, analyze, synthesize, and critique each other's work, with full observability via Langfuse.

Read →
04

What Your AI Agent Actually Costs

Your invoice tells you the total. It never tells you who spent it, which tokens were wasted, or whether the last model upgrade paid for itself.

Read →
05

Every Public Benchmark Is Useless to You

Building a standing internal evaluation suite — and the four ways my harness broke before it produced a single trustworthy number.

Read →
06

Your Agent Is a Confused Deputy

Security lessons from building an autonomous, tool-calling agent for regulated enterprise data — including a cross-tenant vulnerability found and closed before any such data was connected.

Read →
07

Managed Agent Runtimes: Read the Fine Print

I run agents on Kubernetes and evaluated moving them to a managed agent platform. Here's what genuinely improves, what quietly becomes your problem, and why a hybrid split was the honest answer.

Read →
08

Your Agent Is Only As Good As Its Data Pipeline

The agent gets the credit. The data pipeline does the work. Here's the production ingestion architecture that keeps a 19-source research agent reliable.

Read →
09

Releasing an AI System Is Not Like Releasing Software

A tag-only image bump left the UI, presentation, and diagrams stale — the diagram five versions behind. None of it errored. Here's what a real AI system release checklist looks like.

Read →
10

Teaching an Agent to Think Like a Domain Expert — Without Fine-Tuning

Not fine-tuning. Not prompt-stuffing. A skills system that retrieves domain heuristics at query time — and the failure modes that shaped it.

Read →
11

Observability for AI Agents: Beyond Langfuse Traces

Langfuse catches bad tool data. Prometheus catches the stuck pod. CloudWatch catches the throttling. Here's how three layers work together — and where each one falls short alone.

Read →
12

Cross-Session Agent Memory: What Persists, What Shouldn't, and Why It's Hard

Every session starts cold. Users re-teach the same preferences. Here's how to give your agent real memory without creating a privacy disaster.

Read →
13

Closing the Feedback Loop: How User Corrections Become Domain Knowledge

The agent was wrong. The user corrected it. The next user got the same wrong answer. Here's how to close the loop — without fine-tuning.

Read →
14

Ingesting 129 GB for an AI Agent: Five Design Mistakes Caught Before Build

We had a design. The advisor caught five blockers before the first build. Here's what changed and why it mattered.

Read →
15

GPU Digital Pathology AI on Kubernetes: What KEDA ScaledJobs Actually Do Under Load

Burst GPU workloads, silent Karpenter and KEDA version traps, and when a ScaledJob is the wrong primitive.

Read →
16

Multi-Tenant FinOps: The Three Silent Bugs That Made Our AWS Cost Data Untrustworthy

We built the join, ran it, got 1.8% coverage. It looked like a data problem. It was a .lower() call — and two more silent bugs followed.

Read →
17

EKS or Managed Agent Runtime? A Framework for the Decision You'll Face Twice

The migration looked straightforward. Then we found five categories of blockers. Here's the framework we wish we'd had before we started.

Read →
18

AWS DataSync for S3: Same-Account and Cross-Account Transfer Without the Surprises

The console can't see the bucket. The role lives in the account that runs the task. KMS is almost always the first thing that breaks. Here's the complete playbook.

Read →
19

The Two-Hop IAM Pattern: IRSA + Cross-Account AssumeRole on EKS

Hop 1: EKS pod gets IRSA credentials. Hop 2: assumes a role in a different account. Here's how to wire both hops without creating a credential management nightmare.

Read →
20

ArgoCD Multi-Source Helm: Decoupling Chart Versions from Configuration

One source for the chart in your registry. One source for the values in Git. They version independently — and that changes everything about how you manage environments.

Read →
21

Cross-Account CloudWatch: The Silent Zero and How to Prevent It

You query CloudWatch. You get zero results. No error. The logs are there — in a different region. Here's how to build multi-account extraction that doesn't silently fail.

Read →
22

Data Infrastructure Quality for AI Readiness: Lessons from Production Pipelines

AI systems are only as good as the data pipelines that feed them. Here's how to build infrastructure that produces genuinely AI-ready data — and the silent failures that break it before training starts.

Read →
23

Langfuse in Production: LLM Observability Beyond Console Logs

Your agent answered a question. Was it correct? How long did it take? What did it cost? Did it use the right prompt version? Langfuse answers all of those — and makes the answers actionable.

Read →
24

When Your Agent's Memory Starts Lying: Self-Poisoning in Long-Term Agent Memory

A hallucinated tool claim persisted as a behavioral rule, got injected as the user's own voice, and the model obeyed it across sessions. Here is how the loop forms and how to break it.

Read →
25

The Context-Window Overflow Was Not Where We Looked

We suspected loaded session history. Measurement proved it couldn't be. Here's how we traced the real cause — and the conversation-manager trade-offs that shaped the fix.

Read →
26

Silent Wrong Data: Eight Ways a Research Pipeline Lied Without a Single Visible Error

None of these failures surfaced an error. No alert, no visible exception, no bad log line — just quietly wrong answers delivered with full confidence.

Read →
27

One Missing Pricing Key Silently Deleted Three Days of AI Cost Data

A recognised-but-incomplete pricing entry is more dangerous than an unknown model. Here's how a single KeyError inside a pagination callback quietly discarded whole days of Bedrock invocation data — and what it took to find and recover them.

Read →
28

Upgrading an Agent Platform to Claude Opus 5 on Amazon Bedrock: Eight Gotchas We Hit

Billed for a full response, parsed an empty string. A 400 misread as "model unavailable." Thinking that burns max_tokens before the answer starts. The migration was worth it — but not without friction.

Read →
29

Measuring AI Coding-Assistant Adoption: Claude Code on Amazon Bedrock and Kiro

Two tools, two telemetry shapes — and what we learned about cost attribution, daily credits, and why "group coverage" is not "licence utilisation."

Read →
30

Agent-as-Tool: What a 2025 Paper Gets Right, and the Hand-Off Contract It Leaves Open

A review of the Agent-as-Tool paper, an honest reading of its results, and the design choice where our multi-agent research pipeline went the other way: raw structured evidence instead of processed summaries.

Read →
31

A Refusal Is an HTTP 200: Designing for Model Refusals on Amazon Bedrock

The model declined, the API said success, and every safety net built on exceptions stayed silent. How to detect refusals on both API paths and retry the unchanged request on a second model.

Read →
32

Tests That Cannot Fail: Six Ways AI-Written Test Suites Pass Whatever the Code Does

A green suite is a claim, not evidence. A taxonomy of hollow tests written by AI coding agents, the cheap real checks that expose them, and a reviewer checklist.

Read →
33

Streaming an Agent Over SSE: The Traps 192 Green Tests Hid

Tool results that never arrive, a line separator the browser cannot parse, and a generator nobody closes: found only when tests drove the real libraries.

Read →
34

An AI-Written Weekly Digest Where Code Owns Every Number

How a weekly research email keeps the model to prose: code computes every count, validators check every claim against the items it cites, and a fail-closed verifier has the last word.

Read →
35

The Request Never Arrived: How Async Python Services Freeze While Health Checks Stay Green

A spreadsheet upload that hung and restarted its pod, a lock that deadlocked its own thread, an access log that cannot see hung requests, and why a frozen worker inside a two-worker pod passes every health check.

Read →
36

Same Model, Different Answers: Moving a Research Pipeline into a Production Service

The weights matched and the tiling matched, yet the contours did not. Dependency pins as part of the model, a framework default that changed under a checkpoint, reading a multi-gigabyte checkpoint without downloading it, tokens that expire mid-job, and output formats that only the consumer can judge.

Read →
37

Your Policy Is What the Cluster Stored, Not What You Wrote

A Kyverno policy lost its negation anchors to a bare YAML null while every check passed. How to compare the stored object to Git, probe each rule kind, and tell a node replacement from a rollout.

Read →
38

The Guard That Fails Open: Your Policy Proxy Is Only as Strong as Its Exception Path

When a mitmproxy addon throws, the request is forwarded anyway. How to make a guard default-deny, classify what the server will run, place allow-exceptions after every deny check, and make config kill switches fail closed.

Read →
39

Letting an LLM Drive Your Test Framework Is Letting It Run Code

What an agent can make a test-framework MCP server do beyond testing a page, from reading the public rf-mcp source (as of 0.36.0) and Robot Framework documentation, and the controls that close each path.

Read →
40

When a Tool Result Is Too Big for the Context Window

Store the full result, keep a whole-record preview in the context, and hand the model a reference it can open. How that works for list results, for the saved conversation, for the model request and across agent hand-offs.

Read →
41

From Regex-Parsed Verdicts to Native Structured Outputs

A reviewer agent's verdict was recovered with regexes because the models rejected forced tool choice. Then the schema request I had recorded as rejected started to work. What I shipped, how I probed it, and why a capability matrix needs a date on it.

Read →
42

Human Approval for Agent Actions: Interrupts, Not Turn Re-Runs

How an agent asks permission in the middle of a tool call, survives the user walking away, and resumes exactly once, instead of ending the turn and running it again after the click.

Read →
43

Running a Fleet of AI Coding Agents Without Them Clobbering Each Other

Parallel agents are fast right up until two of them write to the same place. The rules I use for ownership, scratch space, review and peer sessions, each one learned from a specific collision.

Read →
44

Upgrading a Self-Hosted LLM-Observability Stack: The Traps Between Two Major Versions

Langfuse v3 to v4 and ClickHouse 24.8 to 26.4 on a single node. A query that was "read-only" and still got killed, a disk that was full of the wrong thing, and the checks that let me empty the old tables only after every historic trace was verified present in the new ones.

Read →
45

The Skill Said ON, the Tool Said OFF: When Prose and Behaviour Drift Apart

An agent reported a setting as on while the tool behind it had sent the opposite. The four-layer fix, why the model trusted the prose, and why a skill must not describe a capability the tool cannot deliver.

Read →
46

Adding Clarifying Questions to an Agent: Five Traps Review Caught Before Ship

A research agent should ask "which compounds?" before it spends a multi-step run finding out nobody named any. The gate that does it is simple. The five ways it would have gone wrong were all in how the gate was wired in.

Read →