The difference between "this works on my laptop" and "this works in production" is mostly about what happens when something breaks at 2am. You don't need a Datadog account on day one. You need three small things.
Three things to add before shipping
- A README with three sections: what it does, how to run it, what to do when it breaks. 200 words is plenty.
- A runbook for the top 3 expected failure modes. "Symptom → cause → fix." Live with the agent for a week and write this from observation.
- Logging that captures: timestamp, input, every tool call + result, final output. JSON lines is fine. Don't store secrets; redact the inputs that matter.
The cheapest possible monitoring
For week one, monitoring can be: a daily summary email or Slack message that says how many runs happened, how many failed, and a link to the log file. That's it. You don't need dashboards; you need to know when something stopped working.
Knowledge check
0/1 answered1. Which of these is the highest-leverage addition before your first user touches the agent?
Discussion
0 commentsBe the first to start the conversation.