AI05 · GenAI, RAG, MCP & agents
Agent Evals, Traces, Budget & Rollback
Build a golden test set, score each agent version, set a cost budget and roll back a version that regresses.
🎯 You will learn to
- Build golden sets
- Score versions
- Gate releases
📘 Guided lab
This mission is a guided walkthrough: follow the steps on your own computer or practice account. The in-browser simulator for it is planned for a later release.
✅ Steps & checkpoints
🗺️ Agent evaluation loop
Each agent version answers a golden test set; scores, cost and traces decide promote or roll back.
The 2D diagram shows the exact flow, ports and trust boundaries. The 3D scene is a concept view of the same parts; packets travel along the same arrows.
🤔 Think about it
Why version prompts and tools like code?
🛡️ Safety & cost
The agents are deterministic simulations: no model is called and no real tool is executed. Write tools always need human approval.
🧹 Cleanup
Nothing to clean up.
📚 Reference documents
- AI Risk Management Framework · NIST
- Agentic AI Lens (Well-Architected) · AWS
- Model Context Protocol · MCP
- OWASP Top 10 for LLM applications · OWASP
- Microsoft Foundry Agent Service · Microsoft
Official sources, checked on 5 Nov 2026. Cloud services change often; check the official page before relying on a feature. All references