Tag: agentic

  • Why AI Workflows Need Audit Trails

    Why AI Workflows Need Audit Trails

    The conversation around AI has shifted. It is no longer just about drafting text or summarising meetings. More and more often, these systems are taking actions on live business processes.

    That is where the risk changes shape.

    A clever prompt can produce a good-looking result. It cannot tell you what happened after the fact if the output was wrong.

    The gap

    When a person makes a decision in a process, there is usually some trace of it. An email, a ticket, a sign-off, a log entry. With an AI system, that trace is often thin or missing.

    If the system updates the wrong record or sends the wrong message, the team is left with a result and very little explanation.

    What breaks without logs

    • You cannot reconstruct the sequence of events.
    • You cannot show who approved what.
    • You cannot improve the workflow with confidence.

    That is not just a debugging problem. It is an accountability problem.

    What a useful audit trail looks like

    At minimum, every execution should capture:

    • the input it received
    • the steps it took
    • the action it actually executed
    • the output it produced
    • the timestamp and identity of the run

    That is enough to answer the questions that matter later.

    The practical bit

    The tooling is already there. Structured logs, append-only storage, and reviewable traces are all enough to get started. The main thing is to design for visibility before the workflow is under pressure.

    If the system is allowed to act, it should also be required to explain itself.

  • Containment Patterns for Autonomous AI Agents

    Containment Patterns for Autonomous AI Agents

    You wouldn’t give a new employee the keys to every office, the admin password to every system, and a company credit card on day one. You’d onboard them gradually — scoped access, supervised actions, and a clear audit trail of what they did and why.

    Autonomous AI agents deserve the same treatment. Arguably more, because they can act faster, at greater scale, and with less inherent judgment than any human hire.

    Yet the pattern I see repeatedly is teams wiring agents directly into production systems with broad API keys, unrestricted network access, and no oversight. The agent works brilliantly — until it doesn’t. And when it goes wrong, nobody can explain what happened or why.

    This is a containment problem. And it’s solvable.

    Why containment matters now

    Autonomous agents are moving from demos into production. They’re reading emails, writing database records, triggering deployments, and calling external APIs. Each of those actions carries real business risk.

    The uncomfortable truth is that today’s agents are semi-trusted at best. Hallucinations aren’t just a chatbot annoyance — they’re an action an agent might take on a live system. A misinterpreted prompt can become a bulk delete. A confused tool call can cascade into an outage.

    Regulators are watching. The EU AI Act’s transparency and risk management requirements are already in force for high-risk systems. UK boards are starting to ask pointed questions about AI governance. If you can’t explain what your agents did and what guardrails were in place, you have a problem that no amount of prompt engineering will fix.

    The five containment patterns

    The patterns below are drawn from how we’d sandbox any untrusted or semi-trusted process — adapted for the specific risks that autonomous agents introduce. I’ve been implementing these in SentinelForge, a platform for running agents with strict governance, auditing, and observability.

    1. Process isolation

    Agents should run in their own isolated environment — a separate container or VM, not on a shared host alongside your production workloads.

    This means:

    • Dedicated runtime per agent or agent class, with its own filesystem and process namespace
    • Resource limits (CPU, memory, disk) so a runaway agent can’t starve other services
    • Immutable infrastructure — the agent’s environment is defined in code, rebuilt from scratch each run, and never patched in place

    If the agent is compromised or behaves unexpectedly, the blast radius stops at the container boundary.

    2. Network segmentation

    An agent should only be able to reach the endpoints it explicitly needs. Nothing more.

    In practice:

    • Egress filtering — whitelist approved domains and IP ranges; block everything else
    • No direct access to internal networks, metadata services (like cloud instance metadata endpoints), or other agents’ environments
    • DNS-level controls to prevent DNS exfiltration or calls to unexpected services

    Think of it like a corporate network with VLANs. The marketing intern doesn’t need access to the finance database. Your agent doesn’t need access to the public internet.

    3. Credential scoping

    This is where most agent deployments go wrong. A single long-lived API key with broad permissions is the agent equivalent of a master key.

    Instead:

    • Time-limited tokens — credentials that expire after a task or session, not months
    • Minimum privilege — if the agent only needs to read from a database, it doesn’t get write access
    • Brokered access — credentials are injected at runtime by a secrets manager, never stored in the agent’s environment or prompt
    • Per-task credentials — different actions get different tokens, so a compromise in one area doesn’t cascade

    4. Output validation

    Before an agent’s action hits a real system, it should pass through a validation layer.

    This means:

    • Schema validation — does the agent’s requested action match an expected structure?
    • Policy checks — is this action allowed given the current context, user, and risk level?
    • Human-in-the-loop gates for high-risk actions (bulk operations, deletions, external communications)
    • Dry-run capability — the ability to simulate what the agent would do before it does it

    This isn’t about slowing agents down. It’s about making sure the action that executes is the action you intended.

    5. Audit logging

    Every action an agent takes should be recorded in a tamper-evident log that’s separate from the agent itself.

    Capture:

    • What the agent did (the action, the target, the parameters)
    • Why it did it (the prompt, the reasoning chain, the tool call context)
    • When it happened (with reliable, centralised timestamps)
    • What the outcome was (success, failure, validation result)

    This log is your forensic trail when something goes wrong. It’s also your evidence for compliance, board reporting, and continuous improvement.

    A practical containment checklist

    If you’re deploying autonomous agents today, here’s a quick audit:

    • [ ] Each agent runs in an isolated container or VM with resource limits
    • [ ] Network egress is restricted to an explicit allowlist
    • [ ] No long-lived credentials — tokens are time-limited and minimum-privilege
    • [ ] Credentials are brokered at runtime, never hardcoded or stored in prompts
    • [ ] Agent actions pass through a validation or policy layer before execution
    • [ ] High-risk actions require human approval
    • [ ] All actions are logged to a tamper-evident, centralised audit trail
    • [ ] Logs are reviewed regularly, not just when something goes wrong

    If you can’t check at least six of those boxes, your agents are running with more freedom than most of your employees.

    The principle is simple

    Autonomous agents are powerful. That power needs guardrails — not because the technology is malicious, but because it’s fast, scalable, and imperfect. The same qualities that make agents valuable are the qualities that make containment essential.

    The patterns aren’t exotic. They’re the same ones we’ve used for decades to manage untrusted code, third-party integrations, and new employees. The difference is that agents can act at machine speed, so your guardrails need to be just as fast and just as automated.

    This is the problem I’m working on with SentinelForge — making these patterns practical and deployable, not just theoretical.

    If agent containment is on your roadmap, get in touch or take a look at the projects I’m building in this space.

  • Building a Secure Agent Control Plane With Observability From Day One

    Building a Secure Agent Control Plane With Observability From Day One

    AI agents are being connected to real business systems at pace. Email inboxes. CRM records. Finance platforms. Document stores. In many organisations, agents now read, write, and decide across these systems — not as a pilot, but as operational infrastructure.

    The problem is that the operational infrastructure around those agents has not kept pace. Teams connect agents to sensitive systems the same way they would connect a chatbot: authenticate, test, deploy. There is no logging layer, no approval gate, no cost tracking, no failure recovery. The agent works until it does not, and when it breaks, nobody knows why.

    The agents themselves are often well-designed. The prompts are thoughtful. The integrations function. But there is no control plane — no layer between the agent and the systems it touches that handles authorisation, logging, rate limiting, approval, and monitoring. Without one, you are running production workloads with no operations team.

    What an Agent Control Plane Actually Is

    An agent control plane is the operational layer that sits between your AI agents and the systems they interact with. It is not a single tool. It is a set of components that collectively ensure agents act safely, their actions are visible, and failures are recoverable.

    Think of it the way you would think about any production system. A web application has authentication, logging, rate limiting, monitoring, and alerting. A database has access controls, backup procedures, and performance monitoring. An agent that reads client records, sends emails, and updates financial data needs the same — arguably more, because its behaviour is less deterministic than a traditional application.

    The control plane answers a set of questions that every production system must answer: Who is allowed to do what? What actually happened? How much did it cost? What do we do when it fails? And can we prove all of this to an auditor, a regulator, or a board?

    The Six Components You Need

    Authentication broker. Agents need credentials to access systems, but those credentials should not live inside the agent or its prompt. A central broker issues scoped, time-limited tokens with minimum permissions. When the token expires, access stops. This is service account management applied to agents.

    Audit logger. Every agent action must be logged in a structured, tamper-evident format. Not just the final output — the full chain: what the agent was asked, which tools it called, what decisions it made, what it executed, and what it returned. Teams log the input and output but miss the intermediate steps. When an agent takes an unexpected action, you need to trace the reasoning that led to it.

    Approval workflow. Not every action should be autonomous. A control plane defines which actions require human approval before execution. Sending an internal summary might be automated. Sending a client-facing email or modifying a financial record should require sign-off. The approval gate sits in the control plane, not in the agent’s prompt — a prompt is a request, not an enforcement mechanism.

    Observability. Tools like Langfuse are designed for this. Observability for agents means tracking execution traces, token usage, latency, error rates, and cost per execution. It means answering questions like: which workflow failed most often last week? Which agent is consuming the most tokens? Is the error rate increasing?

    Cost tracking. Agentic AI is not free. Every model call, every tool invocation, every token consumed has a cost. Without cost tracking per agent, per workflow, and per execution, you cannot budget, optimise, or identify waste.

    Failure recovery. Agents fail. Models time out. APIs return errors. A control plane defines what happens: does it retry? Escalate to a human? Roll back? Without explicit failure recovery, a failed agent either silently drops the task or retries indefinitely, burning tokens and creating duplicate actions.

    A Concrete Example

    Consider an n8n workflow that processes incoming client data: it reads a form submission, enriches the record with data from an external API, updates the CRM, and sends a confirmation email.

    Without a control plane, this workflow runs with a static API key, no logging beyond n8n’s default execution history, no approval gate for the email send, and no cost tracking. If the CRM API returns an error, the workflow fails silently. If the agent sends a confirmation email to the wrong address because the form data was malformed, there is no record of what data it saw or why it made that decision.

    With a control plane, the same workflow runs with a scoped token from the authentication broker, full execution tracing through Langfuse, an approval gate that holds the email send when the confidence score is below a threshold, and automatic retry with escalation on failure. Every execution is logged with its full decision chain. Cost per execution is tracked and surfaced on a dashboard.

    The workflow is the same. The operational posture is completely different.

    Why From Day One Matters

    Retrofitting observability and security into a live system is significantly more expensive than building it in from the start. Not just in money — in time, risk, and organisational friction.

    When you add a control plane after agents are already in production, you must modify every existing workflow to route through the new layer, backfill logs for actions that already happened, and negotiate with teams accustomed to unfettered agent access. In practice, this work costs roughly three times what it would have cost to build the control plane first — before accounting for the risk exposure during the period agents ran without controls.

    Building it from day one means the control plane is part of the deployment process. Every new agent is onboarded through it. Every execution is logged from the first run. Every cost is tracked from the first token. No migration, no retrofitting, no gap.

    What to Do Next

    If you are running agentic AI in your organisation — or planning to — here is where to start.

    1. Inventory your agents. List every agent, workflow, and automated process that takes action on a business system. Include what systems it accesses, what credentials it uses, and what actions it can perform. You cannot secure what you have not catalogued.

    2. Add structured logging. For every agent execution, log the input, the decision chain, the action taken, the output, and the timestamp. Use a consistent schema. Send these logs to a central, append-only store. If you are using n8n, configure each node to log its input and output. Integrate a tracing tool like Langfuse from the start.

    3. Implement approval gates for high-risk actions. Define which actions require human sign-off: anything client-facing, anything that modifies financial data, anything that changes system configuration. Build the approval gate into the control plane, not into the agent’s prompt.

    4. Set up observability. Deploy an observability layer that tracks execution traces, error rates, token usage, and cost per workflow. Build dashboards. Set alerts for anomalies — sudden cost increases, rising error rates, unusual execution patterns.

    5. Define failure recovery. For every workflow, document what happens on failure. Retry logic, escalation paths, rollback procedures. Test these before you need them.

    The organisations that will extract the most value from agentic AI are not the ones that move fastest. They are the ones that move safely — with the infrastructure to observe, control, and recover from the inevitable failures.


    If you are building agentic systems and need help designing the control plane, observability, and security architecture around them, the AI & Automation Architecture service covers exactly this. For examples of what this looks like in practice, see recent projects.

  • Building a Secure Agent Control Plane With Observability From Day One

    Building a Secure Agent Control Plane With Observability From Day One

    AI agents are being connected to real business systems at pace. Email inboxes. CRM records. Finance platforms. Document stores. In many organisations, agents now read, write, and decide across these systems — not as a pilot, but as operational infrastructure.

    The problem is that the operational infrastructure around those agents has not kept pace. Teams connect agents to sensitive systems the same way they would connect a chatbot: authenticate, test, deploy. There is no logging layer, no approval gate, no cost tracking, no failure recovery. The agent works until it does not, and when it breaks, nobody knows why.

    The agents themselves are often well-designed. The prompts are thoughtful. The integrations function. But there is no control plane — no layer between the agent and the systems it touches that handles authorisation, logging, rate limiting, approval, and monitoring. Without one, you are running production workloads with no operations team.

    What an Agent Control Plane Actually Is

    An agent control plane is the operational layer that sits between your AI agents and the systems they interact with. It is not a single tool. It is a set of components that collectively ensure agents act safely, their actions are visible, and failures are recoverable.

    Think of it the way you would think about any production system. A web application has authentication, logging, rate limiting, monitoring, and alerting. A database has access controls, backup procedures, and performance monitoring. An agent that reads client records, sends emails, and updates financial data needs the same — arguably more, because its behaviour is less deterministic than a traditional application.

    The control plane answers a set of questions that every production system must answer: Who is allowed to do what? What actually happened? How much did it cost? What do we do when it fails? And can we prove all of this to an auditor, a regulator, or a board?

    The Six Components You Need

    Authentication broker. Agents need credentials to access systems, but those credentials should not live inside the agent or its prompt. A central broker issues scoped, time-limited tokens with minimum permissions. When the token expires, access stops. This is service account management applied to agents.

    Audit logger. Every agent action must be logged in a structured, tamper-evident format. Not just the final output — the full chain: what the agent was asked, which tools it called, what decisions it made, what it executed, and what it returned. Teams log the input and output but miss the intermediate steps. When an agent takes an unexpected action, you need to trace the reasoning that led to it.

    Approval workflow. Not every action should be autonomous. A control plane defines which actions require human approval before execution. Sending an internal summary might be automated. Sending a client-facing email or modifying a financial record should require sign-off. The approval gate sits in the control plane, not in the agent’s prompt — a prompt is a request, not an enforcement mechanism.

    Observability. Tools like Langfuse are designed for this. Observability for agents means tracking execution traces, token usage, latency, error rates, and cost per execution. It means answering questions like: which workflow failed most often last week? Which agent is consuming the most tokens? Is the error rate increasing?

    Cost tracking. Agentic AI is not free. Every model call, every tool invocation, every token consumed has a cost. Without cost tracking per agent, per workflow, and per execution, you cannot budget, optimise, or identify waste.

    Failure recovery. Agents fail. Models time out. APIs return errors. A control plane defines what happens: does it retry? Escalate to a human? Roll back? Without explicit failure recovery, a failed agent either silently drops the task or retries indefinitely, burning tokens and creating duplicate actions.

    A Concrete Example

    Consider an n8n workflow that processes incoming client data: it reads a form submission, enriches the record with data from an external API, updates the CRM, and sends a confirmation email.

    Without a control plane, this workflow runs with a static API key, no logging beyond n8n’s default execution history, no approval gate for the email send, and no cost tracking. If the CRM API returns an error, the workflow fails silently. If the agent sends a confirmation email to the wrong address because the form data was malformed, there is no record of what data it saw or why it made that decision.

    With a control plane, the same workflow runs with a scoped token from the authentication broker, full execution tracing through Langfuse, an approval gate that holds the email send when the confidence score is below a threshold, and automatic retry with escalation on failure. Every execution is logged with its full decision chain. Cost per execution is tracked and surfaced on a dashboard.

    The workflow is the same. The operational posture is completely different.

    Why From Day One Matters

    Retrofitting observability and security into a live system is significantly more expensive than building it in from the start. Not just in money — in time, risk, and organisational friction.

    When you add a control plane after agents are already in production, you must modify every existing workflow to route through the new layer, backfill logs for actions that already happened, and negotiate with teams accustomed to unfettered agent access. In practice, this work costs roughly three times what it would have cost to build the control plane first — before accounting for the risk exposure during the period agents ran without controls.

    Building it from day one means the control plane is part of the deployment process. Every new agent is onboarded through it. Every execution is logged from the first run. Every cost is tracked from the first token. No migration, no retrofitting, no gap.

    What to Do Next

    If you are running agentic AI in your organisation — or planning to — here is where to start.

    1. Inventory your agents. List every agent, workflow, and automated process that takes action on a business system. Include what systems it accesses, what credentials it uses, and what actions it can perform. You cannot secure what you have not catalogued.

    2. Add structured logging. For every agent execution, log the input, the decision chain, the action taken, the output, and the timestamp. Use a consistent schema. Send these logs to a central, append-only store. If you are using n8n, configure each node to log its input and output. Integrate a tracing tool like Langfuse from the start.

    3. Implement approval gates for high-risk actions. Define which actions require human sign-off: anything client-facing, anything that modifies financial data, anything that changes system configuration. Build the approval gate into the control plane, not into the agent’s prompt.

    4. Set up observability. Deploy an observability layer that tracks execution traces, error rates, token usage, and cost per workflow. Build dashboards. Set alerts for anomalies — sudden cost increases, rising error rates, unusual execution patterns.

    5. Define failure recovery. For every workflow, document what happens on failure. Retry logic, escalation paths, rollback procedures. Test these before you need them.

    The organisations that will extract the most value from agentic AI are not the ones that move fastest. They are the ones that move safely — with the infrastructure to observe, control, and recover from the inevitable failures.


    If you are building agentic systems and need help designing the control plane, observability, and security architecture around them, the AI & Automation Architecture service covers exactly this. For examples of what this looks like in practice, see recent projects.

  • Building a Secure Agent Control Plane With Observability From Day One

    Building a Secure Agent Control Plane With Observability From Day One

    AI agents are being connected to real business systems at pace. Email inboxes. CRM records. Finance platforms. Document stores. In many organisations, agents now read, write, and decide across these systems — not as a pilot, but as operational infrastructure.

    The problem is that the operational infrastructure around those agents has not kept pace. Teams connect agents to sensitive systems the same way they would connect a chatbot: authenticate, test, deploy. There is no logging layer, no approval gate, no cost tracking, no failure recovery. The agent works until it does not, and when it breaks, nobody knows why.

    The agents themselves are often well-designed. The prompts are thoughtful. The integrations function. But there is no control plane — no layer between the agent and the systems it touches that handles authorisation, logging, rate limiting, approval, and monitoring. Without one, you are running production workloads with no operations team.

    What an Agent Control Plane Actually Is

    An agent control plane is the operational layer that sits between your AI agents and the systems they interact with. It is not a single tool. It is a set of components that collectively ensure agents act safely, their actions are visible, and failures are recoverable.

    Think of it the way you would think about any production system. A web application has authentication, logging, rate limiting, monitoring, and alerting. A database has access controls, backup procedures, and performance monitoring. An agent that reads client records, sends emails, and updates financial data needs the same — arguably more, because its behaviour is less deterministic than a traditional application.

    The control plane answers a set of questions that every production system must answer: Who is allowed to do what? What actually happened? How much did it cost? What do we do when it fails? And can we prove all of this to an auditor, a regulator, or a board?

    The Six Components You Need

    Authentication broker. Agents need credentials to access systems, but those credentials should not live inside the agent or its prompt. A central broker issues scoped, time-limited tokens with minimum permissions. When the token expires, access stops. This is service account management applied to agents.

    Audit logger. Every agent action must be logged in a structured, tamper-evident format. Not just the final output — the full chain: what the agent was asked, which tools it called, what decisions it made, what it executed, and what it returned. Teams log the input and output but miss the intermediate steps. When an agent takes an unexpected action, you need to trace the reasoning that led to it.

    Approval workflow. Not every action should be autonomous. A control plane defines which actions require human approval before execution. Sending an internal summary might be automated. Sending a client-facing email or modifying a financial record should require sign-off. The approval gate sits in the control plane, not in the agent’s prompt — a prompt is a request, not an enforcement mechanism.

    Observability. Tools like Langfuse are designed for this. Observability for agents means tracking execution traces, token usage, latency, error rates, and cost per execution. It means answering questions like: which workflow failed most often last week? Which agent is consuming the most tokens? Is the error rate increasing?

    Cost tracking. Agentic AI is not free. Every model call, every tool invocation, every token consumed has a cost. Without cost tracking per agent, per workflow, and per execution, you cannot budget, optimise, or identify waste.

    Failure recovery. Agents fail. Models time out. APIs return errors. A control plane defines what happens: does it retry? Escalate to a human? Roll back? Without explicit failure recovery, a failed agent either silently drops the task or retries indefinitely, burning tokens and creating duplicate actions.

    A Concrete Example

    Consider an n8n workflow that processes incoming client data: it reads a form submission, enriches the record with data from an external API, updates the CRM, and sends a confirmation email.

    Without a control plane, this workflow runs with a static API key, no logging beyond n8n’s default execution history, no approval gate for the email send, and no cost tracking. If the CRM API returns an error, the workflow fails silently. If the agent sends a confirmation email to the wrong address because the form data was malformed, there is no record of what data it saw or why it made that decision.

    With a control plane, the same workflow runs with a scoped token from the authentication broker, full execution tracing through Langfuse, an approval gate that holds the email send when the confidence score is below a threshold, and automatic retry with escalation on failure. Every execution is logged with its full decision chain. Cost per execution is tracked and surfaced on a dashboard.

    The workflow is the same. The operational posture is completely different.

    Why From Day One Matters

    Retrofitting observability and security into a live system is significantly more expensive than building it in from the start. Not just in money — in time, risk, and organisational friction.

    When you add a control plane after agents are already in production, you must modify every existing workflow to route through the new layer, backfill logs for actions that already happened, and negotiate with teams accustomed to unfettered agent access. In practice, this work costs roughly three times what it would have cost to build the control plane first — before accounting for the risk exposure during the period agents ran without controls.

    Building it from day one means the control plane is part of the deployment process. Every new agent is onboarded through it. Every execution is logged from the first run. Every cost is tracked from the first token. No migration, no retrofitting, no gap.

    What to Do Next

    If you are running agentic AI in your organisation — or planning to — here is where to start.

    1. Inventory your agents. List every agent, workflow, and automated process that takes action on a business system. Include what systems it accesses, what credentials it uses, and what actions it can perform. You cannot secure what you have not catalogued.

    2. Add structured logging. For every agent execution, log the input, the decision chain, the action taken, the output, and the timestamp. Use a consistent schema. Send these logs to a central, append-only store. If you are using n8n, configure each node to log its input and output. Integrate a tracing tool like Langfuse from the start.

    3. Implement approval gates for high-risk actions. Define which actions require human sign-off: anything client-facing, anything that modifies financial data, anything that changes system configuration. Build the approval gate into the control plane, not into the agent’s prompt.

    4. Set up observability. Deploy an observability layer that tracks execution traces, error rates, token usage, and cost per workflow. Build dashboards. Set alerts for anomalies — sudden cost increases, rising error rates, unusual execution patterns.

    5. Define failure recovery. For every workflow, document what happens on failure. Retry logic, escalation paths, rollback procedures. Test these before you need them.

    The organisations that will extract the most value from agentic AI are not the ones that move fastest. They are the ones that move safely — with the infrastructure to observe, control, and recover from the inevitable failures.


    If you are building agentic systems and need help designing the control plane, observability, and security architecture around them, the AI & Automation Architecture service covers exactly this. For examples of what this looks like in practice, see recent projects.

  • Building a Secure Agent Control Plane With Observability From Day One

    Building a Secure Agent Control Plane With Observability From Day One

    AI agents are being connected to real business systems at pace. Email inboxes. CRM records. Finance platforms. Document stores. In many organisations, agents now read, write, and decide across these systems — not as a pilot, but as operational infrastructure.

    The problem is that the operational infrastructure around those agents has not kept pace. Teams connect agents to sensitive systems the same way they would connect a chatbot: authenticate, test, deploy. There is no logging layer, no approval gate, no cost tracking, no failure recovery. The agent works until it does not, and when it breaks, nobody knows why.

    The agents themselves are often well-designed. The prompts are thoughtful. The integrations function. But there is no control plane — no layer between the agent and the systems it touches that handles authorisation, logging, rate limiting, approval, and monitoring. Without one, you are running production workloads with no operations team.

    What an Agent Control Plane Actually Is

    An agent control plane is the operational layer that sits between your AI agents and the systems they interact with. It is not a single tool. It is a set of components that collectively ensure agents act safely, their actions are visible, and failures are recoverable.

    Think of it the way you would think about any production system. A web application has authentication, logging, rate limiting, monitoring, and alerting. A database has access controls, backup procedures, and performance monitoring. An agent that reads client records, sends emails, and updates financial data needs the same — arguably more, because its behaviour is less deterministic than a traditional application.

    The control plane answers a set of questions that every production system must answer: Who is allowed to do what? What actually happened? How much did it cost? What do we do when it fails? And can we prove all of this to an auditor, a regulator, or a board?

    The Six Components You Need

    Authentication broker. Agents need credentials to access systems, but those credentials should not live inside the agent or its prompt. A central broker issues scoped, time-limited tokens with minimum permissions. When the token expires, access stops. This is service account management applied to agents.

    Audit logger. Every agent action must be logged in a structured, tamper-evident format. Not just the final output — the full chain: what the agent was asked, which tools it called, what decisions it made, what it executed, and what it returned. Teams log the input and output but miss the intermediate steps. When an agent takes an unexpected action, you need to trace the reasoning that led to it.

    Approval workflow. Not every action should be autonomous. A control plane defines which actions require human approval before execution. Sending an internal summary might be automated. Sending a client-facing email or modifying a financial record should require sign-off. The approval gate sits in the control plane, not in the agent’s prompt — a prompt is a request, not an enforcement mechanism.

    Observability. Tools like Langfuse are designed for this. Observability for agents means tracking execution traces, token usage, latency, error rates, and cost per execution. It means answering questions like: which workflow failed most often last week? Which agent is consuming the most tokens? Is the error rate increasing?

    Cost tracking. Agentic AI is not free. Every model call, every tool invocation, every token consumed has a cost. Without cost tracking per agent, per workflow, and per execution, you cannot budget, optimise, or identify waste.

    Failure recovery. Agents fail. Models time out. APIs return errors. A control plane defines what happens: does it retry? Escalate to a human? Roll back? Without explicit failure recovery, a failed agent either silently drops the task or retries indefinitely, burning tokens and creating duplicate actions.

    A Concrete Example

    Consider an n8n workflow that processes incoming client data: it reads a form submission, enriches the record with data from an external API, updates the CRM, and sends a confirmation email.

    Without a control plane, this workflow runs with a static API key, no logging beyond n8n’s default execution history, no approval gate for the email send, and no cost tracking. If the CRM API returns an error, the workflow fails silently. If the agent sends a confirmation email to the wrong address because the form data was malformed, there is no record of what data it saw or why it made that decision.

    With a control plane, the same workflow runs with a scoped token from the authentication broker, full execution tracing through Langfuse, an approval gate that holds the email send when the confidence score is below a threshold, and automatic retry with escalation on failure. Every execution is logged with its full decision chain. Cost per execution is tracked and surfaced on a dashboard.

    The workflow is the same. The operational posture is completely different.

    Why From Day One Matters

    Retrofitting observability and security into a live system is significantly more expensive than building it in from the start. Not just in money — in time, risk, and organisational friction.

    When you add a control plane after agents are already in production, you must modify every existing workflow to route through the new layer, backfill logs for actions that already happened, and negotiate with teams accustomed to unfettered agent access. In practice, this work costs roughly three times what it would have cost to build the control plane first — before accounting for the risk exposure during the period agents ran without controls.

    Building it from day one means the control plane is part of the deployment process. Every new agent is onboarded through it. Every execution is logged from the first run. Every cost is tracked from the first token. No migration, no retrofitting, no gap.

    What to Do Next

    If you are running agentic AI in your organisation — or planning to — here is where to start.

    1. Inventory your agents. List every agent, workflow, and automated process that takes action on a business system. Include what systems it accesses, what credentials it uses, and what actions it can perform. You cannot secure what you have not catalogued.

    2. Add structured logging. For every agent execution, log the input, the decision chain, the action taken, the output, and the timestamp. Use a consistent schema. Send these logs to a central, append-only store. If you are using n8n, configure each node to log its input and output. Integrate a tracing tool like Langfuse from the start.

    3. Implement approval gates for high-risk actions. Define which actions require human sign-off: anything client-facing, anything that modifies financial data, anything that changes system configuration. Build the approval gate into the control plane, not into the agent’s prompt.

    4. Set up observability. Deploy an observability layer that tracks execution traces, error rates, token usage, and cost per workflow. Build dashboards. Set alerts for anomalies — sudden cost increases, rising error rates, unusual execution patterns.

    5. Define failure recovery. For every workflow, document what happens on failure. Retry logic, escalation paths, rollback procedures. Test these before you need them.

    The organisations that will extract the most value from agentic AI are not the ones that move fastest. They are the ones that move safely — with the infrastructure to observe, control, and recover from the inevitable failures.


    If you are building agentic systems and need help designing the control plane, observability, and security architecture around them, the AI & Automation Architecture service covers exactly this. For examples of what this looks like in practice, see recent projects.

  • Why AI Control Planes Need Observability From Day One

    Why AI Control Planes Need Observability From Day One

    AI agents are getting plugged into email, CRM, finance, and document systems faster than most teams can build the operations around them.

    That is the mismatch.

    The agent might be fine. The prompt might be fine. The integration might even work. What is often missing is the layer that says who can do what, what happened, what it cost, and how the system recovers when something breaks.

    What the control plane does

    Think of it as the operational wrapper around the agent. It handles:

    • authentication and scoped access
    • structured logging
    • approval gates for risky actions
    • observability and cost tracking
    • failure recovery

    That is not overkill. That is the minimum if the agent is touching production data.

    A concrete example

    Imagine a workflow that reads a form, enriches a record, updates a CRM, and sends a confirmation email.

    Without the wrapper, you get a static key, weak logs, and no real visibility when something goes wrong.

    With it, you get scoped access, review points, full traces, and a clearer answer when someone asks, “What did the system actually do?”

    Why day one matters

    Retrofitting this later is painful. It means backfilling logs, reworking workflows, and explaining to people why the guardrails arrived after the risk did.

    It is much easier to build the control plane alongside the agents and treat it as part of the product, not a patch.

    What to do first

    1. List every agent and workflow that can touch a real system.
    2. Log the full chain of inputs, decisions, and actions.
    3. Add human approval where the blast radius is high.
    4. Track failures and costs as first-class signals.

    That is enough to start building something you can actually trust.

  • Why Agentic AI Needs Audit Trails, Not Just Clever Prompts

    Why AI Workflows Need Audit Trails

    The conversation around AI has shifted. It is no longer just about drafting text or summarising meetings. More and more often, these systems are taking actions on live business processes.

    That is where the risk changes shape.

    A clever prompt can produce a good-looking result. It cannot tell you what happened after the fact if the output was wrong.

    The gap

    When a person makes a decision in a process, there is usually some trace of it. An email, a ticket, a sign-off, a log entry. With an AI system, that trace is often thin or missing.

    If the system updates the wrong record or sends the wrong message, the team is left with a result and very little explanation.

    What breaks without logs

    – You cannot reconstruct the sequence of events.
    – You cannot show who approved what.
    – You cannot improve the workflow with confidence.

    That is not just a debugging problem. It is an accountability problem.

    What a useful audit trail looks like

    At minimum, every execution should capture:

    – the input it received
    – the steps it took
    – the action it actually executed
    – the output it produced
    – the timestamp and identity of the run

    That is enough to answer the questions that matter later.

    The practical bit

    The tooling is already there. Structured logs, append-only storage, and reviewable traces are all enough to get started. The main thing is to design for visibility before the workflow is under pressure.

    If the system is allowed to act, it should also be required to explain itself.

  • Why Agentic AI Needs Audit Trails, Not Just Clever Prompts

    Why Agentic AI Needs Audit Trails, Not Just Clever Prompts

    The conversation around AI in business has shifted. For the past two years, most organisations have experimented with generative AI as a productivity tool — drafting documents, summarising meetings, answering questions. The model receives input, produces output, and a human reviews the result.

    That is no longer the whole picture.

    Across the organisations I work with, AI is moving from answering questions to taking actions. Agents execute n8n workflows that move data between systems. They trigger Hermes agents that read, write, and decide. They call APIs, update records, send messages, and make operational decisions — sometimes with human approval, sometimes without.

    This is agentic AI: systems that do not just respond, but act. It creates a governance problem that clever prompts alone cannot solve.

    The Governance Gap

    When a human makes a decision in a business process, there is usually a trace. An email sent, a form submitted, a system log entry, a manager’s sign-off. When something goes wrong, you can reconstruct the sequence of events. You can ask: what were they asked to do, what did they do, and why?

    When an AI agent executes an action, that trace often does not exist. The agent receives a prompt, processes it through one or more model calls, and performs an action. If the action is wrong — if it updates the wrong record, sends a message to the wrong person, or executes a workflow it should not have — the organisation is left with a result and no explanation.

    This is not a theoretical risk. In my own infrastructure, I have built agentic workflows that interact with live systems. The difference between a safe deployment and an unsafe one is not the quality of the prompt. It is whether the system logs enough information to reconstruct what happened after the fact.

    What Happens Without Audit Trails

    Without audit trails, three things break down.

    You cannot reconstruct events. If an agent produces an incorrect output or takes an unintended action, you need to know what input it received, which model or tool it called, what intermediate decisions it made, and what action it executed. Without this, debugging is guesswork. You are trying to diagnose a problem without access to the patient’s notes.

    You cannot establish accountability. When an automated system causes harm — a data breach, a financial error, a compliance failure — someone needs to be able to explain what happened. Under UK GDPR, the accountability principle requires organisations to demonstrate compliance, not just claim it. If your AI agent processes personal data and you cannot show what it did with that data, you are not compliant. It does not matter how good the system is in theory.

    You cannot improve the system. Agentic AI systems iterate. You adjust prompts, change tool configurations, add guardrails. Without structured logs of what each execution actually did, you are optimising in the dark.

    What a Practical Audit Trail Looks Like

    An audit trail for an agentic AI system does not need to be complex. It needs to be consistent and complete. At minimum, each agent execution should capture:

    • Input received. What was the agent asked to do? This includes the user’s request, any system context, and the prompt that was constructed.
    • Decision chain. What steps did the agent take? Which tools did it call? What intermediate outputs did it produce? For multi-step agents, this is the sequence of reasoning that led to the final action.
    • Action taken. What did the agent actually do? Which API was called, which record was updated, which message was sent.
    • Output produced. What was the final result returned to the user or passed to the next step in the workflow.
    • Timestamp and identity. When did this happen, and which agent or workflow executed it?

    This is not excessive. It is the same information you would expect from any business system that takes actions on data. The fact that the system is powered by a language model does not change the requirement — it increases it, because the system’s behaviour is less deterministic and harder to predict.

    The Regulatory Dimension

    For UK organisations, this is not optional. UK GDPR Article 5(2) establishes the accountability principle: you must be able to demonstrate that you comply with data protection principles. If an AI agent processes personal data — and most business agents do — you need to show what data it accessed, what it did with that data, and on what basis.

    Article 30 requires records of processing activities. An agent that processes client records, employee data, or customer information is conducting processing activity. If you cannot produce a log of that activity, you do not have the records the regulation requires.

    For financial services firms, the FCA’s operational resilience framework adds another layer. Important business services must withstand disruption and recover. If your AI agents are part of an important business service — processing transactions, managing client communications, monitoring risk — you need to understand how they behave, what they depend on, and what happens when they fail. Audit trails are the evidence base for your resilience assessment.

    How to Implement Audit Trails in Agent Workflows

    The good news is that the tooling exists. You do not need to build this from scratch.

    Structured logging at every node. In n8n workflows, each node can be configured to log its input and output. For agentic workflows, you should log at minimum the trigger, each decision point, and the final action. Use a consistent schema — timestamp, node name, input summary, output summary, and execution status — so that logs are searchable and comparable.

    Observability platforms. Tools like Langfuse are designed for exactly this purpose. They capture the full execution trace of an agent: prompt, model response, tool calls, and final output. When connected to your workflow engine, they give you a queryable record of every agent execution without building custom logging infrastructure.

    Immutable storage. Audit logs must be tamper-evident. If the log can be modified after the fact, it is not an audit trail — it is a diary. Store logs in append-only storage with access controls that prevent modification. This can be as simple as writing to a write-once bucket or using a logging service that enforces retention policies.

    Structured output from agents. Design your agents to return structured output, not just free text. A JSON response that includes the action taken, the target system, and the rationale is far more useful for auditing than a paragraph of prose. This also makes it easier to validate agent behaviour programmatically — you can check that the action taken is within the set of permitted actions before it executes.

    Regular review. Audit trails are only useful if someone looks at them. Build a review cadence — weekly for high-risk agents, monthly for lower-risk ones — where you sample executions and check for anomalies.

    The Bottom Line

    The organisations that will get the most value from agentic AI are not the ones with the most sophisticated prompts. They are the ones that can trust their agents to act safely, verify what those agents did, and improve them over time.

    Audit trails are the foundation of that trust. They are how you move from hoping your agents behave to knowing they do. They are how you satisfy regulators, reassure boards, and sleep at night.

    If you are deploying agentic AI in your organisation — or planning to — audit infrastructure is not a phase-two consideration. It is a prerequisite.


    If you are building agentic AI systems and need help establishing the governance, architecture, and audit infrastructure to support them, the AI & Automation Architecture service covers exactly this. For a broader conversation about where your organisation stands, get in touch.

  • Why Agentic AI Needs Audit Trails, Not Just Clever Prompts

    Why Agentic AI Needs Audit Trails, Not Just Clever Prompts

    The conversation around AI in business has shifted. For the past two years, most organisations have experimented with generative AI as a productivity tool — drafting documents, summarising meetings, answering questions. The model receives input, produces output, and a human reviews the result.

    That is no longer the whole picture.

    Across the organisations I work with, AI is moving from answering questions to taking actions. Agents execute n8n workflows that move data between systems. They trigger Hermes agents that read, write, and decide. They call APIs, update records, send messages, and make operational decisions — sometimes with human approval, sometimes without.

    This is agentic AI: systems that do not just respond, but act. It creates a governance problem that clever prompts alone cannot solve.

    The Governance Gap

    When a human makes a decision in a business process, there is usually a trace. An email sent, a form submitted, a system log entry, a manager’s sign-off. When something goes wrong, you can reconstruct the sequence of events. You can ask: what were they asked to do, what did they do, and why?

    When an AI agent executes an action, that trace often does not exist. The agent receives a prompt, processes it through one or more model calls, and performs an action. If the action is wrong — if it updates the wrong record, sends a message to the wrong person, or executes a workflow it should not have — the organisation is left with a result and no explanation.

    This is not a theoretical risk. In my own infrastructure, I have built agentic workflows that interact with live systems. The difference between a safe deployment and an unsafe one is not the quality of the prompt. It is whether the system logs enough information to reconstruct what happened after the fact.

    What Happens Without Audit Trails

    Without audit trails, three things break down.

    You cannot reconstruct events. If an agent produces an incorrect output or takes an unintended action, you need to know what input it received, which model or tool it called, what intermediate decisions it made, and what action it executed. Without this, debugging is guesswork. You are trying to diagnose a problem without access to the patient’s notes.

    You cannot establish accountability. When an automated system causes harm — a data breach, a financial error, a compliance failure — someone needs to be able to explain what happened. Under UK GDPR, the accountability principle requires organisations to demonstrate compliance, not just claim it. If your AI agent processes personal data and you cannot show what it did with that data, you are not compliant. It does not matter how good the system is in theory.

    You cannot improve the system. Agentic AI systems iterate. You adjust prompts, change tool configurations, add guardrails. Without structured logs of what each execution actually did, you are optimising in the dark.

    What a Practical Audit Trail Looks Like

    An audit trail for an agentic AI system does not need to be complex. It needs to be consistent and complete. At minimum, each agent execution should capture:

    • Input received. What was the agent asked to do? This includes the user’s request, any system context, and the prompt that was constructed.
    • Decision chain. What steps did the agent take? Which tools did it call? What intermediate outputs did it produce? For multi-step agents, this is the sequence of reasoning that led to the final action.
    • Action taken. What did the agent actually do? Which API was called, which record was updated, which message was sent.
    • Output produced. What was the final result returned to the user or passed to the next step in the workflow.
    • Timestamp and identity. When did this happen, and which agent or workflow executed it?

    This is not excessive. It is the same information you would expect from any business system that takes actions on data. The fact that the system is powered by a language model does not change the requirement — it increases it, because the system’s behaviour is less deterministic and harder to predict.

    The Regulatory Dimension

    For UK organisations, this is not optional. UK GDPR Article 5(2) establishes the accountability principle: you must be able to demonstrate that you comply with data protection principles. If an AI agent processes personal data — and most business agents do — you need to show what data it accessed, what it did with that data, and on what basis.

    Article 30 requires records of processing activities. An agent that processes client records, employee data, or customer information is conducting processing activity. If you cannot produce a log of that activity, you do not have the records the regulation requires.

    For financial services firms, the FCA’s operational resilience framework adds another layer. Important business services must withstand disruption and recover. If your AI agents are part of an important business service — processing transactions, managing client communications, monitoring risk — you need to understand how they behave, what they depend on, and what happens when they fail. Audit trails are the evidence base for your resilience assessment.

    How to Implement Audit Trails in Agent Workflows

    The good news is that the tooling exists. You do not need to build this from scratch.

    Structured logging at every node. In n8n workflows, each node can be configured to log its input and output. For agentic workflows, you should log at minimum the trigger, each decision point, and the final action. Use a consistent schema — timestamp, node name, input summary, output summary, and execution status — so that logs are searchable and comparable.

    Observability platforms. Tools like Langfuse are designed for exactly this purpose. They capture the full execution trace of an agent: prompt, model response, tool calls, and final output. When connected to your workflow engine, they give you a queryable record of every agent execution without building custom logging infrastructure.

    Immutable storage. Audit logs must be tamper-evident. If the log can be modified after the fact, it is not an audit trail — it is a diary. Store logs in append-only storage with access controls that prevent modification. This can be as simple as writing to a write-once bucket or using a logging service that enforces retention policies.

    Structured output from agents. Design your agents to return structured output, not just free text. A JSON response that includes the action taken, the target system, and the rationale is far more useful for auditing than a paragraph of prose. This also makes it easier to validate agent behaviour programmatically — you can check that the action taken is within the set of permitted actions before it executes.

    Regular review. Audit trails are only useful if someone looks at them. Build a review cadence — weekly for high-risk agents, monthly for lower-risk ones — where you sample executions and check for anomalies.

    The Bottom Line

    The organisations that will get the most value from agentic AI are not the ones with the most sophisticated prompts. They are the ones that can trust their agents to act safely, verify what those agents did, and improve them over time.

    Audit trails are the foundation of that trust. They are how you move from hoping your agents behave to knowing they do. They are how you satisfy regulators, reassure boards, and sleep at night.

    If you are deploying agentic AI in your organisation — or planning to — audit infrastructure is not a phase-two consideration. It is a prerequisite.


    If you are building agentic AI systems and need help establishing the governance, architecture, and audit infrastructure to support them, the AI & Automation Architecture service covers exactly this. For a broader conversation about where your organisation stands, get in touch.