Delete and recreate: Why AI needs good governance

Guest article by Carl Selby | Partner, RWK Goodman & Sam Berry | Solicitor, RWK Goodman

RWK Goodman is a Funding & Finance Partner, learn more about the programme here.

In December 2025, Kiro, an autonomous AI agent deployed by Amazon to monitor AWS infrastructure and make necessary updates on the live system, decided that the best way to complete its task was to delete and recreate part of the AWS environment. What could possibly go wrong? The result was a 13-hour outage of AWS’ Cost Explorer in a region in mainland China.

This incident is not an outlier. As organisations deploy AI agents at scale, outages and unintended consequences are likely to become more frequent.

The cost of such incidents is difficult to estimate. For businesses seeking investment, a poorly timed incident can make a significant difference to the amount of money they can raise and, in the worst-case scenarios, make it very difficult to gain investment. For investors, in the age of AI, weak AI governance can be the difference between the investor making money on their investment or not.

The risk of poor AI governance

Beyond outages and errant code, using AI without clear preparation, planning and oversight can create other risks including:

• Bias, discrimination and unfair outcomes – AI can embed or amplify bias if the underlying data, model design or prompts are not tested and monitored for bias.
• Loss of confidential information – Without proper controls, AI tools can expose personal, confidential, or commercially sensitive data (including reportable personal data breaches).
• Hallucinations and unreliable outputs – AI can produce plausible but wrong answers; NIST flags confabulation, information integrity and human–AI configuration risks that can drive flawed legal, commercial, operational, or customer decisions.
• Cybersecurity – AI adoption can expand the attack surface and introduce new security vulnerabilities if not properly governed.
• Reputation – Risk of incurring liability, damaging public trust, and inconsistent standards of output.

Good AI Governance

So, what could you do to reduce the risk that your AI dream turns into an AI powered nightmare?

• Human-in-the-loop oversight – This might sound archaic in the AI age but having a human-in-the-loop at appropriate stage of the process, especially for agentic AI, is, for the time being, an essential part of good governance.
• Adopt an AI governance framework – A good framework will set out which tools can be used, for what purposes, with what data, and what safeguards should be applied. Given the rapid evolution of AI, it will be needed to be updated regularly as use cases evolve. Independent standards such as ISO/IEC 42001 are emerging to set out best practice.
• Implement and audit – A framework that is not implemented is just a document taking up storage on the intranet. It needs to be a living document that is embedded in your operations. Audit how the framework has been implemented regularly to make sure it is working.
• Agentic governance – AI agents should be subject to the same governance rules as humans: least‑privilege access, isolation where possible, and clear points at which human approval is required before higher‑risk actions are taken.
• Don’t forget data protection – Where AI processes personal data, documenting how you process personal data, the lawful basis and, usually in the context of agentic AI, completing a data protection impact assessment to assess impacts on data subjects’ rights and freedoms.
• AI literacy – Lastly, and probably most importantly, anyone that is developing or using AI tools in your organisation needs to understand the technology they are using, its limitations and the way in which it is appropriate to use AI in your business. Not only will this help mitigate the risks but it might help build better strategy on how to deploy AI.

AI has the potential to generate massive growth opportunities for businesses and their investors – but its use does need to be governed if you want to realise the gains without the risk of a creating a pack of cards that might come tumbling down. Don’t let AI delete (and maybe try to recreate) the value you have built in your business. Good AI governance has the potential to give investors a reason to trust that the value they create will endure.

Carl Selby and Sam Berry are lawyers at RWK Goodman and advise clients on AI governance and the legal issues that arise from the deployment of AI. For more information about our services and support visit rwkgoodman.com/tech or contact our team at enquiries@rwkgoodman.com.