AI in financial operations: Where governance gets tested
In early 2024, a finance employee at the engineering firm Arup joined a video call with the company's CFO and several colleagues. Following the call, the employee sent 15 transfers to five accounts, totaling about $25.6 million. The CFO and colleagues on the call had been generated by AI.
The Arup story is usually told as a cautionary tale about deepfakes and impersonation, but it also has a lot to do with internal controls. The fraud was successful because one conversation could authorize a payment, and nothing in the process required the payment request to be confirmed by anyone else in the company. In fact, it should have been the actual CFO reviewing and completing the payment anyway.
AI entered finance departments as a tool for summarizing reports, writing emails and explaining variances. It now categorizes transactions, matches invoices, flags anomalies and, at a growing number of companies, initiates payments through agents with real spending authority.
Most AI governance frameworks were written around models and data. But finance is where AI governance has the most obvious consequences, because a mistake can lead to millions in losses.
See the ROI behind your spend
Use this calculator to understand impact, then manage and track it all in Slash.
Key takeaways
- AI has moved from summarizing financial information to acting on it: categorizing transactions, reconciling accounts and, at a growing number of companies, initiating payments.
- The costliest AI failures in finance happen at the point of payment approval, where a convincing fake or an unchecked automation can move money in minutes.
- AI regulation is still in flux in the US and the EU, but there are steps you can take to be better prepared for changes.
What is an AI governance framework?
An AI governance framework is the set of policies, roles and controls an organization uses to decide how AI is built, bought and used, who answers for its outcomes, and how its risks are found and contained over the life of each system.
Governments and standards bodies publish reference frameworks that describe what good AI governance should include, which we explain in more detail further down. Companies are then responsible for building an internal governance framework based on them: a written set of rules about which AI tools are approved for work, who in the company is responsible for managing each one and what each is allowed to do.
The company framework is mostly organizational, but its efficacy depends on how it’s enforced. Companies can set rules around who has access to certain tools, what data a tool has access to, which actions need approval, and what actions get logged. A policy that says an AI agent can't pay a new vendor only works if the payment system won't let it.
Most frameworks touch on four general principles:
- Transparency: People can understand what an autonomous system did and why.
- Accountability: A named person owns each outcome, so no one can blame "the algorithm."
- Fairness: Outputs are tested so they don't discriminate against protected groups.
- Human oversight: Someone can review, correct or override the system, especially when stakes are high.
Governance is often confused with two neighboring ideas. AI ethics is the set of values a company wants its systems to reflect, and governance is the machinery that turns those values into rules people follow. AI security protects data, models and infrastructure from attack, while governance decides who may use those systems, for what, and with whose approval.
Why finance is where AI governance breaks down
The clearest data on ungoverned AI comes from IBM's 2026 Cost of a Data Breach Report, which IBM and the Ponemon Institute constructed from 602 organizations in 16 countries. The global average cost of a data breach reached a record $4.99 million between March 2025 to February 2026, up 12% from the year before.
Among surveyed organizations that were victims of an AI-related breach, 92% lacked proper AI access controls.
Shadow AI, meaning employees using AI tools their company hasn't approved, showed up in 43% of incidents, more than double the 20% IBM recorded in 2025. Those incidents averaged $5.39 million in losses. In a finance department, shadow AI incidents usually aren’t malicious: maybe an analyst pastes a vendor list into a chatbot to clean it up, or a controller uploads a bank statement for a quick summary.
Attackers are using AI too. IBM found AI-driven attacks rose 56% year over year and added about $1 million to the average breach. Deepfake impersonation made up 45% of them, the largest single category.
Finance teams are already aware of the uptick in AI-related risks, but IBM’s access-control numbers suggest many organizations haven’t acted on it yet A survey of corporate finance leaders published October 8 by Certos found 81% of corporate finance leaders had seen attempted or suspected AI-enabled fraud. Payment diversion was their top concern.
NIST, ISO 42001 and the EU AI Act: The frameworks to know
Global governing bodies have already begun to draft resources and regulations for businesses regarding AI implementation. The three most comprehensive today are:
- NIST AI Risk Management Framework: A free, voluntary guide from the US National Institute of Standards and Technology, a federal agency. It's written for any organization that builds or uses AI, and it lays out the steps for finding and managing AI risks.
- ISO/IEC 42001: An international standard from two standards bodies, ISO and the IEC. It's written for organizations that want outside proof of their AI practices, and it sets requirements an auditor can check a company against.
- EU AI Act: A law passed by the European Union in 2024. It applies to companies that sell AI systems in the EU or whose AI is used there, wherever they're based, and it sets legal obligations that get stricter as a system's potential for harm rises.
Companies can use more than one as a reference point, even if they aren’t in the enforcement jurisdiction. NIST is a good starting point for developing an internal rulebook, ISO 42001 is a standard to follow when you need to prove controls to a third-party, and the EU AI Act can serve as the legal minimum for what constitutes appropriate AI use.
Although the EU already has regulation around AI governance, the US is less settled, as much of the regulation falls to the states. Colorado passed the first comprehensive state AI law in 2024, delayed it, saw a federal court stay its enforcement in April 2026, then repealed and replaced it in May with a narrower law on automated decision-making, effective January 1, 2027. A December 2025 executive order, EO 14365, created a Justice Department task force to challenge state AI laws. Those laws still stand until Congress or the courts say otherwise.
5 financial guardrails for AI governance
Your company’s financial controls shouldn’t wait for the dust to settle regarding the general guidelines. Most internal protections around AI usage and how it connects to your finances are a matter of making common sense decisions, and that starts with getting serious about building an internal policy and seeing how your financial tools can enforce those policies.
1. Identify where AI interacts with your money
Start with an inventory of every tool, approved or not, that reads financial data or can initiate a transaction. That includes AI features inside accounting software, AP automation and expense tools, plus the general-purpose chatbots employees already use. Then sort each tool by what it can do: reading data, recommending an action, or acting autonomously. A tool that only reads data only needs rules about what data goes into it, but a tool that acts on its own needs every control below.
2. Separate the request from the approval
A request to send money can arrive by email, phone, chat or an AI tool. Wherever it comes from, the payment approval should happen inside your company’s finance system, traceable to a named person, under their own login.
Write the approval rules down before they’re needed: who can approve which payments, up to what amount, and when a second approver is required. Confirm new payees and changes to vendor bank details using contact information already on file, and never rely solely on details supplied with the request without confirming.
3. Limit what AI agents can spend
Gartner finds that 17% of organizations have deployed AI agents, and more than 60% expect to within two years. Some of those agents will pay for things on the company's behalf, such as renewing software subscriptions, ordering supplies or settling invoices.
An agent that can spend should be set up like a new employee with a company card. It gets its own card or account instead of access to a shared one, with a spending limit and a list of vendors it's allowed to pay. If it makes a mistake or someone takes control of it, the damage stops at what that one card can spend, and it can be shut off without affecting anyone else.
4. Require a person to approve high-risk payments
Some payments are hard or impossible to reverse once they go out, including wires, first payments to new vendors and payments sent after a vendor's bank details change. Those should require a person's approval, along with anything above a dollar threshold the company sets. AI can still prepare these payments, match them to invoices and flag anything unusual, but a person decides whether the money goes out.
5. Keep records and check the AI's work
Every payment an AI system starts or helps prepare should be logged with what the system did, what information it used and who approved it. When an auditor or regulator asks how a payment happened, that log is the answer.
AI also makes mistakes, and a miscategorized transaction can sit in the books for months before anyone notices. Finance teams should regularly spot-check the AI's work, such as how it categorized transactions, against receipts, invoices and bank statements.
Why guardrails make AI more useful in finance
None of this is an argument for using less AI in finance. IBM found that organizations using security AI and automation extensively cut average breach costs by $1.93 million and found and contained breaches 65 days faster than organizations that didn't. The Association for Financial Professionals’ 2026 payments fraud survey found a similar pattern in payments: only 17% of organizations use AI to fight payments fraud, but 45% of those that do said it improved their ability to detect deepfakes.
For most, the limit to how much you’re willing to pass off to AI comes down to trust. Without approval rules, spending limits, or records, letting AI into your financial processes relies on a lot of hope that nothing goes wrong. Given the risk, many teams may give up the hours AI could save on categorizing transactions, matching invoices, and preparing payments.
Implementing proper guardrails is how your team can start to unlock the benefits of using AI without trading off peace of mind. If an agent can only spend within a set limit, and every payment it prepares still goes through the company's normal approvals, the worst case is known in advance. A team can start AI on low-risk work, review the records it leaves, and give it more responsibility as it proves reliable.
Strengthen your company’s financial controls with Slash
Most AI governance frameworks live in documents. A policy might say that payments over a certain amount need two approvals, or that an AI agent can only pay vendors on an approved list. Rules like these are easy to write and easy to skip, especially when a payment is urgent or the person who's supposed to approve it is out of the office.
Rules hold up better when the software that moves the money enforces them. If the payment system won't release a wire until a second person signs off, no one has to remember the policy.
Slash is a business banking platform where finance teams can write those rules into the account itself.¹ Approval requirements, spending limits and review steps are set in Slash, so an AI agent working in the account is held to the same standards as the people on the team. Those controls include:
- Multi-step approvals: Payments can require sign-off from more than one person under rules the company sets, so no single request moves money on its own.
- Visible markers on AI categorizations: Transactions categorized by AI are clearly marked, so reviewers know which entries to check before the books close.
- Real-time fraud alerts: Slash monitors transactions as they happen and flags activity that doesn't fit, learning from your company's transaction patterns over time.
- Card groups: Companies can group cards by team and set rules for how and where each group can spend. An AI agent's card can sit in its own group with its own rules.
- MCP connections for agents: The Model Context Protocol is an open standard for connecting AI agents to software tools. Through MCP, agents can connect to Slash directly.
With strong controls in place, a company's AI governance framework can cover its money the way it already covers models and data. If your business needs a better way to enforce those rules as AI takes on more of its finance work, open a Slash account and set them up on day one.
Apply in less than 10 minutes today
Join the 10,000+ businesses already using Slash.
Read more from us
