Part of the Mastering Fraud Solution Implementation series, in the preparation phase. The previous article separated ambition from phase-one scope; this one checks the four shelves every fraud AI program needs stocked before the vendor conversation starts.

Have you ever baked a chicken and a cake at the same time? I haven't, and I doubt anyone really has. Let me tease you with that image for a moment - I will explain exactly how it connects before this article is over.

The previous article ended with a warning: some rows on the wish list assume data you do not have. The most expensive version of that assumption usually arrives in two letters: AI.

One requirement repeats near the top of practically every fraud RFP I see:
"We want AI/ML (machine learning model). It should automatically find fraud, including fraud we've never seen, and stay up to date by itself."

Machine learning can do a lot of heavy lifting in fraud detection. I have deployed it, tuned it, and watched it detect patterns that rules missed, move the most suspicious alerts to the top of the queue, and give investigators a stronger starting point. This is not an argument against AI. It is an argument against treating AI as a feature you can procure independently of the conditions that make it useful over the long term.

For fraud leadership, the strategic objective is not to own an AI capability; it is to improve fraud outcomes. Even where regulation mandates an AI/ML model, that model still has to be effective and defensible. The less-voiced requirement is equally important: achieve the outcome without creating a control the organization cannot operate, explain, or govern.

Before AI becomes a vendor question, it needs readiness across the usual three dimensions - people, process, technology - and one extra special area - data. Procure a model without checking it, and you have not acquired a capability. You have acquired an expectation (and a problem).

Note: The points below relate primarily to predictive models - the most commonly deployed AI/ML approach in fraud detection - though the same generally applies broadly to any analytics or AI/ML algorithm.

Data

A supervised fraud model learns from historical examples - marked fraudulent or legitimate. Its ceiling is set not only by how much history you have, but by whether that history resembles what the model will see in production and whether the labels preserve the distinctions that matter.

Data is the KO criterion. No vendor, no matter how capable, can compensate for it inside the fraud platform. An empty shelf, or a shelf missing critical pieces, doesn't become full when they arrive.

Existence and history depth. A real-time fraud detection model usually requires months of transaction history. Six to twelve months is a common planning range, depending on volumes, fraud prevalence, seasonality, and the fraud typology.

Structure and production-representative form. The available history is frequently warehouse-shaped: cleansed, aggregated and transformed for reporting. If important fields, event timing or channel context have disappeared in that transformation, the training set cannot recreate them. It must exist (or be reproducible) in a form that copies or tightly mimics the real production data (events). What trains the model must match what it will score in production. A gap between the two - training-serving skew - is a silent failure: the model validates well in the lab and degrades immediately on go-live.

That does not mean every implementation must wait a year. An organization with well-preserved event history may be able to reconstruct a suitable dataset. A vendor's pre-trained or consortium model may also provide a starting point. But both still need local field mapping, outcome definitions, and validation against the population where the model will operate. A head start is not an exemption from readiness.

For many greenfield deployments, the honest sequence is: launch with rules and other proven controls, capture production data in its native or reliably reconstructed form, train and validate the model, then promote it into production. The model comes after go-live, not necessarily at it. Plan for that sequence rather than discovering it during delivery.

Figure 1: In a greenfield deployment, go-live and production data capture establish the foundation; preparation, training, validation, deployment and monitoring then form a repeating model lifecycle.

Labels. "We marked which transactions were fraud" is not enough if every confirmed case carries the same undifferentiated flag. Account takeover, social-engineering scams, first-party fraud, mule activity, or even internal fraud, leave different, sometimes conflicting, footprints. An account takeover may show a new device and abnormal behavior on a genuine account. A scam may show the genuine customer, on the usual device, completing a payment that a criminal persuaded them to make.

This is where the chicken-and-cake image from the opening earns its place. A funds-transfer fraud model and an application fraud model need different ingredients (events and data fields), different conditions, and different evaluation criteria. Train them together in one undifferentiated dataset, and you do not get a blended result - you get chicken that tastes of cake and cake that tastes of chicken, both damaged. Depending on how prevalent one fraud type is relative to the other, the aggregate score hides a model that performs well on the dominant typology and poorly on the one currently hurting you. A well-designed architecture can support several typology-focused models. The point is that the labels and fraud tags must preserve the differences and specifics of fraud occurrences in the data.

The fix is unglamorous and starts today, whatever your AI timeline: make consistent typology-level labeling part of the investigation process. This week's case handling is shaping your future model, for better or worse. I went deeper on the MLOps expectations in How (not) to train the fraud detection model, Part 1 and Part 2.

People

The quiet assumption inside "we want AI" is often "so we need fewer people." In practice, the expertise relocates: from writing rules to curating data, maintaining labels, monitoring drift, investigating model behavior, and explaining decisions. AI does not reduce the people requirement, but it certainly elevates expectations on what those people need to know.

The question is not headcount. It is whether named, capable people own each of the following.

Model development and retraining. "It stays up to date by itself" may be the most dangerous sentence in an RFP. Fraud patterns don't merely drift; criminals probe controls and change behavior in response.

A naive retraining loop that skips the data preparation step often has a poisoned-feedback problem. If unconfirmed transactions are treated as legitimate, the next training set can contain fraud the current model missed. If wrongly classified fraud categories remain in the historical dataset, we are back to chicken and cake situation. Retraining then teaches the model that at least some of its blind spots are normal behavior.

Delayed reports, chargebacks, and case outcomes make the problem harder: today's apparent legitimate result may become next month's confirmed fraud. Missed fraud left in training data as genuine does not merely represent a missed opportunity to sharpen the model - it actively blunts it, because behavior associated with fraud is also being fed back into training as legitimate.

Figure 2: Without curated outcomes, an automated feedback loop can reinforce the blind spot it was meant to correct.

This is why skilled people are essential: they track performance and data quality by relevant segment, define degradation triggers, retrain on curated outcomes, validate the candidate against the production model, and control its promotion into production. The trigger can be automated. The judgment and accountability cannot be delegated to the loop.

Measurement and defensibility. When someone requires "the model must detect 95% of all fraud," ask: 95% of what? As discussed earlier in Measure It, Prioritize It, Own It, the total fraud that actually occurred is not fully observable. Confirmed fraud is the portion found and reported, often after a delay. That makes "percentage of all fraud" a poor acceptance criterion, however attractive it sounds in an RFP.

A better conversation focuses on questions a business leader can test. Out of the fraud confirmed last quarter, how much did the model actually catch - and against which population (recall)? How often does it block a genuine customer, and what does that cost in declined transactions and friction (false positive rate)? Are the alerts worth investigating, or is the team working through noise (alert yield)? How much confirmed loss did it prevent? Does it score fast enough for the channel to act on it (decision latency)? And does it perform consistently across customer types, channels, and transaction sizes, or only well on average (segment stability)?

None of these is a perfect measure. Each is at least testable. I went deeper on how these metrics work and where they can mislead in Confusion Matrix and the False-Positive Ratio Mantra.

Someone must also own the ability to trace and defend individual decisions. If a model contributes to declining a payment, freezing an account, prioritizing an investigation, or filing a report, the organization should be able to show what information drove the decision, what control surrounded it, and who approved the process. The exact explanation obligation varies by use case and jurisdiction; the need for traceability does not. If you cannot trace the decision, you will eventually be asked to explain it.

Decision rights and governance. You need named people who understand the model's purpose, its limits and the operating response when performance changes. You also need clear decision rights: who can change a threshold, approve retraining, accept a performance trade-off or take the model out of service?

In regulated institutions, applicable model-risk frameworks may require documented governance, effective challenge and validation proportionate to the model's use and materiality. In the US, the banking agencies' revised 2026 model-risk guidance takes this risk-based approach. In the EU, additional AI Act obligations may apply where the particular system and use case qualify as high-risk. These requirements must be assessed before production use, not treated as housekeeping afterward.

Miss this, and you get one of the industry's most expensive anticlimaxes: a sophisticated ML platform operated in rules-only mode for years, with the organization paying for capability it cannot safely use. A model with no accountable owner is a control without a controller. AI does not reduce the governance requirement - quite the opposite, as model governance continues to mature and gain in importance globally.

Processes

Deploying an AI/ML model is not an event. It is the start of an operational commitment.

A few process questions to resolve before go-live:

• Who collects outcome data, at what frequency and in what form?
• What happens when a new data source is integrated - what validation runs?
• Who approves, and how does it propagate to the model?
• How does a threshold change, a retrain, or a model replacement move through the organization - who initiates it, who approves it, and who confirms that the new behavior is acceptable?

These are not exotic questions. They separate a model that gets managed from one that gets ignored. Processes can mature after launch, but minimum controls, named owners, approval paths, and escalation procedures must operate before the model makes production decisions.

Technology

Responsibility varies with the deployment model. The vendor may provide model serving, the real-time decision engine and parts of the feature pipeline, while the customer usually owns-or must coordinate-the source systems and integration boundary. What does not change is the institution's accountability for ensuring that the complete control receives the right signals, operates within the decision window, and fails safely.

The signals the model needs must reach it at the right time, in the right form, and in the right order. No platform compensates for events that never arrive or arrive out of sequence. Latency matters here too: a real-time authorization model that cannot receive and return a score within the decision window (SLA) is not a real-time control, regardless of the vendor's headline response time.

The other question is fallback. What happens when the model is offline or degraded? If the answer is "nothing," the model is a single point of failure in a critical control. The contingency needs to exist and be tested before go-live.

The gap a vendor cannot close

Every capability you ask for in an RFP is, underneath, a claim about the organization that will operate it. AI makes the data claim obvious, but the same logic applies across all four dimensions.

You want to detect account takeover using device, session, and behavioral signals. Does the channel send those signals today, with the identifiers and timing needed to use them? You want real-time scoring. Can your stack deliver the events within the latency window with sufficient context? You want monitored, governed model operation. Do you have the people and processes in place to run it?

If the necessary data doesn't exist or cannot be trusted, no vendor can compensate for it inside the fraud platform. The same is true for each of the other three shelves. Assess the gaps before choosing the partner. A "no" discovered now is a plan. The same "no" discovered at UAT is a crisis.

This is not only an AI problem, nor only a data problem. The same shape appears wherever a requirement quietly assumes something about the organization that is not true:

• You want real-time signals from a source system that only batches overnight.
• You want a single customer view across products that sit in silos and share no reliable customer key.
• You want low-risk alerts closed automatically, but nobody owns the mandate to approve the risk appetite that would allow it.

These are structural gaps. The missing component may be data, integration, process, authority, or capacity; in each case, the fraud platform is not the ground on which the repair must begin.

The device-fingerprinting example in The "I Want It All" Trap showed exactly this pattern. The platform supported the capability, but the required events depended on middleware integration that wasn't ready. One desirable phase-one requirement imported another team's timeline and set the go-live date for everything else.

That is why the previous article's wish list was always going to be longer than the deliverable scope. Some rows are not the vendor's to build. Others belong in parallel workstreams with their own owners and timelines. A few require a business decision before anyone can scope them honestly. Push all of them into the fraud program as "requirements," and you have not closed the gaps; you have hidden them inside someone else's delivery, where they will return during integration testing with a budget attached.

Next in the series: the last stop in preparation. The homework is done; now, and only now, comes the vendor conversation.