Mastering Fraud Solution Implementation - Don't Buy the Demo

Part of the Mastering Fraud Solution Implementation series, closing the preparation phase. The previous article checked whether your data and people are ready for AI; this one looks past the vendor demo to who will build, adapt, and deliver.

I would be uncomfortable signing off on a real-time fraud platform if the strongest argument for choosing it was how well the demo went.

That may sound strange coming from someone who has spent years on the vendor side. Having built a few demos myself, I appreciate the work behind a good demonstration: making a complicated platform understandable and connecting what happens on screen to the customer's problem.

But I also know how much remains outside that room, from a data feed that does not exist yet to the investigator who will handle the resulting alerts, and we can't forget the need to align the solution with the desired team structure and workflow. Something that looked simple on screen may still require another team's work and a lot of data before you can use it.

If you are responsible for choosing the fraud solution, those are the parts I would spend more time on. A convincing demo is a useful start. I would want considerably more evidence before making it the basis of a commitment.

Let the demo do its job

A standard sales demo usually follows a prepared path, with data and scenarios selected to show the product clearly. That is reasonable. Nobody needs to spend the first half of a meeting watching someone repair a data mapping.

You can learn plenty from it: how the product approaches a problem, how an analyst moves through a case, where a capability seems well developed and flexible enough, and where the presenter has to leave the application and return to the slides. It also gives you something concrete to challenge.

The mistake is allowing the conditions behind the demonstration to disappear from the discussion. Seeing a customer, their devices, their payments, and their relationships together on one screen does not establish that your systems can supply and connect all of that information. Or that your team can maintain it.

I would pause on a capability that matters and ask what had to be in place for it to work as shown. The answer may be entirely reassuring. It may also reveal another integration, an additional product, or an assumption that nobody on your side has agreed to yet.

Talk to vendors before the requirements are frozen

There is a qualification to the preparation sequence I have been arguing for throughout this series. Choosing the partner belongs after you have defined what you need and assessed your readiness. Conversations with potential partners can start earlier.

You cannot be expected to know every available approach before speaking to anyone who builds these products. A briefing or a focused workshop with shortlisted vendors can help clarify a requirement, challenge an outdated assumption, or show that something you expected to commission is already supported.

Keep that discussion close to the top of your ranked requirements. You do not need to settle every item on a long wish list before learning whether the solutions can address the few things that justify the investment.

The distinction matters because early conversations can also reshape your priorities without anyone quite noticing. An interesting feature gets added to the list, the list gets rearranged around it, and eventually the original problem has to compete for attention. Take the useful ideas back to the people who own the requirements. They still have to decide whether those ideas belong in scope.

Where I would look for differences

In The "I Want It All" Trap, I argued that established platforms can overlap substantially across a broad list of capabilities. Let me emphasize, though, that it should never be read as permission to treat them as interchangeable.

A long list can hide a decisive difference. Two solutions may both support case management, for example, while handling the work your investigators do every day quite differently. A workflow that fits one team's operation may require substantial adaptation for another.

If the purpose of your program is to improve investigation, let investigators work through a relevant case (I have seen numerous project discussions where the investigation team wasn't even called to participate). Give them room to notice the awkward parts. If the priority is payment intervention, focus on the path from receiving the event to returning an actionable decision. A polished investigation screen will not answer that question.

I would also separate what is available in the proposed product today from what depends on another component, custom development, or a roadmap commitment. Any of those may be acceptable. They carry different dependencies, and the selection discussion should make those visible.

I would spend time on reference conversations, preferably with a customer in your region. A successful deployment elsewhere is useful evidence, but it doesn't prove the same delivery expertise is available locally. Language needs, structural constraints, and dependencies can differ too. A regional customer may have faced some of the same issues.

Even then, find out who will actually deliver your project. A local reference does not guarantee you will get the same team. I would want to hear how the customer worked with the people delivering the solution, what they had to contribute themselves, and what operating it required after go-live.

What the customer thinks is included

One of the common points of ambiguity in my implementation work has been the integration layer. The bank often expected us to develop it. I generally prefer the bank to own and develop it, ideally using its existing technology. We had the capability to do the work; that wasn't the point of contention. Yet the difference in expectations sometimes became clear only during design, with the project already underway.

During selection, discussions tend to focus on what the solution can do. The work needed to connect it to the bank's systems deserves the same attention. Real-time fraud detection depends on contributions from both teams, so I would clarify who will build, test, and maintain each part of the integration before detailed design begins. That gives everyone a shared starting point for delivery.

A related area where expectations can differ is the starting point provided by payment-fraud rules. Customers may reasonably look to the vendor for effective prebuilt rules that they can then update or fine-tune. A common rule library should provide a useful foundation. The key point to clarify is how much adaptation the bank's own products and customer portfolio will require and where more focused rules may be needed. It is also worth agreeing early on how responsibility for developing and maintaining those rules will be divided.

In my experience, the gap in expectations about rule effectiveness usually became visible after go-live. Almost immediately, the rules needed fine-tuning to stabilize alert volumes. Further tuning then focused on improving detection KPIs and reducing false positives. The work involved can be more than the customer expected. That post-launch work belongs in the selection conversation too.

This is where buying technology can get confused with having the hard part of the job done. The bank still needs to own its detection requirements and the rules that address its specific product landscape and gaps. Vendor expertise can help, but a supplied library does not settle that responsibility.

The strongest implementations had people who could confidently own their areas and work with us toward a practical solution. They did not necessarily have the deepest fraud-domain expertise. They did have clear responsibilities, organizational priority, and a willingness to approach the work as partners.

That is why I would bring the unresolved findings from Before You Ask for AI, Check Your Pantry into selection discussions. Work through what the vendor will supply, what your team must provide, and what remains undecided. Those conversations give both sides a chance to discover different expectations before they become a design-phase disagreement.

Before asking for a POC

A proof of concept (POC) is an expensive exercise (for both sides), so I would first establish what you need it to prove. A focused demonstration, technical discussion, or relevant customer reference may already provide the answer. I wouldn't add a POC to the selection process just because it feels like a reassuring next step.

For me, the important distinction is whether the POC can demonstrate the intended functionality without changing the conditions it depends on. Batch preparation, entity resolution, and historical analysis may be representative of some procurement or internal-fraud processes. But if I am buying a real-time platform, a POC built on enriched warehouse extracts may give the detection logic information the intended live setup will not have. I would want the test to preserve the relevant data and integration constraints, even if it focuses on only one part of the solution.

If you do commission one, be clear about which selection decision its findings will inform and what will remain untested. Useful analytical results do not automatically establish that the intended live operation will work. That limitation belongs in the recommendation alongside the findings.

What this means for you

If you are leading the selection, put the unresolved responsibilities on the table before choosing the partner, including integration and the work needed after launch. For each remaining uncertainty, ask what evidence would settle it, starting with existing material and a relevant customer reference before commissioning a POC. Make the final recommendation clear about what has been demonstrated, what remains assumed, and who owns the remaining work.

This article closes the preparation phase of the Mastering Fraud Solution Implementation series. The next step is agreeing on how the selected partner will deliver the requirements you own: The Art of Defining 'What' and 'How'.