When the Engineers Go Home: Due Diligence for the Buy-Side

In short

AI vendor due diligence on the buy-side should look past the demo. Test whether the tool keeps working once the vendor's engineers leave, how strong the data underneath it is and who owns what gets built. Then check how the vendor works with you after signing, whether the platform fits into your workflows or walls them in, and whether its partners would keep you running if it failed.

•    95% of enterprise generative AI pilots showed no measurable impact on revenue or cost (MIT NANDA).

•    For high-materiality AI bought from vendors, only 24% of UK firms surveyed by the Bank of England and FCA claim deep knowledge of how it works.

•    In the same survey, 47% of model provider relationships judged critical and hard to replace had no exit plan.

•    40% of those firms rate critical third-party dependencies a high or severe risk today, and 63% expect to within one to three years.

•    US job postings for forward deployed engineers rose from fewer than 200 in 2023 to more than 7,300 in the first nine months of 2026 (Lightcast data via CNBC).

The demo was built in your office

It usually starts with a great meeting. An AI vendor sends a small team of engineers to your office, and within two weeks they have connected your research archive and built a demo around your analysts' own templates. The investment committee loves it. Procurement signs.

Then the engineers move on to the next client, and someone on your team inherits a tool nobody in the building knows how to change.

A growing number of buy-side firms are choosing AI this way. The buying decision rewards whatever looks best in week two, and the bill for that choice arrives in month nine. Before you sign, it pays to know which kind of vendor is across the table: one selling software your people can run, or one renting you the attention of very expensive engineers.

The fastest-growing job in AI is a services job

Forward deployed engineers are specialists employed by a software vendor who sit inside the client's environment and build the use case on site. The role has exploded: US job postings rose from fewer than 200 in 2023 to more than 7,300 in the first nine months of 2026, according to Lightcast data reported by CNBC.

Help on site has its place, and good vendors offer it during onboarding. The trouble starts when the embedded engineer becomes the product. Total compensation for the role is now pushing past $300,000 in the US, and a vendor that needs one in every account carries a cost that grows with every client it signs.

That cost has to land somewhere. It lands in your renewal price, or it sits in the vendor's burn rate until the next funding round. As a recent Crunchbase News column argued, software should cost less to deliver as it scales, while the cost of people only climbs. If you are paying for people, you should know it before you sign.

Watch what that model does to the product. When every workflow is built by hand for one client, the vendor ends up with a stack of one-off projects and no common platform underneath. The roadmap follows whichever account is loudest, and hiring has to outpace revenue just to keep up.

Buyers keep rewarding the demo

Here is the uncomfortable part: vendors run this playbook because buyers keep paying for it.

Evaluation committees score the pilot. Very few score month nine. A workflow hand-built around one portfolio manager's process looks brilliant in week two, then breaks the day that PM changes a template or compliance asks how an output was produced. Firms that would never accept a black box from a broker are signing one from an AI vendor because the onboarding felt attentive.

The results show up in the data. MIT's NANDA initiative reviewed more than 300 public AI initiatives and found that 95% of enterprise generative AI pilots produced no measurable impact on revenue or cost. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, with escalating costs and unclear business value among the causes.

Gartner also flagged agent washing: vendors relabeling chatbots and assistants as agents. Of the thousands of vendors claiming agentic AI, it estimated only about 130 offer the real thing. A demo built by the vendor's own engineers is the easiest place to hide the difference.

UK regulators found the same habit inside firms. In the Bank of England and FCA's October 2026 survey, 77% of respondents claimed deep knowledge of their own high-materiality AI. For the systems they bought from vendors, that figure fell to 24%, and 81% relied on vendor assurances and contracts instead.

The same MIT research holds the lesson buyers tend to skip. Tools bought from specialized vendors reached deployment about 67% of the time, far more often than projects built internally. Buying works. Buying a project dressed up as a product is where firms get burned.

Three things that outlast the demo

A strong demo is a good sign when it runs on the same product your team will log into on day one. Even then, three things decide whether the relationship lasts, and none of them fit in a demo slot.

1. The data underneath

The value of any research agent sits in the data it reads. A workflow built on thin coverage or unverified sources can look sharp on the five companies in the demo and fall apart across a real coverage universe. Gartner expected at least 30% of generative AI projects to be abandoned after proof of concept by the end of 2025, and put poor data quality first among the reasons.

Ask, too, whether the vendor has data of its own. Many AI platforms own none. They license feeds from the same providers your firm may already pay and put a model on top, so every workflow starts from scratch. That can mean paying again for data you already have, and it leaves the vendor with little of its own to build on.

Ask where the data comes from, how far back it goes, how quickly new filings arrive and whether every answer cites the exact source.

Strong answer: coverage the vendor builds and maintains itself, tested on your own universe, with every figure traceable to the document and page it came from.

Red flag: a demo limited to a handful of large caps the vendor picked in advance, or a data slide made of other providers' logos.

2. A partnership that continues after signing

Some vendors treat the signed contract as the finish line. The people you met during the sale disappear, and every request turns into a change order. Investment processes change every quarter, so the vendor you choose should still be planning the next year with you long after the first renewal.

Buyers fall into the same trap. In the Bank of England and FCA survey, more than 90% of firms reported vendor due diligence as a control on AI data risk, while ongoing monitoring and periodic review were each reported by around half. The work cannot stop at the signature on either side.

Read the contract with the same eye. An unusually short cancellation term can look like flexibility, so check whether it cuts both ways: a vendor that can walk away at short notice can leave your process stranded. Either way, the real cost of leaving sits in the months your team has spent building around the tool.

Strong answer: a named team after go-live and a roadmap shaped by client requests.

Red flag: the only senior contact you ever meet is the one who closed the deal.

3. The vendor's web of integrations and partnerships

Some platforms are built as walled gardens. Your research and agents stay inside the vendor's own interface, with few or no connections to the tools your firm already runs, and every month inside makes leaving harder. Investment teams should stay free to work the way they already do. The platform should fit into those workflows and adapt as they change.

Look at how widely the vendor is plugged in: the data providers and platforms it works with, and the models and tools it connects to. Every established partner has run its own diligence on the vendor, and every integration is a route that keeps your workflows running if priorities change. It also answers the question few buyers ask out loud: what happens to you if the vendor goes under?

Concern is rising. In the same survey, 40% of firms rated critical third-party dependencies a high or severe risk today, and 63% expect to within one to three years. A vendor funding engineers on every client site with venture money is exactly the kind of supplier that can vanish between renewals.

When a vendor like that fails, the license fee is the smallest loss. The analyst hours spent shaping workflows around its tools go with it, along with every integration your team built.

Strong answer: established partnerships with recognized data providers and platforms, plus open APIs and MCP access into the AI tools your firm already uses.

Red flag: a closed system that only talks to itself, with no partner willing to vouch for it.

Your compliance team is already asking

Regulators are now asking the questions buying committees skip.

In the UK, the FCA's October 2026 review of AI good practice reminded firms that they can outsource AI services but stay responsible for them. Its areas for improvement included firms that accepted vendor assurances without testing them, and firms tied to a single platform with no exit options.

The Bank of England and FCA survey published on 26 October covered 196 firms, with investment and capital markets the largest group. Third-party systems now account for 53% of high-materiality AI, up from 35% in 2024. Among model provider relationships judged critical and hard to replace, 47% had no exit plan.

In the US, compliance teams are already moving. In the 2026 Investment Management Compliance Testing Survey of 411 US investment adviser firms, 85% named AI their top compliance topic, up 28 points on 2025. Six in 10 are making significant changes to their third-party risk management programs, yet only 48% have completed compliance testing on vendor due diligence.

In the EU, DORA treats AI providers as ICT third-party service providers. Article 28(8) requires firms to keep documented, tested exit strategies for providers that support critical or important functions.

All three regimes point the same way. An AI tool whose behavior depends on which engineer happened to build it is hard to validate, and harder still to explain to an examiner.

Six questions to ask an AI vendor before you sign

These six questions separate a product your team can run from a project that walks out the door with the vendor's engineers. Ask them early, and ask them in writing.

1. If your engineers left tomorrow, what would we still have?

Ask the vendor to run your workflow live with nobody from their side in the room. If the answer involves a support ticket to the person who built it, you are buying a team on loan, and it will be priced like one at renewal.

Strong answer: a live run on your own data with only your team at the keyboard, plus documentation your analysts can follow.

Red flag: "We'll keep someone on hand for the first few months."

2. Who owns what you build for us?

Prompts, agent logic, connectors and test sets encode your investment process. If they live in a vendor engineer's private repository, you own nothing you can carry forward. Put ownership and access in the contract.

Strong answer: everything built for you is exportable, kept under version control and named as yours in the contract.

Red flag: "The configuration lives in our environment, but we can always walk you through it."

3. Can my analysts change it without calling you?

Investment processes move every quarter. A platform worth buying lets a PM or analyst adjust an agent when the process moves. If every tweak becomes a change request, your research process now runs on the vendor's calendar.

Strong answer: analysts edit an agent in plain language and see the change in the next run.

Red flag: "Send us the change and we'll scope it for the next sprint."

4. How does your cost to serve change as you grow?

Ask about the gross margin trend and the ratio of deployment staff to clients. A vendor that needs more people for every new logo must raise prices or raise capital, and both end up on your desk.

Strong answer: delivery cost per client falls as the install base grows, and services revenue is reported separately from software.

Red flag: a vague line about investing in customer success, next to a hiring plan that doubles the deployment team.

5. What is our exit plan?

Ask how you get your data, outputs, agent configurations and audit logs back, in what format and how fast. For firms in the EU this is already a regulatory requirement, and a vendor that hesitates here is telling you something.

Strong answer: a documented exit plan, standard export formats and contractual notice of model changes.

Red flag: "We've never had a client leave."

6. Can I speak to a client whose embedded team has moved on?

Reference calls usually come from the accounts getting the most attention. Ask for a client where the vendor's engineers left six months ago, and ask them what still works.

Strong answer: a reference who runs the tool alone and can tell you what they changed last month.

Red flag: every reference still has the vendor's engineer on site.

What good looks like

The firms getting real value from AI buy platforms their own people can run, from vendors that stay close long after the contract is signed. In practice that means data you can trace to the source, agents your team configures and keeps, integrations with the tools you already use and pricing you can model before you sign. If you are comparing vendors now, our guide to the best enterprise AI research platforms in 2026 covers the wider field.

That is how we built Orbit. The award-winning Orbit Insight platform gives investment teams prebuilt agents in the Agent Marketplace, including the Orbit Screening Agent for stock and portfolio screening. With Orbit Agent Builder, your analysts describe their methodology in plain language and get a purpose-built agent that runs it across the whole coverage universe, with every source cited and every run auditable.

Those agents draw on Orbit's own Knowledge Base, built and maintained by Orbit from 70M+ documents across 75,000+ companies in 120+ markets, from filings and transcripts to broker research. Every answer cites its source.

Orbit is open by design. Every agent built in Orbit Insight is automatically an Orbit MCP endpoint and available through our APIs, so it runs inside Claude, ChatGPT Enterprise or your own internal tools.

Orbit works with partners including FactSet and SimCorp, and is listed in Claude's Connectors Directory. Pricing is transparent, very competitive and scales with the use case.

Choosing an AI vendor is a decision about who runs part of your investment process for years. We see every client relationship as a long-term partnership, and we expect to be held to that. Ask the six questions. Ask them of us too. These are conversations that matter deeply to us at Orbit.

Put Orbit through the six questions. Book a demo or start a free trial and run it without us in the room.