Buy the Exit, Not the Demo

Twenty-four questions to ask before you buy an investment data platform — and why the ones that matter are not about what it does.

These questions were drafted while evaluating data platforms for an investment firm running advisory, insurance derivatives, private asset servicing and third-party administration off a single stack. They are reproduced here with the client details removed.

Every platform evaluation starts with a demo, and the demo always works.

It works because it was built to. The data is clean, the joins are pre-configured, the dashboard renders in half a second, and the person driving has given this demo two hundred times. You will not learn anything from it that the vendor did not intend you to learn.

What the demo cannot tell you is the only thing you actually need to know: what this looks like in year three, when the person who configured it has left, an auditor is asking how a number was produced, and someone upstairs is asking what it would take to move.

So the questions worth asking are not about capability. They are about three things the demo is structured to avoid.

One: can you explain a number eighteen months later

In an investment context this is not a nice-to-have. A return figure, an attribution result, a valuation — each of these will eventually be questioned by someone with standing to question it. A regulator, an auditor, a client, an acquirer in diligence.

At that point you need to reconstruct not just the number but the path: which inputs, transformed by which logic, on which date, under which version of the schema. If the platform cannot show you that, you do not have a number you can defend. You have a number you can report, which is a different and much weaker thing.

How auditable are data pipelines, tables, triggers, and scheduled jobs?

Ask it exactly that way, and watch whether the answer covers all four. Most platforms have a good story about tables. Fewer have one about triggers and scheduled jobs, which is precisely where the silent failures live — a job that stopped running, a trigger that fired twice, a transformation that was quietly amended.

Then push on versioning, because this is where the honest answer separates from the marketed one. Most platforms version code. Far fewer version data. If a figure changed between two reporting dates, you need to know whether the inputs moved or the logic did, and you can only answer that if both are under version control with rollback and roll-forward.

Two: can you still run your own code

This is the lock-in question, and it is usually asked too politely.

If your valuation library, your interpolation routines, your options Greeks, your risk engine cannot run inside or alongside the platform, then one of two things happens. Either you rewrite them in the vendor's environment — at which point they are the vendor's, in the vendor's dialect, and they leave when you leave. Or you keep them outside and move data back and forth, which reintroduces exactly the reconciliation problem the platform was bought to solve.

The question behind the question is whose model produces your numbers. If you cannot deploy your own quant libraries, the answer is theirs.

Three: who can actually use it

There is a real tension here and most evaluations pick a side without noticing.

A platform only engineers can query becomes a bottleneck: every question routes through a queue, and the people with the questions stop asking them. A platform anyone can query without understanding the model produces confident wrong answers at speed, which is worse, because nobody knows to check.

So ask both halves. What does it take for a non-engineer to explore the data — and what stops them producing a plausible number from a bad join? Auto-join and foreign-key mapping features are the answer vendors give. They are useful. They are also how someone joins two tables that should never have been joined and gets a result that looks fine.

The six areas, and what each is really testing

1 · Investment management and performance attribution — testing whether it was built for this, or adapted to it

• Does the platform support performance and attribution tracking out of the box — time-weighted return, money-weighted return, multi-level attribution?
• Are there native tools or workflows for data validation, reconciliation and exception handling?

2 · Compliance and auditability — testing whether you can defend a number

• Do you offer features that facilitate or accelerate SOC 1, SOC 2 or GIPS certification?
• How auditable are data pipelines, tables, triggers and scheduled jobs?
• Do you provide built-in change management and versioning allowing seamless rollback and roll-forward of schema, logic or data?

3 · External compute and integration flexibility — testing whose model produces your numbers

• Can the platform ingest or interact with compute run externally — Python-based valuation libraries, risk engines?
• How flexible is the compute architecture? Are users locked into the native engine or data warehouse?
• Can users deploy custom quant libraries — options Greeks, interpolations, valuation surfaces?

4 · Usability for non-engineers — testing whether it becomes a bottleneck or a hazard

• Is native visualization supported, or is a third-party BI tool required?
• How much technical expertise is needed to explore and analyze data?
• Are there query builders or auto-join capabilities, including automatic key mapping?
• Is there a no-code or low-code interface for workflow building, automation and reporting?

5 · Structured and unstructured sources — testing whether it handles the data you actually have

• Do you support ingestion of structured and unstructured data — PDF, Excel, email?
• Are there native connectors for market data vendors?

6 · Scalability, security and maturity — testing the bill and the blast radius

• What is the pricing model, and how does it scale with data volume, users and compute intensity?
• Do you support metadata lineage and dependency tracking across tables, pipelines and dashboards?
• Is granular access control available — row-level security, data masking, role-based access?
• Can you show use cases in investment management with complex workflows and regulatory requirements?

Two of those deserve a note. Unstructured ingestion sounds like a nice-to-have until you count how much of your actual data arrives as a PDF statement or a spreadsheet attached to an email. And pricing that scales with compute intensity is the line item that turns a predictable subscription into a bill nobody can forecast — ask for the model, not the quote.

The last question

None of the twenty-four is the one I would ask last. That one is not on the list, because it is not a feature question and vendors are rarely asked it in a first meeting.

What does leaving look like? Not whether you can export — everyone can export. What comes out, in what format, with what lineage attached, and how much of the logic comes with it. If the answer is a pile of CSVs and none of the transformations, then everything you build inside the platform is rented, and the rent is renegotiated at renewal from a position you no longer control.

You are not buying a demo. You are buying the exit, and you should price it before you sign, not after.

---

Scarborough Road works with insurers and asset managers on investment data infrastructure, platform selection, and the performance and reporting systems that run on top of it. If you are mid-evaluation and the demos are starting to look identical, that is the conversation to have.

General commentary on platform evaluation. Not legal, accounting or investment advice. No vendor is endorsed or criticised here, and no client's evaluation, shortlist or selection is described.