Every compliance team eventually asks the same question: should we build our own regulatory-watch tool or buy one? The pitch for building is seductive. A few scrapers, a shared inbox, maybe a keyword alert on a handful of regulator websites. It looks cheap because the first version is cheap. The real cost shows up later, in the maintenance, the coverage gaps and the silent drift that nobody puts in the budget.
This is not an argument that building is always wrong. It is an argument for pricing the whole thing honestly before you commit.
The demo always works
A prototype that pulls the latest press releases from three or four regulators can be stood up in a sprint. It demos well. Someone runs a search, a fresh guidance document appears, and the room nods. The trouble is that a regulatory-watch tool is not judged on the day it works. It is judged on the day a regulator publishes something material and your tool stays quiet.
The gap between "works in the demo" and "trustworthy in production" is where the money goes.
Maintenance is the product
Regulator websites are not APIs. They are living sites maintained by dozens of separate institutions with no shared standard. That means:
- Layouts change without notice. A CSS class you scraped against gets renamed and your feed goes dark. You often find out weeks later.
- Formats vary wildly. Some publish clean HTML, some publish PDF only, some bury documents inside a JavaScript-rendered portal that a simple fetch cannot see.
- Structure is inconsistent. Publication dates, document types and languages are labelled differently at every source, so normalising them into something searchable is ongoing work, not a one-time script.
- Duplication and noise. The same measure can appear as a consultation, then an opinion, then a final regulation. Deduping and classifying that is a genuine data problem.
None of this is glamorous, and none of it ends. A watch tool is not a project you finish; it is a system you feed. Budgeting one engineer to build it and no one to keep it alive is the most common and most expensive mistake.
Coverage is a moving target
The scope of a serious European watch function is broader than most in-house builds assume. You are not tracking one regulator. Depending on your footprint you may care about market authorities (AMF, BaFin, CNMV, CONSOB, CSSF, FINMA, FCA), the European bodies (ESMA, EBA, ECB, EIOPA), data protection (CNIL, the EDPB and national DPAs) and, increasingly, cyber and crypto rulemaking on top. Each new source is another scraper, another format, another thing to monitor for breakage.
Coverage also has a language dimension. A German or Italian regulator publishes in German or Italian. If your team works in English, translation and consistent tagging become part of the pipeline, not an afterthought.
The practical question is not "can we scrape one more site?" It is "can we keep sixty sites reliable at the same time, forever?"
The drift nobody budgets for
Here is the failure mode that rarely appears in the business case. A scraper breaks quietly. It does not throw an error; it just returns an empty list, which looks exactly like "no new publications." Your dashboard stays green. Weeks pass. Then an examiner or an auditor asks why you missed a guidance document that was published in plain sight.
This is silent coverage drift, and it is dangerous precisely because it is invisible. Detecting it requires monitoring the monitor: expected-publication baselines, freshness checks per source, alerts when a normally chatty regulator goes unusually quiet. Very few in-house builds ever reach that level of maturity, because by the time the tool is "done enough" the team has moved on to the next priority.
A fair way to compare
When you run the build-versus-buy numbers, count the parts people leave out:
| Cost item | Often counted | Usually forgotten |
|---|---|---|
| Initial build | Yes | - |
| Ongoing scraper repair | Sometimes | Often |
| New-source onboarding | No | Yes |
| Normalisation and dedup | No | Yes |
| Drift detection | No | Almost always |
| Key-person risk | No | Yes |
The last row matters. An in-house tool is often held together by one person's knowledge. When they leave, the coverage map leaves with them.
Where a dedicated tool earns its keep
The case for buying is not that scraping is hard. It is that reliable, wide, maintained coverage is a full-time job that scales badly inside a compliance team whose actual job is compliance. A dedicated regulatory-watch product spreads that maintenance cost across every customer, keeps the source list current and treats drift as a first-class problem.
Seqlense Doc is built for exactly this shape of work: watch across roughly eighty-five European regulators spanning finance, data protection, audit, insurance and cyber, with a query language (SEQQL) that lets you filter by source:, doctype: and lang: instead of babysitting scrapers. The point is not that you could never build it. It is that someone should be maintaining it every single day, and that someone probably should not be you.
Before you build, price the maintenance, the coverage and the drift. If the honest total still favours building, build. Just do it with your eyes open.