Some Reasons Scraping Federal Procurement Data Goes Wrong

A pretty magical thing in 2026 is that the activation energy needed to start building something is much lower than it ever has been. I hear all the time from folks about how they are building internal dashboards or agents using Claude Code, Codex, Cursor, etc.

I also hear from folks, when deciding whether to use Tango or not, that they are considering building their own ETL pipelines so they can support their tools. And, this is maybe a surprising take from us at MakeGov, but sometimes that's a perfectly fine idea!

After all, federal procurement data is public, and there are public APIs. With a coding agent, a simple, working scraper for SAM.gov or USAspending is an afternoon's work, and for a prototype or a one-time analysis that's absolutely the right way to do it. There are even great open-source tools out there (opens in new tab).

The trouble starts when someone is going to act on the number or you want to make sure that the system is reliable. Getting data can be quick. Getting it right is slow and painful, and it's slow in ways that don't always show up as errors. And each of those footguns can cost time, tokens, or bad business outcomes.

Here are two examples. Each is a real record you can look up, with a test you can run against your own pipeline.

Vehicles aren't in the data

A lot of federal money moves through contract vehicles. But there is no authoritative way of knowing whether a given IDV is part of a vehicle or not. And often times, it requires a lot of effort to figure it out.

Take NMITS, NOAA's Mission IT Services BPAs. It's 83 separate BPAs held by 82 companies, tied together by a description that NOAA's own records type nine different ways. Three of them:

BASE AWARD FOR NMITS (NOAA MISSION IT SERVICES) MAS BPA(S)
BASE AWARD FOR NMITS (NOAA MISSION
ASE PERIOD FOR NMITS (NOAA MISSION IT SERVICES) MAS BPA

If you try to match on a string, you'll get nine vehicles. But if you map them correctly, you can learn something useful: of the 82 companies holding a BPA, only 22 have ever won an order.

Try this at home: How much has gone through NMITS?

Totals sometimes don't agree

If you look up the TRICARE managed care contract awarded to Humana in 1996 (MDA90696C0002), USAspending will say that there has been $2.99 billion obligated. SAM.gov, on the other hand, will say that $4.98 billion has been obligated. What gives?

The reality is that contracting data is not static. Even contracts from decades ago are actively updated in FPDS and SAM.gov, and keeping track of all of the changes is a real effort. And if you expect the federal systems to agree with each other, I hate to break it to you, you will be disappointed.

Try this: Pull that contract from both systems and make the obligations agree. (Fun fact: SAM.gov is also likely wrong about this contract.)

Spend it on what you're building

Neither of these examples is particularly surprising for a federal practitioner. They're ordinary, and there are a lot more like them.

You could solve every problem yourself. The question is whether that's where you want your budget to go. Every hour an engineer spends working out which of nine strings is the same vehicle is an hour not spent on the product.

The same goes for tokens. An agent that has to fetch raw records, reconcile them, and work out what's missing spends its context on plumbing before it reaches the question you asked. When we measured it, reading a contract award from the raw USAspending API took about 2.6 times the tokens of reading the same award from Tango.

The interesting work is on top of the data: the capture tool, the market model, the agent that reads a solicitation and tells you whether to bid. That's where your team knows something nobody else does, and it's where the time and the tokens should go.

So in Tango the grouping is already done, and NMITS is one record:

curl -G "https://tango.dev/api/vehicles/" \
     -H "X-API-KEY: ${TANGO_API_KEY}" \
     --data-urlencode "search=NMITS"

We have the same upstream limits as everyone else. What we've done is the reading, grouping, and cross-checking, so you can start from the answer.

Get started

There's a free plan. Sign up at tango.dev (opens in new tab), or connect the Tango MCP server to Claude or ChatGPT and ask it about NMITS.

And if you have a record that breaks your pipeline, send it to hello@makegov.com. We collect them.