Most commercial data agreements describe the file and stop. The columns are listed. The delivery day is named. Nothing says what happens when the file arrives short, late, or rewritten.
That gap is how a short file reaches the field report.
At one rare-disease biotech, the specialty pharmacy sending four files a day was the single biggest source of trouble. Getting one fix landed took several rounds, and nobody in the chain held a service level agreement worth invoking.
A spec is useful only when each line can fail a delivery. Write every clause so a test can check it.
The checklist
| Clause | What to write down | The test that enforces it |
|---|---|---|
| Grain | One row per what: prescriber-week, pharmacy-patient-day, invoice line. Long format, one period per row | Duplicate check on the stated grain key |
| Cadence and window | Weekly or monthly, the delivery day, the latest acceptable arrival time, exactly one file per period | Freshness: max load date is current; one delivery per period |
| Control record | Row count and the total of each amount column, sent with every file | Row count and sums reconciled exactly at fixed precision |
| Layout | Column names, types, order, and allowed values | Schema drift test fails on any rename or type change |
| Units and scale | Units versus packages; dollars versus cents | Metric trend per product against its own history |
| Event and calendar definitions | What ship, fill, dispense, and referral dates mean; the week definition | Custom condition on milestone order; dates map to your fiscal calendar |
| Identifiers | NPI with check digit; NDC level and form (11-digit package, hyphenated or not); how patient tokens are generated and when they change | Identifier validation; token turnover between deliveries |
| Restatement and corrections | Which periods can restate on refresh; how a corrected file is flagged; that older periods never move | Exact match on the stable tail; shape check on the restated window |
| Change notice | Notice period before any layout, definition, or methodology change | Spec version recorded with each delivery |
| Acceptance | The test set a file must pass; who is told when it fails | Test results kept with a disposition per failure |
| Fix turnaround | How fast a rejected file is replaced; who you can call | Tracked in your ticketing, reported monthly |
Write the numbers (arrival time, notice period, fix turnaround) for your own feeds. We have not published standard values, and a number nobody tracks does nothing.
Where each clause comes from
The rows are the failures already on the record. A partial specialty pharmacy file, 26,000 rows against a normal 40,000, loads with no error. Rename RX_CNT to RxCount and the field fills with nulls. Switch a quantity from units to packages and one product jumps 10x. The vendor feed test answer turns those into the five checks every delivery gets, and the data sources reference lists the characteristic failure of 37 feeds, which is the clause each feed’s spec needs most.
Grain and format clauses exist because vendors love wide files, with periods spread across columns, and those defeat trend logic. Ask for long format. If you can’t get it, normalize before you test.
Identifier clauses need exact forms. 867 files and claims carry package-level NDCs in 11-digit form while product masters carry the two-segment product form, and a 10-digit NDC with its hyphens stripped can be three different products. Specialty pharmacies also tokenize patients their own way, which is why counting unique patients across them gives a range, not a number. The spec can’t fix that. It can require notice when a pharmacy changes its tokens.
The restatement clause is the one most specs leave out. Syndicated data is part actual, part projection, and the vendor reruns it when better inputs arrive. The restatement answer shows how to test the window and the stable tail separately. The spec should say where the line between them sits.
Make acceptance a result, not a conversation
Our data contract post describes the usual fate of a file interchange agreement: a spreadsheet sitting in someone’s inbox that nobody has opened in months. Turn each clause into a test and the agreement runs on every file. When a file fails, the vendor gets the failed clause, the field, and the size of the breach.
Open source DataOps TestGen is free, so the tests can travel too. The same post describes handing a supplier the tool and the contract and asking for a passing run before they ship.
Then put the results where the budget owner sees them. The monthly partner scorecard has a delivery reliability line, and rejected files belong on it. If you’re replacing a vendor, write the spec before the switch, because the tests that pass on the incumbent’s output are the baseline the new feed has to match.