Data sources

The commercial data behind a launch, and how each feed breaks.

You did not create most of this data. It arrives from vendors on their schedule, in their format, and it changes without telling you. Below is the whole estate: 37 sources across four groups, what each one is, how it lands, and the way it characteristically fails.

The shape of the problem

The failures are relational, so column-level checks never see them.

The picture of a prescriber or an account gets assembled from several feeds, so no single table is ever complete. A partial file, a restatement, and a realignment all pass a null check, because the data that arrived is perfectly valid and simply incomplete, restated, or remapped.

Operational

Transactional and operational feeds

Data generated by your own commercial operation and by the partners who dispense and distribute for you. You have more control here than anywhere else, which is exactly why these get governed and the vendor feeds do not.

Source	What it is	How it arrives	How it breaks
CRM activity	Rep calls, sample drops, and field activity out of Veeva or equivalent.	Daily, internal system, generally well structured.	Activity logged against an account that never made it into the account master, so calls disappear from the roll-up.
Specialty pharmacy dispense	Dispense, shipment, and patient status from each pharmacy in the network.	Weekly or daily, one file per pharmacy, each with its own format and calendar.	No two pharmacies agree on what ship date, fill date, or patient status means, and each de-identifies patients its own way, so one patient appears three times under three tokens.
Medical and pharmacy claims	Adjudicated claims showing real utilization and treatment paths.	Weekly or monthly, large, lagged by adjudication.	Malformed NPI, NDC, or ICD values fail the join silently, so every downstream number comes in quietly low with nothing erroring.
Patient services and hub	Enrollment, benefits verification, prior authorization, and copay activity.	Daily or weekly from the hub vendor.	Milestone dates out of sequence, such as a ship date before enrollment. Not a format error, so only a test that understands the sequence catches it.
Copay and adjudication	Copay card redemption and patient out-of-pocket support.	Weekly from the copay vendor.	Redemptions that never reconcile to a dispense, usually a plan or bridge-file mapping change rather than a data error.
REMS program data	Use within the certified components of the delivery chain.	Daily, weekly, or monthly depending on the program.	Certification status changes that are not reflected downstream, which turns a compliance report into a wrong answer.
3PL and distribution	EDI 867 dispense and 852 inventory-style feeds from distributors and 3PL.	Weekly, EDI, positional formats.	A unit-of-measure change from units to packages, or a price arriving in cents. One product shows a 10x jump and the dashboard renders it as a heroic sales week.
Stocking	Which pharmacies actually carry the brand.	Weekly, syndicated or distributor sourced.	Silent coverage gaps when a chain stops reporting, which reads as lost stocking rather than missing data.
Non-personal promotion	Email, web, banner, and other digital channel engagement.	Weekly, one feed per channel or agency platform.	Every channel counts an interaction differently, so clicks, views, and reads cannot be summed into a single engagement number without a definition nobody wrote down.
Speaker programs and events	Medical education events, attendance, and spend.	Periodically from the agency or event platform.	Attendee records that do not resolve to a prescriber, so lift analysis silently drops the attendees who matter most.
DTC coverage	Direct-to-consumer geographic coverage for lift analysis.	Monthly, geography-keyed.	Geography keys that do not match the current alignment, so lift lands in the wrong territory.

Syndicated

Syndicated and purchased datasets

The data you did not create, cannot control, and pay the most for. It feeds your most visible numbers and produces your most characteristic failures. Restatement is normal behaviour here, not an error, which is what makes it hard to test.

Source	What it is	How it arrives	How it breaks
Prescription and prescriber feeds	Prescription-level and prescriber-level syndicated data from IQVIA, ICON/Symphony, and equivalents.	Weekly and monthly, often wide format with periods spread across columns.	Restatement on refresh. The vendor revises recent months on every delivery while older periods stay fixed, so a rule that forbids change fires constantly and a rule that ignores change misses real breaks.
Sub-national volume	Volume by geography, specialty, and payer type, the basis for share.	Weekly or monthly.	A shift in the mix of payer types, specialties, or geographies. Usually an upstream mapping regression rather than real market movement, and by the time an analyst notices, the number has been in three meetings.
Payer, plan, and formulary	Plan tracking, formulary status, and payer hierarchy.	Monthly, often needing a bridge file to join to plan-level data.	Formulary status changes that arrive without effective dates, so share movement gets attributed to the wrong quarter.
Longitudinal patient and source of business	Patient-level treatment history behind adherence and switching analysis.	Monthly, large, de-identified.	Token changes between deliveries that break patient continuity, quietly resetting adherence and persistence curves.
Net price and contracting	Contract terms, net pricing, and rebate detail behind demand-based P&L.	Monthly, finance-adjacent, tightly controlled.	A methodology change between deliveries that moves net price without any structural change to the file.
Real-world evidence	Outcomes and utilization evidence supporting access strategy.	Periodically, study-shaped rather than operational.	Cohort definitions that drift between refreshes, so two analyses of the same question disagree.
Prescriber reference and letter shop	Purchased prescriber reference data used for targeting and mailing.	Monthly or quarterly.	Two prescriber files that disagree. One source says Dr. Smith wrote 30 scripts, the master says there are two Dr. Smiths, and now the targeting list has a ghost and alignment double-counts.
Consumer and genomics	Consumer segmentation and, for some therapies, genomic data.	Periodically.	Join keys that exist in the vendor file and nowhere in your estate, so coverage looks complete and is not.

Reference

Reference and master data

The dimensions everything else joins to. This is where a launch platform is won or lost: a million physicians practise in the US and roughly 40,000 are targets for a given drug, and sales compensation rides on getting that list right.

Source	What it is	How it arrives	How it breaks
HCP master	The prescriber dimension, record-linked across every source that names a prescriber.	Assembled continuously from syndicated, specialty pharmacy, claims, and CRM.	A prescriber present in one source and absent from the master, so their activity never reaches a report.
HCO master	The organization and account dimension.	Assembled from shipments, interactions, and syndicated data, often via Veeva Network.	Mastered records whose alignment or sales credit changes without the downstream marts being rebuilt.
HCO affiliations	Organization-to-organization hierarchy and parentage.	From the MDM system and syndicated affiliation files.	Hierarchy loops and orphaned parents that make roll-ups double-count or drop whole branches.
Account master	The account list the field and the brand team both work from.	Internal, maintained by your team.	Accounts with sales activity that never made it into the master, which column-level checks cannot see because every value is valid.
Territory hierarchy and alignment	Zip-to-territory rosters, sales alignments, and the reporting hierarchy.	Periodically, and always at the worst moment.	A realignment drops a rep 40 percent overnight while the underlying data stays perfectly correct. Alignment has to be tested as a relationship between feed and mapping, not as a property of either.
Product hierarchy and market basket	Product, SKU, and competitor groupings behind share.	Internal, with syndicated market basket definitions.	An NDC remapped from one market basket to another. Nothing errors, both group totals shift, and every trend chart built on them is about to look wrong.
Specialty mappings	Prescriber specialty groupings used for targeting and segmentation.	From reference data and internal overrides.	Unmapped specialties silently bucketing into "other", which quietly shrinks the target segment.
Target lists and call plans	Who the field is meant to see, and how often.	Periodically from sales operations.	Targets referencing prescribers who no longer exist in the current master.
IC quota and grade	The inputs to incentive compensation ranking.	Weekly or by comp period.	A duplicated prescriber splits credit across two records and the comp run pays the wrong reps. Nobody notices until the comp run, and then everybody notices.
Time and calendar	Fiscal periods, comp periods, and vendor week definitions.	Internal.	Three different definitions of a week across three vendors, all of them reasonable, none of them the same.

Derived

Derived and gold datasets

What the business actually consumes. These fail the most expensively because they fail invisibly: every column is populated, every format is valid, and the logic is wrong.

Source	What it is	How it arrives	How it breaks
Market share and brand performance	TRx, NRx, equalized volume, and share by geography, payer, and specialty.	Rebuilt on every load.	Published totals stop tying back to source detail after an integration step starts dropping or double-counting rows.
Funnel and waterfall metrics	Stage-classified patient or account records.	Rebuilt on every load.	A gap in the classification logic and records fall out of the funnel; an overlap and they get counted twice. Either way the funnel stops adding up, and someone finds out in a QBR.
Patient journey and treatment views	Milestone-dated views behind adherence, persistence, and discontinuation.	Rebuilt on every load.	Milestones in an impossible order, which is a logic error rather than a format error.
Field performance and activity	Actual versus plan for calls, samples, and coverage.	Weekly.	Activity and performance computed against different alignment versions, so the two never reconcile.
Promotional mix and ROI	Lift analysis and return calculation across channels.	Monthly.	Attribution windows that shift when a channel feed is late, changing the answer without changing the method.
Forecast versus actual	Brand performance against the current forecast.	Weekly, with quarterly forecast updates.	A silent vendor restatement moves the actuals under a forecast you already briefed to leadership. Same query, different answer.
Demand-based P&L	TRx, SKU, pricing, and contracting combined into a demand view of profitability.	Monthly.	Pricing and contracting effective dates that disagree with the volume periods they are applied to.
Analytic Master	The record-linked single source of analytic truth for prescribers and organizations, mastered and non-mastered alike.	Rebuilt continuously; the MDM system is one input, not the whole answer.	Non-matched records quietly excluded rather than surfaced for stewardship, which is how a launch loses prescribers that no report ever mentions.

Identifiers

The join keys, and why they fail without erroring.

These thread through every feed above. A malformed identifier does not throw an error. It silently fails to join, and the damage surfaces downstream as a quietly wrong number with no obvious cause.

Identifier	What it keys	The trap
NPI	National Provider Identifier, the primary prescriber key.	Carries a Luhn check digit. A regex confirms the value looks like an NPI; only the check digit confirms it is one.
DEA	Prescriber and facility registration number for controlled substances.	Has its own checksum, which pattern matching does not verify.
ME	AMA physician identifier, common in purchased prescriber reference data.	Present in some sources and absent in others, so it works as a linking key only part of the time.
Syndicated provider ID	Vendor-assigned prescriber key.	Stable within a vendor and meaningless across vendors, so it cannot be the master key.
NDC	National Drug Code identifying product, strength, and package.	Segment formats differ, and a 10-digit NDC in an 11-digit field still looks like an NDC. Validity against the spec is only half the question; whether it is an active product needs a current external list.
ICD-9 / ICD-10	Diagnosis coding behind indication and cohort analysis.	Similar enough that a generic format rule accepts both, so the wrong revision passes and joins to nothing.
HCPCS	Procedure and product coding used in medical claims.	Requires validation against the current published set, which changes over time.
CMS place of service	Where care was delivered, used for site-of-care analysis.	A small controlled vocabulary that vendors extend informally.
Plan and formulary IDs	Payer, plan, and formulary keys behind access analysis.	Frequently need a bridge file to join, and the bridge is the thing that goes stale.
De-identified patient tokens	Patient keys in specialty pharmacy and longitudinal data.	Each source tokenizes its own way, so the same patient can appear several times and you cannot tell.

What we do about it

Two checks on every feed, not one.

Most teams check that the file arrived and stop there. The second check is the one that catches a vendor quietly rewriting history you already briefed to leadership.

Extract to load

Did the file arrive intact?

Count rows against the vendor control record and sum the amount column on both sides at fixed precision. Confirm the max load date is today. Where there is no control count, bound the volume against that feed's own history rather than a fixed threshold, because a specialty feed legitimately shrinks on a holiday week. These run in seconds and stop partial, stale, and truncated files in staging before anything reaches a mart.

Period over period

Did last month change underneath us?

Keep a snapshot of each period as it was received and diff the overlapping periods when the next file lands. Expected corrections get reviewed and accepted; silent rewrites get flagged before they reshape a trend line. This is the check most teams skip, and it is the one that protects your credibility with leadership.

Reconciliation on every IQVIA, Symphony, and specialty pharmacy feed is part of the build, not a phase two nobody gets to.

Questions

About the feeds.

What data sources do you work with?

IQVIA, ICON/Symphony, Veeva, specialty pharmacy dispense feeds, medical claims, non-personal promotion, real-world evidence, copay, and EDI 852/867. There is no learning curve on any of them. We work with the data you already receive rather than asking you to standardize it first.

What actually goes wrong with commercial pharma data?

Partial files that look complete, vendor restatements that read as growth, and territory realignments that make a rep's numbers drop 40 percent overnight. None of those trip a null check, because the data that arrived is perfectly valid and simply incomplete, restated, or remapped. The way you find out is a phone call from the field.

Why is commercial pharma data harder than other data?

Because you did not create most of it, and the failures are relational rather than local. A prescriber or account record gets assembled from several vendor feeds arriving on their own schedules in their own formats. Identifiers like NPI, NDC, and DEA fail silently and quietly drop rows on the join, so every downstream number comes in low with nothing erroring.

Can you handle specialty pharmacy data?

Yes, and it is usually the messiest part of a launch. A specialty brand pulls dispense data from a dozen pharmacies or more, each with its own file, schedule, and format. None of them agree on what a ship date, a fill date, or a patient status means, and each de-identifies patients its own way. Stitching that into one patient journey is the work.

How do you know a vendor feed arrived intact?

Two checks, not one. Extract-to-load reconciliation counts rows and sums amounts against the vendor's control record, so partial and truncated files stop in staging. Then a snapshot diff compares each period against what the vendor sent last time, which is the check most teams skip and the one that catches a silent restatement before it reshapes a trend line.

All questions

Which of these is your worst feed?

Tell us and we will walk it end to end on your screen: what reconciliation looks like on it, what breaks today, and what your AI tool would answer if it were pointed at the right shape of data.