Alternative data sounds like an edge and often is. Card-spend panels, satellite imagery, web-extracted signals, job postings, filings scraped from public sources: each promises a read on a company before the read shows up in reported numbers. The trouble is that most of the work that determines whether the signal survives contact with a backtest happens in the pipeline, not the model. A panel that looks predictive in a notebook can fall apart the moment you ask when each row was actually knowable, or which issuer it belongs to.
So this is a piece about plumbing. The unglamorous part is where alternative data either becomes a usable signal or quietly turns into a lookahead bug you ship to production.
Ingestion, normalisation, and the dating problem
Every source arrives in its own shape. Card data comes as aggregated panels with their own merchant taxonomy. Satellite vendors deliver imagery or pre-computed counts on their own cadence. Job postings and scraped filings are text, with all the parsing and deduplication that implies. Ingestion means writing down each feed’s quirks before normalising into a common schema: units, currencies, geographies, what a “week” means to this vendor versus that one.
The part that decides whether the signal is real is point-in-time capture. A row has at least two dates that matter: the event date the data describes, and the date you could first have known it. Vendors frequently restate panels, backfill corrections, and revise counts weeks later. If you store only the latest value, a backtest reads numbers that did not exist on the day it claims to trade, and the strategy looks better than it ever was. The fix is to keep a vintage history, so a signal is dated when it became knowable and corrections are added as new observations rather than overwrites. This is the same point-in-time discipline that fundamentals need; alternative data just hides the leakage better because nobody has a reference series to check against.
Licensing, compliance, and what you are allowed to use
Before any of that matters, the data licensing terms have to permit the use you have in mind. Several questions tend to decide the engagement:
- Does the contract allow derived signal data, or only internal analysis, and may outputs be shared with clients or used to manage outside capital?
- For card and similar panels, is the source consented and de-identified at the origin, and does aggregation actually prevent re-identification?
- For scraping public filings or web pages, do the source’s terms and the relevant jurisdiction’s rules permit collection at the rate and scale you plan?
- Material non-public information: a panel granular enough to read a single issuer’s results before release is a compliance problem, not an edge.
These are constraints to design around, not paperwork to bolt on later. A signal you cannot defend the provenance of is one you cannot put into a regulated process, and the audit trail for where each row came from is the thing that lets you answer the question when it is asked.
Deduplication, entity mapping, and proving the signal is real
Raw feeds repeat themselves. The same filing gets scraped twice, the same posting reappears across boards, a merchant shows up under three spellings. Deduplication has to happen before aggregation, or counts inflate in ways that correlate with collection effort rather than anything about the business.
Then comes entity resolution: mapping each record back to the issuer it concerns. Card merchants, posting employers and filing entities rarely carry clean tickers or LEIs, and the same brand can map to a subsidiary, a parent, or a private peer that is not investable at all. Get this wrong and you attribute a competitor’s signal to your name. We treat each match as an inspectable decision with a confidence, not a string-match accident, and we keep the unmatched residue visible because that is where the bias hides.
The last check is whether the signal reflects the world or the way you collected it. A jump in scraped postings can mean a hiring surge, or it can mean a vendor changed sites or you fixed a parser. The defence is boring and necessary: hold out a validation window, watch for drift when collection changes, and reconcile the alternative series against a ground truth where one exists, even a lagging reported figure. If a signal moves only when your pipeline changes, it is an artefact, and it is far cheaper to find that in evaluation than after someone has traded on it.