We Spent Months Cleaning SEC EDGAR 13F Data So You Don't Have To

The data isn't the hard part. Cleaning it is. SEC EDGAR is public, free, and a mess. Every quarter, 13,000+ institutional investment managers file Form 13F, disclosing their U.S. equity holdings. In theory that's a beautiful dataset — every hedge fund, every pension fund, every bank, all in one place. In practice, the raw filings actively fight you: The same institution files under different names, sometimes even in the same quarter. In our own database right now: BlackRock, Inc. and BlackRock Inc. are two distinct CIK registrations for what most people would call "one" institution. Multiply that across 13,000+ filers and you get a long tail of near-duplicate names that break any naive GROUP BY institution_name . CUSIPs don't map cleanly to tickers. Foreign issuers frequently use CUSIP prefixes that don't resolve through the usual reference data — we ended up building a fallback resolution path (Yahoo Finance lookups plus manual validation rules) just to keep ticker coverage from silently degrading over time. Quarters arrive gradually, not all at once. The SEC gives managers up to 45 days after quarter-end to file. If you naively take "the latest quarter with any data" as your reporting period, you'll rank a barely-started quarter — where only a handful of small filers have reported so far — ahead of the real, complete prior quarter. We learned this the hard way: an unclamped "most recent quarter" query once let 115 newly-onboarded institutions' entire existing portfolios get counted as "new inflow" with nothing to offset them, because a first-time filer has no prior-quarter row to diff against. That's the kind of bug that doesn't throw an error — it just quietly produces a chart that looks plausible and is wrong. Amendments (13F-A) revise, replace, or partially restate earlier filings , and the XML schema itself has shifted over the years, so parsing "just the latest 13F" isn't a fixed target. None of this is exotic — it's the normal cost of working with real-world