Same API standard, four incompatible schemas: scraping state cosmetology license registries

"Just query the Socrata API" is true and also useless advice. Socrata SODA is a real open standard — New York, Connecticut, Colorado, and Texas all expose their professional-license registries through the same $limit / $offset / $where query language. The standard ends there. What each state puts inside that standard is four unrelated data models wearing the same protocol. Quick answer Every state's cosmetology/barber/salon registry is one giant multi-profession table with its own column names, its own beauty-credential filter, and its own idea of what "active" means — and one state (Texas) doesn't expose a status column at all, so an activeOnly toggle is a silent no-op there. A generic Socrata client that assumes one schema will either miss most of the data or crash on the first state whose columns don't match. The fix is a per-state config object that maps each state's real column names to one canonical output row, with the active-license filter applied only where the underlying data supports it. @dataclass ( frozen = True ) class StateConfig : state : str endpoint : str order_key : str col_business_name : str | None col_licensee_name : str | None col_status : str | None base_where : str | None = None active_where : str | None = None Why does the same query return different professions per state? Cosmetology licenses don't get their own dataset — they're rows buried inside each state's entire professional-licensing table, next to electricians, dentists, and notaries. Filtering has to happen server-side, in SoQL, before pagination even starts, or you're downloading (and paying to store) irrelevant rows. Texas needs a starts_with() match across three license-type prefixes plus an Establishment wildcard; Connecticut needs an exact in() list of six credential names; Colorado needs a four-code in() list: TX_BEAUTY_WHERE = ( " starts_with(license_type, ' Cosmetology ' ) " " OR starts_with(license_type, ' Class A Barber ' ) " " OR starts_with(license_type, ' Barber ' ) "