标签：#leadgen

Five states, one query language, and an "active licenses only" checkbox that only actually filters two of them. That's the trap in scraping US childcare-license open-data registries: Socrata SODA makes every state's API look identical, but "active" is defined — or not defined at all — differently in every dataset. Quick answer New York, Connecticut, Colorado, Delaware, and Texas all publish their childcare-facility registries through Socrata, and all five accept the same $where query syntax. But only NY and CT ship a server-side status filter this Actor can apply. Colorado and Delaware have no status column in the dataset at all — there's nothing to filter on. Texas does have a status column ( operation_status ), it's just not wired into the active-only filter, so toggling activeOnly doesn't touch Texas rows either way. Treating "active only" as a global switch that behaves the same everywhere will silently hand you closed and revoked facilities in three of the five states while you believe you filtered them out. STATE_CONFIGS : dict [ str , StateConfig ] = { " NY " : StateConfig (..., col_status = " facility_status " , active_where = " facility_status= ' Active '" ), " CT " : StateConfig (..., col_status = " status " , active_where = " status= ' ACTIVE '" ), " CO " : StateConfig (..., col_status = None ), # no status column to filter on " DE " : StateConfig (..., col_status = None ), # no status column to filter on " TX " : StateConfig (..., col_status = " operation_status " ), # status exists, filter isn't wired } Why does "active only" do nothing in three states? Because the filter is applied per-state, not globally, and only two states have both a status column and a configured $where fragment for it: async def _fetch_page (...): params = { " $limit " : str ( page_limit ), " $offset " : str ( offset ), " $order " : config . order_key } if active_only and config . active_where : params [ " $where " ] = config . active_where return await _get_with_retry ( session

"Just query the Socrata API" is true and also useless advice. Socrata SODA is a real open standard — New York, Connecticut, Colorado, and Texas all expose their professional-license registries through the same $limit / $offset / $where query language. The standard ends there. What each state puts inside that standard is four unrelated data models wearing the same protocol. Quick answer Every state's cosmetology/barber/salon registry is one giant multi-profession table with its own column names, its own beauty-credential filter, and its own idea of what "active" means — and one state (Texas) doesn't expose a status column at all, so an activeOnly toggle is a silent no-op there. A generic Socrata client that assumes one schema will either miss most of the data or crash on the first state whose columns don't match. The fix is a per-state config object that maps each state's real column names to one canonical output row, with the active-license filter applied only where the underlying data supports it. @dataclass ( frozen = True ) class StateConfig : state : str endpoint : str order_key : str col_business_name : str | None col_licensee_name : str | None col_status : str | None base_where : str | None = None active_where : str | None = None Why does the same query return different professions per state? Cosmetology licenses don't get their own dataset — they're rows buried inside each state's entire professional-licensing table, next to electricians, dentists, and notaries. Filtering has to happen server-side, in SoQL, before pagination even starts, or you're downloading (and paying to store) irrelevant rows. Texas needs a starts_with() match across three license-type prefixes plus an Establishment wildcard; Connecticut needs an exact in() list of six credential names; Colorado needs a four-code in() list: TX_BEAUTY_WHERE = ( " starts_with(license_type, ' Cosmetology ' ) " " OR starts_with(license_type, ' Class A Barber ' ) " " OR starts_with(license_type, ' Barber ' ) "