今日已更新 80 条资讯 | 累计 20052 条内容
关于我们

Scrape Google Trends Without an API Key (Including the Scraper Flag Google Hands You)

Ken-Mutisya 2026年07月04日 05:23 1 次阅读 来源:Dev.to

Google Trends has no official API, and most wrapper libraries rot within months. But the Trends site itself runs on a keyless JSON API that anyone can call, and it serves the exact numbers you see in the UI. Here is the full recipe, including one gotcha where Google quietly labels your session a scraper. The two step flow Trends works in two steps. First you call explore , which returns a list of widgets, one per chart on the page, each with a signed token: GET https://trends.google.com/trends/api/explore ?hl=en-US&tz=0 &req={"comparisonItem":[{"keyword":"web scraping","geo":"US","time":"today 12-m"}],"category":0,"property":""} Then you call a widget data endpoint with that widget's request and token : GET https://trends.google.com/trends/api/widgetdata/multiline?hl=en-US&tz=0&req=<widget.request>&token=<widget.token> The widget kinds map to endpoints: TIMESERIES uses multiline , GEO_MAP uses comparedgeo , and both RELATED_QUERIES and RELATED_TOPICS use relatedsearches . The cookie trick Call explore cold and you get a 429. The API wants a NID cookie, and here is the counterintuitive part: you get it by requesting the public explore page first, and that page may itself respond 429 while still setting the cookie you need. const res = await fetch ( ' https://trends.google.com/trends/explore?geo=US&q=test ' ); // res.status may be 429. The Set-Cookie header is still there. const cookies = res . headers . getSetCookie (); Grab the cookie from the 429 response, retry explore , and everything works. Strip the anti JSON prefix Every Trends response starts with a junk line like )]}' to break naive JSON.parse calls. Drop everything up to the first newline: const body = await res . text (); const data = JSON . parse ( body . slice ( body . indexOf ( ' \n ' ) + 1 )); The scraper flag Here is the part I have not seen documented. Look inside the widget request object that explore returns to a keyless session: "userConfig" : { "userType" : "USER_TYPE_SCRAPER" } Google knows. And

本文内容来源于互联网,版权归原作者所有
查看原文