标签：#cloudflare

A scraper fails, someone swaps the User-Agent, someone else adds a proxy, then the job starts passing locally but fails again in CI. That usually happens because Cloudflare did not block “scraping” as one thing. It evaluated several signals, and each failure needs a different fix. This is about authorized automation: your own sites, customer-approved workflows, testing, monitoring, data access you are allowed to perform. If you do not have permission to automate against a site, changing fingerprints or rotating IPs does not make it okay. Start with the failure you actually see Cloudflare failures often get collapsed into “403”, but the page body matters. Common cases: Error 1020 : usually an access denied page from a Cloudflare rule or bot score decision. The HTTP status may still be 403, so inspect the HTML. 403 without a 1020 page : often IP reputation, firewall rules, geo restrictions, or an auth problem. 429 : rate limit exhaustion. Slowing down can help here, but it will not fix a fingerprint problem. Endless Just a moment... page : your client did not complete the browser-side challenge. CAPTCHA or Turnstile loop : Cloudflare still considers the session borderline after earlier checks. Add classification before you add workarounds. Even a basic classifier saves time: import time import requests CLOUDFLARE_MARKERS = { " 1020 " : " cloudflare_access_denied " , " Just a moment " : " cloudflare_js_challenge " , " cf-turnstile " : " cloudflare_turnstile " , " cf-error-code " : " cloudflare_error_page " , } def classify_response ( resp : requests . Response ) -> str : body = resp . text [: 5000 ] if resp . status_code == 429 : return " rate_limited " for marker , label in CLOUDFLARE_MARKERS . items (): if marker in body : return label if resp . status_code == 403 : return " forbidden_unknown " return " ok " if resp . ok else f " http_ { resp . status_code } " def get_with_backoff ( url : str , max_attempts = 4 ): for attempt in range ( max_attempts ): resp = request

Cloudflare has recently introduced temporary accounts that let AI agents deploy Cloudflare Workers immediately, without first creating or authenticating with a permanent account. If left unclaimed, the accounts and their deployments expire automatically after 60 minutes. By Renato Losio

I optimized my architecture for the wrong model. I used to think black-box infrastructure was the right abstraction for AI-driven development. Vercel, Supabase, Cloudflare Workers — a sharp contract in front, a managed backend behind — felt like the obvious fit. The less an agent had to reason about, the fewer places it could get lost. Give it a clean interface, hide the messy backend, move fast. I still think that was right for the agents we had last year. I don't think it's right for the agents we're starting to use now. The shift is not that AWS got simpler. It didn't. Setup still takes longer, CI/CD takes more work to wire, and cost control has real limits. The shift is that agents got better at reading complexity — and once an agent can actually use a large structured context, the things I treated as overhead (resources, provider schemas, IAM policies, explicit queues, explicit alarms, explicit networks) become the highest-signal context I can hand it. To keep this concrete, I'm holding the tool constant. This is HCL/Terraform on AWS vs HCL/Terraform on Cloudflare — same language, same workflow, two providers. The inversion Agents are… Best served by… Context-poor (last year's loops) Black boxes — shrink the surface, hide the backend Context-rich (now) Inspectable systems — describe everything as code The one-line version: The better agents get at reading, the more valuable explicit infrastructure becomes. I didn't become more pro-AWS because AWS got easier. I became more pro-AWS because agents got better at reading it. Same Terraform, two providers The interesting comparison isn't AWS-elegance vs Cloudflare-elegance. It's how much of the infrastructure topology and operational contract an agent can reconstruct from the HCL plus the provider schema alone. Terraform × AWS Terraform × Cloudflare Provider maturity AWS provider is about as battle-tested as IaC gets; enormous public corpus of modules/examples v5 is a ground-up, OpenAPI-generated rewrite — improving

Cloudflare and AWS both implemented x402 stablecoin micropayments at their edge networks within two weeks. The open protocol under the Linux Foundation revives HTTP 402 for agent-to-service payments with sub-cent transaction costs. Coinbase reports 169 million transactions in year one. Enterprise tax and invoicing gaps remain unresolved. By Steef-Jan Wiggers

Every developer has that one project. The passion build that sits in the back of your mind for months—or even years—before you finally sit down, crack your knuckles, and make it a reality. For me, that project was building a modern, open-access bilingual digital lexicon bridging English and Assamese: AssameseDictionary.org . While it started as a personal milestone dream, it quickly turned into a massive data engineering and architecture challenge. Here is how I tackled parsing a massive vocabulary database and serving it globally with near-zero latency. 🏗️ The Core Challenge: Scale vs. Speed A dictionary isn't like a standard SaaS app or landing page. It lives and dies by its database depth. To make this a truly definitive tool, I compiled, cleaned, and programmatically validated an extensive vocabulary index mapping over 293,000 words . The dataset doesn't just hold simple translations; it maps complex bidirectional lookups, phonetic transliterations, advanced English definitions, context usage examples, and cross-linked synonym tokens. If I threw this massive dataset into a traditional relational database hooked up to a standard server setup, I ran into immediate roadblocks: Latency: Heavy search queries on a dataset this size can cause noticeable lag. Cost/Overhead: Maintaining and scaling database servers for unpredictable public traffic gets expensive fast. I wanted the search utility to snap back instantly. To achieve that, I had to ditch traditional server paradigms entirely. ⚡ The Architecture: Serverless Edge Caching To keep things ultra-lightweight, highly cost-effective, and blazing fast, I built the platform around an edge-computing topology: The Runtime: I offloaded the backend logic entirely to Cloudflare Workers . Instead of routing traffic to a centralized origin server, queries are intercepted and executed at serverless edge locations physically closest to the user. The Data Layer: Instead of an active SQL database bottleneck, I mapped the data mat

Cloudflare is giving AI companies until September 15 to separate web crawlers used for search from those used for AI training and agents, or risk being blocked by default on many publisher sites.

Why your Cloudflare Turnstile token works in the browser but 403s from requests You solved the Turnstile widget. You can see the token in the page. You copy it into your script, POST the form from requests, and the server hands you back a 403 — or a JSON body with "success": false. The token clearly worked a second ago in the browser, so what changed? Short answer: a Turnstile token is not a password you can carry around. It's a one-time, short-lived proof bound to a very specific context, and replaying it from a different context is exactly what it's designed to reject. Below is what that context is, how to tell which constraint you're hitting, and the fix for each. The real scenario You're automating a flow on a Cloudflare-protected site. There's a cf-turnstile widget on the form. You get a token one of two ways: you render the page in a real browser (Playwright/Selenium) and read cf-turnstile-response, or you hand the sitekey + page URL to a solving service and get a token back. Either way, you then submit the form with a plain HTTP client requests, httpx, axios) and it fails. The frustrating part: it's intermittent-looking. The reason it feels random is that there are four separate constraints, and you're usually tripping a different one each time. The four things a Turnstile token is bound to 1. It's single-use Once Cloudflare validates a token server-side (the siteverify call your target makes), that token is spent. Submit twice, retry, or test it once by hand, and the second use returns false. You get a fresh one per submission. 2. It has a short TTL Turnstile tokens expire fast — a few minutes. Solve early, do other work, submit later, and the token can be dead on arrival. The widget auto-refreshes in the browser precisely because tokens go stale; a script that grabs the token and sits on it loses that refresh. 3. It's bound to the sitekey and the page URL Multiple widgets. Some pages embed more than one Turnstile (login + newsletter). Solving the wrong site

Cloudflare released the Cloudflare One stack, an open-source library of agent skills for planning, deploying, and managing Zero Trust environments. The skills include automated migration logic for Zscaler and Palo Alto Networks, the same logic used in Cloudflare's Descaler program that has moved enterprise customers in hours rather than months. By Steef-Jan Wiggers

Every AI developer has been here: your app is throwing 503s, users are pinging you, and you have 12 browser tabs open — OpenAI status page, Anthropic status page, the GitHub Copilot health page, three different Discord servers — trying to figure out is this me or is it them? That's the problem we set out to solve. Prismix aggregates status from 77 AI services in one place. Six weeks of running it in production taught us some things that might save you time. The problem is worse than you think AI APIs don't fail like traditional infrastructure. They fail in weird, partial ways: Degraded performance that passes your health checks but makes your product feel broken Regional outages — OpenAI US-East is down while EU is fine, so half your users are affected Silent rate-limit cascades — the API returns 429s but their status page says "operational" for another 20 minutes Incident lag — providers often post status updates 10–30 minutes after engineers are already aware The official status pages are optimistic by design. They're customer-facing communications tools, not real-time engineering dashboards. There's nothing wrong with this — but it means you need a different mental model for "is this service down?" What 77 status pages look like in aggregate When you watch 77 AI services simultaneously, patterns emerge fast. OpenAI is the most-watched service (and has the most incidents to watch). The pattern is almost always the same: investigating → identified → monitoring → resolved , typically in 45–90 minutes. The investigating phase is where most developers panic — it looks bad but usually resolves without action on your end. Anthropic runs noticeably clean compared to its API usage growth. Incidents are rarer and shorter. When they do happen, updates arrive faster than most providers. The long tail is interesting. Services like Replicate, Runway, ElevenLabs, and Suno have incident patterns that don't correlate with OpenAI at all. If you're routing across multiple providers

Last Tuesday at 3am, a multi-agent loop hit 12K KV writes/minute and froze. The loop was a one-line counter bug. That part was fixable. What I found while tracing it was worse. I had --dangerously-skip-permissions enabled on a Claude Code session that was running D1 migrations. I thought it was pointing at staging. It wasn't — I'd misconfigured my env file reference, loading .env.production instead of .dev.vars . Claude didn't ask. The flag told it not to. The migration was ADD COLUMN , not DROP COLUMN , so no data loss. Survivable. But only barely. The thing I got wrong: I treated --dangerously-skip-permissions as "skip the annoying confirmation popups." It's actually "remove the only moment a human sees what command is about to run." Those are very different things. Turning the flag back off helps, but it doesn't constrain what Claude attempts — it just adds a prompt you'll click through anyway at 3am. What actually worked was adding a deny rule in .claude/settings.json : { "permissions" : { "allow" : [ "Bash(wrangler d1 execute * --local*)" ], "deny" : [ "Bash(wrangler d1 execute *)" ] } } The allow rule is more specific than the deny, so --local calls go through and everything else is blocked before execution. Over 2 weeks post-fix, Claude attempted zero production DB commands. Three deny events were logged — all from ambiguous prompts I wrote during fast context-switches, not from Claude going rogue. I ended up running three layers: the settings.json allowlist, a separate git worktree for migration work that physically contains only staging credentials, and a CLAUDE.md that instructs Claude to ask before anything touching production. The CLAUDE.md approach has a real caveat though — in long sessions the instructions lose weight as context grows. Anything critical needs to be restated in the prompt itself. I wrote up the full breakdown — including the worktree setup, the exact CLAUDE.md wording, and why MCP tool permissions behave inconsistently with the deny ru

You run a Telegram bot through OpenClaw on your own Linux server. One day it goes quiet. The bot can't send or receive. But the server itself is fine — SSH works, apt works, other sites load. The reason: your server can't reach api.telegram.org . Some networks block or throttle it, so every call times out while everything else is fine. The clean fix: route only OpenClaw's Telegram traffic through Cloudflare WARP (1.1.1.1) . Everything else on the box stays direct and fast — including your SSH login. Here is the full setup, step by step. First, confirm it's a Telegram-only problem curl --max-time 8 https://api.telegram.org/ # hangs / times out curl --max-time 8 https://www.google.com/ # works instantly If Telegram times out but other sites are quick, this guide is for you. How it works We chain three small tools: OpenClaw ──▶ iptables ──▶ redsocks ──▶ WARP (SOCKS5) ──▶ Cloudflare ──▶ api.telegram.org WARP gives us a local proxy that exits through Cloudflare's network (which can reach Telegram). redsocks turns normal connections into proxy connections (so the app needs no proxy support — OpenClaw has none). iptables picks only OpenClaw's Telegram traffic and sends it to redsocks. The trick is in that last step. We match by the app's user and Telegram's IP ranges, so nothing else is touched. Step 1: Install WARP in proxy mode # Add Cloudflare's package repo curl -fsSL https://pkg.cloudflareclient.com/pubkey.gpg \ | gpg --yes --dearmor -o /usr/share/keyrings/cloudflare-warp-archive-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/cloudflare-warp-archive-keyring.gpg] \ https://pkg.cloudflareclient.com/ $( lsb_release -cs ) main" \ > /etc/apt/sources.list.d/cloudflare-client.list apt-get update && apt-get install -y cloudflare-warp # Sign up (free) and switch to proxy mode warp-cli --accept-tos registration new warp-cli --accept-tos mode proxy # opens a SOCKS5 proxy on 127.0.0.1:40000 warp-cli --accept-tos connect Now check that WARP can reach Telegram: curl --socks5-

I run a content site, do-and-coffee.com . Like everyone else, it gets scraped by AI crawlers. Instead of blocking them, I did something else: I put a paywall in front of the site that returns HTTP 402 Payment Required to bots, with machine-readable payment instructions. If a crawler pays a cent in USDC, it gets the article. If it doesn't, it gets the 402 and nothing else. Then I let it run for 30 days and watched. Here's what actually happened — and it's not the number you'd put on a pitch deck. TL;DR A Cloudflare Worker sits in front of the site. AI crawlers get 402 + x402 payment requirements ; humans and search bots pass through free. Payment is USDC on Base , $0.01 per article, verified and settled through Coinbase's CDP facilitator. 30-day result: 5,811 crawler requests, 5 paid, 5,806 served a 402. Revenue at $0.01/article ≈ $0.05 . The interesting part isn't the revenue. It's who paid: GPTBot paid 4 times out of 48 requests; ClaudeBot paid once out of 651. Architecture do-and-coffee.com/blog/article/* ─▶ x402 Worker (Cloudflare) │ has X-PAYMENT-RESPONSE? ───────────┤─▶ yes ─▶ proxy origin (200) KV cache hit (payer:url)? ─────────┤─▶ yes ─▶ proxy origin (200) no X-PAYMENT? ─────────────────────┤─▶ 402 + payment requirements has X-PAYMENT? ────────────────────┘ │ ├─▶ CDP /verify (is the signed payment valid?) ├─▶ CDP /settle (waitUntil: confirmed — on-chain) └─▶ on success: KV.put(payer:url, receipt, ttl 24h) ─▶ proxy origin The worker speaks the x402 protocol: a 402 response carries an accepts array describing exactly how to pay (scheme exact , network base , asset USDC, amount, payTo wallet). A compliant agent reads that, signs a USDC payment, and retries with an X-PAYMENT header. The worker verifies and settles it through Coinbase's facilitator, then proxies the real article. How it works The 402 response When there's no payment, the worker builds the requirements and returns 402: function buildPaymentRequirements ( resourceUrl : string , env : Env ): Payment

When I built whatsmy.fyi , I assumed I'd need a geolocation provider: MaxMind, ipinfo, ip-api, pick your poison. They all mean the same thing: an external dependency, an API key, a quota, added latency, and someone else's server seeing your users' IPs. Then I found out Cloudflare Workers makes the whole category unnecessary. The cf object Every request that hits a Cloudflare Worker carries a request.cf object, populated at the edge before your code even runs. No lookup, no latency, no key. Here's what's inside: { asn : 34984 , // ISP's autonomous system number asOrganization : " Superonline " , // ISP name city : " Istanbul " , region : " Istanbul " , country : " TR " , continent : " AS " , isEUCountry : undefined , // "1" if EU, undefined otherwise latitude : " 41.01380 " , // string, not number! longitude : " 28.94970 " , postalCode : " 34000 " , timezone : " Europe/Istanbul " , colo : " IST " , // which CF datacenter handled this clientTcpRtt : 12 , // user's RTT to the edge, in ms httpProtocol : " HTTP/3 " , tlsVersion : " TLSv1.3 " , tlsCipher : " AEAD-AES128-GCM-SHA256 " } That last group surprised me most: you get the user's HTTP protocol, TLS version, and actual TCP round-trip time for free. Try getting that from a geo API. A complete IP endpoint in ~30 lines export default { async fetch ( request ) { const cf = request . cf ?? {}; const ip = request . headers . get ( " CF-Connecting-IP " ); return Response . json ({ ip , city : cf . city ?? null , country : cf . country ?? null , isp : cf . asOrganization ?? null , asn : cf . asn ?? null , timezone : cf . timezone ?? null , lat : cf . latitude ? parseFloat ( cf . latitude ) : null , lng : cf . longitude ? parseFloat ( cf . longitude ) : null , protocol : cf . httpProtocol ?? null , tls : cf . tlsVersion ?? null , rttMs : cf . clientTcpRtt ?? null , }); }, }; That's the entire backend. No database, no GeoIP file to update monthly, no vendor. The gotchas (learned the hard way) 1. Coordinates are strings. lati

Building a "paste a YouTube link, get a transcript" feature sounds trivial until you deploy it to a server. The moment your request comes from a datacenter IP instead of a residential one, YouTube responds with LOGIN_REQUIRED or quietly serves nothing. Here's how VidTranscriber handles it. The problem There are two ways to get text from a YouTube video: Existing captions — if the uploader (or YouTube's auto-caption) provides them, you can fetch the caption track directly. Fast, free, no transcription needed. Transcribe the audio — pull the audio stream and run it through a speech-to-text model (Whisper-family). Works for any video, but costs compute. Both start with talking to YouTube from your server — and that's where it breaks. YouTube aggressively gates datacenter traffic: the watch page and InnerTube API return LOGIN_REQUIRED , and naive audio fetching gets reCAPTCHA'd. The approach The fix is to separate where the request originates from where the work happens : A Cloudflare Worker handles the user request and orchestration. Caption/audio fetching is routed through a path whose egress isn't treated as a bot — so the LOGIN_REQUIRED wall doesn't trigger. Captions, when available, become the primary path (no transcription cost). Only when there are no usable captions do we fall back to downloading audio and running Whisper. Long jobs go onto a queue (Cloudflare Queues) so the request returns immediately and the transcript streams in as it completes. Why captions-first matters Most "transcript generator" traffic is for videos that already have captions — talks, tutorials, news. Serving those from the caption track is instant and free, which means the expensive Whisper path is reserved for the minority of videos that actually need it. That's the difference between a tool that's cheap to run and one that isn't. What's still hard IP reputation drifts — what works today can get throttled tomorrow, so the extraction path needs monitoring and fallbacks. Caption quality

Cloudflare recently described how a slowdown in its billing pipeline was traced to contention inside the query planning stage of ClickHouse. The team profiled the bottleneck and patched ClickHouse to replace an exclusive lock with a shared lock, drop the per-query copy of the parts list, and improve part filtering. By Renato Losio

I recently rebuilt my homelab from scratch. The goal was simple: one machine, everything containerised, zero exposed ports, GPU-accelerated local AI, and a fully automated backup setup. No cloud subscriptions for the tools I use every day. This is the full technical breakdown — what I'm running, how it's wired together, and the hard-won fixes that cost me hours so you don't have to repeat them. What I'm Running Eight services, 26 containers, one machine: Service Purpose Portainer Docker management UI Uptime Kuma Service monitoring (7 monitors) NocoDB Self-hosted Airtable — CRM & leads n8n Workflow automation Open WebUI Local AI chat interface Ollama Local LLM inference (GPU) AFF!NE Collaborative docs & whiteboards Plane Project management (roadmaps, sprints) Duplicati Encrypted daily backups Cloudflare Tunnel Zero Trust secure access — no open router ports All external-facing services sit behind Cloudflare Zero Trust with email OTP. No passwords to manage, no VPN clients — Cloudflare handles authentication at the edge. Architecture ┌──────────────────────────────────┐ │ Cloudflare Edge (Zero Trust) │ │ *.yourdomain.com — email OTP │ └──────────────┬───────────────────┘ │ HTTPS ┌──────────────▼───────────────────┐ │ Ubuntu Machine │ │ │ │ cloudflared (outbound tunnel) │ │ │ │ │ ┌─────▼────────────────────┐ │ │ │ homelab-net (bridge) │ │ │ │ │ │ │ │ portainer uptime-kuma │ │ │ │ nocodb n8n │ │ │ │ open-webui affine │ │ │ │ plane-* duplicati │ │ │ │ ollama (GPU passthrough) │ │ │ └───────────────────────────┘ │ └───────────────────────────────────┘ Everything runs on a shared Docker bridge network ( homelab-net ). The cloudflared container maintains an outbound-only encrypted tunnel — no inbound ports open on the router at all. Ollama runs in Docker with NVIDIA GPU passthrough. The AI model inference happens on the GPU, leaving CPU headroom for all other services. Prerequisites Ubuntu 24.04 LTS Docker Engine + Compose v2 NVIDIA GPU with driver 535+ NVIDIA Container Too

If you've ever built a REST API with Express, you've probably felt it. Middleware registration, type definitions, body parser setup, connecting Joi or Zod... the structure is simple, but the boilerplate is excessive. When I first saw Hono, I was skeptical. "Another Express clone," I thought. That changed when I actually ran it. Bottom line: Hono v4 is more than just lightweight and fast. TypeScript type inference flows naturally all the way to route handlers. Zod validation connects via a single official package. On Bun, response times are noticeably faster than Express. Everything in this post is based on what I ran in a sandbox in June 2026. Why Hono — Compared to Express and Fastify Understanding where Hono fits means answering three questions. Bundle size : Hono v4 core is about 12KB. Express is 58KB, Fastify is 77KB. The gap might not sound dramatic, but in edge environments like Cloudflare Workers or Deno Deploy, bundle size directly affects cold start time. Edge functions sometimes initialize a new runtime per request — smaller means faster first response. Runtime compatibility : Express is Node.js-only. Fastify targets Node.js by default. Hono was designed from the start to "run anywhere." The same code deploys to Bun, Deno, Cloudflare Workers, Node.js, and AWS Lambda Edge. TypeScript support : Express requires @types/express as a separate install, and properties added to req via middleware don't get type inference. Hono is written in TypeScript from the ground up, and the Hono<{ Bindings: Env; Variables: Variables }> generic gives you type-safe access to environment variables and middleware state. I'm not saying Hono is the right choice for every situation. If your team is deeply invested in Express, or you need a mature plugin ecosystem, there's no compelling reason to switch. But if edge deployment is the goal, or you want type safety from day one, Hono is the most convincing TypeScript API framework right now. Installation and First Server — Response in

I use "what is my IP" sites maybe once a month. Every time I end up on something covered in ads, calling three different tracking APIs, and showing me results I don't fully understand. So I spent a weekend building whatsmy.fyi. The thing I didn't expect: you don't need an IP geolocation API at all if you're on Cloudflare Workers. Every request comes with a cf object that already has your city, country, ISP, TLS version, HTTP protocol, and RTT. Free. Zero latency. The part I enjoyed most was the WebRTC leak test. It checks whether your browser is exposing your real IP through RTCPeerConnection even when you're on a VPN. I ran it on my own setup. It was leaking. Zero logs. Zero storage. Just your data, shown to you. https://whatsmy.fyi

Cloudflare recently added support for Claude Managed Agents, allowing developers to run and manage Claude agents within Cloudflare. Developers can connect agents to private systems, choose their runtime environment, and monitor agent activity using Cloudflare services. By Renato Losio