标签：#seo

If you've ever built a site for a client, a friend, or your own side project, you've had this conversation: "So... how much would a website cost?" And you've answered with "it depends" — which is true, but useless without context. So here's the breakdown I wish I could just link people to instead of explaining from scratch every time. First: "Website" Is Not One Thing If you've ever built a site for a client, a friend, or your own side project, you've had this conversation: "So... how much would a website cost?" And you've answered with "it depends" — which is true, but useless without context. So here's the breakdown I wish I could just link people to instead of explaining from scratch every time. A landing page and a custom marketplace platform are both "websites" the same way a bicycle and a truck are both "vehicles." Different build process, different skillset, different price tag. Once you separate by type, the numbers actually make sense: Type Typical Range Landing Page / One-Pager $500 – $3,000 Multi-Page Business Site $1,500 – $8,000 E-Commerce Store $2,000 – $20,000+ Custom Web App / Platform $10,000 – $100,000+ The Build-Method Question (This Is the Part Devs Actually Care About) No-code builders (Wix, Squarespace): $15–$50/month. Fast to ship, fine for a hypothesis test. The tradeoff is architectural debt you don't see until you hit it — custom logic, advanced SEO control, and scaling all get harder or impossible without a full platform switch. WordPress / CMS: $50–$500/year for platform + plugins, plus dev time. Flexible, huge plugin ecosystem, no vendor lock-in — but every convenience plugin is also a maintenance and security surface you now own. Custom-coded: starts around $1,000, no real ceiling. This is the only route when requirements exceed what a template or plugin can do — unusual functionality, real performance constraints, or a design that isn't achievable off-the-shelf. The trap: a $20/month builder that gets outgrown in 18 months and rebuilt

Most developers treat SEO as "someone else's job" — a marketing concern that happens after the site ships. But a huge chunk of SEO is actually decided at the code level, long before a marketer ever touches the content. If you're building sites — for clients, for yourself, or as side projects — a few technical fundamentals can make or break how discoverable that work ever becomes. Here's the technical SEO checklist I use when reviewing or building sites, from a digital marketing + web perspective. Core Web Vitals Aren't Optional Anymore Google uses three core metrics as direct ranking signals: LCP (Largest Contentful Paint) — how fast the main content loads INP (Interaction to Next Paint) — how responsive the page feels to input CLS (Cumulative Layout Shift) — how visually stable the page is while loading A site can have perfect content and still underperform in search if these numbers are bad. Common culprits: unoptimized images, render-blocking JS, and layout shifts from late-loading ads or fonts. Quick wins: Lazy-load offscreen images Serve modern image formats (WebP/AVIF) Reserve space for dynamic content (ads, embeds) to avoid layout shift Defer non-critical JavaScript Structured Data Is a Developer Task, Not a Marketing One Schema.org markup (JSON-LD is the recommended format) helps search engines — and increasingly AI-driven search summaries — understand what's actually on the page: is this a product, an article, a recipe, an FAQ? Sites with well-implemented structured data are more likely to get rich results (star ratings, FAQ dropdowns, breadcrumbs) in search — which directly impacts click-through rate even without a ranking change. If you're building a site and skip this step, you're leaving visibility on the table for something that's usually a few hours of implementation work. Rendering Strategy Affects Crawlability Client-side rendered (CSR) React/Vue apps can still get indexed, but it's inconsistent and slower than server-rendered or statically generate

Next.js gives you almost everything you need to rank well out of the box, and most teams still ship sites that Google struggles to read. The framework is not the problem. The problem is that SEO gets treated as a final checkbox instead of an architectural decision, so metadata ends up scattered, content renders on the client, and the structured data never gets written. The App Router changed how all of this works. The generateMetadata function, file-based conventions for sitemap.ts and robots.ts , and Server Components as the default each remove a class of SEO bug that used to be common in the Pages Router. But they only help if you use them deliberately. This is the playbook we follow when we build a Next.js site that has to rank, the same approach behind this site. It is opinionated and concrete: where to put metadata, which files to ship, how to handle structured data and multiple languages, and why Core Web Vitals is an SEO feature rather than a performance afterthought. None of it requires a plugin. Render on the server so Google sees real HTML The single biggest SEO win in Next.js is also the easiest to get wrong: make sure your indexable content is in the HTML on the first byte. Googlebot will execute JavaScript, but it does so on a delay and with no guarantees. Content that depends on a client-side fetch can be missed, indexed late, or indexed empty. Server Components are the default in the App Router, so this is mostly about not opting out. Keep 'use client' at the leaves of your tree, on the button that needs an onClick , not on the page that holds your copy. Fetch your data in the Server Component and pass the rendered result down. If you can view the page source and read your headline and body text without JavaScript, you are in good shape. Master the Metadata API instead of next/head In the App Router you never touch next/head . Every route exports either a static metadata object or a dynamic generateMetadata function, and Next.js merges and d

For fifteen years the goal of search was simple: rank on page one and earn the click. That contract is breaking. More people now ask ChatGPT, Perplexity, Gemini and Google's AI Overviews a question and read the synthesized answer without ever visiting a blue link. If your brand is not in that answer, you are invisible to them, no matter how well you rank. This is the gap Generative Engine Optimization closes. GEO is the practice of structuring your content and your site so that large language models retrieve it, trust it, and cite it when they answer a question in your space. It overlaps with SEO but it is not the same job. One study from the GEO firm Brandlight found the overlap between top Google links and the sources AI tools actually cite has fallen from around 70% to under 20%, and the gap is widening. The payoff is real, not theoretical. AI referrals convert far better than cold organic traffic because the visitor arrives pre-qualified by the answer that sent them. Vercel has reported that roughly 10% of new signups now come from ChatGPT, and LLM-referred visitors have been measured converting at 15.9% from ChatGPT against under 2% for typical organic search. Here is how we approach GEO for the sites we build. SEO earns clicks, GEO earns citations The mental shift is the whole game. Traditional SEO optimizes a page to win a position in a ranked list of links. GEO optimizes a passage to be quoted inside a generated answer. A model does not "rank" your page; it retrieves chunks of it, weighs them against everything else it pulled, and decides whether to repeat your claim and name you as the source. That changes what good content looks like. Models favor passages that are self-contained, factual, and quotable: a clear definition, a specific number, a direct answer in the first sentence. Burying the answer three paragraphs down, the way you might to keep a reader scrolling past ads, is exactly wrong here. Lead with the claim, then support it. Write so a model can

Next.js 16 is the biggest release since the App Router landed, and the upgrade is not a one-line bump. The caching model changed shape, params and searchParams are now promises everywhere, Turbopack runs your builds by default, and middleware.ts is on its way out in favour of proxy.ts . None of that is hard on its own. The trouble is that the changes touch almost every dynamic route in a real app at once, so a rushed upgrade tends to fail in a dozen small places rather than one obvious one. We run this site on Next.js 16, and we have moved client projects across the same gap. The pattern that works is boring and reliable: read the codemod output, fix the async APIs first, decide your caching strategy deliberately instead of letting the old implicit behaviour leak back in, then clean up the renamed files. This guide walks through that order, with the specific gotchas that cost the most time. If you are still on Next.js 13 or 14, the same steps apply, you just have more of them to work through. Run the codemod, then read what it could not fix Start with the official upgrade command. It pulls the right versions of next , react , and react-dom , and runs the codemods that handle the mechanical rewrites for you. npx @next/codemod@latest upgrade latest The codemod is good, but it is not magic. It will happily wrap your params access in await where the shape is obvious, and skip anything indirect, a params object passed into a helper, destructured two functions deep, or read inside a generateMetadata you wrote by hand. Treat the codemod as the first 80%, not the finish line. Once it has run, do a clean install and a type check before you touch anything else. With typescript.ignoreBuildErrors set, as it is on many projects, the build will not catch these for you, so run the type checker yourself. rm -rf node_modules .next && npm install && npx tsc --noEmit The errors that come back are your real to-do list. Most of them will be the async API change, which is the next sectio

Last month I shared how I built 143 free online tools that run 100% in your browser — no signup, no uploads, no watermarks. That post got a great response (and a lot of "how is this free?" comments — answer: it stays free because files never touch a server, so there are no processing costs). Today's update: every single tool now has a full guide series. What's new 144 how-to articles — one per tool — live at toolfyra.vercel.app/blog : Step-by-step guides — every input explained, common pitfalls, pro tips Real competitor comparison tables (we scraped and analyzed who ranks for what, and where their tools annoy users with account walls) FAQ sections with schema markup so answers surface directly in search and AI assistants Unique generated illustrations per article Smart related-tools clusters — finish one task, the next tool is one click away Why guides for calculator tools? Because "how to use a calculator" is what people actually search for. Tools win clicks; guides win trust and rankings . Each article is built from real search-engine data: live SERP results, keyword expansions, and competitor FAQ analysis — zero guesswork. The engineering side (for the dev readers) Every tool is a single HTML page with vanilla JS — calculators run client-side, file tools use Canvas/FileReader APIs The blog is generated (Python build script): schema.org BlogPosting + FAQPage + BreadcrumbList, per-post OG images as optimized SVGs, canonical URLs, sitemap + IndexNow pings on every deploy New site-wide: instant search (type "pdf" → live results dropdown, keyboard-first: / to focus, ↑↓ to navigate), a Tools dropdown with 11 categories, and a mobile hamburger panel — all vanilla JS, no dependencies Privacy by architecture: there is literally no upload endpoint to breach What's next More waves of content (FAQ, mistakes-to-avoid, and comparison articles for every tool) A batch of new tools from our demand-research pipeline (we score thousands of real search phrases before writing a line

The average person does not wake up choosing between "website" and "no website." They choose between opening an app, asking ChatGPT, tapping a map result, or typing a URL. Websites are still relevant when those paths need a place to land: confirm a business is real, compare two options side by side, pay for something, book a slot, or read instructions that outlive a chat thread. They become irrelevant when the destination is slow, broken, or empty, because the next tap is always available. That shift is what agencies miss when the brief says "we need a website" as if presence alone still wins. In 2026 a site is less often where people first find you and more often where they check you are real, pay, or book after they found you somewhere else. Your job is not only to exist on the open web. It is to be the destination that still earns the click when someone is ready to act. What does "still relevant" mean after apps and AI answers? Relevance is not traffic volume. Pew Research analysis of tens of thousands of Google searches in 2025 found users clicked a traditional result on only about 8% of queries that showed an AI Overview, versus about 15% without one. Casual browsing traffic is thinner. The visits that remain often carry sharper intent: someone already heard a name and wants proof, or they are ready to buy and need a form that works on mobile. Google's Search team has argued the same restraint in public: websites are not obsolete, but they are not mandatory for every goal ( Search Off the Record ). Whether you need one depends on audience, control, and what you are trying to deliver. For many businesses the answer is still yes, because apps and social profiles do not replace a site you control when AI systems and search features need structured facts to cite. The relevance question therefore splits in two. Is the open web still where machines and sceptical humans go to verify claims? Yes, for most categories. Is every marketing site still the main place people

Originally published on the WeLocale blog . Most SEO work is a matter of degree. You improve a title, you gain a little. hreflang is not like that. It either forms a valid set that search engines act on, or it does nothing at all, and the failure is completely silent. No warning, no penalty, no message in Search Console telling you the tags you carefully added are being discarded. We build a translation widget, which means we generate hreflang tags for other people's sites. This post is what we have learned about why they get ignored, including the parts where our own approach has real limits. The rule that breaks most setups hreflang is not a property of a page. It is a property of a set of pages, and every page in that set has to agree. If your English page says the German version is at /de/ , the German page has to say the English version is at / . If it does not, the declaration is one-way, and one-way declarations get dropped. Google calls these return links and treats their absence as a reason to distrust the whole set. This is why hreflang fails in a way that feels unfair. Every individual page looks correct when you inspect it. The problem only exists in the relationship between pages, which is exactly the thing you cannot see by viewing source on one URL. The corollary catches people too: each page must list itself . A German page whose tags mention English and French but not German is an incomplete set. Incomplete sets get dropped. The other four failure modes en-UK. The language code comes from ISO 639-1 and the region code from ISO 3166-1. In ISO 3166-1 the United Kingdom is GB. There is no UK. The tag is silently invalid, and it is easily the most common hreflang error on the web. Same class of mistake: lowercase regions, uppercase languages, and a region with no language at all. URLs that redirect. hreflang has to point at the final URL. If it points at http and you redirect to https, or it omits a trailing slash your server adds, the target is a redir

I run TextTimeTools , a free site with speaking-time and reading-time calculators. It's a pure static site deployed to Cloudflare Pages — no backend, no database, no CMS. The calculators themselves are one page. But the site has 50+ pages , each targeting a different keyword like "how many words is a 5 minute speech" or "how long to read 1000 words". Every single one of those pages is generated by a build script. I've never written one by hand. Here's the pattern, and why it's the highest-leverage thing I've done for this site's organic traffic. The problem with a single calculator page A speaking-time calculator answers one query well: "how many words is my speech". But people don't search for tools — they search for answers : "how many words is a 5 minute speech" "how many words for a 3 minute speech" "how long to read 1000 words" "how long to read 5000 words" Each of those is a separate keyword with its own search intent and its own competition. One calculator page can't rank for all of them — a page titled "Speaking Time Calculator" has no reason to show up for "how long to read 1000 words". The classic fix is to write a page per keyword. That works, but it doesn't scale — every new keyword means hand-writing another page, and keeping them consistent is a nightmare. The fix: generate pages at build time The build script ( gen-longtail.cjs ) takes a list of keyword targets and emits a complete, keyword-specific HTML page for each one. The word count pages and reading time pages are both generated this way. const PAGES = [ { minutes : 1 , slug : ' how-many-words-is-a-1-minute-speech ' , variant : ' is-a ' }, { minutes : 2 , slug : ' how-many-words-is-a-2-minute-speech ' , variant : ' is-a ' }, { minutes : 5 , slug : ' how-many-words-is-a-5-minute-speech ' , variant : ' is-a ' }, // ... up to 15 minutes { minutes : 2 , slug : ' how-many-words-for-a-2-minute-speech ' , variant : ' for-a ' }, { minutes : 5 , slug : ' how-many-words-for-a-5-minute-speech ' , variant :

Originally published on michael-kaminski.io . The Genome of Games publishes the same 1,180 records four different ways, and one command writes all four: node build.js , 0.39 seconds, zero npm dependencies. Out come 1,245 static HTML pages for crawlers, an interactive canvas graph for humans, a 129,037-byte search index for the site's own search box, and a Model Context Protocol server exposing 8 tools to agents. The decision worth copying is the one that sounds like a downgrade. The MCP server does not query the site and does not read the source data. It statically imports a 1.9 MB index that the build wrote. There is exactly one place where slugs, lineage, and adoption edges get joined, so an agent and a crawler cannot come back with different answers. The dataset is an ontology of video game mechanics — 168 mechanics, 618 games, 394 companies, 4,366 recorded links, 1962 to 2025. What the records are about does not matter here. The shape of the problem shows up anywhere a structured knowledge base has to serve both a search engine and a model. Four surfaces, one build, a twelve-fold expansion Six hand-edited JSON files under data/ are the source of truth: the feature ontology, the graph, the prose, the company registry, the site copy, and the verified outbound links. Together they are 1,312,577 bytes. The build turns that into 16,644,215 bytes of generated read surface. A 12.7× expansion, and every byte of it is disposable. Surface Consumer Bytes Per entity 1,245 static HTML pages Crawlers, humans 14,613,203 11,728 / page mcp-index.json → MCP server Agents 1,901,975 1,612 search-index.json The site's own search box 129,037 109 /graph/ canvas Humans exploring lineage data injected at build — The build also emits sitemap.xml with 1,245 entries, llms.txt , robots.txt , and a 404 page. The same run reports 96,843 internal links across those pages. Nothing in that list is authored. Delete the whole output directory and the next build restores it in under half a second.

Ranking on Google is no longer the same thing as earning a click. Search engines increasingly answer questions directly through AI Overviews, featured snippets, People Also Ask boxes, local results, and other search features. In early 2026, a SparkToro study reported by Search Engine Land estimated that 68.01% of U.S. Google searches ended without a click during the first four months of the year. The comparison needs to be interpreted carefully because different studies use different data panels, but the direction is clear: a search impression does not automatically become a website visit. For small websites, this does not mean that SEO is dead. It means the goal is becoming broader. A useful page should be easy to discover, easy to understand, worthy of being cited, and valuable enough that a searcher wants to continue reading after seeing the short answer. SEO still matters in AI search Google's official guidance says that SEO remains relevant for generative search features because AI Overviews and AI Mode are grounded in Google's core Search ranking and quality systems. Google recommends the same fundamentals that have always helped users and crawlers: valuable original content, clear organization, crawlability, good page experience, and accurate technical implementation. This is important because there is no reliable shortcut called “GEO magic.” Google specifically says that site owners do not need special AI-only markup or an llms.txt file to appear in Google Search. The practical approach is still to build a website that people can use and trust. The question is therefore not only, “How do I rank for this keyword?” A better question is, “If an AI system or search feature reads my page, will it find a clear, specific, well-supported answer that represents my experience?” The four layers of visibility A small website can think about search visibility in four layers: Layer What it means Example signal Discovery Search engines can find and crawl the page Internal

An RV repair business does not lose a service call because a visitor failed to read a clever headline. It loses the call when a person with a broken slide-out, roof leak, or electrical issue cannot answer four basic questions quickly: Do you handle this exact problem? Do you serve where I am? Are you available and credible? What do I do next? That sounds like marketing. It is mostly a systems-design problem. The implementation goal is not “make more city pages.” It is to make the business's real-world facts available, consistent, crawlable, and usable across the website, Google Business Profile, analytics, and the conversion flow. This post turns SEOG’s RV repair checklist into an implementation pattern a developer can apply to any local-service site. The model: one source of truth, many decision surfaces Local customers do not encounter a business in one place. They may see a Google result, a Maps profile, a service page, a review, or a call button before they ever submit a form. Treat the site as one consumer of a small, canonical business data model rather than a collection of independently written pages. business facts ─┬─> server-rendered service pages ├─> JSON-LD ├─> XML sitemap + canonical URLs ├─> GBP sync/review queue (with human approval) ├─> call/form events └─> audit and change history The important part is the left side. If a mobile RV technician's phone number, service coverage, repair categories, and hours live in five unrelated CMS fields, a mismatch is inevitable. Start with an explicit domain object. type BusinessLocation = { id : string ; legalName : string ; publicName : string ; phoneE164 : string ; website : string ; address ?: { streetAddress : string ; addressLocality : string ; addressRegion : string ; postalCode : string ; addressCountry : " US " ; }; geo ?: { latitude : number ; longitude : number }; serviceAreas : Array < { name : string ; state : string ; proof : string [] } > ; hours : Array < { dayOfWeek : string []; opens : string ; c

I run a search engine that publishes in twelve languages from one static site on Cloudflare Pages. Last week I audited its machine-readable layer — the part crawlers and answer engines read rather than humans — and found four problems. None of them threw an error. None appeared in logs. Every page rendered perfectly. That is the whole point of this post: the multilingual layer fails in a register where nothing tells you. 1. The homepage was serving the wrong language to everyone abroad The site's primary market speaks Hebrew, so / is Hebrew and /en/ , /ar/ , /de/ and nine others sit alongside it. A middleware rule redirected visitors from one specific region to their language. Everyone else — including every English speaker on earth — landed on Hebrew. My first instinct was to fix it with a broader geo-redirect: detect English-speaking countries, send them to /en/ . This would have been a bad idea, and it is worth saying why. Googlebot crawls predominantly from US IPs. A geo-redirect on / that keys off country would take the crawler off the Hebrew homepage and onto the English one almost every time it visited. You do not want your primary-market homepage to become the page the crawler can never reach. The correct tool is hreflang , and it is what search engines built for exactly this. Checking the page, the tags were already there and already right: <link rel= "alternate" hreflang= "he" href= "https://example.com/" > <link rel= "alternate" hreflang= "en" href= "https://example.com/en/" > <link rel= "alternate" hreflang= "ar" href= "https://example.com/ar/" > <!-- …ten more… --> <link rel= "alternate" hreflang= "x-default" href= "https://example.com/en/" > Two things make this work, and both are easy to get wrong: The set must be reciprocal. Every page in the group lists every other page including itself . If /en/ does not point back at / , search engines are entitled to ignore the whole cluster. x-default is not "the default language" — it is the fallback for users

Two numbers off my Search Console this morning, same 28 day window, same site. The free landing page roast tool: 42 impressions, average position 38.0. The blog post I wrote to support that tool: 11 impressions, average position 21.1. Six weeks earlier it was the other way round. On July 4 the tool sat at position 7.5 on 18 impressions and the article was at 17.8 on 38. So the tool has more than doubled its reach since then, and its average position has gotten roughly five times worse over the same stretch. Both of those things are true at once, and working out why changed how I plan the next tool. The tool favors.dev/roast takes a URL and gives back a conversion score out of 100. It screenshots your full public page, then grades the copy and the design together across six categories: clarity, value proposition, trust, CTA, visual design and SEO. You get back the specific issues hurting signups with a fix for each, the things the page already does well, and a one line verdict. No signup, no credit card, no email field. Paste a URL, press "Roast it", read the result. It is deliberately small, and the scoping is most of why it shipped. The cut list was: accounts and password resets, saved history, dashboards, billing and usage limits, settings and themes, support for every edge case, and an admin panel for myself. Every one of those is how a weekend build turns into a month. If a free tool needs a billing system, you have started building a second product by accident. What those numbers actually say Here is the honest read, because "my free tool beat 29 of my 32 blog posts" is technically true and a bit misleading. Reading Tool impressions Tool position Article impressions Article position Jul 4 18 7.5 38 17.8 Jul 11 20 7.1 42 16.9 Jul 19 22 10.4 39 19.1 Aug 15 42 38.0 11 21.1 Impressions climbed because the tool started matching a much wider spread of queries. Average position fell for exactly the same reason. It is not ranking better. It is ranking on more things, m

Build-in-public note, no fireworks. Why bother Search Console in mid-August was blunt: 6 of 1,098 pages indexed, the rest stuck at "Discovered – currently not indexed", and the Links report empty. Zero external backlinks. The site has a thousand free session pages that nobody can find because nothing points at them. Directories are the cheapest way to get the first handful of pointers, and they are also the corpora that AI assistants read when someone asks "what tools show SPX gamma exposure". So the goal was never traffic. It was (a) backlinks and (b) third-party mentions. What went in The same card everywhere: SPX dealer positioning rebuilt from the 0DTE tape; zero-gamma flip, call/put walls, hold band; every finished session free; a backtest Lab; an MCP server; no buy/sell signals. Logo from the favicon, three screenshots (terminal, the measured book, the Lab), category Finance / Investing wherever the menu allowed it. Done and live: Capterra, AlternativeTo, the official MCP registry (and glama.ai, which pulls from it), TradersList, Firsto, TinyLaunch, Startup Fame, Indie Hackers. Submitted and waiting on a human: StartupStash. Product Hunt is scheduled, one shot only. What we skipped, and why — this is the useful part Anything that wants a badge on our homepage for the free tier (Huzzler, Startup Fame's final step). The card is filled and sits unpublished. A measurement terminal with "featured on" stickers on it is a different product. AI-tool directories (Futurepedia, There's An AI For That). $300–$500 for a listing in front of an audience that wants image generators. The MCP server technically qualifies; the economics do not. Hashnode. Published a 1,300-word engineering write-up; AutoMod archived it within the hour as "this type of content" on a free subdomain, with an upsell to Pro. The Markdown is saved; it will go out on dev.to instead. Reddit. Not a channel for this product. Decided, not deferred. Paid "we submit you to 140 directories" packages. Every one

I Tested 5 AI Engines On My Own Sites. None Agreed. In July I wrote that my open-source...

Eight user agents, and the one that decides whether ChatGPT cites you is not among them. An r/SEO post from April, 53 points and 40 comments, says Cloudflare quietly cut the author's site off from ChatGPT, from Perplexity and from Google's AI Overviews. I went and read the block. It names ChatGPT's training crawler and not its search crawler, and it never mentions Perplexity at all. The Google agent it does name is one Google says does not affect inclusion or ranking in Search. Take just two of the eight agents. GPTBot governs inclusion in OpenAI's training data, and Google-Extended governs grounding in Gemini Apps. Those are real things to give up. They are not the three things the warning names. What is actually in the file Cloudflare prints the whole block on its managed robots.txt page : # BEGIN Cloudflare Managed content User - Agent : * Content - signal : search = yes , ai - train = no , use = reference Allow : / User - agent : Amazonbot Disallow : / User - agent : Applebot - Extended Disallow : / User - agent : Bytespider Disallow : / User - agent : CCBot Disallow : / User - agent : ClaudeBot Disallow : / User - agent : Google - Extended Disallow : / User - agent : GPTBot Disallow : / User - agent : meta - externalagent Disallow : / # END Cloudflare Managed Content Read that against Cloudflare's own crawler reference table and a pattern falls out. GPTBot is in, OAI-SearchBot is out. ClaudeBot is in, Claude-SearchBot is out. For both of those pairs the table calls the blocked agent an AI Crawler and the one left alone AI Search. Applebot-Extended is in and plain Applebot is not, the same split again. The table has no Applebot-Extended row, so it cannot tell you what Cloudflare calls that one. The block runs along the training and search seam, and that looks deliberate to me. OpenAI's side of it is one line: "Each setting is independent of the others". I worked through the three OpenAI bots and which one governs search visibility in an earlier post , so I will

Originally published on echoeffect.net . If you have been inside Google Search Console recently and clicked into the Pages report (previously called Index Coverage), you may have seen a section titled "Why pages aren't indexed." That list tells you exactly which URLs Google found on your site but chose not to add to its search index, and the reason for each one. This is not abstract SEO theory. Pages that are not indexed cannot rank. If Google is excluding pages from your site, you are losing search visibility you should have, and the reason is usually fixable once you understand what Google is actually telling you. This post covers the four most common "not indexed" statuses small business websites encounter, what each one means in plain terms, and the exact steps to resolve it. A quick note before diving in: Some pages on your site should not be indexed. Thank-you pages, admin pages, internal search result pages, and duplicate filter pages are examples where non-indexing is correct. Before fixing any of these errors, confirm the flagged URL is actually a page you want in Google's index. 1. Page With Redirect What it means: Google followed one of your URLs and landed on a different URL because a redirect was in place. The original URL is not indexed. Only the final destination URL is eligible to be indexed. This status is usually caused by one of three things: Old URLs still listed in your XML sitemap that have since been redirected (common after a site redesign or domain migration) HTTP versions of pages listed in your sitemap when the live site runs on HTTPS Trailing-slash inconsistencies, where your sitemap lists yoursite.com/page but the server redirects to yoursite.com/page/ The redirect itself is not necessarily a problem. A 301 redirect is the correct way to permanently move a page. The issue is that Google's crawler is spending time and crawl budget following chains to find the real URL, and your sitemap or internal links are pointing to the wrong address.

I run a small tech-interview-prep site. It has 8,664 individual pages, one per concept — each with a real question, what it's actually testing, a model answer and the mistake that sinks candidates. Programmatic SEO, the whole playbook. Here's what 28 days of Google Search Console says: Impressions 6,511 Clicks 9 CTR 0.14% Average position 45.9 Pages with at least one impression 1,575 of 8,664 (18%) Nine clicks. In a month. From nearly nine thousand pages. I want to walk through this honestly, because the conclusion I reached is not the one I expected, and it's not the one most posts about programmatic SEO land on. What I assumed was wrong My working theory for weeks was "Google isn't indexing them." That's the standard programmatic-SEO failure story: you publish thousands of pages, Google decides your new domain hasn't earned the crawl budget, and most of them sit in Search Console under Discovered — currently not indexed forever. And early on that was true. A few weeks ago only 6 pages had ever received an impression. It's now 1,575. Google is indexing them, steadily, without me doing anything new. The crawl budget arrived on its own schedule. The clicks did not. The actual failure mode Here's the distribution that explains everything. Across 1,206 distinct queries: Position Share of queries 1–10 12% 11–20 6% 21–50 23% 51+ 59% Median position: 58. That's page six of the search results. Nobody has ever been to page six of the search results. So the pages aren't missing from the index. They're in the index, ranked below anything a human will scroll to. Indexed and invisible are close to the same thing, and the second one is more annoying because the dashboard fills up with numbers that look like progress. 6,511 impressions is real. It's also what position 58 produces: Google shows your result to enough people that you see the impression, and none of them scroll far enough to see it. The queries are the tell These are my top queries by impressions: 11 imp pos 52.2 com

We moved a server-side SEO engine into a Chrome extension. Measuring the page was the easy half. Saying what we could not measure was the hard half. We had been running an on-page analysis engine on our own servers for years. You give it a URL, it fetches the page, it reports. Ordinary. Then we moved that engine into the browser, because a server cannot reach localhost , a staging box, an intranet, or anything behind a login. The browser can. Porting the analysis was mechanical work. What took the real time was a category of problem that barely exists on the server: in a live tab, half the things you want to measure are sometimes unavailable, and the honest answer is not a number. This post is about the decisions that came out of that, with the code that implements them. The One Rule: Null Is Not Zero Every derivation in the engine returns number | null , and the two mean different things. 0 means we measured it and it is zero. A page with no layout shift really does score zero. null means we could not measure it. No interaction happened yet, the browser does not support that entry type, or the document came from another origin and the size fields were zeroed out. A zero printed where a null belongs is a made-up number. It is worse than an empty cell, because the reader has no way to tell it apart from a real measurement. So the two never collapse: the derivation keeps them separate and the UI renders them differently. That sounds obvious written down. It is surprisingly easy to violate, and the next section is the most common way. PerformanceObserver Fails Silently, So Ask It First Here is the trap. Calling observe() with an entry type the browser does not support does not throw . It does not warn. It quietly does nothing, and your handler is simply never called. Which means an unsupported metric produces exactly the same result as a measured zero. The one thing the rule above forbids. The fix is to ask before you observe, and to record the refusal: js const SUPPOR