标签：#web

I applied to a YC W25 startup the normal way. Filled out the form, wrote a decent cover letter, hit submit. Silence. While waiting, I found their open-source repo on GitHub. Read through the codebase out of genuine curiosity I wanted to understand what they were actually building. Found a bug. Fixed it. Opened a PR. It got merged in 2 days. They still hadn't replied to my application. Here's what that taught me about job hunting in 2025: A cover letter tells someone what you claim you can do. A merged PR shows them. One of those gets read. The other gets filed under "maybe later" -which is just "no" with extra steps. I'm not saying cold applications are dead. I'm saying they're the last resort, not the first move. If a company has a public repo, you have a backdoor that most applicants don't even think to try. Read the code deep and find something small but real. Fix it and Open a PR. Now you're not a stranger in their inbox you're someone who already ships for them. The reply came eventually, by the way. But by then, the maintainers already knew my GitHub handle. That matters more than you think. Have you ever landed something through a contribution instead of an application? Drop it in the comments curious how many people have done this.

A webhook endpoint is a public URL sitting on the internet, and anything on the internet can send it a POST . If your app acts on whatever lands there, an attacker who guesses the URL can forge events: fake an inbound email, trigger a workflow, or feed your system garbage. The fix is to confirm two things before you trust a request, that you own the endpoint and that Nylas actually sent the payload, and both are built into how webhooks work. This post covers verifying webhooks from two angles: the HTTP mechanics your endpoint implements, and the nylas CLI for testing a signature without standing up a server. I work on the CLI, so the terminal commands below are the ones I reach for when I'm debugging a signature mismatch. Two layers of webhook trust There are two separate checks, and they happen at different times. The first is a one-time endpoint challenge: when you register or activate a webhook, Nylas sends your URL a request with a challenge value you echo back, proving you control the endpoint. The second runs on every notification afterward: each delivery carries a cryptographic signature you verify against a shared secret, proving the payload is genuine and wasn't tampered with. You need both because they defend against different things. The challenge stops you from accidentally registering an endpoint you don't own and confirms the URL is live. The signature stops anyone else from posting forged events to that URL once it's known. Skip the signature check and your public endpoint will trust any POST that reaches it, which is the most common webhook security mistake. Pass the endpoint challenge The first time you set up a webhook or flip one to active , Nylas sends a GET request to your endpoint with a challenge query parameter. Your endpoint has to return the exact value of that challenge in the body of a 200 OK response, within 10 seconds, or the webhook won't verify. It's a quick handshake that proves the URL is yours and reachable. // Express: echo the ch

Dein bestes Google-Ranking ist wertlos, wenn die Antwort schon vor dem Klick gegeben wurde. Genau das passiert gerade: Nutzer fragen ChatGPT, Claude oder Perplexity – und bekommen eine fertige Antwort mit drei, vier zitierten Quellen. Bist du nicht darunter, existierst du in diesem Moment nicht. Kein Ranking, kein Klick, keine zweite Chance. Die Disziplin, die das adressiert, heißt Generative Engine Optimization (GEO) . Und sie ist – anders als der Marketing-Lärm vermuten lässt – zu großen Teilen ein Engineering-Problem. Crawler-Zugang, Rendering, strukturierte Daten. Lauter Dinge, über die ein Entwickler entscheidet, nicht das Content-Team. SEO optimiert auf den Klick. GEO optimiert auf das Zitat. Der Unterschied ist nicht kosmetisch. Klassisches SEO will, dass du auf Platz eins rankst, damit jemand klickt. GEO will, dass ein Sprachmodell deinen Absatz wörtlich in seine Antwort übernimmt – inklusive Quellenangabe. Der Klick ist nur noch Bonus. Daraus folgt ein anderer Tech-Stack an Signalen: Aspekt Klassisches SEO GEO / KI-Sichtbarkeit Ziel Top-10 in Google Zitat in ChatGPT, Claude, Perplexity Relevante Bots Googlebot, Bingbot GPTBot, ClaudeBot, PerplexityBot Index-Hinweis sitemap.xml llms.txt + sitemap.xml Strukturierte Daten Rich Snippets Entity-Linking ( Organization , sameAs , @graph ) Rendering Google rendert JS (verzögert) viele KI-Bots rendern kein JS → SSR Pflicht Erfolgskontrolle Search Console, Rank-Tracker Citation- & Mention-Tracking in LLMs Die Hebel überschneiden sich – sauberes HTML, schnelle Antwortzeiten, valides Markup helfen beidem. Aber die Bots, die Index-Signale und die Erfolgskontrolle sind eigenständig. Wer GEO als „SEO mit neuem Namen" abtut, übersieht genau die Stellen, an denen es klemmt. Schritt 1: Lass die Bots überhaupt rein Bevor du über Content-Qualität nachdenkst, klär die banale Frage: Kommt der Crawler durch? Erstaunlich oft lautet die Antwort nein – und niemand merkt es, weil ein Browser die Seite ja problemlos lädt. Die drei Use

If you've ever sent a transaction on Solana and wondered why it landed instantly one time and struggled another, you're not alone. Solana is incredibly fast, but how your transaction enters the network matters just as much as what you're sending. In this article, we'll break down Solana transaction processing in plain English — no developer jargon — and explain why landing services like Lunar Lander and Astralane can dramatically improve speed and reliability. The Big Picture: How Solana Handles Transactions At a high level, Solana works like this: You submit a transaction The network decides which transactions get processed first A validator includes your transaction in a block The transaction is finalized on-chain The key detail most users don't see is step #2 — how Solana decides which transactions get priority when the network is busy. That decision is driven by something called Stake-Weighted Quality of Service (QoS) . Stake-Weighted QoS (Explained Like You're Not a Developer) Solana has a built-in traffic management system. Think of it like traffic control for a highway. A Simple Analogy Imagine a highway with two lanes: 🚗 Fast lane (priority access) 🚙 Regular lane (everyone else) Solana prioritizes transaction traffic based on stake, meaning traffic originating from or routed through high-stake validators is more likely to be processed during congestion. Why? Because validators that stake SOL are financially invested in keeping the network healthy. Giving them priority helps protect Solana from spam and overload. What This Means for You Transactions that enter Solana through stake-backed paths have a much higher chance of landing quickly Transactions that enter through generic or overloaded RPCs compete for a smaller slice of capacity During congestion, non-priority transactions are more likely to be delayed or dropped This is the core idea behind Solana's stake-weighted QoS system. Where Transactions Usually Go Wrong Most wallets and apps send transactions t

Most "Shopware vs Shopify" posts compare dashboards, app stores, and pricing tables. None of that matters to you until the day a client asks for something the platform won't let you build. Then the comparison stops being a feature grid and becomes a question about ceilings: how high can I go before the platform says no, and what happens when I hit it? That's the only axis I care about as a developer, so that's the one I'll argue on. Shopify is an outstanding product. It's also a closed SaaS that decides, on your behalf, where customization ends. Shopware is open source built on Symfony, which means the ceiling is "however far PHP and HTTP will take you." Below are the three places that difference actually bites, with code. Angle 1: The checkout is the wall This is the headline because it's where most agency developers first hit something they cannot do. For years the Shopify answer to "customize the checkout" was checkout.liquid . That era is over. Shopify deprecated checkout.liquid in favour of Checkout Extensibility . Plus stores had to migrate their Thank-you and Order-status pages by August 28, 2025 , and in January 2026 Shopify began auto-upgrading stores — wiping customizations built on additional scripts, script-tag apps, or checkout.liquid . Non-Plus stores have until August 26, 2026 , and legacy Shopify Scripts keep working only until June 30, 2026 . ( Shopify migration timeline ) The replacement, Checkout Extensibility, is genuinely more upgrade-safe. It's also a smaller box. You get Checkout UI Extensions (declarative components that render in slots Shopify defines) and Shopify Functions for backend logic — and that's the surface. You don't own the checkout template; you decorate the pieces Shopify exposes. Worth noting: full visual checkout customization (branding API, custom fields beyond the defaults, full UI extension power) is gated to Shopify Plus anyway. On Shopware, the checkout is a Twig template like every other page, and you override it the sam

Remember how I promised you (or rather myself) two weeks ago that from now on I'd only write light,...

I remember the exact moment my first AI side project died. It was 3 AM, I had just spent two full weeks building an elaborate RAG pipeline with vector databases, custom embeddings, and a fine-tuned model—all for a tool that would "revolutionize how developers read documentation." I hadn't written a single line of user-facing code. I hadn't even validated if anyone wanted it. And when I finally deployed it to a hobby server, the cost of hosting the model alone was $200/month. I killed the project before anyone ever visited the URL. That was three months ago. Since then, I've shipped three AI side projects that actually have users. Not millions—but real people who use them daily. Two of them even cover their own hosting costs now. The difference? I stopped trying to build the perfect AI infrastructure and started shipping the stupidest thing that could work. Here's what I learned from those three MVPs, and how you can break out of the "AI side project graveyard" too. The Trap: Thinking You Need to Build Everything The biggest lie in the AI side project space is that you need to own the stack. Every tutorial screams "self-host Llama 3," "set up your own vector database," "build a custom agent framework." That's great for learning, but it's death for shipping. For my second project—a tool that automatically generates commit messages from diffs—I spent exactly one evening. I used the OpenAI API directly, with no caching, no streaming, no error handling. Here's the core of it: import openai import subprocess def get_diff (): result = subprocess . run ([ " git " , " diff " , " --cached " ], capture_output = True , text = True ) return result . stdout def generate_commit_message ( diff ): response = openai . chat . completions . create ( model = " gpt-3.5-turbo " , messages = [ { " role " : " system " , " content " : " Write a concise git commit message summarizing the changes. " }, { " role " : " user " , " content " : diff } ] ) return response . choices [ 0 ]. message .

Every developer using Cursor , Claude Code , Windsurf , or GitHub Copilot knows this exact frustration: You are building a cutting-edge Angular 22 application. You ask your AI coding assistant to spin up a dynamic form, a lazy-loaded list, or an asynchronous data card. Instead of leveraging modern fine-grained reactive Signals, optimized native block control flows, or proper SSR hydration hooks, the AI drops an unoptimized pile of legacy tech debt full of NgModules , *ngIf , *ngFor , and raw RxJS BehaviorSubjects . The LLM Training Paradox Why does this happen? Large Language Models are trained on historical code datasets. Statistically, more than 90% of the public Angular repositories and StackOverflow threads on the internet represent older paradigms. Left to their own devices, agents default to the statistical average of their training data. They literally default to the past. The Fix: angular22-agent-skills To solve this, I built a public, open-source repository of custom instruction bundles and system guardrails leveraging the new skills.sh tool standard. By injecting this verified context directly into your development environment, you force your local AI agents to bypass their training averages and write pristine, optimized, modern Angular 22 syntax every single time. 👉 Check out the repo here: https://github.com/PavanAnguluri/angular22-agent-skills 🔍 The Difference: Before vs. After To understand why these guardrails are necessary, look at what an AI agent writes out of the box versus what it writes once you apply the angular22-agent-skills harness. 🚫 What AI Agents Generate by Default (Legacy) // The AI falls back to old decorators and heavy RxJS boilerplate for standard state import { Component , Input , OnInit } from ' @angular/core ' ; import { BehaviorSubject } from ' rxjs ' ; @ Component ({ selector : ' app-user-profile ' , template : ` <div *ngIf="visible"> <h3>{{ firstName }} {{ lastName }}</h3> <div *ngFor="let item of items"> {{ item.name }} </div>

Every identity migration guide eventually reaches the same paragraph, and it's always a little apologetic: "users will need to reset their passwords." It gets treated like a law of nature. It isn't. It's a choice, usually forced by a tool that didn't want to do the harder thing. The harder thing is verifying your users' existing password hashes in place, so they sign in after the move with exactly the credentials they had before and never notice anything happened. Whether you can do it comes down to one question: can you get the old hashes, and can the new system verify them? Password hashes are more portable than people think A password hash isn't a secret algorithm. bcrypt is bcrypt. A bcrypt hash carries its own cost factor and salt inside the string, so anything that implements bcrypt can verify a hash any other bcrypt system produced. The same is true of the PBKDF2 format ASP.NET Identity uses: documented, versioned, self-describing. If you know what you're holding, you can check a password against it without ever knowing the password. So a migration that preserves logins doesn't need the plaintext (nobody has it) and doesn't need to re-hash everyone up front. It needs to obtain the stored hashes and verify against them on sign-in, upgrading each one to its own format quietly the first time a user logs in. That last part is lazy migration: carry the old hash, verify it once, replace it transparently. Over a few weeks of normal logins your user table re-hashes itself and the legacy formats age out, with zero resets and zero support tickets. The dual-path bit The wrinkle is that different sources hand you different formats, and a good importer verifies both: From self-hosted Duende / ASP.NET Identity: the V3 PBKDF2 hashes (and any legacy bcrypt) verify natively and rehash on first sign-in. This is the easy case, because it's the same scheme the destination already uses. Most teams are surprised it's that clean. From Auth0: bcrypt hashes verify verbatim. The catch

After a long and deliberate alpha, spartan/ui is now 1.0 . We shipped the first 30 primitives in August 2023 with a simple bet: building accessible, good-looking UI in Angular is harder than it should be, and the community deserved a better starting point. Almost three years later, that bet has grown into a stable, production-ready library of more than 55 components - built on signals, ready for zoneless, and server-side-rendering compatible out of the box. Here's what 1.0 actually means. Stable, and ready to build on We stayed in alpha for a long time on purpose. It let us refine the APIs in the open, with real applications putting real pressure on the design, instead of freezing a v1 we'd regret six months later. That patience is what 1.0 cashes in. The APIs are now stable and semantically versioned, so you can depend on spartan/ui/brain and upgrade with confidence. The copy-in spartan/ui/helm layer stays exactly as it's always been - yours to own, read, and customize. No black boxes, no fighting the library to change a style. Built for modern Angular Every primitive is built on Angular signals and standalone components. spartan is zoneless-ready and SSR compatible out of the box, so it drops cleanly into how Angular apps are actually written today - no extra setup, no adapters. The split that's defined spartan from day one still holds. spartan/ui/brain carries the hard, unglamorous parts - ARIA, keyboard navigation, focus management - and keeps them maintained so you don't have to. spartan/ui/helm gives you full styling control on top, copied into your project like a recipe. Accessibility you can rely on; appearance you fully own. From 30 primitives to 55+ The alpha shipped with 30 components. 1.0 ships with more than 55 - nearly double - including many of the most-requested additions over the past two years: Data Table - sorting, filtering, and selection, the piece people asked for most Sidebar - composable app navigation Calendar and Date Picker Carousel , Auto

I wanted AI code review I could actually own. Not access through a subscription or a black-box service with its own limits. The deployment, credentials, providers, and usage under my control. I kept hitting usage limits mid-week during deep building sessions. The models were capable. The workflow was useful. But access still depended on somebody else's weekly allowance, and centralized platforms can change whenever the company behind them decides to. Pricing, quotas, models, plan boundaries. A workflow that fits this month may sit behind another subscription next month. I could not find a reliable open-source option that gave me the ownership model I wanted. So I built one. That became Codra : A self-hosted AI review engine built around bring-your-own models, your own data boundary, and no Codra-imposed usage ceiling. What Codra Is Codra is an open-source, self-hosted AI code review engine for GitHub pull requests. It listens to pull request events, reviews changed files, posts inline findings, and provides a dashboard for jobs, repositories, model routing, history, usage, and failures. It runs on Cloudflare Workers and uses: Cloudflare Queues for review jobs PostgreSQL through Hyperdrive for storage KV for sessions and cache A React dashboard for operations The GitHub App, model credentials, database, and review history are yours. Provider keys are encrypted with AES-GCM using your deployment secret. Bring Your Own Model, Bring Your Own Limits Changing providers does not require replacing your review history, configuration, or workflow. You configure the provider and model. Supported: OpenAI-compatible APIs OpenRouter Anthropic Google / Gemini Cloudflare Workers AI Why Self-Hosted Matters Here A large frontend repo and a tiny backend repo should not need the same review strategy. Each repository gets its own review settings. You tune triggers, skip generated files, ignore drafts, use mention-triggered reviews, configure labels, set file limits, and define custom ru

TL;DR: We launched Tarotas, a tarot reading app, in five languages (Czech, Slovak, Polish, English, German) on a single domain. Each market behaved completely differently. Here is what the data showed us about multi-locale growth. When we started building Tarotas at Inithouse, the plan seemed straightforward: one product, five languages, one domain. Czech as the base, then Slovak, Polish, English, and German. Same cards, same readings, same UI. Just translated. What we did not expect: each locale acts like a separate product. The setup Tarotas is a tarot card app where you draw a card and read a calm, generic interpretation. No fortune telling, no sign-ups, no paywall. 78 cards across five languages, all on tarotas.com with language detection. We built it in Lovable and deployed it in under two weeks. The multi-language part took another week: content generation for 78 cards times 5 languages, plus locale-specific meta tags and URL structures. What the data told us The Czech and Slovak markets responded first. That was expected: our studio is based in Prague, our existing portfolio (products like zivafotka.cz and magicalsong.com ) already had traction in CZ/SK. But the interesting part was the divergence. CZ/SK users stayed longer. Session duration in Czech and Slovak was noticeably higher than in other locales. Users explored multiple cards, came back for second readings. The "reflection" positioning landed well in these markets, likely because tarot has a quiet cultural niche in Central Europe: not mainstream, but not fringe either. Polish users bounced faster but shared more. The PL locale had higher bounce rates but showed a different signal: social referrals. Polish users who did engage were more likely to share readings. The tarot community in Poland leans more social: Facebook groups, Instagram stories, TikTok readings. Our product caught some of that energy. German users barely showed up. DE was our weakest locale by far. German-language search demand for ta

When I started learning web development, I discovered that creating a webpage is more than making it look good. It is also important to make websites accessible and easy for everyone to use. Two concepts that helped me improve my website were semantic HTML and web accessibility. Semantic HTML means using HTML elements according to their purpose instead of using generic elements for everything. Semantic elements such as , , , , , and make the structure of a webpage clear. They improve readability, help search engines understand the content, and make websites easier for people using screen readers. Before (Non-Semantic HTML) My Website Welcome to my website. After (Semantic HTML) <h1>My Website</h1> Welcome to my website. The semantic version is much easier to understand because each element clearly describes its purpose. During my accessibility audit, I found several improvements that made my website more user-friendly. The first issue was that images needed descriptive alternative text. I added meaningful alt attributes so screen readers can describe the images to users who cannot see them. The second improvement was the heading hierarchy. I used one for the page title and organized the remaining sections with headings. This creates a logical structure that is easier to navigate. The third improvement involved descriptive links. Instead of using vague text, I changed links to clearly describe where they lead. For example, I used "Visit GitHub" instead of a generic phrase. I also ensured that the HTML document included the lang="en" attribute and that all form fields had properly associated elements. These small changes improve accessibility and usability for everyone. Working with semantic HTML and accessibility has shown me that building websites is not only about appearance but also about creating experiences that everyone can use. As I continue learning web development, I will continue applying these best practices in all my projects. My Portfolio Portfolio Websi

If you write software for a living and you're considering a trip to China in 2026, the friction you'll hit is not what you expect. The Great Firewall is the headline, but it's rarely what trips up a first-time visitor. What actually breaks your week is the small stuff: a QR code at a noodle shop, a metro turnstile that won't take your foreign card, a hotel Wi-Fi that quietly drops every request to Google. This is a brief survival guide written from a developer's mindset: what's actually changed in 2026, what you can fix before you leave, and what you should just accept. 1. Visa-free entry now covers most Western devs As of late 2025, China extended its 30-day visa-free transit policy to passport holders from 38 countries, including the US, UK, Germany, France, Australia, the Netherlands, and most of the EU. If you're flying in for a vacation, a conference, or even a short remote-work stretch, you may not need to apply for a visa at all — you just need an onward ticket within 30 days. The catch: the rules per nationality drift quarterly, and the official guidance is scattered across embassy pages. I keep a more current breakdown here: FirstTripChina visa-free guide — worth checking the week you book your ticket. 2. The payment problem is the real "API" you need to integrate China runs on two payment rails: Alipay and WeChat Pay. Cash is technically legal but vendors below the level of a 4-star hotel will look at you like you handed them a stone tablet. Foreign credit cards work at airports and big chains; they do not work at the dumpling place you actually want to eat at. The fix that exists in 2026 — and that did not exist three years ago — is "Tour Card" inside Alipay and "International" mode inside WeChat Pay. Both let you link a Visa/Mastercard issued outside China and pay via the same QR system locals use. Setup steps (roughly): Install Alipay (App Store / Play Store, US/EU regions both work). Verify with passport + selfie (KYC takes about 3 minutes). Tap Tour C

Lucide has released version 1.0 of its open-source icon toolkit, marking its first stable major release. The update features over 1,600 icons and removes trademarked brand icons due to legal and design concerns. Significant performance improvements have also been made, reducing package size and adding context providers for various frameworks. Users upgrading should be aware of breaking changes. By Daniel Curtis

I replaced aaPanel/OpenLiteSpeed with Caddy and shell scripts and turned the process into a benchmark. Two phases (architecture then code), one external code review. The winning model? Not the one you'd expect.

Web MCP, DevTools per agenti e Modern Web Guidance: meno hype, più strumenti e metodo. Negli annunci recenti di Chrome è emersa una cosa interessante: al netto delle novità “appariscenti”, ciò che resta più utile per il lavoro quotidiano è quello che migliora workflow, diagnosi e decisioni tecniche . Tre filoni, in particolare, disegnano una direzione chiara: Web MCP , DevTools per agenti e Modern Web Guidance . Di seguito una sintesi ragionata di cosa significano, perché contano per il frontend, e come prepararsi a sfruttarli. 1) Web MCP: il ponte tra agenti e Web (senza incollaggi fragili) Se stai lavorando con assistenti/agentic workflow, oggi il collo di bottiglia è quasi sempre lo stesso: far sì che un agente capisca e usi le capacità del browser e delle app web in modo affidabile. Web MCP punta a risolvere questo punto creando un linguaggio/protocollo comune per esporre “capacità” (capabilities) e strumenti (tools) che un agente può invocare in modo strutturato, invece di basarsi su prompt lunghi, scraping o integrazioni ad hoc. Perché è importante per chi fa frontend Automazioni più robuste : meno script fragili che si rompono al primo refactor del DOM. Integrazioni più standard : se più strumenti parlano lo stesso “dialetto”, il costo di collegare agenti e applicazioni scende. Esperienze utente nuove : assistenti che completano task complessi dentro l’app (es. compilazioni, ricerca guidata, operazioni amministrative) con maggiore affidabilità. Implicazione pratica Inizia a ragionare sull’app come su un insieme di azioni esplicite (es. “crea ordine”, “esporta report”, “filtra dataset”), non solo come UI. Questa mentalità ti rende pronto a esporre capacità in modo sicuro e controllato, quando lo stack lo renderà semplice. 2) DevTools per agenti: debugging e performance nell’era dell’automazione Se Web MCP è il “ponte”, DevTools per agenti è la cassetta degli attrezzi per controllare quel ponte: osservabilità, diagnosi e iterazione rapida su flussi in cui non è

The Idea Like a lot of people, I sometimes struggle with time. Not in an "I'm just bad at planning" way — more like my brain genuinely has a hard time feeling how long things take. Twenty minutes can feel like five. I'll think "I have time" right up until I definitely don't. So I built Ready. Ready is a PWA (a web app you can install on your phone like a native app) that counts down to your next event — but not just to the event itself. It counts down to when you need to leave, factoring in both how long it takes you to get ready and how long the journey takes. It sends you push notifications before it's time to move. So you don't accidentally forget about time, run out the door ..late again! The app was designed with time blindness in mind — a challenge many people experience. The tone is always encouraging, never stressful. No red warnings. No "you're late." Just a gentle nudge that has your back. It's also my portfolio project. I'm a junior developer learning in public, and this is me documenting the whole messy, rewarding process. (Which also happens to be great for recalling what you learned) The Stack — and Why I Chose It Before writing a single line of code, I had to decide what to build with. Here's what I landed on and why: Next.js — a framework built on top of React (a popular way to build web interfaces). I chose it because it handles both the frontend (what you see and click) and the backend (the logic running behind the scenes) in one project. Less setup, more building. Supabase — think of it as a database with superpowers. It handles storing your data and user authentication (logging in and out) out of the box. It has a generous free tier, which is great when you're learning. Tailwind CSS — instead of writing traditional CSS in separate files, Tailwind lets you style things directly in your code using short class names like rounded-full or text-teal-600 . Web Push API + Service Workers — A service worker is a small script that runs in the background of

Your site ranks on Google. Your Core Web Vitals are clean. Your meta tags are in order. And yet, when someone asks ChatGPT, Perplexity, or Google's AI Overview a question your business should answer your content doesn't show up. Not because your SEO is broken. Because AI search engines don't work like Google. Google Reads Pages. AI Search Reads Passages. Google crawls your page, indexes it, and ranks it based on signals like backlinks, domain authority, and keyword relevance. The unit of ranking is the page. AI search engines ChatGPT, Perplexity, Claude, Gemini don't rank pages. They retrieve passages. They pull specific chunks of content that directly answer a query, synthesize a response, and surface it to the user often without the user ever clicking through to your site. If your content isn't structured to be retrieved at the passage level, it gets skipped entirely. The page might exist. The answer might be buried somewhere in a 1,500-word article. But if the AI can't extract it cleanly and confidently, it moves on to content that makes its job easier. That's the invisibility problem. And most websites have no idea it's happening to them. The Crawler Problem Nobody Is Talking About Before we even get to content structure, there's a more fundamental issue. AI search engines have their own crawl agents. OpenAI sends GPTBot. Anthropic sends ClaudeBot. Perplexity sends PerplexityBot. These bots need access to your site before any retrieval can happen and a significant number of websites are blocking them without realizing it. This happens in a few ways: Blanket disallow rules in robots.txt. Many sites, especially those built on managed platforms, use wildcard disallow rules that were written for a different era when the only crawler worth worrying about was Googlebot. Those same rules now block AI crawlers by default. Overly aggressive bot protection. Security tools and CDN configurations that flag unusual crawl patterns will sometimes block AI bots before they even

Last year I was asked to investigate a startup's AWS bill. It had jumped from roughly $200/month to over $3,000 in a few weeks. Nobody knew why. After digging through logs, metrics, and database traffic, I found the culprit: a polling loop with no backoff strategy. The code looked harmless: async function processQueue () { const jobs = await getJobs () for ( const job of jobs ) { await processFile ( job ) } processQueue () } processQueue () At first glance, this seems reasonable. Process all available jobs, then check again. The problem appears when the queue is empty. When getJobs() returned no work, the loop immediately queried the database again. And again. And again. There was no delay, no backoff, and no event-driven trigger. As a result, the service continuously hammered the database looking for work that didn't exist. Each iteration generated: A database query Network traffic CPU usage Logging overhead Additional infrastructure load Individually, each operation was cheap. Executed hundreds of thousands of times per day, they became expensive. The fix was simple: async function processQueue () { while ( true ) { const jobs = await getJobs () for ( const job of jobs ) { await processFile ( job ) } await new Promise ( resolve => setTimeout ( resolve , 5000 )) } } Even better would have been replacing polling entirely with an event-driven design using a message queue. What this incident taught me: 1. Empty queues are production workloads. Many engineers optimize for peak traffic and forget about idle traffic. Systems often spend more time idle than busy. 2. Polling needs backoff. If you're polling, always define what happens when no work is found. 3. Cost bugs rarely look like bugs. Nothing crashed. No exceptions were thrown. The system was technically working exactly as written. It was just doing useless work 24/7. 4. Always monitor cost alongside performance. CPU, latency, and error rates looked normal. The AWS bill was the first real alert. One question I ask