标签：#AI

AI crawlers now appear in many server logs alongside traditional search bots. Some are used for search retrieval, some for training, and some for broader web indexing. If you care about AI search visibility, you need to know which ones can access your public pages. The most common accidental blocker is simple: a robots.txt rule or CDN bot setting that prevents AI crawlers from reaching the content you want discovered. The major AI crawler tokens to check Here are crawler tokens you may see in logs or robots.txt rules: Crawler token Company Notes GPTBot OpenAI Documented OpenAI crawler token OAI-SearchBot OpenAI Documented OpenAI search-related crawler token ChatGPT-User OpenAI Documented OpenAI user-triggered agent token ClaudeBot Anthropic Documented Anthropic crawler token Claude-SearchBot Anthropic Documented Anthropic search-related crawler token Google-Extended Google Google control token for Gemini Apps and Vertex AI use CCBot Common Crawl Web corpus crawler used by many downstream systems PerplexityBot Perplexity Commonly referenced Perplexity crawler token Crawler names and purposes change. Always confirm against official platform documentation before making sitewide access decisions. First, check what is actually happening Before you change anything, find out who is already crawling. If you have server logs: grep -E "GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|Google-Extended|CCBot|PerplexityBot" access.log If you use Cloudflare, check bot and security events and filter by user agent. Three quick diagnostic steps: Open https://yourdomain.com/robots.txt and look for broad Disallow: / rules. Confirm the sitemap is listed in robots.txt or discoverable at /sitemap.xml . Use our AEO Checker to validate robots.txt and flag restrictive AI crawler rules. The most common mistake The blunt rule that makes sites invisible to many crawlers: User-agent: * Disallow: / This blocks every well-behaved crawler that follows the wildcard rule. If you see it on

OpenAI is investigating issues with Codex usage limits Tibo wrote that the Codex team spent Sunday in a war room, digging through logs and looking for anything that could have caused faster usage drain for some users. As the investigation continues, OpenAI has issued a full reset of Codex usage limits for everyone. The funny part: this week at OpenAI is called RESET week. In US corporate culture, that usually means a lighter week to slow down, clear the calendar a bit, and recharge.

Hello Devs 👋 When teams talk about engineering metrics, the conversation usually moves toward speed....

Nine major suspected cargo thefts happened at Tesla’s Nevada battery factory in January alone, according to sheriff’s records obtained by WIRED.

My non-tech friends still don’t get it. Despite what you’d believe from Twitter, most people still...

While the World’s Fair officially kicks off today a bunch of keen developers were in early, taking...

The AI Engineer World's Fair has attracted a lot of adults, and they brought their kids too, for...

Leading into the AI Engineer event in San Francisco, I’m looking forward to having my mind blown....

Deep into a coding session, you realize you want beta testers to try some new functionality first....

The AI Engineer World’s Fair returns to Moscone West in San Francisco from June 28 through July 2,...

It has been less than three years since Shawn "swyx" Wang coined the term AI engineer in an open...

How I ended up running a local LLM to generate my git commit messages

A new OpenAI report maps how AI could reshape jobs across the EU, highlighting which occupations may face automation, growth, or workflow changes.

Protocol adapters are one of the easiest places for agent-commerce architecture to drift. An adapter begins with the narrow responsibility of translating an external protocol request into something the commerce platform understands. For example, an MCP-style tool may ask for return terms, an ACP-style interaction may ask whether checkout can be prepared, an AP2-related flow may carry payment authority information, and an internal feed may publish product capabilities. Those are adapter concerns at the boundary. The problem starts when the adapter does more than translate. It checks product availability from catalog fields. It interprets policy text. It decides whether checkout is ready. It treats a payment artifact as authority. It turns a domain blocker into a softer protocol response. Each shortcut may solve an integration problem locally, but it also creates a second place where commercial meaning is decided. When several adapters exist, those local decisions begin to diverge. The MCP tool may block return-policy quotation, the ACP adapter may expose the product as purchasable, the feed may publish it as checkout-ready, and the AP2-related flow may reject delegated payment. At that point, the platform does not only have multiple integrations. It has multiple interpretations of the same commercial state. This is the adapter problem in agent-ready commerce: semantic drift at the protocol boundary. The adapter should know how to speak the protocol. It should not decide product truth, policy meaning, eligibility, checkout validity, or payment authority. Those decisions belong inside the commerce platform, where they can be shared, tested, evidenced, and audited. This is the fifth article in the Agent-Ready Commerce series. Part 1 introduced the broader architecture model: Facts → Eligibility → Authority → State transition → Evidence → Audit Part 2 focused on commercial truth. It argued that catalog data is not enough. A platform needs source-backed, freshness-aware p

Engineering leaders have spent the past year trying to get their teams to adopt AI coding tools as quickly as possible. Now, a new set of questions has taken over: how do you measure whether any of it is worth the money, and how do you stop agents from running unchecked on production systems? Developer tooling company Warp , an open agentic development environment built from the terminal up, thinks the answer isn't picking a single agent and standardising on it — it's giving teams a way to run several at once, compare them, and govern all of them from a single control plane. As Tessl wrote back in February, orchestration has emerged as a discipline in its own right — a dedicated layer of tooling for coordinating, supervising and directing multiple agents running in parallel. Back in February, Warp launched Oz as a cloud platform for running and managing coding agents at scale. Now, Warp is taking things a step further. In May, the company expanded Oz into what it's calling the first multi-harness control plane — meaning teams can now run Claude Code, Codex and Warp Agent simultaneously through a single interface, rather than committing to any one of them. Tessl caught up with Warp CEO Zach Lloyd to discuss how engineering leaders are thinking about agent fleets, what the harness layer actually changes, and where the lines between autonomy and human oversight are really being drawn. "The wild west": how the agent gold rush became a budget problem Zach spent several years at Google, leading engineering on Docs and Sheets before co-founding photo-editing startup SelfMade . He later served as interim CTO at Time, before founding Warp in 2020, raising north of $70 million in funding from the likes of Sequoia, Google Ventures, Figma co-founder Dylan Field, and Salesforce’s co-founder Marc Benioff. That background — building collaborative tools at Google scale, then navigating the startup world — gives Zach a particular vantage point on how quickly the engineering tooling

Voice interfaces are rapidly becoming the next major interaction layer after mobile and web UI. Instead of clicking, users will increasingly talk to systems that understand intent, context, and can execute actions in real time. In this article, we’ll build a production-grade architecture for a real-time AI voice system using modern web technologies such as Next.js, WebRTC, and OpenAI’s streaming capabilities. We’ll also explore how this architecture powers modern conversational systems like an AI Voice Agent platform, where AI can handle real-time interactions for business use cases like bookings, support, and sales automation. 1. Why Voice AI is the Next Interface Shift Text-based chatbots solved the first wave of automation. But voice introduces: Faster interaction (no typing) Higher emotional expressiveness Better accessibility Natural multitasking Businesses are now adopting systems like Voice AI for Business to replace traditional call centers and static IVR menus. The key challenge is not just speech-to-text, but building a low-latency conversational loop that feels human. 2. System Architecture Overview A production-ready AI voice system typically consists of: Frontend (Next.js) Audio capture via Web Audio API Streaming audio chunks UI for conversation state Backend (Node.js / Edge Functions) Session management Authentication Tool execution layer AI Layer OpenAI Realtime API (streaming) Function calling Context memory Audio Pipeline Speech-to-text streaming Text-to-speech streaming Optional noise cancellation 3. Core Concept: Real-Time Streaming Loop The core of a voice agent is a continuous loop: User speaks Audio is streamed to server Model transcribes in real time Model generates response token-by-token Response is converted to audio instantly Audio is played back with minimal delay The goal is to keep latency under ~800ms for a natural experience. 4. Building the Frontend (Next.js + Web Audio API) We start by capturing microphone input: const stream = awa

Artificial Intelligence has brought the term "AI Agent" into almost every technology conversation. As a result, many people now use the words agent and automation interchangeably. While both are designed to reduce manual work and improve efficiency, they solve problems in fundamentally different ways. Understanding this distinction is essential if you're building software, automating business processes, or deciding where AI fits into your organization. What Is Automation? Automation is designed to execute predefined instructions. You tell the system exactly what to do, in what order, and under what conditions. Every time those conditions are met, it performs the same sequence of actions. For example: A customer submits a form. An email is automatically sent. A record is created in the database. A notification is sent to the sales team. Every step is predetermined. If the process changes, the workflow must be updated. Automation excels at repetitive, predictable tasks where consistency is more important than decision-making. What Is an AI Agent? An AI agent is not focused on following instructions. It is focused on achieving a goal. Instead of executing a rigid sequence of steps, an agent observes its environment, evaluates available information, makes decisions, and adjusts its actions as circumstances change. If one approach fails, it can try another. If new information becomes available, it can revise its strategy without requiring a developer to define every possible scenario in advance. In simple terms: Automation asks: "What steps should I execute?" An agent asks: "What is the best way to accomplish this objective?" This ability to reason and adapt is what makes agents fundamentally different from traditional automation. A Simple Example Imagine you're booking a business trip. An automated workflow might: Book the airline you specified. Reserve the hotel you selected. Email you the itinerary. It completes exactly what it was programmed to do. An AI agent, howev

There is a sentence my coding agent used to say that I now read as a warning light. You are completely right. For months I took it as a compliment. The machine agreed with me, so I figured I was onto something. I would describe a plan, watch the agent call it a strong plan, and go build it. If you work with an AI agent every day, you have heard your own version of this. Smart call. Solid approach. That makes a lot of sense. Each one is the machine nodding along while you talk. It feels good. That is the problem. What took me too long to admit An agent that agrees with everything I say stops being a thinking partner. It turns into something that flatters me into shipping my first idea. My first idea is rarely my best idea. Nobody's is. The whole point of a second mind in the room is that it pushes back when the first mind is about to walk into a wall. A yes-machine removes the one thing that made a second mind worth having. This has a name Sycophancy. These models are trained to be agreeable, because agreeable scores well in the feedback that shapes them. OpenAI said so out loud in 2025 when they pulled back a version of their model for being, in their words, overly flattering. They were pointing straight at the default behaviour. So your agent is doing exactly what it was tuned to do when it tells you that you are right. No malfunction involved. One opinion most builders have not made peace with Your agent's confident wrong answer costs more than a useless one. A useless answer wastes a minute. You see it is useless and move on. A confident wrong answer wastes a week, because you trusted it, built on it, and found out only when it broke in front of someone who mattered. Occasional wrongness is survivable. Everything is wrong sometimes. What actually bites is being wrong while sounding certain, and agreeable, and exactly like what you wanted to hear. How to tell if your agent is a yes-machine You can test it in a minute. Tell it a bad idea on purpose. Propose somethi

How to Create an AI Agent: A Production Walkthrough The first agent I shipped to production failed at 3am on a Sunday. It looped on a tool call, burned through $40 in tokens before my budget alarm fired, and left a half-written draft in the database with no way to resume. That night taught me more about agent design than any framework tutorial. Since then I have built a pattern I trust enough to leave running unattended for weeks at BizFlowAI, where agents research, write, optimize and publish content without me touching them. This is that pattern, stripped down to what actually matters. Start with the job spec, not the framework Before you pick LangGraph, CrewAI, or roll your own, write the agent's job spec like you would for a junior engineer. One paragraph. What it owns, what it must never do, what "done" looks like, and which signals tell you it failed. Here is the spec for one of my production agents: The Topic Researcher owns generating a ranked list of 20 content topics per site per week. It reads from keyword_pool and search_console_perf , writes to topic_queue . It must never publish, never call paid APIs more than 8 times per run, and must finish in under 6 minutes. Done = 20 topics with score >= 0.6 and zero duplicates against the last 90 days. Failure signal = empty queue after a run, or any topic flagged by the dedupe check. If you cannot write this paragraph, do not build the agent. You will end up with a "do everything" prompt that hallucinates its way through ambiguous tasks. The job spec becomes your evaluation rubric later, so write it carefully. Rule of thumb I use : if the spec needs more than 5 tools or more than 3 decision branches, it is two agents, not one. Design the tools before you write the prompt Most agent failures I have debugged were not prompt failures. They were tool failures. The model called a tool with wrong arguments, the tool returned a 4MB JSON blob, or two tools had overlapping responsibilities and the model picked the wrong

The AI Implementation Process I Use With Every Client Most AI projects do not fail at the model. They fail in the six weeks before anyone writes a prompt, and in the six weeks after the demo lands in a Slack channel and nobody knows who owns it. I have run enough of these now (from one-off automations to multi-agent content systems running unattended) that the process has converged into something stable. This is the version I actually use. It has five phases: scoping, POC, integration, evaluation, operations. Each phase has an exit criterion. If we cannot meet the exit criterion, we do not move forward. That single rule has saved more projects than any clever architecture choice. Phase 1: Scoping (1 to 2 weeks, fixed price) Scoping ends with a written document that names the workflow being automated, the system of record it touches, the success metric in hours or dollars, the data we have access to, and the smallest possible first slice. No model is chosen yet. No code is written. If we cannot produce that document, the engagement stops here and the client keeps the document. The hardest part of scoping is resisting the urge to solve the interesting problem. Clients almost always describe the AI-shaped fantasy ("an agent that handles all support tickets") when the real opportunity is narrower and uglier ("triage tier-1 tickets that mention billing, route to the right queue, draft a reply for human approval"). The narrower version ships. The fantasy does not. I run scoping as three sessions: Workflow walkthrough. Someone who actually does the work shows me their screen for an hour. I record it. I take timestamps. The point is to find the moments where a human is doing pattern matching that an LLM can do, and to find the moments where they are doing judgment that an LLM should not do. Data audit. Where does the input live? Where does the output need to go? What is the auth story? If the data is locked inside a SaaS product with no API and no export, that is the projec