AI 资讯
Building an AI Question Paper Generator: Conquering Google Cloud Document AI, Firestore Vector Search, and Gemini
As part of the Gen AI Academy APAC , I set out to solve a major pain point for educators: manually sifting through textbooks to create grade-appropriate question papers. I built an automated Question Paper Generator using a Serverless Next.js stack, a Retrieval-Augmented Generation (RAG) architecture, and the complete Google Cloud AI suite. Teachers simply upload a textbook chapter (PDF), specify the grade and subject, and let the AI generate a fully formatted assessment quiz. While the architecture sounds straightforward, orchestrating these enterprise-grade APIs in a serverless environment presented several intense technical hurdles. Here is a deep dive into the architecture, the specific roadblocks I hit, and how I ultimately solved them. 🏗️ The RAG Architecture The application is built on Next.js 15 and deployed to Google Cloud Run . The pipeline flows as follows: Document Extraction : The PDF is uploaded and sent to Google Cloud Document AI (Document OCR Processor) to extract the raw text. Chunking & Embeddings : The text is chunked into logical paragraphs and sent to Vertex AI ( text-embedding-004 ) to generate dense vector embeddings. Vector Database : The embeddings and metadata (Grade, Subject) are stored seamlessly in Firestore using native VectorValue support. Retrieval & Generation : When a teacher requests a quiz, the query is embedded, and a findNearest Vector Search runs on Firestore. The retrieved context is passed to Google Gen AI ( gemini-3.5-flash ) to synthesize the structured question paper. 🐛 The Technical Challenges & How I Solved Them Building an end-to-end pipeline using cutting-edge SDKs often means dealing with strict schema validations and opaque error codes. Here are the major technical gotchas I faced. 1. The Document AI Region Endpoint Mismatch The Challenge: I provisioned a Document OCR processor in the asia-south1 region. However, when my Node.js client attempted to send a processing request using the processor's full resource name,
AI 资讯
Cloudflare Workers Accept Inbound TCP, with gRPC the First Protocol on Top
Cloudflare Workers can now accept inbound TCP connections through a new connect(socket) handler routed via Spectrum, ending an eight-year restriction to HTTP. Containers get full-duplex gRPC in any language, while Workers get unary and server-streaming through automatic gRPC-web translation. Everything is private beta. By Steef-Jan Wiggers
AI 资讯
Not Every Workload Belongs on a Free Server: Red Flags and Exit Criteria
The review passed. The deployment failed. An engineer moved a code-review agent to a free server. The model answered correctly in every test. Then the server hit its quota at 2:47 PM on day three. Fourteen pull request verdicts vanished with the session. No state. No logs. No retry. This is the reviewer's blind spot. Teams test models obsessively. They rarely test the runtime underneath. This guide covers one decision: refusing a free server for an agent. It lists red flags, better alternatives, and exit criteria. It also names a concrete example: MonkeyCode's free model access and free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. What "free" actually includes MonkeyCode is an open-source agent platform. It offers free model access and a free server option. The free model access includes 10 million tokens per cycle, per the project's published claim. The free server runs the agent without a paid VM. Those offers are real. They are also constraints. Free infrastructure is a budget, not a promise. Treat it like a trial environment, not a production contract. Free tiers exist to convert users, not to run production. That is fine. The mistake is treating them as infrastructure. Three failure modes Free infrastructure fails in predictable ways. Know all three before committing. Mode one: quota exhaustion. Token budgets reset on a schedule. Heavy days burn the whole cycle. The failure is silent. The agent stops mid-task. Mode two: state loss. Free servers restart without warning. In-memory sessions disappear. Long-running agents lose context. Recovery is manual. Mode three: contention. Shared resources mean cold starts. Neighbors consume CPU. Rate limits appear at peak hours. Latency becomes a random variable. Red flags: check before committing Run this checklist before any migration. One red flag means pause. Two mean stop. Hard deadlines. The agent gates CI or on-call responses. A quota reset cannot wait. Daily burn exce
AI 资讯
Serverless and Agentic Coding Are a Match Made in Heaven
I am not going to spend this whole article making the usual serverless argument. Yes, managed infrastructure is useful. Yes, automatic scaling is nice. Yes, not having to patch servers is a win. Yes, event-driven architectures can be a great fit for modern web applications. All of that is true, but it is not the thing I want to focus on here. The more interesting point is that serverless changes how useful agentic coding can be. It gives AI coding agents a better environment to work in. Not because the agents suddenly become smarter, but because the system they are working on becomes more explicit, more constrained, and easier to inspect. That matters more than I expected. When you build a web application, eventually it needs to be hosted somewhere. You can put it on a VPS, configure nginx, run your app with systemd or a process manager, add a database, bolt on a queue, and wire up whatever else you need. That is a completely valid way to run software. Plenty of serious production systems work that way. But once you start using coding agents, a problem appears. The agent may understand your application code, but not the environment around it. It may not know how your reverse proxy is configured. It may not know how background workers are started. It may not know which scripts run during deployment, which environment variables exist in production, which assumptions live in a README, or which parts of the setup are just tribal knowledge. So when you ask it to make a meaningful architectural change, it has to guess. Sometimes those guesses are fine. Sometimes they are not. The agent may invent a worker process that does not match how you deploy. It may reach for Redis because that is a common queueing answer, even though the rest of your system does not use Redis. It may assume local file storage is available. It may add a scheduler without understanding where that scheduler will actually run. That is where serverless starts to feel less like a deployment choice and mo
AI 资讯
The Model's JSON Was Almost Valid. I Made It Grade Its Own Homework for 48 Hours.
Every extraction pipeline I have ever pointed at a language model shares the same dirty secret: the JSON comes back almost valid. Almost is where the bugs live, because almost passes your eyes and then fails your schema at midnight. So I built a loop where the model grades its own homework, then let it run for 48 hours on a free server to see what breaks. The experiment The idea was simple: take plain-text payloads that look like webhook bodies, extract five fields against a small schema, and give the model exactly one chance to fix its own mistakes. I wrote the rules down before writing any code, because rules written after a failure are just excuses. Pass one asks the model to return the fields as JSON. A validator checks the result against the schema. If validation fails, pass two sends the original payload, the bad JSON, and the exact validation errors back to the model. Every attempt, raw text included, lands in a JSONL log. I ran that loop for 48 hours on MonkeyCode's free server option, using its free model access for both passes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Here is the loop, trimmed to the parts that mattered. import hashlib import json import time from datetime import datetime , timezone import jsonschema import requests SCHEMA = { " type " : " object " , " required " : [ " event " , " customer_id " , " amount " , " currency " ], " properties " : { " event " : { " type " : " string " , " enum " : [ " charge.succeeded " , " charge.failed " ]}, " customer_id " : { " type " : " string " , " pattern " : " ^cus_ " }, " amount " : { " type " : " integer " , " minimum " : 0 }, " currency " : { " type " : " string " , " minLength " : 3 , " maxLength " : 3 }, }, } SEEN : set [ str ] = set () def now_iso () -> str : return datetime . now ( timezone . utc ). isoformat () def call_model ( prompt : str ) -> str : # Point this at the free model endpoint you are testing. resp = requests . post ( " https://your-endpoint.e
开发者
AWS Introduces Specification Driven Composition for Flexible Data Workflows
AWS describes a specification-driven approach for composing flexible data workflows by separating intent from processing logic. Architecture uses declarative specifications, reusable processing capabilities, and validation before execution. AWS reports that the approach can reduce dataset onboarding from weeks to days while supporting traceability, versioning, data classification, and governance. By Leela Kumili
AI 资讯
AWS Serverless Weather Data Pipeline
Building a Serverless Weather Pipeline on AWS: A Step-by-Step Walkthrough This is a build log for someone who's used AWS a bit — deployed a Lambda from the console, poked around S3 — but hasn't touched CDK, Step Functions, EventBridge Scheduler, or GitHub's OIDC setup before. I'll explain each concept the first time it comes up, and show the actual code behind every piece, roughly in the order I built it. Here's what it ends up doing: every 10 minutes, EventBridge Scheduler kicks off a Step Functions workflow that pulls current weather for five cities in parallel from a free public API, reshapes the results into JSON Lines, drops them into S3 in a partitioned layout, and makes them queryable in Athena with plain SQL. No crawler, and no AWS credentials sitting anywhere in the GitHub repo that deploys it. kasukur / serverless-weather-pipeline AWS Serverless Weather Pipeline Serverless Weather Data Pipeline A small but complete serverless data pipeline on AWS walkthrough: EventBridge Scheduler → Step Functions → Lambda → S3 → Glue/Athena , deployed by GitHub Actions with no AWS access keys stored anywhere (authentication is via GitHub's OIDC provider). flowchart TD A["EventBridge Scheduler (every 10 min)"] --> B["Step Functions state machine"] B --> C["PrepareCities (Pass)"] C --> D["ForEachCity (Map, concurrency 4)"] D --> E["FetchWeather (Lambda -> Open-Meteo public API)"] E -.-> F["retries transient errors (up to 2 attempts)"] E -.-> G["FetchFailed (Pass): per-city failure absorbed here, other cities continue"] E --> H["TransformWeatherData (Lambda, pure function, no AWS calls)"] H -.-> I["splits successes vs failures"] H -.-> J["builds JSON-Lines body + partitioned S3 key"] H --> K["LoadToS3 (Lambda, writes to S3 via boto3)"] K --> L["S3 (processed/dt=YYYY-MM-DD/hour=HH/*.jsonl)"] L --> M["Glue Data Catalog table (partition projection -- no crawler)"] M --> N["Athena (query with plain SQL)"] D -.-> … View on GitHub Table of Contents What we're building, and why eac
AI 资讯
React Form Backends Compared: Serverless Functions vs. Form-as-a-Service
React Form Backends Compared: Serverless Functions vs. Form-as-a-Service React makes building a form straightforward. What happens after onSubmit is a different question: you still need somewhere to validate, process, store, or forward the submission. Two common approaches are writing a serverless function yourself or using a hosted form backend such as onsubmit.dev (form backend). This article compares the two, using Vercel/Netlify-style functions for the DIY approach and onsubmit.dev with its React integration as the managed example. The basic problem Imagine a typical contact form: function ContactForm () { return ( < form > < input name = "email" type = "email" required /> < textarea name = "message" required /> < button type = "submit" > Send </ button > </ form > ); } The React component is only the UI. A real application usually needs backend behavior too: accepting the HTTP request validating and sanitizing input handling errors preventing abuse or spam delivering or storing the submission keeping credentials and other secrets off the client There are two broad ways to get that backend. Option 1: Build a serverless function With platforms such as Vercel and Netlify, you can create an HTTP function alongside your application and have your React form submit to it. Conceptually, the architecture looks like this: React form | v Your serverless function | +--> validation +--> email provider +--> database +--> other services The main advantage is control. Your function owns the request lifecycle, so you decide precisely how data is validated, transformed, authenticated, stored, and forwarded. If a submission needs to update PostgreSQL, call an internal API, enqueue a job, and return application-specific data, a custom backend is usually the natural solution. Serverless functions can also reduce product-level vendor lock-in. Although platforms have their own deployment conventions, HTTP handlers and their business logic are generally portable with some work. The tr
AI 资讯
Nvidia senior manager linked to Supermicro scheme smuggling AI servers to China
Nvidia worker indicted after Jensen Huang scolded Supermicro for AI server smuggling.
AI 资讯
AWS Serverless Patterns and Anti-Patterns: What Works, What Breaks, and When to Use What
Serverless on AWS isn't "just use Lambda." It's a design philosophy: let AWS manage the infrastructure, pay only for what you use, and build with managed services that scale independently. But the patterns that work in serverless are fundamentally different from traditional architectures — and the anti-patterns are expensive to learn the hard way. This guide covers the patterns that work in production, the anti-patterns that waste money or cause outages, and the decision framework for when serverless is the right (or wrong) choice. The Serverless Building Blocks ┌─────────────────────────────────────────────────────────────────────┐ │ AWS SERVERLESS STACK │ ├─────────────────────────────────────────────────────────────────────┤ │ COMPUTE │ Lambda | Fargate (serverless containers) │ │ API │ API Gateway (REST/HTTP/WebSocket) | AppSync (GraphQL)│ │ ORCHESTRATION │ Step Functions | EventBridge Scheduler │ │ MESSAGING │ SQS | SNS | EventBridge │ │ STORAGE │ S3 | DynamoDB | Aurora Serverless │ │ STREAMING │ Kinesis | DynamoDB Streams | MSK Serverless │ │ AUTH │ Cognito | IAM | Lambda Authorizers │ │ OBSERVABILITY │ CloudWatch | X-Ray | Application Signals │ └─────────────────────────────────────────────────────────────────────┘ Key principle: In serverless, you compose applications from managed services. Lambda is the glue between them — not the application itself. Pattern 1: Synchronous API (Request/Response) The most common serverless pattern: HTTP API backed by Lambda. Client → API Gateway → Lambda → DynamoDB / Aurora Serverless │ Response ← ─ ─ ─ ─ ─ ─ ┘ Best Practices API Gateway HTTP API (not REST API) — cheaper, faster, simpler for most cases One Lambda per route (single responsibility) — not a monolith Lambda Keep Lambda warm — use Provisioned Concurrency for latency-sensitive endpoints DynamoDB for simple access patterns — scales with traffic, no connection pooling Aurora Serverless v2 for complex queries — but use RDS Proxy to manage connections When to Choose H
AI 资讯
Serverless: When It Helps and When It Hurts
The Allure of Serverless Serverless computing, despite its name, still runs on servers. The difference is that you don't manage them. You deploy functions, and the cloud provider handles scaling, patching, and availability. The promise is simple: you focus on code, not infrastructure. That's genuinely appealing for many projects, but it's not a silver bullet. Let's talk about when serverless shines and when it becomes a headache. When Serverless Helps 1. Spiky and Unpredictable Traffic Serverless scales automatically. If you have a sudden surge of users, functions spin up to handle the load, then scale down to zero when idle. You pay only for what you use. This is ideal for APIs with variable traffic, like a mobile app backend that sees daily peaks and quiet nights. For example, a simple REST endpoint using AWS Lambda and API Gateway: exports . handler = async ( event ) => { const body = JSON . parse ( event . body ); // process request return { statusCode : 200 , headers : { ' Content-Type ' : ' application/json ' }, body : JSON . stringify ({ message : `Hello, ${ body . name } !` }) }; }; No server to configure, no load balancer to set up. It just works. 2. Event-Driven Workloads Serverless excels at reacting to events: file uploads, database changes, messages in a queue. You can glue services together with minimal code. For instance, resizing an image when it's uploaded to S3: import boto3 from PIL import Image import os s3 = boto3 . client ( ' s3 ' ) def handler ( event , context ): bucket = event [ ' Records ' ][ 0 ][ ' s3 ' ][ ' bucket ' ][ ' name ' ] key = event [ ' Records ' ][ 0 ][ ' s3 ' ][ ' object ' ][ ' key ' ] download_path = ' /tmp/ ' + key upload_path = ' /tmp/resized- ' + key s3 . download_file ( bucket , key , download_path ) with Image . open ( download_path ) as img : img . thumbnail (( 200 , 200 )) img . save ( upload_path ) s3 . upload_file ( upload_path , bucket , ' resized/ ' + key ) This is a perfect serverless use case: short-lived, statele
AI 资讯
The Serverless Equation: Conquering the Cold Start in Real-Time AI Inference
In our inaugural issue , we established that the future of enterprise AI lies not merely in raw model parameters, but in the architectural paradigms—specifically Graph Neural Networks (GNNs)—that capture relational intelligence. However, the most sophisticated architectural decision is rendered obsolete if the deployment infrastructure introduces prohibitive latency. At Informatiqs, we emphasize that model deployment is fundamentally an operations research problem. As we transition from batch-processed predictions to real-time Generative AI and dynamic Machine Learning on Google Cloud Platform (GCP), we confront the inherent friction between compute elasticity and system responsiveness: the notorious "Cold Start" problem. In this issue, we dissect the mathematics of serverless inference, the orchestration of Cloud Run and Eventarc, and how minimizing initialization latency is the ultimate enabler for high-frequency, event-driven enterprise intelligence. 1. The Mathematical Anatomy of the Cold Start To engineer a solution, we must first formalize the problem. In a serverless architecture (scale-to-zero), infrastructure scales dynamically with demand. The total response time for an inference request can be understood as a composite of three phases. First, the baseline network latency. Second, the actual inference time—the computational effort of the model itself. The critical variable, however, is the conditional penalty phase. If a serverless container has scaled to zero, the system must endure the time required to provision new compute resources and the heavily taxing process of loading massive neural network weights into memory. If the container is already 'warm', this penalty is completely bypassed. We can model the probability of encountering this cold start using queueing theory. Assuming incoming inference requests arrive as a stochastic process, the likelihood of a cold start is determined by the mathematical relationship between the frequency of incoming requ
AI 资讯
Building a viral Imax ticketing app that never crashes
When 150,000 tickets went on sale for The Odyssey in 70mm IMAX, they sold out almost instantly. But plans change, cancellations happen, and good seats randomly open up at odd hours. To solve this, Andrew Baker from Temporal built IMAXXING : a service that monitors every 70mm IMAX showing across the US and alerts subscribers the moment great seats become available. What started as a fun weekend project quickly scaled, now over 9,000 users. I sat down with Andrew to break down the architecture: how durable execution keeps long-running workflows alive, how to debounce alerts so you don't spam users, and how serverless workers on Google Cloud Run handle sudden spikes in demand without overprovisioning. What's in the video Durable execution 101: How Temporal allows you to rewind history to the point of failure. The Entity Workflow pattern: Why there is one persistent workflow per user subscription and separate monitoring workflows per showing across the country. Signals & smart debouncing: How showing workflows send signals to wake up subscription workflows, and how a 60-second in-workflow timer batches multiple theater alerts into a single digest—without consuming active CPU while sleeping. Serverless workers on Cloud Run : How running Temporal workers as serverless containers lets compute autoscale directly with task queue depth rather than generic CPU metrics. AI agents for ops: How modern coding agents paired with Terraform and the gcloud CLI accelerated the deployment and operational dashboard setup. The point that stuck with me is how durable execution fundamentally changes how you think about long-lived state and retries. Instead of building complex cron jobs, custom retry databases, and alert queues, the workflow state itself is the queue and the timer. Have you experimented with entity workflows or running workflow workers on serverless infrastructure? How do you handle debouncing and noisy downstream APIs in your own apps?
AI 资讯
Physical Server vs Cloud Server: Which Infrastructure Makes More Sense?
When building an application, we usually focus on the frontend, backend, APIs, and database. But there is another important question: Where should the application actually run? Two common approaches are physical servers and cloud/virtual servers. Understanding the difference is important because infrastructure decisions affect scalability, availability, security, maintenance, and cost. What Is a Server? A server is a computer system that runs applications, processes requests, communicates with databases, and provides information to users. A typical request might look like: User → Internet → Application Server → Backend → Database → Response Depending on the application, the server may handle authentication, APIs, user data, file processing, notifications, and other backend operations. In simple terms, the server provides the execution environment behind the application. Physical Server: More Control, Less Flexibility A physical server is a dedicated machine used to run applications. For example: 16 CPU cores + 64 GB RAM + 2 TB SSD Advantages: • Dedicated hardware • Predictable performance • Greater hardware-level control • Suitable for stable workloads Limitations: • Higher initial investment • Hardware maintenance • Hardware failures can cause downtime • Scaling requires additional or upgraded hardware If an application suddenly grows beyond the capacity of the machine, increasing capacity may require purchasing and configuring new hardware. Cloud / Virtual Server: Infrastructure That Can Adapt A cloud server is a virtual server running on physical infrastructure inside a cloud data center. For example: 4 vCPU + 16 GB RAM + SSD Instead of purchasing the entire physical machine, resources can be provisioned according to the application's requirements. Cloud environments also provide different scaling approaches. Scale Up: Increase the resources of an existing server. 4 vCPU → 8 vCPU → 16 vCPU Scale Out: Add additional application instances. Application Server 1 + Ap
AI 资讯
Harper Argues Against the Multi-System Stack and Releases 5.2
The database platform Harper advocates for a single-runtime architecture that keeps application code and data together, with its benchmark against a Vercel-based stack reporting significantly better performance on live, personalized-data workloads. Harper recently released version 5.2, with a new record cache and more throughput per node. By Renato Losio
AI 资讯
Building a Disposable Notion Agent on Cheap Models
TL;DR: We built a one-shot HTTP worker that talks to Notion through MCP. Version one worked. Version two got cheaper and more readable, then failed in a new way. The harness was fine. The tool surface, the model, and the prompt were not the same problem, and we kept treating them as one. We keep seeing the same pitch: put an agent in the cloud, give it tools, let it live in Slack, let it remember you. That is a product. It is not the product we needed. We needed something dumber and more useful. Another service should be able to say "read this Notion page, write a summary somewhere, stop." No chat history. No personality that accretes over weeks. No always-on process. If nobody is calling it, it should cost nothing. We started calling that shape a one-shot agent . One HTTP request. Tools for that request. A JSON result. Then the instance can go away. This is the path we actually walked: first working version, what it got wrong, the Markdown fork, and the cheaper tricks that mattered more than swapping frameworks. The job was never "build a chatbot" The first real task was almost boring. Once a week, pull a skill write-up from Notion, extract what mattered, and append it to a digest page. Callers would name pages in English. They would not paste Notion ids. If a name was ambiguous, the agent should refuse to write rather than guess. If that loop is wrong, people stop trusting write-back. If it is expensive, nobody schedules it. If it needs a human to babysit a terminal, it is not a system. So the constraints were social as much as technical: An external caller owns the schedule. The agent does not. The agent must be allowed to use tools, not just talk about them. Secrets stay in the environment, never in the request body. Idle time should be free. Question you will probably ask: why not a cron script that hits the Notion API directly? Because the task changes every call. This week it is a weekly digest. Next week it is "list in-progress rows and do not write." We did
AI 资讯
Cloudflare Migrates JavaScript CDN Serving 9B Requests a Day to Its Developer Platform
Cloudflare has migrated cdnjs, its open source CDN for JavaScript and CSS libraries, to its Developer Platform. The new architecture uses Workers, R2, KV, Workflows, Queues, Durable Objects and Containers, consolidating publishing and delivery infrastructure while preserving package contents, URLs and SRI hashes at a scale of 9 billion requests per day. By Leela Kumili
AI 资讯
Advanced Server-Side Caching Patterns in Next.js: From Basic ISR to Granular Control
Originally published on tamiz.pro . Caching in modern web development is no longer just about serving static assets faster; it is the primary mechanism for balancing performance, cost, and data freshness. In the context of Next.js, the caching architecture has evolved significantly, shifting from a simple getStaticProps / getServerSideProps dichotomy to a sophisticated, multi-layered system that spans the Edge Runtime, the Server Components architecture, and the Node.js server environment. For software engineers and systems architects, understanding the default behaviors of Next.js caching is insufficient. To build production-grade applications that handle high concurrency without hammering your database, you must master the advanced patterns: granular revalidation, cache tagging, and external cache management. This article dives deep into these mechanisms, explaining how they work under the hood and how to orchestrate them for optimal performance. The Evolution of Next.js Caching To appreciate advanced patterns, we must first contextualize the current caching model. Next.js 13+ (App Router) introduced a new caching paradigm that is both simpler by default and more powerful when customized. The default behavior is now: App Router (RSC): Components are cached by default. Server Components are rendered once and cached on the server. The next request for the same data returns the cached result. Static Generation: Pages and layouts are built at build time and served statically. Server Components: Fetched data is cached in memory on the server, not in the browser. The critical shift here is that caching is opt-out, not opt-in . Previously, you had to explicitly mark things as static. Now, you must explicitly invalidate cache when data changes. This inversion of control places the responsibility of consistency squarely on the developer, requiring precise tools to manage invalidation. Granular Revalidation: The Tag-Based System The most significant advanced caching pattern
AI 资讯
Migrating From S3 to Branch-Aware Storage
If your files already live in Amazon S3, the pitch for storage that branches with your database is appealing but the word "migration" makes it sound like a project. It mostly is not. Neon's object storage speaks the S3 API, so the code you already wrote, the AWS SDK calls and presigned URLs, keeps working. What changes is how you point the client and where the bucket comes from, and that is a small, mechanical diff. The actual data move is a copy loop you can run once. The one thing to do up front is confirm the object operations your app actually relies on: the demo here exercises PutObject , GetObject , listing, and presigned URLs, and I flag the S3 features you should check for yourself further down. This post is the practical version: what stays identical, the exact config that changes, a script to copy the objects across, and an honest list of the S3 features that do not have an equivalent so you know what to check before you commit. The repo with the working client is at the end. TL;DR Neon object storage is S3-compatible. Your @aws-sdk/client-s3 code for the common operations, PutObject , GetObject , getSignedUrl , listing, works unchanged (these are what the demo verifies). Confirm anything beyond that, like multipart for large objects, against the current preview. The diff is the client config: point endpoint at the Neon storage endpoint, pin region: 'us-east-2' , set forcePathStyle: true . The bucket is declared in neon.ts instead of created in the console, and credentials are injected per branch. Move the data with a list-and-copy loop between two S3 clients (source AWS, destination Neon). What does not carry over: S3 bucket policies, event notifications and Lambda triggers, storage classes and Glacier transitions, and cross-region replication. Object CRUD and presigning do. The payoff is everything else in this series: once the files are on Neon, they branch with your database. Prerequisites An existing S3 bucket and credentials that can read it A Neon p
AI 资讯
Stop Standing Up an S3 Bucket Per Preview Environment
If your app stores files and you want real preview environments, you eventually hit the same wall: each preview needs its own storage, so you start provisioning a bucket per environment. That sounds cheap until you write it down. For every ephemeral environment you create a bucket, attach a policy, mint an IAM role or access keys, set CORS, add a lifecycle rule so it eventually cleans up, wire the credentials into the preview's config, and register a teardown step for when the PR closes. Then you find the orphaned buckets the teardown missed, months later, still billing. The reason this is painful is that the bucket is a separate resource from the database, so it needs its own lifecycle. Neon collapses that: the bucket is declared as part of the branch, so it is created and destroyed with the branch and needs no per-environment provisioning at all. This post compares the two approaches and shows the branch version working with no bucket-management code in sight. The repo is at the end. TL;DR Isolated storage per preview usually means provisioning a bucket per environment: policy, IAM, CORS, lifecycle, credential wiring, teardown. It is slow, it drifts, and it leaves orphaned buckets that keep costing money. On Neon the bucket is declared once in neon.ts . Creating a branch brings the bucket (with a copy-on-write copy of the files) and injects scoped credentials; deleting the branch removes it. There is no per-environment bucket to create, no IAM role to mint, and nothing to orphan. Copy-on-write means fifty preview buckets do not cost fifty times the storage, only what each one changes. Prerequisites A Neon project on the platform preview (object storage, us-east-2 ) The Neon CLI, and a CI system that opens/closes preview environments Familiarity with S3 buckets and IAM if you have done the manual version The per-environment bucket, written out Here is what "just give the preview its own bucket" actually expands to, per environment: Create a bucket with a unique nam