标签：#AWS

I used to deploy Node.js apps on EC2 and manage servers like it was my second job. Port configs. PM2 restarts. Nginx rewrites. SSL renewals. Then I ran my first AWS Lambda function. 80% of that work is gone. Here's what Lambda actually does that nobody explains clearly: → You write a function → AWS runs it ONLY when triggered → You pay for milliseconds of execution → It scales from 1 to 1,000,000 requests without you touching anything As a full-stack developer in Bahrain, preparing for my AWS Developer Associate exam, this is the shift that changes how you think about backend architecture. Not "how do I manage a server" but "what should happen when this event fires." That mental model switch took me a week to fully get. I'm documenting everything as I study. Drop a 🔥 if you want me to share my Lambda notes weekly.

TL;DR We built a serverless Lambda pipeline that ships FSx for ONTAP audit logs to Sumo Logic's JP (Tokyo) region deployment. For Japanese enterprises with data residency requirements under APPI (Act on the Protection of Personal Information), this means audit logs never leave Japan. FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → Sumo Logic HTTP Source (JP) │ ▼ ┌───────────────────┐ │ Sumo Logic JP │ │ (Tokyo) │ │ │ │ • 500 MB/day FREE │ │ • Data stays in │ │ Japan │ │ • 7-day retention │ │ (free tier) │ └───────────────────┘ Key advantages: 500 MB/day free tier (~15 GB/month) — covers most FSx for ONTAP deployments at zero vendor cost JP region deployment — data residency in Tokyo Simplest auth model — URL-embedded token, no header management 30-minute end-to-end — HTTP Source URL is the only credential needed Verified on Sumo Logic JP region. Logs searchable via _sourceCategory=aws/fsxn/audit . This is Part 12 of the Serverless Observability for FSx for ONTAP series. Why Sumo Logic for Japanese Enterprises? For organizations operating under Japanese data protection regulations, the choice of observability platform often comes down to one question: where does the data physically reside? Requirement Sumo Logic JP Other Options Data residency in Japan ✅ Tokyo deployment Varies by vendor APPI compliance consideration ✅ Data stays in JP May require cross-border assessment Free tier for validation ✅ 500 MB/day Most offer 14-day trials only No agent installation ✅ HTTP Source (agentless) Some require collectors Sumo Logic's JP deployment ( service.jp.sumologic.com ) processes and stores all data within Japan, making it a straightforward choice for organizations that need to demonstrate data residency compliance. Compliance note : This integration provides a technical path for data residency. Evaluate your specific regulatory requirements with your compliance team — data residency alone does not constitute full regulatory compliance. Architecture ┌────

TL;DR We built a serverless Lambda pipeline that ships FSx for ONTAP audit logs to Dynatrace via the Log Ingest API v2. The real value: Dynatrace's Davis AI can automatically correlate file access anomalies with application performance degradation — answering "why is the app slow?" with "because 500 users hit the same NFS share simultaneously." FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → Dynatrace Log Ingest API v2 │ ▼ Davis AI ┌───────────────────┐ │ Correlates: │ │ • File access │ │ anomalies │ │ • APM metrics │ │ • Infrastructure │ │ health │ │ │ │ → Root cause │ │ in seconds │ └───────────────────┘ Verified on Dynatrace SaaS Trial (Tokyo-equivalent region). Logs visible in Logs Viewer within 1-2 minutes. This is Part 11 of the Serverless Observability for FSx for ONTAP series. Why Dynatrace for FSx for ONTAP? Most observability tools treat storage logs as isolated data. Dynatrace is different — it builds a topology map of your entire stack and uses Davis AI to find causal relationships through time-window correlation and entity connectivity: Scenario Without Dynatrace With Dynatrace App latency spike "Check the logs" Davis AI detects temporal correlation: file access to /vol/data/ increased 10x within the same 5-minute window as app response time degradation, connected via topology (app → NFS mount → SVM) Storage I/O anomaly Manual investigation Automatic correlation via shared topology entities — Davis identifies which services are affected based on entity relationships User reports slow file access Grep through audit logs DQL query + topology view showing the full dependency path from user request to storage operation The key differentiator: Davis AI correlates events across entities that share topology connections within overlapping time windows — not just keyword matching or manual dashboard correlation. Architecture ┌─────────────────────────────────────────────────────────┐ │ Event Sources │ ├─────────────────────────────────────────

We were running a cloud-hosted platform on AWS EKS , with EC2 worker nodes managed by us, MongoDB Atlas for NoSQL workloads, AWS RDS for relational databases, and Amazon ElastiCache for Redis for caching and temporary data. Over time, the infrastructure had grown the way most real systems grow: more services, more data, more backups, more images, more snapshots, and more “temporary” resources that were no longer temporary. The platform worked, but the cloud bill was higher than it needed to be. So we started cutting waste, improving the application, and resizing the infrastructure around how the system actually behaved. The result: around $120,000 in annual savings , without sacrificing reliability. The Problem Was Not One Big Thing When we started reviewing the infrastructure, it was clear that there was no single expensive resource causing the entire problem. The cost came from many places at once. Some services were using more CPU and memory than they needed. Some microservices did not really need to be separate anymore. Some databases were oversized for their actual usage. Some storage had accumulated over time. Some backups and snapshots were kept longer than necessary. Some resources were simply unused. That is usually how cloud costs grow. Not because of one bad decision, but because of hundreds of small decisions that were reasonable at the time and never revisited later. So instead of looking for one magic fix, we approached the problem from multiple angles: application code, architecture, databases, Kubernetes resources, storage, backups, caching, and non-production environments. The Optimizations 1. Making the Application Use Fewer Resources One of the most important parts of the optimization was improving the application itself. It is easy to look at cloud cost as an infrastructure problem only, but inefficient code directly affects infrastructure cost. If the application uses too much CPU or memory, the platform needs more pods, larger nodes, bigger ins

TL;DR We built a serverless pipeline that ships FSx for ONTAP audit logs to Honeycomb, where its high-cardinality query engine turns file access data into actionable insights. Two delivery paths verified: [Path A: Direct] FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → Honeycomb Events Batch API [Path B: OTel Collector] FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → OTel Collector → OTLP → Honeycomb Why Honeycomb for file access logs? Because file access data is inherently high-cardinality : thousands of users × millions of file paths × dozens of operations × multiple SVMs. Traditional log tools force you to pre-aggregate or sample. Honeycomb lets you query the raw events at full resolution. ┌──────────────────────────────────────────────────────┐ │ Honeycomb Query Engine │ │ │ │ "Show me which users accessed /vol/finance/* │ │ between 2am-4am last Tuesday" │ │ │ │ → BubbleUp: auto-detect anomalous dimensions │ │ → Heatmap: visualize access density over time │ │ → GROUP BY user, path, operation — no pre-indexing │ │ │ │ 20M events/month FREE │ └──────────────────────────────────────────────────────┘ This is Part 10 of the Serverless Observability for FSx for ONTAP series. Why Honeycomb for File Access Logs? Most observability tools index a fixed set of fields. When you have high-cardinality dimensions — like file paths ( /vol/data/project-alpha/2026/Q1/report-final-v3.docx ) or Active Directory usernames — you hit index bloat, slow queries, or forced sampling. Honeycomb's columnar storage handles this natively: Capability Traditional Logs Honeycomb Query by arbitrary field Pre-index or full scan Instant (columnar) GROUP BY high-cardinality field Expensive / limited Native BubbleUp (anomaly detection) Manual investigation Semi-automatic (select time range, BubbleUp identifies differing dimensions) Heatmap visualization Requires pre-aggregation Raw events For FSx for ONTAP audit logs, this means you can ask questions like: "Which

Building a Unix-Domain-Socket IPC server for ECS-on-EC2 services that need to talk fast, cheap, and reliably A while back I was looking at a flamegraph of a service that, on paper, should not have been having any performance problems. The producer and the consumer were the same Docker image's worth of trouble — colocated on the same EC2 host, in the same ECS cluster, sharing the same instance type, the same kernel, the same RAM. By every reasonable measure they were neighbours. And yet every event was making a round trip that looked roughly like this: producer → kernel TCP stack → ENI on the producer task → AWS VPC → internal load balancer → ENI on the consumer task → kernel TCP stack → consumer. TLS handshake. HTTP framing. JSON over the wire. Connection pool. Retry policy. The whole circus. I wasn't doing anything wrong. This is what the platform funnels you toward. ECS with awsvpc networking gives every task its own ENI. The default story for "service A talks to service B" is "give B a DNS name, put a load balancer in front of it, configure a security group, point A at the LB." Even if A and B are physically on the same box, the bytes are still leaving the kernel, traversing the VPC, and coming back. There's a fix for this. It's been a fix for fifty-something years. It just hasn't been the default fix, because cloud-native architecture grew up assuming services would be scattered across hosts and the network was the abstraction that mattered. This article is about building a proper IPC server using Unix Domain Sockets, deployed as a sidecar pattern on ECS-on-EC2, with a wire protocol robust enough to ship in production. We're going to design it from scratch — the transport choice, the wire format, the backpressure model, the failure modes, the deployment topology. I'll show you real pseudo-code from the implementation and call out the small number of places where, if you get it wrong, you'll spend a weekend debugging it. The intended outcome is something you coul

How a shared filesystem became the cheapest, fastest outbox I've ever built — and why FSx for OpenZFS is the version of that idea that finally scales I was staring at an AWS bill last quarter where a single Kinesis Data Streams line item was costing more than the entire S3 footprint sitting behind it. The events on that stream had a useful lifetime of about ninety seconds. They were written by one service, read by another, processed, and dropped. We were paying full streaming-bus price for bytes that barely outlived a TCP timeout. That bill is what got me thinking about transitional data as a category that deserves its own architecture, and about why every "use the right tool" instinct I had — Kinesis, Kafka, MSK — was the wrong tool for this particular shape of work. The right tool, it turns out, is a filesystem. Specifically, AWS FSx for OpenZFS, used as an outbox between producers and consumers, with only a tiny pointer message traveling through whatever messaging bus you already have. This article is the case for that pattern. It's also the design, the failure modes, the code, the cost math, and the honest list of when not to do it. I'll walk you through the architecture from first principles, show you the safe-write protocol that makes it correct under crashes and concurrent retries, compare the cost against Kinesis, MSK and EFS at a realistic petabyte-class workload, and explain why the recent addition of FSx Intelligent-Tiering changes the cost story in a way that makes the pattern attractive even for teams that don't ingest petabytes. If you've ever felt the queasy sensation of paying twice for the same bytes — once to land on a stream, again to land in storage — this is for you. What "transitional data" actually means Most data falls into one of two cleanly shaped buckets. Durable data is the stuff you keep — user records, orders, financial events, audit trails. It needs to live for years; you pay storage costs for those years and you get value over those y

Introduction This article is an English translation of the article at the following URL, which was originally written in Japanese. The screenshots are still in Japanese. Sorry about that. https://qiita.com/amarelo_n24/items/e196b74f718c750a0e18 The penetration testing feature for AWS Security Agent (hereinafter referred to only as "Security Agent"), which was announced at AWS re:Invent 2025, has been generally available (GA). Code review and design review are still in preview as of May 25th, so those who haven't been able to try Security Agent yet can still try these features. I wasn't able to try penetration testing during the preview period , so I decided to at least experience code review and made my Security Agent debut! This article reflects the author's personal views. It is based on personal testing and should be used for reference only. Furthermore, the author has no experience in app development, so the terminology used may not be entirely accurate. Any corrections or errors in the content would be greatly appreciated. This article was written based on information as of May 25, 2026. What is a Security Agent? As mentioned above, this service was announced during AWS re:Invent 2025. It is a frontier agent that proactively protects applications throughout the entire development lifecycle in all environments (quoted from the official AWS page). https://aws.amazon.com/security-agent/ It includes three features that became generally available (GA) in April: penetration testing, design review, and code review (the subject of this article). Function name Feature Overview Status（As of 2026/5/25） Penetration testing Attempting to infiltrate the system from an external source to evaluate security measures. GA Design Review Analyze product specifications, architecture documents, and technical designs from a security risk perspective. Preview Code Review Inspect source code and repositories to detect code-level vulnerabilities. Preview Code security review (hereinafter

If you're using Amazon SES and emails to @privaterelay.appleid.com are silently failing, the cause is almost certainly SES's account-level suppression list treating Apple relay DSN errors as hard bounces. Fix: Emails Not Delivered to Apple Private Relay Addresses (Amazon SES) If your app supports Sign in with Apple , some of your users will have a Hide My Email address — a relay address like abc123@privaterelay.appleid.com that forwards to their real inbox. These addresses are easy to break silently. Here's the symptom we ran into and exactly how we fixed it. The Symptom Transactional emails (alerts, welcome emails) worked fine for regular email addresses but never arrived for users who signed in with Apple. We were also receiving bounce notifications like this: Subject : Delivery Status Notification (Failure) An error occurred while trying to deliver the mail to the following recipients: prw8xms8tv@privaterelay.appleid.com Confusingly, some of these emails were being delivered — Apple's relay occasionally returns a DSN error even on successful delivery. But over time, delivery stopped entirely for affected addresses. Root Cause: SES Suppression List Amazon SES has an account-level suppression list . When a send results in a bounce (even a soft or misleading one), SES adds that address to the suppression list and silently drops all future sends to it — no error, no log entry from your code's perspective. Apple's private relay sometimes returns a non-standard response that SES interprets as a hard bounce. Once that happens: Send to Apple relay → Apple returns DSN error → SES logs as hard bounce → Address added to suppression list → Every future send silently dropped We found 9 Apple relay addresses on our suppression list, the oldest suppressed since September 2025 — meaning those users had missed months of emails. The Fix Step 1 — Remove suppressed Apple relay addresses In the AWS Console : Go to Amazon SES (make sure you're in the correct region) Left sidebar → Con

As AI agents move from experiments to production, AWS, Cloudflare, and others are redesigning cloud infrastructure for a future dominated by machine-generated internet traffic instead of human users.

Appendix: Live System Output — Real Pipeline in Production All output below was captured live from the running pipeline on 2026-03-08. These are not mock outputs — they come from actual AWS infrastructure and Kubernetes clusters. ArgoCD — All 50 Applications Across 6 Clusters The following is the live output of argocd app list from the hub cluster ( myapp-production-use1 ). Every component of the pipeline is represented — security, logging, monitoring, backups, and the application itself. $ argocd app list --output wide NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH argocd/argo-rollouts-myapp-production-use1 myapp-production-use1 argo-rollouts production Synced Healthy argocd/argo-rollouts-myapp-production-usw2 myapp-production-usw2 argo-rollouts production Synced Healthy argocd/aws-lbc-myapp-production-use1 myapp-production-use1 kube-system production Synced Healthy argocd/eso-myapp-production-use1 myapp-production-use1 external-secrets production OutOfSync Healthy ← known false positive argocd/eso-myapp-production-usw2 myapp-production-usw2 external-secrets production OutOfSync Healthy ← known false positive argocd/falco-myapp-dev-use1 myapp-dev-use1 falco production Synced Healthy argocd/falco-myapp-dev-usw2 myapp-dev-usw2 falco production Synced Healthy argocd/falco-myapp-production-use1 myapp-production-use1 falco production Synced Healthy argocd/falco-myapp-production-usw2 myapp-production-usw2 falco production Synced Healthy argocd/falco-myapp-staging-use1 myapp-staging-use1 falco production Synced Healthy argocd/falco-myapp-staging-usw2 myapp-staging-usw2 falco production Synced Healthy argocd/fluent-bit-myapp-dev-use1 myapp-dev-use1 logging production Synced Healthy argocd/fluent-bit-myapp-dev-usw2 myapp-dev-usw2 logging production Synced Healthy argocd/fluent-bit-myapp-production-use1 myapp-production-use1 logging production Synced Healthy argocd/fluent-bit-myapp-production-usw2 myapp-production-usw2 logging production Synced Healthy argocd/fluent-bit-myapp-

The tech giant says a breakthrough in data-center networking has dramatically accelerated the flow of information through its massive cloud infrastructure.

Introduction In my previous IAM article we learnt basics of IAM and how to create Users, Groups and attach Policies. You can refer here: https://dev.to/kadhamvj23/aws-identity-and-access-management-explained-for-beginners-cn7 After setting up secure access to our AWS account using IAM, the next question we mostly have is where do we actually run our application? The answer is Amazon EC2 - Elastic Cloud Compute. EC2 is one of the most widely used AWS services and understanding it well is essential for anyone starting their cloud journey. In this article we will cover what EC2 is, why it exists, the different types of instances, pricing models, Regions and availability Zones and finally hands-on walk through of creating your first EC2 instance. Breaking Down the Name -EC2 Let us understand what each word in the name actually means: Elastic --> In AWS you will notice many services have this prefix "Elastic". The reason is simple. Whenever AWS provides a service that can be scaled up or scaled down based on our needs, that service is called Elastic . With EC2 you can increase resources when traffic is high and decrease them when the traffic is low. So in simple terms EC2 = A virtual server on the cloud that you can resize anytime. Cloud: EC2 runs on AWS's public cloud infrastructure, meaning the servers are owned and managed by Amazon across the world. Compute: The word compute means you are asking AWS to provide you CPU, RAM and Disk - basically a virtual machine or server that can run your applications. How does EC2 actually work? When you request a Virtual server from AWS, here is what happens behind the scenes: You request a virtual machine on AWS ⬇️ request goes to a Hypervisor(a software layer sitting on top of physical servers that creates and manages VMs) ⬇️ Hypervisor creates your VM ⬇️ You get the access to your EC2 instance You never touch any physical hardware. AWS manages all of that for you. Why use EC2? Imagine your company wants to host an application. T

Why We Built JaisCloud — A Free, Single-Binary AWS Emulator in Go If you've ever tried to test AWS-dependent code locally, you've probably reached for LocalStack. It works — but it comes with baggage: Python runtime, Docker dependency, and the features most teams actually need locked behind a Pro subscription. JaisCloud is our answer to that problem. What is JaisCloud? JaisCloud is a free, open-source local AWS cloud emulator written entirely in Go. It implements the exact AWS wire protocols — Query/XML, JSON/Target, REST — so your existing SDK code points at JaisCloud and works unmodified. No shims, no proxy rewrites, no SDK patches. # Start it jaiscloud-aws start # Point your SDK at it export AWS_ENDPOINT_URL = http://localhost:4566 export AWS_ACCESS_KEY_ID = test export AWS_SECRET_ACCESS_KEY = test export AWS_REGION = us-east-1 # Your existing code works — no changes needed aws s3 mb s3://my-bucket aws sqs create-queue --queue-name my-queue The Problem With Existing Solutions JaisCloud LocalStack Community Moto Single static binary Yes No - Python + Docker No - Python library Zero runtime deps Yes No No Postgres persistence Yes Pro only No Real Spark/EMR execution Yes No No Apache Iceberg Yes No No Prometheus metrics Yes Pro only No State export / import Yes No No Deterministic time control Yes No No Written in Go Yes No No License Apache-2.0 Apache-2.0 Apache-2.0 Key Features Single Static Binary JaisCloud ships as a single Go binary per cloud. No Python, no Docker, no Node — just download and run. Works on laptop, CI runner, or Kubernetes pod. # Download and run — that is it ./jaiscloud-aws start # Or with Docker docker run --rm -p 4566:4566 rjaisval/jaiscloud-aws:latest Portable State Snapshots Export the complete emulator state — every resource, every account, every region to a single gzip tarball and restore it anywhere in milliseconds. # Capture a baseline jaiscloud-aws export -o baseline.tar.gz # Restore on a teammate's machine or CI runner jaiscloud-aws i

Snowflake has signed a new, enormous five-year deal with Amazon to secure chips for AI usage. Nvidia is once again being put on notice.

Until this past weekend, a contractor for the Cybersecurity & Infrastructure Security Agency (CISA) maintained a public GitHub repository that exposed credentials to several highly privileged AWS GovCloud accounts and a large number of internal CISA systems. Security experts said the public archive included files detailing how CISA builds, tests and deploys software internally, and that it represents one of the most egregious government data leaks in recent history.