标签：#backend

Hi, I'm Heet Sanghani, a Python Developer and AI/ML Engineer from Ahmedabad, Gujarat, India. I currently work at BrainerHub Solutions building backend systems and AI-powered applications. In this post, I want to share how I built BidXpert — a real-time auction platform using FastAPI. What is BidXpert? BidXpert is a real-time bidding platform where users can create auctions, place bids, and get instant updates using WebSockets. Tech Stack Backend: FastAPI + Python Database: PostgreSQL Real-time: WebSockets Auth: JWT Authentication Deployment: Docker What I Learned Building BidXpert taught me how to handle concurrent WebSocket connections efficiently in FastAPI and manage real-time state. About Me I'm Heet Sanghani — Python Developer & AI/ML Engineer based in Ahmedabad. Check out my portfolio and other projects at: 👉 https://heet-sanghani-portfolio.vercel.app/ python #fastapi #webdev #ai

Security is not a feature you bolt on after launch. It is not the CISO's problem alone. It is not a checklist you run through before a compliance audit. It is a shared responsibility across every engineer, every team, every layer of the stack. This guide walks through the three layers where most web vulnerabilities live — Frontend , In Transit , and Backend — using a threat modeling lens: thinking like an attacker so you can build like a defender. What Is Threat Modeling? Before writing a single line of defensive code, you need to think systematically about your system's attack surface. Threat modeling is the process of: Identifying entry points — Where does untrusted data enter your system? Form inputs, URL parameters, uploaded files, third-party APIs? Assessing potential impact — If this entry point is exploited, what can an attacker access or do? Designing defenses proactively — Before the exploit occurs, not after. It shifts your mindset from "let's hope nothing breaks" to "let's assume something will be tried." Part 1 — Frontend Security: Stopping XSS What Is XSS? Cross-Site Scripting (XSS) happens when untrusted data is rendered as executable code in a browser. An attacker injects a script; your application runs it on behalf of your users. The consequences are severe: session hijacking, credential theft, defacement, redirects to malicious sites. There are three flavours: ┌─────────────────────────────────────────────────────────────────┐ │ XSS TYPES │ ├──────────────────┬──────────────────────────────────────────────┤ │ Stored XSS │ Malicious script saved in your DB, │ │ │ served to every user who loads that data. │ │ │ Most dangerous — persistent and broad. │ ├──────────────────┼──────────────────────────────────────────────┤ │ Reflected XSS │ Script lives in a URL parameter. │ │ │ Requires tricking the user into clicking │ │ │ a crafted link. Temporary, per-request. │ ├──────────────────┼──────────────────────────────────────────────┤ │ DOM-Based XSS │ Entir

Previously, we learned how to send and receive data in Go. Now, we will combine those concepts and build a simple CRUD API. CRUD stands for: C reate R ead U pdate D elete These four operations form the foundation of most backend applications. In this tutorial, we will build a simple task API in Go using only the standar library. By the end, you will understand: how CRUD APIs work how to handle multiple HTTP methods how to store data in memory how to send and receive JSON data how backend APIs manage resources Prerequisites To follow along, you should have: Go installed basic familiarity with Go syntax understanding of the net/http package basic understanding of JSON handling You can confirm if Go is installed by running: go version Step 1 — Create the Project Create a new folder for the project: mkdir go-crud-api cd go-crud-api Now initialize a Go module: go mod init go-crud-api This creates a go.mod file for managing project dependencies. Step 2 — Create the Server File Create a file called main.go . Your project structure should now look like this: go-crud-api/ ├─ go.mod └─ main.go Step 3 — Write the CRUD API Open main.go and add the following code: package main import ( "encoding/json" "net/http" ) type Task struct { ID int `json:"id"` Title string `json:"title"` } var tasks [] Task func tasksHandler ( w http . ResponseWriter , r * http . Request ) { w . Header () . Set ( "Content-Type" , "application/json" ) switch r . Method { case http . MethodGet : json . NewEncoder ( w ) . Encode ( tasks ) case http . MethodPost : var task Task err := json . NewDecoder ( r . Body ) . Decode ( & task ) if err != nil { http . Error ( w , "Invalid JSON" , http . StatusBadRequest ) return } tasks = append ( tasks , task ) json . NewEncoder ( w ) . Encode ( task ) default : http . Error ( w , "Method not allowed" , http . StatusMethodNotAllowed ) } } func main () { http . HandleFunc ( "/tasks" , tasksHandler ) http . ListenAndServe ( ":8080" , nil ) } Now let's unpack what is hap

The Logging Nightmare Deploying microservices across Multi-Cloud environments using tools like Terraform is an exhilarating milestone. But the moment something breaks, that excitement quickly turns into a nightmare. The SSH Grind : If you find yourself SSH-ing into disparate instances just to run tail -f and grep through scattered log files, you're doing it wrong. The Agnostic Approach : The industry standard demands Centralized Logging, but chaining your application to vendor-specific solutions like AWS CloudWatch or GCP Cloud Logging limits your architectural freedom. Implementing a true "Cloud-Agnostic" ELK stack gives you back control over your observability data. Clean Architecture & The Non-Blocking Logger Factory Building this robust observability pipeline requires adhering to Clean Architecture principles, specifically through a Non-Blocking Logger Factory. Standardized Interface : By wrapping modern logging libraries like Winston or Pino , we standardize our application's logging interface. The Secret Sauce : The winston-elasticsearch transport module buffers your logs and pushes them directly to your Elasticsearch cluster in the background. Non-Blocking : This architectural choice is crucial: it ensures that high-volume log streaming happens without blocking the Node.js event loop . Here is how the data flows through the system: Resilience Fallback (The Failsafe) A centralized system introduces a dangerous dependency. Your logging infrastructure must never be the reason your application crashes. The Threat : If the remote Elasticsearch cluster is unreachable due to network partitions or rate limits, a poorly configured logger will throw uncaught exceptions, bringing down the app. The Solution : We implement a strict Resilience Fallback (Failsafe) mechanism. The transport module safely catches the connection errors and seamlessly falls back to standard output (console), guaranteeing continuous operation. The 512MB Local-Test Challenge While this setup is a

A survival guide for when everything goes wrong in production. Your Redis dashboard looks perfect. Hit ratio: 99.2%. Latency: sub-millisecond. Memory usage: 60% of available. Every metric says healthy. Then at 2:47 PM, your API starts returning 500s. Response times spike to 30 seconds. Users can't log in. The dashboard still shows 99% hit ratio because the cache is working — it's serving cached errors to everyone equally fast. Redis is doing exactly what you told it to do. The problem is what you told it to do. Why Single-Threaded Is Fast (Until It Isn't) Redis processes commands on a single thread. No locks. No context switching. No synchronization overhead. One CPU core, fully utilized, can handle 100K+ operations per second because it never waits for another thread to release a lock. The event loop model (similar to Node.js) multiplexes thousands of client connections on a single thread using non-blocking I/O. Read a request, process it, write the response, move to the next. When your commands are simple — GET, SET, INCR — each one takes microseconds. The trap: slow commands block everything. KEYS * on a million-key database? That's a full keyspace scan on the main thread. While it runs, every other client waits. SORT on a large set? Same. LRANGE on a list with 10 million elements? Same. Redis 6.0 introduced I/O threading ( io-threads config) for reading and writing network data on multiple threads, but command execution is still single-threaded. Redis 7.0 improved this further, but the fundamental model hasn't changed. Long-running commands on the main thread stall everything. Rules: Never use KEYS in production. Use SCAN instead — it's cursor-based and returns results incrementally. Watch out for O(N) commands on large data structures: LRANGE , SMEMBERS , HGETALL on million-element structures. Use SLOWLOG to find commands that are blocking the event loop. Pipelining: The Easiest 10x You'll Ever Get Every Redis command involves a network round trip: send request

The honest decision framework, not another flame war. The SQL vs NoSQL debate has been running for 15 years and it still generates more heat than light. Here's the framework that actually helps you decide. The Real Question It's not "SQL or NoSQL." It's: what does your access pattern look like? If your application is mostly reading and writing related data through well-defined queries — orders with line items, users with addresses, products with categories — relational databases are purpose-built for this. JOINs are not expensive when they're indexed. Transactions are not slow when they're scoped correctly. PostgreSQL handles 50 million rows comfortably on a single node. If your application is reading and writing self-contained documents with predictable access by a primary key, and you rarely need cross-document queries — user profiles, product catalogs, content management — a document database simplifies your code. No ORM mapping hell. No migration files for adding a field. If your application writes massive volumes and reads by partition key with eventual consistency — time-series data, IoT telemetry, activity feeds at scale — wide-column stores like Cassandra were built for this specific workload. The 2026 Reality PostgreSQL has eaten NoSQL's lunch in many areas. JSONB support means you can store and query unstructured data inside PostgreSQL with GIN indexes. You get the document model flexibility without giving up transactions, JOINs, and a 30-year ecosystem. For 80% of startups and mid-size companies, PostgreSQL is the only database you need. MongoDB has gotten more relational. Multi-document ACID transactions (since 4.0), schema validation, aggregation pipelines that look suspiciously like SQL. It's converging toward what PostgreSQL already does, but with a different starting point. DynamoDB dominates serverless. If you're in AWS and your access pattern is simple key-value with known query patterns, DynamoDB's pricing model (pay-per-request) and operational s

A survival guide for when everything goes wrong in production. There's a moment every engineer who works with Kafka experiences. You check the producer. Messages are sending. You check the consumer. Nothing. The consumer group shows zero lag because there's nothing to lag behind — as far as the consumer knows, the topic is empty. But it's not empty. The messages are there. Somewhere. In some partition, at some offset, behind some configuration you set six months ago and forgot about. Kafka doesn't lose messages. But it's very good at hiding them from you. Consumer Lag: The Number Everyone Watches Wrong Consumer lag is the difference between the latest offset in a partition and the offset your consumer group has committed. Simple concept. Dangerous in practice. The mistake: treating lag as a single number. Lag is per-partition. If you have 30 partitions and one consumer is stuck on partition 17 while the others are healthy, the total lag looks manageable. But partition 17's data is hours behind, and whatever downstream system depends on that data is serving stale results. Monitor lag per partition. Tools like Burrow, Kafka Exporter for Prometheus, or even kafka-consumer-groups.sh --describe break it down. If one partition's lag is growing while others are stable, you have a stuck consumer, a hot partition, or a poison message. A poison message is a record your consumer can't process — malformed data, unexpected schema, null where it shouldn't be null. The consumer throws an exception, the offset doesn't commit, and it retries the same message forever. Lag grows. The consumer looks "alive" because it's processing — just not making progress. The fix: dead letter queues. After N retries, move the message to a separate topic, commit the offset, and move on. Alert on the dead letter topic. Investigate later. Don't let one bad record block millions of good ones. Rebalance Storms: The Silent Killer Consumer rebalancing is Kafka's mechanism for redistributing partitions acro

Si vous utilisez PostgreSQL, vous avez probablement déjà dû choisir entre une clé primaire BIGSERIAL et un UUID. Depuis des années, la version 4 (aléatoire) est le choix par défaut quand on veut un identifiant unique et distribué. Mais en 2026, une alternative plus récente s’impose : UUID v7, qui intègre un timestamp et promet de meilleures performances pour les index. Dans cet article, je vous explique concrètement ce qui change, avec des benchmarks PostgreSQL et des exemples de code, pour que vous puissiez décider en connaissance de cause. UUID v4 : le standard aléatoire et son problème d’index Un UUID v4 est constitué de 122 bits aléatoires. Cette absence totale de tri est sa force pour l’unicité, mais elle devient un handicap dans un index B‑tree, qui est la structure utilisée par PostgreSQL pour les clés primaires. Lorsque vous insérez un nouvel UUID v4, il a autant de chances de se retrouver au début de l’index qu’à la fin. Résultat : l’index se fragmente, les pages se remplissent mal, et les performances d’écriture se dégradent à mesure que la table grossit. J’ai reproduit un test simple sur PostgreSQL 16 avec 10 millions de lignes, en utilisant une table dont la seule différence est la colonne id : -- Table UUID v4 CREATE TABLE events_v4 ( id UUID DEFAULT gen_random_uuid () PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); -- Table UUID v7 (généré côté application, voir plus bas) CREATE TABLE events_v7 ( id UUID PRIMARY KEY , payload JSONB , created_at TIMESTAMPTZ DEFAULT now () ); Après insertion, voici les mesures : Type de clé Taille de l’index Fragmentation Latence moyenne d’insertion BIGINT ~214 Mo 0 % ~0,8 ms/ligne UUID v4 ~428 Mo (2×) 99 % ~4,8 ms/ligne UUID v7 ~428 Mo (2×) ~2 % ~1,1 ms/ligne Ce qui frappe, c’est la fragmentation quasi nulle de l’UUID v7. L’index reste compact et les insertions sont presque aussi rapides qu’avec un BIGSERIAL. L’UUID v4, lui, est plus de quatre fois plus lent à l’insertion sur ce volume. UUID v7 :

Environment Variables in Node.js: The Complete Guide (2026) Environment variables are the standard way to configure apps across environments. Here's how to use them correctly. What and Why What: Key-value pairs set outside your application code Where: OS environment, .env files, CI/CD config, container orchestration Why: → Separate config from code (12-Factor App methodology) → Same code runs everywhere (dev, staging, production) → Secrets never committed to git → Easy to change behavior without redeploying Reading Env Vars in Node.js // Method 1: process.env (built-in, always available) const port = process . env . PORT || 3000 ; const dbUrl = process . env . DATABASE_URL ; const apiKey = process . env . API_KEY ; // ⚠️ process.env values are ALWAYS strings! const timeout = parseInt ( process . env . TIMEOUT , 10 ) || 5000 ; const debug = process . env . DEBUG === ' true ' ; const maxRetries = Number ( process . env . MAX_RETRIES || ' 3 ' ); // Method 2: dotenv (most popular approach) // npm install dotenv import ' dotenv/config ' ; // Loads .env into process.env automatically // Now process.env has all your .env variables! // Or explicit load: import dotenv from ' dotenv ' ; dotenv . config ({ path : ' .env.local ' }); // Custom file path // Method 3: env-cmd (for package.json scripts) // "dev": "env-cmd -f .env.dev node server.js" // "prod": "env-cmd -f .env.prod node server.js" // Method 4: tenv / enve (type-safe alternatives) import { env } from ' tenv ' ; const port = env . number ( ' PORT ' , 3000 ); // Type-safe with default const dbUrl = env . string ( ' DATABASE_URL ' ); // Required, throws if missing const debug = env . bool ( ' DEBUG ' , false ); // Boolean parsing The .env File Ecosystem # .env (committed to git with defaults) NODE_ENV = development PORT = 3000 LOG_LEVEL = debug # .env.local (NOT committed! Gitignored) # Contains local overrides and secrets DATABASE_URL = postgresql://localhost/myapp_dev API_KEY = sk_test_local_key JWT_SECRET = local-de

Most software projects focus on producing outputs. Most AI projects focus on producing predictions. But real organizations don't operate on outputs or predictions alone. They operate on decisions. A decision has consequences. A decision creates risk. A decision consumes resources. A decision changes the future state of a system. Over the last few months, I've been studying and building systems around a simple question: How can we make decisions more explainable, auditable, and repeatable? This led me toward concepts such as: event-driven architectures decision logging risk evaluation pipelines audit trails feedback loops operational intelligence systems Instead of asking: "Can we predict what will happen?" I'm becoming more interested in asking: "Can we explain why a decision was made?" and "Can we reproduce that decision six months later?" Current areas I'm exploring: Financial decision systems Risk infrastructure Event-driven architectures Blockchain compliance workflows Operational intelligence platforms One of the projects I'm currently building is an Event-Driven Decision Logging System (EDDL), designed to explore how organizations can record, audit, and replay critical decisions over time. Still learning. Still building. Still refining my understanding of how complex systems operate under uncertainty. Looking forward to sharing the journey here. systemsdesign #architecture #backend #fintech #softwareengineering #eventdriven #riskmanagement

Advanced JSON Schema Validator Techniques for Power Users Once you're comfortable with basic validation, these advanced techniques will help you handle complex validation scenarios and integrate validation deeply into your systems. 1. Conditional Validation with if/then/else The most powerful feature in modern JSON Schema is conditional validation. Use it to enforce different rules based on the data itself: { "type" : "object" , "properties" : { "type" : { "type" : "string" , "enum" : [ "individual" , "business" ] }, "taxId" : { "type" : "string" }, "businessName" : { "type" : "string" } }, "allOf" : [ { "if" : { "properties" : { "type" : { "const" : "business" } } }, "then" : { "required" : [ "taxId" , "businessName" ] }, "else" : { "properties" : { "taxId" : { "not" : {} }, "businessName" : { "not" : {} } } } } ] } This schema makes taxId and businessName required only when type is "business". For individual accounts, those fields must not be present. 2. Custom Error Messages with Error Message Extension Enhance validation with user-friendly error messages that guide users toward correct input: { "type" : "object" , "properties" : { "password" : { "type" : "string" , "minLength" : 8 , "pattern" : "^(?=.*[A-Z])(?=.*[0-9])" , "errorMessage" : { "minLength" : "Password must be at least 8 characters" , "pattern" : "Password must contain at least one uppercase letter and one number" } } } } While not part of the core JSON Schema spec, many validators (including AJV) support the errorMessage keyword for better user-facing error reporting. 3. Schema Composition with allOf, anyOf, and oneOf Combine multiple schemas to create sophisticated validation rules: { "allOf" : [ { "$ref" : "#/$defs/baseUser" }, { "$ref" : "#/$defs/withTimestamp" }, { "if" : { "properties" : { "role" : { "const" : "admin" } } }, "then" : { "$ref" : "#/$defs/adminPrivileges" } } ] } allOf: Data must match ALL sub-schemas (intersection) anyOf: Data must match AT LEAST ONE sub-schema (union) oneOf: Da

As a backend engineer, I have worked on many services where the hard part was not only writing the code. The hard part was keeping the project clean, reproducible, easy to build, and safe to maintain as the team and codebase grew. In Go, a big part of that discipline comes from understanding Go Modules . At first, Go Modules may look simple: a go.mod file, a go.sum file, and a few commands like go mod init and go mod tidy . But in real projects, these small tools decide how your service builds in CI, how your dependencies are verified, how private repositories are handled, and how other developers can use your package. In this article, I want to explain Go Modules in a practical way, from the mindset of someone building production backend systems. We will cover: What Go Modules are and why Go does not work like NPM or Pip How go mod init , go mod tidy , and go mod vendor actually help How I think about Go project structure without over-engineering How to publish your own Go package A few production tips that matter in real teams What Are Go Modules? A Go module is a versioned collection of Go packages. In simple words, it is the boundary of your project. It tells Go: what your project is called which Go version it targets which dependencies it needs which versions of those dependencies should be used A Go module is defined by the go.mod file. Before Go Modules, Go projects were commonly managed inside GOPATH . That worked, but it created friction around dependency versions and project location. Go Modules solved that by making dependency management explicit and project-based. Today, when I start a serious Go project, one of the first things I do is initialize a module. Why Go Packages Feel Different From NPM or Pip If you come from JavaScript or Python, Go package management may feel a little strange at first. In Node.js, packages are usually published to NPM . In Python, packages are usually published to PyPI . Go is different. Go uses the module path as an import

A lot of AI apps die in the same place. Not during the prototype phase. Not while testing...

The argument sounds reasonable: fewer lines of code mean fewer bugs. Simpler to review, easier to...