标签：#json

Cartography is all about taking the real world and turning it into a picture that people can understand. It’s the process of deciding: what places to show, what details to keep or remove, what colors and symbols to use, how to draw the round Earth on a flat screen or paper Cartography mixes geography (knowing where things are), design (making the map clear and beautiful), and math (flattening the Earth using projections). Every map you see—Google Maps, airport maps, weather maps, D3.js visualizations—is a result of cartography. Line simplification alogorithms are tools used in cartography to reduce the number of points in a geographic shape while keeping the shape recognizable. 🌍 Why do we need line simplification? Real geographic shapes—coastlines, borders, rivers, airport boundaries—are extremely detailed. If you zoom in enough, you can always find more bumps, curves, and tiny wiggles. This is what Lewis Fry Richardson discovered: The more precisely you measure a coastline, the longer it becomes.Because coastlines have infinite detail.But your computer screen does not have infinite detail. It has pixels. If you try to draw a super-detailed coastline - the file becomes huge > the map loads slowly > D3.js rendering becomes slow > zooming becomes laggy > the map looks messy when zoomed out. This is why we need line simplification algorithms. 🎯 What do line simplification algorithms do? They remove unnecessary points from a shape while keeping the overall form. Think of it like: drawing a coastline with fewer squiggles. smoothing a jagged boundary reducing a 10,000‑point shape to 1,000 points. making the map faster and cleaner. The goal is: Keep the important shape, remove the tiny details. 🧩 Why this matters for zoomable maps Zoomable maps (like D3 zoom or Leaflet zoom) need multiple resolutions: When zoomed out → simple shapes When zoomed in → detailed shapes If you use only high‑resolution data: the map becomes slow, too many points are drawn, the user sees clutter

Losing PostgreSQL Gains? Blame Inline JSONB!! PostgreSQL's jsonb is a favorite among developers for its flexibility - but it hides a dark side. When used carelessly, especially in-line within rows under 2KB, it can silently destroy performance, even if you're using indexes. Here's why. 🔍 The Hidden Cost of JSONB (Inline Storage) PostgreSQL stores table rows in 8KB pages, packing as many tuples as possible. For a typical row with 10–12 columns, and small text/integers, 40–100 rows can easily fit per page. Typically row count = Page Size(8kb) / row size + row metadata (30-50 bytes approx.) But the game changes when you add jsonb. Example CREATE TABLE events ( id serial PRIMARY KEY, user_id int, action text, metadata jsonb ); Suppose metadata which is a jsonb column contains: { "ip": "127.0.0.1", "device": "Android", "country": "IN" } This JSON might be just 100–500 bytes, so PostgreSQL stores it in-line inside the same page (no TOASTing). Result Each row size jumps from ~80 bytes → ~200–400 bytes Row count per page drops from 100 → 20–40 Index scan still needs to read each page for matching rows More pages = more I/O, slower performance 🔢 Real Benchmark Insight Performance comparisonEven with a GIN or B-tree index on the JSONB column, PostgreSQL still needs to scan all matching pages to retrieve the full tuple. 🧠 Why Index Doesn't Save You Say you index a JSONB key like: CREATE INDEX ON events ((metadata->>'ip')); And query: SELECT * FROM events WHERE metadata->>'ip' = '127.0.0.1'; PostgreSQL will: Use the index to find matching tuples Still need to fetch the row from disk Because JSONB is in-line, many pages are touched More page fetches = more IO = slower queries 🩹 What You Can Do ✅ Force TOAST: Add padding to make JSONB exceed 2KB: UPDATE events SET metadata = metadata || jsonb_build_object('padding', repeat('x', 2000)); ✅ Split into separate table: If JSONB is rarely queried ✅ Stick to well defined schema and avoid using jsonb unless absolutely necessary. 🧾 TL;DR

You've got a JSON API response and you want TypeScript interfaces for it. Here's how to generate them fast — and where the auto-generators quietly get it wrong. The fast path Paste your JSON, get interfaces: { "id" : 1 , "name" : "Ada" , "roles" : [ "admin" ], "profile" : { "active" : true } } → interface Root { id : number ; name : string ; roles : string []; profile : Profile ; } interface Profile { active : boolean ; } jsonviewertool.com/json-to-typescript does this in the browser (client-side), nesting objects into their own interfaces. Where generators trip up A generator only sees the ONE sample you give it, which causes predictable gaps: Nullable fields. If your sample has "avatar": null , the generator infers null — but the real type is probably string | null . Feed it a populated sample, or fix it by hand. Empty arrays. "tags": [] infers any[] — the element type is unknowable from an empty array. Optional fields. A field missing from your sample won't appear at all. If the API sometimes omits middleName , mark it middleName?: string . Unions. A status that's "active" in your sample becomes string , not the literal union "active" | "banned" | "pending" . Narrow it manually for the safety. Numbers that are really enums or IDs. "currency": 840 types as number ; you may want an enum or branded type. When to use a schema instead If the JSON has a JSON Schema or OpenAPI spec, generate types from that ( json-schema-to-typescript , openapi-typescript ) — it encodes nullability, optionality, and unions the raw sample can't. Sample-based generation is for quick throwaway typing; schema-based is for anything you'll maintain. Rule of thumb Generate from a sample to skip the boilerplate, then read every field — the generator gives you a draft, not a contract. Nullability and optional fields are where the runtime bugs hide.

Software spends a surprising amount of its life just moving structured data around: an API returns JSON, a config file is written in YAML or TOML, a report is exported as CSV, a spreadsheet wants tabular rows. These formats are not interchangeable — each was designed for a particular job, and using the wrong one creates friction. Knowing the strengths of each makes you faster and saves you from a category of frustrating bugs. JSON: the lingua franca of APIs JSON (JavaScript Object Notation) is the default for data exchange between systems, especially web APIs. It represents nested objects and arrays cleanly, every programming language can parse it, and its rules are strict enough to be unambiguous. That strictness is also its main friction for humans: no comments are allowed, every string needs double quotes, and a single trailing comma makes the whole document invalid. JSON is excellent for machine-to-machine communication and data storage; it is merely tolerable for files humans have to edit by hand. YAML: configuration humans edit YAML was designed to be readable and writable by people. It uses indentation instead of braces, supports comments, and drops most of the punctuation that makes JSON noisy. This makes it popular for configuration in tools like CI pipelines and container orchestration. Its strength is also its danger: because structure is defined by indentation, a single misplaced space can silently change the meaning of your file or break it entirely. YAML also has surprising type-coercion quirks (the classic example: the word "no" being read as the boolean false ). Use YAML for human-edited configuration, but validate it. TOML: configuration that stays unambiguous TOML (Tom's Obvious Minimal Language) aims for YAML's readability without YAML's ambiguity. It uses explicit, INI-like sections and clear key-value pairs, supports comments, and has unambiguous typing. It is less prone to the silent indentation mistakes that plague YAML, which is why a number

Here's a bug that looks impossible until you understand Gson. You have this data class in your Pokedex app: data class PokemonStat ( @SerializedName ( "base_stat" ) val baseStat : Int , val stat : StatInfo ) A teammate cleans up the code and deletes what looks like a redundant line: data class PokemonStat ( val baseStat : Int , // @SerializedName removed val stat : StatInfo ) It compiles . No red errors. He runs the app — and every Pokemon's baseStat is 0 . Not the real 45 or 49. Just 0 . Everywhere. And Gson never threw an error, never logged a warning, never said a single word. If you can explain why this happens, you understand Gson better than most juniors. This is also a favorite interview question. Let's break it fully. What Gson actually does When the JSON comes back from the server, Gson goes key by key: Read a key from the JSON — say base_stat . Look for a Kotlin property with the exact same name . Found it? Pour the value in. Not found? Leave that property at its default value and move on. That's the entire matching game — exact name match, or nothing. Now look at the "cleaned up" class. The JSON key is base_stat . The Kotlin property is baseStat . Those are not the same string . Gson looks for a property called base_stat , doesn't find one, shrugs, and leaves baseStat at its default. The default for an Int is 0 . That's your bug. @SerializedName("base_stat") was never redundant. It was the sticky note telling Gson: "this property is called baseStat in Kotlin, but look for base_stat in the JSON." Delete the note, and Gson stops matching. Two ways to fix it: Rename the property to base_stat — works, but breaks Kotlin's camelCase convention. Put @SerializedName("base_stat") back — keeps the clean name and matches. This is the right one. But why 0 ? Why not a crash? This is the part that surprised me. A missing field feels like it should be an error. It isn't. Gson treats a missing key as allowed . It builds your object, fills the fields it found, and leaves

Hi everyone, I want to share textparser, a high-performance, lightweight text parsing and AST generation library written in pure C. 💡 The Core Idea & Why It's DifferentTraditional parser generators (like Flex/Bison) come with a steep learning curve and rigid code generation. On the other end, hand-writing recursive descent parsers or state machines becomes an unmaintainable mess as your language grows.textparser bridges this gap using a hybrid JSON + Python + C workflow:You define your tokens, syntax patterns, and color styles in a clean, human-readable JSON grammar file. A lightweight Python compiler tool processes the JSON, runs optimizations, and emits a dense, static C header array.The C runtime engine loads these pre-compiled arrays instantly. It processes raw text strings into an abstract token tree (textparser_token_item) using a highly optimized regular expression engine (crpe2).By offloading the grammar overhead and heavy state-machine parsing logic to the Python build step, the actual runtime C library stays incredibly lean, memory-efficient, and fast. 🚀 Features At A Glance30+ Languages Out-of-the-Box: Includes ready-to-use JSON grammars for C, C++, Rust, Python, JavaScript, HTML, SQL, and dozens more.Rich Token Metadata: Every parsed token tracks exact code coordinates, structural flags, and custom syntax styling options.Zero Bloat: Ideal for terminal text editors, syntax highlighters, custom linters, and lightweight static analysis tools where bringing in a massive compiler front-end is overkill.

Working With Massive JSON Responses Without Losing Performance Every developer eventually encounters it. You make an API request expecting a few hundred objects, and instead receive a response that's tens—or even hundreds—of megabytes. Suddenly your browser freezes, your editor becomes sluggish, and your application consumes gigabytes of memory. Large JSON responses aren't unusual anymore. Analytics platforms, cloud providers, search engines, AI services, ecommerce catalogs, IoT systems, and data export endpoints routinely generate enormous payloads. The good news is that handling massive JSON efficiently is mostly about choosing the right techniques. This guide covers the best practices that help you inspect, process, and optimize large JSON datasets without overwhelming your tools or your users. Understand Why Large JSON Is Expensive Before optimizing, it's helpful to know where the cost comes from. When an application receives JSON, it usually goes through several stages: Download the response. Store it as a string. Parse it into objects. Allocate memory for every property. Traverse the resulting object graph. For a 100 MB JSON file, peak memory usage can easily exceed 300 MB because both the raw string and the parsed objects coexist temporarily. This explains why applications often run out of memory long before reaching the actual file size. Don't Pretty-Print Gigantic Responses Immediately Pretty-printing is useful—but formatting a huge document all at once can consume significant CPU time and memory. Instead: inspect only the sections you need collapse large objects expand nodes on demand search before formatting If you need to examine a large payload in the browser, using a dedicated formatter designed for large documents can make navigation much easier. Tools like JSON Formatter allow you to validate, format, collapse, and inspect JSON without manually editing thousands of lines. Stream Instead of Loading Everything One of the biggest mistakes is reading an

You change one value in a JSON config, run git diff , and get a wall of red and green — because a formatter reflowed the file, or the serializer reordered the keys, or the indentation shifted by two spaces. The one thing you actually changed is buried in there somewhere. Good luck finding it in review. The problem is that git diff and diff work on lines . JSON isn't lines — it's a tree. So I built jdelta : it compares the data , ignores key order and whitespace entirely, and tells you exactly which values changed, addressed by path. Zero dependencies, no network. What it looks like $ jdelta config.before.json config.after.json Added (1) + features.darkMode true Changed (2) ~ auth.required true → false ~ server.port 8080 → 3000 +1 -0 ~2 auth.required flipped to false and you can see it instantly — no scrolling past 200 lines of reindented braces. Why not git diff / diff ? Because they diff text. Run a formatter, sort your keys, change two spaces of indent, and a line differ lights up the whole file while reporting zero semantic change. jdelta parses both sides and walks the trees: Object keys show as user.profile.age ; array elements as items[2].price . Odd keys fall back to quoted brackets ( ["order-id"] ). A key on only one side is added / removed ; a key on both with a different value is changed . A type change ( number → string , object → array ) is one changed entry tagged with the kinds — not a confusing add + remove. Reordered keys and whitespace produce nothing , because they aren't data changes. Great for reviewing config changes, API-response snapshots, tsconfig / settings.json , lockfile-adjacent files, and test fixtures. In scripts and CI jdelta a.json b.json --json # machine-readable: {added, removed, changed, summary} jdelta a.json b.json --quiet # just the +a -r ~c line jdelta a.json b.json --exit-code # exit 1 if they differ — gate a pipeline on it Install npx jdelta a.json b.json # Node build (npm) pip install jdelta # Python build — same behavior Tw

Working with complex JSON payloads can quickly become a nightmare. You end up chaining .map() , .filter() , and .reduce() calls across multiple lines just to pull out a few nested values. Add optional chaining to avoid crashes and the code becomes nearly unreadable. There is a cleaner way - JSONata . It is a compact, purpose-built query and transformation language for JSON data. Think of it as XPath for XML, but designed from the ground up to work with JSON objects and arrays. What is JSONata? JSONata is an open-source project originally created by Andrew Coleman at IBM. It gives developers a declarative syntax to extract and reshape JSON data without writing procedural JavaScript loops. Where vanilla JS might take 15 lines, a JSONata expression often takes one. It is available as an npm package and integrates naturally into Node.js and TypeScript projects. Simple Path Navigation The foundation of JSONata is its dot-notation path traversal. Given a nested JSON object, you simply trace the path to the value you need: customer.address.city This returns the city value without any need for null checks or defensive coding. JSONata handles missing properties gracefully by returning undefined rather than throwing errors. Automatic Array Mapping When JSONata encounters an array during path traversal, it automatically maps across all items. There is no need to write an explicit .map() call: customer.orders.product This returns an array of all product names from every order in one clean expression. Inline Filtering You can filter arrays directly using bracket notation with a condition: customer.orders[price > 1000].product This returns only the products from orders where the price exceeds 1000. No .filter() callback required. Built-in Aggregation Functions JSONata ships with a solid set of built-in functions for math, strings, and arrays. Aggregating a set of values is straightforward: $sum(customer.orders.price) Other useful functions include $count() , $average() , $string(

I spent yesterday building purejq , a pure-Python implementation of jq. I expected it to be the slow-but-portable option. Then I benchmarked it against the jq package on PyPI (the C bindings everyone uses to run jq from Python) and got this, on a 100k-object array, in-process: workload purejq jq PyPI (C bindings) field-access stream 9 ms 368 ms filter + count 55 ms 442 ms map + aggregate 18 ms 444 ms group_by 112 ms 704 ms transform + sort 136 ms 899 ms Pure Python, 7-40x faster than the C extension. That number looked wrong to me too, so before publishing anything I made the benchmark script verify every output against the actual jq binary first ( tools/bench.py --verify ), re-ran everything as median-of-7, and gave the bindings their best-case API. The gap is real. Here's why. The serialization tax The C bindings wrap real jq, and real jq only speaks JSON. So every call does this: your dicts -> JSON text -> C parser -> jq evaluates -> JSON text -> dicts That round trip costs about 350-450 ms for 100k small objects on my machine, before any actual filtering happens. You can see it in the numbers: even a trivial field access pays the same ~400 ms floor as a group_by. purejq skips the trip entirely. It compiles the jq program once into Python closures and walks your dicts and lists directly: import purejq prog = purejq . compile ( " group_by(.team) | map({team: .[0].team, n: length}) " ) prog . first ( data ) # operates on your objects, no serialization The lesson generalizes beyond jq: when you embed a C library that has its own data model, the marshaling boundary is often more expensive than the work. An interpreter written in your language gets to skip the boundary, and that can buy back an order of magnitude. Surprise number two: the CLI beats the jq binary on big files This one I really didn't expect. End to end on a 93 MB file (1M objects), parse + filter + output: workload purejq CLI jq 1.8.1 binary single lookup 0.51 s 1.68 s filter + count 1.08 s 1.96 s grou

Data Visualizer Live Demo 🌐 Try it live: https://datavisualizer.urlmediainspector.dev/ What It Is Data Visualizer is a visual workspace where developers can explore, transform, execute, and understand data using interconnected nodes on an infinite canvas. Instead of jumping between API tools, JSON viewers, spreadsheets, code editors, schema inspectors, and visualization platforms, everything happens inside a single interactive environment. Each node represents a specific capability and can be connected together to create powerful workflows for data exploration, processing, automation, and analysis. Key Features Infinite Visual Workspace Work on an unlimited canvas where data, code, APIs, documents, and visualizations can be organized as connected workflows instead of isolated files and tabs. API Exploration Connect to APIs, inspect responses, analyze payloads, and build reusable visual pipelines for data processing. JSON & YAML Visualization Navigate deeply nested structures through interactive visual representations that make complex data easier to understand. JavaScript & TypeScript Execution Run JavaScript and TypeScript directly inside workflow nodes to transform, filter, and manipulate data in real time. Browser-Based Python Runtime Execute real Python entirely in the browser without requiring local installations or external servers. CSV & Dataset Analysis Import and explore tabular data visually, making it easier to inspect records, understand relationships, and process large datasets. Schema Exploration Visualize schemas and nested structures to quickly understand how data is organized and connected. PDF, Image & Video Support Work with documents and media assets directly inside the workspace without constantly switching applications. Visual Data Pipelines Create workflows by connecting nodes together, allowing data to flow naturally between APIs, transformations, code execution, schemas, and visualizations. Interactive Data Transformation Modify and reshape

If you've ever hit Unexpected token in JSON at position 42 or Unterminated string , there's a good chance an unescaped character broke your payload. JSON is strict about what's allowed inside a string, and the fix is almost always escaping . Here's the practical version. What does escaping a JSON string mean? A JSON string is wrapped in double quotes. Any character that would confuse the parser must be replaced with a backslash escape sequence. Escaping doesn't change the meaning of your text — it just makes the string valid JSON so parsers can read it. Unescaping is the reverse: turning those sequences back into readable characters (handy when you copy a value out of logs or an API response). The characters you must escape JSON defines exactly seven characters that must be escaped inside a string: Character Escaped as Double quote " \" Backslash \ \\ Newline \n Carriage return \r Tab \t Backspace \b Form feed \f The forward slash / may optionally be escaped as \/ , but it isn't required. Unicode can be written as \uXXXX (four hex digits). JSON escape examples Double quotes — He said "hello" becomes: "He said \" hello \" " Backslashes (Windows paths) — C:\temp\file.txt becomes: "C: \\ temp \\ file.txt" Newlines and tabs — a two-line, tabbed string becomes: "Line 1 \n Line 2 \t Tabbed" How to escape and unescape JSON in code In production you rarely escape by hand — every language has it built in. JavaScript const escaped = JSON . stringify ( text ); // escape const back = JSON . parse ( escaped ); // unescape Python import json escaped = json . dumps ( text ) # escape back = json . loads ( escaped ) # unescape Java (Jackson) ObjectMapper mapper = new ObjectMapper (); String escaped = mapper . writeValueAsString ( text ); String back = mapper . readValue ( escaped , String . class ); C# (.NET) using System.Text.Json ; string escaped = JsonSerializer . Serialize ( text ); string back = JsonSerializer . Deserialize < string >( escaped ); Common escaping mistakes (and f

Previously, we learned how to send and receive data in Go. Now, we will combine those concepts and build a simple CRUD API. CRUD stands for: C reate R ead U pdate D elete These four operations form the foundation of most backend applications. In this tutorial, we will build a simple task API in Go using only the standar library. By the end, you will understand: how CRUD APIs work how to handle multiple HTTP methods how to store data in memory how to send and receive JSON data how backend APIs manage resources Prerequisites To follow along, you should have: Go installed basic familiarity with Go syntax understanding of the net/http package basic understanding of JSON handling You can confirm if Go is installed by running: go version Step 1 — Create the Project Create a new folder for the project: mkdir go-crud-api cd go-crud-api Now initialize a Go module: go mod init go-crud-api This creates a go.mod file for managing project dependencies. Step 2 — Create the Server File Create a file called main.go . Your project structure should now look like this: go-crud-api/ ├─ go.mod └─ main.go Step 3 — Write the CRUD API Open main.go and add the following code: package main import ( "encoding/json" "net/http" ) type Task struct { ID int `json:"id"` Title string `json:"title"` } var tasks [] Task func tasksHandler ( w http . ResponseWriter , r * http . Request ) { w . Header () . Set ( "Content-Type" , "application/json" ) switch r . Method { case http . MethodGet : json . NewEncoder ( w ) . Encode ( tasks ) case http . MethodPost : var task Task err := json . NewDecoder ( r . Body ) . Decode ( & task ) if err != nil { http . Error ( w , "Invalid JSON" , http . StatusBadRequest ) return } tasks = append ( tasks , task ) json . NewEncoder ( w ) . Encode ( task ) default : http . Error ( w , "Method not allowed" , http . StatusMethodNotAllowed ) } } func main () { http . HandleFunc ( "/tasks" , tasksHandler ) http . ListenAndServe ( ":8080" , nil ) } Now let's unpack what is hap