AI 资讯
The Token Bucket Algorithm: Build Server-Side API Rate Limiting in ~40 Lines
The Token Bucket Algorithm: Server-Side API Rate Limiting in ~40 Lines Plenty of tutorials teach you how to survive someone else's rate limit with retries and backoff. Far fewer show you how to build one. If you run an API, you need rate limiting on your side too — to protect your database from a runaway client, keep one noisy tenant from starving everyone else, and give abusive traffic a polite 429 instead of a melted server. The cleanest algorithm for the job is the token bucket . Let's implement it from scratch, then make it production-ready. How token bucket works Picture a bucket that holds up to capacity tokens. Every request removes one token. The bucket refills at a steady refillRate (tokens per second), up to its cap. If a request arrives and the bucket is empty, it's rejected. This gives you two useful properties at once: A sustained rate — the long-run average, set by refillRate . A burst allowance — clients can spend the whole bucket at once, set by capacity . That burst tolerance is why token bucket feels fair. A user who's been quiet for a minute can fire off a batch of requests without being punished for it. A minimal implementation Here's a self-contained bucket in JavaScript. No dependencies, no timers — we compute refill lazily based on elapsed time, which is both simpler and more accurate than a background interval. class TokenBucket { constructor ( capacity , refillRatePerSec ) { this . capacity = capacity ; this . refillRate = refillRatePerSec ; this . tokens = capacity ; this . lastRefill = Date . now (); } _refill () { const now = Date . now (); const elapsedSec = ( now - this . lastRefill ) / 1000 ; this . tokens = Math . min ( this . capacity , this . tokens + elapsedSec * this . refillRate ); this . lastRefill = now ; } take ( cost = 1 ) { this . _refill (); if ( this . tokens >= cost ) { this . tokens -= cost ; return { ok : true , remaining : Math . floor ( this . tokens ) }; } const deficit = cost - this . tokens ; const retryAfter = Mat
AI 资讯
Web Scraping with Python in 2026: Best Libraries and Anti-Bot Strategies
Web Scraping with Python in 2026: Best Libraries and Anti-Bot Strategies Web scraping in 2026 looks very different from 2020. Sites are smarter, anti-bot systems are more aggressive, and the legal landscape has evolved. Here's what actually works now. The 2026 Scraping Landscape Challenge 2020 Solution 2026 Solution Bot detection Rotate User-Agent Fingerprint randomization + residential proxies CAPTCHAs Manual solving Turnstile/hCaptcha solvers JavaScript rendering Selenium Playwright (faster, more reliable) Rate limiting Sleep between requests Adaptive pacing + request signing IP blocking VPN rotation Residential proxy pools Best Libraries in 2026 1. Playwright (Best for JS-heavy sites) from playwright.sync_api import sync_playwright def scrape_with_playwright ( url ): with sync_playwright () as p : browser = p . chromium . launch ( headless = True ) page = browser . new_page () page . goto ( url , wait_until = " networkidle " ) data = page . query_selector_all ( " .job-item " ) results = [] for item in data : title = item . query_selector ( " h2 " ). text_content () results . append ( title ) browser . close () return results 2. httpx + Selectolax (Fast, no JS needed) import httpx from selectolax.parser import HTMLParser def scrape_static ( url ): resp = httpx . get ( url , headers = { " User-Agent " : " Mozilla/5.0 " }) tree = HTMLParser ( resp . text ) for node in tree . css ( " .listing " ): print ( node . text ()) 3. API-First Approach (Always check first!) Many sites have hidden or public APIs that make scraping unnecessary: url = " https://www.freelancer.com/api/projects/0.1/projects/active/?query=python " data = httpx . get ( url ). json () Anti-Bot Strategies That Work 1. Request Fingerprint Randomization import random def get_random_headers (): browsers = [ " Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " , " Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 " , ] return { " User-Agent " : random . choice ( browsers ), " A
AI 资讯
How We Translate 300-Page Books Using Claude Without Hitting Token Limits
Breaking long documents into overlapping chunks, preserving context, and reassembling with FastAPI At LectuLibre, we’ve built an AI‑powered platform that translates entire books—EPUBs and PDFs—using large language models. When we first hooked up Claude’s API, we naively fed it a 300‑page PDF in one request. It failed immediately. Claude 3 Opus has a 200K token window, but a 300‑page book can easily run to 300K tokens or more. Even if we squeezed it in, the output would be truncated and the quality would degrade at the extremes of the context window. So we faced a classic long‑document problem: how do you translate a book that’s larger than the model’s context window? Here’s the real approach we ended up with, the code we wrote, and the lessons we learned. The Problem: Token Limits Are Real Claude 3 Opus and Haiku models (and most LLMs) have a maximum context length—200,000 tokens for Opus. A token is roughly ¾ of a word. A 300‑page novel with ~75,000 words translates to about 100K tokens, so it should fit, right? But translations from English to Spanish can expand by 15–20%, and the prompt instructions, system message, and the user message itself all eat into that budget. Plus, we needed to send the entire source text in every call to give the model full context. That’s not feasible. We could have tried a simple split: cut the book at arbitrary page boundaries and translate piecemeal. That fails spectacularly. Narrative breaks mid‑sentence, and phrases like “the previous chapter” lose their referents. We needed a more intelligent chunking strategy. Our Approach: Sliding Window with Overlapping Paragraphs We settled on a sliding window chunking algorithm based on paragraphs, with a generous overlap. Here’s the idea: Split the source text into paragraphs (using \n\n ). Build chunks of max_chunk_tokens (we used 180,000 to keep a safety margin), adding paragraphs one by one and counting tokens with tiktoken . When the chunk exceeds the limit, we start a new chunk but we
AI 资讯
Maintaining WordPress sites behind HTTP Basic auth — Playwright, urllib, and encrypted credentials
It's pretty common to throw a layer of HTTP Basic auth on a WordPress site: a staging environment before launch, an internal test instance only employees should see, or any environment that wants an extra gate before the WordPress login screen itself. From a maintenance-tool point of view, this setup creates a peculiar "half-working, half-broken" asymmetry. The SSH/WP-CLI side runs fine. But everything HTTP-based — visual checks, thumbnail generation, browser-based fallback updates — hits 401 and dies. This post walks through how we resolved that asymmetry. What was breaking — two parallel paths, both blocked A maintenance tool actually touches a Basic-auth-protected site through two distinct paths: Playwright path : visual checks, thumbnail capture, browser fallback updates when SSH isn't available. browser.new_context() → navigation → screenshot urllib path : HTTP status checks (pre/post-update 200/5xx/4xx monitoring, rollback decisions) With no credentials, both paths see a 401 Unauthorized from the protected site. The Playwright symptom is the obvious one: the screenshot you save is the browser's "authentication required" dialog. The thumbnail grid fills with dark auth-prompt images, and you start wondering whether anything actually works. The urllib symptom is much worse — it silently breaks rollback decisions . A 401 baseline followed by another 401 after the update looks like "nothing changed = healthy." Real failures can hide behind that match, and the rollback that should have fired never does. The design — consolidate credential extraction into one helper When the same credentials need to flow through multiple code paths, picking them out of the site dict separately at each call site invites format-mismatch and missed-update bugs. So the first thing we did was build a small core/basic_auth_utils.py module that owns every form of credential extraction . # core/basic_auth_utils.py def get_basic_auth_tuple ( site ): """ Return (user, password), or None if not
AI 资讯
Build a Minimal WebMCP Agent with Playwright and Gemini
WebMCP lets a web page expose tools that AI agents can discover and execute inside the browser. That...
AI 资讯
Learn DynamoDB by running it - accesspatterns.dev
I've been building on DynamoDB since around 2015, and these days I build tools for it: dynoxide , a DynamoDB engine, and Nubo , a native client. So I'm not neutral about it. It's the first database I reach for, and with reason - the operational overhead is close to nil, no connections to pool or instances to size, and it holds the same single-digit-millisecond reads whether the table has a thousand items or a billion. The data modelling is a craft, and a satisfying one. It's also one of the harder databases to learn, and that's the part I keep coming back to. DynamoDB punishes the instincts you bring from SQL. You don't normalise and join at read time; you work out the questions your app will ask first, and shape the data around those access patterns, until one table answers all of them. It's a real shift in how you think, and it's where a lot of people bounce off - it feels backwards right up until it clicks. The people who teach it best all teach it the same way. Alex DeBrie's The DynamoDB Book , the arc.codes team's examples, Rick Houlihan's re:Invent talks - the legendary ones, where he models half a dozen access patterns onto a single table at a hundred miles an hour - none of them hand you rules to memorise. They show you patterns and make you run them. I learned a lot of my DynamoDB from all three, and it stuck because I was building as I went. That last part is the bit that's hard to come by on your own. Reading about an access pattern and having it in your fingers are different things, and to run one - build the table, write the items, fire the query and see what comes back - you need an AWS account, or a local emulator installed and seeded. Enough friction that plenty of people read about single-table design without ever building one. There was a second thing pulling the same way. dynoxide had learned to run in the browser only last month, compiled to WebAssembly with no server behind it, and it was a preview I didn't fully trust. What it needed was a real
AI 资讯
Building a Denim Collection API: A Practical Guide to Handling Product Variants
If you've ever worked with e-commerce data, you know that "a pair of jeans" is never just one product. A single style might come in 5 washes, 8 sizes, and 3 inseam lengths. That's 120 potential SKUs. Handling this correctly in an API can be tricky, so let me share a pattern I've used for structuring product variants. The core problem is balancing flexibility with performance. You want customers to filter by size, color, and fit without making dozens of API calls. Here's a simple but effective approach using a normalized database schema with a flat query layer: -- Products table (the "parent") CREATE TABLE products ( id UUID PRIMARY KEY , name TEXT NOT NULL , description TEXT , base_price DECIMAL ( 10 , 2 ), category TEXT ); -- Variants table (the actual sellable items) CREATE TABLE variants ( id UUID PRIMARY KEY , product_id UUID REFERENCES products ( id ), sku TEXT UNIQUE NOT NULL , size TEXT , color TEXT , wash TEXT , inseam TEXT , price DECIMAL ( 10 , 2 ), -- can override base price stock_quantity INT , image_url TEXT ); The key insight? Keep the product metadata (description, care instructions, brand story) in the products table, but put all the sellable attributes in variants. This lets you run queries like: -- Find all size 28 jeans in "mid wash" under $80 SELECT p . name , v . color , v . wash , v . price , v . stock_quantity FROM products p JOIN variants v ON p . id = v . product_id WHERE p . category = &# 039 ; women - jeans &# 039 ; AND v . size = &# 039 ; 28 &# 039 ; AND v . wash LIKE &# 039 ; % mid %&# 039 ; AND v . price & lt ; 80 AND v . stock_quantity & gt ; 0 ORDER BY v . price ; For the frontend, I usually return a flattened structure: { "product": { "id": "abc -123 " , "name": "Classic Straight Leg Jean" , "description": "High-rise fit in stretch denim..." , "availableSizes": [ " 24 " , " 25 " , " 26 " , " 27 "
AI 资讯
Beyond ChatGPT: Understanding the Core Building Blocks of Generative AI
Most developers have experimented with ChatGPT or GitHub Copilot. But when it comes to building AI-powered applications, simply calling an LLM API isn't enough. Understanding what's happening behind the scenes helps you design systems that are scalable, reliable, and cost-effective. In this article, we'll explore four concepts every software engineer should know: tokens, embeddings, transformers, and Retrieval-Augmented Generation (RAG). 1. LLMs Think in Tokens, Not Words One of the biggest misconceptions about Large Language Models (LLMs) is that they understand words like humans do. In reality, they process tokens, which are smaller units of text. For example: Prompt: Explain dependency injection in Spring Boot. is first converted into a sequence of tokens before the model processes it. Why does this matter? API pricing is based on the number of input and output tokens. Longer prompts increase latency and cost. Every model has a maximum context window measured in tokens. When building AI applications, prompt design isn't just about getting better answers—it's also about optimizing performance and cost. 2. Transformers: The Breakthrough Behind Modern AI Before 2017, language models processed text one word at a time using architectures like RNNs and LSTMs. They struggled with long conversations because earlier context was gradually forgotten. The introduction of the Transformer architecture changed this with a mechanism called self-attention. Instead of reading text sequentially, transformers analyze the relationships between all tokens in a sentence simultaneously. Consider this sentence: "The server restarted because it ran out of memory." The model understands that "it" refers to "the server", not "memory", by assigning attention to the relevant words. This ability to capture context efficiently is what powers modern LLMs like GPT, Gemini, Claude, and Llama. 3. Embeddings Enable Semantic Search Suppose a customer searches: "How can I get my money back?" But your
AI 资讯
React useIntersectionObserver Hook: Lazy Load & Detect Visibility (2026)
React useIntersectionObserver Hook: Lazy Load & Detect Visibility (2026) You want to load an image only when it scrolls near the viewport. Or fire an analytics event the first time a card is actually seen . Or trigger "load more" when the user reaches the bottom of a list. Every one of these is the same question — is this element on screen yet? — and for years the answer was a scroll listener that fired hundreds of times a second, re-read getBoundingClientRect() on each tick, and still managed to miss the edge cases. IntersectionObserver is the browser API that answers that question correctly, asynchronously, and off the main thread. useIntersectionObserver is the hook that wires it into React without the useEffect / useRef /cleanup boilerplate — and without the leak-on-unmount and stale-closure bugs the hand-rolled version always ships. This post covers the real @reactuses/core API, the three patterns you'll actually reach for, and how to tune threshold , rootMargin , and root . SSR-safe and typed. Why Not Just Use a Scroll Listener? The old way to know whether an element was visible looked like this: listen to scroll , and on every event measure the element against the viewport. useEffect (() => { function onScroll () { const rect = el . getBoundingClientRect (); if ( rect . top < window . innerHeight ) { setVisible ( true ); } } window . addEventListener ( ' scroll ' , onScroll ); return () => window . removeEventListener ( ' scroll ' , onScroll ); }, []); This has two problems baked in. First, scroll fires on the main thread, dozens of times per second, and getBoundingClientRect() forces a synchronous layout each time — that's exactly the recipe for janky scrolling. Second, it only catches elements crossing the viewport ; the moment your scroll happens inside a container, you're re-deriving geometry by hand. IntersectionObserver flips the model. You hand the browser a target and a threshold, and it tells you — asynchronously, batched, off the scroll path — when
开发者
🚀 Build Your First Space Shooter Game with Limn Engine
🚀 Build Your First Space Shooter Game with Limn Engine A Complete Step-by-Step Tutorial for JavaScript Beginners Welcome! In this tutorial, you'll build a complete space shooter game using Limn Engine — a zero‑configuration 2D game engine that runs in your browser. What you'll build: A spaceship that moves, shoots bullets, fights waves of enemies, and keeps score. All in about 100 lines of code . By the end, you'll understand: How to create a game loop How to handle keyboard input How to detect collisions How to use particles for visual effects How to manage game state (lives, score, game over) 🎮 Want to play the finished game? Click here to play Space Shooter Live! Before We Start What You Need A text editor (VS Code, Notepad, or any code editor) A web browser (Chrome, Firefox, Edge) Limn Engine — download epic.js from limn-engine-doc.vercel.app What You Should Know Basic JavaScript (variables, functions, arrays, if-statements) How to open an HTML file in a browser No game development experience required! Step 1: The HTML Structure Every Limn Engine game starts with a simple HTML file. <!doctype html> <html> <head> <script src= "asset/epic.js" ></script> </head> <body> <script> // All your game code goes here </script> </body> </html> What's happening: <script src="asset/epic.js"> — loads the Limn Engine library Everything inside the second <script> tag is your game code Save this as game.html and open it in your browser. You should see a blank canvas with a blue gradient background. Step 2: Setting Up the Game The first thing we need is a Display — this is the engine that creates the canvas, runs the game loop, and handles input. const display = new Display (); display . perform (); // Activates performance mode (dual-canvas rendering) display . start ( 800 , 600 ); // Creates an 800×600 canvas What's happening: new Display() — creates the engine display.perform() — turns on high-performance mode display.start(800, 600) — creates a canvas 800 pixels wide and 600 p
AI 资讯
When paramiko's defaults silently get your IP banned — the look_for_keys and allow_agent trap
One day a multi-site administrator reported a strange bug: "After running the app's SSH connection test 2-3 times, my IP can't reach SSH on that server for a long while ." The errors came back as Connection refused or Connection closed by ... . The server wasn't down, and SSH from a different IP worked fine. The source IP was being temporarily banned at the server. Two external investigation reports gave the cause: server-side protection mechanisms ( fail2ban or PerSourcePenalties in OpenSSH 25+) detect short-windowed authentication failure spikes and temporarily ban the source IP. But the user had only clicked the test button 2-3 times — why were failures "spiking"? The answer turned out to be paramiko's default behavior . paramiko's default — trying many keys per connection paramiko.SSHClient.connect() defaults two options to True : client . connect ( ' host ' , pkey = my_key , # The following are True by default: # look_for_keys=True, # also try ~/.ssh/id_* files # allow_agent=True, # also try ssh-agent registered keys ) When the explicitly passed pkey fails, paramiko falls back through ssh-agent registered keys → ~/.ssh/id_* files → password auth in order. Convenient for developers with a single key. Disastrous for a multi-site administrator: The SSH agent has multiple per-site keys registered ~/.ssh/ holds several id_rsa / id_ed25519 files A single connect call ends up trying 5-10 keys in sequence That blows past the server's MaxAuthTries (default 6) on a single connection So what looked to the user like "one connection test" was being seen by the server as " a suspicious IP racking up 5-10 auth failures in a row ." Repeat that 2-3 times and the protection mechanism declares the IP "exceeded threshold" and bans it. The fix — look_for_keys=False and allow_agent=False paramiko exposes options to scope key trial. We set them explicitly in connect_kwargs : connect_kwargs = { ' pkey ' : my_key , ' look_for_keys ' : False , # don't try ~/.ssh/id_* ' allow_agent ' : F
AI 资讯
How to Fix Excel CSV Date Import Problems (US / UK Format Guide)
You export a CSV from a UK system and double-click it in US Excel — 28/05/2026 suddenly looks like May 28, or something even stranger. The file isn't corrupt. Excel is guessing your date format based on your computer's locale. This short guide covers why that happens, how to import CSV correctly, and how to batch-fix dates that are already wrong. For the full walkthrough with examples, see the official guide: How to Fix Excel CSV Date Import Problems . Why Excel breaks CSV dates A CSV is plain text. It does not store whether a value is a date or a string. When you double-click to open, Excel parses using your regional settings: US Excel: MM/DD/YYYY UK / Europe: DD/MM/YYYY Dates like 03/04/2025 are the most dangerous — both parts are ≤ 12. US Excel may read April 3; UK Excel reads March 4. Excel won't warn you. Other common traps: Dates exported as serial numbers (e.g. 44927 ) Two-digit years triggering century guesses Re-saving as CSV destroys formats a second time The right way: don't double-click the CSV Open Excel first — don't double-click the .csv file Data → Get Data from Text/CSV (older Excel: Data → From Text) Set date columns to Text if you need to preserve the original string Confirm the source region (US / UK), then convert to a single format For team data exchange, agree on ISO 8601: YYYY-MM-DD in your schema — unambiguous, sorts correctly as text, and works in JSON APIs and databases. Ambiguous value US reads as UK reads as Safe format (ISO) 03/04/2025 2025-04-03 2025-03-04 Confirm source region 28/05/2026 — 2026-05-28 2026-05-28 05/28/2026 2026-05-28 — 2026-05-28 Already broken? Fix dates in the browser (no server upload) I built a free tool cluster on FormatList — everything runs locally in your browser : 1. Date Format Fixer — bulk repair Paste a date column or upload .csv / .txt Ambiguous rows highlighted; optional US / UK preference Export ISO / US / UK or download a fixed CSV Best for: normalizing a whole column to ISO before a database or API imp
AI 资讯
AI Crawlers Are Scanning Your Site Right Now - How to Check and Control Access
AI crawlers now appear in many server logs alongside traditional search bots. Some are used for search retrieval, some for training, and some for broader web indexing. If you care about AI search visibility, you need to know which ones can access your public pages. The most common accidental blocker is simple: a robots.txt rule or CDN bot setting that prevents AI crawlers from reaching the content you want discovered. The major AI crawler tokens to check Here are crawler tokens you may see in logs or robots.txt rules: Crawler token Company Notes GPTBot OpenAI Documented OpenAI crawler token OAI-SearchBot OpenAI Documented OpenAI search-related crawler token ChatGPT-User OpenAI Documented OpenAI user-triggered agent token ClaudeBot Anthropic Documented Anthropic crawler token Claude-SearchBot Anthropic Documented Anthropic search-related crawler token Google-Extended Google Google control token for Gemini Apps and Vertex AI use CCBot Common Crawl Web corpus crawler used by many downstream systems PerplexityBot Perplexity Commonly referenced Perplexity crawler token Crawler names and purposes change. Always confirm against official platform documentation before making sitewide access decisions. First, check what is actually happening Before you change anything, find out who is already crawling. If you have server logs: grep -E "GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|Google-Extended|CCBot|PerplexityBot" access.log If you use Cloudflare, check bot and security events and filter by user agent. Three quick diagnostic steps: Open https://yourdomain.com/robots.txt and look for broad Disallow: / rules. Confirm the sitemap is listed in robots.txt or discoverable at /sitemap.xml . Use our AEO Checker to validate robots.txt and flag restrictive AI crawler rules. The most common mistake The blunt rule that makes sites invisible to many crawlers: User-agent: * Disallow: / This blocks every well-behaved crawler that follows the wildcard rule. If you see it on
AI 资讯
The Predictive Power of Philosophy: Why You Can’t Ask a Gun to Read a Bedtime Story
I want to talk about why philosophy is actually far more important than people think, especially when it comes to software engineering, systems design, and AI. When most people hear the word "philosophy," they roll their eyes. They think of abstract, circular arguments that don't matter in the real world. But true philosophy, good philosophy, is more like base mathematics. It is base physics. It is the raw understanding of the essence of a concept and how that translates into real-world action. If you don't understand the origin of a thing, you are left playing a game of perceptions. You will circle around a problem, coming up with endless rationalizations, but you will be completely unable to predict where it is going to go next. The origin of something is it fundamental nature. This origin is actually its bounding box. It dictates the absolute limits of its trajectory. Knowing this gives you predictive capability before you execute. It is the a priori knowledge that separates actual engineers from people who just copy-paste solutions. (When should and how should you copy paste, for example, 'it depends'.) The Gun Analogy and Inherent Limitations Imagine you are at a shooting range, and you point a gun downrange. As long as you point that gun in the general direction of the targets, it is not going to shoot directly behind you, or 90 degrees to the left. The inherent nature of the gun, and the velocity of the bullet, give it strict limitations. Because of those limitations, you can heavily rely on the fact that the bullet won't leave that bounding box. Therefore, shooting on a range is actually very safe. It only becomes unsafe when you turn the gun in a different direction. You have to understand that you cannot ask a tool to do more than its inherent nature allows. If you are firing an M16, it is not going to act like a guided missile and hit a target in another country hundreds of miles away. It does not have that capability. * Furthermore, a gun cannot read you
AI 资讯
Fix Web Performance Issues Faster with Modern Web Guidance and Chrome DevTools for AI Agents
Performance optimization has always been one of the hardest parts of web development. You run...
开发者
How I Explored a US Health Dataset with Python — EDA + Hypothesis Testing
I recently completed an exploratory data analysis project on the NHANES (National Health and Nutrition Examination Survey) dataset from Kaggle. It's a real-world health survey collected by the CDC covering body measurements, lifestyle habits, and demographic data from thousands of US adults. In this article I'll walk you through exactly what I did — from loading and cleaning the data all the way to running statistical tests — and share what I found along the way. The Dataset The dataset has 5,735 rows and 28 columns , but for this project I focused on 8 columns that were relevant to the questions I wanted to answer: Column Description smoking Has the person smoked at least 100 cigarettes? gender Male or Female age Age in years education Highest level of education weight Weight in kg height Height in cm bmi Body Mass Index Step 1 — Loading and Selecting Columns import pandas as pd import numpy as np import matplotlib.pyplot as plt import seaborn as sns db = pd . read_csv ( ' NHANES.csv ' ) data = db . loc [:, ( ' SEQN ' , ' SMQ020 ' , ' RIAGENDR ' , ' RIDAGEYR ' , ' DMDEDUC2 ' , ' BMXWT ' , ' BMXHT ' , ' BMXBMI ' )] data = data . rename ( columns = { ' SEQN ' : ' id ' , ' SMQ020 ' : ' smoking ' , ' RIAGENDR ' : ' gender ' , ' RIDAGEYR ' : ' age ' , ' DMDEDUC2 ' : ' education ' , ' BMXWT ' : ' weight ' , ' BMXHT ' : ' height ' , ' BMXBMI ' : ' bmi ' }) One thing worth knowing about NHANES: all the columns come in as numeric codes. 1 means Male, 2 means Female. 1 means the person smoked, 2 means they didn't. You have to map these to readable labels before doing any analysis, otherwise your charts are meaningless. Step 2 — Cleaning the Data Drop the ID column and remove nulls data . drop ( ' id ' , axis = 1 , inplace = True ) data . dropna ( inplace = True ) This brought us from 5,735 rows down to 5,406 — about 6% lost, which is acceptable. Remove outliers using the IQR method The IQR (Interquartile Range) method flags values that fall too far outside the middle 50% of
AI 资讯
I Built a Free Apache Kafka Course from Scratch — Here's the Full Curriculum (and What I Got Wrong)
I Built a Free Apache Kafka Course from Scratch — Here's the Full Curriculum (and What I Got Wrong) I spent months building a free Apache Kafka course covering everything from first principles to a real-time analytics platform final project. No paywall. No "premium tier." 9 modules, 470 minutes of content, completely free. Here's the full syllabus, the Python code that actually works, and the honest mistakes I made building the curriculum — so you don't repeat them. Why I Built This Every time someone asked me "how do I learn Kafka?", I sent them to the same 3 places: The official Confluent docs (dense, assumes you already know what you're doing) A $15 Udemy course that spends Module 1 explaining what a computer is A YouTube playlist where half the videos are deleted None of them answered the real question beginners have: why does Kafka exist, and what problem does it actually solve before I write a single line of code? That's the gap I built for. The Problem With Most Kafka Tutorials Most tutorials start with: "Kafka is a distributed event streaming platform..." And then they immediately show you a Docker Compose file with 6 services. Beginners copy-paste it, something breaks, they don't know why, they quit. The real problem is that Kafka is an answer to a specific architectural problem — and if you don't understand the problem first, the solution makes no sense. So Module 1 and 2 of this course don't touch Kafka at all. They build the problem statement from scratch. The Full Syllabus (9 Modules, 470 Minutes) Module 1: Introduction to Kafka — 35 min Not "what is Kafka" — but why event streaming exists at all. What breaks in traditional request-response architectures at scale. Module 2: The Problem Statement — 30 min A real-world scenario: you're building an e-commerce platform. Orders, inventory, notifications, analytics — all tightly coupled. What happens when one service goes down? This module makes the pain visceral before Kafka enters the picture. Module 3: How
AI 资讯
Your console.log Is Lying to You
Open your browser DevTools and run this: const user = { name : " Bob " } console . log ( user ) user . name = " Alice " You would expect the log to show { name: "Bob" } , the value at the time of the console.log call. The collapsed line is what you expect: ▶ Object { name: "Bob" } But expand it, and you will see: name: "Alice" Oops. So what's going on? console.log() is the most-used debugging tool in JavaScript, but it can be subtly unreliable. Not because it is broken, but because it optimizes for speed and interactivity rather than for accuracy . It was built for fast exploration in a live, interactive environment, and those priorities come with tradeoffs that can genuinely mislead you during debugging. Over the next sections, we'll look at a few ways the console can mislead you - and, more importantly, why each one exists. Objects Aren't Snapshots When you pass an object to console.log() in browser DevTools, the browser does not immediately serialize it into a string. Instead, it stores a live reference to that object and defers the actual rendering until you expand the entry. This is called lazy evaluation, and it is what caused the surprise. The collapsed ▶ Object you see is essentially a placeholder: the properties shown inside it are evaluated at the moment you click the arrow, not at the moment you called console.log() . By then, your code has already continued running. That means what you're seeing is not a frozen record of the object at the time of logging, but a live view into whatever the object happens to look like when DevTools renders it. In the example: You log { name: "Bob" } DevTools stores a reference to the user object The code continues executing user.name is mutated to "Alice" You expand the logged object later and see the current state This behavior can feel unintuitive at first, because most developers mentally model console.log() as "print this value right now", but in browser DevTools, it is closer to "show me this object as it exists when
AI 资讯
From Regex Hell to AI: How I Finally Tamed Messy PDF Invoices
Last month, I spent three days wrestling with 500 PDF invoices. Each one had the same data—vendor name, invoice number, total amount—but the layouts were all over the place. Different fonts, missing headers, tables that somehow broke across pages. I tried regex. I tried OCR with layout analysis. I even tried building a rule-based parser that looked for keywords like "Total:" . Nothing worked reliably. Every time I fixed one pattern, another invoice broke. I was one commit away from throwing my laptop out the window. Then I took a step back. I realized I didn't need to understand every layout variation. I just needed to understand the data . And that's where AI came in. What didn’t work Let me be clear: I tried the usual suspects first. Regex. Classic. I wrote patterns like r"Total\s*:\s*\$?(\d+\.\d{2})" . Worked on 60% of invoices. The rest had "Total Due" or "Amount Total" or the dollar sign in a different place. Regex is great when you control the input. I didn't. OCR with layout parsing. I used Tesseract with --psm 6 and tried to extract lines by bounding boxes. It helped a bit, but tables with merged cells or rotated text threw it off. Plus, I had to write code to guess which box was a field name and which was a value. Rule-based parser. I built a dictionary of known vendors and their layouts. That worked … until I got an invoice from a new vendor. Maintenance became a nightmare. I was solving the wrong problem. Instead of fighting formatting, I needed to focus on meaning . The AI approach that saved me I remembered that large language models are surprisingly good at understanding context. If I could give the model the raw text from a PDF and a description of what I wanted, maybe it could extract the fields directly. Here’s the core idea: treat extraction as a structured generation task. Provide a prompt with a few examples (few-shot) or just describe the schema, and let the model output JSON. I found an API that did exactly this with a simple HTTP call. (Full d
AI 资讯
When SSH commands hit a csh login shell — wrapping every command in /bin/sh -c across the codebase
One day a user reported an oddly asymmetric bug. In the "add new site" modal, picking an SSH profile and clicking "auto-detect WordPress install path" always failed with "no path found." But clicking the WP-CLI path test button on the same SSH connection worked fine . Same credentials, same host — one succeeded, the other failed. Tracing it down, the culprit was an old foe: csh / bash incompatibility on the server side . This post walks through the fix, sweeping the same bug across the rest of the codebase, and the static-analysis test we added to keep it from coming back. The smoking gun — find: 2: unknown primary or operator The server-side error log gave it away: find: 2: unknown primary or operator find itself is POSIX-standard, but it was dying with a mysterious 2 argument. That 2 is the leading number of 2>/dev/null — a redirect that was being passed as a literal argument to find because the shell never interpreted it as a redirect in the first place . Note: 2>/dev/null is the standard way to silently discard stderr in Bourne shell (sh) and bash. csh (C shell) uses different syntax and doesn't recognize it. Sakura Internet defaults users to csh We've documented this before in the four-host investigation of why WP-CLI doesn't run : on Sakura Internet (Japanese host), the default user login shell is csh / tcsh , not bash. This collides with how paramiko (Python's SSH library) works: exec_command runs the command through the user's login shell. Sending find ... 2>/dev/null to a Sakura host means csh tries to interpret it and chokes . That's the real error. The bash/sh idioms that fall over on csh include: 2>/dev/null (redirect) [ -f path ] (test syntax) for X in ...; do ... done (loop) cmd1 && cmd2 (short-circuit) \( ... \) (subshell) These all blow up with "unknown primary or operator" or "Missing }" on csh. "I fixed one site, so they're all fixed" — but they weren't This wasn't our first encounter with this issue. A few release rounds earlier, we'd noticed test