AI 资讯
How I Put PgCache in Front of a 16-Million-Row Postgres Database
Disclaimer: This is a side project, not a production story. The slow-query problem is real, but the database is synthetic data I generated to make it show up on demand. I have no connection to PgCache. Everything here is in a repo you can clone and run. I tested version 0.6.2. A handful of dashboard queries on one of my projects were fine for a year and then weren't: count users by tier, revenue grouped by country, best-selling products per category. Nothing exotic, just aggregates and joins over tables that had gotten big. The usual fixes didn't sit right with me. A materialized view means picking a refresh interval and serving slightly stale numbers in between. Redis in front of Postgres means writing and maintaining code that knows which cache entries to throw away on every write. A read replica just runs the same slow query on another machine. PgCache offers a different trade. It's a proxy that talks the Postgres wire protocol, so your app connects to it as if it were the database. It caches reads. And instead of expiring entries on a timer, it follows Postgres's replication stream and refreshes a cached result when the rows behind it change. That stream is the same feed Postgres uses to copy data to a standby server , a running log of every insert, update, and delete. The "no timers, no manual invalidation" part is the interesting claim. Here's how it held up. A database big enough to be slow First I needed a database where "slow" was real and not a rounding error. I wrote a seed script for a small e-commerce schema and filled it to about 16 million rows: Table Rows Notes users 1,000,000 10 countries; tiers 50% free / 33% pro / 17% enterprise products 2,000 10 categories orders 5,000,000 four statuses, random totals, spread over two years order_items 10,000,000 about two per order I added indexes on every foreign key and on every column the test queries filter or group by. That was on purpose. I wanted to compare PgCache against a Postgres that had been tuned p
AI 资讯
🤿 Diving Deep into Google SecOps: From Log Abyss to Automated Playbooks
Introduction: The Telemetry Abyss In information security, just like in technical deep-sea diving, we face a vast, silent, and potentially hostile environment. Modern corporate telemetry is an ocean: millions of gigabytes of data in constant motion. Without the right gear, security analysts risk "data narcosis." Google Security Operations (Google SecOps) acts as our autonomous breathing gear (SCUBA). It provides planet-scale visibility, allowing us to descend safely into the depths of logs, maintain control under pressure, and emerge with clear answers regarding potential incidents. In this field log, we document one possible professional workflow for structuring detection engineering in Google SecOps from scratch, using the Model Context Protocol (MCP) and a "Buddy System" with intelligent AI. Pre-Dive Check: Security in Memory Before jumping, every technical diver performs a rigorous equipment check. In SecOps, this means configuring our local environment and authenticating securely to Google Cloud Platform (GCP). A golden rule of diving is to avoid "gas leaks." In development, this means avoiding credential leaks by never writing API keys or tokens to persistent disk. We use a memory-native PowerShell loader (load-secops-env.ps1) that requests parameters interactively, keeping them strictly in RAM and destroying them upon closing the terminal. PowerShell # Security-First Environment Loader $projectID = Read-Host "Introduce el GCP Project ID" $customerID = Read-Host "Introduce el Chronicle Customer ID" $ env : CHRONICLE_PROJECT_ID = $projectID $ env : CHRONICLE_CUSTOMER_ID = $customerID $ env : CHRONICLE_REGION = "us" By launching your IDE from this active terminal, sub-processes inherit these variables securely without leaving secrets on your local drive. Guided Descent: Validating APIs and Currents Once submerged, we monitor pressure and currents. We perform structured checks to validate API activation and IAM permissions. During the descent, we may hit "thermoc
创业投融资
Microsoft 365 outage drags on, but things are improving
Microsoft 365 and Outlook are still seeing service degradations on Tuesday, the company's status page indicates.
AI 资讯
Sonos introduces new headphones, soundbar, and software in its biggest announcement in years
At an event in New York City, Sonos announced two new audio products - the Sonos Beam Ultra soundbar and Sonos Ace Ultra headphones, details of which were found in an FCC filing in early August - and a major update to its audio operating system dubbed Sonos 27. It's the biggest single Sonos announcement […]
创业投融资
Tim Cook’s Apple: his 10 biggest wins and misses
Expectations for Tim Cook were almost impossibly high when he stepped in to replace Apple's visionary cofounder in 2011. He inherited a company on a blockbuster run, after Steve Jobs returned and revitalized the Mac, launched the iPod, and oversaw the launch of the iPhone. Now, after 15 years in the CEO role, Tim Cook […]
AI 资讯
What are the alternatives to Xcode? Use these tools to restructure your iOS development workflow
Once, just to change an interface field, I spent nearly half an hour switching back and forth between several tools. The code was modified in VSCode. Because the project includes not only Swift but also Flutter modules and some script files. After making changes, I switched back to Xcode to compile, then the test package was handed over to an automation script, and finally I had to open another tool to upload. That day I suddenly realized something: many developers are actually no longer completely dependent on Xcode. To be more precise, it's not that they 'don't use Xcode,' but that the development workflow is being broken apart. Editors, compilers, build tools, and upload tools are each taking on different responsibilities. What many people really want to replace is not Xcode itself When discussing Xcode alternatives, we actually need to know what developers really want to replace. Xcode actually contains many parts: code editing, project management, compilation and building, simulator, on-device debugging, Archive, signing and distribution. Some people want to replace the editing experience, some want to reduce dependence on a full IDE, and others simply want to put different technology stacks into a unified workflow. Therefore, many current 'alternatives' are not complete replacements, but rather split some of these aspects. VSCode: The most common alternative Now more and more iOS developers use VSCode to write code. The reason is that many projects are no longer just native Swift; Flutter, Node services, Shell scripts, JSON configuration, and Web frontends may all be in the same repository. If everything is handled in one editor, the development context becomes more continuous. Swift plugins, Git plugins, and AI-assisted tools have also made VSCode increasingly used in iOS projects. However, it mostly replaces the 'editor' layer. When it comes to compilation and runtime, many projects still return to the Xcode toolchain. AppCode: Another route with a JetBrains
AI 资讯
Scaling Kafka Consumers in Spring Boot: How We Cut Lag and Saved Latency
Scaling Kafka Consumers in Spring Boot: How We Cut Lag and Saved Latency When scaling high-throughput event-driven microservices in fintech, default Spring Kafka consumer configurations often run into throughput limits under peak loads. Here is the exact production setup we engineered to resolve consumer lag and reduce API processing latency by 35%. 1. Concurrency Tuning Over Single-Threaded Listeners By default, @KafkaListener operates with concurrency = 1. When a partition receives high message volume, processing gets backlogged. @Configuration @EnableKafka public class KafkaConsumerConfig { @Bean public ConcurrentKafkaListenerContainerFactory < String , PaymentEvent > kafkaListenerContainerFactory ( ConsumerFactory < String , PaymentEvent > consumerFactory ) { ConcurrentKafkaListenerContainerFactory < String , PaymentEvent > factory = new ConcurrentKafkaListenerContainerFactory <>(); factory . setConsumerFactory ( consumerFactory ); factory . setConcurrency ( 6 ); // Matches number of partition splits factory . getContainerProperties (). setAckMode ( ContainerProperties . AckMode . MANUAL_IMMEDIATE ); return factory ; } } 2. Explicit Batch Processing and Idempotency Instead of committing offset per message, processing batches with manual acknowledgments ensures atomic handling: @Service public class PaymentEventConsumer { @KafkaListener ( topics = "payment.settlement.v1" , containerFactory = "kafkaListenerContainerFactory" ) public void consume ( ConsumerRecord < String , PaymentEvent > record , Acknowledgment ack ) { try { processPayment ( record . value ()); ack . acknowledge (); } catch ( Exception ex ) { log . error ( "Failed processing record key: {}" , record . key (), ex ); // Route to Dead Letter Queue (DLQ) handleDeadLetter ( record ); ack . acknowledge (); } } } 3. Key Takeaway Scaling Kafka consumer pipelines requires matching topic partition count with container concurrency, tuning database connection pools and implementing dead letter queues for fail
AI 资讯
Domain-Driven Design for Microservices: Building Scalable Systems with Clear Domain Boundaries
Domain-Driven Design for Microservices: Building Scalable Systems with Clear Domain Boundaries Introduction Domain-Driven Design (DDD) isn't just another architecture pattern—it's a philosophy that aligns technical decisions with business reality. When building microservices at scale, DDD becomes essential. Without it, you end up with services that don't respect business domains, unclear responsibilities, and integration nightmares. Why DDD Matters for Microservices Microservices force you to make decisions about boundaries. The question isn't whether you'll decompose your system—it's whether you'll do it thoughtfully using DDD principles, or accidentally create distributed monoliths. DDD answers three critical questions: Where should a service boundary exist? (Bounded Contexts) How do we communicate across services without coupling? (Domain Events, Anti-Corruption Layers) How do distributed teams understand the same problem? (Ubiquitous Language) Core Concept 1: Bounded Contexts A Bounded Context is a boundary within which a domain model is applicable. Each microservice should typically map to one or more Bounded Contexts. Java Example: E-commerce System // Ordering Context - Bounded Context 1 public class Order { private String orderId ; private List < OrderLineItem > lineItems ; private OrderStatus status ; // PENDING, CONFIRMED, SHIPPED, DELIVERED private LocalDateTime createdAt ; public void confirmOrder () { if ( this . status != OrderStatus . PENDING ) { throw new InvalidOrderStatusException ( "Cannot confirm non-pending order" ); } this . status = OrderStatus . CONFIRMED ; } } // Inventory Context - Bounded Context 2 public class InventoryItem { private String skuId ; private Integer availableQuantity ; private Integer reservedQuantity ; public void reserveStock ( Integer quantity ) { if ( availableQuantity < quantity ) { throw new InsufficientStockException ( "Not enough stock to reserve" ); } this . reservedQuantity += quantity ; this . availableQuantity -
科技前沿
Trump admin shelves Cyclospora research despite record-breaking outbreak
The CDC has tallied nearly 30,000 confirmed and probable cases this summer.
AI 资讯
Microsoft tests fix for latest hours-long Outlook outage
Microsoft says it's testing a fix for the widespread Outlook issues that have led to email delays and failures.
AI 资讯
Model Predictive Control for Real-Time Robot Navigation
Model Predictive Control for Real-Time Robot Navigation A path planner tells a robot where it should go. A controller determines how the robot should move to follow that path. Model Predictive Control (MPC) repeatedly predicts future behavior and chooses control inputs that optimize a short horizon. MPC Concept Current State | v Predict future states | v Optimize control sequence | v Apply first control | v Measure new state | +----> Repeat The key idea is that the entire control sequence is not executed at once. Only the first action is applied before the problem is solved again. Robot Model For a simple differential-drive robot: x_dot = v cos(theta) y_dot = v sin(theta) theta_dot = omega The controller can predict where the robot will be after applying candidate velocity commands. Optimization Objective A typical objective might penalize: Distance from reference path Heading error Excessive control effort Rapid control changes Collision proximity Conceptually: Cost = tracking_error + control_effort + smoothness_penalty + obstacle_penalty Prediction Horizon Suppose the controller predicts: t0 -> t1 -> t2 -> t3 -> t4 For each candidate control sequence it estimates the resulting trajectory. The optimizer selects the best feasible sequence. Obstacle Handling A cost function can strongly penalize trajectories near obstacles: Obstacle ### ##### ### \ predicted trajectories \---- safe \--- unsafe Hard constraints can also be used when collision avoidance must be guaranteed by the optimization formulation. ROS 2 Architecture /global_plan | v /mpc ^ | /odom /imu /local_costmap | v /cmd_vel Real-Time Requirements MPC is computationally heavier than simple feedback controllers. Monitor: Optimization time Control frequency Solver failures CPU utilization Prediction horizon Sensor latency If optimization misses its deadline, the system needs a safe fallback. Practical Implementation Strategy Start simple: Define a robot model. Implement trajectory prediction. Define tracking
AI 资讯
Implementing A* and RRT Motion Planning for Robotics
Implementing A* and RRT Motion Planning for Robotics Two classic planning approaches are A * and RRT (Rapidly-exploring Random Tree) . A* is particularly useful when the environment can be represented as a graph or grid. RRT is useful when planning in continuous or high-dimensional configuration spaces. A* Planning A* combines the cost already traveled with an estimate of the remaining cost. Conceptually: f(n) = g(n) + h(n) Where: g(n) is the cost from the start. h(n) estimates the cost to the goal. f(n) ranks candidate nodes. Grid Example S . . # . . . . . . # . . . . . . . . # . . # # # . # . . . . . . . G The planner explores promising cells while avoiding blocked cells. Python Implementation Skeleton import heapq def astar ( graph , start , goal , heuristic ): queue = [( 0 , start )] cost = { start : 0 } parent = { start : None } while queue : _ , current = heapq . heappop ( queue ) if current == goal : break for neighbor in graph [ current ]: new_cost = cost [ current ] + 1 if neighbor not in cost or new_cost < cost [ neighbor ]: cost [ neighbor ] = new_cost priority = new_cost + heuristic ( neighbor , goal ) heapq . heappush ( queue , ( priority , neighbor )) parent [ neighbor ] = current return parent RRT Planning RRT works differently. Instead of systematically exploring grid cells, it samples points and gradually grows a tree. x / x------x / S-----x x----x------G A typical loop is: Sample a random configuration. Find the nearest existing node. Steer toward the sample. Check collision. Add the new node if valid. Repeat until the goal is reached. RRT Skeleton for _ in range ( max_iterations ): sample = random_configuration () nearest = nearest_node ( tree , sample ) new_node = steer ( nearest , sample ) if collision_free ( nearest , new_node ): tree . add ( new_node ) tree . connect ( nearest , new_node ) if reached_goal ( new_node ): return extract_path ( tree , new_node ) A* vs RRT Property A* RRT Representation Grid/graph Continuous space Search Determinis
开发者
Building Global and Local Path Planners for Autonomous Robots
Building Global and Local Path Planners for Autonomous Robots Autonomous navigation is not just about finding a route from A to B. A robot must plan a useful route through a map and continuously adapt that route to obstacles, other robots, people, and changes in its environment. A practical navigation system therefore separates global planning from local planning . Global vs Local Planning Global Map | v +------------------+ | Global Planner | +------------------+ | v Global Path | v +------------------+ Sensors>| Local Planner | +------------------+ | v Velocity Commands | v Robot Global Planner The global planner considers the larger environment. Its job is typically to find a route such as: Start ---> Corridor ---> Door ---> Room ---> Goal Common approaches include: A* Dijkstra Graph search Grid-based planning Sampling-based planning Local Planner The local planner operates closer to the robot and reacts to current observations. It considers: Nearby obstacles Robot velocity Robot footprint Dynamic objects Current trajectory Short-term goal direction Why Both Are Needed Suppose the global path is: Robot -----> Hallway -----> Goal A person suddenly walks into the hallway. The global route may still be valid, but the robot needs to slow down, stop, or temporarily move around the person. That is the local planner's job. Grid-Based Global Planning Represent the environment as a costmap: . . . . . . . . . # # . . . . . # # . . . . . . . . . . . . . . . G . S . . . . . . A planner searches through free cells while assigning higher costs to undesirable regions. Local Planning A local planner can generate multiple candidate trajectories: obstacle ### Robot --> / | \ / | / | candidate trajectories Each trajectory can be scored based on: Collision risk Distance to path Distance to goal Smoothness Velocity Clearance ROS 2 Architecture /map | v /global_planner | v /global_plan | v /local_planner <--- /scan /pointcloud | v /cmd_vel Keep the global and local planners modular so
AI 资讯
Building a Real-Time SLAM System for Mobile Robots
Building a Real-Time SLAM System for Mobile Robots SLAM means Simultaneous Localization and Mapping . A mobile robot must answer two questions: Where am I? What does the environment look like? The challenge is that the robot needs the map to localize while also needing localization to build the map. SLAM Architecture Sensors | +--> Frontend | | | +--> Odometry | +------------------+ v State Estimator | v Map Builder | v Map Sensor Options Typical systems use: 2D LiDAR 3D LiDAR Cameras IMUs Wheel encoders The right sensor combination depends on the environment. SLAM Frontend The frontend extracts motion constraints. For LiDAR: Scan | v Feature / Point Processing | v Scan Matching | v Relative Motion For visual SLAM: Image | v Feature Extraction | v Feature Matching | v Relative Pose Backend Optimization The backend can represent the robot trajectory as a graph: Pose 1 ---- Pose 2 ---- Pose 3 ---- Pose 4 \ / +------ Loop Closure ---+ Loop closure recognizes that the robot has returned to a previously observed location. This can significantly reduce accumulated drift. Real-Time Constraints SLAM is not useful if it produces excellent maps several seconds too late. Monitor: Sensor processing latency Pose estimation latency Map update time CPU/GPU utilization Queue sizes Frame/scan drops Map Resolution Higher resolution gives more detail but costs more memory and computation. Choose resolution based on: Robot size Environment Navigation requirements Available compute Failure Modes SLAM can struggle with: Repetitive environments Dynamic objects Feature-poor walls Rapid motion Poor sensor calibration Incorrect timestamps A robust system should monitor confidence and detect tracking failures. Production Pipeline Camera / LiDAR / IMU | v Sensor Calibration | v Odometry Frontend | v Pose Estimation | v Loop Detection | v Graph Optimization | v Map Server | v Navigation The goal of production SLAM is not just map quality. It is stable localization, predictable latency, and grac
产品设计
ROS 2 QoS Profiles: Reliable vs Best-Effort Robot Communication
ROS 2 QoS Profiles: Reliable vs Best-Effort Robot Communication Robot systems continuously exchange data with very different requirements. A dropped camera frame is usually acceptable. A dropped emergency command may not be. ROS 2 Quality of Service (QoS) lets you express these requirements. The Two Common Reliability Modes Reliable Reliable communication attempts to ensure that samples reach compatible subscribers. Useful for: Commands Configuration Important state transitions Critical application data Best Effort Best effort prioritizes timely delivery and may tolerate lost samples. Useful for: Cameras LiDAR High-frequency IMU streams Other continuously refreshed sensor data Example Imagine a camera producing 30 frames per second. If frame 100 is lost, the system can often process frame 101 immediately. For a command: MOVE_FORWARD losing the message may be unacceptable. Therefore: Camera -> Best Effort Command -> Reliable is often a sensible starting point. QoS Dimensions Reliability is only one QoS policy. Important policies include: Reliability Durability History Depth Deadline Lifespan Liveliness C++ Example auto sensor_qos = rclcpp :: SensorDataQoS (); auto publisher = create_publisher < sensor_msgs :: msg :: Image > ( "/camera/image" , sensor_qos ); For important application data, you might explicitly configure reliable communication: auto qos = rclcpp :: QoS ( rclcpp :: KeepLast ( 10 )) . reliable (); auto publisher = create_publisher < std_msgs :: msg :: String > ( "/robot/status" , qos ); QoS Compatibility A publisher and subscriber need compatible QoS settings. A common mistake is: Publisher: Best Effort Subscriber: Reliable and then wondering why messages are not received as expected. Always inspect the effective QoS of both endpoints. A Practical Decision Table Topic Suggested Starting Point Camera image Best Effort Point cloud Best Effort IMU Best Effort Navigation command Reliable Configuration Reliable Robot state Reliable Diagnostics Reliable These
AI 资讯
Building a High-Performance Robot Communication System with DDS
Building a High-Performance Robot Communication System with DDS Modern robots may have dozens of processes distributed across CPUs, edge computers, and embedded devices. ROS 2 uses DDS (Data Distribution Service) as its underlying communication technology. Understanding DDS helps you design robot systems that remain responsive as message traffic grows. The Communication Model Instead of connecting every process directly: Camera ---> Perception LiDAR ---> Perception IMU ---> Localization | v Planning | v Control ROS 2 nodes communicate through DDS topics and discovery. A simplified model is: Publisher | v DDS DataWriter | v Topic | v DDS DataReader | v Subscriber Why DDS Is Useful for Robotics DDS provides mechanisms for: Discovery Reliability Durability Deadline management History Resource limits Data delivery policies These features are important because different robot data has different requirements. A camera stream may prioritize low latency. A configuration message may prioritize reliability. High-Performance Design Avoid treating every topic identically. For example: Data Typical Priority Camera frames Low latency LiDAR scans High throughput IMU Low latency Robot commands Reliability Configuration Reliability + durability Diagnostics Reliability Reduce Copying Large sensor messages can consume substantial CPU and memory bandwidth. Good practices include: Avoid unnecessary serialization/deserialization. Reuse buffers where possible. Keep image resolution appropriate for the workload. Compress only when bandwidth savings justify CPU cost. Separate high-rate sensor topics from low-rate metadata. Separate Data Paths A useful architecture is: +--> Vision Camera ----------+ | LiDAR -----------+--> Perception --> Planning --> Control | IMU -------------+ Diagnostics ---------------------> Monitoring Configuration ------------------> Lifecycle Manager Not all traffic needs the same QoS or processing path. Measuring Performance Do not optimize based on intuition alone.
AI 资讯
What I Learned Partitioning a Billion-Row Table in Production
Adding an index stops working eventually. Here's what we did when a nationwide logistics platform's core table crossed a billion rows — and the parts nobody warns you about. There's a specific moment in a backend engineer's life when the usual advice stops working. A query gets slow. You check the execution plan, you add an index, it gets fast again. This works for years. It works so reliably that it starts to feel like a law of nature. Then one day you add the index and nothing happens. Or worse — the index takes six hours to build, locks the table while it does, and the query is still slow at the end of it. That's roughly where we were on a nationwide logistics platform processing tens of thousands of orders a day. The tracking events table — one row per scan, per parcel, per status change — had crossed a billion rows. Every parcel generated a dozen or more events on its journey. The table only ever grew. This is what we did about it, and more usefully, what nobody told us beforehand. First: are you sure you need this? Partitioning is not a performance trick you reach for when a query feels sluggish. It carries real operational cost, and most tables that people want to partition should just be indexed properly. Some honest signals that you're actually at the boundary: Your indexes no longer fit comfortably in memory, so index reads hit disk Index maintenance — REINDEX, VACUUM, ANALYZE — takes so long you can't schedule it Deleting old data is impossible in practice, because a DELETE of a hundred million rows will destroy your write throughput for hours Your queries almost always filter on a single obvious dimension, usually time That last one matters more than the rest. Partitioning only helps if your access pattern lines up with how you split the data. If your queries hit every partition anyway, you have added complexity and gained nothing. For us the alignment was clean: nearly every query on the events table was scoped to a date range. Operations dashboards loo
AI 资讯
Foundry Model Router Expands from Two Regions to 28, Refreshing Its Model Pool
Microsoft expanded Foundry's model router from two regions to 28 for global standard and 21 for data zone deployments, while adding Claude Opus 4.8 and GPT-5.6 and removing four deprecated models. Default deployments receive pool changes automatically; configured subsets exclude new models until added. The effective context window equals the smallest model in the pool. By Steef-Jan Wiggers
AI 资讯
Pocket's AI made my game ideas real. Now Meta controls the results.
Interactive mobile "gizmos" are easy to make, hard to share outside Meta's platform.
AI 资讯
Setting Up Your Own VPS: A Secure Starting Point
Every self-hosted project I run starts the same way: a brand new VPS and about twenty minutes of setup before I install a single application. That twenty minutes is what separates "my server" from "someone else's crypto miner." A fresh box with a public IP starts getting probed within minutes, and the default configuration on most images is built for convenience, not safety. This is the secure baseline I set up on every new server, before Docker, before n8n, before anything else. It is also the starting point our production n8n guide assumes you already have. Every command below was checked against current Ubuntu LTS documentation, and I flag the parts that genuinely need a real server to verify. Key takeaways Never do daily work as root. Create a sudo user and log in as that instead. Use an SSH key and turn password login off, but only after you confirm the key works. Deny everything at the firewall by default, then open only the ports you actually use. Turn on automatic security updates so patches land while you sleep. If you plan to run Docker, remember that published ports skip UFW. Bind them to 127.0.0.1 . Prerequisites A VPS running a current Ubuntu LTS. Both 24.04 "Noble Numbat" and 26.04 "Resolute Raccoon" work well. I run long-lived boxes on Hostinger VPS hosting , which is also what powers the n8n guide. An SSH key pair on your own machine. If you do not have one yet, Step 3 creates it. A terminal, and a note of your provider's recovery console. Most hosts, Hostinger included, give you a browser based console in their control panel. That is your way back in if you ever lock yourself out, so find it before you start. Disclosure: some links in this guide, including the Hostinger link above, are referral or affiliate links. If you sign up through them we may earn account credit or a commission, at no extra cost to you. We only point at tools we actually run. Step 1: Log in and update the system Right after the server boots, log in with the credentials your pr