标签：#learn

Over the past few months, I've been working on a high-scale scraping pipeline to aggregate listings directly from company job boards and applicant tracking systems. Mapping over 100,000 distinct companies to their career pages turned out to be a massive engineering headache, but it's finally stable. The result is a unified database of more than 2 million active job postings, which I'm opening up to everyone for free. I am running daily delta refreshes to keep it current. Dataset Overview Scale: 2M+ active job listings across 100,000+ unique companies. Format: Parquet. (To keep storage costs to minimum) Core Fields: job_title, company_name, company_website, job_description, location, post_date, and the original tracking URL. For more detailed info check here . Update Cadence: Refreshed daily straight from the source. View the stats here . (Currently it contains only minimal stats, but I plan on improving it based on the comments) Why I Built This Finding a clean, scaled, and up-to-date job dataset is surprisingly difficult. Most available options are either heavily gatekept by expensive subscription APIs or restricted to a single job board like LinkedIn. By scraping the actual employer sites directly, this collection sidesteps the noise and captures a much cleaner cross-section of the live market. How to Access It I set up a dedicated project space where you can grab the data directly: Open Job data Let me know what kind of analysis or projects you end up running with it. If you have questions about the engineering architecture behind handling this scale, or ideas for specific fields you'd like to see enriched next, let's discuss in the comments. submitted by /u/Invicto_50 [link] [留言]

Why C Is Still the GOAT (And Why People Like Me End Up Hating C++) Disclaimer: This is a personal opinion from a developer who enjoys simple tools, simple languages, and simple debugging sessions. The Eternal Question Every few months, somebody asks: "Why are you still writing C in 2026?" My answer is always the same: Because C knows exactly what it is. No hidden magic. No surprise abstractions. No template metaprogramming black holes. No compiler errors longer than my source code. Just you, a compiler, and a Segmentation Fault waiting patiently around the corner. C Is Honest One thing I love about C is that it never pretends to protect me. If I allocate 8 bytes and write 16 bytes? Boom. If I dereference a bad pointer? Boom. If I forget to free memory? Boom (eventually). But here's the important part: I know exactly why I got cooked. The language didn't hide anything from me. C basically says: "Here's the loaded foot-gun. Try not to shoot yourself." And honestly? I respect that. Meanwhile in C++ C++ often feels like this: template < typename T > concept SomethingComplicated = requires ( T t ) { // 300 lines of magic }; Then the compiler responds with: error: instantiation of recursive template ... required from ... required from ... required from ... required from ... ...followed by 14 pages of diagnostics. At this point, I'm no longer debugging my code. I'm debugging the language itself. My Personal Villain Origin Story I started using Visual C++. You know. The gigantic Microsoft ecosystem. The IDE. The project files. The build settings. The mysterious compiler flags. Eventually I found myself spending more time fighting tools than writing software. And somewhere along the way, I started associating that frustration with C++ itself. Fair or unfair? Probably unfair. But emotions aren't always rational. Modern C++ Feels Like Three Languages Wearing a Trench Coat Old-school C++? Pretty understandable. Modern C++? Sometimes it feels like: C Object-Oriented C++ Template

Most people use these technologies every day. Almost nobody knows they exist. Every time you open YouTube, browse Instagram, join a Zoom meeting, or play an online game, your router is quietly performing a series of networking tricks behind the scenes. Those tricks have names: NAT SNAT DNAT PAT Port Forwarding They sound intimidating. They're actually much simpler than they appear. Let's break them down using something familiar: your home Wi-Fi. The Problem the Internet Had to Solve Imagine a family of five living in one house. Everyone owns a device: Laptop Phone Smart TV Gaming Console Tablet Each device needs internet access. The problem? Your Internet Service Provider usually gives you only one public IP address . Something has to manage all those devices sharing a single internet connection. That's where NAT comes in. NAT: The Receptionist of Your Network NAT stands for Network Address Translation . Think of NAT as a receptionist in an office building. People inside the building have room numbers: Laptop = Room 101 Phone = Room 102 TV = Room 103 But when communicating with the outside world, everyone uses the building's main address. The receptionist keeps track of who sent what. Your router does exactly the same thing. What Happens When You Visit Google? Inside your home: Laptop 192.168.1.10 Your router: Public IP 49.x.x.x When you open Google: 192.168.1.10 ↓ Router ↓ 49.x.x.x ↓ Google Google never sees your private IP. It only sees your router's public IP. That's NAT in action. SNAT: Changing the Sender's Address SNAT stands for Source Network Address Translation . The keyword is: Source It changes the sender's address. Before leaving your network: Source: 192.168.1.10 After SNAT: Source: 49.x.x.x The router replaces your private IP with its public IP. Without SNAT, websites wouldn't know how to send responses back to you. Real-Life Example Imagine mailing a letter. Instead of writing your bedroom number as the return address, you write the house address. Tha

I don't identify as a gamer. I occasionally participate in board games or an amateur football match,...

The dominant production failure mode for LLM agents in 2026 isn't bad reasoning — it's capacity. Here's what the data shows, why nobody demos it, and the capacity-engineering patterns that actually keep agents alive under load.

Let me start with a confession: I'm a data scientist who's been burned by hype more times than I care to admit. When everyone told me "Model X is the next GPT-killer," I'd run my own benchmarks and find... well, let's just say the results were rarely as advertised. So when I started seeing claims about Chinese AI models catching up to (and sometimes surpassing) Western counterparts, I did what any self-respecting data nerd would do: I put them through my own rigorous testing pipeline. Over the past three months, I've run over 2,000 API calls across four major Chinese model families — DeepSeek, Qwen, Kimi, and GLM — using Global API's unified endpoint (more on that later). I tracked latency, token costs, output quality across multiple benchmarks, and even threw in some real-world tasks that mattered to me personally. Here's what I found, with all the numbers you'd expect from someone who still gets excited about statistical significance. The Testing Methodology (Because Anecdotes Aren't Data) Before we dive into results, let me be transparent about my approach. I ran each model on the following standardized tests: Code Generation : HumanEval (Python) and MBPP (multi-language) — 164 problems total Reasoning : GSM8K (math word problems) and MMLU-Pro (general knowledge) — 1,200 questions Chinese Language : CLUE benchmarks (text classification, NER, reading comprehension) — 3,500 samples English Language : LAMBADA and Hellaswag — 2,000 samples Speed : Average tokens per second over 100 consecutive requests with consistent prompt lengths I also tested vision tasks where applicable, but let's be real — Kimi doesn't support vision at all, and DeepSeek's implementation is... experimental at best. More on that later. All tests were conducted using the same global-apis.com/v1 endpoint, which normalizes API compatibility to OpenAI's format. This isn't an ad — I genuinely found it made my testing easier because I could swap models without rewriting code. The Big Picture: Pricing

Please post your personal projects, startups, product placements, collaboration needs, blogs etc. Please mention the payment and pricing requirements for products and services. Please do not post link shorteners, link aggregator websites , or auto-subscribe links. -- Any abuse of trust will lead to bans. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. -- Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads. submitted by /u/AutoModerator [link] [留言]

79% of enterprises have adopted AI agents. Only 11% run them in production. We've spent the past year building agent systems for banks, clinical operations teams, and engineering orgs. The problem isn't that agents don't work — they work fine. The problem is that every framework leaves compliance, cost governance, and crash recovery as exercises for the team. After the framework fails them in production. We built MeshFlow to close that gap. **The core idea:** treat governance as infrastructure, not middleware. Every agent step passes through a 15-step kernel that handles identity, rate limiting, budget enforcement, compliance profiles, input/output guardrails, PII detection, risk classification, tool permission, the LLM call itself, audit ledger write, and SLA recording — in that order, always, without configuration. ```python from meshflow import Workflow, CostCap, Agent wf = Workflow(cost_cap=CostCap(usd=5.00)) wf.add(Agent('researcher'), Agent('analyst'), Agent('writer')) result = wf.run('Write a competitive analysis of our market') # Compliant. Durable. Audited. Cost-capped. Done. ``` ```bash pip install meshflow ``` **What's technically interesting:** **Token optimization layer** — five compounding mechanisms that reduce LLM spend 70-85%: - `cache_control` on every system prompt and tool definition (Anthropic: 10% of normal price on cached tokens) - `ModelRouter`: task-type classification routes simple tasks to nano models (keyword + token-count heuristic, zero LLM call) - `ContextCompactor`: sliding window summarization activates at configurable token threshold - `RAGTokenBudget`: hard `max_chars` cap on knowledge injection with truncate/drop/tail strategies - `ContextDeduplicator`: shared context sent once for N parallel agents, not N times **SHA-256 audit chain** — each step record stores `prev_hash` (SHA-256 of the previous record) and `entry_hash` (SHA-256 of its own canonical fields). Modify any log entry and `verify_chain()` breaks. This is the artifact

Hey ML community, We’ve just open-sourced **MeshFlow** , a code-first, framework-agnostic runtime designed for governing and optimizing multi-agent systems in production. Most agent frameworks focus on rapid prototyping, but ML and platform engineering teams usually run into hard bottlenecks around LLM cost scaling, evaluation alignment, and execution safety. MeshFlow tackles these from a runtime/infrastructure perspective. Here are the key ML and system features: * **Task-Based Model Routing** : Before an agent executes a node, MeshFlow runs an evaluation on task complexity, routing the execution to one of four model tiers (`nano`, `small`, `medium`, `large`). This cuts overall API costs by 50-60% by utilizing smaller local models (e.g. LLaMA-3-8B) for standard formatting or extraction and reservation of frontier models (e.g. Claude Opus) for high-complexity reasoning. * **Context Compactor & Summary Pruning Middleware** : Implements sliding window summarization and context deduplication across parallel agent teams to limit prompt length growth. * **System Prompt Caching** : Native injection of Anthropic `cache_control` tags when system prompts exceed 1024 tokens. * **Cost Regression Evaluation Gate** : Integrates with CI pipelines to evaluate agent changes against a golden scenario baseline, throwing failures if code updates introduce token cost regressions. * **Resilient State Persistence** : Multi-backend state serialization (Redis, PostgreSQL, S3) that preserves checkpoint frames and allows resuming paused workflows. Here is the basic API contract: ```python from meshflow import Workflow, Agent, CostCap wf = Workflow(cost_cap=CostCap(usd=5.00)) wf.add(Agent('researcher'), Agent('critic'), Agent('writer')) result = wf.run('Compile comparative literature review of LLM reasoning pathways') print(result) ``` We'd love to discuss: 1. How do you handle token budget enforcement and model routing in your agent loops? 2. What evaluation pipelines do you use to detect co

Hi everyone, I missed the ICML conference tickets because I was waiting for some travel funding confirmation and now they are sold out. Do you know any other ways I could still purchase one? There seems to be no waiting list… or if you know anyone who needs to cancel theirs, please let me know 🙏🏻 submitted by /u/TopPerformance1255 [link] [留言]

It seems that there are two ways to build voice AI: Half-duplex: strict turn-taking. You speak, the other side waits until you’re done, one direction of speech at a time. ← This is how almost every voice assistant works today. Full-duplex: two channels, both sides can talk at any time - no more waiting for your “turn”. ← This is the way humans actually talk. In fact, there are three crucial things half-duplex voice models can't really do: Overlap - talking and listening at the same time without falling apart Backchannels - the "mhms," "rights," and "yeahs" you drop in while the other person is still going Barge-in - getting interrupted mid-sentence and recovering gracefully These three features are a big reason why voice agents still feel “robotic” to this day. But what exactly is the spectrum from half-duplex to full-duplex? Is a Moshi-style architecture the only way to approach full-duplex natural voice conversations? What are ways half-duplex systems could imitate full-duplex? Would love to hear others' thoughts on this. submitted by /u/Chilly5 [link] [留言]

This article was originally published on davidohnstad.net . I cross-post here to reach the Dev.to community. { " @context ": " https://schema.org ", " @graph ": [ { "@type": "Person", " @id ": " https://davidohnstad.com/#author ", "name": "David Ohnstad", "url": " https://davidohnstad.com ", "sameAs": [ " https://www.linkedin.com/in/davidohnstad/ ", " https://orcid.org/0009-0007-9023-7456 ", " https://davidohnstad5.mystrikingly.com/ ", " https://github.com/davidohnstad40-netizen ", " https://hashnode.com/@davidohnstad ", " https://davidohnstad.com ", " https://davidohnstad.net ", " https://davidohnstad.info ", " https://david-ohnstad.com ", " https://davidohnstadminnesota.com " ], "jobTitle": "Senior Data Product Manager", "worksFor": { "@type": "Organization", "name": "Veeam Software", "url": " https://www.veeam.com " }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "College of St. Scholastica" }, "address": { "@type": "PostalAddress", "addressLocality": "Duluth", "addressRegion": "MN", "addressCountry": "US" }, "description": "Senior Data Product Manager at Veeam Software, MS and MBA from the College of St. Scholastica, based in Duluth, Minnesota. Specializes in data architecture, AI/ML integrations, and SaaS platform development." }, { "@type": "Article", " @id ": " https://davidohnstad.net/why-enterprise-ai-projects-fail-platform-first#article ", "headline": "Why Enterprise AI Projects Fail: Platform-First Thinking", "description": "David Ohnstad reveals why enterprise AI initiatives fail despite massive investment. Learn the platform-first trap and how successful teams build differently.", "url": " https://davidohnstad.net/why-enterprise-ai-projects-fail-platform-first ", "datePublished": "2026-05-29T14:06:46Z", "dateModified": "2026-05-29T14:06:46Z", "author": { "@type": "Person", " @id ": " https://davidohnstad.com/#author " }, "publisher": { "@type": "Organization", "name": "David Ohnstad", "url": " https://davidohnstad.net ", "logo": { "@type": "Ima

Hey everyone, hope this is ok to post here. I built a free EU AI Act risk assessment tool and would love some feedback from people who actually know this space. You fill out a 10-question form describing your AI system, it classifies your EU AI Act risk tier, and emails you a PDF report with your applicable Articles and priority actions. Takes about 2 minutes, no account required. https://assessment.aiella.com Eventually I want to build a monitoring SDK that works like a Python library and automatically documents compliance of the technically measurable requirements at inference time. Looking for design partners for that down the road. Genuine feedback welcome, especially from anyone who has been through a real EU AI Act compliance process. Happy to answer questions about the classification methodology or the AWS architecture behind it. submitted by /u/aiandi [link] [留言]

We recently hit a classic gradient boosting trap with our pricing engine (Flyback), and I wanted to share the ablation data. We run LightGBM quantile regression to forecast secondary market watch prices. We engineered a variant-conditioned Bayesian target encoder to isolate within-reference pricing dynamics. LightGBM absolutely loved it. It ranked #1 in feature importance at q90 by a wide margin, with gains several times the next-highest feature, across all our multi seed runs. But when we ran a strict 4-seed × 3-variant ablation on the hold-out set, the results inverted. Test MAPE regressed by +0.28pp and the between-variant delta was 7x the within-variant standard deviation. The encoder was finding effective splits that completely failed to generalize because the signal it was learning was driven by irreducible label variance: unobserved factors like condition nuance, seller behavior, and timing that no feature can capture. I wrote a full post breaking down the architecture, the ablation methodology, and the mechanism behind the divergence. Happy to discuss LightGBM split mechanics, target encoding leakage, or the ablation setup. Full post and ablation results: https://flyback.ai/engineering/target-encoding-divergence submitted by /u/Nj-yeti [link] [留言]

From CFD and FEA to digital twins, carmaking now involves a lot of virtualization.

Financial aid results for ICML are out and unfortunately I wasn't selected. I was wondering, does this mean I wasn't selected for Volunteering as well? Or should I expect a separate email? submitted by /u/RussB3ar [link] [留言]

Hello, I have a task to fine-tune small LLMs on annotated conversational data. The dataset contains not only the final answers, but also reasoning traces and tool-calling decisions (i.e., when the model should think and when it should call a tool). I am wondering what the best training approach would be and why. My current dataset is stored in a chat format similar to this: ```text system user assistant_think assistant_tool assistant_answer user assistant_think assistant_tool assistant_answer ... ``` My current idea is to split each conversation into multiple training samples. For example, if a conversation contains two user turns, I would create two samples: Sample 1 text system user assistant_think assistant_tool assistant_answer Sample 2 ```text system user assistant_think assistant_tool assistant_answer user assistant_think assistant_tool assistant_answer ``` In other words, each sample contains all previous conversation history up to the assistant response being trained. For training, the loss would be computed only on the assistant-generated tokens: text assistant_think assistant_tool assistant_answer while the system and user messages would be masked out from the loss. Is this approach correct, or is there a better way to structure the training data for reasoning and tool-calling behavior? My second question is about reinforcement learning. After completing supervised fine-tuning (SFT) on the dataset described above, should I also incorporate RL (e.g., PPO, GRPO, DPO, or another approach) to further train the model on when a tool should or should not be called? If so: What advantages would RL provide over SFT alone for tool use and reasoning? How would you design the reward function? Under what circumstances is RL actually necessary, and when is SFT sufficient? I would appreciate any practical advice, papers, blog posts, or open-source examples related to training reasoning and tool-calling models. ``` submitted by /u/zdeneklapes [link] [留言]

Scroll through social media today, and you'll likely come across AI-generated images everywhere. From anime-style portraits and fantasy landscapes to hyper-realistic photographs of places that don't even exist, AI image generators have quickly become one of the most fascinating applications of artificial intelligence. What makes this technology so impressive is its accessibility. A few years ago, creating professional-quality artwork required design skills, expensive software, and hours of effort. Today, anyone can generate stunning visuals simply by typing a few words. But what actually happens behind the scenes when you enter a prompt and click "Generate"? Turning Ideas into Images At a basic level, AI image generators convert text into visuals. When a user enters a prompt such as: "A futuristic Mumbai skyline at sunset with flying cars" the AI doesn't search for an existing image online. Instead, it creates a completely new image based on patterns it learned during training. These models are trained using millions of image-text pairs, allowing them to understand concepts such as objects, colors, lighting, artistic styles, and even relationships between different elements within a scene. As a result, the AI can interpret the user's description and transform it into a visual representation. Starting with Random Noise One of the most interesting aspects of modern AI image generation is that the process usually begins with random noise. Imagine the static pattern seen on an old television screen. Initially, the AI starts with something similarly meaningless. It then gradually removes the noise while adding details that match the prompt. This process is known as a diffusion model , and it is the foundation of many modern AI image generators. To understand the idea, consider the following simple Python example: import random prompt = " A futuristic Mumbai skyline at sunset " noise_level = random . randint ( 1 , 100 ) print ( f " Prompt: { prompt } " ) print ( f " Start

I built a routing-based approach to lightweight real-time multilingual ASR as part of my research at Gladia. The core problem was how multilingual models that accurately handle mid-conversation language switches are often too big for most local hardware and have poor accuracy. So rather than relying on one massive multilingual model, the system routes audio between smaller, specialized monolingual models (~100M parameters each). Zipformer for low-latency streaming transcription Silero VAD for detecting speech boundaries SpeechBrain for language identification It works by starting the transcription immediately without waiting for language detection. A coordinator buffers audio, monitors language confidence, and when a switch is detected above a threshold, it rolls back to the last speech boundary and re-transcribes with the correct model. Users may briefly see incorrect text, but it self-corrects quickly. Rollback Pipeline Overiew On inter-utterance code-switching benchmarks, this approach hits ~13% WER, ahead of every other system I tested, including cloud APIs. Intra-utterance switching (mid-sentence Spanglish, etc.) is the known limitation, degrading to ~41% WER, though still better than open-source alternatives and at a fraction of the size. Open-source repo with instructions and the detailed benchmark results. https://github.com/gladiaio/realtime-multilingual-asr-router Let me know what you think. Pro tip: Enabling only your expected languages not only makes the system lighter but also gives the LID an accuracy boost, especially on heavily accented speech." submitted by /u/JeanMichelRanu [link] [留言]

This is a follow-up to SynaptoRoute: A Study in Local Semantic Routing . If you haven't read it, the short version is: SynaptoRoute is a zero-token semantic routing engine that classifies user queries into intents using local embeddings instead of LLM API calls. SynaptoRoute v0.3.0: Matching Semantic Router While Scaling to 50,000 Routes What Changed Since v0.2.0 When I published the first post, SynaptoRoute had just shipped dynamic batching and O(1) hot-reload. The throughput numbers were promising, but the accuracy story was incomplete. I had internal benchmarks but no comparison against a widely adopted baseline under identical, reproducible conditions. That gap is now closed. v0.3.0 is live on PyPI: pip install synaptoroute == 0.3.0 The Benchmarking Journey Getting to these numbers took multiple benchmark revisions. Early synthetic datasets produced catastrophic accuracy collapse and initially suggested that both SynaptoRoute and Semantic Router were performing poorly. After deeper investigation, the root cause turned out to be flaws in the dataset generation pipeline rather than limitations of the routing engines themselves. Several rounds of validation, failure analysis, threshold tuning, adversarial testing, and external benchmarking followed. All final results presented in this article come from independent public datasets with strict train/test separation, eliminating dataset leakage and benchmark inflation. That process was valuable because it forced the project to validate assumptions against real-world data instead of relying on synthetic benchmarks. The Benchmark That Actually Matters I evaluated SynaptoRoute against Semantic Router on two standard NLU datasets. Same embedding model ( BAAI/bge-small-en-v1.5 ). Same hardware. Same evaluation script. Same train/test splits loaded from HuggingFace. CLINC150 150 intents spanning 10 domains, plus an out-of-domain class. This is the standard stress test for intent routers. Metric SynaptoRoute Semantic Router