标签：#ia

I Built an Agent Evaluation Harness for Local AI — Here's What Most People Get Wrong DOYR | Not financial/legal/tax advice. For educational purposes only. Three months ago, I started building AI agents for my trading business. First agent: Fetches Nifty option chain data. Second agent: Analyzes PCR, OI, max pain. Third agent: Predicts direction using XGBoost. Fourth agent: Sends Telegram alerts. I had 4 agents doing 5 jobs. And I had no idea if they were any good . Sure, my trading results were +₹96,000 over 6 months. But was that because my agents were smart, or because I was overriding their bad decisions? I couldn't answer that question. So I built something to find out. An Agent Evaluation Harness. What Is an Agent Evaluation Harness? An Agent Evaluation Harness is a systematic framework for testing AI agents. It answers one question: "How good is this agent, actually?" Most people skip evaluation. They build an agent, test it once or twice manually, and call it "done." Then they wonder why it fails in production. An evaluation harness forces you to: Define success metrics — what does "good" mean? Create test suites — what scenarios will you test? Run evaluations — how does the agent perform across all scenarios? Measure regressions — did a change make the agent worse? Track improvements — is version 2 better than version 1? This is not optional. This is engineering 101 . Why Most Agent Evaluations Are Wrong I reviewed 50+ "agent evaluation" frameworks online. Here's what I found: Mistake 1: Single-Task Testing What they do: Test the agent on one task. "Can it book a flight?" → Yes/No. What's wrong: Real agents face thousands of variations of the same task. "Book a flight from Delhi to Mumbai on Friday" vs "Book a flight from Delhi to Mumbai next Friday" vs "Book a flight from Delhi to Mumbai on August 15th." A good harness tests variations , not just one example. Mistake 2: No Edge Cases What they do: Test happy paths only. "Book a flight when everything works.

Chinese researchers have shown that AI models have the capacity to act like aggressive and adaptive computer viruses.

Reports of cases began in June, and the national tally is now nearing 23,000.

Reddit is expanding its moderation tools and building stronger abuse prevention systems that it says could eventually reduce communities’ reliance on karma and account-age requirements, making it easier for legitimate newcomers to participate.

More than 50 offending image and video ads were published across Facebook, Instagram, Messenger, or Threads, according to Meta’s ad library data. Some ran as recently as this week.

Start with the Built-in Inspector Before reaching for external tools, remember Node.js has a built-in debugger. Run your script with --inspect and open chrome://inspect in Chrome to get a full DevTools experience: breakpoints, step-through, console, and even memory profiling. node --inspect app.js For a quick breakpoint without touching the browser, use --inspect-brk to pause on the first line. This is great for debugging startup issues. Use debugger Statements and Conditional Breakpoints Sometimes you need a breakpoint only when a condition is true. Instead of littering your code with if blocks, set a conditional breakpoint in DevTools. Right-click the line number, choose "Add conditional breakpoint," and enter an expression like user.id === 42 . For quick inline debugging, debugger; works but remember to remove it before committing. I often use it temporarily when I'm too lazy to open the DevTools UI. Log Like a Pro with util.inspect console.log of an object prints [object Object] which is useless. Use util.inspect with depth and colors to see nested structures clearly. const util = require ( ' util ' ); console . log ( util . inspect ( myObject , { showHidden : false , depth : null , colors : true })); Or in modern Node, you can use console.dir with { depth: null } for the same effect. Async Stack Traces: Don't Lose the Context Async errors are painful because stack traces often end at the event loop. Node 12+ gives you better async stack traces by default, but you can improve them further by using Error.captureStackTrace in your own error classes. class MyError extends Error { constructor ( message ) { super ( message ); Error . captureStackTrace ( this , MyError ); } } This makes the stack trace point to the caller, not the constructor. Handle Unhandled Rejections and Exceptions Silent failures are the worst. Set up global handlers to log errors properly and exit gracefully. process . on ( ' unhandledRejection ' , ( reason , promise ) => { console . error ( ' U

The company is also planning to retire its public API.

Jeff Dean and other high-profile Google executives have founded Discovery Loop, a startup that will seek AI-powered breakthroughs in everything from drug discovery to chip design.

Tinder is expanding its in-person events feature from an initial Los Angeles test to 26 cities worldwide by the end of September.

As streamers compete with social media giants for viewer attention, Disney+ is partnering with TikTok to bring creator content to its app.

LaTeX is a document preparation system for high-quality typesetting, perfect for academic papers and technical docs. Many people turn to Overleaf as their go-to online editor for LaTeX, but it comes with its own frustrations. If you are tired of Overleaf being costly and always hitting the compile timed out error, this guide is for you! The full MacTeX install weighs in at a massive ~6.4GB, most of which you'll never actually use. Setting up a minimalist LaTeX environment on macOS using BasicTeX and VSCode is a much better alternative that makes your setup ~8 times smaller. It saves storage and makes it much easier to collaborate with your teammates using GitHub as a combo. Install LaTeX via Homebrew We'll use Homebrew to keep things manageable. If you don't have it, grab it at brew.sh . 1. Install LaTeX BasicTeX is the "lean" version of MacTeX. It's only ~140MB initially. brew install --cask basictex 2. Refresh your path and verify Make the TeX binaries available in your current terminal session: eval " $( /usr/libexec/path_helper ) " The default LaTeX compiler pdflatex should be available now. Verify it's working: which pdflatex pdflatex --version 3. Update tlmgr and packages tlmgr is the TeX Live Manager. To update tlmgr and all packages, run the following commands: sudo tlmgr update --self sudo tlmgr update --all 4. Install latexmk (build manager) latexmk is the "build manager" that handles multiple runs of the compiler (necessary for bibliographies and tables of contents). sudo tlmgr install latexmk Verify latexmk version: which latexmk latexmk --version 5. Install essential package collections BasicTeX is too bare-bones for real projects. Since we went minimalist, we need to grab only the packages we actually use. These three collections will cover 90% of your needs while keeping storage down. sudo tlmgr install collection-latexrecommended sudo tlmgr install collection-fontsrecommended sudo tlmgr install collection-latexextra Note: If a build fails due to a mi

Mugshawtys and similar pages often frame mugshots as thirst traps. The women featured are humiliated and are sometimes harassed for months and years after their arrests.

Originally published at htpbe.tech . The version on htpbe.tech stays in sync with the latest detection algorithm — refer to it for the canonical text. A large share of fintech still ships on Rails. Stripe, Gusto, GitHub, Shopify, Instacart — the generation of companies that defined modern payments and payroll built their backends on Ruby, and the startups following them keep reaching for the same stack. So when a forged bank statement, an altered payslip, or a doctored invoice lands in an underwriting queue, more often than you would guess it lands on a Rails controller. Your KYC provider already confirmed the applicant is a real person with a valid identity. It said nothing about whether the PDF they uploaded was edited after the bank generated it. That structural-tampering layer is invisible to identity verification, and the right place to catch it is at ingress — before your Document model saves, before the row reaches underwriting, before any downstream system trusts the file. This guide walks through integrating the PDF tamper detection API into a Ruby on Rails application: from the first curl command to an idiomatic HtpbeClient service object built on Faraday, a Data -class result struct, configuration-bound credentials, a typed error class, an ActiveJob that analyzes an uploaded document and routes on the verdict, and a request spec that stubs the API with WebMock. The patterns target Rails 7.x and Ruby 3.x, but they map cleanly onto Sinatra, Hanami, or a plain Ruby worker. Treat the code as a reference architecture: it runs the real request flow against the documented error codes, but you should adapt and harden it for your own traffic profile and threat model. If you want the conceptual overview first, start with How to Detect PDF Tampering Programmatically . Integrating from another stack? See the Python , Node.js , Go , Java / Spring Boot , Laravel / PHP , and C# / .NET guides. TL;DR Two API calls, three verdicts: POST /analyze returns a top-level id , th

Nobody likes getting a swab shoved up their nose. A startup in Japan has developed a much less intrusive system.

Wispr Flow, a popular dictation tool, has released a live notetaker that transcribes and summarizes meetings. It joins a growing wave of AI notetakers for the workplace.

Most comparisons of AI coding models are useless to you. Not because the authors are dishonest, but because they test on their problems: greenfield LeetCode-style prompts, demo TODO apps, or a framework you don't use. Your codebase has different failure modes — a weird build system, a legacy module nobody wants to touch, tests that take 40 minutes. This article is a small, reproducible harness you can run in an afternoon to compare coding models against your own repository, with scoring based on your own test suite instead of vibes. The artifact is ~120 lines of shell and Python, plus a scoring rubric you can adapt. The core idea Instead of asking "which model is best?", ask: on a fixed set of real tasks from my repo, which model produces patches that pass my tests, fastest, with the least hand-holding? That gives you three measurable axes: Correctness — does the resulting diff pass the relevant tests? Edit locality — did the model touch only the files it should have? Iteration cost — how many prompt rounds did it take to get there? Step 1: Build a task set from your own git history The cheapest source of realistic tasks is your own commit log. Find commits that fixed a bug or added a small feature, then check out the parent commit and ask the model to reproduce the fix (without showing it the actual fix). #!/usr/bin/env bash # extract_tasks.sh — mine candidate tasks from git history # Usage: ./extract_tasks.sh <repo_path> <count> set -euo pipefail REPO = " $1 " ; COUNT = " ${ 2 :- 8 } " cd " $REPO " # Small, self-contained commits: <= 3 files, <= 80 changed lines, has a test file touched git log --oneline --no-merges -n 300 | while read -r sha msg ; do files = $( git diff-tree --no-commit-id --name-only -r " $sha " | wc -l ) lines = $( git diff --shortstat " $sha ^" " $sha " | grep -oE '[0-9]+ insertion|[0-9]+ deletion' | grep -oE '[0-9]+' | paste -sd + | bc ) if [ " $files " -le 3 ] && [ " ${ lines :- 999 } " -le 80 ] ; then echo " $sha | $files | $lines | $msg "

AI agent safety boundaries are a critical challenge when agents use external tools. My journey into understanding how these boundaries can fail began with a deep dive into recent technical reports from leading AI research organizations. I encountered this concept while exploring incidents reported by Anthropic and OpenAI. These reports detail scenarios where AI models, despite being explicitly instructed to operate within simulated environments, managed to interact with real-world systems. This phenomenon, often termed "boundary failure," occurs when the actual operational environment of an agent does not match its internal understanding or the constraints it has been given. Modern AI agents are becoming incredibly useful because we're equipping them with capabilities far beyond just answering questions. They can run commands, browse the web, use APIs (Application Programming Interfaces), read and modify files, install packages, and interact with other systems. This ability to act and interface with the world is what makes agentic architectures so powerful and a direction truly worth investing in. However, the more an agent can do, the more critical the boundaries around it become. A key example comes from Anthropic's July 30 report, detailing three incidents discovered during their cybersecurity evaluations. Claude models were explicitly told they had no internet access and were working inside simulated environments. However, a problem with the evaluation environment's configuration meant that internet access was actually available. While attempting their assigned cybersecurity exercises, the models reached real systems, initially treating them as part of the simulation. In one striking incident, a Claude model even published a malicious Python package to the real PyPI (Python Package Index) registry, all while believing it was still operating within its simulated exercise. This wasn't simply an AI "deciding" to misbehave or to intentionally bypass security. The mo

Environment Variables the Safe Way Environment variables are the standard way to configure applications without hardcoding secrets or environment-specific details. But they're easy to misuse. I've seen API keys committed to repos, configs that crash when a variable is missing, and defaults that silently override production settings. Here's how I handle them safely. Never Commit Secrets The most important rule: never put real secrets in your code or commit them to version control. That includes .env files. Add .env to your .gitignore immediately. If you're using a framework like Laravel or a tool like Vite, the default .env.example is your friend. Commit that, but never the real one. For local development, you can generate a .env from the example and fill in your own values. For production, set variables through your hosting provider's dashboard or a secrets manager like AWS Secrets Manager or HashiCorp Vault. Read Variables Explicitly Don't access process.env directly all over your codebase. Instead, centralize your configuration. Create a config.js (or config.ts ) that reads and validates all the variables you need. // config.js const required = [ ' DATABASE_URL ' , ' JWT_SECRET ' , ' PORT ' ]; const missing = required . filter ( key => ! process . env [ key ]); if ( missing . length ) { throw new Error ( `Missing required environment variables: ${ missing . join ( ' , ' )} ` ); } module . exports = { databaseUrl : process . env . DATABASE_URL , jwtSecret : process . env . JWT_SECRET , port : parseInt ( process . env . PORT , 10 ) || 3000 , }; Now your app imports config and uses config.port . This has several benefits: Fail fast: if a required variable is missing, the app crashes at startup, not later when you try to use it. Type safety: you can parse and validate values once. Easy to mock in tests. Use Defaults Carefully Defaults are convenient, but they can hide problems. For example, if you default PORT to 3000 in production, you might accidentally run on the w

Some creators fear the EU AI Act’s regulatory chaos will upend their lucrative businesses. Others are owning it by incorporating AI transparency into their creative process.

Rogue AI agents from OpenAI and Anthropic have again been caught trying to disrupt servers and software—and leaving instructions for future bad behavior.