标签：#rails

LLM Evaluation System Prompts Scored Rubrics Runtime Guardrails: A Practical Guide for Production Learn how to evaluate LLM outputs in production using system prompts, scored rubrics, and runtime guardrails to prevent hallucinations and ensure quality. TL;DR: To evaluate LLM outputs in production, combine system prompts that define evaluation criteria, scored rubrics using LLM-as-a-judge for dimensions like correctness and relevance, and runtime guardrails that filter or flag unsafe outputs. This approach scales better than human review, adapts via prompt changes, and catches failures that status codes miss, as seen in the Air Canada chatbot case. Why Production LLM Evaluation Demands More Than Status Codes A 200 status code only confirms the server processed the request—it says nothing about whether the generated text is factual, safe, or useful. The Air Canada chatbot that invented a non-existent bereavement discount returned perfectly valid HTTP responses, yet the hallucinated policy led to a tribunal ruling against the airline. Production evaluation must therefore separate operational health (latency, error rates) from output quality (correctness, relevance, harmlessness). Consider a typical API call that succeeds operationally but fails qualitatively: import requests response = requests . post ( " https://api.example.com/v1/chat " , json = { " model " : " gpt-4o " , " messages " : [{ " role " : " user " , " content " : " What is Air Canada ' s bereavement policy? " }]}, headers = { " Authorization " : " Bearer $KEY " } ) print ( response . status_code ) # 200 print ( response . json ()[ " choices " ][ 0 ][ " message " ][ " content " ]) # Output: "Air Canada offers full refunds for bereavement-related cancellations..." A 200 status code and a well-formed JSON body mask a completely fabricated policy. To catch this, you need a separate evaluation layer that scores the output against a rubric. LLM-as-a-judge is a common approach, using a second model to assess the

"Context bombing" tricks hacking agents into shutting down before they can do harm.

There's a comfortable thing people say when they see an AI agent query a code map. "Nice crutch. For now." The logic underneath it is reasonable. Coding agents are young. Context windows are small and getting bigger. Models are dumb today and will be smart tomorrow. So a structural index, the thing that hands the agent a dependency graph it would otherwise have to reconstruct, looks like a patch over a temporary weakness. Wait two releases. The model will just hold the whole repo in its head and the map becomes a quaint workaround, like a spellchecker for someone who learned to spell. I build one of those maps Sense . I went looking for the data that would kill it. I didn't find it. I found the opposite. What a map hands an agent is a computed fact. What a better model hands you is a more confident guess . No amount of model progress turns the second into the first, because the difference between them isn't a quality gap that closes with scale. It's a difference of kind. The rest of this piece is the two findings that forced me there. The belief, stated fairly The claim at full strength, because a weak version is easy to knock over. A code map exists to compensate for what the model can't do yet. Today's agent greps, samples, and guesses at structure because it can't read the whole codebase at once. Tomorrow's agent reads all of it, reasons over all of it, and the guessing stops. Bigger windows plus better weights equal no more blind spots. The map is scaffolding you'll tear down once the building stands. If that's true, the right move is to skip the tool and wait. Both findings, in order. Proof one: the best model available was still blind The benchmark ran the same task on thirteen real Ruby repos. Pick the hub model of an app, the Inbox , the MergeRequest , the Spree::Order , and ask the agent to find every place that depends on it before a teardown change. The non-obvious dependents, the ones scattered through concerns and workers and config-string registries, w

You've seen the claim in every Ruby thread for the past year. Ruby and Rails are the most AI-friendly stack. Fewer tokens, less hallucination, the model just writes it cleanly. Half of that claim I'll concede without a fight . The other half I measured, across thirteen real Ruby codebases, and that's where a line shows up, sharp enough to put every repo on one side or the other. Including yours. The half that's true: writing Ruby is solved Start with the part that holds up, because it really does. A model that has seen ten thousand Rails apps knows where the model lives, where the job goes, what a concern does, what has_many implies, before it reads a line of yours. Convention over configuration was always written partly for the next human reading the code. It turns out the model is the next reader too, and the conventions answer half its questions before it asks them. So "write me a service object," "add a scope," "refactor this controller"? The stack carries the model. Fewer wrong guesses, tighter loops, less to hallucinate because the shape is already known. Anyone who builds on Rails has lived this, and the AI-friendly reputation earned it. I'm not here to take that away. I'm here to point out it answers a question nobody dangerous is asking. The half that isn't: navigating Ruby at scale "Can AI write Ruby" is settled. The question that ships broken deploys is different: can AI navigate Ruby? What breaks if this model changes, who depends on it, where the blast radius ends. Reading and navigating feel like the same skill when you're fluent. They are not the same skill for an agent. Reading a file is local, the answer is right there in the text. Navigating is structural, the answer lives in the edges between files, what calls what, what breaks what, and no single file contains it. So I ran the structural question on all thirteen repos. Same task each time: take the hub model, the Inbox , the MergeRequest , the Spree::Order , and find every dependent before a tear

Every application we use today—from banking apps to social media platforms—has something working behind the scenes. That hidden engine is called the backend. The backend is responsible for processing requests, storing data, handling authentication, enforcing business rules, and ensuring everything works as expected when users interact with an application. One of the first decisions backend developers make is choosing a framework. A framework provides the tools, structure, and best practices needed to build applications faster and more securely. Today, let's look at three popular backend frameworks: Django, Gin, and Ruby on Rails. Django (Python) Django is one of the most mature and feature-rich backend frameworks available. Built using Python, it follows the philosophy of "batteries included." This means many features developers need are already built into the framework, including: User authentication Admin dashboard Database ORM Security protections URL routing Form validation Because so much comes ready to use, developers can spend more time solving business problems instead of rebuilding common features. Best for: Content management systems E-learning platforms Business applications APIs Startups building products quickly Advantages: Fast development Excellent security features Large community Extensive documentation Scales well for many applications Trade-offs: The framework includes many components, so it can feel heavier than minimalist frameworks. Gin (Go) Gin is a lightweight web framework built for the Go programming language. Unlike Django, Gin keeps things minimal. It gives developers speed and flexibility while letting them choose many of the additional tools they want to use. One reason many developers enjoy Gin is its impressive performance. Since Go is a compiled language designed for concurrency, Gin can efficiently handle many requests simultaneously while using relatively few system resources. Best for: REST APIs Microservices High-performance syst

Pagination is a key component on web-applications that let users navigate through pages making easy to read/find records. Also, pagination is a great strategy to improve performance by avoiding to load entire dataset at once. However, while working with Kaminari, a popular pagination gem in Rails, I encountered an unexpected issue that revealed an interesting edge case. Identify the issue Basically, pagination in frontend was not working properly. Datatable should load 316 total rows, although when the user started to load 15 records per page, frontend is showing inaccurate total rows. Multiple of 15 should ends at 0 or 5. There were pages with 64 records, crazy world. Some pages were loading 12 or 11 rows. There is no issues or error messages in the frontend or backend. Lost in debugging-land After discarding Angular frontend errors, I started to dig into backend controller and Kaminari configuration. Nothing seems wrong. Everything looked good: test suite, smoke tests, desktop debugging. Despite of test results, I started to wonder: what if returned-data is wrong after all? and... Bingo! Finally, after checking every response I noticed that there were duped records in two different pages(pagination requests). Those duped records were skipped from Angular data-table and that's why loaded/total rows did not match. Bingo: a sort of mistake This tricky bug has a simple explanation: bad sorting. Kaminari uses a SQL query using LIMIT/OFFSET strategy: SELECT * FROM posts ORDER BY id LIMIT 25 OFFSET 0 ORDER BY : sort the collection. LIMIT : number of records per page. OFFSET : is used to skip a specified number of rows before starting to return rows from a query. This works perfectly using ORDER BY id because primary key is unique. Check table A. id title body created_at lock 101 Welcome to the Platform First post introducing the new platform features. 2026-06-16 08:15:22 false 102 Summer Update Announcing the latest improvements and updates. 2026-06-16 08:15:22 true 103

Imagine a user trying to upload a 100MB video or a high-resolution photo to your app. If you use the standard Rails file upload, that file travels from the user's browser to your Rails server, and then your server sends it to S3 or Google Cloud. This is a terrible way to do it. While that 100MB file is transferring, your Rails worker (Puma) is frozen. It can't handle other users. If three people upload large files at once, your whole app will stop responding. In 2026, the professional way to handle this is Direct Uploads . With Direct Uploads, the file goes directly from the user's browser to your cloud storage (S3, R2, etc.). Your Rails server only handles a tiny bit of metadata. It is faster for the user and much safer for your server. Here is how to set it up in Rails 8. STEP 1: Configure Your Storage First, make sure you aren't using the local disk for production. You need a cloud provider like AWS S3 or Cloudflare R2. In your config/storage.yml : amazon : service : S3 access_key_id : <%= ENV['AWS_ACCESS_KEY_ID'] %> secret_access_key : <%= ENV['AWS_SECRET_ACCESS_KEY'] %> region : us-east-1 bucket : my-app-uploads # Crucial for Direct Uploads! public : true Note: You must configure CORS in your S3/R2 dashboard to allow requests from your domain. If you don't do this, the browser will block the upload. STEP 2: The Rails Form Rails makes the backend part incredibly easy. You just add one attribute to your file field: direct_upload: true . <!-- app/views/users/_form.html.erb --> <%= form_with ( model: user ) do | f | %> <div class= "field" > <%= f . label :avatar %> <%= f . file_field :avatar , direct_upload: true %> </div> <%= f . submit "Save Profile" %> <% end %> When you add direct_upload: true , Rails automatically includes a JavaScript library that handles the "handshake" with S3. STEP 3: Adding a Progress Bar (The UX Win) Direct uploads can take a few seconds. If nothing happens on the screen, the user will think your app is broken. We can use the built-in Ac

Guardrails positions itself as a populist political movement that runs on small donations from people in the trenches of the AI boom.

Rails GuardDog: Advanced Security Scanner for Rails Introduction Today I'm excited to announce Rails GuardDog v0.1.0 — an open-source security scanner for Rails that goes beyond traditional tools like Brakeman. While Brakeman is excellent for catching basic Rails vulnerabilities, Rails GuardDog focuses on newer vulnerability classes that most tools miss: AI/LLM prompt injection, DoS/ReDoS patterns, supply chain attacks, and more. The Problem Modern Rails applications face new security challenges: AI/LLM Integration - How do you prevent prompt injection when integrating with ChatGPT, Claude, or Anthropic? ReDoS Attacks - Catastrophic backtracking in regex can bring down your app Supply Chain Attacks - Typosquatted gems that look like popular libraries IDOR Gaps - Objects accessible without proper authorization checks Advanced Secrets - Hardcoded API keys that Brakeman misses Rails GuardDog detects all of these. What is Rails GuardDog? Rails GuardDog is a lightweight gem that adds comprehensive security scanning directly to your Rails applications. 12 Security Checkers SQL Injection - String interpolation in queries XSS - Unescaped output in views CSRF - Disabled protection verification Mass Assignment - permit! vulnerabilities (fixes Brakeman #1942, #1918) Open Redirect - User input in redirects Hardcoded Secrets - API keys, tokens, passwords (always-on, fixes #1989) DoS/ReDoS - Unbounded queries, dangerous regex patterns IDOR - Object access without authorization AI/LLM Prompt Injection - User input flowing to LLMs Rate Limiting - Missing rack-attack configuration Supply Chain - Typosquatted gems using Levenshtein distance GraphQL - Missing field-level authorization Features 📊 Multiple report formats : Console, HTML, JSON 🔍 AST-based analysis : Uses parser gem for deep code understanding ⚡ Async support : Built-in Sidekiq integration 📈 Zero dependencies : Only requires parser and ast gems 🚀 Production-ready : Tested and battle-ready 📝 CWE/OWASP mappings : Every find

My Previous Deployment Choices Since I mostly do web development using Ruby on Rails, these were my go-to options for deployment: PaaS like Heroku, Render, or Railway Serverless setups (Cloud Run + NeonDB) Honestly, I didn’t have any major complaints. PaaS costs a bit more, but in return, you get a clean UI and dead-simple workflows like GitHub integration. If the cost bothered me, I’d just go serverless. For my personal servers—where huge traffic isn't exactly a concern—going serverless meant the app would just sleep when inactive, allowing me to run services for around 50 yen a month. Compared to the headache of clicking through complex AWS or GCP dashboards to piece things together based on architecture diagrams, it was a walk in the park. I was perfectly content. Seriously. Kamal Became the Default in Rails 8 Everything changed when Rails 8 dropped. I heard they adopted Kamal as the official deployment tool. Kamal — Deploy web apps anywhere From bare metal to cloud VMs using Docker, deploy web apps anywhere with zero downtime. kamal-deploy.org Kamal? Is deploying really going to get any easier? I mean, I’m doing completely fine right now, though... That’s what I thought. But once I gave it a shot, it felt like being struck by lightning. This is an absolute game-changer. All you have to do is run rails new , throw your server's IP address into deploy.yml , and run kamal setup . That’s it—your app is deployed. For every release after that, it's just kamal deploy . I couldn't believe how simple the deployment workflow was. # deploy.yml service : my-app image : my-user/my-app servers : web : - 192.0.2.1 # Just swap in your VPS IP address here proxy : ssl : true host : app.example.com # Set up your domain here Sure, you have to bring your own server, but Kamal prides itself on being able to deploy absolutely anywhere. I rented a couple of VPS instances from Hetzner for about $10 a month each to host SuperRails and LazyCafe . From what I looked into, you can't really

Anyone who works with Ruby on Rails knows that, despite the framework being incredible for productivity, there are some classic workflow deficiencies that haunt almost every project. You are focused on writing code, but suddenly you need to open an external tool like Postman to test a route. Then, you run a complex script to generate a static database diagram. And at the end of the day, you still need to manually update the API documentation, which will inevitably become outdated in the next sprint. This constant context switching and manual maintenance generates enormous friction. To cover these deficiencies, I developed three gems that bring these tools inside your application. They are so practical that they quickly become indispensable in any Rails project. Meet each one of them: 1. rails-api-docs : The End of Outdated Documentation The deficiency: API documentation always starts with good intentions, but as the system evolves—new routes, parameters, and response fields—it quickly stops representing reality. Keeping this updated manually is repetitive and frustrating work. The solution: The rails-api-docs gem solves this by leveraging what Rails already knows. It inspects your routes, controllers (via AST analysis using Prism), and the ActiveRecord schema to automatically generate the first draft of your documentation. Everything is saved in a single YAML file ( config/rails-api-docs.yml ), which serves as the single source of truth. Why it is indispensable: Append-only strategy: When adding new routes and running the generator, the gem only appends what's new. Your descriptions, custom examples, and tags are never modified or deleted, making the documentation a living document. Zero development friction: You edit the YAML in one window and view the updated documentation in the browser at localhost:3000/rails/api-docs instantly, with no build step required. For production, it exports a single static HTML file without any external dependencies. 2. rails-http-lab

My talk “Frontend Ruby on Rails with Glimmer DSL for Web” went well at RubyConf Austria 2026 . Especially given that after the talk, Chad Fowler (the starter of RubyConf and famous book author of The Passionate Programmer , among other books) told me “good job”, and Obie Fernandez (a famous entrepreneur and book author of The Rails Way , among other books) told me he will try Glimmer DSL for Web because he doesn’t like React.js. Presentation Slides Direct Original Long Link: https://docs.google.com/presentation/d/e/2PACX-1vQ9oBnZpzK_eicVLGSqDmVzhsXsblONEKepnw5_xGHGXTM52JSjaS_ObYUJbx-zkb1M2ul9N2A2MnvU/pub?start=false&loop=false&delayms=60000&slide=id.g140fe579a5a_0_0 Glimmer DSL for Web GitHub: https://github.com/AndyObtiva/glimmer-dsl-web I ran a poll at the beginning of my talk, and everyone agreed that they love Ruby and that Ruby is superior to JavaScript, plus the majority indicated that they’d like to write less JavaScript and more Ruby during their Rails web development work. Several attendees told me my talk was great after the talk. Charles Nutter had me help him with his JRuby workshop afterwards by showcasing my other Glimmer project, Glimmer DSL for SWT , which runs on JRuby. In about 1 minute, I scaffolded a Hello World desktop app from scratch and then packaged it as a native executable on the Mac. Attendees were impressed. So, I’ve participated in presenting 2 events at this conference. I am very grateful for having such a great experience at RubyConf Austria 2026 overall, especially given that it uniquely included several classical/neoclassical/jazz concerts in between talks that entertained us and relaxed us. Chad Fowler concluded the conference with a beautiful Jazz piano and sax performance. Shout out to Hans Schnedlitz, Muhamed Isabegovic, and Zuzanna Kusznir (plus everyone who helped out) for organizing and hosting such a special Ruby conference!!! Original blog post version: https://andymaleh.blogspot.com/2026/06/rubyconf-austria-2026-frontend-r

--- title : " Introducing rails-tenantify: Row-Level Multi-Tenancy for Rails 7+" published : true description : " A modern, safe, and robust row-level multi-tenancy gem for Ruby on Rails. Prevent data leaks, protect bulk writes, and preserve tenant context in background jobs." tags : rails, ruby, opensource, saas --- ## The Problem Every multi-tenant SaaS app eventually needs to answer the same questions: * How do we make sure School A never sees School B's data? * How do we scope every query to the right organization? * How do we keep tenant context alive in background jobs and Sidekiq retries? * How do we stop a careless `update_all` from wiping another tenant's rows? The typical answer is *"use acts_as_tenant"* or *"switch to Apartment."* But in modern Rails development, that often means: * Fighting unmaintained APIs on Rails 7+ * Losing tenant context when a background job retries * Dealing with schema-per-tenant complexity (Apartment) and heavy DevOps overhead * Rolling your own `default_scope` and crossing your fingers that nobody calls `unscoped` For most Rails apps, you just need **row-level tenancy** : one database, one `organization_id` column, and strict scoping. The pattern is simple. Getting it **safe** in production is not. --- ## What I Built **`rails-tenantify`** is a Ruby gem that adds row-level multi-tenancy directly to your Rails models and controllers. No external services, no extra databases per tenant—just your own PostgreSQL (or SQLite in dev). ruby class Project < ApplicationRecord include Tenantify::Scoped belongs_to_tenant :organization end ### Set the tenant once per request ruby class ApplicationController < ActionController::Base set_tenant_by :subdomain # acme.yourapp.com → Organization end ### Everything scopes automatically ruby Tenantify.current_tenant = current_organization Project.all # Only this org's projects Project.create!(name: "Q2 Roadmap") # organization_id is set automatically ### Switch context safely for admins or scripts