标签：#Mac

This article is about running a hand-written Gemma 4 port in pure JAX on three different accelerators, and about the two places the abstraction leaks. The code is here: github.com/xbill9/gemma4-dev What is this project trying to Do? This project aims to serve one Gemma 4 checkpoint from one JAX port across every accelerator I can rent, and to find out — by measurement, not by reading docs — which parts of "it's just JAX" are true. The port lives in ports/gemma4/ and is driven by a generation loop behind an OpenAI-compatible server. No PyTorch, no vLLM, no torch_xla . The same code runs on Cloud TPU v5e and v6e, and on an NVIDIA T4G attached to an AWS Graviton2 host. "Pure JAX" is the whole experiment. If the port is really portable, the only thing that should change between those rigs is a config file. It mostly is. Two things are not, and they are the interesting part. Gemma 4 E2B is not a stock transformer Any port has to carry four irregularities, and none of them are optional: Two attention geometries. Sliding layers use head_dim=256 , global layers use 512 . Most inference stacks assume one head dimension per model. 8:1 MQA , so the KV budget is nothing like the parameter count would suggest. A KV-share map that collapses 35 layers onto 15 caches . A 512-slot sliding ring , plus per-layer embeddings (PLE) held in a 4.70 GB table that gets quantized to 4 bits on load. That first one is worth dwelling on, because it is what breaks other stacks. On the vLLM path, the heterogeneous head dims force the Triton attention backend: Gemma4 model has heterogeneous head dimensions {'sliding_attention': 256, 'full_attention': 512}. FA4 not available, forcing TRITON_ATTN backend. And on a Turing GPU that backend then asks for shared memory the hardware does not have: triton.runtime.errors.OutOfResources: out of resource: shared memory, Required: 98304, Hardware limit: 65536 JAX never enters that conversation. Attention is ordinary XLA rather than a hand-tiled kernel, so ther

This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g instance using pure JAX. The code is here: github.com/xbill9/gemma4-dev What is this project trying to Do? This project aims to serve a modern open model on the cheapest whole CUDA GPU AWS will rent you, and to measure honestly what that costs. Aren't You Using The Wrong GPU? Probably! The T4G is a Turing chip from 2018. It has no bfloat16 and no fp8. But it is cheap, it is available when nothing else is, and it is attached to a Graviton2 host — which makes G5g the rare hardware axis that almost nothing in the ML ecosystem targets: aarch64 and CUDA together . So let's give pure JAX a shot on G5g! AWS EC2 G5g G5g instances pair an AWS Graviton2 (64-bit Arm) processor with NVIDIA T4G Tensor Core GPUs. At g5g.xlarge they are the cheapest EC2 instance carrying a whole NVIDIA GPU , and the only Arm-based GPU family AWS offers. Two GPU instances are cheaper per hour and neither can serve this model (us-east-1, Linux, on-demand, checked against the Pricing API on 2026-08-28): g6f.large at $0.2020 is genuinely NVIDIA and genuinely CUDA — but it is one eighth of a GPU with 3 GB , and the weights alone are 6.155 GB. The first g6f that fits is g6f.4xlarge at $0.9500, which is 1.7x this rig's g5g.2xlarge . g4ad.xlarge at $0.3785 carries an AMD Radeon Pro V520 — no CUDA at any price. Among whole NVIDIA GPUs, G5g is the floor: g5g.xlarge at $0.4200, and the next one up is g4dn.xlarge at $0.5260. More information is available here: https://aws.amazon.com/ec2/instance-types/g5g/ The default in this rig is g5g.2xlarge — 1 GPU, 8 vCPU, 16 GiB RAM. Note- the T4G reports 15,360 MiB of device memory, not the nominal 16 GB. Budget against the measured number. Gemma 4 Gemma is Google's family of open models built from the same research as Gemini. This rig serves google/gemma-4-E2B-it , the instruction-tuned reference release. JAX JAX is Google's array computing library — NumPy semantics, c

Classic Machine Learning Through the Eyes of an SRE — Part 9: Isolation Forest The algorithm in one line: Isolation Forest scores how anomalous a point is by how few random cuts it takes to separate that point from everything else. No model of normal, no loss function, nothing optimized. ← Previous: Part 8 — Hierarchical Clustering Fails Beautifully · Next: this is the series finale — start at Part 1 . Every anomaly detector I had studied models what NORMAL looks like, then calls the leftovers outliers. K-Means: far from every centroid. DBSCAN: in the noise bucket. Sensible, and intuitive. Isolation Forest does not bother. It never models normal at all. It goes straight at the rare points with a single question: how few random cuts does it take to isolate you? Random cuts, literally. Pick a feature at random, pick a split value at random between that feature's min and max, repeat. A point that separates from the crowd in three cuts is anomalous. A point buried in the middle of a dense mass takes thirty. Grow hundreds of these random trees, average the isolation depth for each point, and you get an anomaly score. There is no loss function here. No optimization, not even the local kind that decision trees do at every split. Every cut is a coin flip, and the power comes entirely from averaging, which is the forest trick from the supervised half of this series now applied to pure randomness. Cheap randomness plus averaging beats careful modeling, as long as the target is something randomness naturally exposes. Rarity is exactly that. Sometimes the winning move is to optimize less. That sentence would have gotten me laughed out of my first ML study session. It is also this finale's thesis. The part I had completely backwards Here is the thing I did not know until I read the original paper properly, and it is the opposite of every instinct a decade of ops gave me. Isolation Forest deliberately trains each tree on a small subsample of your data, and this is not a performan

Hey everyone! I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets. GitHub: https://github.com/tanopereira/py-evoFE PyPI: pip install py-evoFE License: MIT The Problem It Solves Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own. Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage. What py-evoFE Does py-evoFE searches the space of possible feature recipes using genetic programming: 1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3)) ). 2. 40+ Built-in Transformers: - Non-linear arithmetic & log-ratios - Target encoding (multiclass, pooled, WoE, quantile target encodings) - String similarity (MinHash, Gap encodings) - Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA) - Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring) 3. Performance & Speed: - Vectorized computation powered by Polars and PyArrow . - Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds. - Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation. 4. Island Model & Caruana Ensembling: - Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration. - Post-search greedy Ca

Which research papers (old or new) do you think a PhD student/early researcher must read to improve their writing skills? Do you have a personal favorite researcher whose papers tend to be well-written, in your opinion? Let's define a "well-written paper" as one that clearly explains the problem it is trying to solve, how the method is developed, and the details of the method, while keeping it easy to understand for a general reader (with a basic knowledge of ML, obviously). Also, post-2015-ish papers usually have nice figures to explain their problem/method, and so they tend to be easier to understand. But I am looking for "well-written papers" in terms of the text. PS: I know the best way to learn writing is by actually writing manuscripts, but I am looking for additional reading resources. submitted by /u/fakeaccountlegitme [link] [留言]

Can an AI make other AIs better? And what stops it from just cheating? Last month, an OpenAI eval agent escaped its sandbox and broke into Hugging Face, apparently to grab test solutions from a benchmark. It's exactly what you'd expect from a system that rewrites agents and reads its own grades. We set out to measure recursive self-improvement anyway, with the exam locked outside its sandbox. We introduce HarnessOpt-Bench, which scores an LLM on how much it improves another agent's harness. On the development split, the optimizer sees per-case traces. Upon validation, it receives a single aggregate score. On test, nothing — until a trusted server scores its final candidate harness. API keys, budget enforcement, and held-out data never enter the optimizer's sandbox. That isolation holds by construction, not by instruction: the held-out evaluator and permission control sit outside the loop that evolves the harness. 5 frontier models, 4 downstream tasks, 111 runs to test 2 hypotheses: 1️⃣ Same coding harness, swap the model: Claude Opus 5 under OpenCode tops 3 of 4 tasks. Walk the releases from Nov 2025 to Jul 2026 on one task, and GPT climbs from 3% to 49% of the headroom, Claude Opus from 37% to 59%. 2️⃣ Same model, swap the coding harness: does a model do best in its own? No consistent home-field edge: opencode beats native harnesses (Claude Code, Codex, Kimi CLI) in 11 of 20 model–task pairs. Model choice moves gains 1.8× more than harness choice. Paper: https://arxiv.org/abs/2608.06301 Code (MIT, built on our team's ICML 2026 VeRO): https://github.com/scaleapi/vero Original post: https://www.linkedin.com/posts/shehabyasser_can-an-ai-make-other-ais-better-and-what-share-7498801902260981760-xuCo/ submitted by /u/shehio [link] [留言]

I put together a small model to estimate NeurIPS acceptance based on scores and an assumed acceptance rate. Try it out here: https://levilingsch.github.io/neurips-acceptance-estimator/ submitted by /u/levydawg [link] [留言]

Hey guys, sorry if this is not the appropriate forum for this question. Is anyone going To ECCV and staying in Lund? Apparently a few days back i saw discounted travel pass available for both Malmo and Lund zone but now today I was going to buy it and the registration site says only Malmo pass. Did ECCV remove them? Because deadline to buy them is 28th august. I dont know why they removed it but the organisation this year feels like a mess. Can anyone access it on their registration site if Malmo Lund passes are available? submitted by /u/Marion-De [link] [留言]

How AI Helps Us Explore the Universe Modern telescopes and space missions generate more data in a single night than a team of human astronomers could review in a lifetime. The Vera C. Rubin Observatory in Chile, for instance, is expected to produce up to seven million alerts every night once it reaches full operational cadence, each one flagging something in the sky that changed since the last image. No group of humans can look at that stream and make sense of it in real time. Machine learning can, and increasingly does. This is the quiet story behind most recent breakthroughs in astronomy: it is not just bigger telescopes, but bigger telescopes paired with models that can filter, classify, reconstruct, and predict faster than any manual pipeline. Here is a tour of where AI is actually doing that work, and why it matters to anyone who writes code. The Data Problem Comes First Space science has quietly become a big data problem. The Rubin Observatory's ten-year Legacy Survey of Space and Time will produce roughly 60 petabytes of raw imagery and catalog around 20 billion galaxies and a similar number of stars. Every image the telescope takes is compared, pixel by pixel, against previous images of the same patch of sky, and any meaningful difference (a moving asteroid, a brightening supernova, a flaring galactic nucleus) triggers an alert within about two minutes of the exposure being taken. That alert stream is too large and too fast for manual triage. So astronomers built software "brokers": machine learning classifiers that sit between the telescope's raw output and the scientists, deciding in near real time which alerts are worth a second look. This is a pattern you will see across almost every domain of modern astronomy: instruments generate more signal than humans can parse, and a model is inserted into the pipeline to do the first pass of filtering. Finding Planets in a Sea of Noise Exoplanets are found mostly through the transit method: a planet passes in front

I created a simple text to image benchmark. I curated 192 prompts that are difficult for T2I models in various ways: text rendering, spatial reasoning, human realism, negations, etc... I then asked a VLM to judge every output against a pre-specified binary question with the ground truth baked in. I'm publishing all the results including the images. (Most public T2I leaderboards don't publish the actual images and that's a shame IMO) There is currently 52 model tested! more than 9k images have been generated and analysed! Full methodology: https://imagebench.ai/methodology-v1 Hugging face dataset: https://huggingface.co/datasets/dh7/imagebench (it contains the prompts to reproduce the results AND the results) Github: https://github.com/dh7/image-bench-ai Gallery to inspect the results: https://imagebench.ai/gallery Leaderboard: https://imagebench.ai/imagebench-v1 Limitations: it's text to image only, and VLM are not perfect as a judge. Let me know what could be useful from there! submitted by /u/dh7net [link] [留言]

Every internal tool I need sits behind SSH. Grafana, Prometheus, the staging clusters, internal AI tooling—none of it answers on a public address, and the only door is a bastion I have a key for. That is the right setup for anything with real data behind it, and I wouldn't change it. What I did change is typing ssh -N -L 3000:localhost:3000 -J bastion prod-1 from memory four times a day across three different machines. So one weekend, I started writing SSH Config Manager . It is a native macOS app that edits ~/.ssh/config without wrecking formatting, saves tunnels as presets, and opens those tunnels in-process instead of shelling out to ssh . I wrote it for my own workflow first. Putting it on the App Store came later, once it was genuinely useful to me and I figured others were struggling with the exact same friction. The VPN Question The first thing people ask is: why not run a VPN and be done with it? Fair question, but the honest answer is that SSH is the tool I already understand inside and out. I have configured sshd enough times to know what PermitRootLogin no and PasswordAuthentication no actually change. When a connection stops working, I can usually name the exact line that broke it. A VPN introduces a whole second network layer underneath, complete with its own credentials, its own background daemon to keep patched, and its own unique failure modes to debug at 2:00 AM when production is down. SSH is already on every Linux server I touch and every developer machine I own — there is nothing new to roll out and nothing new to secure. The tradeoff is real, and I would rather acknowledge it up front. Operating without a VPN means no transparent network routing: every internal service I want to reach must be explicitly forwarded to a local port in advance, and a colleague without my config reaches none of them. Still, I would far rather maintain a clean list of port forwards than maintain another background daemon. Shell Aliases Do Not Survive Three Machines Th

Author here. Ibteda Digital Library is a private community archive in Pakistan — for ten years we digitized rare Urdu books (lithographs, dictionaries, periodicals) on a DIY camera rig, finishing every page by hand in Photoshop. When we wound down daily operations, I realized those 575,729 finished pages across 1,765 books recorded a decade of crop decisions, so I registered them back to their raw photos (SIFT + MAGSAC with conservative acceptance gates) and used the recovered geometry as supervision. The negative results are probably the most interesting part for this sub. Scaling from 378 to 572 training books didn't move unseen-book pass@80 . Neither did ResNet-50 (better training fit, flat held-out, worse after calibration), 1024px inputs, or a spatial head. Per-book error analysis showed why: the failures were near-constant offsets per volume — our operator's preferred margin inset, which simply isn't present in the pixels of a new book. Ten operator-corrected crops per book (element-wise median residual) took pass@80 from 0.71 to 0.83 on held-out volumes. Ten labels beat every scaling lever we tried. For retouching (stain/stamp removal), we kept the neural net to detection only — a U-Net proposes removal support, classical OpenCV reconstructs the paper, and everything outside the mask is byte-identical to the original. Labels used REMOVE/KEEP/IGNORE states, and any erased Urdu diacritic vetoed deployment regardless of IoU. The stricter label cut both improved mark IoU (0.56 → 0.60) and got diacritic false positives to zero. Two things I'd genuinely like input on: (1) has anyone modeled document boundaries that depend on an invisible human preference rather than visible structure — is there prior work on per-instance residual calibration like this? Our own next step is conditioning the model on the calibration examples directly (few-shot inset inference) instead of a post-hoc median. (2) Is there any constrained diffusion/inpainting setup you'd trust to guarant

Have you ever tried to use the classic Gang of Four (GoF) State Pattern in real code? You might have hit a wall. You might have thought, "Wait, this feels way too connected." You are not wrong about that. In school and many engineering interviews, the GoF State Pattern looks great. It promises to fix big, ugly switch statements. But real business rules are hard. When you use this pattern in real life, it can become a huge mess. Every state knows too much about the other states. Let us look at why this happens. We will learn the difference between the GoF pattern and a Finite State Machine (FSM). We will also learn when to use each one. The False Promise of the GoF State Pattern The main idea of the GoF State Pattern is to spread out the work. The main object gives its work to state objects. But there is a catch. The state classes themselves must trigger the change to the next state. Example: The Traffic Light Think about a simple traffic light. It goes Red to Green to Yellow to Red. It does this forever. class RedState implements TrafficLightState { change ( context : TrafficLight ): void { console . log ( " RED light, Stop " ); context . setState ( new GreenState ()); // Very connected! } } The Problem: RedState is forced to know about GreenState . This is fine for a simple traffic light. It is a closed loop. The rules will never change. But what happens when business rules change? Imagine the city council makes a new rule. From midnight to 5:00 AM, the light must flash yellow. Now, you must open your RedState and YellowState classes. You have to add new time checks. You have to add the new flashing state. The more states you add, the messier your code gets. The Better Choice: The Central FSM In the real world, things do not always happen in a straight line. An online order does not just go from Pending to Shipped to Delivered. It can jump from Pending to Cancelled. It can go from Shipped to Returned. If you use GoF here, your PendingState needs to know about many

I've been working on an open-source project called Millwright , an attempt to explore what an end-to-end machine learning workflow could look like in Rust. https://millwright-rs.dev/ This started while I was learning and building ML tooling in Rust. I kept finding capable individual libraries, but also gaps between them. Training a model was rarely the problem. Building the workflow around it — preprocessing, model selection, evaluation, explainability, deployment and monitoring — often meant integrating several unrelated crates and data representations. I initially started implementing some of those missing pieces as smaller independent crates. Eventually I realized I was more interested in the integration problem itself. That became Millwright. The current idea is to cover the classical ML lifecycle: ingest → explore → preprocess → select → fit → assess → explain → export → serve → monitor without trying to reimplement every ML algorithm. Instead, Millwright provides a common abstraction layer over existing Rust libraries and uses adapters for different ML backends. One architectural decision I'm experimenting with is having the framework own a small 2D data boundary ( Frame ) rather than exposing a particular backend's ndarray/dataframe representation throughout the API. That allows models and components backed by different libraries to participate in the same pipeline, at the cost of conversions at backend boundaries. The project currently includes work around: preprocessing and composable pipelines cross-validation and hyperparameter optimization multiple ML backends ensembles regression diagnostics SHAP-based explainability ONNX export model serving and registry drift monitoring time-series workflows incremental learning AutoML There are also Python bindings. I'm not building this on the assumption that Rust should replace Python for ML. Python's ecosystem is enormously more mature, and there would be little value in simply recreating scikit-learn in another l

sklearn 1.9 fixed a bug in how BayesianRidge computes its uncertainty. We traced predict on 1.8 and 1.9 and compared the two formulas it actually computes, see if you can spot what changed before the notebook tells you. https://github.com/aadya940/scikit-verify/blob/master/examples/sklearn_bug_hunting.ipynb submitted by /u/Lost-Dragonfruit-663 [link] [留言]

The same zsh script could list ~/Documents when I ran it in Terminal. Started as a LaunchAgent, it failed with: ls: /Users/administrator/Documents: Operation not permitted The LaunchAgent had the same user ID, the same $HOME , and the same script. That combination makes this look like a Unix permission problem. In this test it was not. The useful discriminator was the launch context: access succeeded from Terminal, failed from launchd , and still succeeded for a path outside the protected folder. I reproduced this on macOS 15.6.1 (Darwin 24.6.0) with a LaunchAgent in gui/501 . The probe was removed after the test. Why chmod is the wrong first check The obvious suspects were file ownership, a wrong home directory, or a job running as another user. The probe printed those facts before touching the files: #!/bin/zsh print -- "user= $( id -un ) uid= $( id -u ) " print -- "home= $HOME pwd= $PWD " /bin/ls " $HOME /Documents" 2>&1 | /usr/bin/head -5 /bin/cat " $HOME /Documents/vinh/working/CLAUDE.md" 2>&1 | /usr/bin/head -1 # Negative control: outside Documents /bin/ls " $HOME /.pf004" 2>&1 | /usr/bin/head -5 The two runs produced this difference: Check Terminal LaunchAgent in gui/501 User / uid administrator / 501 administrator / 501 $HOME /Users/administrator /Users/administrator ls ~/Documents Listed entries Operation not permitted cat inside ~/Documents Read the file Operation not permitted ls ~/.pf004 Listed entries Listed entries The working directory differed, but the script used absolute paths under $HOME , so PWD=/ did not explain the denial. The negative control mattered more: the LaunchAgent could read another directory owned by the same user. Changing ownership or mode bits would not explain why only the launch context changed the result. The owning layer is the privacy context On this machine, the access decision was attached to how the process was launched, not just to uid 501. Terminal had a privacy context that allowed access to the user's Documents folder.

When someone asks whether to use Azure OpenAI Service or the direct OpenAI API, the starting point is this: the models running on both platforms are identical. GPT-4o, GPT-5, and the o-series models you deploy on Azure have the same weights, the same capabilities, and the same output quality as the ones you call from platform.openai.com, and what changes between the two platforms is the infrastructure where they run, the authentication mechanism, and the compliance guarantees the provider can offer on those requests. What changed in 2026 Azure AI Foundry was renamed Microsoft Foundry on January 1, 2026, and Azure OpenAI Service now lives inside that unified platform alongside the model catalog, development tooling, and agents. References to Microsoft Foundry in new documentation point to what used to be Azure AI Foundry. In July 2026, the GPT-5.6 family arrived with Sol, Terra, and Luna available on Azure the same day as on the direct OpenAI API. Historically Azure lagged four to eight weeks behind new model releases because Microsoft validates them within their compliance frameworks before making them available, and while that gap still exists for some specific features and APIs, for the main models in the GPT-5 family availability is converging. Where data is processed When you call GPT-4o from the OpenAI API, the request goes to OpenAI's own infrastructure, which is centralized and gives you no control over which region processes your data. For most use cases that doesn't matter, but for organizations with data residency requirements, regulatory compliance needs, or industries like healthcare, banking, or government, that detail can determine whether the service is usable at all. Azure OpenAI runs the same models within the boundary of your Azure tenant, so the data you send in prompts doesn't leave to OpenAI's infrastructure but processes in the Azure regions you choose. That's what makes it possible to meet HIPAA, SOC 2, EU data residency, and other certificati

Foundation models are expensive. A trading agent that calls GPT-4 for every price prediction burns budget fast. Lightweight forecasters are cheap to run but expensive to train, especially when you only have a handful of examples. MetaCaster introduces a meta-harness architecture where agents don't forecast directly. Instead, they train specialized lightweight models on-demand from few-shot examples and textual context. This is not another AutoML wrapper. The meta-agent orchestrates data generation, architecture selection, and training loops to produce task-specific forecasters in minutes. The result is a deployable model that runs inference without touching the foundation layer again. The Economic Gap Time-series forecasting in production faces a resource trap: Foundation models (TimeGPT, Chronos) deliver strong zero-shot performance but cost $0.002 to $0.02 per prediction at scale. Lightweight forecasters (PatchTST, DLinear, FEDformer) run for pennies but need thousands of training samples and hours of GPU time. Few-shot scenarios (new trading pairs, emerging markets, privacy-sensitive health data) don't have enough history to train from scratch. MetaCaster targets the intersection: resource-constrained environments where you need specialized models but can't afford foundation API calls or long training cycles. Meta-Harness Architecture The system has three layers: 1. Meta-Agent Orchestrator The top-level agent receives a few-shot time series (as few as 5-10 examples) and optional textual context (domain descriptions, seasonality hints). It decides: Which lightweight forecaster architecture to instantiate (PatchTST, DLinear, Autoformer, etc.) What synthetic data generation strategy to apply How to configure the training harness (learning rate, epochs, augmentation) The meta-agent uses a learned policy, not heuristics. It's pre-trained on a meta-dataset of diverse forecasting tasks so it generalizes to new domains. 2. Data Generation Agents These agents expand the f

Hi everyone, I’m researching how to design an AI agent for a medicine-reminder system. The agent has to decide, at each relevant time, whether to: send a reminder, wait (do nothing for now), or notify another person (e.g. caregiver), when it does not have complete information about the patient (has the dose already been taken? is the person nearby/attentive? are there adherence barriers? etc.). I’m trying to frame this properly before diving into implementation. Right now I’m looking at it as a sequential decision problem under partial observability (POMDP / belief-state RL territory), but I’m not sure how far that framing is actually useful in practice for this kind of system. I’d really appreciate any pointers on: Is a POMDP / belief-state approach overkill here, or is it the right formalization? What simpler alternatives (contextual bandits, MDP with engineered features, rule-based + uncertainty thresholds, etc.) have people used successfully for similar “remind vs wait vs escalate” decisions? Papers, open-source projects, or real systems that tackle medication adherence / context-aware reminders with uncertainty or incomplete observations. Common practical pitfalls (reward design, observation noise, alert fatigue, safety/escalation logic, evaluation metrics) that aren’t obvious from the theory. Any recommended starting points for someone new who wants to move from “I understand the concepts” to a small working prototype or simulation. I’m mainly in research/preparation mode right now, so even high-level advice, key papers, or “here’s what I’d do differently” comments would be very helpful. Thanks! submitted by /u/Senior_Disaster_7307 [link] [留言]

FP8 gave us a clean 1.5x on Qwen3-8B serving throughput on an RTX PRO 6000 Blackwell (1,725 to 2,597 tok/s at concurrency 32, vLLM). The uncomfortable question is always the same: did the model get dumber. This post is the exact check we ran before recommending the switch, with numbers, so you can run the same one. Why "run an eval suite" is usually the wrong first answer Standard benchmarks (MMLU and friends) are noisy instruments for quantization deltas at 8B scale. Score movement inside the error bars tells you nothing about whether YOUR prompts changed behavior. What you actually want to know is narrower: on the workload you serve, does the FP8 checkpoint produce materially different outputs than BF16, and are any of the differences wrong. That is answerable directly, cheaply, and per prompt. The method Both configurations run the same fixed workload: 20 prompts covering reasoning, code, summarization, translation, extraction, classification, math, and instruction following. Greedy decoding, temperature 0, 256-token cap, streamed. Greedy matters: it removes sampling noise, so any output difference is attributable to the numerics. Then a three-stage comparison: Byte equality. outputs_bf16[i] == outputs_fp8[i] . Anything identical is settled. Similarity triage. For non-identical pairs, difflib.SequenceMatcher.ratio() sorts near-identical wording drift from real divergence. Side-by-side review under a written rubric. Every non-identical pair gets read. The rubric asks one question: is there a factual or numerical claim that one precision gets right and the other gets wrong. Wording changes, reordering, and equally-defensible readings are recorded but not counted as regressions. The core loop is small: import difflib , json bf16 = json . load ( open ( " vllm_bf16_conc1.texts.json " )) fp8 = json . load ( open ( " vllm_fp8_conc1.texts.json " )) for i , ( a , b ) in enumerate ( zip ( bf16 , fp8 )): if a == b : print ( i , " identical " ) continue r = difflib . Sequenc