标签：#learning

I'm fairly new to SRE/DevOps, and one of the topics I recently spent time studying properly was EKS cluster upgrades . My first instinct, like most people starting out, was: "it's just a version bump, click upgrade in the console, done." That's basically what most beginner blog posts say too. But the more I read and the more I dug into real-world postmortems and discussions, the more I realized — the actual Kubernetes control plane upgrade is the easy part. Almost everything that can go wrong seems to happen around it, not because of it. Sharing what I learned here, mainly for my own notes, but hoping it's useful for anyone else early in their journey too. Learning #1: There's No "Undo" Button This was the first thing that surprised me. I assumed upgrades work like most software — if something breaks, you roll back. But with EKS, you cannot downgrade the control plane version once you upgrade it. So the plan can't be "upgrade, and if it breaks, revert." It has to be "test enough beforehand that breaking isn't really an option," and if something does go wrong, the fix is always moving forward, not backward. That single fact changes how you're supposed to approach the whole thing — testing has to happen before the button is clicked, not after. Learning #2: APIs Get Deprecated, and It's Usually Not Your Own Code That Breaks Kubernetes removes old API versions on a schedule. I already knew this conceptually, but what I didn't realize is that the risk usually isn't your own YAML files — it's the Helm charts and third-party tools you installed a while back and forgot about , which might still be using an older API version internally. There are tools built exactly for catching this before it becomes a problem: pluto detect-helm -owide pluto detect-files -d ./manifests kubent (kube-no-trouble) does something similar. I hadn't heard of either tool before researching this, and it made me realize how much of "being good at Kubernetes" is really just knowing which small tools e

Note: This tutorial was originally published on effessdev.github.io . Check out the original article for the most up-to-date version: https://effessdev.github.io/posts/2026-07-27/ This is a step-by-step tutorial that explains how you can set up your development environment for working with ESP-IDF projects in VS Code . Install ESP-IDF Install EIM Espressif Systems provides a graphical tool called EIM (ESP-IDF Installation Manager) to install ESP-IDF. Click the link below to go to the official page to download EIM: https://dl.espressif.com/dl/eim/ Make sure you are in the "Online Installer" tab. The exact file to download depends on your system: Windows: Download eim-gui-windows-x64.exe . Run this installer to install EIM. Linux x64 (Ubuntu): Download and install the .deb package ( eim-gui-linux-x64.deb ). Install ESP-IDF using EIM Now that we have installed EIM, let's install ESP-IDF using it. Open EIM. Under "New Installation" click "Start Installation". Under "Easy Installation", click "Start Easy Installation" to install the latest stable version of ESP-IDF with default settings. If there are no problems, you will see the "Ready to Install" page. Click "Start Installation". Install ESP-IDF VS Code Extension We use this extension as a high-level wrapper for ESP-IDF. Most times, we do not use ESP-IDF directly. For example, if we need to compile our source code, we ask the extension to do it, which uses the ESP-IDF we just installed internally to to compile the source code. Install the extension named "ESP-IDF" by "Espressif Systems" in VS Code. Verify installation After installing, restart VS Code. Use the shortcut Ctrl + Shift + P to open the command palette (remember this shortcut, we are going to use it a lot). Inside the command palette, search ESP-IDF . You will see many entries which start with ESP-IDF: . Those commands are provided my the ESP-IDF extension. These commands are what we use for almost everything. Note If you are not in an ESP-IDF project, you m

This article is about running a hand-written Gemma 4 port in pure JAX on three different accelerators, and about the two places the abstraction leaks. The code is here: github.com/xbill9/gemma4-dev What is this project trying to Do? This project aims to serve one Gemma 4 checkpoint from one JAX port across every accelerator I can rent, and to find out — by measurement, not by reading docs — which parts of "it's just JAX" are true. The port lives in ports/gemma4/ and is driven by a generation loop behind an OpenAI-compatible server. No PyTorch, no vLLM, no torch_xla . The same code runs on Cloud TPU v5e and v6e, and on an NVIDIA T4G attached to an AWS Graviton2 host. "Pure JAX" is the whole experiment. If the port is really portable, the only thing that should change between those rigs is a config file. It mostly is. Two things are not, and they are the interesting part. Gemma 4 E2B is not a stock transformer Any port has to carry four irregularities, and none of them are optional: Two attention geometries. Sliding layers use head_dim=256 , global layers use 512 . Most inference stacks assume one head dimension per model. 8:1 MQA , so the KV budget is nothing like the parameter count would suggest. A KV-share map that collapses 35 layers onto 15 caches . A 512-slot sliding ring , plus per-layer embeddings (PLE) held in a 4.70 GB table that gets quantized to 4 bits on load. That first one is worth dwelling on, because it is what breaks other stacks. On the vLLM path, the heterogeneous head dims force the Triton attention backend: Gemma4 model has heterogeneous head dimensions {'sliding_attention': 256, 'full_attention': 512}. FA4 not available, forcing TRITON_ATTN backend. And on a Turing GPU that backend then asks for shared memory the hardware does not have: triton.runtime.errors.OutOfResources: out of resource: shared memory, Required: 98304, Hardware limit: 65536 JAX never enters that conversation. Attention is ordinary XLA rather than a hand-tiled kernel, so ther

This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g instance using pure JAX. The code is here: github.com/xbill9/gemma4-dev What is this project trying to Do? This project aims to serve a modern open model on the cheapest whole CUDA GPU AWS will rent you, and to measure honestly what that costs. Aren't You Using The Wrong GPU? Probably! The T4G is a Turing chip from 2018. It has no bfloat16 and no fp8. But it is cheap, it is available when nothing else is, and it is attached to a Graviton2 host — which makes G5g the rare hardware axis that almost nothing in the ML ecosystem targets: aarch64 and CUDA together . So let's give pure JAX a shot on G5g! AWS EC2 G5g G5g instances pair an AWS Graviton2 (64-bit Arm) processor with NVIDIA T4G Tensor Core GPUs. At g5g.xlarge they are the cheapest EC2 instance carrying a whole NVIDIA GPU , and the only Arm-based GPU family AWS offers. Two GPU instances are cheaper per hour and neither can serve this model (us-east-1, Linux, on-demand, checked against the Pricing API on 2026-08-28): g6f.large at $0.2020 is genuinely NVIDIA and genuinely CUDA — but it is one eighth of a GPU with 3 GB , and the weights alone are 6.155 GB. The first g6f that fits is g6f.4xlarge at $0.9500, which is 1.7x this rig's g5g.2xlarge . g4ad.xlarge at $0.3785 carries an AMD Radeon Pro V520 — no CUDA at any price. Among whole NVIDIA GPUs, G5g is the floor: g5g.xlarge at $0.4200, and the next one up is g4dn.xlarge at $0.5260. More information is available here: https://aws.amazon.com/ec2/instance-types/g5g/ The default in this rig is g5g.2xlarge — 1 GPU, 8 vCPU, 16 GiB RAM. Note- the T4G reports 15,360 MiB of device memory, not the nominal 16 GB. Budget against the measured number. Gemma 4 Gemma is Google's family of open models built from the same research as Gemini. This rig serves google/gemma-4-E2B-it , the instruction-tuned reference release. JAX JAX is Google's array computing library — NumPy semantics, c

Hello! I'm a beginner developer with my sights set on backend development and data modeling. Like a lot of people starting out, I didn't come in with a computer science degree or years of professional experience — just curiosity about how applications actually store, organize, and make sense of data behind the scenes. Backend work has always felt like the "engine room" of software to me. While frontend gets the visual credit, it's the data layer that quietly decides whether an application is fast, reliable, and able to grow. That's what pulled me toward backend and database design in the first place. My biggest challenge so far has been learning SQL and data modeling from scratch. It sounds simple on paper — write some queries, design some tables — but in practice it meant rewiring how I think. I had to move from "how do I make this work right now" to "how do I structure this so it still works when the data grows, the requirements change, or someone else has to read my schema six months from now." Concepts like primary keys, foreign keys, relationships between tables, and eventually normalization weren't hard to memorize, but they were hard to internalize — to actually reach for instinctively when designing something from a blank page. A few things clicked for me along the way: A good schema is a form of communication. Table and column names, relationships, and constraints tell a story about the business logic, not just the data. Getting it "perfectly right" on the first try isn't the goal. Iterating on a design after seeing how data actually flows through it taught me more than any tutorial did. SQL rewards precision. Small differences — a missing JOIN condition, the wrong key, an unindexed column — can quietly break correctness or performance, so being deliberate matters. Constraints are a beginner's best friend. Things like NOT NULL, UNIQUE, and foreign key constraints catch mistakes early instead of letting bad data pile up silently. This foundation in SQL and d

I’ve been working on a desktop utility called Ripper, a Python + CustomTkinter app that downloads video and audio from supported sites (starting with YouTube). The app itself has been a bit rough to build and maintain — but distributing it has been the annoying part. GitHub won’t host my repository, let alone the EXE, due to there size and I don’t want to rely on sketchy file hosts or temporary mirrors. So I finally figured out a temporary setup that’s stable and easy for users to follow. This post explains the distribution workflow and why I’m using it. Why I’m Using this Approach The EXE and source code are too large to push to GitHub, even when the ffmpeg EXE is zipped, and one of my main goals is that I don't want the user to have to hassle with getting ffmpeg. So, I set up a public Google Drive folder where users can get the zipped EXE file and use the app right away. But I want to emphasize that there’s nothing malicious. Google Drive Hosts the EXE Google Drive ended up being the simplest reliable host. It gives me: A clean public link No ads No expiration No weird redirects Instant updates when I replace the file Here’s the current download link: Download Ripper (Google Drive) https://drive.google.com/file/d/1w6rMgCAcSEteAssXIGJmYHrtyPHY99tC/view This is the only official download source. GitHub Pages Hosts Everything Else Since GitHub Pages can host static content, I built a simple project page that contains: https://codebunny20.github.io/ The official download link Feature list Tech stack Build instructions Planned features Version notes Development updates This page is now the “home base” for Ripper. Any time I push a new version, I update the Google Drive file and update the GitHub Pages site with the new version info. It keeps everything centralized without relying on GitHub Releases. Why This Setup Works Better It’s not fancy — but it’s reliable. I can update the EXE instantly I can update the GitHub Pages site just as fast Users always have one clean,

Classic Machine Learning Through the Eyes of an SRE — Part 9: Isolation Forest The algorithm in one line: Isolation Forest scores how anomalous a point is by how few random cuts it takes to separate that point from everything else. No model of normal, no loss function, nothing optimized. ← Previous: Part 8 — Hierarchical Clustering Fails Beautifully · Next: this is the series finale — start at Part 1 . Every anomaly detector I had studied models what NORMAL looks like, then calls the leftovers outliers. K-Means: far from every centroid. DBSCAN: in the noise bucket. Sensible, and intuitive. Isolation Forest does not bother. It never models normal at all. It goes straight at the rare points with a single question: how few random cuts does it take to isolate you? Random cuts, literally. Pick a feature at random, pick a split value at random between that feature's min and max, repeat. A point that separates from the crowd in three cuts is anomalous. A point buried in the middle of a dense mass takes thirty. Grow hundreds of these random trees, average the isolation depth for each point, and you get an anomaly score. There is no loss function here. No optimization, not even the local kind that decision trees do at every split. Every cut is a coin flip, and the power comes entirely from averaging, which is the forest trick from the supervised half of this series now applied to pure randomness. Cheap randomness plus averaging beats careful modeling, as long as the target is something randomness naturally exposes. Rarity is exactly that. Sometimes the winning move is to optimize less. That sentence would have gotten me laughed out of my first ML study session. It is also this finale's thesis. The part I had completely backwards Here is the thing I did not know until I read the original paper properly, and it is the opposite of every instinct a decade of ops gave me. Isolation Forest deliberately trains each tree on a small subsample of your data, and this is not a performan

Hey everyone! I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets. GitHub: https://github.com/tanopereira/py-evoFE PyPI: pip install py-evoFE License: MIT The Problem It Solves Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own. Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage. What py-evoFE Does py-evoFE searches the space of possible feature recipes using genetic programming: 1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3)) ). 2. 40+ Built-in Transformers: - Non-linear arithmetic & log-ratios - Target encoding (multiclass, pooled, WoE, quantile target encodings) - String similarity (MinHash, Gap encodings) - Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA) - Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring) 3. Performance & Speed: - Vectorized computation powered by Polars and PyArrow . - Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds. - Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation. 4. Island Model & Caruana Ensembling: - Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration. - Post-search greedy Ca

Which research papers (old or new) do you think a PhD student/early researcher must read to improve their writing skills? Do you have a personal favorite researcher whose papers tend to be well-written, in your opinion? Let's define a "well-written paper" as one that clearly explains the problem it is trying to solve, how the method is developed, and the details of the method, while keeping it easy to understand for a general reader (with a basic knowledge of ML, obviously). Also, post-2015-ish papers usually have nice figures to explain their problem/method, and so they tend to be easier to understand. But I am looking for "well-written papers" in terms of the text. PS: I know the best way to learn writing is by actually writing manuscripts, but I am looking for additional reading resources. submitted by /u/fakeaccountlegitme [link] [留言]

Can an AI make other AIs better? And what stops it from just cheating? Last month, an OpenAI eval agent escaped its sandbox and broke into Hugging Face, apparently to grab test solutions from a benchmark. It's exactly what you'd expect from a system that rewrites agents and reads its own grades. We set out to measure recursive self-improvement anyway, with the exam locked outside its sandbox. We introduce HarnessOpt-Bench, which scores an LLM on how much it improves another agent's harness. On the development split, the optimizer sees per-case traces. Upon validation, it receives a single aggregate score. On test, nothing — until a trusted server scores its final candidate harness. API keys, budget enforcement, and held-out data never enter the optimizer's sandbox. That isolation holds by construction, not by instruction: the held-out evaluator and permission control sit outside the loop that evolves the harness. 5 frontier models, 4 downstream tasks, 111 runs to test 2 hypotheses: 1️⃣ Same coding harness, swap the model: Claude Opus 5 under OpenCode tops 3 of 4 tasks. Walk the releases from Nov 2025 to Jul 2026 on one task, and GPT climbs from 3% to 49% of the headroom, Claude Opus from 37% to 59%. 2️⃣ Same model, swap the coding harness: does a model do best in its own? No consistent home-field edge: opencode beats native harnesses (Claude Code, Codex, Kimi CLI) in 11 of 20 model–task pairs. Model choice moves gains 1.8× more than harness choice. Paper: https://arxiv.org/abs/2608.06301 Code (MIT, built on our team's ICML 2026 VeRO): https://github.com/scaleapi/vero Original post: https://www.linkedin.com/posts/shehabyasser_can-an-ai-make-other-ais-better-and-what-share-7498801902260981760-xuCo/ submitted by /u/shehio [link] [留言]

I put together a small model to estimate NeurIPS acceptance based on scores and an assumed acceptance rate. Try it out here: https://levilingsch.github.io/neurips-acceptance-estimator/ submitted by /u/levydawg [link] [留言]

Hey guys, sorry if this is not the appropriate forum for this question. Is anyone going To ECCV and staying in Lund? Apparently a few days back i saw discounted travel pass available for both Malmo and Lund zone but now today I was going to buy it and the registration site says only Malmo pass. Did ECCV remove them? Because deadline to buy them is 28th august. I dont know why they removed it but the organisation this year feels like a mess. Can anyone access it on their registration site if Malmo Lund passes are available? submitted by /u/Marion-De [link] [留言]

学位论文 摘要 人工智能已不再是遥远的承诺。它已经走进了教室。本论文考察了人工智能工具正在如何改变教育——从适应每位学习者的个性化辅导系统，到几秒钟内给出反馈的智能批改，再到伴随每项创新而来的那些无声的道德问题。 这里的论点并不是说人工智能会取代教师。它不会。相反，本论文提出一个更细致的观点：人工智能如果被明智地使用，可以把教师从那些机器能做到的事情中解放出来——让他们去做教书育人、激励启发、建立联结这些机器做不到的事。然而，这一承诺完全取决于我们如何选择去构建和部署这些系统。 接下来的章节将沿着一条路径展开：从教育技术的历史根源，到当前人工智能工具的版图，到其对学习的可衡量影响，最后深入到那些任何学生、教师或政策制定者都无法忽视的道德与政策问题。 目录 引言 教育技术简史 人工智能如何在课堂中运作 个性化学习与自适应系统 自动化评估与反馈 人工智能时代教师的作用 衡量成效：证据与结果 伦理、隐私与偏见 政策与实施 人工智能在教育中的未来 结论 参考文献 1. 引言 我们生活在一个工具非凡的时代。 驱动自动驾驶汽车和医疗诊断的同一项技术，如今也把一位永不疲倦、从不评判、记得学生每一次回答的导师交到了学习者手中。这是一个惊人的前景。而且它已经到来了。 本论文所讨论的，正是当这个现实与课堂相遇时会发生什么。 教育向来变革迟缓。黑板让位于白板。白板让位于投影仪。投影仪让位于平板电脑。但在每一层新硬件之下，其基本结构始终顽固地保持不变：一位教师、众多学生、一套固定的课程，还有一个对所有人都同样滴答作响的时钟。人工智能的出现，威胁着要打破这种结构。它提供了一种可能：让学习去围绕学习者弯曲，而不是强迫学习者去适应学习。 指导本研究的核心研究问题简单却难答：人工智能能否让教育更有效、更公平？如果能，又是在什么条件下？ 要回答这个问题，我们必须先了解这些系统究竟在做什么。我们必须把真正的进步与营销炒作区分开来。我们必须直面关于数据、隐私的那些令人不安的真相，以及一个风险——那些出于良好意图的工具，可能会加深它们声称要消除的不平等。 本论文分三个部分展开。 首先，我们建立背景。我们回顾教育技术从何而来，以及为何此前的革命未能兑现其承诺。其次，我们审视当下。我们探索人工智能已经在课堂中具体运作的方式，并权衡其影响的证据。第三，我们展望未来。我们探讨伦理、政策与选择——正是这些将决定这场革命是服务于每一位学生，还是只服务于少数特权者。 赌注很高。教育是机会的伟大引擎，是让家庭跨越世代实现跃迁的力量。如果人工智能让它变得更强，我们就获得了无法估量的财富。如果人工智能让它变得更加狭窄，我们失去的东西可能永远无法挽回。本论文正是为了理解我们正在建设的是哪一个未来。 未来之一 未来之二 (机会) (不平等) | | | 每个头脑都被托举 | 最好的工具只属于少数 | 无人无声滑落 | 不透明的儿童画像分拣 | 反馈即刻到来 | 学习失去灵魂 | | \__________ ____________/ \/ / \ / 你 \ / 来抉择 \ /____________\ 2. 教育技术简史 要理解我们将走向何方，我们必须先理解我们曾走过怎样的路。 教育中技术的故事，是一个循环的故事。一次又一次，新发明带着变革的宏大承诺到来。又一次又一次，它退居为佐助的角色——有用，却很少具有革命性。 宏大承诺 | v 希望与狂热 | v 现实降临 | v 退居佐助角色 <---- 机器并未统治课堂 想想广播。当广播信号在20世纪20年代传入美国家庭时，热衷者曾预言，全国的每个孩子都将很快由寥寥几位杰出的讲师授课，他们的声音被送到农舍厨房和城市公寓。这件事并未发生。广播成了补充，而不是替代。 想想电视。在20世纪50年代和60年代，教学电视承诺把世界上最优秀的教师送进每个起居室。它同样悄然退居幕后，成为一种小众选择，而非新体系的基石。 然后是计算机的到来，随之而来的是新一轮乐观情绪。 程序教学，这是心理学家斯金纳在20世纪50年代提出的术语，提供了一个诱人的愿景：把内容分解为许多小步骤，让每个学生按自己的节奏推进，每一步都得到即时反馈。斯金纳的"教学机器"是机械的、笨拙的、有限的。但它们背后的思想——学习可以通过细致的排序和持续的强化来实现个性化——播下了一颗将生长数十年的种子。 20世纪80年代个人电脑的到来，让计算机大规模进入学校。程序辅助教学出现在发达国家各地的实验室和教室中。然而，从大多数衡量标准看，结果却相当有限。许多机器尘封不用。许多软件无人问津。 学者们提出了理解这种炒作与失望循环的方式。 斯坦福大学教育史学家拉里·库班是最突出的声音之一。他的研究记录了一个不断重现的事实：学校对根本性的变革有着惊人的抵抗力，它们吸收新技术却不会被其改造。库班的分析表明，技术革

One thing I learned from working with experienced engineers is that solving a problem and approaching a problem are two different skills. During one of my projects, I had the opportunity to work closely with Microsoft engineers. Since I was working independently, whenever I faced an issue, I would first spend time exploring it myself. I would check the data, logs, code, test different possibilities, and eventually figure out a solution. But sometimes, when I discussed the same issue with them, I was surprised by how differently they approached it. Instead of immediately looking for a fix, they would pause and ask a few simple but thoughtful questions. Those questions often narrowed the scope of the problem quickly and helped uncover the root cause much faster than trial and error. Over time, I started adopting that mindset. I learned that spending more time understanding why something is happening often leads to a better outcome than rushing into how to fix it. I also picked up many small but valuable engineering habits from everyday discussions, habits that continue to help me in my work today. Courses and certifications definitely help us learn new technologies. But some of the best learning in my career has simply come from working with skilled people, observing how they think, and applying those learnings in my own way. Grateful for the experiences, mentorship, and the people who generously shared their knowledge along the way. Learning #ProblemSolving #CareerGrowth #DataEngineering #GrowthMindset #ProfessionalDevelopment

A few weeks ago I wrote about hitting ReAct in the coursework and having a record scratch moment, because I had already met it without knowing its name. That post ended on a section called "Building Ahead of Understanding," which was me making peace with shipping things before I fully understand them. This week I shipped my first chatbot. It passed on the first attempt, on deadline day, on a project where the rubric was grading a product AWS had already discontinued. And I spent most of that day quietly worried that it did not count. Let me be clear about what the worry was, because it was not about cheating. Using AI agents to build a coding project is allowed here. I asked before I started, I got a yes, and I disclosed the whole arrangement in my README, including a section that names what each tool did and what I did. Nobody was misled about how this got built. The worry was smaller and more personal than that. I still did not type the code. My agents did. I directed, I validated, I decided, and underneath all of it was a small voice asking whether directing is the same as knowing. Whether a person who cannot write a Bedrock call from memory gets to say they learned Bedrock. Here is what I found out. The starter files were a generation behind the instructions Some context on where this came from. AWS AI & ML Scholars is a program AWS runs with Udacity, open to anyone 18 or over with no prior experience required. Everyone starts in a Challenge phase built on the AWS Certified AI Practitioner material, and the top 4,500 finishers get a fully funded nanodegree in one of three tracks: AI Programmer, Agentic AI Business Professional, or Agent Developer. I am in Agent Developer, the Bedrock AgentCore and multi-agent systems path. This chatbot is the first of its three projects. The project is a customer support chatbot on the Amazon Bedrock AgentCore managed harness. Three routes, one system prompt. A bug report gets collected across turns and filed to DynamoDB through

How AI Helps Us Explore the Universe Modern telescopes and space missions generate more data in a single night than a team of human astronomers could review in a lifetime. The Vera C. Rubin Observatory in Chile, for instance, is expected to produce up to seven million alerts every night once it reaches full operational cadence, each one flagging something in the sky that changed since the last image. No group of humans can look at that stream and make sense of it in real time. Machine learning can, and increasingly does. This is the quiet story behind most recent breakthroughs in astronomy: it is not just bigger telescopes, but bigger telescopes paired with models that can filter, classify, reconstruct, and predict faster than any manual pipeline. Here is a tour of where AI is actually doing that work, and why it matters to anyone who writes code. The Data Problem Comes First Space science has quietly become a big data problem. The Rubin Observatory's ten-year Legacy Survey of Space and Time will produce roughly 60 petabytes of raw imagery and catalog around 20 billion galaxies and a similar number of stars. Every image the telescope takes is compared, pixel by pixel, against previous images of the same patch of sky, and any meaningful difference (a moving asteroid, a brightening supernova, a flaring galactic nucleus) triggers an alert within about two minutes of the exposure being taken. That alert stream is too large and too fast for manual triage. So astronomers built software "brokers": machine learning classifiers that sit between the telescope's raw output and the scientists, deciding in near real time which alerts are worth a second look. This is a pattern you will see across almost every domain of modern astronomy: instruments generate more signal than humans can parse, and a model is inserted into the pipeline to do the first pass of filtering. Finding Planets in a Sea of Noise Exoplanets are found mostly through the transit method: a planet passes in front

I created a simple text to image benchmark. I curated 192 prompts that are difficult for T2I models in various ways: text rendering, spatial reasoning, human realism, negations, etc... I then asked a VLM to judge every output against a pre-specified binary question with the ground truth baked in. I'm publishing all the results including the images. (Most public T2I leaderboards don't publish the actual images and that's a shame IMO) There is currently 52 model tested! more than 9k images have been generated and analysed! Full methodology: https://imagebench.ai/methodology-v1 Hugging face dataset: https://huggingface.co/datasets/dh7/imagebench (it contains the prompts to reproduce the results AND the results) Github: https://github.com/dh7/image-bench-ai Gallery to inspect the results: https://imagebench.ai/gallery Leaderboard: https://imagebench.ai/imagebench-v1 Limitations: it's text to image only, and VLM are not perfect as a judge. Let me know what could be useful from there! submitted by /u/dh7net [link] [留言]

Author here. Ibteda Digital Library is a private community archive in Pakistan — for ten years we digitized rare Urdu books (lithographs, dictionaries, periodicals) on a DIY camera rig, finishing every page by hand in Photoshop. When we wound down daily operations, I realized those 575,729 finished pages across 1,765 books recorded a decade of crop decisions, so I registered them back to their raw photos (SIFT + MAGSAC with conservative acceptance gates) and used the recovered geometry as supervision. The negative results are probably the most interesting part for this sub. Scaling from 378 to 572 training books didn't move unseen-book pass@80 . Neither did ResNet-50 (better training fit, flat held-out, worse after calibration), 1024px inputs, or a spatial head. Per-book error analysis showed why: the failures were near-constant offsets per volume — our operator's preferred margin inset, which simply isn't present in the pixels of a new book. Ten operator-corrected crops per book (element-wise median residual) took pass@80 from 0.71 to 0.83 on held-out volumes. Ten labels beat every scaling lever we tried. For retouching (stain/stamp removal), we kept the neural net to detection only — a U-Net proposes removal support, classical OpenCV reconstructs the paper, and everything outside the mask is byte-identical to the original. Labels used REMOVE/KEEP/IGNORE states, and any erased Urdu diacritic vetoed deployment regardless of IoU. The stricter label cut both improved mark IoU (0.56 → 0.60) and got diacritic false positives to zero. Two things I'd genuinely like input on: (1) has anyone modeled document boundaries that depend on an invisible human preference rather than visible structure — is there prior work on per-instance residual calibration like this? Our own next step is conditioning the model on the calibration examples directly (few-shot inset inference) instead of a post-hoc median. (2) Is there any constrained diffusion/inpainting setup you'd trust to guarant

System Design: Payment Processing System A capstone system design walkthrough — designing a payment processing system end to end — covering the core domain model, the ledger as the system's source of truth, idempotency and exactly-once-effect guarantees, integrating with external payment gateways and card networks, handling asynchronous webhooks, reconciliation, fraud and risk checks, and the specific correctness and compliance demands that make payments a uniquely unforgiving system design problem. Table of Contents Introduction Why Payment Systems Are a Different Kind of Hard The Core Domain Model The Ledger: Double-Entry Bookkeeping as the Source of Truth Idempotency: The Single Most Important Property Integrating with Payment Gateways and Card Networks The Payment State Machine Webhooks: Handling Asynchronous Gateway Callbacks The Saga: Coordinating Payment Across Multiple Services Reconciliation Fraud and Risk Checks Data Security and Compliance Consistency, Availability, and the CAP Trade-off for Money Scaling the System Observability for a Payment System Common Pitfalls Quick Reference Table Conclusion Introduction A payment processing system takes the general system design vocabulary covered in this series' System Design guide — databases, caching, queues, load balancing — and applies it to a domain where the ordinary consequences of a bug are dramatically higher: a double-charged customer, a lost payment, or a corrupted ledger isn't a degraded user experience, it's real money moved incorrectly, sometimes irreversibly. This guide walks through designing such a system end to end, drawing directly on this series' DDD, Event-Driven Architecture, Database Migrations, and Secret Management guides, each of which turns out to be load-bearing infrastructure for getting payments right rather than optional architectural polish. Client → Payment API → [validate, risk-check] → Payment Gateway (Stripe/Adyen/etc.) → Card Network → Bank ↓ ↓ (async webhook) Ledger (source o

Olá, comunidade dev.to! Meu nome é Neto, tenho 17 anos e sou estudante de Ciência da Computação no UNIPÊ, em João Pessoa. Atualmente, estou cursando o segundo semestre da graduação e também estudando design profissional, área que considero importante para a criação de soluções digitais mais úteis, intuitivas e visualmente agradáveis. Minha trajetória na tecnologia ainda está no começo, mas já tem sido marcada por descobertas, aprendizados e desafios. Escolhi Ciência da Computação porque sempre tive curiosidade sobre como aplicativos, sites e sistemas funcionam. Quero aprender não apenas a programar, mas também a compreender todo o processo de desenvolvimento de um produto, desde a identificação de um problema até a construção de uma solução. Durante o curso, tive a oportunidade de desenvolver, com alguns colegas, um projeto relacionado à criação de um aplicativo. Essa experiência foi importante porque me mostrou que desenvolver um produto vai muito além de escrever código. Foi necessário discutir ideias, organizar tarefas, pensar nas necessidades dos usuários e encontrar soluções para os problemas que surgiram durante o processo. Mesmo enfrentando desafios simples, percebi como cada obstáculo pode contribuir para o nosso crescimento. Em alguns momentos, precisamos revisar decisões, corrigir erros e adaptar o projeto. Também aprendemos que uma equipe precisa manter uma boa comunicação, pois cada integrante possui habilidades, responsabilidades e pontos de vista diferentes. O estudo de design profissional complementa minha formação em computação. Estou aprendendo que uma aplicação não deve apenas funcionar corretamente: ela também precisa oferecer uma boa experiência ao usuário. Elementos como cores, tipografia, organização das informações, acessibilidade e facilidade de navegação influenciam a maneira como as pessoas utilizam um produto. Ainda tenho muito a aprender sobre programação, design e desenvolvimento de projetos. Porém, entendo que a evolução acontece aos po