标签：#learn

Problem Statement For roughly a decade, vision-language models have been declared to be approaching or matching human performance on scene description (captioning). The evidence for that claim has almost always come from the same family of benchmarks—most famously MS-COCO. Those images are typically clean, well-lit, and depict either no people or people performing simple, isolated actions (sitting, walking, holding an object). They rarely require the model to parse multi-agent social dynamics, subtle intentions, or the kind of relational reasoning humans perform effortlessly when watching a movie scene or a street interaction. Because the evaluation data are easy, the reported numbers look excellent. Automatic metrics such as BLEU-4, CIDEr, or even embedding-based scores like BERTScore further inflate the impression of progress: they reward surface lexical overlap more than genuine semantic fidelity. At the same time, almost no work has systematically catalogued which visual-cognitive failures models still commit, or how those failure modes have changed as architectures moved from CNN+LSTM captioners to today’s multimodal large language models (MLLMs). The result is a field that can claim “human-level performance” while remaining largely blind to whether the models actually understand the scenes that matter most in real applications—scenes full of people interacting. The authors therefore set out to answer two concrete questions that the existing literature left open: (1) How much of the apparent progress is an artifact of easy data? (2) Which specific error types have been eliminated and which stubbornly remain? Core Idea The core insight is that progress looks dramatically different once you force models to describe complex social behavior and once you measure not only overall accuracy but a taxonomy of visual-cognitive errors. By constructing a new 100-image Complex Social Behavior (CSB) dataset drawn from movie frames that require reasoning about multi-person in

I recently found an old project I built with a friend around 2017–2018: a perk calculator for the game Firefall. The application allowed players to browse perks by category, drag them into a build, track the available perk points and automatically filter incompatible options based on the selected class. Looking at the code today, there are many things I would structure differently. The JavaScript could be better organised, responsibilities could be clearer, and the overall architecture would benefit from more modern practices. Still, I decided to preserve it as it is. Older projects are useful reminders that progress is not only visible in the technologies we use, but also in how we model problems, organise code and make technical decisions. It is not a showcase of how I would build the same application today. It is a snapshot of how I approached a real problem at that point in my career. Repository: https://github.com/lksvn/firefall-perk-calculator

skip에서 partition overwrite로: business_date 재처리를 Iceberg로 다시 표현하기 이전 글에서는 같은 source_hash 가 다시 들어왔을 때 기존 successful run을 재사용하는 idempotency를 다뤘다. 하지만 재처리에는 두 종류가 있다. 1. 같은 입력이 다시 들어온 경우 -> skip이 맞다. 2. 같은 business_date의 정정 입력이 들어온 경우 -> skip하면 안 된다. -> 같은 날짜의 gold 결과를 중복 없이 교체해야 한다. manufacturing-data-platform-mini 의 B5 slice는 두 번째 문제를 아주 작게 다룬다. 전체 Spark pipeline을 만든 것이 아니다. gold_daily_metrics Iceberg table 하나를 local Spark에서 만들고, business_date partition overwrite와 snapshot evidence만 검증했다. Scenario 이미 아래 gold row가 있다. business_date=2026-06-29 plant-a / line-1 / gearbox-a units_produced=120 defect_count=3 나중에 같은 business_date=2026-06-29 에 대한 정정 source가 들어온다. 운영자가 원하는 것은 append가 아니다. 원하지 않는 상태: 2026-06-29 old row 2026-06-29 corrected row -> 같은 날짜 결과가 중복됨 원하는 상태: 2026-06-29 corrected row만 남음 2026-06-30 같은 다른 날짜 partition은 그대로 유지됨 재처리 전후 snapshot evidence가 남음 그래서 이 slice의 질문은 이렇다. 같은 business_date의 정정 source를 처리할 때, gold table에서 해당 날짜 partition만 중복 없이 교체하고, 어떤 run이 어떤 Iceberg snapshot을 만들었는지 남길 수 있는가? Decision Pressure Slice1의 CSV pipeline은 already-successful source를 안전하게 skip할 수 있다. dataset_id + business_date + source_hash 이 key가 같으면 같은 입력이다. 다시 계산해도 같은 결과이므로 기존 run을 재사용한다. 하지만 source_hash 가 달라졌다면 의미가 다르다. same business_date different source_hash 이건 retry가 아니라 correction이다. CSV run-folder 방식에서는 새 run output을 만들 수는 있지만, "현재 gold table에서 해당 날짜를 원자적으로 교체한다"는 table-level 의미가 약하다. Iceberg를 붙이는 이유는 여기 있다. source_hash -> 같은 입력인지 판단하는 idempotency key business_date partition -> 정정 시 교체할 gold table 범위 snapshot_id -> table commit의 evidence 즉 Spark/Iceberg는 도구 이름을 추가하려고 붙인 것이 아니라, 재처리 상태 전이를 더 명확히 표현하기 위해 붙였다. Options Option 장점 문제 판단 same source면 항상 재계산 단순함 retry 때 불필요한 commit이 계속 생김 제외 corrected source를 append 구현 쉬움 같은 날짜 gold row가 중복될 수 있음 제외 whole-table overwrite 단순함 다른 날짜 partition까지 지울 위험 제외 business_date partition overwrite correction 범위가 명확함 Spark/Iceberg 설정과 test가 필요 선택 MERGE/upsert 강력함 이번 skeleton에 과함 backlog 이번 구현은 DataFrameWriterV2.overwritePartitions() 를 사용했다. corrected_d

wide CSV 여러 개를 EAV로 모아 gold mart 만들기 현실의 데이터 소스는 한 가지 모양으로 오지 않는다. 같은 의미의 값도 어떤 파일에서는 생산수량 , 다른 파일에서는 units , 또 다른 파일에서는 made 로 올 수 있다. 온도도 어떤 곳은 섭씨, 어떤 곳은 화씨일 수 있다. 이걸 매번 pipeline code에 if source == ... 로 박기 시작하면 source가 늘 때마다 코드가 지저분해진다. manufacturing-data-platform-mini 의 EAV mini slice는 이 문제를 작게 다룬다. 여러 wide CSV를 mapping config로 표준 attribute에 맞춘 뒤, EAV long format으로 모으고, 다시 gold metric mart로 pivot/aggregate한다. 데이터는 모두 synthetic이고, 회사 코드·고객 데이터·실제 schema는 쓰지 않았다. 1. Scenario 서로 다른 공장/라인/벤더에서 비슷한 제조 지표 파일이 들어온다. 예: plant_a.csv: 설비ID, 생산수량, 불량수, 온도C, 압력kPa plant_b.csv: machine_id, output_units, defects, temp_f, pressure_bar vendor_d.csv: unit_name, made, scrap, deg_c, kpa 비즈니스적으로는 같은 지표를 보고 싶다. units_produced defect_count temperature_c pressure_kpa 문제는 source마다 column name과 unit이 다르다는 점이다. 2. Decision Pressure 단순 구현은 source마다 코드를 늘린다. if source == "plant_a": 생산수량을 units_produced로 읽는다 if source == "plant_b": output_units를 units_produced로 읽는다 temp_f를 섭씨로 변환한다 if source == "vendor_d": made를 units_produced로 읽는다 이 방식은 작게는 빨라 보이지만 source가 늘수록 문제가 된다. 새 파일 형식마다 pipeline code를 고쳐야 한다. column mapping과 transform logic이 섞인다. unit conversion이 흩어진다. quality check가 source별로 갈라진다. gold mart grain을 설명하기 어려워진다. 그래서 mapping은 config로 빼고, pipeline은 표준 attribute를 처리하게 만들었다. 3. Options option result risk source별 hard-coded parser 처음엔 빠름 source가 늘 때 code change 반복 모든 source를 wide table 하나로 합치기 보기 쉬움 sparse/heterogeneous column 폭발 EAV long format 이종 attribute를 표준 형태로 모음 pivot/quality 설계가 필요 full mapping DSL/rules engine 유연함 mini project에는 과함 이 프로젝트의 선택은 단순한 JSON mapping + EAV long + gold pivot이다. 4. Decision 각 source는 JSON config로 자신의 column을 표준 attribute에 매핑한다. source column -> standard attribute output_units -> units_produced temp_f -> temperature_c with f_to_c pressure_bar -> pressure_kpa with bar_to_kpa pipeline 흐름: wide CSVs -> mapping configs -> EAV long rows -> gold entity_daily_metrics -> quality checks -> catalog/lineage EAV row의 핵심 shape: entity_id business_date attribute value v

schema drift를 fail이 아니라 warn으로 둔 이유 데이터 파이프라인에서 source schema가 바뀌는 순간은 애매하다. 무조건 무시하면 운영자는 입력 구조가 바뀐 사실을 모른다. 반대로 모든 schema 변화를 실패로 처리하면, 정상적인 컬럼 추가까지 daily run을 막아버린다. manufacturing-data-platform-mini 에서는 이 문제를 작게 다뤘다. synthetic manufacturing CSV의 실제 header를 기준으로 schema_hash 를 만들고, previous successful run과 비교해 달라졌으면 schema_drift quality check를 warn 으로 남긴다. 단, required column이 빠져 silver/gold contract를 만들 수 없는 경우는 현재 ValueError 로 빠르게 실패한다. 1. Scenario 어느 날 source CSV에 새 컬럼이 추가된다. 기존 header: event_time,plant_id,line_id,work_order_id,machine_id,product_code, operation,units_produced,defect_count,cycle_time_ms,business_date 새 header: event_time,plant_id,line_id,work_order_id,machine_id,product_code, operation,units_produced,defect_count,cycle_time_ms,business_date,operator_id operator_id 는 아직 silver/gold mart에서 쓰지 않는다. 하지만 source 구조가 바뀐 사실은 기록되어야 한다. 2. Decision Pressure schema drift에서 중요한 질문은 단순히 "바뀌었나?"가 아니다. 바뀐 것을 운영자가 알 수 있는가? 정상적인 컬럼 추가 때문에 pipeline을 멈춰야 하는가? downstream gold mart contract가 조용히 바뀌지는 않는가? 이전 successful run과 지금 run의 schema identity를 비교할 수 있는가? 초기 구현에서는 한 가지 실제 버그가 있었다. schema_hash 가 고정된 required column 목록에 너무 묶여 있어서, 추가 컬럼이 들어와도 hash가 바뀌지 않았다. 즉 operator_id 가 추가되어도 drift가 보이지 않았다. 이 문제를 고치기 위해 read_rows 가 실제 CSV header를 반환하고, 그 실제 header 기준으로 schema_hash 를 계산하도록 바꿨다. 3. Options option result risk ignore drift pipeline은 계속 돈다 source 변화가 보이지 않음 fail every drift 변화에 강하게 반응 정상적인 컬럼 추가도 막음 warn and continue 변화가 보이고 run도 계속됨 warning을 inspect해야 함 auto-evolve silver/gold 새 컬럼을 바로 사용 가능 downstream contract가 조용히 바뀔 수 있음 full schema registry production에 가까움 mini slice에는 무거움 이 프로젝트의 선택은 warn and continue 다. 4. Decision 현재 contract는 이렇다. previous successful run이 없으면: schema_drift = pass baseline schema established current schema_hash == previous successful schema_hash: schema_drift = pass current schema_hash != previous successful schema_hash: schema_drift = warn quality_passed는 true 유지 run/lineage record에 previous/current schema_hash 저장 required column missing: V

source_hash로 같은 입력 재처리를 안전하게 skip하기 작은 데이터 파이프라인도 한 번만 실행된다고 가정하면 금방 거짓말이 된다. 실제로는 같은 파일을 다시 실행할 수 있다. 실패한 run을 재시도할 수도 있고, 과거 날짜를 backfill할 수도 있고, 운영자가 실수로 같은 입력을 다시 넣을 수도 있다. 이때 결과가 중복되면 gold metric은 더 이상 믿을 수 없다. 이 글은 개인 포트폴리오 프로젝트 manufacturing-data-platform-mini 에서 source_hash 를 이용해 같은 입력 재처리를 안전하게 skip하도록 만든 작은 설계 판단을 정리한 글이다. 데이터는 모두 synthetic이며, production platform이 아니라 검증 가능한 mini slice다. 1. Scenario 같은 business_date 의 제조/로봇 이벤트 파일을 다시 처리해야 하는 상황이 있다. 예: retry: 앞 run이 중간에 실패해서 다시 실행한다. backfill: 과거 날짜를 다시 채운다. operator mistake: 같은 파일을 실수로 다시 실행한다. 단순히 매번 append하면 같은 날짜의 gold metric이 중복될 수 있다. 2. Decision Pressure 단순 CSV pipeline은 보통 이렇게 끝난다. CSV 읽기 -> silver 만들기 -> gold 집계 -> 결과 저장 하지만 운영 관점에서는 질문이 생긴다. 이 입력은 전에 처리한 파일과 같은가? 같은 파일을 다시 돌리면 중복 output이 생기나? 다른 파일로 같은 날짜를 다시 돌리면 어떻게 구분하나? 어떤 run이 어떤 source에서 만들어졌나? 그래서 재실행을 판단할 identity가 필요했다. 3. Options option result problem always append 모든 run 결과를 계속 추가 같은 입력 재실행 시 중복 always overwrite 결과를 항상 덮어씀 이전 결과/원인 추적이 약함 skip by business_date only 같은 날짜면 무조건 skip 정정 파일을 반영할 수 없음 skip by dataset_id + business_date + source_hash 같은 입력만 no-op 정정 파일은 새 run으로 처리 가능 이 프로젝트의 Slice1은 마지막 선택지를 쓴다. 4. Decision 현재 mini pipeline은 입력 파일의 content hash를 source_hash 로 계산한다. idempotency key: dataset_id + business_date + source_hash 이미 성공한 run이 있으면 새로 처리하지 않고 기존 run을 재사용한다. same dataset_id same business_date same source_hash prior successful run exists => status = skipped 이 선택은 작지만 중요하다. 같은 파일 재실행: skip -> 중복 없음 같은 날짜의 정정 파일: source_hash가 다름 -> skip하지 않고 새 run으로 처리 가능 단, 여기서 조심해야 할 경계가 있다. Slice1은 다른 source_hash 를 새 run으로 처리할 수 있지만, 이전 gold partition을 원자적으로 교체하는 Iceberg-style overwrite까지 구현한 것은 아니다. 그 문제는 다음 Slice2의 business_date partition overwrite 주제다. 5. Evidence 관련 코드와 검증 evidence: src/manufacturing_data_platform/pipeline/lakehouse.py tests/test_lakehouse_pipeline.py VERIFICATION_LOG.md README.md 검증 로그: 2026-07-08 publication readiness check: pytest: 33 passed lakehouse JSON CLI: passed, status=processed, quality_passed=true EAV JS

Stop Guessing: How I Pick AI API Architecture at Every Scale I've been on both sides of this. Two years ago I was the lone backend engineer at a Series A startup, duct-taping API calls together at 2 AM because the founders wanted a chatbot demo by morning. Last quarter I sat in a procurement meeting at a Fortune 500 where we spent six weeks evaluating three vendors for a single inference workload. Same job title on LinkedIn, wildly different problems. Most AI API guides I've read treat both scenarios like they're the same conversation. They're not. The startup CTO optimizing for burn rate and the enterprise architect worrying about a 99.9% uptime SLA are solving fundamentally different equations. After enough of these conversations, I've developed a framework I'd like to share — and yes, I'll talk about Global API because it's what I actually use, but I'll also explain the reasoning behind each choice so you can adapt it to your own stack. What I Look at First: The p99 Question Before I look at price, I look at the latency distribution. Specifically, the p99. Mean latency tells you almost nothing useful. If your median response is 200ms but your p99 is 4 seconds, your users will see janky behavior on the long tail and you won't know why until production is on fire. For startups in the MVP phase, you can usually get away with best-effort routing. A p99 of 2-3 seconds is fine if you're building an async summarization feature. But the moment you put AI in the synchronous request path — like a customer-facing chatbot or a real-time code suggestion — p99 starts to bite. I learned this the hard way when our startup's "AI assistant" feature had users complaining about slowness that I couldn't reproduce locally. The culprit? Provider cold starts hitting our 1% of users who happened to get routed to a freshly spun-up instance. For enterprises, p99 isn't a nice-to-have, it's a contractual obligation. Most B2B SLAs I've negotiated pin uptime at 99.9% and require reporting on m

Detecta si tu modelo de materiales hace trampa con la "huella bibliográfica" Un modelo de ML puede predecir la propiedad de un material sin entender la química: basta con que "aprenda" qué autores, revistas o años suelen ir con cada resultado. Esta herramienta aplica el test de falsificación de Clever Materials para descubrirlo. El problema: cuando el modelo lee el membrete, no la ciencia Imagina que entrenas un modelo para predecir si un material es estable. El modelo no mira la química: descubre que los artículos del grupo X (publicados en la revista Y, en torno al año Z) casi siempre reportan "estable". Así que aprende a clasificar por el membrete bibliográfico , no por la estructura. Funciona en el papel y se rompe en la práctica. A esto se le llama confounding bibliográfico (o leakage por metadata). No es un error de código: es una señal espuria que el modelo aprovecha. El paper Clever Materials (Jablonka et al., 2026) mostró que este patrón está generalizado en cinco tareas reales de materials science. Qué hace la herramienta materials-confounding-check es una CLI ( mcc check ) que corre cuatro sub-tests de falsificación sobre tu dataset (descriptores químicos + metadata bibliográfica + propiedad objetivo): Clasificador de metadata — ¿se puede predecir la bibliografía (autor/revista/año) a partir de los descriptores químicos? Si es above-chance , hay una señal bibliográfica presente. Huella bibliográfica — ¿un modelo que usa solo la metadata predicha se acerca al modelo con descriptores? Entonces el dataset no descarta hacer "trampa" por bibliografía. Split por grupo/tiempo — ¿colapsa el rendimiento si separas por autor/año en vez de al azar? Veredicto — un score low / medium / high de riesgo de confounding. El rigor que exige el test (para especialistas) El punto delicado de cualquier "test de significancia" es fijar el umbral a mano. Si ajustas el margen hasta que tu fixture pase, el test no prueba nada: es el anti-patrón Clever-Hans que el propio proyecto d

You have seen the posts. AI is coming for the junior engineer first. Why hire someone to write code a model can write for free? The career ladder's bottom rung is gone, so start saving your pity for anyone about to graduate into this market. I think the premise is wrong, and it is wrong in a specific, fixable way. Look closely at what these predictions actually describe. Not a junior engineer. A person whose entire job is turning a finished spec into working code. That role is real, and it is shrinking fast, but it was never the same thing as "junior engineer." We just let the two collapse into one job title for forty years because, until recently, spec-to-code translation was the canonical, critical thing a junior had the skill to do. The task and the title are not the same thing. AI is eating the task. It does not follow that it eats the title too, unless we insist on keeping them welded together. So the real question is not "does the junior engineer survive." It is "what do we train a junior engineer to do now that the translation work is cheap." And the honest answer is: not much of what we have been doing. I think we landed on "junior engineers are doomed" for a reason that has nothing to do with whether it is true. It is the easy conclusion. It requires nothing from us. Training a junior into a senior was never straightforward, even in the old world, and figuring out how to do it without the years of tickets we used to lean on is genuinely hard. "They're doomed" lets everyone off the hook. "How do we train juniors into seniors now" does not, but it is the question with a future in it. The first one just has a shrug. The apprenticeship we built no longer exists For as long as I have been in this field, the plan was the same. Hire someone who can code. Hand them small, well-specified tickets. Let them grind through years of execution: bugs, edge cases, code review, the slow accumulation of pattern recognition that eventually turns into judgment. Somewhere around

Image-to-video generation is often described as a simple interaction: upload image -> describe motion -> get video That description hides the real problem. A single still contains only one view of a subject. When we ask a model for a fast camera orbit, a full-body walk, or expressive gestures, we are asking it to invent information that was never present in the source. That is where identity drift, unstable lighting, texture flicker, and waxy faces come from. The useful way to approach Seedance 2.0 image-to-video is not as a prompt-writing contest. It is a constraint-management workflow. Give the model a strong identity anchor, request motion that the source image can support, and evaluate one variable at a time. This post explains that workflow in a way that is useful whether you are animating a product render, a character portrait, an approved client still, or a visual asset for a prototype. Note: Model capabilities, pricing, model availability, and input limits change quickly. Check the current documentation and the terms of the platform you use before committing a production workflow. Why image-to-video is different from text-to-video Text-to-video is excellent when invention is the point. You describe a scene and let the model make creative decisions about characters, lighting, composition, and motion. Image-to-video is the better tool when those decisions have already been made and must remain stable. Situation Better starting mode Why Product hero shot Image-to-video Label, shape, material, and color must remain recognizable Character-led sequence Image-to-video One strong reference can anchor a character across clips Approved campaign still Image-to-video The source already represents the accepted art direction Atmospheric B-roll Text-to-video Exact subject identity matters less than visual exploration Abstract concept film Text-to-video Inventing a scene is more valuable than preserving one Existing brand-photo library Image-to-video Stills become reusable

Check this out: migrating Off OpenAI: A Backend Engineer's Notes From Production I still remember the morning I opened our team's monthly invoice and nearly spilled cold brew on my mechanical keyboard. We were burning through OpenAI credits like it was nobody's business — specifically, north of $500/month for what amounted to a chat-completion endpoint and some embedding lookups. As the backend engineer who had inherited the LLM integration six months prior, I felt personally responsible. So I did what any self-respecting engineer does at 2 AM with too much caffeine: I benchmarked alternatives. What I found annoyed me. DeepSeek V4 Flash was sitting there at $0.25/M output tokens while GPT-4o charges $10.00/M. That's a 40× price difference for output that, in my blind tests, 80% of users couldn't distinguish. The $500/month bill could plausibly become $12.50. My CFO would weep tears of joy. This post is the migration journal I wish I'd had before I started. fwiw, I've already done the swap across three production services. Here's what worked, what didn't, and exactly how much coffee I drank. The Math That Made Me Pick Up a Keyboard Before I show you code, let's talk numbers — because if you're going to convince your team or your boss, you'll need a slide that fits on one screen. I pulled together the pricing for the models I actually considered routing traffic through. All figures are per million tokens, USD: Model Provider Input $/M Output $/M Relative to GPT-4o GPT-4o OpenAI $2.50 $10.00 1× (baseline) GPT-4o-mini OpenAI $0.15 $0.60 16.7× cheaper DeepSeek V4 Flash Global API $0.18 $0.25 40× cheaper Qwen3-32B Global API $0.18 $0.28 35.7× cheaper DeepSeek V4 Pro Global API $0.57 $0.78 12.8× cheaper GLM-5 Global API $0.73 $1.92 5.2× cheaper Kimi K2.5 Global API $0.59 $3.00 3.3× cheaper Let me be clear about something: those numbers come straight from the provider's pricing pages at the time I ran the analysis. I have not invented, rounded up, or "adjusted" anything her

series: Prediksi Survival Klub Debutan Kenapa Project Ini? Setiap musim, klub yang promosi ke liga top (Premier League, La Liga, dst.) menghadapi risiko besar: sekitar 2 dari 3 klub yang naik biasanya kembali terdegradasi di musim pertama mereka. Saya penasaran — bisakah performa di beberapa laga awal musim memberi sinyal dini soal peluang klub tersebut bertahan? Ini jadi project portofolio pertama saya sebagai data scientist yang baru mulai (0-1 tahun pengalaman). Saya sengaja pilih topik yang saya suka (sepak bola) supaya prosesnya tetap enjoyable, bukan cuma "tutorial project" generik. Rencana Project Pertanyaan utama: Berdasarkan performa 8 laga pertama musim debut, seberapa besar peluang klub promosi bertahan hingga musim berikutnya (tidak degradasi)? Data yang dipakai: football-data.co.uk — data hasil pertandingan tiap musim sejak 1993/1994 Wikipedia (halaman musim liga) — daftar klub promosi & klasemen akhir musim Tech stack: pandas , requests untuk data collection scikit-learn untuk modeling (mulai dari Logistic Regression sebagai baseline) imbalanced-learn untuk handle class imbalance Streamlit + Plotly untuk dashboard interaktif Deploy ke Streamlit Community Cloud Timeline (Build in Public) Saya bikin timeline ini publik supaya ada tekanan yang sehat untuk benar-benar menyelesaikannya, bukan cuma jadi ide yang menguap: Checkpoint Target Tanggal Yang Harus Selesai Part 1 (post ini) 11 Juli 2026 Kickoff, rencana, environment siap Part 2 15 Juli 2026 Dataset jadi, push ke GitHub Part 3 17 Juli 2026 EDA selesai, insight awal Part 4 24 Juli 2026 Model final dipilih + evaluasi Part 5 31 Juli 2026 Dashboard live di Streamlit Cloud Part 6 (final) 8 Agustus 2026 Project selesai, recap lengkap Tantangan yang Sudah Saya Antisipasi Data leakage — fitur harus dihitung dari laga awal musim saja, bukan seluruh musim, biar model beneran memprediksi bukan "menyontek" hasil akhir Dataset kecil — kemungkinan hanya ~60-100 sampel klub, jadi saya mulai dari model sederhana (Lo

The RabbitMQ upgrade looked like a straightforward infrastructure task: move from RabbitMQ 3.X to 4.X, provision the new broker, review the client setup, confirm queues still declare correctly, restart consumers, watch the logs, and move on. But infrastructure upgrades rarely test only infrastructure. They also test the assumptions your application has been making for years. In this case, the upgrade forced a more important question: is our messaging system reliable by design, or has it simply been relying on stable conditions? That distinction matters because a message queue can appear healthy when the broker is running, the network is stable, consumers are alive, and messages are acknowledged quickly. But production systems are not judged only by how they behave when everything is fine. They are judged by how they behave during restarts, closed channels, slow handlers, bad configuration, deployment windows, and partial failure. The RabbitMQ upgrade exposed those edges. It revealed assumptions around connection lifecycle, acknowledgements, dead-letter routing, retry behavior, observability, and operational simplicity. The real lesson was not just how to upgrade RabbitMQ. The real lesson was how to build a messaging layer that is easier to operate, easier to reason about, and safer to fail. Simplicity Is an Operational Feature One of the first things the upgrade exposed was complexity. Over time, messaging code can quietly become a small internal framework. A connection helper becomes a connection manager. A consumer wrapper becomes a consumer framework. Retry helpers appear, dead-letter helpers appear, failure handlers appear, and monitoring logic gets layered on top. Each addition may have been reasonable when introduced, but during an incident, complexity has a cost. Every abstraction becomes another place to inspect. Every helper becomes another assumption to validate. Every unused file becomes a possible source of false confidence. RabbitMQ integration code doe

This is a submission for Weekend Challenge: Passion Edition What I Built I told an agent Never write directly to the database . A long session later, context window full, it wrote directly to the database. The rule loading mark was still sitting in the prompt. The model had just stopped weighting and attending to it. It's an invisible failure. No error is being thrown. The task comes back subtly wrong, and the rule reads perfectly fine when you go back and check it. I wanted to make it visible, so I built an interactive field you can drag around. Every rule you write for an agent is a hill. Its height is how well the rule is written: a directive-led, backtick -anchored rule stands tall, a hedged and vague one sits low. Then you raise the water. The water is context load. As it rises the low rules go under first, in order of how well they were written. The weak ones drown while you watch. Three of the hills are high-stakes prohibitions, the Never... rules. They drown too. That is the whole point of the piece. A rule you cannot afford to lose does not belong in prose at all; it belongs on a runtime hook that runs as code, not attention. The field flags those in red the moment they go under. Underneath the field is a second tool: a client-side lint that reads an instruction and names the surface tells (hedges, shouting, politeness, a ban placed before its directive). It is deliberately not a score. It catches what a little regex can honestly catch, and points at the real analysis for the rest. Demo Play it on its own page. Drag to orbit, drag the load slider to raise the water: ▶ Open the live demo Each of the nine instruction patterns in the demo links to its rule page on reporails.com/rules . Code Code is available on Codepen: https://codepen.io/editor/G-bor-M-sz-ros-the-reactor/pen/019f4cad-e344-78bf-b7bc-919972f42a4e The whole thing is one self-contained HTML file: no build step, no dependencies, no backend. The CodePen above is the full source, so you can read eve

When learning a new technology, most of us follow a familiar path. We start with the official documentation. Then we search GitHub repositories. We read blog posts. We watch YouTube tutorials. Eventually, we ask an AI assistant when we get stuck. Each resource solves a different problem, and the best developers know when to use each one. Documentation Is the Foundation Official documentation should almost always be your first stop. It tells you how a framework or library is intended to work. The information is usually accurate, maintained, and version-specific. If you're learning React, Next.js, or Node.js, the official docs provide the most reliable starting point. But documentation has limits. It explains what something does, not always why developers use it in real projects. Community Content Fills the Gaps That's where blog posts, conference talks, and open-source repositories become valuable. Experienced developers share: Real-world architecture decisions Common mistakes Performance considerations Debugging strategies Project structure Deployment workflows These practical insights often don't belong in official documentation, but they're essential for becoming a better engineer. AI Has Changed the Workflow AI assistants have become another tool in the developer toolbox. Instead of searching through multiple pages, developers can ask targeted questions like: Why is this hook re-rendering? What's the difference between these two approaches? How can I improve this query? Can you explain this error message? AI doesn't replace documentation. It helps you understand it faster. The most effective workflow is using documentation as the source of truth while letting AI explain concepts, compare approaches, or clarify confusing examples. Build Your Own Reference Library One habit that's improved my productivity is creating a personal knowledge base. Whenever I solve a difficult problem, I write down: The issue Why it happened The solution What I learned Links to relevant

**_Hi everyone! We're three 16-year-old friends learning to code. Instead of building "just another app," we want to solve a real problem that developers actually face. So we have one question: Think about a moment when you caught yourself saying, "Why hasn't anyone built a good solution for this yet?" What was the problem? It can be anything: something that wastes your time, something frustrating, a repetitive task, a confusing workflow, or anything that made you wish a better tool existed. We're not trying to sell anything. We're simply listening and looking for real problems worth solving. Every answer means a lot to us. Thank you!_**

Adapted from an appendix of my MS thesis. Markov Chain Monte Carlo Almost as soon as computers were invented, they were used for simulation. Markov chain Monte Carlo (MCMC) was invested as Los Alamos, Metropolis et al (1953) simulated a liquid in equilibrium with its gas phase. Their tour de force was the realization that they did not need to simulate the exact dynamics, they only needed to simulate some Markov chain with the same equilibrium distribution. The Metropolis algorithm was widely used by chemists and physicists, but was not widely known among statisticians until after 1990. Hastings (1970) generalized the Metropolis algorithm, and simulations following his scheme are said to use the Metropolis-Hastings (MH) algorithm [1]. A special case of the MH algorithm was introduced by Geman et al (1984) discussing optimization to find the posterior mode rather than simulation. Algorithms following their scheme are said to use the Gibbs sampler. It took some time for the spatial statistics community to understand that the Gibbs sampler simulated the posterior distribution, thus enabling full Bayesian inference of all kinds. Gelfand et al (1990) made the wider Bayesian community aware of the Gibbs sampler, and then it was rapidly realized that most Bayesian inference could be done using MCMC, whereas very little could be done without MCMC. Green (1995) generalized the MH algorithm as much as it could be generalized [1]. Theoretical Foundations A sequence X 1 ​ , X 2 ​ , … of random elements of some set is a Markov chain if the conditional distribution of X n + 1 ​ given X 1 ​ , … , X n ​ depends on X n ​ only. The set in which the X i ​ take values is called the state space of the Markov chain. A Markov chain has stationary transition probabilities if the conditional distribution of X n + 1 ​ given X n ​ does not depend on n . This is the main kind of Markov chain of interest in MCMC. The joint distribution of a Markov chain is determined by the following [1]. The ma

Modern applications, especially AI agents and distributed systems, need more than logs to understand what is happening. That's why I explored SigNoz, an open-source observability platform built on OpenTelemetry. Setting up SigNoz with Docker was simple. After connecting a sample application, I could view logs, metrics, and traces from a single dashboard within minutes. My favorite feature is distributed tracing. Instead of guessing where requests slow down or fail, SigNoz clearly shows the complete request journey across services, making debugging much easier. The built-in dashboards provide valuable insights into CPU usage, memory, request latency, throughput, and error rates. Having centralized logs alongside metrics and traces saves time by eliminating the need to switch between multiple tools. I also liked the alerting feature, which helps detect issues before they affect users. For AI applications, observability is essential. AI agents make multiple API calls, use tools, and perform complex workflows. SigNoz makes it easier to understand each step, identify failures, measure latency, and optimize performance. Overall, my experience with SigNoz was excellent. It combines logs, metrics, traces, dashboards, and alerts into one intuitive platform. Among all its features, distributed tracing impressed me the most because it provides deep visibility into application behavior and simplifies troubleshooting. I'm excited to use SigNoz in future AI and cloud-native projects.

My GPT got dumber after the update" gets blamed on the model regressing, or on you prompting worse. Both are unfalsifiable, and both send you to fix the wrong layer. The layer that actually moved is the one you can pin. "The model" is two layers. The weights — the trained network, slow to change, and when they do change it's announced under a new name. And the wrapper — the router that picks which model answers, the system prompt, the default reasoning effort, verbosity caps. The wrapper changes silently, on its own schedule, per product. It's almost always what moved under you. So stop re-tuning prompts to chase it. Pin the wrapper: Force the route. Don't leave it on Auto — set Thinking (or say "think hard") so the router can't quietly demote your prompt to a faster, weaker model. OpenAI's own GPT-5 launch post describes exactly this router (it scores prompts "simple" vs hard); after the backlash they put the picker back (Auto/Fast/Thinking — TechCrunch, Aug 2025). Pin the version. If you build on a model, call its exact versioned ID via the API. A model ID's weights don't change — new versions ship under new IDs — so router and system-prompt churn can't reach you. Own the harness. Running agents? Set the system prompt, reasoning effort, and verbosity yourself instead of inheriting a default. Anthropic's own April 23 post-mortem is the proof: six weeks of "Claude Code got worse" traced to three wrapper changes (a reasoning-effort downgrade, a reasoning-history bug, a verbosity cap their ablations put at ~3% quality) — API weights never touched. A real weights change — a new model — will still move behavior. But that's announced, and you choose when to adopt it. The silent stuff is all wrapper, and the wrapper is the part you can pin. Sources: OpenAI GPT-5 launch (router + "think hard"); TechCrunch, Aug 2025 (model picker reinstated); Anthropic April 23 post-mortem (anthropic.com/engineering/april-23-postmortem); InfoQ and VentureBeat (corroboration); Claude platfor

There's a moment in every Solana project where the work stops being about whether the program works and starts being about whether it's ready . You've tested it, the logic holds, the constraints are tight. Then you point it at mainnet, and a different set of questions shows up: questions about money, permanence, and strangers. This post is about that transition. Not the commands, which are short and well documented, but the shift in what you're responsible for once real users can touch your code. If you've been building on devnet and you're starting to think about a live launch, this is the mental model to carry in. Devnet was a sandbox. Mainnet is not. Devnet is a practice field. The SOL is free, you airdrop more whenever you run low, and if you deploy something broken, the only casualty is your afternoon. That safety is the whole point of devnet: it lets you fail cheaply and often, which is exactly how you should be learning. Mainnet removes the safety net, and three things change the moment you cross over. The SOL is real. Deploying a program allocates an on-chain account sized to your compiled binary, and you pay rent for that space in actual SOL. Larger programs cost more. This isn't a huge sum for a typical program, but it's real money leaving a real wallet, and that alone tends to sharpen how carefully you check things before you hit deploy. The audience is real. On devnet the only person calling your program is you. On mainnet, anyone can find your program and send it any transaction they like, the moment it's live. Everything from the security arc stops being theoretical: the accounts strangers pass in, the inputs you didn't expect, the edge cases you hoped no one would hit. Mainnet is where "every account is attacker-controlled until proven otherwise" becomes a live condition rather than a lesson. The mistakes are visible. A bad devnet deploy disappears into the noise. A bad mainnet deploy is a public event, on a permanent ledger, in front of the users you