The Day My AI Taught Me That Passing Tests Means Nothing

I never set out to build VentureTwin AI as just another chatbot. The idea was much bigger than answering questions. I wanted to build a digital twin that could understand a student's entire journey—their projects, certifications, technical skills, academics, achievements, and career interests—and use all of that to provide meaningful career guidance. Instead of simply recommending jobs based on keywords or certificate counts, I wanted the system to answer a much harder question: What is this student actually good at, and where are they most likely to succeed? To make that possible, I designed the platform as a collection of independent intelligence modules. The Certificate Intelligence module retrieved and verified certifications. Resume Intelligence evaluated technical skills and experience. Project Intelligence analyzed project metadata such as technology stack, complexity, implementation, and impact. Each module produced its own output, which was then passed to a scoring engine that generated a Career Readiness Score. Individually, every module worked exactly as expected. Then I compared two student profiles. The first student had completed more than 20 online certifications but had only a couple of basic projects. The second student had fewer certifications, but had built full-stack applications, worked with AI models, contributed to open-source projects, and actively participated in hackathons and technical competitions. I expected the second profile to receive stronger recommendations. It didn't. Instead, the student with the larger collection of certificates consistently received the higher Career Readiness Score. At first, I assumed something was broken. I traced every stage of the scoring pipeline, inspected API responses from every module, verified the PostgreSQL records, and even recalculated the scores manually. Every value matched. Every API response was correct. The database contained exactly what it should. The scoring engine was behaving exactly as I