I stopped letting LLMs guess financial facts

LLMs can be surprisingly useful for company research. But I kept running into a strange split: parts of the reasoning were useful, while the financial facts underneath them were much harder to trust. A model could identify an accounting risk in one paragraph, then mix fiscal periods, accounting scopes, or currencies in the next. Missing values might quietly become zeros. A deterministic calculation could be performed probabilistically. A citation could point to a real filing without actually supporting the claim. Those are different failure modes, and treating all of them as one giant prompting problem did not feel like a reliable architecture. So I started building OpenThesis , an Apache-2.0 desktop system for evidence-first, AI-assisted company research. The project is not a stock picker or a trading bot. The idea is simpler: use ordinary software for work that should be deterministic, and give the LLM a bounded evidence set for the reasoning work where it can actually help. The monolithic prompt is doing too many jobs A common company-research workflow looks roughly like this: company question ↓ LLM ↓ answer That single model call is implicitly responsible for remembering reported values, selecting the right fiscal period, recognizing the accounting scope, finding sources, performing calculations, comparing scenarios, identifying risks, and writing a conclusion. Some of those tasks are probabilistic by nature. Others are not. Qualitative reasoning, connecting evidence, forming scenarios, and challenging an assumption are reasonable uses of a language model. Remembering an exact reported value, deciding whether a value is missing, and calculating a margin or valuation are poor places to accept probabilistic behavior. My design rule became: Deterministic work should stay deterministic. Use LLMs for reasoning, not as the database and calculator underneath the reasoning. Evidence before reasoning OpenThesis starts from official filings rather than from model memory o