How to Test AI-Powered Web Apps Without Treating the Model Like a Normal API

AI-powered web applications look familiar on the surface. They have text boxes, buttons, menus, loading indicators, and API calls. That makes it tempting to test them like any other web application: submit an input, wait for a response, and compare the output with an expected string. That approach breaks quickly. Model output is variable. Safety behavior depends on context. A response can be semantically correct but displayed in the wrong conversation. An agent can produce a convincing final message after calling the wrong tool. A prompt-injection defense can block obvious attacks while failing when malicious instructions arrive through a webpage, document, image, or previous message. Testing these applications requires two kinds of evidence at the same time: Deterministic product evidence: the UI, state, permissions, tool calls, and workflow behaved correctly. Probabilistic model evidence: the output stayed within an acceptable range across repeated and adversarial inputs. Prompt injection is a workflow problem Prompt injection testing is often reduced to pasting “ignore previous instructions” into a chat box. That is a useful smoke test, but it does not represent how browser-based agents encounter untrusted content. An agent may read instructions from: A webpage. A support ticket. A PDF. A hidden DOM node. An email. A retrieved knowledge-base entry. A tool response. A previous conversation turn. The guide on testing prompt injection defenses in AI-powered browser workflows provides a good foundation. The test should verify more than the final sentence. It should inspect whether the agent: Treated external content as data rather than authority. Attempted a prohibited tool call. Exposed secrets in an intermediate step. Navigated to an unapproved domain. Changed its goal after reading untrusted content. Requested confirmation before a sensitive action. Preserved the original user instruction. A safe final answer does not prove that the workflow was safe. The agent ma