Human-in-the-Loop Playwright Automation: Best Way to Stream Backend Browser for OTP/CAPTCHA Handling?

Hi everyone, We're building an automation platform using Playwright where all browser automation runs on the backend. For portals that require manual intervention (OTP, CAPTCHA, MFA, document uploads, etc.), we're exploring a way to let users temporarily view and interact with the running backend browser from our React application, after which automation would resume automatically. Our goals are: Keep all automation logic on the backend Support human intervention only when necessary Scale to bulk processing workflows Deploy reliably in production We're currently evaluating approaches such as CDP screencasting, VNC/noVNC, and WebRTC-based browser streaming. Has anyone built something similar in production? What architecture did you choose, and what were the biggest challenges around scalability, latency, security, session management, and CAPTCHA/OTP workflows? Also, is there a better alternative than live browser streaming for this use case? Any advice, experiences, or open-source projects would be greatly appreciated. submitted by /u/Loud_Ice4487 [link] [留言]