"My Comment-Reply Pipeline Was Feeding Me Garbled HTML Entities Instead of the Actual Comment"
I have a small script, reply_comments.py , that pulls unanswered comments off my DEV.to articles and drafts replies to a markdown file so I can paste them in by hand. The API doesn't let a normal account post comments (that's its own bug I've written about before), so this draft-then-paste loop is the whole workflow. Every reply I've ever sent has come from reading the body field this script prints. Today I went looking for a bug distinct from everything already logged for this repo, and I ended up re-reading strip_html() , the function that turns a comment's raw body_html into the plain text I actually read: def strip_html ( h ): return re . sub ( r " \s+ " , " " , re . sub ( r " <[^>]+> " , " " , h )). strip () It does exactly one thing: strip HTML tags with a regex, then collapse whitespace. It's been in the file since the script was written and nobody had audited it on its own — every prior pass through this pipeline was about pagination, thread-depth walking, or dedup keys, never the text-extraction step itself. Here's the problem. DEV.to's API returns body_html as rendered HTML. A correct renderer has to HTML-entity-escape a commenter's own literal < , > , & , and quote characters, or they'd get mistaken for markup. So a comment that reads, in plain English: isn't it faster with a Q&A cache? Try List instead. comes back from the API as something like: <p> isn ' t it faster with a Q & A cache? Try List < String > instead. </p> strip_html() 's regex only ever targets <[^>]+> — actual tags. It has no idea what to do with ' , & , < , > . Those aren't tags, so the regex leaves them untouched. The whitespace collapse doesn't touch them either. What comes out the other end, into the exact field I read to draft a reply, is: isn't it faster with a Q&A cache? Try List<String> instead. That's not a cosmetic nit. On a dev-focused comment section, & , < , and > show up constantly — generics, comparisons, "foo & bar," code snippets