Why pasted text keeps breaking search and formatting (and the regexes I ended up using to clean it)

I kept running into a boring problem that was harder to debug than it should have been: text that looked normal, but behaved wrong the moment I pasted it into a CMS, a spreadsheet, or a code comment. Search would fail. Line breaks would get weird. A heading copied from ChatGPT would drag Markdown markers along with it. Sometimes the only visible clue was that the punctuation felt slightly "off." What finally made this manageable wasn't some big NLP trick. It was going back to the dumb, reliable layer: exact character matching. The tool I built for this is basically a pile of small, deterministic cleanups for the specific junk that copied text tends to accumulate — full-width punctuation mixed into ASCII, invisible Unicode code points, curly quotes, em dashes, leftover Markdown, and whitespace noise. The most useful part is the invisible-character scan, not the cleaning The piece I trust most in the whole component is the part that explicitly names which invisible characters it cares about, then counts them by code point. It's not doing a vague "this text seems suspicious" pass. It has a hard-coded inventory: const invisibleDefs = [ { key : " zwsp " , codes : [ 0x200b ] }, { key : " zwnj " , codes : [ 0x200c ] }, { key : " zwj " , codes : [ 0x200d ] }, { key : " bomZwnbsp " , codes : [ 0xfeff ] }, { key : " wordJoiner " , codes : [ 0x2060 ] }, { key : " softHyphen " , codes : [ 0x00ad ] }, { key : " bidiMarks " , codes : [ 0x200e , 0x200f , 0x202a , 0x202b , 0x202c , 0x202d , 0x202e ] }, ]; const codesToRegex = ( codes ) => new RegExp ( `[ ${ codes . map (( c ) => " \\ u " + c . toString ( 16 ). padStart ( 4 , " 0 " )). join ( "" )} ]` , " g " ); const analyzeInvisible = ( str ) => { const breakdown = invisibleDefs . map (( def ) => ({ key : def . key , count : ( str . match ( codesToRegex ( def . codes )) || []). length , })); const total = breakdown . reduce (( sum , row ) => sum + row . count , 0 ); return { breakdown , total }; }; I like this because it's brutall