Search code examples
emacsline-breakscarriage-returnline-endingseol

How to find and remove the invisible characters in text file using emacs


I have a .txt file named COPYING which is edited on windows.
It contains Windows-style line breaks :

$ file COPYING 
COPYING: ASCII English text, with CRLF line terminators

I tried to convert it to Unix style using dos2unix. Below is the output :

$ dos2unix COPYING 
dos2unix: Skipping binary file COPYING

I was surprised to find that the dos2unix program reports it as a binary file. Then using some other editor (not Emacs) I found that the file contains a control character. I am interested in finding all the invisible characters in the file using Emacs.

By googling, I have found the following solution which uses tr :

tr -cd '\11\12\40-\176' < file_name

How can I do the same in an Emacs way? I tried the Hexl mode. The Hexl mode shows text and their corresponding ASCII values in a single buffer which is great. How do I find the characters which have ASCII values other than 11-12, 40-176 (i.e tab, space, and visible characters)? I tried to create a regular expression for that search, but it is quite complicated.


Solution

  • Emacs won't hide any character by default. Press Ctrl+Meta+%, or Esc then Ctrl+% if the former is too hard on your fingers, or M-x replace-regexp RET if you prefer. Then, for the regular expression, enter

    [^@-^H^K-^_^?]
    

    However, where I wrote ^H, type Ctrl+Q then Ctrl+H, to enter a “control-H” character literally, and similarly for the others. You can press Ctrl+Q then Ctrl+Space for ^@, and usually Ctrl+Q then Backspace for ^?. Replace all occurrences of this regular expression by the empty string.

    Since you have the file open in Emacs, you can change its line endings while you're at it. Press C-x RET f (Ctrl+X Return F) and enter us-ascii-unix as the new desired encoding for the file.