Search code examples
bashsedeol

sed: replacing end of line


I need to quote end of line and using this solution to replace end of line which working perfectly when file has multiple lines.

Problem arises when I try to use this method with files that have no \n or have single line ending with \n. In this case sed does not replace anything, even second s/ command does not work. Also looks like last end of line is never replaced.

My command is:

sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' <file>

(replace \n to \\n and replace a to _ for example).

I use hexdump -C to display files here:

There are tests I made:

$ # Test1. single line without \n
$ echo -n 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63                                          |abc|
# but expected "_bc"

$ # Test2. single line with \n
$ echo 'abc' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  61 62 63 0a                                       |abc.|
# but expected "_bc\n"

$ # Test3. two lines
$ echo -e 'abc\n' | sed ':a;N;$!ba;s/\n/\\n/g;s/a/_/g' | hexdump -C
00000000  5f 62 63 5c 6e 0a                                 |_bc\n.|
# but expected "_bc\n\n"

Questions: Why second s/ does not replace anything in Test1 and Test2? Is there any method to fix replacing of s/\n/\\n/ and s/a/_/g in all this tests?

P.S.: I don't want some workaround like adding newline at end of stream before sed processing and removing it after.

EDIT: looks like N command does not read single line, even if it followed by '\n'. Any idea how to fix it?

EDIT2: Seems like this is expected behavior of N command. Is there any method on how to replace last line ending?


Solution

  • Sad, but I don't find any method to prevent sed from adding new line to output when N command is used. I didn't want to some workaround to like add new line before processing and removing it after, but seems like there is no other method.

    This is the final solution with workaround:

    sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' <<< "$VAR" | tr -d '\n'
    

    <<< like echo adds new line into input string, tr removes it.

    With that line all test are OK:

    $ echo -e "abc" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
    00000000  5f 62 63                                          |_bc|
    
    $ echo -e "abc\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
    00000000  5f 62 63 5c 6e                                    |_bc\n|
    
    $ echo -e "abc\n\n" | sed ':a;$!{N;ba};s/\n/\\n/g;s/a/_/g' | tr -d '\n' | hexdump -C
    00000000  5f 62 63 5c 6e 5c 6e                              |_bc\n\n|
    

    P.S.: thanks potong for figuring out the :a;$!{N;ba}.

    P.P.S.: If someone will give working answer without tr, I will re-accept it.