Search code examples
bashunix

Uniq many files in place

I have many *.dat files. What's a bash script that could remove duplicate adjacent lines in each one?

Solution

  • You mean like this?

    #!/bin/bash
for f in "$@"
do
   cp "$f" /tmp/tmp.dat
   uniq /tmp/tmp.dat > "$f"
done

    Where you can run in the directory that has your many *.dat files. If you put this in a script called uniq_dat, and make it executable, you can run it like:

    uniq_dat *.dat