How to use awk to generate fixed width columns with colored input?

Question

I am taking the output from 'ls -l' and passing it through awk to reformat it. This works:

list=$(ls --color=none -l | tail -n+2)
printf '%s' "$list" | awk '{printf "%-40s more stuff\n", $9}'

It produces something like:

env_profiles                             more stuff
ls_test.sh                               more stuff
saddfasfasfdfsafasdf                     more stuff
test                                     more stuff

But with --color=always it produces:

env_profiles                 more stuff
ls_test.sh                   more stuff
saddfasfasfdfsafasdf             more stuff
test                             more stuff
                                         more stuff

"env_profiles" is a directory, "ls_test.sh" is an executable file, so they are both colored and end up with different alignment. Also there is an extra line.

EDIT: Modified answer based on Ed Morton's post. Gets rid of extra line, handles filenames with spaces:

ls --color=always -l | tail -n+2 | awk '
{
    $1=$2=$3=$4=$5=$6=$7=$8=""
    field = substr($0,9)
    nameOnly = $0
    gsub(/\x1b[^m]+m/,"",nameOnly)
    if( length(field) - length(nameOnly) >= 0 ) {
        printf "%-*s more stuff\n", 40 + length(field) - length(nameOnly), field
    }
}'

Answer 1

The field ($9) that contains your colored file names starts and ends with control characters to produce the color on your terminal, e.g. in this case foo is colored on the screen but bar is not:

$ cat o1
-rwxr-xr-x  1 emorton Domain Users     21591 Nov 12  2011 foo
-rwxr-xr-x  1 emorton Domain Users     21591 Nov 12  2011 bar

$ cat -v o1
-rwxr-xr-x  1 emorton Domain Users     21591 Nov 12  2011 ^[[01;32mfoo^[[0m
-rwxr-xr-x  1 emorton Domain Users     21591 Nov 12  2011 bar

so when you printf that field in awk and give it a field width of N characters, the color-producing strings are counted as part of the width but then if they are non-printing or backspaces or whatever then the end result will not not show them and it'll look like it's using less space a field that did not contain those characters. Hope that makes sense.

It looks to me like the coloring strings always start with the character \x1b then some coloring instruction and end with m so try this:

$ awk '{
    nameOnly = $NF
    gsub(/\x1b[^m]+m/,"",nameOnly)
    printf "<%-*s>\n", 10 + length($NF) - length(nameOnly), $NF
}' o1
<foo       >
<bar       >

Note that your approach of using a specific field only works if there's no spaces in the file names