bash and readline: tab completion in a user input loop?

Question

I'm making a bash script which presents a command line to the user.

The cli code is as this:

#!/bin/bash

cmd1() {
    echo $FUNCNAME: "$@"
}

cmd2() {
    echo $FUNCNAME: "$@"
}

cmdN() {
    echo $FUNCNAME: "$@"
}

__complete() {
    echo $allowed_commands
}

shopt -qs extglob

fn_hide_prefix='__'
allowed_commands="$(declare -f | sed -ne '/^'$fn_hide_prefix'.* ()/!s/ ().*//p' | tr '\n' ' ')"

complete -D -W "this should output these words when you hit TAB"

echo "waiting for commands"
while read -ep"-> "; do
    history -s $REPLY
    case "$REPLY" in
        @(${allowed_commands// /|})?(+([[:space:]])*)) $REPLY ;;
        \?) __complete ;;
        *) echo "invalid command: $REPLY" ;;
    esac
done

Clarification: made and tested in Bash 4

So, "read -e" gives readline capabilities, i can recall commands, edit the input line, etc. What i cannot do in any way is to have readline's tab completion to work!!

I tried two things:

1. How it should be supposedly done: using the bash builtins "complete" and "compgen", which is reported to work here Update: it's not reported to work in scripts.

2. This ugly workaround

Why doesn't readline behave correctly when using "complete" inside the script? it works when i try it from bash in interactive mode...

Answer 1

After trying a custom completion script that I know works (I use it every day) and running into the same issue (when rigging it up similar to yours), I decided to snoop through the bash 4.1 source, and found this interesting block in bash-4.1/builtins/read.def:edit_line():

old_attempted_completion_function = rl_attempted_completion_function;
rl_attempted_completion_function = (rl_completion_func_t *)NULL;
if (itext)
  {
    old_startup_hook = rl_startup_hook;
    rl_startup_hook = set_itext;
    deftext = itext;
  }
ret = readline (p);
rl_attempted_completion_function = old_attempted_completion_function;
old_attempted_completion_function = (rl_completion_func_t *)NULL;

It appears that before readline() is called, it resets the completion function to null for some reason that only a bash-hacking long beard might know. Thus, doing this with the read builtin may simply be hard-coded to be disabled.

EDIT: Some more on this: The wrapping code to stop completion in the read builtin occurred between bash-2.05a and bash-2.05b. I found this note in that version's bash-2.05b/CWRU/changelog file:

• edit_line (called by read -e) now just does readline's filename completion by setting rl_attempted_completion_function to NULL, since e.g., doing command completion for the first word on the line wasn't really useful

I think it's a legacy oversight, and since programmable completion has come a long way, what you're doing is useful. Maybe you can ask them to add it back in, or just patch it yourself, if that'd be feasible for what you're doing.

Afraid I don't have a different solution aside from what you've come up with so far, but at least we know why it doesn't work with read.

EDIT2: Right, here's a patch I just tested that seems to "work". Passes all unit and reg tests, and shows this output from your script when run using the patched bash, as you expected:

$ ./tabcompl.sh
waiting for commands
-> **<TAB>**
TAB     hit     output  should  these   this    when    words   you
->

As you'll see, I just commented out those 4 lines and some timer code to reset the rl_attempted_completion_function when read -t is specified and a timeout occurs, which is no longer necessary. If you're going to send Chet something, you may wish to excise the entirety of the rl_attempted_completion_function junk first, but this will at least allow your script to behave properly.

Patch:

--- bash-4.1/builtins/read.def     2009-10-09 00:35:46.000000000 +0900
+++ bash-4.1-patched/builtins/read.def     2011-01-20 07:14:43.000000000 +0900
@@ -394,10 +394,12 @@
        }
       old_alrm = set_signal_handler (SIGALRM, sigalrm);
       add_unwind_protect (reset_alarm, (char *)NULL);
+/*
 #if defined (READLINE)
       if (edit)
        add_unwind_protect (reset_attempted_completion_function, (char *)NULL);
 #endif
+*/
       falarm (tmsec, tmusec);
     }

@@ -914,8 +916,10 @@
   if (bash_readline_initialized == 0)
     initialize_readline ();

+/*
   old_attempted_completion_function = rl_attempted_completion_function;
   rl_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/
   if (itext)
     {
       old_startup_hook = rl_startup_hook;
@@ -923,8 +927,10 @@
       deftext = itext;
     }
   ret = readline (p);
+/*
   rl_attempted_completion_function = old_attempted_completion_function;
   old_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/

   if (ret == 0)
     return ret;

Keep in mind the patched bash would have to be distributed or made available somehow wherever people would be using your script...