ZSH: Why aren't some commands in the builtin array `$commands`?

I was googling how to verify the availability of commands.
In the process, I found the builtin variable $commands.
https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#index-commands

But when I tried, some commands were not in $commands even though they were available.

I want to get insights whether it can be useful to check for the existence of commands by knowing that timing and conditions.

Solution

tl;dr

You may have command(s) moved or installed that have never been run or the shell has never traversed the location of the command(s) subsequent to their move or installation and as a result the command hash table, which $commands accesses, is not up-to-date with these commands.

If you need a reliable way to check if a command exists use command -v, see this answer.

If you just want to see the command(s) in the command hash table run hash -r or rehash (ZSH only).

Context

What is `$commands`?

You've found this in the documentation you've linked in your question:

This array gives access to the command hash table. The keys are the names of external commands, the values are the pathnames of the files that would be executed when the command would be invoked. Setting a key in this array defines a new entry in this table in the same way as with the hash builtin. Unsetting a key as in ‘unset "commands[foo]"’ removes the entry for the given key from the command hash table.

– https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#index-commands

"Gives access to the command hash table"... let's see what this is.

What is the "command hash table"?

For this we'll go to chapter 3 of the ZSH Guide, specifically section 1 (External Commands).

The only major issue is therefore how to find [external commands]. This is done through the parameters $path and $PATH [...].

There is a subtlety here. The shell tries to remember where the commands are, so it can find them again the next time. It keeps them in a so-called 'hash table', and you find the word 'hash' all over the place in the documentation: all it means is a fast way of finding some value, given a particular key. In this case, given the name of a command, the shell can find the path to it quickly. You can see this table, in the form key=value, by typing hash.

In fact the shell only does this when the option HASH_CMDS is set, as it is by default. As you might expect, it stops searching when it finds the directory with the command it's looking for. There is an extra optimisation in the option HASH_ALL, also set by default: when the shell scans a directory to find a command, it will add all the other commands in that directory to the hash table. This is sensible because on most UNIX-like operating systems reading a whole lot of files in the same directory is quite fast.

– https://zsh.sourceforge.io/Guide/zshguide03.html

Ok, so we now know $commands gives access to the command hash table which is essentially a cache of locations. This cache needs to be updated, right?

When does the command hash table get updated?

1. When the command is found for the first time

We know this from the documentation above:

The shell tries to remember where the commands are, so it can find them again the next time. It keeps them in a so-called 'hash table' [...].

There is additional documentation, for HASH_CMDS, that supports this:

Note the location of each command the first time it is executed. Subsequent invocations of the same command will use the saved location, avoiding a path search.

– https://github.com/zsh-users/zsh/blob/daa208e90763d304dc1d554a834d0066e0b9937c/Doc/Zsh/options.yo#L1275-L1283

2. When your shell scans a directory looking for a command and finds other commands

Again, we know this because of the above documentation:

There is an extra optimisation in the option HASH_ALL, also set by default: when the shell scans a directory to find a command, it will add all the other commands in that directory to the hash table.

Ok this is all the context.

Back to the problem, why?

Why are the commands you are looking for not appearing in the $commands array? Why are they not in the command hash table?

Well, we now know when the command hash table is updated, so we can surmise that the udpate conditions weren't met and some possibilities as to what situation you may be in:

When the command is found for the first time
- You have recently installed a new command and have never run it.
- You have had an existing command moved and have never run it.
When your shell scans a directory looking for a command and finds other commands
- You have not run a command that necessitated a path search that would have traversed the location where your new/moved command is installed.

Anything that can be done?

It depends on what you're doing.

It's critical I know the existence of a command

Don't use $commands. Use command -v <command_name>, see this answer.

I just want the see the command in the command hash table

You can force the command hash table to update with hash -r or in ZSH rehash.