In section 2.5.3 of IEEE Std 1003.1, 2004 edition's sh, regarding the PS1 environment variable, it is defined:
Each time an interactive shell is ready to read a command, the value of this variable shall be subjected to parameter expansion and written to standard error. The default value shall be "$ ". For users who have specific additional implementation-defined privileges, the default may be another, implementation-defined value. The shell shall replace each instance of the character '!' in PS1 with the history file number of the next command to be typed. Escaping the '!' with another '!' (that is, "!!" ) shall place the literal character '!' in the prompt. This volume of IEEE Std 1003.1-2001 specifies the effects of the variable only for systems supporting the User Portability Utilities option.
So it seems ! can be used in the PS1 variable to be replaced by the history file number of the next command to be typed. If you try that in a Bourne shell implementation such as dash, you find it not implemented. It's also not implemented in today's GNU bash.
Which shell implements that?
bash implements this if you set the POSIXLY_CORRECT environment variable or invoke bash with the --posix option:
PS1='!$ ' POSIXLY_CORRECT=1 bash
2$ echo foo
foo
3$ !2
echo foo
foo
3$