Search code examples
windowswinapiconsole

Non documented Console Input Mode flags

I'm working on my library in Rust for making GUIs in terminals using Windows API.

While I implemented getting and setting the console mode, I noticed that for input mode the default value is 503. I went to check the documentation for the flags, but there are no values 128 and 256 documented, without which you can't get 503.

Either way, they are enabled by default, as said in the documentation:

When a console is created, all input modes except ENABLE_WINDOW_INPUT and ENABLE_VIRTUAL_TERMINAL_INPUT are enabled by default.

Which, when included with 128 and 256, evaluates to 503, the value I'm getting!

Are these two flags a thing of the past that Microsoft just doesn't want to remember? What was/is their function?

I've been already looking through the Internet, but couldn't find any remarks about the two flags missing from the documentation.

Solution

  • The mode is a bitmask. The integer value 503 contains the following flags, as defined in wincon.h and consoleapi.h:

    0x0001 ENABLE_PROCESSED_INPUT
0x0002 ENABLE_LINE_INPUT
0x0004 ENABLE_ECHO_INPUT
0x0010 ENABLE_MOUSE_INPUT
0x0020 ENABLE_INSERT_MODE
0x0040 ENABLE_QUICK_EDIT_MODE
0x0080 ENABLE_EXTENDED_FLAGS
0x0100 ENABLE_AUTO_POSITION

    The two flags you are asking about are ENABLE_EXTENDED_FLAGS and ENABLE_AUTO_POSITION.

    The only mention of ENABLE_EXTENDED_FLAGS I can find on MSDN is in the SetConsoleMode() documentation:

    Value Meaning
    ENABLE_QUICK_EDIT_MODE 0x0040 This flag enables the user to use the mouse to select and edit text. To enable this mode, use ENABLE_QUICK_EDIT_MODE | ENABLE_EXTENDED_FLAGS. To disable this mode, use ENABLE_EXTENDED_FLAGS without this flag.

    See How to disable user selection in Windows console for an example.

    ENABLE_AUTO_POSITION is not documented on MSDN. The only thing I can find for it anywhere online is the following blurb:

    https://txwizard.github.io/WizardWrx_NET_API/api/WizardWrx.ConsoleStreams.StandardHandleInfo.ConsoleModes.html

    Name Description
    ENABLE_AUTO_POSITION Though defined in WinCon.h, this flag is otherwise undocumented.

    My initial best guess is that it is related to the "Let system position window" check box on the property sheet of a CMD.EXE window, though I have yet to test this theory.

    That same page also mentions ENABLE_EXTENDED_FLAGS, too:

    Name Description
    ENABLE_EXTENDED_FLAGS Required to enable or disable extended flags.

    See ENABLE_INSERT_MODE and ENABLE_QUICK_EDIT_MODE.
    ENABLE_INSERT_MODE When enabled, text entered in a console window will be inserted at the current cursor location and all text following that location will not be overwritten. When disabled, all following text will be overwritten.

    To enable this mode, use ENABLE_INSERT_MODE | ENABLE_EXTENDED_FLAGS.

    To disable this mode, use ENABLE_EXTENDED_FLAGS without this flag.
    ENABLE_QUICK_EDIT_MODE This flag enables the user to use the mouse to select and edit text.

    To enable this mode, use ENABLE_QUICK_EDIT_MODE | ENABLE_EXTENDED_FLAGS.

    To disable this mode, use ENABLE_EXTENDED_FLAGS without this flag.