So apparently it's called the Options Button, and in later versions there is an
IsOptionsButtonVisible
Which can be set to false.
For some reason this isn't available in my current version so just setting
OptionsButtonText = string.Empty;
worked for me.