I need to convert from UTF-8 to CP 850, and would like to support as many different combinations as possible, for example accented characters as well as normal character plus combining diacritic.
Right now I am trying to find out if ICU handles the codes 1-31 (possibly except 9, 10, 13) in CP 850. Will they be the graphical symbols or will they be ASCII control codes?
By definition what you're talking about are control codes and are mapped as such by the Unicode specification to U+0001..U+001F (1..31). See the ICU Converter Explorer for example.
If you wanted the graphical representations instead, you'd need to find the graphical representations' Unicode equivalent and convert each code point yourself.