What markup language for richly formatted content?

When you are developing a web-based application and you want to allow richly formatted text from the user you have to make a choice about how to allow that input. Many different markup languages have been created because it is arguably more difficult to sanitize HTML.

What are the advantages and disadvantages of the various different markup languages like:

Or to put it differently, what factors do you consider when choosing to use a particular markup language.

Solution

Markdown, BBCode, Textile, MediaWiki markup are all basically the same general concept, so I would really just lump this into two categories: HTML, and plain text markup.

HTML

The deal with HTML is the content is already in a "presentable" form for web content. That's great, saves processing time, and it's a readily parse-able language. There are dozens of libraries in pretty much any language to handle HTML content, convert to/from HTML to other formats, etc. The main downside is that because of the loose standards of the early web days, HTML can be incredibly variable and you can't always depend on sane input when accepting HTML from users. As pointed out, tidying or santizing HTML is often very difficult, especially because it fails to follow normal markup rules the way XML does (i.e. improperly closed tags are common).

Plain Text Markup

This category is frequently used for the following reasons:

Easy to parse into multiple forms from one source - PDF, HTML, RTF
Content is stored in readable plain text (usually much easier to read than raw HTML) if needed at some later date, rather than needing to extract from the HTML
Follows specific defined rules where HTML can be annoying variable and unstructured
Allows you to force a subset of content formatting that's more appropriate in many cases than simply allowing full HTML
In addition to forcing a subset of HTML makes it easy to sanitize input and prevent cross site scripting problems etc.
Keeping the "raw" data in an abstracted format means that at a later date, if you for instance wanted to convert your site from HTML 4 to XHTML, you only need to change the parsing code. With HTML formatted user input, you're stuck now having to convert all the HTML to XHTML individually, which as HTML Tidy shows, is not always a simple task. Similarly if a new markup language comes along at some point or you need to move to an alternative format (RTF, PDF, TeX) an abstracted restricted subset of text formatting options makes that a much simpler task.

Bottom line is what is the user input being used for. If you're planning to keep the data around and may need to shuffle formats etc. then it makes sense to use a careful abstract format to store the information. If you need to work with the raw data manually for any reason, then bonus points if that format is easily human-readable. If you're only displaying the content in a web page (or HTML doc for a report etc.) and you have no concerns about converting it or future-proofing it, then it's a reasonable practice to store it in HTML.