Search code examples
unicodeuwptextboxright-to-left

Why does the TextBox character ordering change with FlowDirection RightToLeft

In my UWP app I have a textbox. I want the user to be able to type Farsi / Persian text (right to left) into the textbox so I set the FlowDirection property to RighToLeft.

The text can be entered and is displayed correctly:

enter image description here

When I save the text, and inspect the property during debugging, i see the same character order as on screen:

enter image description here

The same character order applies for the stored value when viewed with mssql management studio:

enter image description here

When I add a '.' or a '!' at the end of the text, the WPF textbox still displays what I expect,

enter image description here

but the text I get back from the text property puts the exclamation mark at the right side of the string.

enter image description here

It is also stored this way in the sql database:

enter image description here

When loading the database value (with the exclamation point on the right) into the textbox it shows the exclamation point correctly on the left side. There must be some magic happening here that I am not aware of, or maybe the problem is that the debug preview / mssql preview does not support displaying RTL values.

My problem is that this magic does not work in other situations. When I load the database value and put it in a microsoft word document, it seems to do no conversion and place the text in the document exactly as it is in the database, resulting in the exclamation point to be shown on the 'wrong' side.

I would like to understand the 'magic' that takes place in displaying / storing these strings, so I can output it correctly in MS Word. And Yes, I have set the paragraph where I output the values in word to RTL.

Solution

  • In Unicode, all characters have directional properties that get used in the Unicode Bidirectional Algorithm for determining how characters are ordered visually. Most characters have a "strong" directional property, but not all. In particular, most punctuation characters are considered directionally neutral.

    The visual ordering of neutral characters is determined by the characters that surround them. For example, the exclamation mark ! is neutral; if it occurs between two left-to-right characters, it will be treated as though it also is a left-to-right character. But if it occurs between two right-to-left characters, it will be treated as though it is a right-to-left character.

    In your example, though, the exclamation mark occurs at the end of the string. So, it has a strong-direction character on one side, but nothing on the other side. In this case, another factor comes into play, which is that the paragraph as a whole has a base direction.

    The Unicode Bidi Algorithm allows two ways that apps can handle the paragraph base direction:

    • the app can set the base direction explicitly, regardless of the string content in the paragraph; or
    • the app can let the base direction be derived implicitly from the string: the base direction is determined by the first strong-directional character in the string.

    In your UWP app, when you set the flow direction to RTL, then the paragraph base direction (for purposes of the Bidi Algorithm) is RTL. With an Arabic-script string that ends with the exclamation mark, the directionality of the exclamation is set to RTL because of the paragraph base direction, and so it appears at the left end of the string. But when you view the control property value in an IDE, the IDE is presenting that property string in a control that has LTR base direction. That is causing the exclamation at the logical end of the string to appear visually at the right end.

    Note that apps will often conflate base direction and alignment, though these are really distinct things. In Word, you can set the paragraph base direction in the Paragraph settings dialog, and when you do it will set the alignment to match by default:

    enter image description here



    But you can override the paragraph alignment to have a RTL base direction with left alignment:

    enter image description here



    Note that the visual order of the exclamation mark is affected by the paragraph base direction but not by the alignment. The Unicode Bidi Algorithm doesn't pay attention to the alignment.

    This article gives a good overview of how the Bidi Algorithm works: https://www.w3.org/International/articles/inline-bidi-markup/uba-basics.

    If you want to explore how the Bidi Algorithm works in more detail, you can read the spec, Unicode Standard Annex #9, Unicode Bidirectional Algorithm; and check out this Unicode utility that explains how the rules of the algorithm apply to sample strings you can provide.