Braille cursor tracking confused when moving through unicode characters

[mdcurran]

When viewing the following line on a braille display with NVDA and UEBC grade 1 and no expand to computer braille:

The middle character in xx is not handled properly.

(there is a 0x7f character between the two x characters).

When arrowing through the characters up to the first x, everything is fine.
When on the x, the braille cursor is on the x (after the letter sign).
When moving one character right (on to the unicode 0x7f character) the braille cursor is now under the lettersign of the first x (i.e. it moved one cell to the left).
When moving one character right (now on the second x) the braille cursor is correctly on the second x (after the lettersign).

Note that this problem seems to occure for any unicode character where its escape hex number is printed on the display. E.g. if I type some english but include some Chinese characters on the same line.

Guessing this is an issue with libloui and mapping input offsets to output offsets?

[mdcurran]

Some further odities, which may be related:
*A hex unicode character will not be printed at all on the display if it is the first character on the line and there is less than two characters after it.
*A hex unicode character as the first character on the display with two alphanumeric characters after it: when arrowing through the line the braille cursor starts moving through the \xHex representation.
*A hex unicode character at the end of the line will only be printed on the display if there are at least 4 other characters before it.

[mdcurran]

One more note:
Using English 6 dot computer braille as the table the original problem no longer exists. However the further problems (unicode characters not appearing depending on amount of characters) still exist.

[jteh]

Definitely a liblouis bug. The position arrays aren't being updated correctly when the unknown characters are being replaced.

[jteh]

Replying to mdcurran:

Some further odities, which may be related:
*A hex unicode character will not be printed at all on the display if it is the first character on the line and there is less than two characters after it.
*A hex unicode character as the first character on the display with two alphanumeric characters after it: when arrowing through the line the braille cursor starts moving through the \xHex representation.
*A hex unicode character at the end of the line will only be printed on the display if there are at least 4 other characters before it.

These are all due to the Python bindings not providing a large enough output buffer to liblouis. I've proposed some ideas and am awaiting feedback on the liblouis community on the best way to fix this in the Python bindings.

[jteh]

Replying to jteh:

Definitely a liblouis bug. The position arrays aren't being updated correctly when the unknown characters are being replaced.

I fixed this in liblouis svn r365.

[jteh]

Replying to jteh:

These are all due to the Python bindings not providing a large enough output buffer to liblouis.

I fixed this in liblouis svn r371.

However, we also now have another option. We can specify that undefined characters be displayed with a certain character (e.g. space) instead of displaying the hexadecimal value. I think this is more user friendly for screen reader users.

All of this will be in the next liblouis release. I can try to request one, but I want to wait to see what happens regarding the proposed changes to the Danish tables, as an NVDA user has requested this.

[jteh]

(In #484) Confirmed. These are liblouis bugs.

The problems exhibited in steps 8 and 9 should be fixed by my changes to liblouis as part of #499.

I understand what is causing the earlier problem, but don't know how to fix it yet.

[jteh]

Fixed in liblouis 2.1.0. We now require liblouis 2.1.0 in changeset:main,3731.