Create a "Flat review" mode

[cnorman]

In most windows screen readers, they create the "Virtual focus" mode, or be it, possibly under a different name.

The (opensource) screen reader Orca has a "Flat review" mode, where the user uses the keys on the number pad to navigate the screen, 7 is previous line, 8 speaks the currentline, 9 speaks the nextline down, 6 thenext word, 5 the current word, 4 the previous word, and 1, 2, and 3 are the same with characters, I think this would be a great feature for NVDA, to be able to read the hole screen from the number pad. Also, there whould be a feature for laptops, which works on the u, i, o, j, k, l, m, _{and . keys. I also think there should be two sets of key bindings, but that's a different ticket.}

[pvagner]

Under windows creating flat representation of a window means hooking display functions or creating video intercept driver.
Currently there are some experients happening you may like to see ticket 40. Anyway I think it will take some time to make it really usable.
Display hooks seems to be good compromis for NVDA because it is still know to help with such issues and in most cases does not require administrative privileges to install.

[aleksey_s]

let me disagree with you. you can use msaa info to create flat review of whole window as well, ofcourse not all possible info will be available, but even in this case it might be very usefull. msaa provides info also about real coords of controls, however we must use msaa even we will make display hooks, so user might know how control it is when reviewing virtual model as in jaws.

[pvagner]

yes you are right it is possible to do such calculations with info provided by MSAA. But given the fact object navigation is working well it will show no benefit. Therefore in extreme cases this might be seen as a duplicate feature.

[aleksey_s]

again, i am disagree. my opinion is that flat navigation will be more usefull and confortable. however, all people i recommend nvda, if they even do not know jaws feel theyselves not comfort with undersdanding and using navigation model as in nvda. problem is that most controls are ungrouped and there can be e.g. text after buttons and before it (as in mycomputer properties etc). so implementing flat review for me is very priority feature.

[mdcurran]

As much as I eventually want to see some kind of screen layout that can be reviewed, I don't think this should be waited on by 0.6. There are too many other bugs. However, as I said, I do believe this is important, but just not for 0.6.

[aleksey_s]

i guess, this might be implemented with virtual buffers concept. so user press some keystroke and goes into virtual representation of current window, even with some specific keystrokes working inside it as in current buffers. is it such problematicaly to do it or developers things it is unnecessary?
however, i thing we do not need to abuse display hooks here. my opinion is that the display hooks nvda must use to receive text of specific controls (or if nvda can not recognize some as control but hook library reports there are portion of text - nvda must create object whatever), and creating flat review by taking position and other info from each nvda object.

[pvagner]

I have got quite a lot of request to do something like this from the czech and slovak community. Therefore I have to adapt to all the thoughts and ideas flying here because I don't have better approach my-self.
How do you see this interfacing to the virtual buffers. Should this be written in python or in c++?

[mdcurran]

I think at the moment, the best chance we have for this is to reply on NVDA's display models. It would be then possible to review a flat representation of the entire application. However some applications do not write their text in a standard way, so the display models will not pick them up (e.g. Java, Gecko). But in general, this would work well enough. We also may consider key strokes to move to the deepest NVDAObject at the particular review location, e.g. you arrow down to the ok button in a dialog then you should be able to get the navigator object to that button as well.
If virtualBuffers were used instead, we would have a tough time getting the screen layout right, especially for objects that are beside each other. Plus, in-process is fast by is limited to one accessibility API. Out-of-process is slow, but is not limited to one accessibility API.
For now I think display model is more what people are looking for.

[mdcurran]

NVDA main as of now already allows you to review a flat representation of an object hyerarchi, either by a treeInterceptor (virtualBuffer or compoundDocument) if it exists, or by a display model, at least for window IAccessibles.
However, the interaction model between object navigation, review cursor, and these particular flat representations is currently a little complex.
A branch has been started to come up with a much more understandable way of interacting.

This will add functionality to:

• allow for users to move from their current navigator object, up to the closest flat representation, though having the review cursor end up at the position of the original navigator object, within this flat representation.

E.g. you are on the Ok button of the File properties dialog in Windows Explorer. Pressing this "moveToFlatReview" script would set the review cursor so that it was reviewing the displayModel for that dialog, however it would be positioned on the word "Ok" in that representation.

• Allow for users to set their navigator object to the deepest object at the current position of the review cursor in the flat representation.

E.g. you are reviewing the flat representation of the Windows Explorer file properties dialog and you are on the word "Ok". Pressing this script would now set your navigator object to the Ok button object.

With this functionality in place, NVDA would also no longer automatically set the navigator object to the deepest object when reviewing a flat representation. Rather they would need to execute the script.

This should hopefully make the experience a little more easy to understand as the navigator object and review cursor arn't anymore magically mvoing around due to the other moving.

The branch is at:
 http://bzr.nvaccess.org/nvda/flatReviewPosition

[aleksey_s]

(In #788) Since it seems to be not very difficult to implement, but it will bring a great convenience for users, I would like to see it taken for 2010.2.

[jteh]

Merged in changeset:main,3666.