NVDA's responsiveness and stability are affected by applications that stop responding

[ianr]

I have noticed that when applications stop responding for a period of time it usually causes nvda to stop reading and stop reacting to nvda keyboard shortcuts.
I have noticed this with notepad, EdSharp?, Internet Explorer, and Visual Studio recently but I believe the problem would manifest itself similarly in any program that stops responding.
I have made a small application to help test this scenario.
I am attaching it to the ticket.

The application has a text box and a button, enter the number of seconds you would like the program to stop responding for and press the button, then you can see how nvda responds to it.
When the application begins responding again it will use the default SAPI voice on your machine to let you know.

In my personal testing I have noticed many different results, I am outlining them here:
1 NVDA stops responding for the entire time that the test application is not responding
2 NVDA becomes unresponsive for a few seconds and then becomes responsive again.
3 NVDA will continue reading items but ignores my key presses in the sense that it will not interrupt speech as it normally does when you press a key
4 NVDA continues to output to my braille display correctly though speech has stopped
5 NVDA stays responsive

The test application requires .NET Framework 4 client profile to be installed, I believe this can be obtained through windows updates.

I hope the application can be helpful in testing and fixing this issue.

[jteh]

NVDA depends on obtaining information from applications. Due to the way that accessibility in Windows works, if an application stops responding (freezes), queries for information from that application will also freeze, and in turn NVDA will freeze. Certain requests can be cancelled, so if we detect this situation, we cancel the request after 10 seconds or if the foreground window changes (e.g. you switched applications using alt+tab). Unfortunately, certain requests cannot be cancelled (without very nasty hacks), so in these cases, NVDA freezes until the application starts responding again. I experimented with some very evil code to try to work around these situations a while ago, but it has the potential to cause nasty problems. (In technical terms, the only way to get around this is to hook some pretty basic Windows API functions and override their behaviour.)

#1189 also addresses this issue, though it was originally specific to applications which freeze on exit, so I've closed it as a duplicate.

[ianr]

I wonder if the 10 seconds time limit could be reduced to 5 without causing any problems, it would be ideal to get this as low as is reasonable to improve responsiveness.

I tested with JAWS and it stays responsive when I cause the test application to stop responding so they have obviously found a way to work around this.

If you are willing to take some time to document the experimental code you were working on I am interested in looking at it.

I believe that fixing this problem would make NVDA feel much more stable but I realize that overriding low level windows API functions is not something to be undertaken lightly.

None the less, I am still interested in seeing the experimental code and gaining a better understanding of how this could be solved.

[jteh]

Replying to ianr:

I wonder if the 10 seconds time limit could be reduced to 5 without causing any problems, it would be ideal to get this as low as is reasonable to improve responsiveness.

There are times when the system is just legitimately slow due to load. If we reduce this, the problem is that it might kill off some legitimate calls. This causes real problems in, for example, MS Office, where if we fail to retrieve the native object model, everything falls over rather nastily.

I tested with JAWS and it stays responsive when I cause the test application to stop responding so they have obviously found a way to work around this.

Interesting. I've certainly seen cases where other screen readers freeze as well due to unresponsive applications (the application error dialog is a case that often seems to freeze), but each case is slightly different.

Note that most screen readers work more in-process than NVDA does, which means that their performance in other processes is sometimes unaffected by a freeze in one process. This may explain why JAWS seems to cope better here. This is an architectural difference; NVDA is primarily out-of-process and bringing the entire program in-process is not an option.

If you are willing to take some time to document the experimental code you were working on I am interested in looking at it.

There isn't really much to document. Basically, I hooked SendMessage and SendMessageTimeout for oleacc and UIAutomationCore. For SendMessage, i redirect the call to SendMessageTimeout and enforce a 1 second timeout. For SendMessageTimeout, I ensure that the timeout can never be more than 2 seconds, which I do because Windows accessibility uses ridiculous timeouts like 20 seconds.

None the less, I am still interested in seeing the experimental code and gaining a better understanding of how this could be solved.

The code is in this branch:  http://bzr.nvaccess.org/nvda/lessFreezing2
You can also view the changelog.
Note that this branch is pretty outdated now; I haven't synced it with main in some time.

[jteh]

Btw, one of the big reasons I stopped working on this code is that UIA seems to call GetComboBoxInfo (or its low level equivalent) on top level windows when you register for events, which happens at startup. This seems to freeze if you call it on an unresponsive window; we proved this in a separate test case. This is why we freeze at startup if there is an unresponsive application on systems with UIA. Unfortunately, this seems to go straight to a kernel call, so we probably can't override this in any useful way. Alas, this means we're stuck with this freeze.

[jteh]

Merged main into lessFreezing2: changeset:lessFreezing2,3906.

[jteh]

A completely different experimental approach to solving this problem:  http://bzr.nvaccess.org/nvda/experimental/cancellableCalls/
At present, this is far from complete and broken in many areas.

[jteh]

(In #1528) Is this app free or paid? If paid, we're unfortunately not going to be able to debug this specific case.

[jteh]

See also duplicate #1543.

[parham]

The application mentioned in #1528 is free, however, the one in #1543 is not. There is still a way though: you could get Visual Web Developer (which is free), or Visual C# Express (which is also free).

[jteh]

We've abandoned the pure cancellableCalls approach for now and instead improved lessFreezing2, including borrowing some code from cancellableCalls.

As of now, snapshots are being generated for lessFreezing2. Anyone experiencing NVDA freezes in various situations should try lessFreezing2 snapshots and report. However, be aware that this is experimental code; the usual warnings apply.

[dwillemv]

Where might I download a snapshot of less freezing2? I could not find it on the snapshots page.

[jteh]

It's showing up fine for me. Try refreshing the page (and perhaps refreshing the iframe containing the table as well).

[jteh]

lessFreezing2 merged in changeset:main,4628. There will doubtless be more things to fix, but these should be filed as separate tickets.

[jteh]

I discovered that until now, these fixes weren't working on Windows XP systems without the UI Automation Client API (part of the Platform Update). See #1817. This has just been fixed.