NVDA Hack Fest November 2008

Repackaging NVDA

We need to make NVDA in to a proper Python package, to facilitate generation of better API documentation, and management of sub packages. Possible ideas:

Move all NVDA-specific Python code in to a directory called NVDA
Create a init.py file in the NVDA directory which contains any possible platform/situational independent code from nvda.pyw, such as playing startup sound, initializing logging, starting the core.
nvda.pyw should be kept outside the package as a Windows NVDA launcher which will pretty much just import the NVDA package
appModules, synthDrivers and brailleDisplayDrivers should be made in to real Python sub packages of the NVDA package, by containing a init.py file, and Py2exe should be instructed to compile these packages in to the executable, like NVDAObjects and virtualBuffers.

Some questions that must be answered:

Where should .kbd files be stored if appModules are compiled in to the executable?
Should we have a 'plugins' directory so that users can still provide 3rd party code to hook in to NVDA (such as appModules, synthDrivers, brailleDisplayDrivers)?

This was deferred, as although it improves our design and packaging, we don't think it should be a priority at present.

managing IAccessible, MSHTML, IAccessible2 and UIA interfaces better

Originally win events were added to Windows in order to provide an event mechonism for MSAA (IAccessible). However now it is possible to query service from an IAccessible to many different interfaces. These include IAccessible2, MSHTML and UIA. Currently NVDA contains a 'normalizeIAccessible' function in IAccessibleHandler.py which tries to convert a COM object to an IAccessible, and possibly an IAccessible2. How should we handle other interfaces such as MSHTML and UIA?

MSHTML and UIA (unlike IAccessible2) do not inherit from IAccessible so they can not be used as IAccessibles.
IAccessible2 frouns upon the usage where only the IAccessible2 interface is used to access both IAccessible2 and IAccessible properties. NVDA currently does this.
- This currently isn't causing any problems. Therefore, we will continue to do this until the specification changes to disallow it.
Is IAccessible2 still close enough to IAccessible to keep it supported by NVDAObjects.IAccessible?
- Yes.
Should MSHTML and UIA be handled completely separate (with their own handlers and NVDAObjects)?
- MSHTML objects will be composed of an IAccessible object, not inherited
- IAccessibleHandler will use MSHTML objects if appropriate
- UIA objects would be inherited from IAccessible for now
  - This allows for a partial implementation initially; we only implement the additional features provided by UIA
  - We suspect that many initial UIA implementations will not be complete and will still need MSAA
How can we improve movement between APIs (e.g. going parent on a UIA object and landing on an IAccessible)?
- There is no special way to handle this
- Specific NVDAObjects simply need to know which parent they should use
- Same as currently done for JAB

Improving Input with Input events

mouse and keyboard handlers are extremely win32 specific
No clear mechonism to handle braille display button input
No mechonism for speech recognition input
scripts can only be bound to keys (keyboard) not mouse jestures or braille display buttons or speech recognition verbs
User can only rebind scripts in appModules, not virtualBuffers (including quick keys) or NVDAObjects.
- Not high priority at this stage
- Probably something like the keyboard map file we discussed last hack fest
Do we need some kind of hyerarchy of input, such as input, keyboardInput, win32KeyboardInput, mouseInput, win32MouseInput, speechInput, win32SpeechInput, brailleInput
- Yes
- There would be a base InputGesture class
  - Every InputGesture would provide a unique string identifier in the form "type:value"; e.g. "keyboard:NVDA+f12", "braille:brltty:LNUP"
  - send() method could be used to send input gestures where possible; not implemented in many cases
- Each subclass would be responsible for converting internal representation to NVDA InputGesture identifier and implementing send() if appropriate
- Constructors can take internal representation
- Win32KeyboardInputGesture (subclasses KeyboardInputGesture), Win32MouseInputGesture (subclasses MouseInputGesture), InputGesture for each braille display (subclasses BrailleDisplayInputGesture)
Would need to be platform specific keyboard and mouse handlers which dispatch InputGestures to singleton InputManager
- InputManager would allow to query whether a particular InputGesture is useful and then to handle it if appropriate
- InputManager would handle input help mode as well
Can scripts be bound to Input objects rather than keys?
- Scripts are bound using the identifier of InputGestures to allow for quick hash lookups
- However, scripts will be passed the InputGesture instance
InputGestures will have translated label property

Braille

How to best avoid rendering braille regions for common context upon a focus change
See the ticket for the approach we discussed

Merging selected parts of secureDesktop into main

manifest for Vista
Saving of config in Application Data
- How will we handle speech dictionaries?
  - builtin dictionary resides in main NVDA distribution; not modifiable via UI
  - User's dictionaries will be in config dir
  - The default dictionary is in config dir
Merged in r2493

Project management

policy on inclusion of drivers or appModules

Will accept Python code for any driver which will not be explicitly included in a commercial NVDA distribution
- Core developers happy to update these drivers where possible when API changes occur
- Must have a maintainer who can be contacted if there is a problem lest the driver be removed
Will not accept full product dlls into the main distribution