NVDA 0.6p2 has just been released. The release is almost two weeks later than the original estimate proposed at the recent hack fest, as we decided to implement some additional noteworthy features, user interface changes, bug fixes and documentation updates.

Please note that this is a preview release, meaning that there are still some major issues to be fixed before the final 0.6 release. For more information about the current status of releases, see this article. Nevertheless, this release is recommended for most users. The old stable release, 0.5, is no longer recommended.

Download links and change log:

• NVDA 0.6p2 installer
• NVDA 0.6p2 portable version
• What's new in 0.6p2 [text file]

NVDA features prominently in this post (and accompanying video) on the Yahoo UI blog: Enhancing TabView Accessibility with WAI-ARIA Roles and States

This highlights one of the great benefits and potential uses of NVDA. The fact that NVDA is free and open source software allows developers to test the accessibility of their web sites and/or applications with a fully functional screen reader without having to purchase an expensive product they would not otherwise use.

Aside from the great plug for NVDA (thanks!), this is an excellent, well documented use of WAI-ARIA to improve web accessibility.

Last week, Mick and I met in person in Melbourne, Australia for the second NVDA hack fest. It was a rather intense, brain-melting five days. Despite the name "hack fest", we actually didn't do a great deal of serious coding/hacking. Much of the time was spent in gruelling discussion and debate. We did some long overdue planning of NVDA milestones. In particular, we decided to make another preview release, 0.6p2, to be released by 25 July. We also covered difficult topics such as support for tables, better implementation of formatting support, flexible mapping of widgets to NVDAObjects, centralisation of key bindings for NVDAObjects and virtual buffers, language detection and automatic voice change, live regions, and hierarchical documents (e.g. Lotus Symphony and OpenOffice). See NVDAHackFestJun2008 for full details of the topics and outcomes.

We are now using 7-Zip self-extracting archives instead of zip archives; i.e. for portable NVDA snapshots and the NVDA Miscellaneous Dependencies package. 7-Zip archives provide better compression than zip archives and thus facilitate smaller file sizes. Also, self-extracting archives are more convenient for some users. If you want to perform operations other than extracting the entire archive, you can open these archives using 7-Zip.

A new version of the NVDA Miscellaneous Dependencies package has been released. This version rearranges some files, which was necessary to fix #122.

This will be included in NVDA snapshots from r2161.

Users running from source must update to this new version if running r2160 or later.

A new version of the NVDA Miscellaneous Dependencies package has been released. This includes a new version of the virtual buffer library containing changes necessary for the implementation of the new quick navigation keys outlined in ticket #102. It also introduces a change to allow quick navigation and links list to detect some elements which were previously skipped.

This will be included in NVDA snapshots from r2149.

Users running from source must update to this new version if running r2149 or later.

A new version of the NVDA Miscellaneous Dependencies package has been released. This includes a new version of the virtual buffer library. It also includes mfc71.dll, which is required to compile NVDA into an executable and is missing on many systems.

Noteworthy changes in the new version of the virtual buffer library include:

• Don't render unlabelled, non-interactive (i.e. not linked or clickable) graphics. This removes a lot of unnecessary clutter from many pages.
• Fix a bug where text would eventually mysteriously disappear for virtual buffers for dynamic sites such as GMail and PennyTel?.
• Eliminate extraneous whitespace in several circumstances.
• Treat sections, separators, tables, iframes and unknown objects as block elements. This means that these elements will never be rendered on the same line as another element.
• Ignore unlabelled graphics inside links if the graphic is not the link's only content and is not clickable.
• Filter the URL for links with no content in the same way that URLs for graphic links are filtered. Implements #104.
• Improvements to the method used to avoid rendering invisible elements.
• Other minor optimisations and fixes.

This will be included in NVDA snapshots from r2138.

I am happy to announce that we have just launched the new NVDA web site on which we have been working for the past couple of weeks. This new site is based entirely on Trac and integrates all of our web services, including general information pages, downloads, wiki, issue tracker and blog. This allows for seemless referencing and interaction between services, greater community collaboration and ease of maintenance, among other benefits.

Note that www.nvda-project.org is the preferred address for this new site. trac.nvda-project.org redirects to this address.

Feedback is very much welcome.

A new version of the NVDA Miscellaneous Dependencies package has been released. This includes a new version of the virtual buffer library.

Highlights include:

• Fix some potential lag in NVDA on very dynamic sites such as MSDN.
• Fix crashes on certain sites such as http://www.pennytel.com/.
• Fix crashes in documents that only contain an empty node.
• For images which aren't contained within a link, use the URL of the image itself. This means that images which aren't links will now be rendered with some information. Addresses #51.
• Filter the URLs rendered for images to make them more readable. This should make for much nicer reading on pages with unlabelled graphics. Addresses #51.

This will be included in NVDA snapshots from r2079.

For users running from source, please update to at least r2079 before installing this package.

It has been quite some time since the last general post. A great deal has happened over the past couple of months, including the CSUN conference and preparation therefor, about which I posted separately. I have thus been dreading writing this, as I struggle to remember some of the minor, but nonetheless important, happenings of the last couple of months.

Perhaps the most exciting work on NVDA has been that relating to the new in-process virtual buffers for Mozilla Gecko 1.9, which includes Firefox 3 and Thunderbird 3. Current NVDA users would certainly be aware of this, even if they haven't tried it personally. NVDA 0.6p1, released just before CSUN, was the first release to feature these new buffers. Aside from massive improvements in reliability and accuracy, the new code allows for almost instantaneous rendering of pages in most cases thanks to its in-process workings. The new code also sports a far better design which makes many exciting features possible that were either impossible or extremely difficult to implement in the old code. For example, it took me only about half an hour to plan and implement the links list, a feature coveted by many users and which I use frequently myself. This was a great reinforcement of our design choices. The code in both the virtual buffer library and NVDA itself has continued to improve steadily since CSUN. While still under steady development, the goals in the web access grant from the Mozilla Foundation are almost complete. One major feature which is currently missing is the ability to efficiently navigate and read tables. Also, NVDA currently does not automatically report changes in live regions, although it does dynamically update the buffer. Nevertheless, Mick and I both use NVDA on the web full time and are very satisfied. Mick, who did the majority of the work on this project, has done an absolutely fantastic job.

Aside from assisting Mick with the virtual buffer work, I have worked on a lot of bug fixes, general improvements and code cleanup. One of the largest of the improvements is perhaps the refactoring of the NVDA GUI, which was included in 0.6p1. The NVDA window is now gone and everything is instead accessed from the NVDA system tray menu. A lot of issues with NVDA windows not gaining focus were fixed. In the last few days, I introduced more fixes in this area, including the elimination of the freezing in NVDA dialogs which occurred on some systems, particularly in Windows Vista. These are important changes in terms of user experience. I have also spent some time working on two tools useful in NVDA development. The log viewer simply allows the user to quickly view the NVDA log file right from the NVDA menu, rather than having to find the NVDA log file and open it with a text editor. It can be refreshed with a single key press and refreshes automatically when switching back to the window. In addition, it allows the user to save the current log content. It is thus useful to both users and developers alike. The other tool is the NVDA Python console, which allows developers to interact with the running internals of NVDA using a Python interpreter. Not only is this extremely useful in debugging NVDA, but it can also assist in inspecting the accessibility architecture of other applications.

I have been holding the fort somewhat more than usual in the last month, as Mick became the proud father of a baby girl in April. Congratulations, Mick! This has been a bit of a challenge for me, as I have had to adapt to working with less of his valuable feedback and collaboration. Even so, despite the exhaustion and chaos of early fatherhood, Mick has still managed to make some very significant contributions to NVDA and has been steadily increasing his working hours over the last couple of weeks.

Other than coding, we have done a lot of work in relation to the NVDA project resources and collaboration tools. I have been strongly encouraging users to use Trac to report issues and Mick and I now make extensive use of it ourselves. Thanks to all of the users who have started to use this resource. It is certainly improving the organisation of the project and makes it much easier for developers and other users to keep track of reported issues. Due to our previous hosting provider becoming ever more unreliable, we moved all of our internet services to a new server. I moved all of the important articles from our old wiki, taken down due to increasing spam and lack of maintenance, to the Trac wiki. Some work still needs to be done to the Trac front page to allow these articles to be found and to provide better direction for new users. I made some long needed updates to the NVDA web site. Most recently, we migrated the NVDA Wordpress blog to Trac, which allows for easier posting, maintenance and integration with the rest of Trac.

Outside of NVDA, both Mick and I have spent a great deal of time testing Mozilla Firefox 3. We have filed several significant bug reports relating to accessibility, some of which have resulted in noteable improvements to Mozilla accessibility, sometimes not just for NVDA, but for other assistive technologies as well. We have continued to attend meetings and contribute to the IAccessible2 effort. Due to the increasing number of external issues we are reporting and tracking, we have started the ExternalBugs wiki page to list and provide links to all of these reports.

Although Mick and I talk via phone on a daily basis, we decided in a recent discussion that we will meet in person for another NVDA hack fest some time in June. We have plans for some major improvements to the core of NVDA; specifically, the creation and handling of NVDAObjects. Topics such as proper support for tables and future plans will also be covered. We will probably make another 0.6 preview release some time over the next few months. I will then embark on implementing support for braille in NVDA.

As many of you can probably attest, services on nvda-project.org, including the web site and Trac, have been extremely unreliable over the past few months. There has been frequent downtime, sometimes lasting for hours at a time, and even when services were functional, they were often extremely slow. Thus, a few weeks ago, we decided that it was time to move to a new server. Rather than moving to another service hosting package, we opted for our own virtual server. While this increases the technical administration tedium that Mick and I must endure, it also allows for much greater flexibility.

Despite the rather shaky transition, all services for nvaccess.org and nvda-project.org have now been moved to the new server. There certainly does appear to be a definite improvement in speed and reliability and we're hoping it will continue thus.

As I begin writing this, I'm sitting on a plane enduring the 13 hour flight back to Sydney from Los Angeles. As many of you know, the week prior to CSUN was insanely busy, as Mick and I hurried to make the 0.6p1 release in time for the conference. (In fact, we ended up deferring the release until soon after we arrived in the U.S.) CSUN was similarly busy, which, alongside far too little sleep, has left us exhausted. Despite a consequent need for some serious R&R, CSUN 2008 was an absolutely fantastic experience, both for NVDA and for Mick and I personally.

We arrived in L.A. late on Monday morning and were settled into our hotel room by around lunch time. No one else we knew was to arrive before Tuesday afternoon, so we spent the first two days working on some final touches for 0.6p1, which was released some time on Tuesday afternoon. (Unfortunately, we also had to endure a dodgy internet connection during this work, which persisted in its dodginess throughout the week. Arrrg!) We also burnt around 30 CDs containing NVDA, Firefox 3beta4 and information about both for distribution to potential new users whom we encountered during the conference.

10 a.m. on Wednesday morning saw us at the IAccessible2 face to face meeting. This ended up exceeding its 1 hour time slot by almost another hour! Discussion was quite broad, covering IAccessible2 itself but also extending beyond into many other topics relating to open accessibility standards. Nevertheless, i think the meeting was very successful. We were told that IAccessible2 support for OpenOffice?.org would hopefully ship in a 3.x release some time within the next year, which is very exciting. There was a great deal of discussion in terms of the future of IAccessible2, especially relating to the establishment of guidelines beyond the specification for application implementations. Currently, there are widely varying ideas on how IAccessible2 should be implemented, with applications such as Lotus Symphony taking a rather flat approach as compared with Mozilla Gecko's extremely hierarchical approach. Mick and I believe that there need to be guidelines for the ways in which IAccessible2 should be implemented in various applications to prevent this getting out of hand. This idea was met with overall approval and a great deal of discussion ensued as to how this might be achieved. Other issues included problems with the use of IAccessible2 for portable applications and assistive technologies, feedback regarding Accessibility Probe, problems with the dependence of IAccessible 2 on MSAA and potential minor changes to the specification. Aside from the useful outcomes, it was great for me to meet the team behind this excellent accessibility API face to face.

On Wednesday evening, we had a dinner meeting with the NVDA Japanese localisation team. Our meeting spanned several hours, although it was interrupted several times by the need to adjourn to different locations. We discussed the status of the NVDA Japanese localisation and a demonstration was given with a commercial Japanese speech engine. We then covered a couple of localisation problems. The first concerned certain Japanese punctuation marks which aren't handled by the synthesiser used. While we handle expansion of English punctuation marks, there is no provision for doing this for other language specific symbols. Second, we discussed Windows input method editors (IME), which are necessary for entry of characters in pictographic languages such as Japanese. NVDA currently has no support for this. We would certainly like NVDA to support this, so Mick and I asked many general questions about how this works. We then spoke about the possibility of open source Japanese speech synthesisers. Mick and I suggested that efforts be made to improve the current eSpeak Japanese language so that an additional synthesiser would not be needed for Japanese NVDA users. Finally, the team explained the current state of screen readers in Japan. There are no good free alternatives and Japanese commercial screen readers are quite expensive, which is why they believe NVDA is important for Japan. It was a very pleasant meeting and it was great to receive some face to face feedback from a localisation team.

Thursday was the first day on which the exhibit hall was open. We spent much of our time for the remainder of the conference assisting at the Mozilla booth in the exhibit hall, preaching the goodness of Firefox 3 accessibility and, where appropriate, NVDA. We had very few opportunities to actually demonstrate NVDA. On the other hand, we spoke to many people from widely varying groups and levels of experience and gave away most of our CDs. It was humbling and gratifying to have quite a number of people, both users and otherwise, visit just to tell us that they appreciate the work we're doing. A highlight for us was a visit from a group of vision impaired primary school students accompanied by a teacher. She did not know about NVDA and was very pleased when we told her of the project. She told us that it would be great for some of her students whose families probably could not afford to purchase any of the commercial screen readers and took several of our CDS for her students, even coming back to take more for others. This is a fantastic validation of the mission of NV Access and NVDA: to lower the barrier to accessible computing. Furthermore, young students like these are not biased by prior use of another product, so in some ways, they are perhaps most likely to make the most of NVDA.

In terms of Mozilla, many people, existing users and otherwise, were impressed by the new accessibility features in Firefox 3. Being primarily concerned with accessibility for screen reader users, I must confess to having paid little attention to accessibility features for other users. There was a great deal of interest in the new full page zoom feature for low vision users. As well, the rich accessibility API support offered by Firefox 3 has potential benefits for users of voice dictation software. Again, many people visited just to say that they appreciated the work of Mozilla. There was also a great deal of interest in the merchandise up for grabs, which included an abundance of stickers, badges, temporary tattoos and brochures. Almost all of this had been taken by the end of the conference.

Thursday night was spent at a dinner with all of the people assisting at the Mozilla booth. (Well, actually, we first had to endure a rather frustrating, crazy cab ride, but I digress.) Most were either fellow grantees or people otherwise involved with Mozilla who were volunteering. Frank Hecker, the CEO of the Mozilla Foundation, was also present. It was great to spend an evening with such a fantastic, sincere group of people.

On Friday afternoon, we attended the IAccessible2 development panel, a session open to the public in which "Lead software applications and assistive technology developers [got] together to share their experiences in supporting IAccessible2". Present were representatives from IBM, Dolphin, GW Micro, Sun Microsystems, Freedom Scientific, Ai Squared, Adobe Systems and NV Access, with Mick speaking for NV Access. Each spoke a little about their current implementation of or plans to implement IAccessible2 support, some also speaking of their involvement in its design. Mick spoke of the IAccessible2 support in NVDA and why we believe it to be important for accessibility. He mentioned the current shortcomings in NVDA's support, including the lack of support for tables (which, by the way, is something we want to rectify very soon). Most of the assistive technology developers have also implemented at least partial support for IAccessible2, although some currently use it only for certain applications. We learnt that Adobe are starting to implement support, which is great news for us. In the discussion of problems and future plans, Mick raised the issue of the current inability to run IAccessible2 applications and assistive technologies portably due to the need to register a proxy library. A potential solution was proposed, which we have since investigated and found to be quite promising.

As is often the case at such conferences, even socialising after hours provided a fantastic opportunity for networking. One such occasion was a discussion with some of the core development team for Orca, the open source screen reader for the Gnome GUI under Linux and Solaris. One outcome of this discussion was that both teams agreed to try to collaborate wherever possible. We have always been open to such collaboration, but this was a great opportunity to become properly acquainted. On another occasion, we were introduced to the person at Microsoft responsible for accessibility of Internet Explorer 8.

One of the few conference sessions we attended was on Saturday afternoon. This was entitled "Low-Cost and Free Screen Access Solutions Versus Full-Feature Software" and was presented by the National Federation of the Blind. We had no idea whether NVDA would be covered or how it would fare if it was. Initially, we felt that perhaps the description of the free and low-cost solutions as not being "full-featured software" was a bit unfair, but I guess it is true that we still have a lot of work to do to match some of the more expensive products. Conversely, we were pleasantly surprised by the fair demonstration of NVDA. We waited with bated breath to find out which version of NVDA would be demonstrated and were relieved to discover that 0.6p1, the release we had prepared only days earlier, was featured. (NVDA has certainly come a long way since 0.5.) The other featured screen readers were Thunder and Serotek System Access. Overall, it was a worthwhile presentation and I hope it inspired some people to at least give solutions such as NVDA a try.

Over the duration of the conference, we met and conversed with several key accessibility people from large organisations such as IBM, Sun, Adobe and Microsoft. In addition, we spoke to several people who were interested in collaborating with us in some way or potentially distributing NVDA with their products. It was gratifying once again to realise the respect we have attained in the field of accessibility. We were astounded to learn of the amount of interest NVDA has attracted. Mick and I also took the opportunity on occasion to investigate some of the other exhibits.

Overall, CSUN 2008 was a very enjoyable, benefitial and worthwhile experience, both for NVDA and for us personally. I very much hope that both Mick and I are able to attend the conference again next year.

We have just released NVDA 0.6p1. This is a preview release, meaning that there are still major issues to be fixed before the final 0.6 release. However, this provides a preview of some of the new functionality that can be expected in 0.6. This release will be featured at CSUN 2008. For more information about the current status of releases, see this article.

Download links:

• NVDA 0.6p1 installer
• NVDA 0.6p1 portable [zip file]
• NVDA 0.6p1 changes [text file]

Mick and I will be attending the Technology & Persons with Disabilities Conference (better known as CSUN) commencing in approximately two weeks. Once again, this is thanks to the generosity of the Mozilla Foundation. This should be an exciting event for NVDA, as NVDA has progressed a great deal since last year's conference.

Perhaps the most exciting recent development is the new in-process virtual buffer back-end for Mozilla Gecko 1.9, which is used by Firefox 3 and Thunderbird 3. In practical terms, this means that rendering pages into a virtual buffer in these programs is now practically instantaneous and is far more accurate than the previous code. Although it is not quite yet as stable as we would like, it is almost ready for public testing. We're very keen to show this off at the conference! :) We are also working to address several other items in NVDA prior to CSUN.

Please see CSUN 2008 Plans for more information, including our "to do" list and a rough schedule. If any of you are interested in joining or collaborating with the NVDA project in some way and want to meet at CSUN, please drop Mick and I an email.

I began working full time for NV Access on NVDA three weeks ago (on Monday, 4 February, to be precise). This was interrupted last week, as I had to work for another week with my previous employer, so I've now been working full time on NVDA for two weeks.

It has certainly been a busy but interesting two weeks. I am obviously quite familiar with the project, given that I have been a core developer almost since the beginning. However, working on a project full time is unsurprisingly somewhat different to casual contribution in one's spare time. Nevertheless, I am very much enjoying the work and am looking forward to realising exciting potential for NVDA as the year progresses.

Mick and I are currently focusing on preparing both ourselves and NVDA for the upcoming Technology & Persons with Disabilities Conference, better known as CSUN. I will post more on this later. I have spent most of my time so far discussing all things NVDA with Mick, fixing small (but nevertheless annoying and problematic) bugs, and helping Mick to test the new virtual buffer code.

I would like to once again extend my personal thanks to the Mozilla Foundation for making this possible. I am extremely grateful for and excited about the opportunity to devote all of my working hours to further develop NVDA.

Thanks to the generocity and support of the Mozilla Foundation NV Access has been able to hire James Teh as a full-time developer to work on NVDA. The Mozilla Foundation has taken a keen interest in NVDA as one of NVDA's goals is to provide excellent support for Mozilla products, such as Firefox and Thunderbird.

The grant (which provides NV Access with US$80,000 over 2008) allows NV Access to employ James Teh (Jamie) full-time to work on improving and maintaining NVDA, with a major focus on Mozilla products. The grant will be also used to cover overheads for the running of NV Access, which a part from general administration, also includes project promotion and the seeking of further funding.

NV Access and Mozilla worked together to draw up a list of grant goals for NVDA, which both organizations see as the most important things that should be achieved to make the project a success. Although the grant will be reviewed before the end of this year, all the goals listed are to be completed with in a three year timeline.

Jamie will hopefully be starting work in the next month or so, once all the admin has been organized. I for one am very excited to have Jamie join the project on a much more full-time basis, and I know he is also very excited to be able to put all his working time to open-source projects that hopefully can improve the lives of people in the community in some way.

On behalf of NV Access, and the other developers of NVDA I would like to thank the Mozilla Foundation for its support over the last year. Together we can make sure that blind users will always have both a free choice when it comes to access to applications on the Microsoft Windows Operating System, and also a choice to move forward with the rest of the community, to use free and open-source products (such as Firefox and Thunderbird).

Since my last blog post on the web access grant a lot has happened in regards to this stopic. A few features talked about in previous postings for the storage code have been implemented, but most importantly, I have taken that next step of actually completely integrating the storage code in to NVDA and now giving myself and other NVDA developers the choice of using the new code to interact with Gecko 1.9 documents (such as Ff3 and ThunderBird?3).

The last post I talked about the fact that Jamie and I had talked about allowing arbitrary properties on nodes in the buffer, rather than just locking it down to just role, value, states, keyboardShortcut and contains. Well, this has been achieved, so now functions such as addTagNodeToBuffer and findBufferFieldIDByProperties take an array of attributes (name and value pares), or sometimes name and multiValue pares, if using findBufferFieldIDByProperties to search on multiple values of a given property. There are still particular properties of a node which have their own specific member variables (such as ID, and some other new ones that are generic to nodes or tagNodes)

The coding that has probably taken up most of my time through out the last month or so is the code that manages lines in the storage module. A virtualBuffer's job is to render a representation of a document in a flat layout, meaning that every character in the buffer has an index, from 0 to the length of the buffer - every character has an ordered place. But, it also has to have an idea of what lines are, as in it must allow the user (through the AT) to be able to arrow up and down through the buffer by lines of information that are not too long, but also not too short as to make the user have to press keys more than they need to. Working out exactly how to implement this was hard, what it uses now is my third go at implementing it.

NVDA itself has pretty good text management through its TextInfo? classes, so all the storage module had to do to communicate line placement was to allow the querying of line offsets using a particular offset as reference, with the getBufferLineOffsets function.

My first attempt was to simply scan back from the offset, looking for a line feed character, and then do the same forward. This works ok if the only information stored in the buffer is itself basic text broken up by line feeds. However, if for some reason the AT wanted to some how tweek where line breaks occured (perhaps for ease of reading), it would have to insert its own line feeds in along with the original information.

This way of doing things was ok for testing. In fact, with some initial tests in NVDA, I had NVDA place a line feed at the end of each node it inserted, plus I also had it scan each block of text it added and got it to insert line feeds with in the text, to break it up in to reasonable sized chunks.

Mutating text in this way is not only bad because when the text is navigated by the user, they will see a line feed character at the end of each line, even if the line was only broken due to line length rules, not because a paragraph actually ended. The other major problem is that because Mozilla Gecko provides text with imbedded objects, who's events depend on the text offsets staying the same as what they internally have, things could get out of sync pretty quickly.

I then designed a way so that the AT or backend, when adding the text to the storage buffer, could provide a list of offsets where lines should be broken. These would be soft line breaks that did not actually appear in the text, but the buffer would know about them and when asked for line offsets, could take those in to account.

I was happy with this approach for quite a while, as it meant 1. that we were not mutating the text at all and Gecko events would be happy, and 2. Users could arrow to the end of a line in the middle of a paragraph and not see line feeds that shouldn't be there.

There were two major problems with this approach. The first was that the AT or backend needed to know the user's chosen maximum line length at render time, and although individual text blocks would not contain lines longer than the chosen length, there was nothing stopping two text blocks (say part of a paragraph and then some links) from all together added up being much longer than the chosen length. Of course this wouldn't be a problem if a line break was inforced at the end of all nodes (such as in many popular windows screen readers), but if NVDA was to support a screen layout, then this problem could be quite evident.

Eventually I decided on the third approach. This way was to allow getBufferLineOffsets to receive a maximum line length int , and also an int that indicated whether a screen layout was to be used, and then it would calculate the offsets itself by a set of steps. To accomidate the new way, tag nodes in the buffer also needed to take a new member variable, addTagNodeToBuffer also needed to be able to receive this. This was an int that indicated if this tag node was a block element or not, as in, should the buffer assume that this node has to both inforce the start and end of lines at its edges.

So, the steps that getBufferLineOffsets takes are: *Set some initial line offsets to the start and end of the buffer *Locate the deepest node at the offset given *Move up the node's ancestors until it locates a tagNode that is indicated as being a block element. If one is found, then the line offsets are set to this node's start and end offsets. Also record the start and end of any tagNodes passed in a possibleLineBreaks set. *Then from the given offset, do a traversal search both backwards and forwards in the tree locating the closest block elements. If one is closer than the ancestor block element, then set the line offsets to this node's offset. Again also while traversing, save the start and end offsets of any tagNodes in the possibleLineBreaks set. *Then scan the text between the now found line offsets, looking for both line feeds and beginnings of words. If a lineBreak is found before the given offset, and its the closest one to the offset, than it now becomes the line's startOffset. Same for a line feed on or after the given offset, if its the cloest it becomes the end offset. The beginning of word offsets are saved in the possibleLineBreaks set. *Finally, The line start to line end is counted up to make sure it doesn't exceed the maximum line length the user requested. If it does, then the line start is brought forward to an offset either at the max line length, or before (using the possibleLineBreaks set as indication of where its healthy to break), and then the line length is counted up from there again. Of course this does not ever pass the original given offset, and the line end of course will not end up being before, or too far after, the given offset.

Note that if the user chooses not to use a screen layout, then rather than searching for block elements, it just uses any tagNode, meaning lines will seem to always break at the end of links and other fields etc.

A rather complex set of actions, however in c++ they really do not take too much time at all. I didn't really like this approach at first as it has a danger of being non-cemetrical, in that there could be a chance that asking for two different offsets that should be on the same line, it may give back two different lines, due to the fact that a maximum line length has to be checked. Though, I foun that as long as I always calculated all soft line breaks, even before the given offset, between clear block line breaks, this would never be a problem.

Around the same time I was improving upon the line offset code, I started re-writing NVDA to use the new virtualBuffer code. At this point in time, the new virtualBuffers for Gecko 1.9 applications have improved quite a lot in comparison to the old virtualBuffers NVDA was using before the grant. Although the backend rendering code is still in Python, the technique of using imbedded objects in text with IAccessible2 and so forth proved to be a rendering time improvement of over 50%. This means when NVDA loads a document in Firefox3, it now takes just under half the time it used to.

As the low-level management of nodes and text is all maintained in c++, this has made sure that its much more accurate, and we no longer have large chunks of documents mysteriously not being rendered, or complaints from the virtualBuffers that some ID doesn't exist and other fun things we used to have.

We have been waiting for a long time to be able to convert NVDA's virtualBuffer interface code to using the TextInfo? classes I spent a lot of time on last year. As we needed to make NVDA work with the c++ virtualBuffer storage module, and because we needed to improve NVDA's rendering patterns for imbedded objects and such, I made sure the new virtualBuffers were designed around the TextInfo? classes. this now means users of NVDA now have the ability to select text with in virtualBuffers, and also copy that text to the clipboard if they wish. They can also choose to read the buffers as a screen layout, or as a more conventional node per line layout. Its taken a little while, but I've also now added the quick key navigation (as in press h to jump to a heading, l for a list etc) in to the new virtualBuffers; its great to see that the findBufferFieldIDByProperties function actually works like I'd hoped.

At the moment we're still in discussion on the development list about how particular fields such as links etc should be spoken: should the word link be spoken before or after the text etc. Though I think we've come to a pretty good agreement on most of the fields.

The new virtualBuffers (at least in Gecko 1.9 applications) can be interacted with in regards to activating links, toggling on and off a pas-through mode to interact with edit fields and combo boxes etc, though the one thing that makes the new virtualBuffers incomplete still is that they have no support for events etc, as in if content changes dynamically in the document, the buffers do not pick up this change. This code however will be added when I re-write the rendering code in c++, as it needs to be very fast, and for best accuracy, it should really be in-process so that things don't start disappearing before NVDA's process gets around to actually asking Gecko for it. However in the mean time I've added a key stroke to tell NVDA to manually re-render the current document, so for most websites, they are able to be tested well enough.

Over the next little while its probably going to be more work on NVDA and virtualBuffers in general, to make sure that the user experience is the best it can be. Once this is ok, then my next task wil be to re-write the rendering code for Gecko virtualBuffers in c++. this should give load times an estimated speed up of about a multiple of three. Then after that the fun work will begin on trying to integrate all of the virtualBuffer c++ code so that the rendering code is injected in to the Gecko application, and rendering takes place in-process. Which by estimates should speed up load times by a multiple of twelve or so.

Since my last post on the web access grant, the virtual buffer library code has undergone quite a few changes, both for code readability, and for makeing sure it will really work the way it should, in all situations.

The first change is that the wm module (that manages all the window messages) has now been removed. So rather than having individual window messages for each API call that must cross a process boundary, we only now have one window message which takes a pointer to the internal function, and a pointer to a struct of arguments, as its wParam and lParam arguments. This means that only one window message needs to be registered, plus it means one less code change when adding new functions to the API.

The next change is that rather than client and internal functions using the storage buffer directly, they use a buffer container instead. This buffer container is a pointer to a struct that contains a handle to the window being virtualized, a handle to the current backend dll being used for this buffer, a pointer to the storage buffer, and a pointer to a win32 Critical Section, which is used to serialize access to the storage buffer. These changes make it much more possible to have multiple buffers for the same window, and it makes sure that the storage buffer can not be read from while its being written to etc. The latter of course depends on the fact that any backends will also properly use that critical section when accessing the storage buffer.

Previously, nodes were only used to represent tags, as in actual nodes with properties, not just text. The only way that text manifested itself was if a node was wider than 0 offsets and it had no children, or as a gap of more than 0 offsets between two sibling nodes. This was very dificult to manage when inserting and removing text, so now text has its own node type.

As Firefox3 accessibility has a *very* different way of handling text and child nodes (through its embedded object approach) compared to Firefox2, or other web browsers, a lot of long phone calls between Jamie and I were had, and a lot of code restructurig was done, to make sure that we can handle the two very different approaches as efficiently as possible.

The main problem was that the virtual buffer library was very ID-centric, meaning that all our API functions took node IDs as arguments. However, in Firefox3, text itself does not have a unique ID, so this approach just doesn't work at all. So the calls for adding and removing nodes have now been changed to take actual nodes as arguments, not just IDs. Also, the call to add a node also actually returns the node as its added, which allows the backend to gain a reference to the created node for later use. To avoid code duplication, much of the adding/removal code has ben broken down in to much smaller reusable functions, which make the code easier to read, and probably even more efficient.

Two extra functions have been added which handle the merging and splitting of text nodes. When removing a node which is flanked by two text nodes, its probably best to actually merge the two text nodes together so as not to cause fragmenting over a long period of time. Otherwise we could end up with a whole bunch of one-character wide text nodes all over the place, if there were a lot of arbitrary removals. The reason for the splitting function, is that if firefox instructs us that we need to add a node in to a parent at offset n, offset n may actually be right in the middle of a text node, so before adding the node, we need to split the text node in two at this position, and then add the new node directly after the first text node. Note that a function has not yet been written to actually take a parent and an offset from firefox and calculate where exactly in its children the new node must be added, this will get written along with the Gecko backend as its quite specific to Gecko.

Many other little fixes and tweeks have been made to the code, making sure that it handles different situations properly.

All through the writing of this code, there has been a testing program that tests different actions to perform on the storage buffer, to make sure we don't break anything.

Lately, I have branched NVDA trunk to a virtual buffer testing branch, which has allowed me to pull apart NVDA's old virtual buffers, and start writing a test one using the storage module from the virtual buffer library. Its very basic, only printing about three or so lines to a virtual buffer, with a few links and headings etc, but this is really just to enable me to start writing the necessary code in NVDA that will be used to navigate the new virtual buffer etc. I must say its quite nice to finally be able to navigate around the new virtual buffer, it does prove that this code is actually going somewhere.

One thing that Jamie and I were talking about on the phone last night was the use of hard-coded properties in the virtual buffer library, such as role, value, states, contains and shortcut. It has always been thought up until this point that backends will convert their own role and states values to virtual buffer library - sspecific ones, then NVDA only has to deal with one set. However, this seems to create a lot of work for the backends, plus rather large mappings need to be written for all the different accessibility APIs. I think we have agreed that the backends will now just use API specific values for roles and states, and NVDA itself will do any conversions after fetching them from the virtual buffer.

We are also worried about exactly what properties a node should have. Obviously a node needs an ID, as that is what makes it unique, but as far as role, value, states, contains and shortcut is concerned, these are really quite arbitrary to a virtual buffer, and specific to the API used. Our thought is that perhaps rather than having hard-coded properties, we will allow arbitrary properties instead, meaning that when a node is created, the backend who created it can specify a string of name=value pares, which denote the properties. The property names should also probably have namespaces for the different accessibility APIs, though there may be some properties which are not specific to any API.

All this needs to be thought out a little more, but what we are starting to realize is, is that the virtual buffer library should only act as a pipe line for information, at the same time tweeking the sintax and structure (i.e. converting a hyerarchical model in to a flat model), but not in any way changing the actual content. For example, the virtual buffer library should not have any idea about what a 'role' is, it should only know that nodes have properties. Its up to the actual accessibility API to inforce the semantics.

The advantage of this is that we don't need to keep changing the virtual buffer library interface when we want to add some other property, perhaps its to do with live regions, or its something to do with tables. Instead, the backend just needs to make sure it adds that particular property to the node, and of course NVDA needs to know to use that property on the other end, when its reading from the buffer.

Although work is slow, I think we are certainly making progress, and at the very least I am certainly learning a lot. This is new grround, for us at least, and I think we'll get there.

Over the last week or so, I have been busily setting up the new testing/development server that was kindly donated to NV Access. This server is going to be used as a server for the organization: hosting a virtual private network, allowing for the collaboration of business-related work and access to NVAccess's printer/copier/scanner/fax (bought through an Australian government grant). But more importantly it will be used as a testing/development server. Once its finally set up, this server will be able to automatically build daily snapshots of NVDA (if the source code changes), and also hopefully run some automated tests on the snapshots, to make sure that changes made don't break any previous changes.

Hardware-wise, the server has a Pentium 4, 3.20 GHz processor, and 1 gig of ram. Previously it did have 2 gigs, but the second chip seemed to be not very healthy, and after taxing the memory quite a bit the other day, we were getting all sorts of fun errors, so for now that chip has been removed. It has two hard drives, one 40 gig for the main Operating System and applications, and an 80 gig for files and Virtual machines for building/testing. It also has no shortage of USB sockets, in fact I already took out one card that had four on it. It has a floppy drive, sound card, and two network cards (one for access to the internet and my home network, and one purely for NV Access, to access the printer, and any other NV Access specific devices).

I have installed Ubuntu Linux 7.10 as the server's Operating System. We chose this OS because a: both Jamie and I are used to using Debian Linux (which Ubuntu is based on) and B: Ubuntu's accessibility seems to be growing all the time. And as NV Access, this is something we'd like to keep an eye on.

For testing and building NVDA, we are going to run MS Windows inside VMWare. For those who don't know, VMWare is software that emulates an entire computer system, so you can run one operating system, inside another. There is a free version of VMWare for Linux, which suits our needs, and I have successfully installed it and its running quite nicely on the server now.

Although the server is pretty much all set up, we're still a little way off from complete automated building/testing of NVDA. One thing that needs to be completed is the re-writing of the build scripts Jamie currently uses to build NVDA snapshots on his laptop. The plan is that we'd no longer like to keep compiled copies of eSpeak, charHook, keyHook and the virtual buffer library in subversion, but instead build them along with the snapshots, and have them all included in the snapshots, and also as a separate download so that people can still run from source. However the most important part holding us back is getting access to the MS Windows Operating System, so we can install it in the virtual machines on the server. I have tested virtual machines with some other free operating systems such as Ubuntu and Gentoo Linux so I know they work, but over the next little while NV Access would like to investigate how to acquire licensed copies of the needed Windows versions. We plan to build the snapshots in Windows XP, though we would much like to be able to test NVDA with Windows 2000, Windows Vista, possibly Windows 98/ME. Of course testing MS Office would be also very useful.

Many thanks go to the donator of the server, already we are seeing just how useful it is, plus we believe it really will change the way NVDA development happens in the future.

One other advantage of the server running Ubuntu is that I'm able to run Orca (a Gnome X-Windows screen reader for Linux / Solaris). Both the Orca and NVDA projects do have many things in common (as they are both free and open-source screen readers), and even though they are written for two entirely different operating systems, there should be much the projects can learn from each other, both in coding and user experience. I recently tested Orca with Firefox 3, and it is very clear that orca's web support is coming in leaps and bounds.

Other than server stuff, I of course have been working on the new virtual buffer library for NVDA. Work is slow but I'm definitely getting there. Design decisions need to be made very carefully as we need to make sure the code is as efficient as possible, but also make sure it will be compatible with lots of different web content.

Last weekend, I flew to Melbourne, Australia (I live in Brisbane) for two exciting events related to NVDA.

On Friday, Mick, Amy, Matt and I attended the Blind Citizens Australia Technology Expo held prior to the BCA national convention, where we staffed a booth for NV Access. Mick and I were providing one on one demonstrations of NVDA under both Windows XP and Vista. We were a bit concerned about the lack of interest for the first hour and a half or so, but after that, things started to heat up. People from widely varying backgrounds and degrees of computing experience visited the booth. We made between six and eight demonstrations (I can't remember the exact number), some of which were quite detailed. Between 15 and 20 people provided us with contact details, requesting further information. We also spoke to several people who were interested in passing the word about NVDA to larger numbers of people through various channels. Everyone was quite impressed by the project and most were somewhat astonished by its $0 price tag. :) Overall, it was a valuable, worthwhile experience. The interest we attracted was somewhat heartening. I think Mick and I learnt a lot about presenting the project to prospective users and other interested parties. We had also underestimated the potential for networking with people who can help further spread the word.

All of Saturday and Sunday morning was devoted to what we are calling an "NVDA Summit" or hack fest. Mick and I sat in his lounge room discussing, coding, living and breathing NVDA... as well as eating Amy's yummy, home-made cookies, among other delicious morsels. (Thanks Amy!) We covered quite a lot of ground, answering all but one of the items on our agenda. The agenda and outcomes are outlined here. Aside from this agenda and discussion of the direction of NVDA in general, the time allowed us to revitalise our working relationship and zeal for the project. It also enabled me to familiarise myself with complex parts of the code which I haven't had a chance to examine in detail due to being somewhat busy with other things of late.

The benefits of this "summit" are already being realised. This week, we have made major leaps forward with regard to the way we read dialogs. Rather than reading the entire dialog or only the dialog title and the current control, we attempt to read the descriptive text or query and then the focused control. This makes for a much nicer user experience when using most dialogs. Also, we have found and squashed a major memory leak which has been plaguing us for a while now. We have improved the speed and accuracy of cursor keys and backspace and delete, although backspace and delete still need some work.

These two events were made possible by many generous donations to NV Access. Many thanks to everyone who has donated.