Documentation should be converted into wiki format for ease of maintenance.

[orcauser]

I have investigated this, and couldnt find any wiki format that allowed us to specify element attributes, so I have extended the creole wiki markup to include element attributes.
We need them so that we can use same page links.

patch to be attached.

Two commits here, the first one adds in the wikiTools directory, and the conversion scripts, and a sconstruct file for automatically creating the user guides from wiki markup.
The sconstruct file should not be used until all languages have been converted into wiki format. Also please check it, I havent ran it under windows, the point of question is if the Glob() function returns the correct list of files.

The second commit includes user guide.txt which is the wiki markupped english, and
user guide-fromwiki.html, which shows the result of running the wiki markup through the conversion process.

If we do diff -u -b -B -w user\ guide.html user\ guide-fromwiki.html
We can see that all the diffrences are due to spacing, and to &quot.
The user guide-fromwiki.html should replace user guide.html once the work is accepted.

If you are happy with this, I can run through the other languages and convert them in a simular way, assuming no one is currently working on them.

Best

[mdcurran]

Thanks a lot for your work on this.
I am going to review it in the next little while and will comment further then.

Thanks again.

[clev]

In case it isn't too late to suggest another format, please give a look in txt2tags:
 http://txt2tags.sourceforge.net/

The advantage is mainly that you can continue writing in txt, just placing some marks here and there, then convert it to a lot of formats quickly.

[mdcurran]

A requirement:

• The converter that we use must support utf8 or some kind of unicode. As we have many languages, and codepages for specific languages are fiddly and old.

I'm not sure if wikiTools or text2tags supports this?

A nicety:

• Table of contents can be generated automatically.

I think that this point is pretty important and will be the reason why we would move to language other than HTML.

It makes the documentation a lot easier to maintain, and there is less room for errors. It also lessens the work for translators.

I see that the patch does not have an auto-generated table of contents. is wikiTools capable of this?
Is text2tags capable of this?

[clev]

Mick, txt2tags supports both UTF8 and automatic table of contents generating. I don't right now recall the sintax you must put in the source text file, but it's easy to locate some tutorials in the project web site. Well I can try to make some sample later.

[clev]

In the following page you can convert online a text file to HTML by just inserting some marks. A sample text source is provided at the multiline edit field. There are options for customizing the target including a table of contents. It's not as powerful as the desktop version of the program, but I think some documents like even the NVDA user guide may be converted online.
 http://txt2tags.sourceforge.net/online.php

[orcauser]

Replying to clev:

In case it isn't too late to suggest another format, please give a look in txt2tags:
 http://txt2tags.sourceforge.net/

Thanks, Wish I had come across it before I worked my hack, but looking at it now it
seems that txt2tags is the better choice for us.

[clev]

I'm tonna paste here a txt2tags little source for you to test. I'm also attaching it as a file, in case the formating is lost here. You may install the program and then either use the GUI or run the command:
txt2tags demo.t2t
where demo.t2t is the source file I'm attaching. You may also go to
 http://txt2tags.sourceforge.net/online.php
paste the text source and hit convert.

See the comments in the source text below for details on the sintax used.

NVDA 2009.1 User Guide
Last updated %%date(%c)

%!target: html
%!encoding: UTF-8
%!options: --toc

%Lines beginning with a percent sign are ignored when inside the document body.
%Plus signs denote numbered headings.
+ Introduction +
++ General Features ++

Providing feedback by synthetic speech, Nonvisual Desktop Access allows blind and vision impaired people to access and interact with the Windows operating system and many third party applications.

Major highlights include:

%Hyphens denote lists. Two blank lines close a list.

• Ability to run entirely from a USB stick or other portable media without the need for installation
• Easy to use talking installer

%Notice the brackets to make a line.

• Browsing the web with [Mozilla Firefox 3  http://www.getfirefox.com/]
• Working with email using [Mozilla Thunderbird 3  http://www.mozillamessaging.com/en-US/thunderbird/early_releases/]
• Support for Microsoft Internet Explorer
• Basic support for Microsoft Outlook Express / Windows Mail
• support for Microsoft Word and Excel
• Support for accessible Java applications
• Support for Adobe Reader
• Early support for IBM Lotus Symphony
• support for Windows Command Prompt and console applications
• Automatic announcement of text under the mouse and optional audible indication of the mouse position
• Support for many refreshable Braille displays

++ System Requirements ++

• Operating Systems: all 32-bit and 64-bit versions of Windows XP, Windows Vista and Windows 7 (including Server operating Systems), may partially work on Windows 2000.
• Memory: 256 mb or more of RAM
• Processor speed: 1.0 ghz or above
• About 50 MB of storage space.

[clev]

See my last comment.

[orcauser]

Clev, Do you know if there is any existing tools for converting back, i.e. from the existing html manual to txt2tags format, or should i rewrite the html2wiki that was attached earlier.

Probably nothing will give us the complete t2t document, but at least it will do the main leg work for us, then we have to edit and varify the final t2t file before inclusion.

[clev]

Replying to orcauser:

Clev, Do you know if there is any existing tools for converting back, i.e. from the existing html manual to txt2tags format, or should i rewrite the html2wiki that was attached earlier.

I don't know; have just asked the TXT2Tags author.

[aleksey_s]

Guys, i think that adding yet-another-utility to generate documentation is unnecessary. We have already a wiki system on the site, so i think that best option is converting current manual in the trac wiki format and uploading it as a wiki. Then we will be able to export html and include it in NVDA releases. This way we enable people to contribute to NVDA documentation and simplify the maintenance.

[orcauser]

Replying to aleksey_s:

Guys, i think that adding yet-another-utility to generate documentation is unnecessary. We have already a wiki system on the site, so i think that best option is converting current manual in the trac wiki format and uploading it as a wiki. Then we will be able to export html and include it in NVDA releases. This way we enable people to contribute to NVDA documentation and simplify the maintenance.

Are you saying that if I give you a trac formatted wiki text file, you have a method to give me back a standalone html file?

The idea of using txt2tags was that if it was included, the user could modify the manual, and if they so wished they could create the html file to check/proofread.

[aleksey_s]

Replying to orcauser:

Are you saying that if I give you a trac formatted wiki text file, you have a method to give me back a standalone html file?

Sure.  http://lists.edgewall.com/archive/trac/2005-April/002729.html
We can write something similar. As you see, script is pretty simple.

The idea of using txt2tags was that if it was included, the user could modify the manual, and if they so wished they could create the html file to check/proofread.

I do not think that we need to include wiki2html converter into NVDA distribution. into sources - yes, but not in the binary package.

[orcauser]

Replying to aleksey_s:

Sure.  http://lists.edgewall.com/archive/trac/2005-April/002729.html
We can write something similar. As you see, script is pretty simple.

Yes, thats essentially what the original patch did, i.e. write a parser for wiki format. wiki2html

One thing to keep in mind is that if we write something, then we have to maintain it, txt2tags seems to have been around for 7 years, and is well developed.

I do not think that we need to include wiki2html converter into NVDA distribution. into sources - yes, but not in the binary package.

Yes, sorry I didnt mean to include it into binary, but into sources would be useful.

I guess now we need input from Mike or Jamie to take the choice of which we should go with.

Thanks

[jteh]

Replying to aleksey_s:

Guys, i think that adding yet-another-utility to generate documentation is unnecessary. We have already a wiki system on the site, so i think that best option is converting current manual in the trac wiki format

I certainly have no problems using the Trac wiki format. In fact, this is what I have done for the what's new document. This way, NVDA contributors only need to learn one format. Unfortunately, requiring the Trac environment to render to HTML isn't realistic for most users.

and uploading it as a wiki. Then we will be able to export html

I don't think the documentation should be maintained in the actual wiki. For a start, this causes problems with versioning; i.e. we need to be able to access documentation corresponding to a particular release or revision of NVDA. Therefore, I think the source for the documentation (in whatever format that may be) should be maintained in bzr.

[orcauser]

So what is the consensus?

[pvagner]

I have just came accross a python library called lcg.
Perhaps it would still do the task.
See its page at:  http://www.freebsoft.org/lcg
And a complete manual at:  http://cvs.brailcom.org/doc/lcg/

[jteh]

I've just had a look at txt2tags and am thinking it could work quite well for us.

Mick thinks LaTeX works well. LaTeX is probably the most powerful, but potentially the most complex.

Looking briefly, lcg also seems to be okay and uses wiki style markup. Peter, anything about lcg that is better than txt2tags in your opinion?

[jteh]

Having looked more at txt2tags, it still looks very nice, but there are a few problems:

• In HTML, h1 really should be the title of the document, as although there is a title element, it is not seen on the page itself. Therefore, h2+ should be used for headings. With txt2tags, if you start using numbered headings at heading level 2, your first number will be 0.1. I can easily hack txt2tags to output h2 for first level headings and so on, but it'd be nice not to have to hack our document generation tool.
• It does not support specification of alt attributes for images. All images get alt="".
• It does not support auto-numbered section references. For example, you cannot do "See section [#NVDAPortable]" and have it output "See section 4.2". We don't currently need this and I suspect we'll have to go for a much more complex tool like docbook to manage it, but it's worth noting anyway.

[clev]

Replying to jteh:

• It does not support specification of alt attributes for images. All images get alt="".

There is a FAQ for TXT2TAGS written in Portuguese that explains how to do that. You should use a %!postproc rule. The TXT2TAGS user guide explain how it works, see parts 7 and 8, starting from
 http://txt2tags.sourceforge.net/userguide/PartVIIMasteringSettings.html#8

The sintax for inserting an alt attribute is like this:
%!postproc(html): '("beach.jpg.*?alt=")' '\1 a really nice beach'
%!postproc(html): '("city.jpg.*?alt=")' '\1 this is my city'

I think it's possible to solve other problems you mentioned by using such features as well. Just for info, the Portuguese FAQ is at:
 http://www.aboutwilson.net/txt2tags/faq.html

[aleksey_s]

What do you think about RestructuredText?? I've just invented that the django framework documentation is maintained in it and it looks quite pretty both in html and in source variant. I believe, that we should prefer a wide-known solution rather than hacking around.
Anyway, I highly appreciate the work the ticket author has done!

[jteh]

I investigated restructured text a while ago. One of the big issues with it is that headings require you to pad the following line with symbols up to or greater than the length of the text. For example, if you write a heading which is 50 characters long, you have to have 50 equals symbols on the next line. You can get around this by just always pasting 80 symbols or something (as more is okay), but I still think it's a bit painful.

[jteh]

Bzr branch:  http://bzr.nvaccess.org/nvda/t2tDocs/

[jteh]

Branch merged into pending. Tasks yet to be completed:

• Document the txt2tags dependency and figure out how we're going to request that it be installed. (It's just one Python script.)
• Create a script to automate the generation of all docs.
• Translators: Translate the documentation based on the new txt2tags stuff.
  • The only thing that should be different is the translated text. All other aspects (structure, headers, etc.) should be identical to the English documentation.
  • Note the new names: userGuide.t2t and changes.t2t.

[jteh]

As of changeset:main,3442, txt2tags dependency has been documented and generate.py now generates the docs, so the only thing left is translations.

[Bernd]

Hi Jamie,

how can I correct german umlauts for example ä, ö, ü and so on? Should I use normal html-tags?

[pvagner]

Replying to Bernd:

how can I correct german umlauts for example ä, ö, ü and so on? Should I use normal html-tags?

I am not sure I do understand. t2t files are UTF-8 encoded and can contain non-ascii characters. When these are converted to html it works fine for me.

[jteh]

Make sure you save the file as UTF-8 encoded. If your editor does not support this, you may need to use a different editor.

[Bernd]

thanks. Now I know the bug.
Peter: Would you convert german documentation into UTF8 or should I send you the converted files?

[jteh]

Right now:

• These languages have no user guide: ru, nl, th, uk, ja
• These languages have no changes doc: zh, uk, pt_BR, pt_PT, th, ja
• These languages have a user guide which needs to be converted: zh, pt_BR, it, ar, es, gl, pl
• These languages have a changes doc which needs to be converted: ru, nl, it, ar, pl, cs, gl, es

[Bernd]

I think it would be the best if such tickets could be send to all translators. (in addition via the mailinglist) So they read it sure.