Lokalize in 4.8

What's new in Lokalize from KDE 4.8:

Translation memory based QA: you can have a permanent list of translator's false friends and check your TM (which can be made to reflect all your project) against them whenever you like. Because the check is done based on TM and not real files, this should work much faster [than pology]. And you can even add this list to your language dir in SVN repository to share it among other team members, the way it is done with glossaries.

Also:
* better support for real world glossaries, including large ones;
* improvements in XLIFF support (part of them contributed by Albert Astals Cid)
* faster UI
* some 14 bugfixes (apart from what was available in 4.7.x updates)

I'd like to say 'thank you' to all bug reporters. Apart from QA feature, all other changes were made as a reaction to user requests.

Testing KDE 4.3 translation with Lokalize

I created Lokalize script to automate UI translation testing: it compiles currently opened PO file and puts it under ~/.kde where KDE can see it, so that when you start your application next time, it will use your translation.

To use it for your language (you'll need Lokalize coming with KDE 4.3):

cd trunk/l10n-kde4
svn up scripts
dolphin scripts/lokalize

then open Lokalize with project for your language, go to Project -> Configure project -> Scripts and drag-n-drop msgfmt.rc from dolphin to the settings area in Lokalize. Restart Lokalize. You should see a new item under Tools menu.

To make this script available automatically for all your team members copy this msgfmt.rc to l10n-kde4/LANG/lokalize-scripts (they will also need to svn up trunk/l10n-kde4/scripts).


Now a few words on packages.
I have been given write access to Debian KDE packaging repository (thanks to Sune Vuorela), where I was able to specify right dependencies and recommended packages for Lokalize. If any of other distro packagers are reading this, please sip on the spec.

P.S. I installed kde 4.2.96 packages from debian experimental (it conflicts with kdevelop package, so I reinstalled kdevelop manually with dpkg -i kdevelop_3.9.94-1_i386.deb afterwards).

Scripting Lokalize

I started working on scripting support in Lokalize several months ago, but never really announced this because I wanted to write my own scripts and refine/extend API accordingly along the way.

KDE 4.3 will bring us 'officially' scriptable Lokalize. Among recent additions are new project wizard with anonsvn checkout support (screencast; it also checks out templates/ and scripts/ folders), actions to update .po from template and to compile .po into .mo and place it in local .kde folder so it could be picked up automatically by applications on the next start (see here and here for python sources).

Recently I also extended pology with ability to communicate to Lokalize, so now you can run xml checks against your translations and get erroneous files opened in Lokalize and positioned on appropriate entry.

So how it works: on each project open Lokalize scans PROJECTDIR/lokalize-scripts folder for .rc files and adds them to a 'cache' file called PROJECTDIR/lokalize-scripts/scripts.rc (so you shouldn't generally want to add it to version control system). RC files contain descriptions of scripts, particularly their paths. The path is relative to .rc file folder (or to a system scripts folder - they are tried both). So you for example can specify "../../scripts/lokalize/opensrc.py" to load script from global kde-l10n4 scripts folder (i.e. not specific to your language). Each script is represented with action in application menu (and you may assign a shortcut to it).

I wrote a script (in qtscript/javascript) to do a check specific to Russian language: %1 should be mentioned in _all_ plural forms, because, in Russian, the first form is also used for 21, 31 and so on. You can find it here. The script contains no immediate code, but only a function. The name of the function (fileSaved) is essential, because it matches the signal emitted by Editor object, which allows us to communicate with current editor tab in Lokalize. The function is automatically called on each files save, and if file contains errors, a warning is issued and wrong entries are shown. Scripts are actually don't get loaded until user explicitly triggers their action in the menu, so to override this behaviour RC file specifies <property name="autorun" >true</property>.

Below are links to API references. Everything marked as Q_SCRIPTABLE may be used from scripts
Editor object API reference
Lokalize object API reference
Project object API reference

You are welcome to write your own scripts and polish existing ones.

Lokalize enforces workflow

Last week I added workflow phases support to Lokalize (XLIFF only). A person defines his/her role in the project [settings], and then corresponding phase is created automatically when file first changes (an old phase opened by same person is reused if found).

You may list phases, create them manually as well as add notes to them.


Also I added xliff extended states support. I did this in a smart way - there is still two-state button, which selects extended state depending on kind of the current phase (needs-translation/translated for translation phase, needs-translation-review/signed-off for review phase),
with drop-down menu to select specific state manually.

So what for translator is ok (translated), for reviewer is not (i.e. rendered in italics), and needs to be signed-off.



This allows us to achieve something interesting: we now able to see in which phase (and hence by whom) the translation unit was modified last time. This info is shown in Metadata toolview (previously known as 'Message Context').

First results implementing OpenDocument translation workflow

This is how OpenDocument translation workflow will look like in KDE 4.3: Screencast

Don't try to do it yourself right now ;) This requires the latest (unreleased) versions of translate-toolkit trunk and kdelibs trunk.

Implications of achieving this
I plugged kross nicely into Lokalize: I introduced 'project kinds' -- each has it's own set of scripts, so that you have 'Merge into ODF' for OpenDocument translation, and 'Widget Text Capture' for GUI/KDE projects.

These actions, as well as few others (see kde l10n svn and kde ru l10n svn for their plug example) are written in python. I plan to create a separate scripting API docs for Lokalize, but for now you can just rgrep Q_SCRIPTABLE KDE/kdesdk/lokalize to get list of available methods.

This mechanism will make implementing business-logic easy. Examples are action to send your translation to project's editor (translation team leader) and action to commit your work to [svn] repository if you have write access to it.

Thanks
I want to say thanks to Sebastian Sauer for giving me the green light for kross-extending commits, and Wynand Winterbach for commiting the patch to translate-toolkit.
Once I've got the thanks section, I'd also like to mention NLNet Foundation which funds my current efforts and KDevelop team, which makes those efforts efficient (today I opened whole kdelibs in KDevelop, and it took 15 minutes and 0 crashes to scan it completely ;) )

Comment your translations

It it sometimes useful to leave a note for an individual translation (for example when you're unsure about some term). Since some time ago scripty (a daemon which updates KDE translation files to reflect the current state) started to preserve comments in GETTEXT PO files. XLIFF standard supports notes too. So this week I was addding notes support to Lokalize.

Notes mechanism may also be used for bookmarking, as Lokalize now allows quick searching including notes (I added 'options' button to set additional filtering critera). Check this out on the video.


I wrote a simple python-based action to merge translated XLIFF file back to .odt document. Next week I'll be writing patches for traslate-toolkit to support on-the-fly conversion and filling <file origin""> attribute, and now I'm preparing a patch for Kross to help me make organizing project-specific scripts easy.

I also added UI reflection of 'recovered from autosave' state. You'll need kdelibs>=4.2.1 for autosave to work properly though.

Compiling Lokalize from trunk under KDE 4.2

XLIFF markup editing in Lokalize

Please correct me if I'm wrong, but today Lokalize became the first open source tool to support XLIFF inline markup editing.

I tried the latest svn trunk revision of virtaal, but it didn't display markup and mangled my test XLIFF file. AFAIK inline markup editing in proprietary world is only supported by Java Heartsome l10n Suite.

But this of course would mean nothing w/o Translate-Toolkit's (same laboratory as Virtaal's) odf2xliff converter, which will be integrated into Localize thanks to KDE's kross technology ;)

Obligatory screencast.

My plans also include support for inserting markuped text from other entries and translation memory (Lokalize will have to match tag id's for that).