Dada Mail Mailing Lists: [dadadev] Localization in Dada Mail

From: Justin J
Subject: [dadadev] Localization in Dada Mail
Date: Sat Feb 4 12:01:15 2012

I've been working on the problem of localization in Dada Mail.

What makes this an interesting project within Dada Mail is because:

  • This is becoming the #1 requested feature: "Is this available in my language?"

And I keep having to say, "no, but..." and then tell them to go through all the templates and manually edit them. This makes people go, "well, what happens when I upgrade?" And, I have no good answer to that. And that sucks.

  • What a great way to get more people involved in Dada Mail

And this will happen two ways:

#1 For this project to work, people will need to do the language translations. I can't do every darn language :) It would be great to have English, French, German and Spanish to start out.

I do want to take a stab at the French translation, since that's my personal second language that I'm currently studying actively. I think other people, even on this list, would be interested in other language translations. These translations can also be collaborative, since we live in a world of version control systems ;)

#2 More people will use Dada Mail, since it'll be localized to their language, etc. This is a great thing, since more users means a more healthier project, as far as I'm concerned.

I've been doing a lot of research on how to pull this off. There's a lot of problems:

  • There's text that needs to be translated in Dada Mail that lives in template files and in the source of the program itself. There will always be these two things.

No one - not one person, wants to slog through every file in Dada Mail, looking for text to translate.

  • Then, there's the upgrade problem. Woo - boy, how do you handle when the program gets upgraded? All the translated work has to be thrown out?
  • Text needs to be in one central place (for each language) that can be easily edited by non programmers to be able to edit
  • How do you do this, without needing a gigantic dependency tree?

A lot of what makes Dada Mail popular is that you don't have to install CPAN Perl modules yourself to get up and running. This has to be true for anything new installed, dealing with localization.

I've finally gotten around to figuring out how to pull all these things together.

What I propose to use is a module called: Locale::Maketext::Lexicon

http://search.cpan.org/dist/Locale-Maketext-Lexicon/lib/Locale/Maketext/Lexicon.pm

This module has a lot of neat features.

One of them is, there's no real nasty dependencies - it's in Perl core, itself. The few things that aren't already in the core Perl distro, I can easily bundle myself. It hasn't been updated in about a year, so it's not exactly a fast-moving target

The way it works is this:

It has the idea of something called, ".po" files. In it, there are key/value entries for all the strings you want/need to translate. The key has the original text in the original language, the value is what your translation will be. For example, here is a key/value pair in a .po file:

msgid "Hello, World!"
msgstr "Bonjour, tout le monde!"

This is in a .po file for the English -> French translation.

These key/value pairs can do some pretty fancy things (but still be fairly readable), if you have to deal with localization thingies, like dates, times, numbers - stuff like that.

I've made a simple program to illustrate all this - you can download it here:

http://dadamailproject.com/dev/local.tar.gz

Here's the directory structure - if you'd like to download and walk along with me:

local
hello.pl
lang
hello_fr.po
MyProgram
Config.pm
L10N.pm
tmpls
hello.tmpl

"hello.pl" is our program. It's going to parse and print out the template in, "tmpls/hello.tmpl". It has strings in English, that we're going to have translate into French. Looks like this: 

        <html>
        <head><title>{{Hello}}!</title>
        <body>
        <p><!-- tmpl_var greeting --></p>
        <p>
                {{My name is Justin! It's very nice to meet you!}}
        </p>
        
        </body>
        </html>

See those double-braces? That's the little signifier used to tell this little program that within these, are strings that should be translated. There's also going to be another variable we're going to pass to the template, <!-- tmpl_var greeting --> that we're going to have translated in the program, itself.

Every single template file will need to have these double-braces applied to the strings we want translated, but it's not that scary.

The program has a config module (just like Dada Mail). All it does is set the language. In, MyProgram/Config.pm, you'll see the line:

$lang = 'fr';

That tells it to use French.

All the fancy-footwork that does the localization is in the, MyProgram/L10N.pm file. It's not too scary.

In the, hello.pl this takes care of the localization:

use MyProgram::L10N;
my $lh = MyProgram::L10N->get_handle($MyProgram::Config::lang) || die "What language?";

Once you have that, "$lh" object, you can change simple, "print" statements, like:

print "Hello, World!";

to,

print $lh->maketext("Hello, World!");

and the French localization will happen! That's it and that's how inline translation in perl code will work. You won't need to hunt for this yourself. It'll be a little grudgework to get that all set in Dada Mail, but it's not really that difficult.

And here's the big problem - especially for something so large and organically grown as Dada Mail -

How do we find all these strings that need translated?

Well,

Local::Maketext::Lexicon has a nifty utility:

http://search.cpan.org/~drtech/Locale-Maketext-Lexicon/script/xgettext.pl

That does just that.

If you run it on the, "local" directory:

justin$ xgettext.pl -D ./local

It'll give you a, ".po" file, that'll have this as the contents:

#: local/tmpls/hello.tmpl:2
msgid "Hello"
msgstr ""

#: local/hello.pl:13
msgid "Hello, World!"
msgstr ""

#: local/tmpls/hello.tmpl:6
msgid "My name is Justin! It's very nice to meet you!" msgstr ""

All I need to do, is fill in the translations that are in the, "msgid" thingies, and put them in the, "msgstr" thingies. For French, it would look a little like this:

#: tmpls/hello.tmpl:2
msgid "Hello"
msgstr "Bonjour"

#: hello.pl:13
msgid "Hello, World!"
msgstr "Bonjour, tout le monde!"

#: tmpls/hello.tmpl:6
msgid "My name is Justin! It's very nice to meet you!" msgstr "Je m'appelle Justin! C'est tr�s gentil vous conn�itre!"

Now, here's the kicker - you can update this file (say, when you make changes - big changes, to the program), using the same utility! Woo!

This system will also work for partially translated stuff - it won't blow up if there's a missing key/value pair. Nice.

So, if you run the script, with English inline and within the template file, but with the program set to localize into French (and with a English -> French .po file), it'll output:

<html>
<head><title>Bonjour!</title> <body>
<p>Bonjour, tout le monde!</p> <p>

Je m'appelle Justin! C'est tr�s gentil vous conn�itre! </p>

</body>
</html>

Pretty much exactly what we need.

Here's some further reading:

Locale::Maketext::TPJ13 -- article about software localization

http://search.cpan.org/~ferreira/Locale-Maketext-1.13/lib/Locale/Maketext/TPJ13.pod

Web Localization in Perl
http://cpansearch.perl.org/src/DRTECH/Locale-Maketext-Lexicon-0.77/docs/webl10n.html

You don't have to know this - I barely understand it, and the people doing the translating definitely DO NOT need to know how this is implemented. All they have to do, is to fill out the translations of the strings in those .po files - and those .po files are made automatically.

So, can I "sign up" anyone that wants a language? :) Like I said, collaboration on the same language is def. possible. If you know github, it'd be the best to fork your own copy of Dada Mail and get hacking.

BEFORE you do that, I need to put all the localization code in Dada Mail, change the strings so that the localization works (and can be found by the automatic find thingy) and normalize some of the strings, so that translation is easier - and probably make the French localization to use as a guide.

Let me know if this sounds like something you can help on ;)

    << Previous: [dadadev] Re: Change Email Address

    | Archive Index |

    Next: [dadadev] Re: Localization in Dada Mail >>

    (archive rss , atom )

    Forward to a Friend




    As the final step, please type in the string of letters that you see in the below image into the text box:

    this list's archives:

    This mailing list is to discuss the nerdy programming development of Dada Mail -

    If you are just looking for support Dada Mail, consult the message boards at:

    http://dadamailproject.com/support/boards

    To post to this list, send a message to:

    dadadev@dadamailproject.com

    All subscribers of this list may post to the list itself.

    Some on topic... topics include:

    • Positive Crits on the program (I like, "x", but, "y" needs some work - here's an idea on how to make this better...)
    • Bug/Error reports
    • Bug fixes
    • Request For Comments on any changes to the program
    • Help customizing Dada Mail for your own internal needs
    • Patches
    • Language Translations
    • Support Documentation/Doc editing, FAQ's, etc.
    • Discussion of any changes that you would like to be committed to the next version of Dada Mail -

    At the moment, there aren't many people with CVS access for Dada Mail - if you would like CVS access, please first talk about the changes you propose and how it will affect the program. If the idea is sound and agreed upon, the change will be comitted. A good track record of this will allow you to have CVS access. Some reasons that patches will not be accepted is if the patch breaks compatibility with a previous version of the program, the patch is too centric to your own problem or the patch simply isn't very good.

    Please, please please familiarize yourself with the documentation at:

    http://dadamailproject.com/support/documentation/

    Since no one wants to answer the same question twice.

    Another sneaky reason for this mailing list is to test out the discussion list capabilities of Dada Mail, since Dada Mail is used for the mailing list itself.

    NOTE - because of this, there may be times that this list will be somewhat broken. Although we're not planning on breaking the program by using it, we're giving you the heads up that this may well happen anyways.

    Subscribe/Unsubscribe on Dada Mail Developers

    * Required

Loading