I18N data files updated

January 31, 2009 · Published by Avatar of Fabian Lange Fabian Lange

symfony uses components from other frameworks when it makes sense. The i18n components accessing localized country names and much more were imported from Prado framework. Unfortunately the Prado people have not updated these files for quite some time. Also the script they used to initially import the data was not published.

Because the Common Locale Data Repository continues to update their data, countries have changed names, data formats and timezones, the data files in symfony are no longer up to date. There have been a number of reports on the quality of the data files on Trac which we couldn't close because we had no idea how to update the files without support from Prado.

This week I finally decided to reverse engineer the conversion script and was able to produce nearly the same data we originally had. Because I don't want this icu i18n conversion script to be lost, I added it to the symfony svn. The script contains instructions on how to use it and takes the ICU locale data files as input, which happen to be close to yml. The script is not perfect, but good enough to solve the current conversion problem and most likely will be adopted anyway for the next run. If you are interested in the creation process of the script you can read more in my private blog.

Unfortunately the amount of changes to the sources, especially in respect to timezones (where CLDR added a "meta timezone" concept) the imported data is not 100% backwards compatible to the data bundled with symfony 1.0, 1.1 and 1.2. This is especially true when custom code accessed the raw data without using the symfony accessors. The main change in the data files is that many values that formerly had no named key now have one (again the timezones had the biggest change here). I also changed the sorting of the select_* helper to respect the locale dependent sort order.

Because of this I decided not to put the new data files into the older symfony versions. If you really need this you should be able to hack my script to produce something that could satisfy your personal update needs. As alternative you could "try" to use the new data files if you just use a specific feature and want newer data.

Because 1.3 is still "living on the edge", I am very happy to receive feedback and suggestions on how to make the i18n more compatible, or remaining bugs, or ideas on how to improve performance of the parsing times.

At least the following tickets have been resolved:

Published in #Living on the edge

Help the Symfony project!

As with any Open-Source project, contributing code or documentation is the most common way to help, but we also have a wide range of sponsoring opportunities.

Comments

Avatar of puentesdiaz
puentesdiaz said on Feb 1, 2009
awesome, just what i need
Avatar of Christian
Christian said on Feb 2, 2009
Well done. I wonder if the following trac although got solved automagically... http://trac.symfony-project.org/ticket/4413

One plea though: I read that you weren't able to update the dat-files due to Prado's "fault". I would have loved to get this response, instead my trac just stayed uncommented/unknown.
Avatar of Fabian
Fabian said on Feb 2, 2009
@Christian: yes that should be fixed as well.. The reason why you did not get a response was that you closed the ticket yourself. It way only open for 2 weeks in which we were especially busy with getting 1.2 out. If you would have left it open it would have gotten more attention.
Avatar of peterVG
peterVG said on Mar 2, 2009
This is good news. We make heavy use of symfony's i18n features in our project. To comply with some standards we need to import and export ISO language codes in the 639-2 format (three characters). Unfortunately, it looks like the ICU language codes mix 639-1 (2 characters) with the 639-2 codes in the same array. We need these to be either one or the other, or better provide a way to switch between the two (e.g. a format_iso639() helper that converts between the two). At least now we can experiment with your conversion script to see if we can clean up and splice the list. Thanks.

Comments are closed.

To ensure that comments stay relevant, they are closed for old posts.