View Issue Details

IDProjectCategoryView StatusLast Update
0001097Main CAcert Websitetranslationspublic2013-05-30 14:10
ReporterNEOatNHNGAssigned ToNEOatNHNG 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version2012 Q3 
Target VersionFixed in Version2013 Q1 
Summary0001097: Special characters which have no HTML-entities are not properly escaped
DescriptionSome special characters which do not have a special HTML entity such as ä will not be escaped and therefore misinterpreted as ISO-8859-1 although they are UTF-8 (which is generated by the translations system)
TagsNo tags attached.
Reviewed bydastrath, NEOatNHNG
Test Instructions

Relationships

duplicate of 0000005 closed Main CAcert Website Incorrect cyrillic characters encoding 
related to 0001011 closedNEOatNHNG Main CAcert Website HTML tags in translations are not escaped 
related to 0000769 new Main CAcert Website Client certificate broken with unicode 
related to 0000770 new Main CAcert Website PGP keys signing fails with unicode 
related to 0000641 closed Main CAcert Website Database query failed. Error received from database was 0001267 
related to 0000771 new Main CAcert Website WOT form displays messed up name 
related to 0001039 new Main CAcert Website Problems to create CAP form languages that are not using English letters 
related to 0000851 new Main CAcert Website Problems with diacritical letters in CAP-Form and certifcate 
related to 0000992 closedNEOatNHNG Main CAcert Website Problem with diacritic characters while adding PGP/GPG public key 
related to 0000895 closededgarwahn Main CAcert Website Login to Testserver-Mgmt-System doesn't work (identified to have special char "§" in password) 
related to 0000610 needs work CATS.cacert.org use utf-8 as encoding 
related to 0000367 closedepilitimus Main CAcert Website e-mail contact via contact formular on www.cacert.org to find assurer seems not to be working 
related to 0000087 confirmed Main CAcert Website Issuing certificates for Jabber servers/users 
related to 0000991 needs workNEOatNHNG Main CAcert Website commonName is wrongly burned on CSR 
related to 0000979 needs feedbackBenBE Main CAcert Website HTML <meta> tag states charset=utf-8 on some pages when it is not 
related to 0000869 new Wiki Wrong characters while creating the pdf for COAP 
related to 0000883 new Main CAcert Website Special Character not properly displayed in CAP 
related to 0001067 new Main CAcert Website White screen error during forms generation with partially installed system (test jig setup, fix attached) 
related to 0000530 closed Main CAcert Website XMPP extension not present after renewal 
related to 0000518 new Main CAcert Website Valid E-Mail addresses with specal chars are rejected 
related to 0000714 needs work Main CAcert Website Valid local part gets rejected when adding new mail address to account 
related to 0001081 closedINOPIAE Main CAcert Website https://secure.cacert.org/account.php shows funny characters 
related to 0000936 closedUli60 Main CAcert Website Special character problem with one special account 
related to 0000935 new Main CAcert Website Pending of client certificates with an email address contains a special character 
related to 0000928 closed test.cacert.org Wrong characters by using special chars in Location 
related to 0000903 new Main CAcert Website error on locality name when content an apostrofe 
related to 0000885 closedNEOatNHNG bugs.cacert.org Polish national characters (and probably most others) cannot be entered in Real Name 
related to 0000798 new Main CAcert Website IDN certificate issuing - update in accordance with policy decisions 
related to 0000725 needs workteus Wiki CAP and COAP Generator fail with special Caracters 
related to 0000651 closedSourcerer Main CAcert Website Useless use of UTF-8 MIMEWords in E-Mail subjects 
related to 0000585 closed Main CAcert Website Issues with escaping on web-site e-mail forms 
related to 0000567 closedINOPIAE Main CAcert Website Cannot assure someone with uppercase letters in the email address 
related to 0000816 closedNEOatNHNG Main CAcert Website language encoding not correct for non-english pages 
related to 0000008 needs reviewSourcerer Main CAcert Website Normalization of special characters when comparing names 
related to 0001100 needs work Main CAcert Website findings from David 

Activities

NEOatNHNG

2012-09-13 17:01

administrator   ~0003196

I have implemented a fix and deployed it on the test server. Please review and test.

INOPIAE

2012-10-25 07:22

updater   ~0003272

I just checked with Russian where the characters are not shown properly on the productive system. In the test server it seems to be correct.
Ticket s20121024.282

hsw

2012-10-25 10:00

reporter   ~0003273

Last edited: 2012-10-25 10:02

View 2 revisions

s20121024.282 works on https://cacert1.it-sls.de/

But the use of html-entities
 "& 0001042;& 0001099; & #1085;& 0001086;& 0001074;& 0001080;& 0001095;& 0001086;& 0001082; & 0001074; CAcert?"
instead of Unicode characters
 "Вы новичок в CAcert?"
looks ugly.

Why do you use charset=iso-8859-1 when the whole world is using utf-8 for multilingual sites?

PHP from v5.4 use $encoding = 'UTF-8' by default.

NEOatNHNG

2012-10-29 00:42

administrator   ~0003280

@hsw because the software we use is code that was originally written for PHP4 and switching to UTF-8 for the whole thing as much as I would want that, is a major task. Right now we just don't have the man power to do such a thing.

INOPIAE

2012-11-06 23:12

updater   ~0003318

Last edited: 2012-11-06 23:28

View 4 revisions

For Bulgarish, Russian, Japanese show wrong characters.

OK are
Portugese Brasileiro
Greek
Polish
Soumi

I found the following a chinese/japanese name is shown correct while assuring, and in points 10.php.
In the SE console and in 15.php it is not displayed correct.

JensK

2012-12-08 11:39

reporter   ~0003384

Last edited: 2012-12-08 11:42

View 2 revisions

Retested due to mail from Ulrich. Still found wrong characters though. For example, the bottom line of the bulgarian home page shows some characters as numbers: https://dl.dropbox.com/u/6874986/broken_bulgarian.png
The same thing also happens in japanese.

NEOatNHNG

2012-12-10 00:22

administrator   ~0003389

Last edited: 2012-12-10 00:24

View 2 revisions

@INOPIAE & JensK broken Bulgarian is because of broken translation. The translation as on the translation server contains the literal string "Политика на &0001087;оверително&0001089;т" while they should just contain the characters (e.g. copied from the symbol table if entering via keyboard is not possible). I have corrected the string in the translation server to "Политика на поверителност" and updated the translation files on the test server. Now the characters look all right. I guess that's the same for the other languages, could you please check that?

So as I see it this is no bug in the patch but in the translations entered into the translation server.

JensK

2012-12-30 11:30

reporter   ~0003582

Just checked all available languages again. I did not find any misinterpreted characters, so I'd say this bug is fixed.
I found a few more errors like the one mentioned above though. I checked some of them on the translation server, and they were all in the translation like NEOatNHNG wrote.

L10N

2012-12-31 15:43

reporter   ~0003584

Lingála (not yet activated) could become another challenge for misinterpreted characters. At http://translations.cacert.org the accents are well shown on a-e-i-o-u vowely, but not on open e (ɛ) and open o (ɔ). In this case, they apears on the following consonant.

Example: PDF Form no 13: "Adɛlɛ́sɛ"

(When copying it here, it is shown as "Adɛlɛ´sɛ" and I do not know, if it becoms with the accent on the ɛ as it should or on the s as it should not after saving the note.

At ln.wikipedia.org, the solution is (since 2005) a CSS indicating the fonts that put the accents well, [1] because until this two letters are added also with accents to unicode, they are shown correctly only with some fonts, but not with all (a lot of tests done by Moyogo and me in 2005).

[1] https://ln.wikipedia.org/wiki/Discussion_utilisateur:Eruedin#Affichage_de_.C9.94.CC.81_et_.C9.9B.CC.81 (in french, with link to the css)

BenBE

2013-01-16 02:12

updater   ~0003674

Tested by 2, Lingala currently out-of-scope, should be done once activated in life system.

2nd Review by Dirk.

wytze

2013-01-17 15:33

developer   ~0003687

The patch has been installed on the production server on January 17, 2013. See also: https://lists.cacert.org/wws/arc/cacert-systemlog/2013-01/msg00006.html

Issue History

Date Modified Username Field Change
2012-09-11 22:46 NEOatNHNG New Issue
2012-09-11 22:46 NEOatNHNG Assigned To => NEOatNHNG
2012-09-11 23:55 NEOatNHNG Relationship added related to 0001011
2012-09-13 16:55 NEOatNHNG Source_changeset_attached => cacert-devel testserver 10edfca6
2012-09-13 16:55 NEOatNHNG Source_changeset_attached => cacert-devel testserver c25bf27b
2012-09-13 17:01 NEOatNHNG Reviewed by => NEOatNHNG
2012-09-13 17:01 NEOatNHNG Note Added: 0003196
2012-09-13 17:01 NEOatNHNG Status new => needs review & testing
2012-10-25 07:22 INOPIAE Note Added: 0003272
2012-10-25 10:00 hsw Note Added: 0003273
2012-10-25 10:02 hsw Note Edited: 0003273 View Revisions
2012-10-29 00:42 NEOatNHNG Note Added: 0003280
2012-11-06 23:12 INOPIAE Note Added: 0003318
2012-11-06 23:16 INOPIAE Note Edited: 0003318 View Revisions
2012-11-06 23:19 INOPIAE Note Edited: 0003318 View Revisions
2012-11-06 23:28 INOPIAE Note Edited: 0003318 View Revisions
2012-11-21 00:38 INOPIAE Status needs review & testing => needs work
2012-12-08 11:39 JensK Note Added: 0003384
2012-12-08 11:42 JensK Note Edited: 0003384 View Revisions
2012-12-10 00:22 NEOatNHNG Note Added: 0003389
2012-12-10 00:22 NEOatNHNG Status needs work => needs review & testing
2012-12-10 00:24 NEOatNHNG Note Edited: 0003389 View Revisions
2012-12-16 20:15 INOPIAE Relationship added duplicate of 0000005
2012-12-22 20:16 Werner Dworak Relationship added related to 0000769
2012-12-22 20:16 Werner Dworak Relationship added related to 0000770
2012-12-22 20:21 Werner Dworak Relationship added related to 0000641
2012-12-22 20:24 Werner Dworak Relationship added related to 0000771
2012-12-22 20:25 Werner Dworak Relationship added related to 0001039
2012-12-23 07:33 Werner Dworak Relationship added related to 0000851
2012-12-23 07:34 Werner Dworak Relationship added related to 0000992
2012-12-23 07:34 Werner Dworak Relationship added related to 0000895
2012-12-23 07:35 Werner Dworak Relationship added related to 0000610
2012-12-23 07:36 Werner Dworak Relationship added related to 0000367
2012-12-23 07:36 Werner Dworak Relationship added related to 0000087
2012-12-23 07:44 Werner Dworak Relationship added related to 0000991
2012-12-23 07:45 Werner Dworak Relationship added related to 0000979
2012-12-23 07:49 Werner Dworak Relationship added related to 0000869
2012-12-23 10:04 Werner Dworak Relationship added related to 0000883
2012-12-23 10:07 Werner Dworak Relationship added related to 0001067
2012-12-27 20:52 Werner Dworak Relationship added related to 0000530
2012-12-30 11:30 JensK Note Added: 0003582
2012-12-31 15:43 L10N Note Added: 0003584
2013-01-07 09:09 Werner Dworak Relationship added related to 0000008
2013-01-08 06:21 Werner Dworak Relationship added related to 0001100
2013-01-11 17:10 Werner Dworak Relationship added related to 0000518
2013-01-11 17:11 Werner Dworak Relationship added related to 0000714
2013-01-11 17:11 Werner Dworak Relationship added related to 0001081
2013-01-11 17:17 Werner Dworak Relationship added related to 0000936
2013-01-11 17:18 Werner Dworak Relationship added related to 0000935
2013-01-11 17:18 Werner Dworak Relationship added related to 0000928
2013-01-11 17:21 Werner Dworak Relationship added related to 0000903
2013-01-11 17:22 Werner Dworak Relationship added related to 0000885
2013-01-11 17:24 Werner Dworak Relationship added related to 0000798
2013-01-11 17:28 Werner Dworak Relationship added related to 0000725
2013-01-11 17:30 Werner Dworak Relationship added related to 0000651
2013-01-11 17:32 Werner Dworak Relationship added related to 0000585
2013-01-11 17:32 Werner Dworak Relationship added related to 0000567
2013-01-15 21:19 Uli60 Relationship added related to 0000816
2013-01-15 23:17 NEOatNHNG Source_changeset_attached => cacert-devel testserver-stable a07b3ea7
2013-01-16 02:12 BenBE Reviewed by NEOatNHNG => dastrath, NEOatNHNG
2013-01-16 02:12 BenBE Note Added: 0003674
2013-01-16 02:12 BenBE Status needs review & testing => ready to deploy
2013-01-16 02:12 BenBE Description Updated View Revisions
2013-01-16 04:30 BenBE Source_changeset_attached => cacert-devel release 98c12d45
2013-01-17 15:33 wytze Note Added: 0003687
2013-01-17 15:33 wytze Status ready to deploy => solved?
2013-01-17 15:33 wytze Fixed in Version => 2013 Q1
2013-01-17 15:33 wytze Resolution open => fixed
2013-05-01 01:10 INOPIAE Status solved? => closed
2013-05-30 14:10 INOPIAE Product Version => 2012 Q3