View Issue Details

IDProjectCategoryView StatusLast Update
0001354Main CAcert Websitesource codepublic2021-08-25 13:38
ReporterBenBE Assigned ToBenBE  
PriorityhighSeverityminorReproducibilitysometimes
Status needs reviewResolutionopen 
Product Version2015 Q1 
Summary0001354: Problems with diacretics and non-latin1 characters
DescriptionBased on a mail report the following issue was found:

When trying to sign a PGP key with diacretics in the name or other non-latin1 characters the name is not accepted by the software. This is due to the way data is encoded in the database.

Further investigations showed several instances of double-escaping of the displayed name throughout the software.
Steps To ReproduceRegister an account with non-latin1 characters in it. Throughout the software you'll notice HTML entity encoding for the name.
Additional InformationThe storage in the database is as follows:
- Everything in latin1 is stored AS IS.
- Everything outside latin1 is stored as &# 1234; with 1234 being the decimal code point of the character.
TagsNo tags attached.
Reviewed byBenBE
Test Instructions0001354:0005216

Relationships

related to 0000008 needs reviewSourcerer Normalization of special characters when comparing names 
related to 0000769 needs workTed Client certificate broken with unicode 

Activities

BenBE

2015-01-02 14:24

updater   ~0005214

A priliminary patch has been applied on the test server which for now ignores the issue mentioned in 0000008 but is written so the necessary changes from there could be easily incorporated. Also this ignores simplifications of the name, thus "Mueller" instead of "Müller" in the account is not yet accepted when signing. Such simplifications are subject to 0000008 and others.

BenBE

2015-01-03 01:53

updater   ~0005215

A simple set of transliteration rules has been added. Please see the source for which exactly as the list is a bit longer.

Please note that Müßli can be written Muessli or Mussli, but neither Mußli nor Müssli are acceptable. Thus as given in the Practice on Names a transliteration always has to be complete.

felixd

2015-01-03 02:12

updater   ~0005216

Last edited: 2015-01-03 02:15

Test instructions:

create an account with strange characters in his name (ä,ö,ü,ß and many more).

try to transliterate them all and add this as an id to your gnupg key.
signing this id should work.
signing any other ids (especially ones where the transliteration is not complete) should fail

felixd

2015-01-03 02:15

updater   ~0005217

Test:

An account named "ÞÞ ÀÆÐÒÓÔÕÖØÛÜÝ" had

- thth aaedjoooooouuey signed
- thth aaedoooooouuey not signed
- Þth aaedjoooooouuey got not signed

Test was therefore PASSED

maze

2015-01-03 20:25

reporter   ~0005224

Test now passes.

Created an account with first/middle/last name of "Maciej" "Arthur" "Żenczykowski".

My last name now displays correctly in "My Details - View/Edit" (both places last name shows up in account history looks good too).

Signing a gpg key now passes UID verification.

Key used:

$ gpg2 -K AB316850
sec 4096R/AB316850 2010-01-22
uid Maciej Arthur Żenczykowski (Personal)
<zenczykowski@gmail.com>
ssb 4096R/FA401B8D 2010-01-22

Blob to sign generated via:

$ gpg2 --export --export-options export-minimal --armor AB316850

Eva

2015-01-13 22:25

updater   ~0005235

I created an account with below data and got it assured to 100 points.
first name: Äüößčéěęæ
last name: ŇÁÝŘŁŒŸÇ
no middle name and no suffix
email: bug1354es@acme.com

I created a pgp key with UID: Äüößčéěęæ ŇÁÝŘŁŒŸÇ <bug1354es@acme.com>

And tried to get it signed (without a comment)

The key was not accepted.

=> fail

BenBE

2015-01-14 00:46

updater   ~0005236

Last edited: 2015-01-14 00:58

A note on Eva's test using account bug1354es (ÄT) acme {DOT} com:

While conversion from UTF-8 of the name "Äüößčéěęæ ŇÁÝŘŁŒŸÇ" (from GnuPG) is escaped properly as
string(103) "&#196;&#252;&#246;&#223;&#269;&#233;&#283;&#281;&#230; &#327;&#193;&#221;&#344;&#321;&#338;&#376;&#199;"

the same string when fetched from the database (ISO-8859-1) is escaped only partially:
string(93) "&#196;&#252;&#246;&#223;&#269;&#233;&#283;&#281;&#230; &#327;&#193;&#221;&#344;&#321;ŒŸ&#199;"


There seems to be a PHP bug though ???

Eva

2015-01-20 21:50

updater   ~0005249

I performed the same test as at 5235. I entered "Äüößčéěęæ ŇÁÝŘŁŒŸÇ <bug1354es@acme.com>" as comment for the key.
It is working now.
-> ok
The result page showed all characters correctly.
-> ok
I performed a pgpdump with the new key and it did look good (with correct names).
-> ok
I went to the view pgp keys page. The key was there, but the comment for the key was missing.
-> fail

Eva

2015-01-20 22:11

updater   ~0005251

I did the same as 5249.
Everything was displayed / entered correctly.
=> ok

felixd

2015-03-03 21:03

updater   ~0005341

I named myself "Äüößčéěęæ ŇÁÝŘŁŒŸÇ" and got a gpg key with that name signed.

Additionally I re-executed 0001354:0005217 and everythig was still as expected

=> PASSED

Eva

2015-03-03 21:17

updater   ~0005349

as there are two positive tests, please review

Issue History

Date Modified Username Field Change
2015-01-02 14:01 BenBE New Issue
2015-01-02 14:01 BenBE Assigned To => BenBE
2015-01-02 14:01 BenBE Status new => needs work
2015-01-02 14:10 BenBE Source_changeset_attached => cacert-devel testserver-stable ac92de83
2015-01-02 14:10 BenBE Source_changeset_attached => cacert-devel testserver-stable 1db86a2a
2015-01-02 14:24 BenBE Note Added: 0005214
2015-01-02 14:24 BenBE Status needs work => needs review & testing
2015-01-02 14:25 BenBE Source_changeset_attached => cacert-devel testserver-stable b5a28349
2015-01-02 14:25 BenBE Source_changeset_attached => cacert-devel testserver-stable ae1040f5
2015-01-02 17:45 BenBE Source_changeset_attached => cacert-devel testserver-stable e793d97b
2015-01-02 17:45 BenBE Source_changeset_attached => cacert-devel testserver-stable 0b5be4b2
2015-01-02 17:45 BenBE Source_changeset_attached => cacert-devel testserver-stable cb9e52bb
2015-01-03 00:32 BenBE Relationship added related to 0000008
2015-01-03 01:50 BenBE Source_changeset_attached => cacert-devel testserver-stable e59d438b
2015-01-03 01:50 BenBE Source_changeset_attached => cacert-devel testserver-stable ba8c4e28
2015-01-03 01:50 BenBE Source_changeset_attached => cacert-devel testserver-stable 2441652c
2015-01-03 01:50 BenBE Source_changeset_attached => cacert-devel testserver-stable c12fe39f
2015-01-03 01:50 BenBE Additional Information Updated
2015-01-03 01:51 BenBE Additional Information Updated
2015-01-03 01:53 BenBE Note Added: 0005215
2015-01-03 02:05 BenBE Source_changeset_attached => cacert-devel testserver-stable fe18b916
2015-01-03 02:05 BenBE Source_changeset_attached => cacert-devel testserver-stable 3324a701
2015-01-03 02:05 BenBE Source_changeset_attached => cacert-devel testserver-stable de351d13
2015-01-03 02:12 felixd Note Added: 0005216
2015-01-03 02:15 felixd Note Added: 0005217
2015-01-03 02:15 felixd Note Edited: 0005216
2015-01-03 02:15 felixd Test Instructions => 0001354:0005216
2015-01-03 15:10 BenBE Source_changeset_attached => cacert-devel testserver-stable 8e38a22d
2015-01-03 15:10 BenBE Source_changeset_attached => cacert-devel testserver-stable 468c9730
2015-01-03 15:20 BenBE Source_changeset_attached => cacert-devel testserver-stable 02672231
2015-01-03 15:20 BenBE Source_changeset_attached => cacert-devel testserver-stable 88de5d85
2015-01-03 18:50 BenBE Reviewed by => BenBE
2015-01-03 20:25 maze Note Added: 0005224
2015-01-13 22:25 Eva Note Added: 0005235
2015-01-14 00:46 BenBE Note Added: 0005236
2015-01-14 00:49 BenBE Note Edited: 0005236
2015-01-14 00:54 BenBE Note Edited: 0005236
2015-01-14 00:58 BenBE Note Edited: 0005236
2015-01-20 21:50 Eva Note Added: 0005249
2015-01-20 22:11 Eva Note Added: 0005251
2015-03-03 21:03 felixd Note Added: 0005341
2015-03-03 21:17 Eva Note Added: 0005349
2015-03-03 21:17 Eva Status needs review & testing => needs review
2021-08-25 13:38 bdmc Relationship added related to 0000769