View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0001306 | Main CAcert Website | certificate issuing | public | 2014-09-15 14:25 | 2021-08-07 19:24 |
Reporter | wytze | Assigned To | GuKKDevel | ||
Priority | normal | Severity | major | Reproducibility | always |
Status | fix available | Resolution | open | ||
Product Version | 2014 Q3 | ||||
Summary | 0001306: expired certificates should not be listed in the CAcert CRLs | ||||
Description | The size of the current CAcert Class1 CRL (http://crl.cacert.org/revoke.crl) is 6.5 megabyte. Even the CAcert Class3 CRL (http://crl.cacert.org/class3-revoke.crl) is already 0.75 megabyte. This is causing an unacceptable huge amount of CRL download traffic (currently over 130 GB *per day*). In addition, it is causing verification failures for certain clients, e.g. the Microsoft Crypto API, due to the long time required for downloading the CRL. The main cause for the large size of the CRLs is the inclusion of *all* certificates revoked since the start of CAcert (in 2003) in there. As a result, most of the certs listed as revoked have expired a long time ago already, and are thus invalid anyway. There is no RFC requirement to include such expired certs in the CRL; omitting them will result in CRLs of a much more manageable size. | ||||
Steps To Reproduce | The attached logfile shows an example of failure on the Microsoft platform for the command: certutil -f -verify -urlfetch -t 30 server.crt | ||||
Additional Information | See also http://social.technet.microsoft.com/Forums/windowsserver/en-US/7e69d0d1-1df2-4830-8d22-f887b6261062/cacert-revocation-server-offline?forum=w7itprosecurity | ||||
Tags | No tags attached. | ||||
Attached Files | crl-size-issue.log (5,228 bytes)
Verlener: CN=CAcert Class 3 Root OU=http://www.CAcert.org O=CAcert Inc. Onderwerp: CN=bocanium.soleus.nu Serienummer van certificaat: 010c5c dwFlags = CA_VERIFY_FLAGS_CONSOLE_TRACE (0x20000000) dwFlags = CA_VERIFY_FLAGS_DUMP_CHAIN (0x40000000) ChainFlags = CERT_CHAIN_REVOCATION_CHECK_CHAIN_EXCLUDE_ROOT (0x40000000) HCCE_LOCAL_MACHINE CERT_CHAIN_POLICY_BASE -------- CERT_CHAIN_CONTEXT -------- ChainContext.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) ChainContext.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40) ChainContext.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000) SimpleChain.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) SimpleChain.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40) SimpleChain.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000) CertContext[0][0]: dwInfoStatus=104 dwErrorStatus=0 Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc. NotBefore: 6-11-2012 16:09 NotAfter: 6-11-2014 16:09 Subject: CN=bocanium.soleus.nu Serial: 010c5c SubjectAltName: DNS-naam=bocanium.soleus.nu, Andere naam:1.3.6.1.5.5.7.8.5=0c 12 62 6f 63 61 6e 69 75 6d 2e 73 6f 6c 65 75 73 2e 6e 75 de 55 08 57 34 ba 81 24 56 af dd 94 e7 eb 1c 75 fe 26 50 ca Element.dwInfoStatus = CERT_TRUST_HAS_NAME_MATCH_ISSUER (0x4) Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) ---------------- Certificaat AIA ---------------- Geen URL's "Geen" Tijd: 0 ---------------- Certificaat CDP ---------------- Gecontroleerd "Basislijst met ingetrokken certificaten" Tijd: 0 [0.0] http://crl.cacert.org/class3-revoke.crl ---------------- Basis-CRL CDP ---------------- Geen URL's "Geen" Tijd: 0 ---------------- Certificaat-OCSP ---------------- Gecontroleerd "OCSP" Tijd: 0 [0.0] http://ocsp.cacert.org/ -------------------------------- CRL (null): Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc. 79 0d 8e 2a 39 7b 7b 69 da ec b0 e0 48 f1 b2 b6 19 1e f5 ff Application[0] = 1.3.6.1.5.5.7.3.2 Clientverificatie Application[1] = 1.3.6.1.5.5.7.3.1 Serververificatie Application[2] = 2.16.840.1.113730.4.1 Application[3] = 1.3.6.1.4.1.311.10.3.3 CertContext[0][1]: dwInfoStatus=101 dwErrorStatus=1000040 Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA NotBefore: 23-5-2011 19:48 NotAfter: 20-5-2021 19:48 Subject: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc. Serial: 0a418a ad 7c 3f 64 fc 44 39 fe f4 e9 0b e8 f4 7c 6c fa 8a ad fd ce Element.dwInfoStatus = CERT_TRUST_HAS_EXACT_MATCH_ISSUER (0x1) Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) Element.dwErrorStatus = CERT_TRUST_REVOCATION_STATUS_UNKNOWN (0x40) Element.dwErrorStatus = CERT_TRUST_IS_OFFLINE_REVOCATION (0x1000000) ---------------- Certificaat AIA ---------------- Gecontroleerd "Certificaat (0)" Tijd: 0 [0.0] http://www.CAcert.org/ca.crt ---------------- Certificaat CDP ---------------- Geen URL's "Geen" Tijd: 0 ---------------- Certificaat-OCSP ---------------- Gecontroleerd "OCSP" Tijd: 0 [0.0] http://ocsp.CAcert.org/ -------------------------------- Issuance[0] = 1.3.6.1.4.1.18506 CertContext[0][2]: dwInfoStatus=109 dwErrorStatus=0 Issuer: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA NotBefore: 30-3-2003 14:29 NotAfter: 29-3-2033 14:29 Subject: E=support@cacert.org, CN=CA Cert Signing Authority, OU=http://www.cacert.org, O=Root CA Serial: 00 13 5c ec 36 f4 9c b8 e9 3b 1a b2 70 cd 80 88 46 76 ce 8f 33 Element.dwInfoStatus = CERT_TRUST_HAS_EXACT_MATCH_ISSUER (0x1) Element.dwInfoStatus = CERT_TRUST_IS_SELF_SIGNED (0x8) Element.dwInfoStatus = CERT_TRUST_HAS_PREFERRED_ISSUER (0x100) ---------------- Certificaat AIA ---------------- Geen URL's "Geen" Tijd: 0 ---------------- Certificaat CDP ---------------- Gecontroleerd "Basislijst met ingetrokken certificaten" Tijd: 5 [0.0] https://www.cacert.org/revoke.crl ---------------- Certificaat-OCSP ---------------- Geen URL's "Geen" Tijd: 0 -------------------------------- Exclude leaf cert: 96 aa e8 9d 5c cf b0 0c 60 7e 3c b9 f6 25 de ff 3d 86 1b 66 Full chain: ee 9e fa 78 60 a6 73 74 8d 97 c1 a9 11 35 0c 45 64 7e d1 e8 Issuer: CN=CAcert Class 3 Root, OU=http://www.CAcert.org, O=CAcert Inc. NotBefore: 6-11-2012 16:09 NotAfter: 6-11-2014 16:09 Subject: CN=bocanium.soleus.nu Serial: 010c5c SubjectAltName: DNS-naam=bocanium.soleus.nu, Andere naam:1.3.6.1.5.5.7.8.5=0c 12 62 6f 63 61 6e 69 75 6d 2e 73 6f 6c 65 75 73 2e 6e 75 de 55 08 57 34 ba 81 24 56 af dd 94 e7 eb 1c 75 fe 26 50 ca De intrekkingsfunctie kan het intrekken niet controleren omdat de intrekkingsserver offline is. 0x80092013 (-2146885613) ------------------------------------ Intrekkingscontrole is overgeslagen: de server is offline Het certificaat is een eindentiteitscertificaat Intrekkingscontrole van certificaat voltooid CertUtil: - de opdracht verify is voltooid. | ||||
Reviewed by | |||||
Test Instructions | |||||
|
At test.cacert.org is a first workaround available und /home/GuKKDevel/bug-1306/EliminateExpired.pl. Since the CRL is built from the Database-file index.txt in the directory named in the configfile, above module reads this file and writes them either to the file for eliminated records or to the next index.txt-file, depending on date of revokation and expiration. both are to be younger than 62 days (2 months) in the past. At this stage after that the files index.txt and index.temp.new have to be renamed manually. |
|
There is a retention time of three months after the last certificate expired/was revoked before an account can be closed for support. I suggest the same duration for CRL. |
|
aggreed so lets make it 100 days |
|
I did a fix. appended are two version to choose. EliminateExpired.pl (4,694 bytes)
#!/usr/bin/perl -w # # module EliminateExpired.pl # # parameter: cert for CRL class1 or class3 # # Eliminates certificate-records from file index.txt for CRL, which # are revoked at least for 100 days and their expirationdate also older than 100 days # # if a certificate is revoked, it remains at index.txt # max{expirationDate,revokationDate} + 100 days # # this has to be proofed against the parameters of the openssl - config files # for next update of the CRL # # also $fileNameStart has to be checked agains the config files # /etc/ssl/CA for class1 # /etc/ssl/class3 for class3 # use strict; use warnings; use POSIX; use open qw< :encoding(UTF-8) >; # # initializing # my $debuglvl = 0; my $lz = "\n"; # Counters my $countInput = 0; my $countOutput = 0; my $countEliminated = 0; # # Date handling # my $actualDate = time(); my $actualDateS = strftime( "%Y-%m-%d", gmtime($actualDate) ); my $eliminationDateLimit = $actualDate - 100 * 24 * 60 * 60; #subtract 100 days from current time my $eliminationDateLimitS = strftime( "%Y-%m-%d", gmtime($eliminationDateLimit) ); my $eliminateDate = substr( $eliminationDateLimitS, 2, 2 ) . substr( $eliminationDateLimitS, 5, 2 ) . substr( $eliminationDateLimitS, 8, 2 ); # #Logging functions: # my $lastdate = ""; sub SysLog($) { return if ( not defined( $_[0] ) ); my $timestamp = strftime( "%Y-%m-%d %H:%M:%S", gmtime ); my $currdate = substr( $timestamp, 0, 10 ); if ( $lastdate ne $currdate ) { close LOG if ( $lastdate ne "" ); $lastdate = $currdate; open LOG, ">logfile$lastdate.txt"; } print LOG "$timestamp $_[0]"; flush LOG; } # # Handling filenames and parameters # my $filenameInput = undef; my $filenameOutput = undef; my $filenameEliminated = undef; my $filenameSave = undef; sub SelectFilenames { my $cert = ""; my $fileNameStart = undef; if ( not defined $ARGV[0] ) { $cert = "class3"; } else { $cert = $ARGV[0]; } if ( $cert eq "class1" ) { $fileNameStart = "/etc/ssl/CA/"; } else { $fileNameStart = "/etc/ssl/class3/"; } $filenameInput = $fileNameStart . "index.txt"; $filenameOutput = $fileNameStart . "index.temp.new"; $filenameEliminated = $fileNameStart . "index.elim.$actualDateS"; SysLog( "File to read from -> $filenameInput" . $lz ); SysLog( "File to write kept lines -> $filenameOutput" . $lz ); SysLog( "File to save the removed lines -> $filenameEliminated" . $lz ); if ( $debuglvl > 8 ) { $fileNameStart = "/home/GuKKDevel/bug-1306/tmp/"; $filenameInput = $fileNameStart . "index.txt"; print "File to read from -> $filenameInput" . $lz; $filenameOutput = $fileNameStart . "index.temp.new"; print "File to write kept lines -> $filenameOutput" . $lz; $filenameEliminated = $fileNameStart . "index.elim.$actualDateS"; print "File to save the removed lines -> $filenameEliminated" . $lz; SysLog( "DEBUGGING:File to read from -> $filenameInput" . $lz ); SysLog( "DEBUGGING:File to write kept lines -> $filenameOutput" . $lz ); SysLog( "DEBUGGING:File to save the removed lines -> $filenameEliminated" . $lz ); } } # # open the neccessary files # SysLog( "Start of program\n"); SysLog( "Removing all revoked certificates expired and revoked before: $eliminationDateLimitS\n"); SysLog( "Allocating files\n"); SelectFilenames(); my $fileInput = undef; open( $fileInput, "<", "$filenameInput" ) || die "$0: can't open $filenameInput for reading: $!"; my $fileOutput = undef; open( $fileOutput, ">", "$filenameOutput" ) || die "$0: can't open $filenameOutput for reading: $!"; my $fileEliminated = undef; open( $fileEliminated, ">", "$filenameEliminated" ) || die "$0: can't open $filenameEliminated for reading: $!"; SysLog( "Start reading\n"); while (<$fileInput>) { my $record = $_; my @field = split /\s+/, $record; my $flag = $field[0]; $countInput++; if ( $flag ne "R" ) { # certificate is unrevoked print $fileOutput $record; $countOutput++; } else { # certificate is revoked my $expirationDate = $field[1]; my $revokationDate = $field[2]; if ( $expirationDate lt $eliminateDate and $revokationDate lt $eliminateDate) { print $fileEliminated $record; $countEliminated++; } else { print $fileOutput $record; $countOutput++; } } } if ($!) { die "unexpected error while reading from $fileInput: $!"; } SysLog( "End reading" . $lz ); SysLog( "Closing files" . $lz ); close $fileInput; close $fileOutput; close $fileEliminated; SysLog( "STATISTICS" . $lz ); SysLog( "Read lines: " . $countInput . $lz ); SysLog( "Kept lines: " . $countOutput . $lz ); SysLog( "Eliminated: " . $countEliminated . $lz ); SysLog( "End of program"); EliminateExpired.V2.pl (7,889 bytes)
#!/usr/bin/perl -w # # module EliminateExpired.pl # # parameter: cert for CRL class1 or class3 # # Eliminates certificate-records from file index.txt for CRL, which # are revoked at least for 100 days and their expirationdate also older than 100 days # puts the resulting file of kept records in place of the original index.txt # # if a certificate is revoked, it remains at index.txt # max{expirationDate,revokationDate} + 100 days # # this has to be proofed against the parameters of the openssl - config files # for next update of the CRL # # also $fileNameStart has to be checked agains the config files # /etc/ssl/CA for class1 # /etc/ssl/class3 for class3 # use strict; use warnings; use POSIX; use open qw< :encoding(UTF-8) >; # # initializing # my $debuglvl = 0; my $cert = ""; # Counters my $countInput = 0; my $countOutput = 0; my $countRevokedUnexpired = 0; my $countRevokedExpired = 0; my $countUnrevokedUnexpired = 0; my $countUnrevokedExpired = 0; my $countEliminated = 0; # Filenames my $filenameInput = undef; my $filenameOutput = undef; my $filenameEliminated = undef; my $filenameSave = undef; my $filenameTemp = undef; my $fileNameStart = undef; # # Date handling # my $actualDate = time(); my $actualDateS = strftime( "%Y-%m-%d", gmtime($actualDate) ); my $eliminationDateLimit = $actualDate - 100 * 24 * 60 * 60; #subtract 100 days from current time my $eliminationDateLimitS = strftime( "%Y-%m-%d", gmtime($eliminationDateLimit) ); my $eliminateDate = substr( $eliminationDateLimitS, 2, 2 ) . substr( $eliminationDateLimitS, 5, 2 ) . substr( $eliminationDateLimitS, 8, 2 ); # #Logging functions: # my $lastdate = ""; sub SysLog($) { return if ( not defined( $_[0] ) ); my $timestamp = strftime( "%Y-%m-%d %H:%M:%S", gmtime ); my $currdate = substr( $timestamp, 0, 10 ); if ( $lastdate ne $currdate ) { close LOG if ( $lastdate ne "" ); $lastdate = $currdate; open LOG, ">>logfile$lastdate.txt"; } print LOG "$timestamp $_[0]"; print "$timestamp $_[0]" if ( $debuglvl > 0 ); flush LOG; } # # Handling filenames and parameters # # getParameters allowed: class1, class3, test # getFilename returns te next valid name of a specific file # selectFilenames depending on given arguments sub getParameters () { if ( not defined $ARGV[0] ) { $cert = "test"; } else { $cert = $ARGV[0]; } if ( ( $cert ne "test" ) and ( $cert ne "class1" ) and ( $cert ne "class3" ) ) { dieOut( "no valid parameter $cert allowed are 'class1', 'class3' and 'test'\n" ); } } sub getFilename($$) { my $name = $_[0]; my $qualif = $_[1]; my $namepart = $name . $qualif; if ( -f $fileNameStart . $namepart . $actualDateS ) { while ( -f $fileNameStart . $namepart . $actualDateS ) { $namepart .= $qualif; } } return $namepart . $actualDateS; } sub SelectFilenames { if ( $cert eq "class1" ) { $fileNameStart = "/etc/ssl/CA/"; } else { if ( $cert eq "class3" ) { $fileNameStart = "/etc/ssl/class3/"; } else { $fileNameStart = "/home/GuKKDevel/bug-1306/tmp/"; } } $filenameInput = $fileNameStart . "index.txt"; $filenameOutput = $fileNameStart . getFilename( "index.", "kept." ); $filenameEliminated = $fileNameStart . getFilename( "index.", "elim." ); $filenameSave = $fileNameStart . getFilename( "index.", "save." ); $filenameTemp = $fileNameStart . "index.temp"; SysLog("File to read from -> $filenameInput\n"); SysLog("File to write kept lines -> $filenameOutput\n"); SysLog("File to save the removed lines -> $filenameEliminated\n"); } # # Pre-/Post-processing # # StartStopDemon stopping and starting the signer-demon # dieOut Start the signer-demon before die # preProcessing stopping the signer-demon # postProcessing renaming the kept-file as new index.txt-file my $restartDemon = 0; sub startStopDemon($) { if ( $_[0] eq "Stop" ) { # Stop demon SysLog("Stopping signer-demon\n"); system("service commmodule-signer stop"); if ($!) { die("$0: Couldn't stop signer"); } SysLog("Signer-demon stopped"); $restartDemon = 1; } else { # Start demon SysLog("Restarting signer-demon\n"); system("service commmodule-signer start"); if ($!) { die("$0: Couldn't start signer"); } SysLog("Signer-demon started"); } } sub dieOut ($) { SysLog( $_[0] ); if ($restartDemon) { startStopDemon("Start"); } die( $_[0] ); } sub preProcessing () { SysLog("\n"); SysLog("Start Pre-prossessing\n"); startStopDemon("Stop"); SysLog("End Pre-processing\n"); SysLog("\n"); } sub postProcessing () { SysLog("\n"); SysLog("Start Post-processing\n"); SysLog("Copying $filenameOutput to \n"); SysLog("$filenameTemp\n"); system("cp $filenameOutput $filenameTemp"); if ($!) { dieOut("$0: can' t copy $filenameOutput to $filenameTemp: $! \n "); } SysLog(" Copying done \n "); SysLog(" Copying $filenameInput to \n "); SysLog("$filenameSave \n "); system(" cp $filenameInput $filenameSave "); if ($!) { dieOut("$0 : can't copy $filenameInput to $filenameSave \n "); } SysLog(" Copying done \n "); SysLog(" Renaming $filenameTemp to \n "); SysLog("$filenameInput \n "); # now the index.txt file will be replaced # whilst Error server-demon MUST BE NOT RESTARTED $restartDemon = 0; system(" mv $filenameTemp $filenameInput "); if ($!) { dieOut("$0 : can't replace $filenameInput by $filenameTemp \n "); } SysLog(" Renaming done \n "); # restart demon $restartDemon = 1; startStopDemon(" Start "); SysLog(" End Post-processing \n "); SysLog(" \n "); } # # start of the main structure # # open the neccessary files # SysLog(" Start of program \n "); SysLog(" Reading the parameters \n "); getParameters(); SysLog( " Remove all revoked certificates, issued by $cert -certificate, expired and revoked before : $eliminationDateLimitS \n " ); preProcessing(); SysLog(" Allocating files \n "); SelectFilenames(); my $fileInput = undef; open( $fileInput, " < ", "$filenameInput " ) || dieOut("$0 : can't open $filenameInput for reading : $! \n "); my $fileOutput = undef; open( $fileOutput, " > ", "$filenameOutput " ) || dieOut("$0 : can't open $filenameOutput for writing : $! \n "); my $fileEliminated = undef; open( $fileEliminated, " > ", "$filenameEliminated " ) || dieOut("$0 : can't open $filenameEliminated for writing : $! \n "); SysLog(" Start reading \n "); while (<$fileInput>) { my $record = $_; my @field = split /\s+/, $record; my $flag = $field[0]; my $expirationDate = $field[1]; $countInput++; if ( $flag ne " R " ) { # certificate is unrevoked print $fileOutput $record; $countOutput++; if ( $expirationDate lt $eliminateDate ) { $countUnrevokedExpired++; } else { $countUnrevokedUnexpired++; } } else { # certificate is revoked my $revokationDate = $field[2]; if ( $expirationDate lt $eliminateDate and $revokationDate lt $eliminateDate ) { print $fileEliminated $record; $countEliminated++; $countRevokedExpired++; } else { print $fileOutput $record; $countOutput++; $countRevokedUnexpired++; } } } if ($!) { dieOut(" unexpected error while reading from $fileInput: $! "); } SysLog(" End reading \n "); SysLog(" Closing files \n "); close $fileInput; close $fileOutput; close $fileEliminated; postProcessing(); SysLog(" \n "); SysLog(" STATISTICS \n "); SysLog(" Read lines : $countInput \n "); SysLog(" Kept lines : $countOutput \n "); SysLog(" unrevoked and unexpired : $countUnrevokedUnexpired \n "); SysLog(" unrevoked but expired : $countUnrevokedExpired \n "); SysLog(" revoked but unexpired : $countRevokedUnexpired \n "); SysLog(" Eliminated : $countEliminated \n "); SysLog(" revoked and expired : $countRevokedExpired \n "); SysLog(" End of program \n "); |
|
Current signer configuration can be found at https://svn.cacert.org/CAcert/SystemAdministration/signer/ |
|
Created branch bug-1306 (on github) and merged the pull request of @GuKKDevel into it |
Date Modified | Username | Field | Change |
---|---|---|---|
2014-09-15 14:25 | wytze | New Issue | |
2014-09-15 14:25 | wytze | File Added: crl-size-issue.log | |
2018-05-01 12:42 | egal | Assigned To | => GuKKDevel |
2018-06-06 10:34 | GuKKDevel | Status | new => fix available |
2018-06-06 10:34 | GuKKDevel | Note Added: 0005594 | |
2018-06-06 10:40 | egal | Note Added: 0005595 | |
2018-06-06 10:57 | GuKKDevel | Note Added: 0005597 | |
2018-06-06 10:58 | GuKKDevel | Note View State: 0005594: public | |
2018-06-06 10:58 | GuKKDevel | Note View State: 0005597: public | |
2018-11-01 13:03 | GuKKDevel | File Added: EliminateExpired.pl | |
2018-11-01 13:03 | GuKKDevel | File Added: EliminateExpired.V2.pl | |
2018-11-01 13:03 | GuKKDevel | Note Added: 0005634 | |
2019-04-05 21:33 | Ted | Note Added: 0005791 | |
2021-08-07 19:24 | Ted | Note Added: 0006064 |