reconnaissance

SecurityTrails Blog · Oct 25 · SecurityTrails team

What Are Certificate Transparency Logs?

SecurityTrails has introduced to you our new Feeds page, SurfaceBrowser™, and the option of getting certificate transparency logs, all of them in unified format so you’re able to extract any information needed. And while Google has announced they’ll be requiring Certificate Transparency for all newly-issued publicly trusted certificates, highlighting transparency and accountability through Certificate Authorities, there are still people who don’t get the importance of Certificate Transparency logs and the value of the information within them.

We’ve covered the massive adoption of SSL certificates and the risks behind it, and today we’ll tackle the subject of Certificate Transparency and what it means. To make our new Feeds easier to use, especially the CTLogs feature, let’s start with the basics, the benefits and how you can use the information to monitor your domains.

Certificate Transparency

SSL certificates, files containing cryptographic keys and elementary to all HTTPS connections, have gained so much popularity that there are almost no websites or blogs left on the Internet without them. But, the structural flaws of this system incur many risks.

In 2011, Dutch certificate authority (CA) DigiNotar issued a fraudulent certificate for google.com and all of its subdomains to Iranian attackers who used the certificate for man-in-the-middle attacks in Iran. The certificate was revoked, but because the CA did not come clean about how or why the certificate was issued in the first place, Google and Mozilla released patches to blacklist the entire CA.

Since spotting and revoking fraudulent certificates takes a substantial amount of time, the monitoring and auditing of unauthorized certificate issuing has been made accessible and easy with Certificate Transparency.

The main goal of Certificate Transparency is to provide a publicly available system of logs, where any domain owner can verify whether a certificate was issued by a trusted CA or issued maliciously, and to prevent users from being tricked by any fraudulent certificates.

How Certificate Transparency works

Certificate Transparency is an open framework that monitors and audits TLS/SSL certificates. It’s made up of three important components:

Certificate Transparency logs

Every Certificate Transparency log is a record of all publicly trusted digital certificates. Those certificates contain information about the public key, the subject and information about the issuer.

Certificate Transparency logs keep cryptographically secured records of certificates which are append-only, meaning that certificates can only be added to the log — it’s impossible to delete, modify or in any way retroactively change or insert certificates into the log.

”Cryptographically secured” means that those logs append new certificates to the cryptographic mechanism Merkle hash tree, allowing for systematic and secure verification.

Certificate monitors

Certificate monitors check logs to see if they are behaving correctly and if there are any in consistencies to indicate a log is not behaving like it should. They are publicly run servers that periodically check all log servers and monitor the certificates they contain. Monitors primarily make sure that all certificates are visible in the log, and they also check to see if they have strange extensions and even if they have Certificate Authority capabilities. Many third-party services have launched their own Certificate Transparency monitoring tools, such as Facebook did with their free online Certificate Transparency Monitoring Developer Tool.

Certificate auditors

Using information they fetch about a log, certificate auditors verify it against other information to audit the integrity of the logs and verify that they are cryptographically consistent. Certificate transparency auditors are software components which can verify if a certificate is visible in a log, and once again, this is done by periodically verifying log proofs. A log that isn’t registered or visible is a sign of suspicious behaviour, and connections to sites with those certificates may be refused. All logs need to provide their log proofs.

Auditing can be a secondary function of a monitor or a separate service, although it’s mostly certificate authorities who run the auditors as they are a way to verify the integrity of that certificate authority.

Log proofs

As we mentioned, Certificate Transparency logs use the cryptographic mechanism known as the Merkle hash tree, a binary tree that is the root hash from which all nodes and leaves originate. It’s made of hashed leaves and nodes that are hashes of certificates approved for the log. Non-leaf nodes are labelled with hashes of their child nodes.

The Merkle hash tree provides logs with the ability to verify if all certificates are consistently appended to them. Verification is done with two cryptographic proofs — consistency and audit proof.

Those proofs help auditors gain cryptographic information to verify logs and their consistency. As we mentioned, all logs are required to provide their proofs.

How Certificate Transparency logs work

Most certificates are submitted by Certificate Authorities, but really, anyone can submit a certificate to a log. When you submit a valid certificate, the log will respond to you with an SCT - Signed Certificate Timestamp. During the TLS handshake, the TLS server delivers the SCT with the certificate. SCTs are basically proof that certificates were added to the log. SCTs must be made available to web browsers so they can check the certificate.

Three methods of delivering the SCT to the browser exist:

1. Certificate extension — X.509v3 extension

This is the most common method for delivering SCTs. Certificate Authorities include an SCT into the certificate by using an X.509v3 extension. When a certificate is submitted to the log, the log returns the SCT. After that, certificate authorities attach the SCT as a certificate extension — X.509v3 extension.

2. TLS extension

With this method, the website operator provides an SCT to the browser by using a TLS extension in the TLS protocol. Those website operators submit certificates to logs and get the SCTs. The TLS extension is included in the TLS handshake, and it’s called “signed_certificate_timestamp”.

3. OCSP stapling

SCTs are also delivered by using the OCSP protocol — Online Certificate Status Protocol. Here, the Certificate Authority issues a certificate to both the webserver and the website operator. The operator needs to support the OCSP stapling and makes a query to the Certificate Authority, and the CA responds to it with the SCT. If the operator is not configured to support OCSP stapling, the Certificate Authority needs to configure their OCSP service to support SCT stapling.

Benefits of Certificate Transparency logs

Certificate Transparency and CTLogs offer many benefits that stretch far beyond previous versions of monitoring certificates.

A major benefit for domain owners is that they’ll be able to monitor logs and see what certificates are issued for their domains. It provides domain security by monitoring for fraudulent certificates.

For researchers (and basically anyone), CTLogs provide the ability to evaluate certificates and determine whether they are truly compliant for certificate obligations.

Previously when monitoring and auditing certificates, a fraudulent certificate would take some time to be revoked. With CT and CTLogs, this can be completed in just a few hours, and any domain owner possibly affected is notified immediately.

Summary

Making HTTPS connections even more secure and reliable for fending off attacks and interception needed a new framework to help monitor all certificates issued. Fortunately, Certificate Transparency came forth as an open-framework solution to detect and blacklist forged or malicious certificates. Monitoring the certificate system and auditing certificates for validity is now publicly available for all domain owners with the common goal of a safer Internet environment.


Use SecurityTrails’ SurfaceBrowser or Feeds to get the scoop on information found in CTLogs, and all in unified format, ready for download and information extraction! Sign up now for your new API and try out these features — designed to help you understand data security better than ever.