Most people don’t have to deal with TLS directly—beyond noticing that the browser marks the website they are visiting as being secure. For those who have to think about how to protect the traffic moving around the Internet, Facebook has open sourced Fizz, a “performant” TLS library it used internally to deploy TLS 1.3.
HTTPS, the encrypted connection that servers use when talking to other servers, is made possible by Transport Layer Security, the successor to SSL. Over the past four years, the Internet Engineering Task Force (IETF) has been working on updating TLS—the 28th version of the TLS 1.3 proposal was approved earlier this year—to support stronger encryption schemes and drop legacy encryption algorithms. TLS 1.3, which hasn’t been officially published (as RFC 8446) yet, is also designed to be more efficient, which is important considering the nature of modern networking means servers are regularly communicating with servers that are geographically very far away. Latency is a given as traffic bounce back and forth between servers, and the speed gets reduced further over a secure connection.
TLS 1.3 “incorporates several new features that make internet traffic more secure, including encrypting handshake messages to keep certificates private, redesigning the way secret keys are derived, and a zero round-trip connection setup, which makes certain requests faster than TLS 1.2,” Facebook engineers Kyle Nekritz, Subodh Iyengar, and Alex Guzman wrote in the Fizz announcement.
There are other TLS libraries, such as Google’s BoringSSL and the widely used OpenSSL. Fizz is different because TLS 1.3 features are implemented in a way to reduce latency and resource overhead, making it a “robust, highly performant TLS library,” Nektriz, Iyengar, and Guzman said.
"Fizz has reduced not only the latency but also the CPU utilization of services that perform trillions of requests a day," said Nekritz, Iyengar, and Guzman. They noted that Facebook's load balancer synthetic benchmarks exhibit roughly 10 percent better throughput with Fizz than with the company's previous stack.
Fizz Implementation
Facebook’s engineers highlighted four areas Fizz reduced latency and resource overhead: memory handling, asynchronous processing, 0-RTT, and early data.
Most applications tend to keep data in several chunks in different memory locations, but many TLS libraries require contiguous memory space to encrypt the block. In those cases, the applications first have to copy data into a new memory location before calling the library. Fizz avoids this latency overhead because it utilizes scatter/gather IO APIs, to encrypt chunked data in place. There is no need to copy data so applications using Fizz perform fewer memory allocations and copies.
Since servers routinely make calls to services in other locations in the middle of negotiating a handshake with clients--TLS load balancers can be in a different location from the server with the certificate signing keys--asynchronous IO is very important. Fizz natively supports asynchronous server operations so it doesn’t block the service from processing other handshakes while waiting for response from other services.
Zero round trip time resumption data, 0-RTT data, is a latency-reducing feature that allows “the client to send data to a server in the first round trip of a connection, without waiting for the TLS handshake to complete if the client has spoken to the same server recently.” The server can do some computation in parallel with the rest of the handshake, and send reply in its very first packet. Applications like Facebook and Google Search benefit tremendously from this feature. Previous efforts used Zero Protocol to establish 0-RTT secure connections, but Facebook engineers found that Fizz “delivers the reliability and performance of TLS 1.3 on par with Zero Protocol.”
Another way to reduce latency is to send early data immediately after the TCP connection is established, such as during mobile app cold start. Early data isn’t always a good idea in TLS 1.3 because an attacker can “replay” the request and cause the server to process it twice. Fizz mitigates the risk of a replay attack by sending only specific requests--a whitelist of allowed requests--as early data and deploying a cache with its load balancers to detect reply attempts.
Secure Code
Fizz has extra security baked in--an important consideration for a library that is going to be used to secure online communications. Fizz is written in C++ 14, and takes advantage of the C++ type system with variadic templates and variant types in C++, which provides a “high level of safety by preventing accidental bugs,” Facebook said.
The TLS state machine in Fizz is defined explicitly in a single location to avoid attacks that attempt to change the code’s state, such as the CCS (ChangeCipherSpec) Injection Vulnerability in OpenSSL. If a state handler uses an incorrect state transition that is not defined in the explicit state machine, the code will not compile.
“This helps us catch bugs during compile time rather than at runtime, thereby preventing mistakes,” the engineers wrote.
Why TLS 1.3?
TLS 1.3 getting approved is a significant milestone, but it doesn’t mean that widespread deployment is going to happen anytime soon. It takes a while for this kind of rollout to happen because it requires changes in the network architecture. Servers and clients need to support TLS 1.3, but more importantly, “middleboxes,” or networking appliances that inspect network traffic, also need to support TLS 1.3.
Earlier versions of TLS 1.3 caused a lot of deployment headaches. For example, thousands of Chromebooks were no longer able to connect to the Internet after being updated to Chrome OS 56. The culprit was the middleboxes—in this case Symantec’s BlueCoat appliances—which had been configured to use TLS 1.2 and was dropping TLS 1.3 connections completely instead of downgrading the connection to TLS 1.2.
For many organizations, the question becomes “Is this going to break things? What will break?” said Tony Lauro, a manager for Akamai’s Enterprise Security Architecture team. For many, the choice is between the risks of attackers exploiting TLS 1.2 and users not being able to be online. The decision may very well boil down to strengthening security controls to mitigate the risks of keeping TLS 1.2.
Financial services organizations and others that rely on middleboxes to inspect network traffic for malicious activity need to make sure their vendors have added TLS 1.3 support or risk seeing a lot of things break in their network.
Facebook said its early work in Fizz helped identify these kind of issues with middleboxes, and that Fizz has some workarounds to handle these kinds of situations. In one case, Fizz ensures "the ClientHello is sent in its own TCP packet and that Fizz will fall back to a full 1-RTT handshake if a 0-RTT handshake repeatedly fails,” the engineers wrote.
Even so, most organizations will likely treat TLS 1.3 as a gradual rollout and not an all-or-nothing proposition, Lauro said. Operations that won’t break while using TLS 1.3 would likely be switched over, first. Segmented networks and specialized processes between specific machines would be able to switch to TLS 1.3 sooner. Eventually, there will be a point where the majority of the systems would be using TLS 1.3--and that is the time to discuss turning off 1.2 altogether.
“Rolling out [TLS] 1.2 took a decade. The world only really did it because of a crisis in [TLS] 1.0,” said Brian Sniffen, Akamai’s chief architect, information security.
Facebook is currently running more than 50 percent of its traffic through TLS 1.3 and Fizz, which Nekritz, Iyengar, and Guzman said called the largest implementation of TLS 1.3 to date.
Most server operators can treat TLS 1.3 as a drop-in replacement for TLS 1.2, as it uses the same keys and certificates. The server and client automatically negotiate to using TLS 1.3 if they both support it. Chrome and Firefox already have TLS 1.3 on by default, for example. However, if the server operator had specified which cipher suites to use, then that will have to be reviewed. For example, if the server is configured to accept only the “RSA” key-exchange mechanism and nothing from “ECDHE,” that would be a problem because with TLS 1.3, all key exchange is ECDHE, Sniffen said.
Facebook released Fizz in hopes that the library would “help speed up deployment of TLS 1.3 across the internet and help others make their apps and services faster and more secure,” the engineers wrote.