HTTP/2 and You: When hypertext gets too/2hyper
Introduction
If you are reading this, CVE-2023-44487 aka HTTP/2 Rapid Reset is upon us. New DDoS records have been set to the tune of 7.5 times the previous all-time high. Some admins and enterprises will be in scramble mode. The culprit: a protocol-level oversight in HTTP/2 (HTTP Version 2) in the handling of stream closure that allows an attacker the opportunity to request and immediately close streams between client and server at an arbitrary rate. When executed en masse, the result is an amplification attack that leverages the underlying speed advantage of HTTP/2 and its introduction and allowance of multiple streams over a single TCP connection.
How did we get here?
In working our way into current events, we must first understand the basic advantages of HTTP/2 that led to its broad adoption. Fundamentally, these advantages come down to efficiencies needed to improve responsiveness and availability in dynamic web applications, microservices, and much of the rich content served to us as part of the modern internet experience. Despite some additional complexity, HTTP/2 allowed a shift from the management of TCP connection pools to what are essentially many sub connections (connection multiplexing) over a single TCP connection. The result is a much more resource savvy deployment from the networking perspective especially when factoring in many users as streams were asynchronously managed under a single pre-established TCP session.
Additionally, HTTP/2 brought with it some security advantages via Transport Layer Security (TLS) stipulated by HTTP/2's Request for Comments (RFC) and loosely enforced by many HTTP/2 browser requirements. The move towards TLS thus implied additional consideration and overhead regarding Public Key Infrastructure (PKI), comprising the brunt of HTTP/2's complexity. In some ways, HTTP/2 was catalytic in this way and perhaps leading in front of impending compliance directives calling for increased PKI requirements since HTTP/2's RFC publication in 2015. Though HTTP/2 may deserve some applause here, the focus on security up front may have distracted us from some basic implementation considerations that ultimately left us with a vulnerability in plain sight.
So, what about the bad part?
The basic implementation malady effecting HTTP/2 is one of handling the streams within the TCP connection. All streams are independent and stream handling is configurable to allow for efficient resource allocation and availability. In the case of the HTTP/2 Rapid Reset, the main vulnerability focuses on the relationship between HEADERS frames to the server, the SETTINGS_MAX_CONCURRENT_STREAMS, and the way that backlog requests are handled when an RST_STREAM frame is issued to cancel the streams. Essentially, concurrent streams are counted in terms of either open (actively transacting) or half-closed (RST) whereby available streams are in an idle state. Ordinarily, this is not an issue under ideal conditions, however, any lag between the request and closure process is punished by adding stream requests in excess of the cap indicated by the SETTINGS_MAX_CONCURRENT_STREAMS configuration. An attacker can upset this balance by issuing packets containing many stream requests then issuing a RST_STREAM frame then repeating the process to overfill the stream limit leaving the impression that there are no idle streams available, i.e. Denial-of-Service. To see this in action, please refer to the Cloudflare blog here that contains both a detailed breakdown of the process as well as a PCAP so that folks may follow along.
Who is affected?
As with most denial-of-service attacks, resources constrained entities are first the come time mind in terms of worry. Large enterprises and cloud providers were able to get ahead of these attacks by simply deploying more resources at the edge to protect against any service interruptions. This is simply not viable for many enterprises.
What's more, one of the more common reagents that added to the efficacy of the HTTP/2 Rapid Reset attack is the simply the use of intermediary technology such as proxies and load balancers. These intermediary technologies add overhead which often creates the perfect level of lag for HTTP/2 Rapid Reset attack to exact its effect. Since enterprises of all sizes leverage these technologies, from government to finance to commerce, there is a broad range affected parties.
How can I prevent this from happening?
The best means of prevention against HTTP/2 Rapid Response are wide ranging. Worried parties will be best served by auditing their web-facing deployments and take stock of any load balancing, proxy services, and/or general use of HTTP/2. Vendors should be noted, and any mitigation guidance or updates should be implemented as soon as possible. Additionally, enterprises with aging architecture may consider transformative measures such as moving web applications behind larger DDoS protection providers and CDNs such as AWS, Cloudflare, Google Cloud, and/or Microsoft Azure. These providers have had a substantial amount of lead time before the public disclosure of CVE-2023-44487 and have already implemented mitigations for HTTP/2 Rapid Response attacks. Enterprises may also attempt to implement rate limits on their current infrastructure; however, mileage may vary based on the vendor and cooperation between technologies.
Another major proactive measure that enterprises can leverage is to monitor incoming web application performance and telemetry at proxies and load balancers. Suspected malicious IP addresses can also be blocked once detected. It is important to note that monitoring will become increasingly important since the vulnerability is at the protocol level and no universal fix exists so long as HTTP/2 is implemented.
The final mitigating measure might be to begin your organization's move toward the embrace of HTTP/3. HTTP/3 remains relatively new and still has some lagging adoption and partial support by a few major browsers. With HTTP/3 most functionality is delegated to the underlying QUIC protocol which features quite a few more protections against volumetric and amplification attacks. QUIC also expands upon the use of streams, leveraging an expression of UDP to offer many additional performance benefits over TCP-based HTTP. This will be a strategic initiative for most enterprises, though we expect HTTP/2 Rapid Response to expedite adoption pressures and support. Early adopters might be comprised of government or other restrictive entities that can strictly enforce browser and server configurations as well as those that have hosted web applications expressly for business processes accessed by enterprise-controlled machines.
How can CyberPoint help?
The CyberPoint Security Research Team (SRT) maintains a portfolio of services that can help you be resilient against cyber-attacks like HTTP/2 Rapid Reset. From scanning and testing your infrastructure using our offensive services team to sitting in the seat next to you as we collaboratively Purple Team our way through your digital transformation, we are here to realize your secure enterprise journey. Additionally, keep an eye on our blog for additional context on current and emerging cyber topics.
About CyberPoint's Security Research Team (SRT)
CyberPoint's Security Research Team is a public-facing research unit that works on relevant challenges within information security (infosec), we specialize in vulnerability research, malware analysis, threat intelligence, and technical evaluation. Our team's been doing this kind of work for years. We're passionate about it on both a professional and a personal level, and we're excited to do research that helps our customers and the larger community. The mission of CyberPoint's Security Research Team is to improve the security knowledge and capabilities of our community, our customers, our products, and our programs.
Contact SRT
For our customers, we support incident response with repository querying and analysis. We also respond to customer inquiries about a wide array of infosec topics.
If your organization needs immediate assistance with a suspected or confirmed cyber incident, contact us.