Can malware be spotted in TLS without having to decrypt the traffic?

Malware may be hiding in TLS traffic.Decryption of traffic encrypted by Transport Layer Security is still a controversial topic. TLS, which has succeeded the deprecated Secure Sockets Layer as the acceptable standard technology for HTTPS and encrypted email, is crucial for ensuring that messages are not subject to prying eyes. In recent years, its importance has only increased in the wake of revelations about the extent of government surveillance worldwide, as well as the emergence of game-changing exploits such as Heartbleed. The latter made it crucial for everyone from hosting providers to e-commerce merchants to move to at least TLS 1.1 (but preferably TLS 1.2).

Encryption as a disguise for malware: Assessing the scope of the issue

At the same time, encryption has become something of a double-edged sword for many enterprises. Even if no one would imagine abandoning HTTPS altogether, its ability to hide malware inside encrypted traffic has become a growing issue. TLS is layered over everyday protocols such as HTTP (web) and SMTP (email), and its libraries can also be implemented within malware communications. According to research from Cisco, the percentage of malware using TLS rose from less than 1 percent in July 2015 to around 12 percent by the end of the year (the peak was at over 14 percent that November).

This trend is a challenge for enterprise security teams, and one that is likely to become more prominent in the years ahead:

  • In February 2016, 77 percent of requests sent to Google servers were sent via HTTPS; that was up from only 52 percent at the end of 2013.
  • Dell had similar findings, reporting that nearly two-thirds (65 percent) of traffic it observed in 2015 was encrypted, according to Dark Reading.
  • IT research firm Gartner has projected that by 2017, half of all networks will take advantage of SSL/TLS encryption.
  • A Vanson Bourne survey of 500 CIOs found that 90 percent of them had experienced, or were bracing themselves for, a SSL/TLS attack.
  • Many security architects remain wary of the theft of digital certificates and encryption keys, which are implicitly trusted by mostly servers and applications.

Encrypted SSL/TLS traffic is essentially a blind spot for many security devices, since they cannot tell what is going on behind the cipher. HTTPS accordingly has the effect of potentially masking malicious activity, as was demonstrated with the Backoff point-of-sale malware that exfiltrated data via encrypted POST requests. Even so, SSL/TLS, SSH, code signing keys and certificates are becoming more common mechanisms for authenticating websites and ensuring secure VPN connections.

“Because traditional security devices are unable to decrypt and inspect this content, [viruses]/malware and other threats embedded in HTTPS traffic can pass unobstructed through your security defenses and on to your enterprise network,” explained a piece of Trend Micro documentation about the InterScan Web Security Virtual Appliance.

TLS decryption and its limitations

Special appliances can be set up to decrypt HTTPS decryption and inspect it for malware. However, there are several limitations to this approach, including the relatively high technical overhead (i.e., the required computational resources) of the decryption process, along with the wide range of bureaucratic issues it can trigger across the organization.

For an example of the latter, consider that any shift to decryption must be accompanied by consultation with legal and human resources departments. It cannot be done unilaterally. Employees should be informed of what streams will be subject to decryption, so that they do not worry about, say, their online banking sessions being targeted by a firewall. Ideally, a splash page would notify an end user that a decryption operation is occurring.

A new approach to TLS inspection?
But what if there were a way to screen SSL/TLS traffic for threats, without having to undertake the intensive and invasive process of decryption? It sounds like fantasy, but Cisco researchers believe that it may now be possible.

In a paper published at Arxiv, they argued that malware leaves behind recognizable traces in TLS flows. Using deep packet inspection, a security team could in theory find malware by searching for:

  • Bytes in and out
  • Byte distribution
  • Network port numbers (443 is so far the most popular)
  • Sequences of packet lengths
  • TLS header information

Sniffing out clientHello and serverHello messages and TLS versions may also be helpful on this front. The researchers claimed more than 93 percent accuracy in identifying malware from several major families (including Zedbot, Dynamer and Bergat), using encrypted flows within a 5-minute window.

The long term feasibility of this approach is undetermined. For now, encryption protects sensitive data, but it can also disguise malware, so decryption solutions may be advisable to inspect traffic for potential threats.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.