Automated Integrity Verification for Web Downloads (AIVWD)
Background
Since the early days of the World Wide Web, web sites that distribute downloadable files (such as downloadable software programs, documents, etc.) have offloaded the hosting of these files to mirror sites. For example, the popular open-source software platform SourceForge.Net uses a network of mirror sites to host the thousands of files that it distributes through its site. More recently, Content Delivery Networks (CDN’s) have emerged, which have built on the concept of mirror sites, to distribute static files across hundreds, or even thousands, of servers globally. Some web sites also utilize third-party cloud storage providers to host downloadable files.The use of mirror sites, CDN’s, and other third-party providers to host downloadable files offers enormous benefits. By offloading the hosting of these static files to third-party providers, sites are able to realize significant increases in performance, availability, and scalability. However, this also introduces another attack vector. If a downloadable file is compromised while hosted by the third-party provider (for example, by a rogue system administrator of a third-party server, or by a hacker that is able to gain access to the third-party server), then these files can easily become vectors for malware, ransomware, and other cyber attacks.
This is not just a hypothetical scenario. In 2012, a SourceForge mirror site was compromised. Attackers inserted a backdoor into a widely used database administration tool hosted on the compromised mirror. In 2013, MaxCDN's servers were compromised by attackers using the credentials of a recently laid-off engineer. This laid the groundwork for an attack on thousands of web sites utilizing a widely-used web development framework. A recent report by Threatpost cites trojanized files as one of the most common attack vectors for ransomware and malware, and the 2020 Solar Winds attack reminds us of just how destructive compromised files that go undetected can be.
As a way of mitigating this threat, sites often post checksum hashes near the download link for a file hosted by a third-party provider. For example, see this page at Dell's web site, where Dell users can download a BIOS update for a Dell laptop computer. Notice on this page that the expected checksum hashes for the downloadable file are posted, just below where it reads, 'To ensure the integrity of your download, please verify the checksum value'. The expectation is that after downloading the file to their system, the user will take the checksum of the file, and verify that the actual checksum of the file matches the expected checksum as published on the site. If not, then this is an indication that the file may have been compromised or tampered with.
However, this procedure almost always goes unused, as reported in a 2018 paper by Mauro Cherubini et al. Most non-technical users are simply unfamiliar with the verification procedure and how to apply it. And, even technical users often neglect to perform the procedure out of complacency. Yet, failure to apply this procedure can have disastrous consequences if the file has been compromised. Computer science researcher Adrian Colyer wrote about this subject in 2018. Colyer's article highlights Cherubini’s findings, and goes on to underscore the need for an automated solution to this problem. Cherubini and Colyer note that standards such as HTTPS, SSL/TLS, and subresource integrity are not solutions to this problem, and propose a browser extension as a solution.
US patent #10,505,736 teaches a solution for solving this problem. The solution disclosed in the '736 patent is an automated solution for verifying the integrity of files downloaded from the web that runs transparently to the user, without user intervention, and without the need for the user to install a browser extension.
The AIVWD Solution
The solution claimed in the '736 patent is referred to as AIVWD (Automated Integrity Verification for Web Downloads). AIVWD verifies the integrity of a downloaded file in an automated fashion using client-side scripting served by the trusted server, which runs in the user's web browser.AIVWD automates integrity verification of downloadable files using a short javascript function. After the user clicks the button to download the file, the user's web browser makes a request to the trusted web server. The trusted web server responds with the URL of the downloadable file, the expected checksum of the file, and a javascript function that downloads the file and performs the integrity verification, as shown below in Figure 1:
Next, the javascript function running in the user's web browser reqeusts the file from the third-party server, and the third-party server responds with the file, as shown below in Figure 2:
Finally, the javascript function running in the user's web browser takes the actual checksum of the downloaded file, and compares it with the expected checksum, as shown below in Figure 3. Then, if (and only if) the actual checksum of the file matches the expected checksum of the file, the user is prompted to open or save the file on their system. Otherwise, the user is warned that the file may have been compromised, the user is not given the opportunity to open or save the file, and the file is discarded.
The entire process runs without user intervention, and without the need for the user to install a browser extension. The '736 patent also claims variation of the solution where the functionality of the javascript is implemented directly in the web browser, alleviating the need for the javascript.
Demo
AIVWD is implemented in the demo below. In this demo, the file resides on a third-party server on Amazon’s AWS network. The trusted server (hosting this web page) serves the URL of the file hosted on the third-party server (https://ivdfw.s3.us-east-2.amazonaws.com/download.dl), the expected checksum of the file (f17214c34b5d37d1578949f66018b64528e6fc32d01018a67bb1b989691fe782), and the javascript function that performs the file download and integrity verification. After clicking the button below, the user sees that the integrity verification is successful, and the file is made available to the user to open or save to their system.
Below is a second demo, simulating a case where the file may have been tampered with, as the actual checksum of the file does not match the expected checksum. After clicking the button below, the user sees that the integrity verification fails, and the file is not made available to the user to open or save to their system.
Tails Implementation
In 2020, MTI had the honor of working with The Tails Organization to implement a variation of AIVWD. The Tails Organization distributes the widely used Tails operating system, for use with the TOR network. Tails uses a network of mirror sites to host the ISO files for its operating system. Realizing how disastrous a compromised ISO file on a mirror site could be, Tails implemented a variation of AIVWD to provide its users with a simple way to ensure the integrity of ISO files downloaded from its mirror sites. The solution replaced Tails' previous solution, which required a browser extension. See https://tails.boum.org/news/verification_extension_deprecation/index.en.html and https://tails.boum.org/contribute/design/download_verification/ for more information.
Conclusion
We are continuously reminded of the devastating effects of attacks stemming from malware injected into downloadable files on the web. These attacks impact numerous parties, including:- victims that unwittingly download compromised files
- software vendors that distribute software through supply chains that they don't control
- companies and organizations that offload hosting of downloadable files to third parties
- third-party mirrors, CDN's, and cloud storage providers
- web browser vendors that aim to protect their users from cyber attacks on the web
- security firms charged with protecting their clients from cyber attacks on the web