When Larry Cashdollar, a security researcher for Akamai’s Security Intelligence Response Team, found that it was possible to upload malicious files to the server using Blueimp’s File Upload jQuery plugin, (CVE-2018-9206), it started him “down a rabbit hole” investigating other projects.
While the code in this particular plugin has been fixed, there are many software projects relying on Apache's .htacess to protect them, not realizing that security control is no longer available by default, Cashdollar warned.
While investigating Cashdollar's report, the plugin’s developer, Sebastian Tschan, discovered the problem was related to the changes Apache made in its Apache HTTPD server nearly eight years ago. The plugin relied on a custom .htaccess file to restrict the permissions on the server’s upload folder, a common-enough practice. The problem was that starting with version 2.3.9 (November 2010), Apache HTTPD had switched on a new setting that allowed server administrators to ignore custom (user-owned) .htaccess files. With this new setting, it didn’t matter what permissions the plugin defined in the .htaccess file because the web server ignored them, with the unfortunate result that the upload directory was left unprotected. In this case, malicious individuals could potentially upload backdoors and web shells through the PHP application and compromise the server.
“The internet relies on many security controls every day in order to keep our systems, data, and transactions safe and secure,” Cashdollar wrote. “If one of these controls suddenly doesn't exist it may put security at risk unknowingly to the users and software developers relying on them.”
[Developers] don’t realize that the [security] control is gone, and that their application has been vulnerable for years.
It makes sense for Apache to put in a setting to prevent .htaccess files from overriding server configurations, especially since these files impact server performance. The .htaccess file can also pose a potential security issue in multi-tenant environments. In a situation like a university web server shared by thousands of students, a single student can override the administrator’s decision to block PHP code execution on the server and run PHP code.
“There may be a limited impact on the server, or it can allow the server to be compromised,” Cashdollar said.
Apache documented the fact that .htaccess is disabled by default, but that doesn’t mean the fact was well-known. A prolific developer, Tschan was unaware of the change. Cashdollar checked with a number of security professionals and hackers who have used Apache’s web server software for twenty-odd years, and found none of them knew that Apache had tightened up the web server configuration in this way. Some of the most plugged-in people in web development and application security missed this change, so it is no surprise that many project developers would not know they had to change how they should use .htaccess.
There was “no parade or giant announcement” from Apache informing developers that a commonly used practice had been changed to require some upfront coordination with the server administrator, Cashdollar said. There was no public guidance informing developers of what they needed to do if they wanted to keep using .htaccess.
Down the Rabbit Hole
The plugin is widely used. Cashdollar has found 7,800 forks (a new project was created using this project as a base) of the plugin on GitHub or somehow integrated into other projects. In one case, the fork is provided in a Docker image. So even though Tschan has fixed the issue, the problematic code can still be present in these other projects. Cashdollar has already found a thousand forked projects that have the vulnerability, and about 10 or 15 projects where developers had removed the code, thus removing the vulnerability. There is no easy way to tell if the developer understood the implications of the code they had removed or why they didn’t communicate the changes upstream to the original project.
"As it turns out, the problem is much larger than a single jQuery project," Cashdollar wrote in a follow-up post.
Many of the projects have not been maintained for years and do not appear to be under active development, adding to the challenge of getting the issue fixed or notifying impacted users. Cashdollar said in a follow-up post that he and GitHub are currently discussing how they can contact the owners of vulnerable forked projects en masse and encourage them to pull the latest update.
This is where the rabbit hole gets even more convoluted, because any PHP application that relies on .htaccess would be affected by this change Apache made years ago, not just Tschan’s file upload plugin or other applications that borrowed his code. Focusing on applications that integrated the plugin or on forked projects would be shortsighted, because the real issue lies in the fact that developers may still be using .htaccess and not instructing application users to verify how the server is configured. Many plugins for WordPress and other CMS platforms use .htaccess, as do CRM software and other enterprise applications. Affected projects may not even be in GitHub at all. Investigating and remediating projects would be a major, multi-year undertaking.
Don’t make the assumption that .htaccess is available.
“[Developers] don’t realize that the [security] control is gone, and that their application has been vulnerable for years,” Cashdollar said. He predicted that this disconnect between how the web server handles .htaccess and how applications rely on the file would be the “source of breaches to come.”
Cashdollar is still trying to get a feel for the magnitude of the problem and said this was “just the tip” of a messy situation, since applications using .htaccess are “all over the internet.”
It wasn’t immediately clear whether administrators who updated their servers suddenly went from having .htaccess enabled to disabled, or if the change was visible only on fresh installs. Cashdollar said he planned to test to see if the update process would overwrite the existing apache.conf file or keep existing configurations intact. If the file is not overwritten, then the application would keep working as designed. Of course, as soon as the administrator tried to re-install the application on a fresh server, the .htaccess part would stop working.
Bottom line, “Don’t make the assumption that .htaccess is available,” Cashdollar said.
This post was updated with reference to Cashdollar's follow-up blog post.