Introducing our new virus scanning Robot for safer uploads
As people are moving more parts of their life to digital platforms, stakes are getting higher and keeping them safe becomes more important than ever.
Whether building native apps, APIs, or web applications, it is our responsibility as developers / ops to invest a decent amount of thought and time into keeping our users safe from those with malicious intent.
Sanitizing input, firewalling, rolling out upgrades, limiting what SSL versions can be used — the list is only getting longer.
Additional challenges are introduced when you allow end users to upload files, since you essentially allow anyone who can operate a keyboard to store potentially malicious files onto your otherwise carefully secured platform.
It is, of course, possible that a malicious file just sits there, and nothing happens. Even so, there is always the risk that bad things might happen and you should be wary of the following:
First of all, you will have to be very careful that a malicious file isn't executed somehow. You probably have 100 mechanisms in place to prevent this: unsetting execution bits, escaping any user input, and so forth. But there is always a 101st way to get around all of that. Given enough time and motivation, a clever hacker will find that way and execute his/her file on your platform. Risks range from secrets being exposed, to data being destroyed.
Secondly, you are probably allowing user uploads in order for other users to consume the content. This means that if a malicious file has been uploaded, you are now a distributor of malware. There is no way of telling whether consumers will execute the file, whether their local antivirus kicks in and they will badmouth you, or whether they unknowingly pass on the virus. Additionally, if your content is accessible by web-crawlers, search engines like Google will notice, which will hurt your ranking and, as a result, your business.
Which ever way you look at it, if you could have some kind of gatekeeper that would reject trojans, viruses, malware and other malicious threats before they reach your platform — any risks would be dramatically reduced.
That is why the Transloadit team got to work and designed an all new robot to help you with this: /file/virusscan.
While 100% safety is a myth, /file/virusscan is built on the widely adopted ClamAV. By using it as a gatekeeper, it will be able to recognize and reject nearly 4 million trojans, viruses, malware, and other malicious threats.
Enabling this protection is as simple as adding a few lines to your instructions.
Maybe your instructions previously converted uploads to iPad format, extracted some preview thumbnails, and exported the results to your private server via SFTP:
steps:
ipad:
use : ":original"
robot : "/video/encode"
ffmpeg_stack: "v2.2.3"
preset : "ipad-high"
thumbnails:
use : "ipad"
robot : "/video/thumbs"
store:
use : [ ":original", "ipad", "thumbnails" ]
robot : "/sftp/store"
user : "transloadit-uploader"
host : "my.website.com"
path : "./transloadit-uploads"
url_template: "https://my.website.com/transloadit-uploads/${file.url_name}"
note: These instructions have been enhanced for readability, but should be encoded as valid JSON when you feed them to our service.
To reject malware, add a scanned
step at the beginning:
steps:
scanned:
use : ":original"
robot : "/file/virusscan"
ipad:
use : "scanned"
robot : "/video/encode"
ffmpeg_stack: "v2.2.3"
preset : "ipad-high"
thumbnails:
use : "ipad"
robot : "/video/thumbs"
store:
use : [ ":original", "ipad", "thumbnails" ]
robot : "/sftp/store"
user : "transloadit-uploader"
host : "my.website.com"
path : "./transloadit-uploads"
url_template: "https://my.website.com/transloadit-uploads/${file.url_name}"
As you can see, we only prepended three lines. We made it consume :original
, and changed other
Steps that were previously relying on :original
to our newly added scanned
step. Much
like piping together a bunch of unix tools. Don't you love the flexibility of our
Assembly flow logic? 😄
In this typical example, we are gatekeeping at the first step. However, much like how our /file/filter Robot works, you can insert virus scanning at any step. This makes sense if you import content from other places, like you would with our /http/import Robot. In such a case, you would import first, then scan, and only when we could not find any threat, would you continue encoding and exporting.
To learn more about this Robot, or its parameters like error_on_decline
, please head
over to its documentation.
You can start using /file/virusscan today for your plan's $/GB pricing.