Our /file/filter Robot

Filter files

🤖/file/filter directs files to different encoding Steps based on your conditions.

Think of this Robot as an if/else condition for building advanced file conversion workflows. With it, you can filter and direct certain uploaded files depending on their metadata.

The Robot has two modes of operation:

  • Constructing conditions out of arrays with 3 members each. For example, ["${file.size}", "<=", "720"]
  • Writing conditions in JavaScript. For example, ${file.size <= 720}. See also Dynamic Evaluation.

Passing JavaScript allows you to implement logic as complex as you wish, however it’s slower than combining arrays of conditions, and will be charged for per invocation via 🤖/script/run.

Conditions as arrays

The accepts and declines parameters can each be set to an array of arrays with three members:

  1. A value or job variable, such as ${file.mime}
  2. One of the following operators: ==, ===, <, >, <=, >=, !=, !==, regex, !regex
  3. A value or job variable, such as 50 or "foo"

Examples:

  • [["${file.meta.width}", ">", "${file.meta.height}"]]
  • [["${file.size}", "<=", "720"]]
  • [["720", ">=", "${file.size}"]]
  • [["${file.mime}", "regex", "image"]]

Warning: If you would like to match against a null value or a value that is not present (like an audio file does not have a video_codec property in its metadata), match against "" (an empty string) instead. We’ll support proper matching against null in the future, but we cannot easily do so right now without breaking backwards compatibility.

Conditions as JavaScript

The accepts and declines parameters can each be set to strings of JavaScript, which return a boolean value.

Examples:

  • ${file.meta.width > file.meta.height}
  • ${file.size <= 720}
  • ${/image/.test(file.mime)}
  • ${Math.max(file.meta.width, file.meta.height) > 100}

As indicated, we charge for this via 🤖/script/run. See also Dynamic Evaluation for more details on allowed syntax and behavior.

Available job variables

Note: Conditions on properties that a file doesn’t have will be ignored. For example, an image doesn’t have ${file.meta.bitrate}. Also, note that since ${file.width} will be ignored, use ${file.meta.width} instead.

  • ${assembly.id}

    The ID of the Assembly representing the current upload, which is a UUIDv4 without dashes.

  • ${assembly.region}

    The AWS region where the Assembly is being processed. You could use this to import files from a bucket in the same region, to reduce data transfer costs and latencies.

  • ${assembly.parent_id}

    The ID of the parent Assembly when replaying that parent Assembly.

  • ${date.now}

    The current date and time represented as the number of milliseconds elapsed since the UNIX epoch, which is defined as the midnight at the beginning of January 1, 1970, UTC.

  • ${unique_prefix}

    A unique 33-character prefix used to avoid file name collisions, such as "f2/d3eeeb67479f11f8b091b04f6181ad".

    Please notice the / in the prefix. If you use ${unique_prefix} in the path parameter of 🤖/s3/store for example, then it will create sub-directories in your S3 bucket. This may or may not be desired. Use ${file.id} if you require a unique prefix without slashes.

  • ${unique_original_prefix}

    This is similar to ${unique_prefix}, with the exception that two different encoding results of the same uploaded file (the original file) will have the same prefix value here.

  • ${previous_step.name}

    The name of the previous Step that produced the current file.

  • ${file.id}

    The ID of the file being processed, which is a UUIDv4 without dashes.

  • ${file.original_id}

    The ID of the original file that a certain file derives from. For example, if you use an import Robot to import files and then encode them somehow, the encoding result files will have a ${file.original_id} that matches the ${file.id} of the imported file.

  • ${file.original_name}

    The name of the original file (including file extension) that a certain file derives from. For example, if you use an import Robot to import files and then encode them somehow, the encoding result files will have a ${file.original_name} that matches the ${file.name} of the imported file.

  • ${file.original_basename}

    The basename of the original file that a certain file derives from. For example, if you use an import Robot to import files and then encode them somehow, the encoding result files will have a ${file.original_basename} that matches the ${file.basename} of the imported file.

  • ${file.original_path}

    The import path of the original file that a certain file derives from. All of our import Robots set ${file.original_path} accordingly.

    For example, if you use 🤖/s3/import to import files from Amazon S3, the imported files, as well a all files that are derived from them, will have a file.original_path that equals the path to the file on S3, but without the filename. So if the S3 path was "path/to/file.txt", then file.original_path will be "/path/to/". If the path was "/a.txt", ${file.original_path} will be "/".

    file.original_path will always have sufficient slashes in order for you to safely use it in the path parameter of your export step, like this: "path": "${file.original_path}${file.name}". This is handy if you want to import files from, for example, S3, convert them somehow and store them again on S3 in the same (or similar) file structure.

  • ${file.name}

    The name of the file being processed, including the file extension.

  • ${file.url_name}

    The slugged name of the file.

    Any characters other than A-Z a-z 0-9 -_. are replaced with underscores, and spaces are replaced with dashes. This includes the file extension as well.

    Note that if you have two files ッッ.jpg and チチ.jpg, they will both be called __.jpg. So you'll want to take extra care to only use ${file.url_name} in conjunction with ${unique_prefix} or ${file.md5hash}.

  • ${file.basename}

    The name of the file being processed, without the file extension.

  • ${file.user_meta}

    Tus uploads, which you use to upload to Transloadit, can carry meta data values. All the extra values sent along will end up in user_meta. This allows you to do things dynamically per file instead of per assembly with fields.

  • ${file.url_basename}

    The slugged basename of the file (the file name without the file extension).

    Any characters other than A-Z a-z 0-9 -_. are replaced with underscores, and spaces are replaced with dashes.

    Note that if you have two files ッッ.jpg and チチ.jpg they will both be called __.jpg. So you'll want to take extra care to only use ${file.url_basename} in conjunction with ${unique_prefix} or ${file.md5hash}.

  • ${file.ext}

    The file extension.

  • ${file.size}

    The file size in bytes.

  • ${file.mime}

    The file's MIME type.

  • ${file.md5hash}

    The file's MD5 hash. This is a hash over the file's contents, not only over the file's name.

  • ${file.*}

    Any file property available in the final results array, such as ${file.meta.width}. Not all meta keys are available for all file types.

  • ${fields.*}

    The fields submitted together with the upload.

    For example, in the case of a Form submission where Uppy was set to allow fields: ['myvar'], and the form had a tag like <input type="hidden" name="myvar" value="1" />, ${fields.myvar} would contain a value of 1.

    Alternatively, fields could also be populated programmatically like so:

    {
  "steps": {
    "store": {
      "use": "encoded",
      "robot": "/s3/store",
      "credentials": "YOUR_S3_CREDENTIALS_NAME",
      "path": "${assembly.id}/${fields.subdir}/356"
    }
  },
  "fields": {
    "subdir": "bar"
  }
}

    In the case of a conflict, variables derived from form fields take precedence over those derived from the fields key.

Available operators for array conditions

Operator Description Example

==

Equals without type check

["${file.ext}", "==", "jpg"]

===

Strict equals with type check

["${file.ext}", "===", "jpg"]

<

Less than

["${file.size}", "<", "1024"]

>

Greater than

["${file.meta.width}", ">", "${file.meta.height}"]

<=

Less or equal

["${file.meta.audio_bitrate}", "<=", "256000"]

>=

Greater or equal

["${file.meta.audio_samplerate}", ">=", "44100"]

!=

Simple inequality check without type check

["${file.ext}", "!=", "jpg"]

!==

Strict inequality check with type check

["${file.ext}", "!==", "jpg"]

regex

Case-insensitive regular expression based on RE2 .match()

["${file.mime}", "regex", "image"]

!regex

Case-insensitive regular expression based on RE2 !.match()

["${file.mime}", "!regex", "image"]

includes

Check if the right element is included in the array, which is represented by the left element

['${file.meta.descriptions}', 'includes', 'Bridge']

!includes

Check if the right element is not included in the array, which is represented by the left element

['${file.meta.descriptions}', '!includes', 'Bridge']

empty

Check if the left element is an empty array, an object without properties, an empty string, the number zero or the boolean false. Leave the third element of the array to be an empty string. It won’t be evaluated.

['${file.meta.descriptions}', 'empty', '']

!empty

Check if the left element is an array with members, an object with at least one property, a non-empty string, a number that does not equal zero or the boolean true. Leave the third element of the array to be an empty string. It won't be evaluated.

['${file.meta.descriptions}', '!empty', '']

Usage example

Reject files that are larger than 20 MB:

{
  "steps": {
    "filtered": {
      "robot": "/file/filter",
      "use": ":original",
      "declines": [["${file.size}", ">", "20971520"]],
      "error_on_decline": true,
      "error_msg": "File size must not exceed 20 MB"
    }
  }
}

Parameters

  • use

    String / Array of Strings / Object required

    Specifies which Step(s) to use as input.

    • You can pick any names for Steps except ":original" (reserved for user uploads handled by Transloadit)

    • You can provide several Steps as input with arrays:

      "use": [
  ":original",
  "encoded",
  "resized"
]

    💡 That’s likely all you need to know about use, but you can view Advanced use cases.

  • accepts

    Array of Arrays / String ⋅ default: []

    Files that match at least one requirement will be accepted, or declined otherwise. If the array is empty, all files will be accepted. Example:

    [["${file.mime}", "==", "image/gif"]].

    If the condition_type parameter is set to "and", then all requirements must match for the file to be accepted.

  • declines

    Array of Arrays / String ⋅ default: []

    Files that match at least one requirement will be declined, or accepted otherwise. Example:

    [["${file.size}",">","1024"]].

    If the condition_type parameter is set to "and", then all requirements must match for the file to be declined.

  • condition_type

    String ⋅ default: "or"

    Specifies the condition type according to which the members of the accepts or declines arrays should be evaluated. Can be "or" or "and".

  • error_on_decline

    Boolean ⋅ default: false

    If this is set to true and one or more files are declined, the Assembly will be stopped and marked with an error.

  • error_msg

    String ⋅ default: "One of your files was declined"

    The error message shown to your users (such as by Uppy) when a file is declined and error_on_decline is set to true.

Demos

Related blog posts