Assembly Notifications

As briefly touched on in our concepts, Transloadit supports two basic modes of operation:

• Sync Mode: where the client waits for encoding to complete and then optionally notifies your back-end.
• Async Mode: where the client resumes as soon as uploading is done, and Transloadit notifies your back-end when encoding completes.

Async Mode is slightly more work to set up, but recommended as it results in:

• A smoother user experience — the user does not have to wait on encoding.
• Reliability — we can retry encoding as well as sending Notifications.

To get the results to your back-end, Transloadit sends an Assembly Notification to your back-end. This is a multipart POST with a field called transloadit, which contains the full Assembly Status JSON. You can find an example of this in our API Response docs.

You specify where we should send this payload in a notify_url webhook of your choosing, like so:

{
  "steps": { ... },
  "notify_url": "https://example.com/transloadit_pingback"
}

Your back-end needs to respond with a 200 header, otherwise Transloadit assumes the Notification has failed and retries 5 times over a period of 1 minute, and only once after 3 seconds for Smart CDN Assemblies.

When you add a notify_url, Transloadit is going to inform your back-end, no matter your client integration. If you don't want your users/program to wait for encoding, this often also involves setting a flag. In the case of Uppy, set the waitForEncoding parameter to false. In many back-end SDKs, waiting for encoding involves explicitly polling the Assembly Status, so just refraining from that will do the trick.

Example

Let's assume you had indeed specified "notify_url": "https://example.com/transloadit_pingback" and that the back-end server that would accept incoming POSTs there was written in Node.js. That could look like this:

const crypto = require('node:crypto')
const http = require('node:http')

const formidable = require('formidable')

const PORT = process.argv[2] || 3020

if (!/^[A-Za-z0-9]{40}$/.test(process.env.AUTH_SECRET)) {
  throw new Error(`Please pass the secret from https://transloadit.com/c/template-credentials
    via the AUTH_SECRET environment var. It must be the auth secret that belongs to the auth key you used for the original Assembly.`)
}

const checkSignature = (fields, authSecret) => {
  const receivedSignature = fields.signature
  const payload = fields.transloadit

  // If the signature contains a colon, we expect it to be of format `algo:actual_signature`.
  // If there are no colons, we assume it's a legacy signature using SHA-1.
  const algoSeparatorIndex = receivedSignature.indexOf(':')
  const algo = algoSeparatorIndex === -1 ? 'sha1' : receivedSignature.slice(0, algoSeparatorIndex)

  try {
    const calculatedSignature = crypto
      .createHmac(algo, authSecret)
      .update(Buffer.from(payload, 'utf-8'))
      .digest('hex')

    // If we are in legacy signature mode, algoSeparatorIndex is -1 and we are
    // comparing the whole string. Otherwise we slice out the prefixed algo.
    return calculatedSignature === receivedSignature.slice(algoSeparatorIndex + 1)
  } catch {
    // We can assume the signature string was ill-formed.
    return false
  }
}

const respond = (res, code, messages) => {
  if (code !== 200) {
    console.error({ messages, code })
  }

  res.writeHead(code, {
    'Content-Type': 'application/json',
    'Access-Control-Allow-Origin': '*',
    'Access-Control-Allow-Methods': 'OPTIONS, POST, GET',
  })

  if (messages) {
    res.write(JSON.stringify({ messages }))
  }

  res.end()
}

http
  .createServer((req, res) => {
    if (req.method === 'OPTIONS') {
      return respond(res, 204)
    }
    if (req.url === '/transloadit_pingback' && req.method === 'POST') {
      const form = new formidable.IncomingForm()
      form.parse(req, (err, fields) => {
        if (err) {
          return respond(res, 500, [`Error while parsing multipart form`, err])
        }

        if (!checkSignature(fields, process.env.AUTH_SECRET)) {
          return respond(res, 403, [
            `Error while checking signatures`,
            `No match so payload was tampered with, or an invalid Auth Secret was used`,
          ])
        }

        let assembly = {}
        try {
          assembly = JSON.parse(fields.transloadit)
        } catch (err) {
          return respond(res, 500, [`Error while parsing transloadit field`, err])
        }

        console.log(`--> ${assembly.ok || assembly.error} ${assembly.assembly_ssl_url}`)

        for (const upload of assembly.uploads) {
          // save upload.ssl_url and metadata to your db here
          console.log(`    ^- uploaded '${upload.name}' ready at ${upload.ssl_url}`)
        }

        for (const stepName in assembly.results) {
          for (const result of assembly.results[stepName]) {
            // save result.ssl_url and metadata to your db here
            console.log(`    ^- ${stepName} '${result.name}' ready at ${result.ssl_url}`)
          }
        }

        return respond(res, 200, [`Success!`])
      })
    } else {
      return respond(res, 500, [
        `Welcome! I only know how to handle POSTs to /transloadit_pingback`,
        `No handler for req.url=${req.url}, req.method=${req.method}`,
      ])
    }
  })
  .listen(PORT, () => {
    console.log(`Server started, listening on http://0.0.0.0:${PORT}`)
  })

You could run this script like so:

$ env AUTH_SECRET=******** node notification-backend-node.js 3020
Server started, listening on http://0.0.0.0:3020

While you're testing locally behind a NAT, you could use a tool like ngrok so Transloadit can still reach your local script. Ngrok is a paid service, but limited use is free forever, and sufficient for our testing purposes. In a new tab:

$ brew cask install ngrok || snap install ngrok # || https://ngrok.com/download
ngrok 2.3.35 from Khiem Doan (khiemdoan) installed

$ ngrok http 3020
Forwarding                    http://2cf99440.ngrok.io -> http://localhost:3020

You can now create a Template and paste the following Instructions. Don't forget to replace the ngrok code of 2cf99440 to whatever ngrok reported in your case:

{
  "notify_url": "http://2cf99440.ngrok.io/transloadit_pingback",
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "faces_detected": {
      "use": ":original",
      "robot": "/image/facedetect",
      "crop": true,
      "faces": "max-confidence",
      "crop_padding": "10%",
      "format": "preserve"
    }
  }
}

Note: At the time of writing ngrok appears to have issues with AWS ranges. If this is the case for you, an alternative to try is localtunnel.

Now you're ready to test right inside the browser. The Instructions we used detect a face, so for optimal results, upload a photo of someone using the Template Editor's testing area. You can use Uppy's Webcam feature if you don't have a picture available.

Your Node.js script should report it has successfully received the Assembly Notification when the Assembly completes:

-- > ASSEMBLY_COMPLETED https://api2.transloadit.com/assemblies/b2b580bdc969427091a48f1f0d3d9d40
^- uploaded 'avatar.jpg' ready at https://s3.amazonaws.com/tmp.transloadit.com/ff89be82...
^- faces_detected 'avatar.jpg' ready at https://s3.amazonaws.com/tmp.transloadit.com/fd2f61b9...

In addition, you'll see a record of the notification on the Assembly page, where you can also manually retry it for further testing. Retry as many times as you like!

Signature Authentication for Notifications

While the Assembly Status JSON resides in the transloadit field of the multipart POST, you'll find there is another field transmitted too, called signature. You can use that field to ensure that only Transloadit can send you data to your notify_url.

We generated this for the request. You can use your Auth Secret to calculate the signature of the request on your end, too. If it matches, you can be sure the response came from Transloadit and was not tampered with, assuming that we are the only two parties that know this secret.

To verify, you'll need to calculate the signature yourself for the entire body of the transloadit field of the Assembly Notification. When the signature in the signature field matches the one you calculate, the request is valid. If not, then the request was tinkered with and should be ignored! And we should call the police too! 🚔

The Example code higher up shows how to do this in Node.js, but there are examples for many other languages in the Signature Authentication docs.