Manual content moderation can be incredibly inefficient and expensive, particularly for platforms with high volumes of user-generated content. Fortunately, it is often a possibility to perform the bulk of this work automatically. Tools such as the Transloadit /image/describe Robot make it easy to filter out explicit content without any human intervention, allowing your moderators to do their job more effectively. Let's dive in and see how it works.

Pixelated background with the text 'NSFW Image Filter'

Why do NSFW filters matter?

Before getting into the technical details, let's briefly go over why NSFW filters are so important:

  1. User protection: maintain a safe environment for all users, especially if your platform aims to be child-friendly.
  2. Brand safety: safeguard your brand's reputation from users posting inappropriate content.
  3. Legal compliance: meet content regulation requirements, which can be strict in places like the EU.
  4. User experience: enhance overall user satisfaction, as most users prefer not to encounter gore or nudity while browsing.

As you can see, there are plenty of reasons to be serious about moderating your image content. Having this set as a task for humans, though, is far from ideal, to say the least. Manually going through a bulk of images can be time consuming under the best of circumstances – and traumatizing under the worst.

Perhaps there is something to be done about that...

Automating the content moderation process

NSFW filters and content moderation have been notoriously challenging for many online platforms. However, thanks to advancements in machine learning, image classification can now be largely automated. This is precisely what the Transloadit /image/describe Robot leverages under the hood. More specifically, we use AWS and GCP ML services to ensure your images are classified using the most sophisticated models available.

So, how can you use this to stop NSFW images from reaching your site?

Leveraging the /image/describe Robot

Transloadit's /image/describe Robot can analyze images and provide detailed descriptions, which we can use to identify potentially NSFW content. We then combine these descriptions with the /file/filter Robot to disregard any images containing explicit tags that we don't want to permit.

Here's a sample Assembly:

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "described": {
      "use": ":original",
      "robot": "/image/describe",
      "provider": "aws",
      "explicit_descriptions": true,
      "format": "meta",
      "granularity": "list",
      "result": true
    },
    "filtered": {
      "result": true,
      "use": "described",
      "robot": "/file/filter",
      "explicit_descriptions": true,
      "declines": [["${file.meta.descriptions}", "!=", ""]],
      "error_on_decline": true,
      "error_msg": "One file contains explicit content!"
    },
    "exported": {
      "use": "filtered",
      "robot": "/s3/store",
      "credentials": "YOUR_AWS_CREDENTIALS",
      "url_prefix": "https://demos.transloadit.com/"
    }
  }
}

This Assembly does the following:

  1. Handles the uploaded file.
  2. Analyzes the image using AWS Rekognition.
  3. Filters out images that contain any explicit keywords (which can be found here for AWS and here for GCP).
  4. Stores the images in S3, if they all pass.

Enhancing your NSFW blocker

Of course, this approach may be too broad for you. Many sites allow some NSFW content, but not all. That's why a better strategy for some might be to use custom keyword filtering.

Check out the example below:

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "described": {
      "use": ":original",
      "robot": "/image/describe",
      "provider": "aws",
      "explicit_descriptions": true,
      "format": "meta",
      "granularity": "list",
      "result": true
    },
    "filtered": {
      "result": true,
      "use": "described",
      "robot": "/file/filter",
      "declines": [["${file.meta.descriptions}", "includes", "Visually Disturbing"]],
      "error_on_decline": true,
      "error_msg": "One file contains explicit content!"
    },
    "exported": {
      "use": "filtered",
      "robot": "/s3/store",
      "credentials": "YOUR_AWS_CREDENTIALS",
      "url_prefix": "https://demos.transloadit.com/"
    }
  }
}

Once again, we follow the same pattern as before – except this time, instead of checking for any explicit descriptors, we only filter images that contain visually disturbing content. Naturally, you can fine-tune this to suit your site's needs, depending on the type of content you wish to allow.

Conclusion

By using Transloadit's /image/describe Robot, you can create a robust NSFW filter to maintain a safe online environment.

Looking for more? We've covered content moderation extensively in our previous blogs. So if you're wondering what to focus on next, why not take a look at how to automatically blur faces in photos?