Upload Plugin SDK Docs

What are plugins?

Upload Plugins allow you to write custom code to transform your files.

Plugins are written in JavaScript (Node.js) and run on Upload's infrastructure.

The transformation pipeline

File transformations run on-demand in response to transformation requests.

A transformation request looks like the following:

https://files.upload.io/AYU81093TgPQauaR/an_amazing_transformation                      \____File ID___/ \__Transformation Slug__/

When Upload receives a transformation request, the following occurs:

  • If this is the first time the transformation request has been made, then:
    1. The "transformation slug" from the URL is resolved to a "transformation definition":
      • The "transformation definition" is looked-up in the account that owns the file, and is matched based on the definition's slug matching the slug from the request.
      • The resolved "transformation definition" will contain one or more "transformation steps":
        • Each "transformation step" references an Upload Plugin (the code to run for that step).
        • Each "transformation step" may also include some configuration to pass to the Upload Plugin. For example, an image resizing plugin might require a width and height to be set. This is configured in the Upload Dashboard.
    2. Upload will generate a "transformation function" from the "transformation definition":
      1. Upload will pull the Upload Plugins from the NPM registry.
      2. Upload will construct a function that executes the Upload Plugins one-by-one, in the order dictated by the transformation steps.
    3. The resulting "transformation function" is invoked for the requested file:
      1. The original file is downloaded to the machine where the function is running.
      2. The function will then execute each of its transformation steps in sequence.
      3. Each transformation step will read and write to the file, with each subsequent step seeing the changes from the preceding step.
      4. Finally, the resulting file is uploaded to the "transformation cache", before being sent to the user as the HTTP response.

        Note: the original file is not modified — only a copy of the file is.

  • On subsequent requests for the same URL, the previous result is returned from a permanent cache. (You can clear this cache via the Upload Dashboard.)

The anatomy of a plugin

Creating an Upload Plugin requires writing JavaScript and publishing your code to the NPM registry. You will need to register a free NPM account to do this.

Node.js version

Upload Plugins must target Node.js 12.x.

package.json

Your package.json must contain the following three lines:

package.json
{
  "name":    "your-plugin-name-here",
  "version": "0.0.0",
  "main":    "dist/bundle.js",

  ...
}

handler.js

Your handler.js should contain your transformation function as its "default export":

handler.js
module.exports = async function(invocation) {
  // Todo: modify the file referenced by 'paths.originalFile'
  //       using your own transformation logic.

}

bundle.js

Your bundle.js must be a "fat bundle".

This means your node_modules must be bundled into your bundle.js file -- tools like Webpack and Rollup.js will do this for you.

Fat bundles are required since Upload will not perform an npm install after pulling plugins from NPM: it will simply download and run them.

CommonJS default export

Your bundle.js must export the transformation function you defined in handler.js as its default export, using the CommonJS module system:

bundle.js
module.exports = ... the exported function from handler.js ...

schema.json & ui.json

The files schema.json and ui.json are both optional, but improve the user experience for the person configuring the plugin in the Upload Dashboard:

  • schema.json is a JSON schema of your plugin's configuration (if any). When someone provides configuration for your plugin in the Upload Dashboard, that configuration will be passed to your transformation function in handler.js as a deserialized params object (see Transforming Files).
  • ui.json includes hints about how your plugin's configuration should be represented as a form in the Upload Dashboard. The ui.json file is based on react-jsonschema-form.

Creating a plugin

Installing the plugin SDK

Install the upload-plugin-sdk via NPM or YARN:

npm install upload-plugin-sdk
yarn add upload-plugin-sdk

Transforming files

Firstly, overwrite the handler.js file from earlier with the code below.

This will give you access to several helper methods from the Upload Plugin SDK:

handler.js
const { transform } = require("upload-plugin-sdk");

module.exports = transform(async ({
  transformation,
  step,
  paths,
  params,
  resolve,
  log
}) => {
  ...
});

Next, use resolve("/") to get the original file's path:

handler.js
const { transform } = require("upload-plugin-sdk");

module.exports = transform(async ({
  transformation,
  step,
  paths,
  params,
  resolve,
  log
}) => {

  const originalFilePath = resolve("/");
});

To perform the file transformation, we will set the file's length back into the file as its new content. This effectively creates a transformation that "returns the size of any given file":

handler.js
const { transform } = require("upload-plugin-sdk");
const { promises as fsAsync } = require("fs");

module.exports = transform(async ({
  transformation,
  step,
  paths,
  params,
  resolve,
  log
}) => {
  const originalFilePath = resolve("/");

  const fileStats        = await fsAsync.stat(originalFilePath)
  const fileSize         = fileStats.size.toString()
  await fsAsync.writeFile(originalFilePath, fileSize)
});

Now, when someone requests file X through this transformation, they will receive the size of file X as the HTTP response.

And that's it: we have implemented our first plugin!


Transformations are typically more complex than this, and may even invoke native libraries. Nonetheless, the principle is the same: your function receives a copy of the original uploaded file, overwrites it, and the overwritten file becomes the result of your transformation, which is then returned to the user as the HTTP response.

Creating artifacts

Transformation functions can output multiple files:

https://files.upload.io/AYU81093TgPQauaR/example_trans/https://files.upload.io/AYU81093TgPQauaR/example_trans/file_1https://files.upload.io/AYU81093TgPQauaR/example_trans/file_2https://files.upload.io/AYU81093TgPQauaR/example_trans/file_2/file_3

To write to any part of the URL, use resolve(...), passing the relative path you want to write to. The resolve method will return a local file path for you to write to. If you write a file to this path, that file will be hosted from that relative location on the URL.

For example, if you saved "Hello world" to the file at the location resolve("/foo/bar"), then the following URL would return the string "Hello world":

https://files.upload.io/AYU81093TgPQauaR/example_trans/foo/bar

Logging

The Upload Plugin SDK comes with a log method.

Each message you pass to the log method will be appended to the /logs/ artifact:

https://files.upload.io/AYU81093TgPQauaR/example_trans/logs/

Deploying your plugin

To use your plugin: publish it to NPM, then create a transformation in the Upload Dashboard that uses your plugin in one of its steps.

Publish to NPM

If you haven't already, you will need to create an NPM account.

Then login to NPM:

npm login
yarn login

To publish your plugin to the NPM Registry:

npm publish
yarn publish

Installing on Upload

Coming soon! If you need assistance installing your plugin, please contact us.

Creating a transformation

Coming soon! If you need assistance configuring file transformations, please contact us.

Using your transformation

To use your transformation, get the URL of the file you'd like to transform, and then append your transformation's slug to it:

https://files.upload.io/AYU81093TgPQauaR/an_amazing_transformation                      \____File ID___/ \__Transformation Slug__/\_____________File URL______________/

Example plugins

The following plugins are open source and provide a great starting point for writing your own:

Upload PluginDescription

upload-compression-plugin

Compresses and decompresses files.

upload-image-plugin

Resizes, crops and converts images.

You are using an outdated browser.

This website requires a modern web browser -- the latest versions of these browsers are supported: