Tasks

Overview

Tasks are the individual steps that are run sequentially in Flex.io Pipes. Tasks can be chained together such that the Input from the caller is passed to the first Task, which operates on the input and returns an output; then the output of that Task is passed to the next Task and so on, until the Tasks are finished and the process returns a final output.

The primary Task type is the Execute Task, which is used to run Python or Node.js scripts. Other Task offer assists to your Execute function, such as bindings for accessing connections or reading files.

Tasks are accessed and used in the Tasks section of the Pipe Builder.

Pipes: Tasks Section

Tasks options discussed in this guide include the following:

Tasks options include the following:

Tasks may be added above or below other tasks by clicking the “+” button in the Pipe Builder. To add a Task, simply click the Task box you’d like to use.

Execute

Click the “Execute” box to add an Execute Task. The Execute Task is used to run Python or Node.js scripts.

Tasks: Execute

Note the following controls in the Execute dialog:

OAuth

Click the “OAuth” box to add an pre-populated Execute Task. The OAuth Task is a wizard that helps you set up or utilize an existing Connection that uses OAuth.

Tasks: Execute

To use the OAuth task, do the following:

  1. If you have an existing Connection set up in Flex.io, select your Connection. If you haven’t set up a Connection yet, click on the “Set up a New Connection” button and follow the prompts to authenticate your OAuth connection.

  2. After selecting your Connection, choose the language you’d like to use for your resulting Execute Task in the language drop-down box. When finished, click the “Save Changes” button to get your Execute Task.

Your resulting function will look something like this, annotated here with comments for each section:

# this function returns an OAuth access token to access the service specified

def flex_handler(flex):

# get the OAuth access token for the connection from the connection
# identifier, which can be the connection alias or connection eid
connection_identifier = 'id'
auth_token = flex.connections[connection_identifier].get_access_token()

# echo the OAuth token; use this token to make authenticated API calls
# to the service defined in the connection
flex.end(auth_token)

Once you have the OAuth token, you can use it to authenticate your API requests. Often, you use it with the HTTP Authorization header, but each API may differ. Here’s an example of how it might be used:

def flex_handler(flex):

# get the authorization token from the Flexio connection
connection_identifier = 'id'
auth_token = flex.connections[connection_identifier].get_access_token()

# url = 'https://api.company.com/endpoint'
# headers = {'Authorization': 'Bearer ' + auth_token}
# response = requests.get(url, headers=headers)
# flex.output.write(response.text)

Read

Click the “Read” box to add an Read Task. The Read Task is used to set up or utilize an existing Connection Storage Type Connection and choose the file you’d like to read.

Tasks: Read

To use the Read task, do the following:

  1. If you have an existing Connection set up in Flex.io, select your Connection. If you haven’t set up a Connection yet, click on the “Set up a New Connection” button and follow the prompts to authenticate your storage-type connection.

  2. After selecting your Connection, choose the file you’d like to read from the Connection file system by clicking the Browse button. When finished, click the “Save Changes” button. The output of the Read Task is written to stdout to be consumed by the following Task or written to the pipe output. Here’s an Execute task you can add after the read task that converts the output of the read task to uppercase:

def flex_handler(flex):
buf = flex.input.read() # read the output from the previous step
buf = buf.upper() # convert the buffer to uppercase
flex.output.write(buf) # write to the output of this step

Further, you may also Read files from your storage-type Connections directly using an Execute task. See the Connection guide for example Read script bindings.

Write

Click the “Write” box to add an Write Task. The Write Task is used to set up or utilize an existing Connection Storage Type Connection and choose the destination to which you’d like to write files.

Tasks: Write

To use the Write task, do the following:

  1. If you have an existing Connection set up in Flex.io, select your Connection. If you haven’t set up a Connection yet, click on the “Set up a New Connection” button and follow the prompts to authenticate your storage-type connection.

  2. After selecting your Connection, choose the directory you’d like to write files to by clicking the Browse button. When finished, click the “Save Changes” button. The input of the Write Task is the stdout from the previous task in your Pipe.

Further, you may also Write files to your storage-type Connections directly using an Execute task. See the Connection guide for example Read script bindings.

Request

Click the “Request” box to add an Request Task. The Request Task is used to create a HTTP request.

Tasks: Request

To use the Request task, do the following:

  1. Add your method (e.g., GET, POST) and endpoint in the “Method & URL” form box.

  2. Add any authorization, form data or headers in the following section.

  3. After filling out the form, click the “Save Changes” button. You may test your request by clicking the “TEST” button from the Pipe Builder. The HTTP response from the HTTP request in this task is written to stdout to be consumed by your next task.

Further, you may create requests directly from your Python or Node.js scripts using an Execute Task.

Convert

Click the “Convert” box to add an Convert Task. The Convert Task is a simple junction box that helps convert from one format to another.

Tasks: Convert

To use the Convert task, do the following:

  1. The Convert task requires a data stream from a previous task (e.g., an HTTP request that returns JSON). Select the input format you want to conver from.

  2. The Convert task will then convert to a format of your choice, including CSV, PDF, RSS, among others. Select the conversion output from the Output dropdown menu.

  3. When you are finished, click Save Changes to save your Task. You may test your request by clicking the “TEST” button from the Pipe Builder. The output of the Convert Task is written to stdout to be consumed by the following Task.

Further, you may create conversions directly from your Python or Node.js scripts using an Execute Task.

Email

Click the “Email” box to add an Email Task. The Email Task is used to set up an email notification.

Tasks: Email

To use the Email task, do the following:

  1. The Email tasks simply sends a notification at a given step in your pipe. Enter the “To” address, a “Subject” line and a Message Body.

  2. Note that your body text can include normal text as well as variables. For example, you may take the stdout from the previous Task using the variable ${input}.

  3. When you are finished, click Save Changes to save your Task. You may test your request by clicking the “TEST” button from the Pipe Builder and checked from your inbox.

Connect

Click the “Connect” box to add an Connect Task. The Connect Task binds a Connection to a variable reference for use in the Pipe. This enables the Pipe to be portable by referencing the Connection without have to include the Connection’s eid or alias.

Tasks: Connect

To use the Connect task, do the following:

  1. If you have an existing Connection set up in Flex.io, select your Connection. If you haven’t set up a Connection yet, click on the “Set up a New Connection” button and follow the prompts to authenticate your Connection.

  2. After selecting your connection, add a local alias for the connection that you can call from your Execute Tasks.

  3. When you are finished, click Save Changes to save your Task.