Getting Started

Welcome

Welcome to Flex.io!

Flex.io is an API for building and deploying feeds to deliver data—with just a few lines of code.

These data feeds are defined using Flex.io pipes. Once defined, pipes can be scheduled or triggered to run with an API request, webhook or by sending an email.

What is a Pipe?

A pipe is a collection of tasks that run sequentially. These tasks can read and write from data sources like cloud storage, databases and APIs using connections. Tasks can also process data, including converting file formats, transforming data and running custom Python or Javascript code on content being passed through the pipe.

In this getting started guide, we’ll show you how to build and run your first pipe. Then we’ll point you to additional resources to help you create your own custom data feeds.

Get Help

Got more questions? Check out our FAQ for some quick answers. And, feel free to message us from the icon in the lower-right of this page; we’re happy to help.

Ok, let’s get started!

Build your First Pipe

To get you started, we’ll create a simple pipe that gets some data and peforms a process on it:

  1. Request a CSV file from a URL.
  2. Convert the CSV file into JSON format.

Once you create and save this pipe, you can deploy it. For example, you might trigger a pipe like this any time you need to utilize a converted CSV file.

In Flex.io, there are two ways to save and run a pipe defition, through the Flex.io Web App or programmatically using the API. We’ll show you how to build a pipe using each method.

Create a Pipe via Web App

The simplest way to demonstrate building and deploying a pipe is to copy and paste some code into the web-based pipe builder.

Build the Pipe

First, sign in to Flex.io, navigate to the “Pipes” tab and you’ll see a sample list of pipes:

Getting Started: Pipe List

Click on the “New Pipe” button in the upper right corner to access the Pipe Builder:

Getting Started: Pipe Builder New

In the builder, give your pipe a Name, like “CSV to JSON Endpoint”. Also give your pipe an Alias like “json-endpoint” (we’ll use this in just a second).

Then, in the Configuration area, paste in the following code snippet:

Your builder area should now look something like this:

Getting Started: Pipe Builder with Definition

Run the Pipe

The builder uses the Flex.io Javascript SDK syntax to define pipes. In this case we’re using a request task to read a URL of sample contact names and then a convert task to convert the CSV to JSON.

To run the pipe, click the “Run” button. Now, after running, you should see a response in JSON like this (note: for comparison, see the original CSV file here):

[
{
"id": "1",
"first_name": "Kimbra",
"last_name": "Pettigrew",
"email": "kpettigrew0@dedecms.com",
"gender": "Female",
"ip_address": "253.123.30.10"
},
{
"id": "2",
"first_name": "Cornall",
"last_name": "Rowlett",
"email": "crowlett1@sitemeter.com",
"gender": "Male",
"ip_address": "223.20.241.246"
},
{...}
]

Save and Deploy the Pipe

Congratulations, you’ve defined your first pipe!

Now we just need to save the definition and call it. To save, click the “Save” button in the upper right of the builder. This will save your pipe to the Pipe List.

You can deploy your pipe in a myriad of ways, but let’s simply call this via cURL.

Next to your API Endpoint, click on the “How do I deploy this pipe?” link. One of the options will be a cURL command that will look something like this (but will include your own API key):

curl -X POST 'https://api.flex.io/v1/me/pipes/json-endpoint/run' -H 'Authorization: Bearer {YOUR-API-KEY}'

Fire this off in cURL and, as expected, you’ll see the same JSON as you saw from within the builder.

Create a Pipe Programmatically

While it is sometimes nice to build definitions in the Flex.io Web App, it is often more useful to define pipes via code. In this section we’ll show you how to define and run the same pipe as discussed above, but using the Flex.io REST API.

Build the pipe

To build a pipe programmatically, you’ll first need your API key. To get your API key, do the following:

  1. Sign up or sign in if you already have an account.
  2. Copy the API key from your Dashboard.
  3. You can also get and manage your API key from your Account.

Next, create the pipe with an HTTP Request in CURL, using your API in place of {YOUR-API-KEY}:

Run the pipe

Once we’ve created the pipe, we can run it with an HTTP POST request in CURL as follows:

curl -X POST 'https://api.flex.io/v1/me/pipes/csv-to-json-endpoint/run'
--header "Authorization: Bearer {YOUR-API-KEY}"

Here, the pipe is referenced in the API endpoint using the alias csv-to-json-endpoint which we gave to it when we created the pipe.

Save and Deploy the Pipe

When pipes are created from the API, they are automatically saved.

Deploying the pipe in code works the same as when we ran it: simply invoke the pipe with an HTTP request.

Learn More

Congratulations, you’ve created, run and deployed your first pipe! While simplistic, this pipe demonstrates the end-to-end nature of a pipe:

  1. Get files (or data).
  2. Process them.
  3. Deliver them.

Now that you have the basics, you can dive into the additional resources below to build more complex pipes. And, again, should you have any questions at all, don’t hesitate to ask!

Reference Material

Flex.io API reference docs can be found here:

Furthermore, to take a deeper dive into the core concepts of Flex.io data feeds, you can take a look at the following guides:

Tutorials, Template & Examples

In addition to the reference material above, you can also take a look at some real-world use cases. Tutorials will provide step-by-step details on how to build various data feeds. Templates provide a simple, graphical approach to setting up data feeds, with a focus on quickly setting up connections. And, finally, examples provide a variety of starter data feed code snippets that you can copy and paste into your own code.