REST Action
  • 10 Feb 2021
  • 12 Minutes To Read
  • Print
  • Share
  • Dark
    Light

REST Action

  • Print
  • Share
  • Dark
    Light

What are Actions in Rivery?

Actions are the ability to connect to anywhere on the web.

The REST Action is the place to send REST requests in order to get data from Web services (APIs etc.) or post data in Web services.

Create a REST Action

Click on “Create new river” from any screen in Rivery and select the “Action” option.

image.png

Insert a name for the river and proceed to the next step called “Action Steps”.


image.png


An action in Rivery can be Multi-Action or a single REST Action.

REST Action-image33

  • Multi-Action is an action which is composed of multiple REST Actions, which allows for a single workflow with multiple endpoints, and the passing of values between endpoints.
  • REST Action is a generic process which can be configured to send any kind of REST request. 

Click on Rest Action in order to proceed to the REST Action screen.

The REST Action is composed of two major parts: Request & Results.

In addition, the REST Action has its own Custom Connection, its own Variables and Time Period inputs.

image.png

  • Request Configuration has options to insert all the inputs related to the request of the REST Action such as the URL, parameters, headers and more.
  • Results Configuration contains the options for handling the results of the REST request.
  • Custom Connection is an optional input to define the connection of the action. This connection can be reused in multiple Actions. All the details of the selected custom connection will be added to the variables of the action and can be encrypted in the connection.
  • Variables menu allows for defining variables to be used in the Action. Variables can be used in any input in the Action.
  • Time Period allows for defining a time range of the rest action. Select the type of the dates (datetime or date), and select the time format which is required for this request. Once a time range is defined, corresponding variables {time_range.start_date} and {time_range.end_date} will appear in the Variables menu. 


Custom Connection

Create or select an existing custom connection.

It is possible to leave the connection empty in case it is not required.

The keys in the connection can be encrypted by using the Is Password checkbox:


Request Configuration

The Request section contains all the required settings for sending a proper REST request.

In any of the inputs, it is possible to use the variables that were configured in the Variables menu.

  • API URL: The HTTP address of the service. This should start with “http://” or “https://”. If the request has parameters, these can be managed in the Params inputs. 

REST Action-image21

  • REST method of the request. Rivery currently supports in GET, PUT, POST, PATCH, and DELETE requests.
  • Params - Click on the Params button in order to add parameters to the request.
    image.png
    Click on the Add button to add required parameters and their corresponding values. Important: The key of each parameter must be equal to the name of the parameter that the service is expecting.
    The value of each parameter is the exact value which will be sent in the request. It is possible to use  variables as values.
  • Authentication type - In some requests it is required to add an authentication method to the request. Rivery currently supports Oauth2.0, Basic Auth, and No Auth. Use the 'No Auth' option if the API only requires some sort of authentication through the Headers, Parameters, or in the API URL.
  • Headers - The headers that will be sent in the request. (Leave this empty if no headers are required). 
  • Request Body - The body that will be sent in the request. Text must be in JSON format.

  • Advanced Options - Options which are not mandatory when sending a REST request but can be useful in some cases:
  1. Sleep after request - declare if and for how long Rivery will wait after sending the request.
  2. Timeout request - declare how long to wait for a response. If the given time is passed the Action will return an error. If the input is empty, the default waiting time is 300 seconds.

Testing the Request

Once the request is configured, try the “Test Rest Action” button to determine if the request returns a successful response. This will send the request with all the configurations inserted in the in the request section and will return the response that was received for this request in the user interface.

If there is an error in the response it will be shown with the corresponding error code and error message received from the API.

Results Configuration

The Results section contains all the required settings about how to get the results from the response that was received from the REST request.

Rest Actions can return one of the following types of results:

  • Response - The action will only return the response code for the given request. If selected, no options will be shown except the advanced options. This option is most commonly used with POST and PUT calls.

  • Variables - The action will return variables which stores the values that were returned in the request’s response. This kind of action can be used inside a Multi-Action or a Logic River

  • Data - The action will return data which can be loaded into a Rivery supported target. This is most commonly used with GET calls, and requires creating an additional Data Source to Target river, with the source set to 'REST API'. For this use case the Action is used as a template for the Source to Target ingestion.

Data Location

image9.png

Insert the key in the response where the desired results are located. If the required data is the whole response, leave the data location empty. If the data is located in a nested key, use ‘.’ to indicate it.

For example, If the response looks like this and the desired results are in the 'data' array:

 {‘results’:{‘data’: [....], ‘status’: ‘success’}}

The data location would be “results.data” to obtain only the response data located in the 'data' array.

Important
The Data Location input uses standard JSON Path format as outlined here: https://github.com/json-path/JsonPath#operators.

Data Position

image14.png

If the desired response is a JSON array and the required results are located in one of the elements, data position should be set to the position of that item (starting from 0). If this is not the use case then leave this option empty.

Variables to return (relevant only if the Results Type is set to Variable)

  1. Click on “Add” to add variables to return.

  2. Name of the output variable to return - Insert the desired name of the variable.

Key in the response - Insert the key in the response that contains the value to be stored in the variable. Leave this empty to store the entire response into the output variable.

Use ‘.’ notation if the value for this variable is located in a nested key in the response.

Example:

We would like to send a request to generate a token and return this token as a variable called new_token.

The response is:

{"auth_res": {"token": "abcde12345", "expired_at":"2017-07-01 22:00:00"}}

In the data location input, we’ll insert auth_res (because the data we are interested in is located inside auth_res).

In the variables to return we’ll add a new variable with key: new_token and value tokenResults Details - Relevant only for 'Data' result type. 
Select the results structure -
JSON - If the data is in JSON structure (meaning a list of rows when each row is an object of key-value pairs)
CSV - If the data is in a table structure. CSV can be the whole response if the response is just text or can be split to a key in the response which stores the heading and values separately.

1. JSON Structure:

REST Action-mceclip1

If the response is returned as JSON data, but the results data are in a CSV structure (as in the example below the toggle button in the screenshot) it is necessary to indicate where the headings of the data are located (fill the key of the JSON response - for example, headers ).

Note that this option requires filling both 'Headers Location' and 'Data Location' inputs.

2. CSV Structure:

image.png

If CSV is selected, it is necessary to indicate where the headings of the data are located. If the headers are in the first row of the data, keep the 'First row in values is headers' toggle enabled. If the headers are in different row, disable this toggle and fill in the headers row number.


Pagination Details

* Relevant on Data result type only

Some services return the data in pages (also knows as chunks) and require extra logic for 'paging' through the results. Pagination is the mechanism for pulling all of the results from the service if it is returned in pages.

Important:

The selected pagination will be in actual use only when running the action from a REST Source to Target River. When running the action, it will return only the first page as a sample of the data.


Pagination Methods

The REST Action supports the following pagination methods:

image18.png

Paginate by offset

image22.png

This pagination method expects that there is some way to get data based on an 'offset', or sending in a value for the starting point of the pull.

  1. Offset start - The start position for pulling the data (typically this is 0 or 1).

  2. Offset key - The name of the key where to send the offset in the request (most of the time called 'offset').



  3. page_offset variable

    Enable the page_offset variable option when the REST API request requires different usage of an offset value than the typical 'offset' = N in a request URL parameter.

    For example - the REST request should be sent as POST method and the offset variable has to be sent in the request body. Use the page_offset variable in the request body with {page_offset} and the river will replace it with the offset value when iterating over pages.

    image.png

    Paginate by page

    image17.png

    This pagination method splits the response data into pages, and each request contains the page number to pull, usually starting from 1.

    Page key - The name of the URL parameter to update (usually called 'page').

    page_value variable

    Enable the page_value variable option when the REST API request requires the page value to be updated somewhere other than a URL parameter (such as the body or just an increasing number in the URL itself).   

    image.png

    Next page in response

    Each response contains the details of the next page to pull.

    In this method, we need to indicate where to find this information in the response.

  • Next Page Type

1. Static URL with param from the response

The response contains some value to be sent in the next request in order to pull the next page. The next request can be sent with the initial URL or with a new URL.

image28.png

Next page key - the key in the response which contains the value to be sent in the next request.

Send next page in key - the parameter’s name in the next request to send the value of the next page.

Next page in new URL - The URL to send the pagination requests with. Leave empty if using the same URL.

  • Url in response

The response contains a full URL ready to be sent to get the next page’s data.

image.png

Next page key - the key in the response that contains the URL of the next page.

Stop according to

Select how to determine when the paging will end. 

image26.png

  1. Page size - the pagination stops when the page size (in records) is smaller than the value that was inserted to the input of page size. 

    image15.png

Key in response - the pagination will stop when the response contains a given value in a given key.

image27.png


1. Insert the name of the key in the response which is supposed to contain the value. If the data is located in a nested key, use ‘.’ to indicate it.
2. Insert the value that should cause the paging to stop.

Sleep after each page
Hold the process for the given amount of seconds.
It is recommended to use that input in order not overload the service with too frequent requests when using pagination.
image12.png

Results Advanced Options

Replace invalid character in headers

Add Dates Variables to Results Data

This option will take the values passed through Time Period variables ({time_range.start_date} and {time_range.end_date}) and automatically populate them in the Column Mapping of the Data Source to Target river when the Action is set as the Source. This is especially useful for use cases where dates are required input parameters but the corresponding response data does not include the date values.


Additional Advanced Options:

Insert the number of seconds to hold the process. Insert the name of the key in the response that is supposed to contain some value. Insert the value that should cause the process to hold.

image5.png

  • Insert the number of seconds to hold the process. Insert the response code that should cause the process to hold.

image.png

  • Insert the key in the response that is supposed to contain the value. Insert the value that should cause the process to exit.

image.png

  • Insert the response code that should cause the process to exit.

image.png

Keys to monitor

image11.png

Insert any key in the response that you’d like to monitor during the Action’s run.

If the Action found that key in the response it will store its value and will return it in the action results and also in the activities screen (when opening the action details).

The keys to monitor can be useful when the response might contain messages about errors or the runs.

In that case, if the action failed, thanks to the “keys to monitor” you’ll have some information about the error when opening the action results in the activities screen.

Testing the results

image72.png

Once the results are configured, it is a good time to test the REST Action.

This test will consider all of the configuration of the Action results, such as data location and position.


Variables

The Variables menu is available on the top right of the Actions screen.
Insert as many variables as required. These can be used as placeholders for dynamic values to be sent to the Action by a Logic River. In a Logic Step, an Action can be called and any variables defined will show up as possible input parameters.

Time Period

The time period allows for defining a start date and end date of the request and is available by selecting the Time Period menu in the top right of the Action screen:

REST Action-image20

These two parameters will be automatically added to the variables list of the action and can be used anywhere in the action (usually will be sent as parameters in the request).

The time format input decides how those dates, when used in the action, will be sent in the request - this format should be the expected format of the service which the request is being sent to.

If the required format is not in the list of the time formats, it is possible to insert manually the required time format according to python syntax.

REST Action-image13

Creating a dynamic date range

In order to pull a dynamic date range, select 'Time Period' on the top right of the Action screen and define the date format and initial start and end values:

  1. Set the date period:
    image.png

  2. This will add two new parameters in the variables section:

  1. Create a new Data Source to Target river and choose the REST API as the source. Then choose the desired date range:
    image.png

Rest Action That Returns Data

If the Action is meant to get data into a Rivery supported target, this will require the creation of an additional Data Source to Target river, with the source connection set to 'REST API'.

In the Source tab of the Data Source to Target river, select the corresponding REST Action as the source. If any Time Periods or Variables were used as part of the configuration, these will be made available in the Source tab.

Finalize the REST Action

Once the test of the Action returns the expected results, the Action is ready to be used. 

The Action created can be used in any of the following rivers in Rivery:

  1. Data Source to Target River (only if this REST Action has “data” results type)

  2. Multi-Action River

  3. Logic River

Was This Article Helpful?