You can import data from REST API sources which return JSON, XML or CSV data. Data import is similar to Excel and CSV file upload but you do not need to export source data to file and upload to flex.bi. In addition you can schedule regular daily import from provided REST API data source.
...
Go to Source Data tab and Add new source application and select REST API application type.
If you have created already another similar REST API data source then you can export its definition and paste it in Import definition to create new REST API source application with the same parameters.
...
In the next step you will need to provide REST API source parameters which will be used to retrieve the data. See example:
In Source data URL provide REST API URL which returns the data. In this example, we are using GitHub issues REST API to get all issues from https://github.com/rails/rails/issues. In addition, we have added parameters per_page=100 (to indicate that we want to get 100 issues per page) and state=closed to indicate that we want to get just closed issues.
...
In this example GitHub uses OAuth 2.0 for their REST API authentication. At first we need to register flex.bi application in our GitHub account settings / Applications:
Please specify there Authorization callback URL as Redirect URL that you see in flex.bi REST API source parameters page.
...
If REST API supports additional parameters to return just the recently updated data then you can use the incremental import option to update just the recent source data after the initial full import. Click Incremental import parameters to specify additional parameters:
- Select Use incremental import to enable it.
- If REST API does not provide parameters for selecting just the recent updated data then you can use the option "Stop incremental import when no source data are changed in a returned page". If during the import some REST API results page will have all the same data that have been imported before then the import will be stopped. But please be aware that if data are changed when stored in flex.bi (e.g. decimal numbers are truncated when stored) then the source data page will not be exactly the same as in flex.bi.
Therefore it is recommended to specify Additional URL parameters to limit just the recently updated source data. Typically you need to specify a URL date parameter that limits results to recently updated source data.
For example, in our GitHub issues example REST API hassinceparameter to return only issues updated at or after the specified date and time. We can use the following additional URL parameter:since={{ 3 days ago | %Y-%m-%d }}This will dynamically get a relative date and time 3 days ago and will format it using a strftime format string %Y-%m-%d (see available strftime format options). The following relative time units can be used – years, months, weeks, days, hours, minutes, seconds. And instead of ago also from now can be used to get a date and time in future.
If incremental import is used then it will be required to specify a Source ID column in the source columns mapping step. Source ID column value should provide a unique results row identifier. It is used to identify when some existing imported rows in flex.bi should be replaced with updated source data during the incremental import.
...
In case of OAuth authentication in the first time you will be redirected to source application authorization page where you will need to authorize flex.bi access to source application data:
After successful authorization you will be redirected back to flex.bi Source columns mapping page. If there will be any authentication errors then you will see corresponding error messages - please review your authentication settings and try again.
...
See example of GitHub issues columns mapping:
When you need to import several columns as the same dimension attributes then click Show options and specify the following options:
...
If source columns mapping was saved without any validation errors then REST API source application will be queued for background import. You will see the updated count of imported rows during the import:
And after successful import it will automatically refresh the status of source application:
You can later visit Source Data tab again and click Import button again to import the latest data from REST API source. During each import it will at first delete all data that were imported previously from this source and then import new data. In addition you can also click Delete data to delete imported data from this source (you need to delete imported data also if you want to change source columns mapping).
...
As it was mentioned in the beginning you can export REST API source application definition:
and copy this definition and paste in Import definition field when creating different source application to create a new copy of this definition which can be modified later.
...