Page Comparison

To use REST API as a data source you have to do the following steps:

...

If you want to return a certain amount of data use Offset and limit. The limit tells REST API how many records to return and offset tells REST API from which record it should start getting the data. If offset is est to 20 and the limit to 30, REST API will return records from 20-50

It is recommended to use page or offset and limit parameters if possible as it will enable concurrent REST API requests and will make the import faster. By default, up to 10 concurrent REST API requests will be made at the same time. Starting from the flex.bi version 4.3.0 you can adjust the maximum number of concurrent requests using the Concurrency parameter (for example, reduce it if too many concurrent requests are causing errors in the source application).

Authentication parameters

...

• Select Use incremental import to enable it.
• If REST API does not provide parameters for selecting just the recent updated data then you can use the option "Stop incremental import when no source data are changed in a returned page". If during the import some REST API results page will have all the same data that have been imported before then the import will be stopped. But, please, be aware that if data are changed when stored in flex.bi (e.g. decimal numbers are truncated when stored), then the source data page will not be exactly the same as in flex.bi.

• Therefore it is recommended to specify Additional URL parameters to limit just the recently updated source data. Typically you need to specify a URL date parameter that limits results to recently updated source data.
  For example, in our GitHub issues example REST API has since parameter to return only issues updated at or after the specified date and time. We can use the following additional URL parameter:
  

  Code Block
  since={{ 3 days ago | %Y-%m-%d }}

  This will dynamically get a relative date and time 3 days ago and will format it using a strftime format string %Y-%m-%d (see available strftime format options). The following relative time units can be used – years, months, weeks, days, hours, minutes, seconds. And instead of ago also from now can be used to get a date and time in future.

If incremental import is used then it will be required to specify a Source ID column in the source columns mapping step. Source ID column value should provide a unique results row identifier. It is used to identify when some existing imported rows in flex.bi should be replaced with updated source data during the incremental import.

...