Page Comparison

You can import data from REST API sources which return JSON, XML or CSV data. Data import is similar to Excel and CSV file upload but you do not need to export source data to file and upload to flex.bi. In addition you can schedule regular daily import from provided REST API data source.

...

• Select Use incremental import to enable it.
• If REST API does not provide parameters for selecting just the recent updated data then you can use the option "Stop incremental import when no source data are changed in a returned page". If during the import some REST API results page will have all the same data that have been imported before then the import will be stopped. But please be aware that if data are changed when stored in flex.bi (e.g. decimal numbers are truncated when stored) then the source data page will not be exactly the same as in flex.bi.

• Therefore it is recommended to specify Additional URL parameters to limit just the recently updated source data. Typically you need to specify a URL date parameter that limits results to recently updated source data.
  For example, in our GitHub issues example REST API has since parameter to return only issues updated at or after the specified date and time. We can use the following additional URL parameter:
  since={{

  3

  days

  ago

  |

  %Y-%m-%d

  }}
  

  This will dynamically get a relative date and time 3 days ago and will format it using a strftime format string %Y-%m-%d (see available strftime format options). The following relative time units can be used – years, months, weeks, days, hours, minutes, seconds. And instead of ago also from now can be used to get a date and time in future.
  

If incremental import is used then it will be required to specify a Source ID column in the source columns mapping step. Source ID column value should provide a unique results row identifier. It is used to identify when some existing imported rows in flex.bi should be replaced with updated source data during the incremental import.

...

Here is example of JavaScript code which will change title property to capitalized version (first capital letter and then lowercase letters):

function capitalize(s) {
  return s.charAt(0).toUpperCase() + s.slice(1).toLowerCase();
}
doc.title = capitalize(doc.title);

If you would like to skip some data rows and do not import them in flex.bi then use return false; in these cases. Here is example which will skip data rows which do not have title property:

if (!doc.title) return false;

You can also create new properties for the doc object when you need to construct flex.bi dimension level names or calculate additional measures. Here is example how to create full_name property:

doc.full_name = doc.first_name + " " + doc.last_name;

Here is an example how to create a measure that counts data source rows (returns value 1 for each row):

doc.count = 1;

You can map one source data row to multiple data rows that should be imported into flex.bi. For example, if doc is an invoice object with several lines then you can return an array of invoice lines which contains both invoice and line attributes:

return _.map(doc.lines, function(line) {
  return {
    invoice_number: doc.number,
    invoice_customer: doc.customer,
	invoice_date: doc.date,
    line_product: line.product,
    line_item_count: line.item_count,
    line_item_amount: line.item_amount
  };
});

You can use Underscore.js functions (like _.each) in your custom JavaScript code.

...