6.5.2 Connecting to Web

Data Connector builds connection between Origin worksheet and data file from an online path or a table on a webpage. Once the data source is changed, you can simply reimport the data to get everything update.

To open Data Connector

  1. With a worksheet/matrix active, select menu Data: Connect to Web and further choose the desired file type. Many data types, including CSV, JSON, HTML table, Origin project, etc., are supported. You can also install the data connector App of new type from ‘’’Add New’’’ menu.

The Connect to Web dialog will open.

Basic operation

Data connector from web basic.png

  1. On the Basic tab, enter or paste URL link in the URL edit box. The URL Preview box below shows the full url text. You can predefine LabTalk variables and then use the use them for substitutions in the URL link. The LabTalk variables must be percent encoded, such as something like
    https://.....%(var$)...
    And you can see the actual URL string in the URL Preview.
    Data connector from web lt variable.png
  2. Click OK to connect to the web server.
  3. In the Selection dialog, select the desired table and click down-arrow button to add it to the bottom panel. Click OK to import

Advanced operation

Some Data Connector web requests may be more complex and will require specifying Advanced settings in the Connect to Web dialog.

For example, the following URL consists or a series of variables in the form of path components or parts.

https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/cid/2244

In this case the path components can be broken down into:

https://pubchem.ncbi.nlm.nih.gov/rest/pug
/compound
/cid
/2244

While it is not always strictly necessary to break down the URL into different parts, there may be times when it is appropriate to do so for clarity such as in the case above. In the Connect to Web dialog, those parts would be specified as illustrated below:

Data connector from web advance2.png

Observe that the Add Part button will add a new group of controls for specifying URL Parts. Also observe that, for each part, there are up and down arrows for changing the order of the parts along with a × button for removing the part.

Note: LabTalk substitution is supported for URL parts.

The optional Command timeout in minutes control allows the user to specify the number of minutes after which the request for data will abort. If left blank (which is generally recommended), the request will take as long as necessary to complete or until the server decides to abort the request itself.

The optional HTTP request header parameters control section is most likely the reason for using the Advanced settings. In addition to a URL, some web service APIs may require one or more arbitrary HTTP request headers be provided with the request. This is highly dependent on the requirements of the web service- there are no set rules in this regard.

Selecting from a drop down list of common request header names and then clicking the Add button generates two text box controls, one for the header name and one for the header value. Custom headers may also be added by choosing --- Custom --- from the drop down or simply overwriting the name of a header already added.

For the URL provided in this particular example, an Accept header is required and it's value must be set to the media type for the type of data to be return. In this case, it is application/json.

Note: A request header named Authorization is included in the list. Do not assume that this header is automatically a password! Not all web services require this header and it is up to the web service itself to define what the value should be. There are no set rules with regard to what constitutes a proper value.

Examples

Example 1

tutorial of Origin Web Data Connector in 2019b

Example 2

Open Signal Processing - FFT Gadget with Data Connector.opju to see an example of connecting web file for sunspot data:

  1. Select Help: Learning Center.
  2. In the dialog, activate Analysis Samples tab. Select Signal Processing from Samples in drop down list.
  3. Double click on FFT Gadget with Data Connector to open it.