Wednesday, July 24, 2019

#724 OIC - Salesforce Adapter - Bulk API

As the name suggests -
It allows one to perform Bulk Data Operations:
Inserts, updates, upserts, or deletes of a large volume of records.

For those interested, the SFDC Bulk API Doc is here

An extract from the aforementioned -

Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, queryAll, insert, update, upsert, or delete many records asynchronously by submitting batches. Salesforce processes batches in the background.

SOAP API, in contrast, is optimized for real-time client applications that update a few records at a time. You can use SOAP API for processing many records, but when the data sets contain hundreds of thousands of records, SOAP API is less practical. Bulk API is designed to make it simple to process data from a few thousand to millions of records.

The easiest way to use Bulk API is to enable it for processing records in Data Loader using CSV files. Using Data Loader avoids the need to write your own client application.

Data Loader is an app you can download from your Salesforce instance -

Leveraging the Bulk API via OIC -

First, an extract from the Oracle SFDC  adapter docs -

The Salesforce Bulk API enables you to handle huge data sets asynchronously with
different bulk operations. For every bulk operation, the Salesforce application creates
a job that is processed in batches.
A job contains one or more batches in which each batch is processed independently.
The batch is a nonempty CSV/XML/JSON file that is limited to 10,000 records and is
less than 8 MB in size. Because the batches are processed in parallel, no execution
order is followed. A batch can contain a maximum of 10,000,000 characters in which
5,000 fields in a batch are allowed with a maximum of 400,000 characters for all its
fields and 32,000 characters for each field.

Here is a simple example where I will read a file of Leads from an ftp server and leverage the SFDC Bulk API to insert them into SFDC.

BulkLoadLeads is configured as follows -

I upload a test csv file to my ftp server -

As this is a scheduled orchestration, I click Submit Now -

As you can see, this is the second run -

I go to Monitor Bulk Data Load Jobs in Salesforce -

No comments: