The Salesforce Bulk API is optimized for loading large sets of data asynchronously by submitting batches of data that are processed in the background. It’s possible to use the Bulk API from the Data Loader using CSV files, however handling large CSV files and dealing with errors during loading can be quite daunting even for couple thousand records.
- sfdc:upsert-bulk service call to submit the records to be uploaded
- sfdc:batch-result service call to asynchronously check bulk upload completion
- basic exception handling that retries the completion check (since the connector throws an exception when the batch is not completed)
- processing the returned BatchResult object to match the results with each data record