How can one make Salesforce Bulk API calls via simple_salesforce?

I'm using the module simple-salesforce, and I'm not seeing anything in the docs about making bulk API calls. Anybody know how to do this?

https://github.com/simple-salesforce/simple-salesforce

Solution

The code does have some comments. There's also this readthedocs page but, even that looks like it could use some help.

Good stuff first, explanation below.

Code example (written assuming you're running the whole block of code at once):

from simple_salesforce import Salesforce

sf = Salesforce(<credentials>)

# query
accounts = sf.bulk.Account.query('SELECT Id, Name FROM Account LIMIT 5')
# returns a list of dictionaries similar to: [{'Name': 'Something totally new!!!', 'attributes': {'url': '/services/data/v38.0/sobjects/Account/object_id_1', 'type': 'Account'}, 'Id': 'object_id_1'}]

# assuming you've pulled data, modify it to use in the next statement
accounts[0]['Name'] = accounts[0]['Name'] + ' - Edited'
# update
result = sf.bulk.Account.update(accounts)
# result would look like [{'errors': [], 'success': True, 'created': False, 'id': 'object_id_1'}]

# insert
new_accounts = [{'Name': 'New Bulk Account - 1', 'BillingState': 'GA'}]
new_accounts = sf.bulk.Account.insert(new_accounts)
# new_accounts would look like [{'errors': [], 'success': True, 'created': True, 'id': 'object_id_2'}]

# upsert
accounts[0]['Name'] = accounts[0]['Name'].replace(' - Edited')
accounts.append({'Name': 'Bulk Test Account'})
# 'Id' is the column to "join" on. this uses the object's id column
upserted_accounts = sf.bulk.Account.upsert(accounts, 'Id')
# upserted_accounts would look like [{'errors': [], 'success': True, 'created': False, 'id': 'object_id_1'}, {'errors': [], 'success': True, 'created': True, 'id': 'object_id_3'}]

# how i assume hard_delete would work (i never managed to run hard_delete due to insufficient permissions in my org)
# get last element from the response.
# *NOTE* This ASSUMES the last element in the results of the upsert is the new Account.
#  This is a naive assumption
new_accounts.append(upserted_accounts[-1])
sf.bulk.Account.hard_delete(new_accounts)

Using simple_salesforce, you can access the bulk api by doing

<your Salesforce object>.bulk.<Name of the Object>.<operation to perform>(<appropriate parameter, based on your operation>)

<your Salesforce object> is the object you get back from constructing simple_salesforce.Salesforce(<credentials>)
- <credentials> is your username, password, security_token, and sandbox(bool, if you're connecting to a sandbox) or session_id. (these are the 2 ways that i know of)
<Name of the Object> is just Account or Opportunity or whatever object you're trying to manipulate
<operation to perform> is one of the below:
- query
- insert
- update
- upsert
- hard_delete (my account did not have appropriate permissions to test this operation. any mention is pure speculation)
<appropriate parameter> is dependent on which operation you wish to perform
- query - a string that contains a SOQL
- insert - a list of dictionaries. remember to have a key for all fields required by your org when creating a new record
- update - a list of dictionaries. you'll obviously need a valid Object Id per dictionary
- upsert - a list of dictionaries and a string representing the "external id" column. The "external id" can be the Salesforce Object 'Id' or any other column; choose wisely. If any dictionary does not have a key that is the same as the "external id", a new record will be created.
What's returned: depends on the operation.
- query returns a list of dictionaries with your results. In addition to the columns your query, each dictionary has an 'attributes' key. This contains a 'url' key, which looks like it can be used for api requests for the specific object, key and 'type' key, which is the type of the Object returned
- insert/update/upsert returns a list of dictionaries. each dictionary is like {'errors': [], 'success': True, 'created': False, 'id': 'id of object would be here'}

Thanks to @ATMA's question for showing how was using query. With that question and the source code, was able to figure out insert, update, and upsert.