Big Object in Salesforce

Salesforce has a limitation on performing queries on a large number of records. Especially when your data is massive (assume more than 10k records per object) performing queries and performance of the system downgrades. That is where Big Object in Salesforce Comes into the picture. Let’s learn what Big Object is in Salesforce and Big Object implementation guide.

Leveraging Big Objects we can perform asynchronous queries to retrieve huge sets of data, without affecting the performance in a faster way. Big Objects can help you improve performance and scale your Implementation when your Data is huge in Salesforce!

What is Big Object in Salesforce?

A big object that stores and manages massive data values within Salesforce without affecting performance. Salesforce Big objects provide consistent performance for a billion records, and are accessible with a standard set of API’s to your org or external system.

Custom big objects can be created by the Metadata API. To define a custom big object, you need to create an object file that contains its definition, field’s, and index, with a Permission-set to define the permissions for each field, and package file to define the contents of the object metadata.

When to use Big Object in Salesforce?

Here are some reasons or use cases when we can use Big Object in Salesforce.

Capture User Activity: Code reviews, time entries, page views, field audits etc.
Retain Historical Data: Historical data is stored for compliance.
360 Customer View: Ancillary customer data e.g. Purchase Details, Transactions.

Deploying your Schema

Big Object in Salesforce Video

How to create big objects in Salesforce?

You need to create the below three files.

1. Create Big Object Files

1.1. Create an Object File:- Defines the custom big object’s fields and index, Each custom big object needs its own object file.

My_First_Big_Object__b.object

<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
    <deploymentStatus>Deployed</deploymentStatus>
    <fields>
        <fullName>MyTextField__c</fullName>
        <label>My Text Field</label>
        <length>16</length>
        <required>false</required>
        <type>Text</type>
        <unique>false</unique>
    </fields>
    <fields>
        <fullName>Account_Lookup__c</fullName>
        <label>Account Lookup</label>
        <referenceTo>Account</referenceTo>
        <relationshipName>Account_Lookup</relationshipName>
        <required>true</required>
        <type>Lookup</type>
    </fields>
    <fields>
        <fullName>Date_Time__c</fullName>
        <label>Date Time</label>
        <required>true</required>
        <type>DateTime</type>
    </fields>   
    <fields>
        <fullName>Number_Field__c</fullName>
        <label>Number Field</label>
        <required>false</required>
        <type>Number</type>
        <scale>2</scale>
        <precision>16</precision>
    </fields>
    <indexes>
        <fullName>MyFirstBigIndex</fullName>
        <label>My First Big Index</label>
        <fields>
            <name>Account_Lookup__c</name>
            <sortDirection>DESC</sortDirection>
        </fields>
        <fields>
            <name>Date_Time__c</name>
            <sortDirection>DESC</sortDirection>
        </fields>
    </indexes>
   
    <label>My First Big Object</label>
    <pluralLabel>My First Big Object</pluralLabel>
   
</CustomObject>

1.2. Create A Permissionset file:- with a Permissionset to define the permissions for each field.

My_First_Big_Object.permissionset

<?xml version="1.0" encoding="UTF-8"?>
<PermissionSet xmlns="http://soap.sforce.com/2006/04/metadata">
    
    <fieldPermissions>
        <editable>true</editable>
        <field>My_First_Big_Object__b.MyTextField__c</field>
        <readable>true</readable>
    </fieldPermissions>
        
    <fieldPermissions>
        <editable>true</editable>
        <field>My_First_Big_Object__b.Number_Field__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <label>My_First_Big_Object Permission Set</label>
</PermissionSet>

1.3. Create package.xml file:- file to define the contents of the object metadata

package.xml


<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>*</members>
        <name>CustomObject</name>
    </types>
    <types>
        <members>*</members>
        <name>PermissionSet</name>
    </types>
    <version>41.0</version>
</Package>

2. Deploy big Object

2.1 Create the zip file for deployment.

Save all three file in one folder. Create the folder Structure like below.

Add My_First_Big_Object__b.object file in Object Folder
Add My_First_Big_Object.permissionset file in permissionset Folder
package.xml file should in root of the folder.

2.2 Deploy with the help of Workbench or ANT Tool.

2.2.1 Login on workbench

2.2.2 Click on the Migration tool in the workbench and select the Single Package option like below image.

2.2.3 Now validate the big object in your Salesforce org.

How to insert a record

My_First_Big_Object__b obj = new My_First_Big_Object__b();
obj.Account_Lookup__c ='0019000001ukhMF';
obj.Date_Time__c=System.now();
obj.MyTextField__c='hello';
obj.Number_Field__c=10;
database.insertImmediate(obj);

Big Object Data Manipulation

Let’s understand what all we can do with big Object

Apex CRUD
- Create / Update (Idempotent Behavior)
- insertImmediate(sobject) OR insertImmediate(sobjects)
Read
- SOQL Queries
- Async SOQL
CSV Files
- API (Bulk API, SOAP API)

Using Standard SOQL with Big Objects

Executes synchronously.
All Indexes are mandatory.
Comparison Operators (=, <, >, <=, >=, IN).
Not Supported Operators.
(!=, LIKE, NOT IN, EXCLUDES, INCLUDES).

SOQL Vs Async SOQL Usage Considerations

Feature	Standard SOQL	Async SOQL
Mode of Execution	Synchronous	Asynchronous
Immediate Response Required?	Yes	No
Expected Result Set Size	Smaller Data Sets (Thousands of records)	Large Data Sets (Millions of records)
Best Suited For	Displaying Data on UI Manipulations within Apex	Aggregation Summarizing for Analytics
Filter using Non Index fields	Yes	No
Sample Format	SELECT Game_Platform__c, Play_Date__c FROM Customer_Interaction__b WHERE Game_Platform__c=’PC’ AND Play_Date__c=’2017-09-06′	{“query”: “SELECT Account_c, In_Game_Purchase__c FROM Customer_Interaction__b WHERE Play_Date__c=’2017-09-06′”, “operation”: “insert”, “targetObject”: “Customer_Interaction_Analysis__c”, “targetFieldMap”: {“Account__c”:”Account__c”,“In_Game_Purchase__c”:”Purchase__c”}, “targetValueMap”:{“$JOB_ID“ : “BackgroundOperationLookup__c”, “Copy fields from source to target“ : “BackgroundOperationDescription__c”}}

Difference Between Custom Object and Big Object

Let understand the difference between custom object and big object in Salesforce.

	Custom Object	Big Object
Create	Manual and Metadata	Metadata
API Name	__c	__b
Track Activity	Yes	No Option
Field History Tracking	Yes	No Option
Data Type	All	Text, Date Time, Lookup, Number, Long Text Area
Edit Field	Yes	Yes
Delete Field	Yes	No
Trigger	Yes	No
Report	Yes	No
Storage	It Count Against Storage	It doesn’t count against Storage

Considerations for Big Objects Usage

General
- Metadata API
- Max. 100 Big Objects per org
- Supports DateTime, Lookup, Number, Text, Long Text Area field types only
- Triggers, Flows, Processes, Salesforce App are unavailable
- Async SOQL is restricted to specific licenses
UI / UX
- Standard UI unavailable (Tabs, Detail Pages, List Views)
- Works with Visualforce Pages or Lightning Components
Data Security & Access
- Supports Object & Field Permissions only
Analytics
- No support for Report Builder
- Einstein Analytics supported
Packaging
- Included in Managed Packages