Big Object in Salesforce

Salesforce has a limitation on performing queries on a large number of records. Especially when your data is massive (assume more than 10k records per object) performing queries and performance of the system downgrades. That is where Big Object in Salesforce Comes into the picture. Let’s learn what Big Object is in Salesforce and Big Object implementation guide.

Leveraging Big Objects we can perform asynchronous queries to retrieve huge sets of data, without affecting the performance in a faster way. Big Objects can help you improve performance and scale your Implementation when your Data is huge in Salesforce!

What is Big Object in Salesforce?

A big object that stores and manages massive data values within Salesforce without affecting performance. Salesforce Big objects provide consistent performance for a billion records, and are accessible with a standard set of API’s to your org or external system.

Custom big objects can be created by the Metadata API. To define a custom big object, you need to create an object file that contains its definition, field’s, and index, with a Permission-set to define the permissions for each field, and package file to define the contents of the object metadata.

When to use Big Object in Salesforce?

Here are some reasons or use cases when we can use Big Object in Salesforce.

  • Capture User Activity: Code reviews, time entries, page views, field audits etc.
  • Retain Historical Data: Historical data is stored for compliance.
  • 360 Customer View: Ancillary customer data e.g. Purchase Details, Transactions.

Deploying your Schema

Big Object in Salesforce

Big Object in Salesforce Video

YouTube video

How to create big objects in Salesforce?

You need to create the below three files.

1. Create Big Object Files

1.1. Create an Object File:- Defines the custom big object’s fields and index, Each custom big object needs its own object file.

My_First_Big_Object__b.object

<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
    <deploymentStatus>Deployed</deploymentStatus>
    <fields>
        <fullName>MyTextField__c</fullName>
        <label>My Text Field</label>
        <length>16</length>
        <required>false</required>
        <type>Text</type>
        <unique>false</unique>
    </fields>
    <fields>
        <fullName>Account_Lookup__c</fullName>
        <label>Account Lookup</label>
        <referenceTo>Account</referenceTo>
        <relationshipName>Account_Lookup</relationshipName>
        <required>true</required>
        <type>Lookup</type>
    </fields>
    <fields>
        <fullName>Date_Time__c</fullName>
        <label>Date Time</label>
        <required>true</required>
        <type>DateTime</type>
    </fields>   
    <fields>
        <fullName>Number_Field__c</fullName>
        <label>Number Field</label>
        <required>false</required>
        <type>Number</type>
        <scale>2</scale>
        <precision>16</precision>
    </fields>
    <indexes>
        <fullName>MyFirstBigIndex</fullName>
        <label>My First Big Index</label>
        <fields>
            <name>Account_Lookup__c</name>
            <sortDirection>DESC</sortDirection>
        </fields>
        <fields>
            <name>Date_Time__c</name>
            <sortDirection>DESC</sortDirection>
        </fields>
    </indexes>
   
    <label>My First Big Object</label>
    <pluralLabel>My First Big Object</pluralLabel>
   
</CustomObject>

1.2. Create A Permissionset file:- with a Permissionset to define the permissions for each field.

My_First_Big_Object.permissionset

<?xml version="1.0" encoding="UTF-8"?>
<PermissionSet xmlns="http://soap.sforce.com/2006/04/metadata">
    
    <fieldPermissions>
        <editable>true</editable>
        <field>My_First_Big_Object__b.MyTextField__c</field>
        <readable>true</readable>
    </fieldPermissions>
        
    <fieldPermissions>
        <editable>true</editable>
        <field>My_First_Big_Object__b.Number_Field__c</field>
        <readable>true</readable>
    </fieldPermissions>
    <label>My_First_Big_Object Permission Set</label>
</PermissionSet>

1.3. Create package.xml file:-  file to define the contents of the object metadata

package.xml


<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>*</members>
        <name>CustomObject</name>
    </types>
    <types>
        <members>*</members>
        <name>PermissionSet</name>
    </types>
    <version>41.0</version>
</Package>

2. Deploy big Object

2.1 Create the zip file for deployment.

Save all three file in one folder. Create the folder Structure like below.

  • Add My_First_Big_Object__b.object file in Object Folder
  • Add My_First_Big_Object.permissionset file in permissionset Folder
  • package.xml file should in root of the folder.

2.2 Deploy with the help of Workbench or ANT Tool.

2.2.1 Login on workbench

2.2.2 Click on the Migration tool in the workbench and select the Single Package option like below image.

2.2.3 Now validate the big object in your Salesforce org.

How to insert a record 

My_First_Big_Object__b obj = new My_First_Big_Object__b();
obj.Account_Lookup__c ='0019000001ukhMF';
obj.Date_Time__c=System.now();
obj.MyTextField__c='hello';
obj.Number_Field__c=10;
database.insertImmediate(obj);

Big Object Data Manipulation

Let’s understand what all we can do with big Object

  1. Apex CRUD
    • Create / Update (Idempotent Behavior)
    • insertImmediate(sobject) OR insertImmediate(sobjects)
  2. Read
    • SOQL Queries
    • Async SOQL
  3. CSV Files
    • API (Bulk API, SOAP API)

Using Standard SOQL with Big Objects

  • Executes synchronously.
  • All Indexes are mandatory.
  • Comparison Operators (=, <, >, <=, >=, IN).
  • Not Supported Operators.
  • (!=, LIKE, NOT IN, EXCLUDES, INCLUDES).

SOQL Vs Async SOQL Usage Considerations

FeatureStandard SOQLAsync SOQL
Mode of ExecutionSynchronousAsynchronous
Immediate Response Required?YesNo
Expected Result Set SizeSmaller Data Sets (Thousands of records)Large Data Sets (Millions of records)
Best Suited ForDisplaying Data on UI
Manipulations within Apex
Aggregation Summarizing for Analytics
Filter using Non Index fieldsYesNo
Sample FormatSELECT Game_Platform__c, Play_Date__c
FROM Customer_Interaction__b
WHERE
Game_Platform__c=’PC’ AND Play_Date__c=’2017-09-06′
{“query”: “SELECT Account_c, In_Game_Purchase__c FROM Customer_Interaction__b WHERE Play_Date__c=’2017-09-06′”,

“operation”: “insert”,
“targetObject”: “Customer_Interaction_Analysis__c”,

“targetFieldMap”: {“Account__c”:”Account__c”,“In_Game_Purchase__c”:”Purchase__c”},
“targetValueMap”:{“$JOB_ID“ : “BackgroundOperationLookup__c”,
“Copy fields from source to target“ : “BackgroundOperationDescription__c”}}

Difference Between Custom Object and Big Object

Let understand the difference between custom object and big object in Salesforce.


Custom ObjectBig Object
CreateManual and MetadataMetadata
API Name__c__b
Track ActivityYesNo Option
Field History TrackingYesNo Option
Data TypeAllText, Date Time, Lookup, Number, Long Text Area
Edit FieldYesYes
Delete FieldYesNo
TriggerYesNo
ReportYesNo
StorageIt Count Against StorageIt doesn’t count against Storage

Considerations for Big Objects Usage

  • General
    • Metadata API
    • Max. 100 Big Objects per org
    • Supports DateTime, Lookup, Number, Text, Long Text Area field types only
    • Triggers, Flows, Processes, Salesforce App are unavailable
    • Async SOQL is restricted to specific licenses
  • UI / UX
    • Standard UI unavailable (Tabs, Detail Pages, List Views)
    • Works with Visualforce Pages or Lightning Components
  • Data Security & Access
    • Supports Object & Field Permissions only
  • Analytics
    • No support for Report Builder
    • Einstein Analytics supported
  • Packaging
    • Included in Managed Packages

FAQ’s

Difference Between Custom Object and Big Object

Custom objects can be created manually in Salesforce, but Big objects can be created by metadata API.

How many big objects are there in Salesforce?

You can create Max. 100 Big Objects per org.

How many records can a big object hold?

It can store up to one billion records

What is async SOQL in Salesforce?

Async SOQL queries are run in the background over Salesforce big object data.

Amit Chaudhary
Amit Chaudhary

Amit Chaudhary is Salesforce Application & System Architect and working on Salesforce Platform since 2010. He is Salesforce MVP since 2017 and have 17 Salesforce Certificates.

He is a active blogger and founder of Apex Hours.

Articles: 465

Leave a Reply

Your email address will not be published. Required fields are marked *