Data API Overview

The Data API provides comprehensive methods for reading and writing data in DealCloud. HTTP is handled by intapp-rest-client (RestClient); see Advanced configuration and Error handling for retries and errors.

This overview covers key concepts before diving into specific operations.

Output Formats

⚠️

Breaking change (dealcloud-sdk 1.x): The output parameter is required for read_data(). See the Migration Guide for details.

All read operations support multiple output formats:

Format	Return Type	Best For
`"list"`	`List[dict]`	API integration, custom processing
`"pandas"`	`pandas.DataFrame`	Data analysis, transformations
`"polars"`	`polars.DataFrame`	Large datasets, high performance
`"polars_lazy"`	`polars.LazyFrame`	Query optimization

# List of dictionaries
records = dc.read_data("Company", output="list")
 
# Pandas DataFrame
df = dc.read_data("Company", output="pandas")
 
# Polars DataFrame (10-100x faster for large datasets)
df = dc.read_data("Company", output="polars")
 
# Polars LazyFrame (for query optimization)
lf = dc.read_data("Company", output="polars_lazy")
result = lf.filter(pl.col("Revenue") > 1000000).collect()

🚀

Polars is recommended for datasets over 100k rows. See Polars Integration.

Data Operations

Read Operations→Write Operations→Cell Operations→Delta Sync→

Key Concepts

Objects vs Views

Source	Parameter	Description
Object	`object_id="Company"`	Read all data from an object
View	`view_id="My View"`	Read filtered data from a configured view

# Read from object
companies = dc.read_data("Company", output="pandas")
 
# Read from view
active = dc.read_data(view_id="Active Companies", output="pandas")

Field References

Fields are identified by their API Name (not display name):

Display Name	API Name	Notes
Company Name	`CompanyName`	No spaces
Date Created	`CreatedDate`	System field
Contact - Primary	`ContactPrimary`	Varies by config

To find API names:

# Get all fields for an object
fields = dc.get_fields("Company")
for f in fields:
    print(f"{f.name} -> {f.apiName}")

Reference Fields

Reference, choice, and user fields return rich objects by default:

# Default: full reference objects
contacts = dc.read_data("Contact", output="list")
# contacts[0]["Company"] = {"id": 123, "name": "Acme Corp", "entryListId": 2011, ...}
 
# With reference_format: just names
from dealcloud_sdk import ReferenceFormat
contacts = dc.read_data(
    "Contact",
    output="list",
    reference_format=ReferenceFormat.NAME
)
# contacts[0]["Company"] = "Acme Corp"
 
# With reference_format: just IDs
contacts = dc.read_data(
    "Contact",
    output="list",
    reference_format=ReferenceFormat.ID
)
# contacts[0]["Company"] = 123

Pagination

The SDK handles pagination automatically:

# Reads ALL records, automatically paginating
all_companies = dc.read_data("Company", output="pandas")
 
# Configure page size if needed
config = DealCloudConfig(
    # ...credentials
    querySettings={"pageSize": 500}  # Default: 1000
)

Streaming for Large Datasets

For memory-efficient processing of large datasets:

# Generator - processes one record at a time
for company in dc.read_data_streaming("Company"):
    process(company)
    # Memory usage stays constant regardless of total records
 
# Async generator
async for company in dc.aread_data_streaming("Company"):
    await process_async(company)

Quick Reference

Read Methods

Method	Returns	Use Case
`read_data()`	`DataFrame` or `List[dict]`	General reads
`read_data_streaming()`	`Iterator[dict]`	Large datasets, memory-efficient
`aread_data_streaming()`	`AsyncIterator[dict]`	Async applications
`typed_read_data()`	`List[T]`	Type-safe with Pydantic

Write Methods

Method	Purpose	Input	Output
`insert_data()`	Create new records	list, pandas, polars	list, pandas, polars
`update_data()`	Update existing records	list, pandas, polars	list, pandas, polars
`upsert_data()`	Insert or update based on key	list, pandas, polars	list, pandas, polars
`delete_data()`	Delete records by ID	list	list
`write_cells()`	Write cell-level data	list, pandas, polars	-

💡

Write methods now support both DataFrame input (pandas and Polars) and configurable output format via the output parameter.

Typed Methods (Pydantic)

Method	Wraps	Returns
`typed_read_data()`	`read_data()`	`List[T]`
`typed_read_data_streaming()`	`read_data_streaming()`	`Iterator[T]`
`typed_aread_data_streaming()`	`aread_data_streaming()`	`AsyncIterator[T]`
`typed_insert_data()`	`insert_data()`	`List[T]`
`typed_update_data()`	`update_data()`	`List[T]`
`typed_upsert_data()`	`upsert_data()`	`List[T]`

Error Handling

from dealcloud_sdk import DealCloud, ErrorHandling
 
# Fail fast (default) - raises on transport HTTP errors
dc.insert_data("Company", data, error_handling=ErrorHandling.FAIL_FAST)
 
# Collect transport errors - returns BatchResult
result = dc.insert_data("Company", data, error_handling=ErrorHandling.COLLECT)
row_errors = result.row_errors  # HTTP 200 rows with "Errors"
transport_errors = result.errors
 
# Inspect row errors on FAIL_FAST (default) without raising
from dealcloud_sdk import split_row_results
rows = dc.insert_data("Company", data)
_, row_errors = split_row_results(rows)
 
# Strict row errors (per-call or via DealCloudConfig.raiseOnRowErrors)
dc.insert_data("Company", data, raise_on_row_errors=True)

Next Steps

Basic Reads - Start with read_data()
Insert Data - Creating new records
Typed Data - Using Pydantic models

Schema Export Basic Reads