Upsert Data

The upsert_data() method inserts new records or updates existing ones based on a match field. It composes insert_data / update_data behavior; same concurrency and HTTP stack as other writes (Advanced configuration, intapp-rest-client).

When to Use Upsert

Scenario	Method
Creating new records	`insert_data()`
Updating known records	`update_data()`
Syncing from external system	`upsert_data()`
Unknown if record exists	`upsert_data()`

Basic Upsert

from dealcloud_sdk import DealCloud, DealCloudConfig
 
config = DealCloudConfig(
    siteUrl="yoursite.dealcloud.com",
    clientId=12345,
    clientSecret="your-secret",
)
dc = DealCloud.from_config_object(config)
 
# Upsert based on ExternalId field
records = [
    {
        "ExternalId": "CRM-001",
        "CompanyName": "Acme Corporation",
        "Revenue": 1000000
    },
    {
        "ExternalId": "CRM-002",
        "CompanyName": "Beta Industries",
        "Revenue": 500000
    }
]
 
result = dc.upsert_data(
    "Company",
    records,
    match_field="ExternalId"  # Field to match on
)

💡

If a record with matching ExternalId exists, it's updated. If not, a new record is created.

Match Field Requirements

The match_field must be:

A unique field (or have unique values in your data)
Present in every record you're upserting
An existing field in DealCloud

# Common match fields
dc.upsert_data("Company", records, match_field="ExternalId")
dc.upsert_data("Contact", records, match_field="Email")
dc.upsert_data("Deal", records, match_field="DealNumber")

Upsert with EntryId

You can also include EntryId for known records:

records = [
    {
        "EntryId": 12345,  # Known record - will update
        "CompanyName": "Acme Corp Updated"
    },
    {
        # No EntryId - will match on field or create new
        "ExternalId": "CRM-003",
        "CompanyName": "New Company"
    }
]
 
result = dc.upsert_data("Company", records, match_field="ExternalId")

Input Formats

records = [
    {"ExternalId": "CRM-001", "CompanyName": "Acme Corp", "Revenue": 1000000},
    {"ExternalId": "CRM-002", "CompanyName": "Beta Inc", "Revenue": 500000},
]
dc.upsert_data("Company", records, match_field="ExternalId")

Using Lookup Values

# Default: Use IDs for references
records = [
    {
        "ExternalId": "CRM-001",
        "CompanyName": "Acme Corp",
        "Industry": 101,  # Choice ID
        "PrimaryContact": 67890  # Contact EntryId
    }
]
 
dc.upsert_data("Company", records, match_field="ExternalId")

Parameters

Parameter	Type	Default	Description
`object_id`	`str \| int`	Required	Object API name or ID
`data`	`List[dict] \| DataFrame`	Required	Records to upsert (accepts pandas or Polars)
`match_field`	`str`	Required	Field to match existing records
`use_dealcloud_ids`	`bool`	`True`	Use IDs vs lookup values
`lookup_column`	`str`	None	Field for reference lookups
`output`	`str`	`"list"`	Output format: `"list"`, `"pandas"`, or `"polars"`

Output Formats

# List (default)
result = dc.upsert_data("Company", records, match_field="ExternalId")
 
# Pandas DataFrame
result = dc.upsert_data("Company", records, match_field="ExternalId", output="pandas")
 
# Polars DataFrame
result = dc.upsert_data("Company", records, match_field="ExternalId", output="polars")

Return Value

result = dc.upsert_data("Company", records, match_field="ExternalId")
 
# Result includes EntryId for all records
for record in result:
    print(f"{record['CompanyName']}: EntryId={record['EntryId']}")

Typed Upsert

Using Pydantic models:

from pydantic import BaseModel
from typing import Optional
 
class Company(BaseModel):
    EntryId: Optional[int] = None
    ExternalId: str
    CompanyName: str
    Revenue: Optional[float] = None
 
# From external system
external_data = [
    Company(ExternalId="CRM-001", CompanyName="Acme Corp", Revenue=1000000),
    Company(ExternalId="CRM-002", CompanyName="Beta Inc", Revenue=500000),
]
 
result = dc.typed_upsert_data(
    "Company",
    external_data,
    Company,
    match_field="ExternalId"
)
 
for company in result:
    print(f"Upserted {company.CompanyName} (ID: {company.EntryId})")

Sync Workflow Example

Complete external system sync:

from dealcloud_sdk import DealCloud, DealCloudConfig
import pandas as pd
 
config = DealCloudConfig(
    siteUrl="yoursite.dealcloud.com",
    clientId=12345,
    clientSecret="your-secret",
)
dc = DealCloud.from_config_object(config)
 
def sync_from_crm(crm_export_file: str):
    """Sync companies from CRM export."""
    
    # 1. Load external data
    crm_data = pd.read_csv(crm_export_file)
    
    # 2. Rename columns to match DealCloud API names
    column_mapping = {
        "CRM ID": "ExternalId",
        "Company": "CompanyName",
        "Annual Revenue": "Revenue",
        "Type": "CompanyType"
    }
    crm_data = crm_data.rename(columns=column_mapping)
    
    # 3. Upsert to DealCloud
    result = dc.upsert_data(
        "Company",
        crm_data,
        match_field="ExternalId",
        use_dealcloud_ids=False  # Use names for choices
    )
    
    # 4. Report results
    print(f"Synced {len(result)} companies")
    return result
 
sync_from_crm("crm_companies.csv")

Handling Duplicates

If your data has duplicate match values:

# Check for duplicates first
import pandas as pd
 
df = pd.DataFrame(records)
duplicates = df[df.duplicated("ExternalId", keep=False)]
 
if not duplicates.empty:
    print(f"Warning: {len(duplicates)} duplicate ExternalIds")
    # Remove duplicates (keep first)
    df = df.drop_duplicates("ExternalId", keep="first")
 
dc.upsert_data("Company", df, match_field="ExternalId")

Performance Considerations

Use specific fields - Only include fields you need to update
Batch appropriately - SDK handles batching, but consider logical groupings
Unique match field - Non-unique fields cause multiple updates

# Good: Specific fields
records = [
    {"ExternalId": "CRM-001", "Revenue": 1000000},  # Only updating Revenue
]
 
# Less efficient: All fields
records = [
    {
        "ExternalId": "CRM-001",
        "CompanyName": "Acme",  # Unchanged
        "Industry": 101,  # Unchanged
        "Revenue": 1000000,  # Changed
        "Status": 201,  # Unchanged
        # ... many more unchanged fields
    }
]

Error Handling

upsert_data delegates to insert_data / update_data and supports the same error surface: error_handling, output="write_result", raise_on_row_errors, and BatchResult on COLLECT.

from dealcloud_sdk import ErrorHandling, split_row_results, DealCloudValidationError
 
result = dc.upsert_data("Company", records, match_field="ExternalId")
 
ok_rows, row_errors = split_row_results(result)
print(f"OK: {len(ok_rows)}, row errors: {len(row_errors)}")
 
batch = dc.upsert_data(
    "Company",
    records,
    match_field="ExternalId",
    error_handling=ErrorHandling.COLLECT,
)
print(batch.row_errors, batch.errors)
 
try:
    dc.upsert_data("Company", records, raise_on_row_errors=True)
except DealCloudValidationError as e:
    for row in e.row_errors:
        print(row["EntryId"], row["Errors"])

Best Practices

Choose match field carefully - Should be unique and stable
Validate data before upsert - Check for duplicates and required fields
Log sync results - Track what was created vs updated
Handle errors gracefully - Use COLLECT for batch operations
Use typed upserts - Better validation and code clarity

Insert Data - Creating records
Update Data - Updating records
ID Mapping - Building ID caches

Update Data Delete Data