Sync

Description

This job triggers a connector using this endpoint.

Use Cases

We recommend creating a Sync Task for every "sync" within Census.

This way, you can use Orchestra to trigger your reverse ELT on a cron or event based schedule. This has a number of advantages vs. using Census' in-built scheduler:

• You can co-ordinate tasks outside of Census - these would typically be specific dbt job runs or operations that you know need to succeed before running an rELT sync

  • A common use-case is to have a dbt run job that tests the state of the data before triggering a sync in Census. This way, you can ensure data pushed to underlying source systems such as Salesforce adheres to data quality constraints.

• You can use Orchestra to trigger jobs across Census Cloud Accounts

• When Census jobs run, Data Warehouse cost is incurred. Running these operations on a schedule you set explicitly ensures these costs do not go out of hand

• We aggregate metadata from the dbt Task in the same place as the metadata from other operations in your Pipeline

Parameters

These parameters are required to run the Job run Task

Name	Data type	Restrictions	Example
Sync ID	Number	N.A.	12345

Error handling

API Requests

If we receive the following error codes from Census, we'll raise an error and the task will move to a failed state.

Code	Description	Handling
401	Unauthorised	We will raise an error and parse the raw error message from the Census response as the Orchestra message
404	Not Found	We will raise an error with the HTTP Reason as the Orchestra message
Other error code		We will raise an error with the HTTP Reason as the Orchestra message

Failed syncs

When the sync completes, the sync response from Census looks like this:

{
    "status": "success",
    "data": {
        "id": 12345678,
        "sync_id": 12345,
        "source_record_count": null,
        "records_processed": null,
        "records_updated": null,
        "records_failed": null,
        "records_invalid": null,
        "created_at": "2023-11-03T11:55:38.753Z",
        "updated_at": "2023-11-03T11:55:38.892Z",
        "completed_at": null,
        "scheduled_execution_time": null,
        "error_code": null,
        "error_message": null,
        "error_detail": null,
        "status": "queued",
        "current_step": "Running sync",
        "canceled": false,
        "full_sync": false,
        "sync_trigger_reason": {
            "ui_tag": "API Request",
            "ui_detail": "Sync was triggered by an API request"
        }
    }
}

Where the top-level "status" is not equal to "success", Orchestra will fail the task and parse the error_code, error_message and error_detail into the Orchestra message field.