Sync

Description

This job triggers a connector using this endpoint.

Use Cases

We recommend creating a Airbyte Sync Task for every "source" within Airbyte, which should itself have a single connector in Airbyte.

This way, you can use Orchestra to trigger your data ingestion on a manual or event based schedule. This has a number of advantages vs. using Stitch's in-built scheduler:

• You can co-ordinate tasks outside of Airbyte, like other ingestion tools for connectors Airbyte doesn't support, using Orchestra

• When Airbyte replication jobs run, Data Warehouse cost is incurred. Running these operations on a schedule you set explicitly ensures these costs do not go out of hand

• We aggregate metadata from the AirbyteTask in the same place as the metadata from other operations in your Pipeline

Parameters

These parameters are required to run the Sync Task

Name	Data type	Restrictions	Example
Connection ID	GUID	N.A.	0368bfcb-946f-4d0f-8b14-2252cdc1f5e0
Workspace ID (Optional)	GUID	N.A.	0368bfcb-946f-4d0f-8b14-2252cdc1f5e0

Error handling

API Requests

If we receive the following error codes from Airbyte, we'll raise an error and the task will move to a failed state.

Code	Description	Handling
401	Unauthorised	We will raise an error and parse the raw error message from the Airbyte response as the Orchestra message
404	Not Found	We will raise an error with the HTTP Reason as the Orchestra message
Other error code		We will raise an error with the HTTP Reason as the Orchestra message

Job failure

When a connection is manually triggered this creates a new Job. Information about the job history can be found in the Airbyte docs here. Orchestra maps the sync status as follows:

Orchestra Status	Airbyte Sync Status	Airbyte Description
Succeeded	Succeeded	100% of the data has been extracted and loaded to the destination
Failed	Partially Succeeded	A subset of the data has been loaded to the destination
Failed	Failed	None of the data has been loaded to the destination
Cancelled	Cancelled	The sync was cancelled manually before finishing
Running	Running	The sync is currently running