Dataset materialisation

Description

This job triggers a dataset materialisation using this endpoint.

Use Cases

We recommend creating a Dataset materialisation task for every 'Dataset' in Sigma.

This way, you can use Orchestra to trigger your refreshes on a cron or event based schedule. This has a number of advantages vs. using Sigma's in-built scheduler:

• You can co-ordinate tasks outside of Sigma - these would typically be specific dbt job runs or operations that you know need to succeed before building your dashboards.

  • A common use-case is to have a dbt run job that transform the data so it is ready for Sigma to use. This way, you can ensure data pushed to Sigma adheres to data quality constraints.

• You can use Orchestra to trigger jobs across Sigma accounts

• When Sigma jobs run, cost is incurred. Running these operations on a schedule you set explicitly ensures these costs do not go out of hand

Parameters

These parameters are required to run the Dataset materialisation task

Name	Data type	Restrictions	Example
Dataset ID	string	N.A.	random_dataset_id

Error handling

API Requests

If we receive the following error codes from Sigma, we'll raise an error and the task will move to a failed state.

Code	Description	Handling
400	Bad Request	Will raise an error and parse the message in the repsonse. Often associated with the wrong dataset id.
401	Unauthorised	We will raise an error and parse the raw error message from the Sigma response as the Orchestra message
404	Not Found	We will raise an error with the HTTP Reason as the Orchestra message
Other error code		We will raise an error with the HTTP Reason as the Orchestra message