Dagster Cloud Code Previews#

The Dagster Cloud CI/CD GitHub Action lets you automatically create previews of your Dagster Cloud code locations when pipeline code is updated in a pull request. The action builds a Docker image, and uses the Dagster Cloud API to create a viewable Dagit link for review, before the pipeline code is merged.

Quickstart Template Repo#

We provide a quickstart template repo which you can use to get CI/CD for your Cloud instance up and running quickly.

Usage#

To use Dagster Cloud Code Previews, you will first need to create a new GitHub Actions Workflow. The job must first clone your Git repository using the actions/checkout action.

Next, you must set up a locations.yaml file which describes each of the Dagster Cloud repo locations to be built and updated. If this locations.yaml file is not located at the repo root, it must be specified with the location-file input.

Example Job#

name: Create Dagster Cloud Code Preview
on: [pull_request]

jobs:
  build-and-preview:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2

      - name: Create Dagster Cloud Code Preview
        uses: dagster-io/dagster-cloud-cicd-action/preview@v0.2.0
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          location-file: ./locations.yaml
          dagit-url: https://hooli.dagster.cloud/prod
          api-token: ${{ secrets.DAGSTER_AGENT_TOKEN }}

Customization#

Inputs#

NameDescription
github-token(Required) Github API token. You can pass your repository's token using ${{ secrets.GITHUB_TOKEN }}.
dagit-url(Required) URL to your Dagster Cloud instance, including the deployment path.
api-token(Required) Dagster Cloud Agent API token.
location-filePath to the locations.yaml file defining the repo locations to update. Defaults to /locations.yaml in the repo root.
image-tagTag for the built Docker images, defaults to the first 6 chars of git hash.
parallelWhether to build and push Docker images in parallel. Defaults to true.

locations.yaml Properties#

Each location specified in the locations.yaml can have the following properties:

NameDescription
build(Required) Path to the build directory relative to the locations.yaml's folder. Build directory must contain a Dockerfile or requirements.txt file.
registry(Required) Docker registry to push the built Docker image to.
package_namePython package containing the Dagster Repository. Can alternatively use python_file or module_name.
module_namePython module containing the Dagster Repository. Can alternatively use python_file or package_name.
python_filePython file containing the Dagster Repository. Can alternatively use package_name or module_name.
working_directory(Optional) Working directory to use for importing Python modules when loading the repository.
executable_path(Optional) Path to reach the executable to use for the Python environment to load the repository. Defaults to the installed dagster command-line entry point.
attribute(Optional) Specifies either a repository or a function that returns a repository. Can be used when the code contains multiple repositories but only one should be included.
base_imageIf the build directory only contains a requirements.txt file and no Dockerfile, specifies the base Docker image to use to build the code location.
targetIf providing a multistage Dockerfile, can be used to specify the target stage to build.