Configure a pipeline for Mantle

​

Introduction

Once you have a pipeline written that you want to be availabe to run on Mantle, you need to Configure it in the UI. This is a simple process to give Mantle access to your pipelines code and the ability to run it in your AWS account.

In this guide we will cover where to get the data needed to configure a new pipeline in Mantle.

​

Github Configuration

Mantle will need acccess to the github repo that contains your pipeline code. This is done by adding

​

Prerequisites

• github account created.
• appropriate access to the github organization which pipeline belong to.

​

Steps

1. In your browser (on any machine does not have to be the machine you will run pipelines on) sign in to you github account
2. Navigate to https://github.com/settings/personal-access-tokens/new
   You may be prompted to confirm access if this happens you can use whatever authentication method you prefer
3. Once you are on the page you should see a header at the top that reads New fine-grained personal access token Fill out the fields as follows:

• Name: mantle-access
• Description: Mantle access to pipelines
• Expire: 90 days (or whatever you prefer)
• Resource Owner: Organization (or User) that the pipeline belongs to.
• Repository access: Only select repositories and then select the repositories that contain the pipeline code.
• Permissions: Expand Repository permissions and select Contents and Read-only

4. Once you have filled out the form click Generate token at the bottom of the page.
   NOTE: IF YOU NAVIGATE AWAY FROM THIS PAGE BEFORE COPYING YOU WILL NOT BE ABLE TO GET THE TOKEN AGAIN AND YOU WILL NEED TO DELETE IT ANT FOLLOW THESE STEPS AGAIN
5. Copy the token and save it in a secure place like a password manager.

​

AWS Batch Configuration

You will need the arn of the batch queue you want to run your pipeline on. This is the arn created in the SetUp guide.

The arn will look something like this:

arn:aws:batch:us-west-2:123456789012:job-queue/SomeQueue

​

Input Configuration Format

In order for mantle to create a UI to run the pipeline and provide the inputs through the SDK, we need an expected set of inputs for the pipeline. This is done by creating a JSON file that describes.

The Format of the JSON file is as follows:

{
  {field_name}: {
    "type": {type},
    "allow_multiple": {boolean}
    "required": {boolean}
  }
  ...
}

Where:

• type: The type of the input. This can be one of the following:
  • string
  • int
  • float
  • bool
  • file
  • any of the dataset types that are avaiable in your account.
• allow_multiple: If the input can be a list of values or not.
• required: If the input is required or not.

​

Configuration

To configure a new pipeline in Mantle, go to {tenant}.app.mantlebio.com/pipelines and click the Import Pipeline button.

You will be prompted to fill out the following fields to fill out as follows:

• Pipeline Name: The name of the pipeline, this will be the name that is displayed in the UI.
• Pipeline Version: The version of the pipeline, this will be the version that is displayed in the UI.
• Description: A description of the pipeline, this will be displayed in the UI.
• Github Repository Name: The name of the repository that contains the pipeline code.
• Github Repository URL: The URL of the repository that contains the pipeline code.
• Github Access Token: The access token that was created in the Github Configuration section.
• AWS Batch Queue ARN: The arn of the batch queue that the pipeline will run on. This was created in the SetUp guide.
• Revision Number: The revision number of the pipeline code that you want to run.
• Entry (not required): The entry point of the pipeline code if you are using a monorepo. See Using A Monorepo for more information. If Entry is not provided, Mantle will use the default entry point.
• Input Configuration: The JSON file that describes the expected inputs for the pipeline. This was created in the Input Configuration Format section.

Once you have filled out the form click Import at the bottom of the page.