Workflow Execution Service (1.0.1)

Download OpenAPI specification:Download

Executive Summary

The Workflow Execution Service (WES) API provides a standard way for users to submit workflow requests to workflow execution systems, and to monitor their execution. This API lets users run a single workflow (currently CWL or WDL formatted workflows, other types may be supported in the future) on multiple different platforms, clouds, and environments. Key features of the API:

  • can request that a workflow be run
  • can pass parameters to that workflow (e.g. input files, cmdline arguments)
  • can get information about running workflows (e.g. status, errors, output file locations)
  • can cancel a running workflow

Introduction

This document describes the WES API and provides details on the specific endpoints, request formats, and response. It is intended to provide key information for developers of WES-compatible services as well as clients that will call these WES services. Use cases include:

  • "Bring your code to the data": a researcher who has built their own custom analysis can submit it to run on a dataset owned by an external organization, instead of having to make a copy of the data
  • Best-practices pipelines: a researcher who maintains their own controlled data environment can find useful workflows in a shared directory (e.g. Dockstore.org), and run them over their data

Standards

The WES API specification is written in OpenAPI and embodies a RESTful service philosophy. It uses JSON in requests and responses and standard HTTP/HTTPS for information transport.

Authorization and Authentication

Users must supply credentials that establish their identity and authorization in order to use a WES endpoint. We recommend that WES implementations use an OAuth2 bearer token, although they can choose other mechanisms if appropriate. WES callers can use the auth_instructions_url from the service-info endpoint to learn how to obtain and use a bearer token for a particular implementation.

The WES implementation is responsible for checking that a user is authorized to submit workflow run requests. The particular authorization policy is up to the WES implementer.

Systems like WES need to also address the ability to pass credentials with jobs for input and output access. In the current version of WES, the passing of credentials to authenticate and authorize access to inputs and outputs, as well as mandates about necessary file transfer protocols to support, are out of scope. However, parallel work on the Data Object Service is addressing ways to pass around access credentials with data object references, opening up the possibility that a future version of WES will provide concrete mechanisms for workflow runs to access data using credentials different than those used for WES. This is a work in progress and support of DOS in WES will be added in a future release of WES.

WorkflowExecutionService

GetServiceInfo

get/service-info
https://{defaultHost}/{basePath}/{version}/service-info

May include information related (but not limited to) the workflow descriptor formats, versions supported, the WES API versions supported, and information about general service availability.

Responses

200
400

The request is malformed.

401

The request is unauthorized.

403

The requester is not authorized to perform this action.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "id": "org.ga4gh.myservice",
  • "name": "My project",
  • "type":
    {
    },
  • "description": "This service provides...",
  • "organization":
    {},
  • "contactUrl": "mailto:support@example.com",
  • "documentationUrl": "https://docs.myservice.example.com",
  • "createdAt": "2019-06-04T12:58:19Z",
  • "updatedAt": "2019-06-04T12:58:19Z",
  • "environment": "test",
  • "version": "1.0.0",
  • "workflow_type_versions":
    {
    },
  • "supported_wes_versions":
    [
    ],
  • "supported_filesystem_protocols":
    [
    ],
  • "workflow_engine_versions":
    {
    },
  • "default_workflow_engine_parameters":
    [
    ],
  • "system_state_counts":
    {
    },
  • "auth_instructions_url": "string",
  • "tags":
    {
    },
  • "required": null
}

ListRuns

get/runs
https://{defaultHost}/{basePath}/{version}/runs

This list should be provided in a stable ordering. (The actual ordering is implementation dependent.) When paging through the list, the client should not make assumptions about live updates, but should assume the contents of the list reflect the workflow list at the moment that the first page is requested. To monitor a specific workflow run, use GetRunStatus or GetRunLog.

query Parameters
page_size
integer <int64>

OPTIONAL The preferred number of workflow runs to return in a page. If not provided, the implementation should use a default page size. The implementation must not return more items than page_size, but it may return fewer. Clients should not assume that if fewer than page_size items are returned that all items have been returned. The availability of additional pages is indicated by the value of next_page_token in the response.

page_token
string

OPTIONAL Token to use to indicate where to start getting results. If unspecified, return the first page of results.

Responses

200
400

The request is malformed.

401

The request is unauthorized.

403

The requester is not authorized to perform this action.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "runs":
    [
    ],
  • "next_page_token": "string"
}

RunWorkflow

post/runs
https://{defaultHost}/{basePath}/{version}/runs

This endpoint creates a new workflow run and returns a RunId to monitor its progress.

The workflow_attachment array may be used to upload files that are required to execute the workflow, including the primary workflow, tools imported by the workflow, other files referenced by the workflow, or files which are part of the input. The implementation should stage these files to a temporary directory and execute the workflow from there. These parts must have a Content-Disposition header with a "filename" provided for each part. Filenames may include subdirectories, but must not include references to parent directories with '..' -- implementations should guard against maliciously constructed filenames.

The workflow_url is either an absolute URL to a workflow file that is accessible by the WES endpoint, or a relative URL corresponding to one of the files attached using workflow_attachment.

The workflow_params JSON object specifies input parameters, such as input files. The exact format of the JSON object depends on the conventions of the workflow language being used. Input files should either be absolute URLs, or relative URLs corresponding to files uploaded using workflow_attachment. The WES endpoint must understand and be able to access URLs supplied in the input. This is implementation specific.

The workflow_type is the type of workflow language and must be "CWL" or "WDL" currently (or another alternative supported by this WES instance).

The workflow_type_version is the version of the workflow language submitted and must be one supported by this WES instance.

See the RunRequest documentation for details about other fields.

Request Body schema: multipart/form-data
workflow_params
string
workflow_type
string
workflow_type_version
string
tags
string
workflow_engine_parameters
string
workflow_url
string
workflow_attachment
Array of strings <binary>

Responses

200
400

The request is malformed.

401

The request is unauthorized.

403

The requester is not authorized to perform this action.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "run_id": "string"
}

GetRunLog

get/runs/{run_id}
https://{defaultHost}/{basePath}/{version}/runs/{run_id}

This endpoint provides detailed information about a given workflow run. The returned result has information about the outputs produced by this workflow (if available), a log object which allows the stderr and stdout to be retrieved, a log array so stderr/stdout for individual tasks can be retrieved, and the overall state of the workflow run (e.g. RUNNING, see the State section).

path Parameters
run_id
required
string

Responses

200
401

The request is unauthorized.

403

The requester is not authorized to perform this action.

404

The requested workflow run not found.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "run_id": "string",
  • "request":
    {
    },
  • "state": "UNKNOWN",
  • "run_log":
    {
    },
  • "task_logs":
    [
    ],
  • "outputs": { }
}

CancelRun

post/runs/{run_id}/cancel
https://{defaultHost}/{basePath}/{version}/runs/{run_id}/cancel

Cancel a running workflow.

path Parameters
run_id
required
string

Responses

200
401

The request is unauthorized.

403

The requester is not authorized to perform this action.

404

The requested workflow run wasn't found.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "run_id": "string"
}

GetRunStatus

get/runs/{run_id}/status
https://{defaultHost}/{basePath}/{version}/runs/{run_id}/status

This provides an abbreviated (and likely fast depending on implementation) status of the running workflow, returning a simple result with the overall state of the workflow run (e.g. RUNNING, see the State section).

path Parameters
run_id
required
string

Responses

200
401

The request is unauthorized.

403

The requester is not authorized to perform this action.

404

The requested workflow run wasn't found.

500

An unexpected error occurred.

Response samples

Content type
application/json
Copy
Expand all Collapse all
{
  • "run_id": "string",
  • "state": "UNKNOWN"
}