Build pipelines

HLA-Compass registers and runs versioned Nextflow definitions. Execution requires Pipelines to be enabled for the selected organization, access to the Pipeline, an allowed resource profile, available credits, and a working deployed runner. A 403 FEATURE_DISABLED means the organization does not have the feature; there is no release-wide prohibition on creating Pipeline Runs.

These instructions use SDK 5.9.2 and its matching API contracts. Registration, local validation, or an accepted run request does not prove that a scientific workflow completed. New scientific pipeline execution and strict filesystem hardening of all Nextflow head/worker images are not yet verified by this readiness release.

Choose the publication path

Path	What you publish	How it starts
Registered Pipeline	An imported nf-core/GitHub revision or a custom Nextflow ZIP	Pipeline run form, Pipeline SDK/CLI, or hosted run_pipeline MCP tool
Pipeline-backed Module	A module manifest, configure Lambda, Nextflow package, and optional custom UI	Module run form or prepared Module API/MCP plan

An internal Pipeline marked launchedViaModuleOnly: true belongs to a Module. Launch and manage it through that owning Module; direct Pipeline mutation and launch endpoints reject the internal record.

Authenticate and discover

Install the dev/staging SDK candidate and select your target environment. Only the exact SDK wheel comes from TestPyPI; dependencies come from PyPI:

python -m pip download --index-url https://test.pypi.org/simple/ --no-deps --only-binary=:all: --dest sdk-wheel "hla-compass==5.9.2"
python -m pip install "hla-compass[authoring]==5.9.2" --index-url https://pypi.org/simple/ --find-links sdk-wheel
hla-compass auth login --env dev
hla-compass auth status --check --env dev
HLA_COMPASS_ENV=dev hla-compass pipeline list
hla-compass pipeline --help

Pipeline CLI subcommands use the active SDK environment; they do not accept a per-command --env. The explicit HLA_COMPASS_ENV=dev prefix above avoids using an unintended active environment. Use your existing platform login and organization permissions. Publishing/importing also requires the relevant role; a module publishing API key is not a general Pipeline credential.

Register a Pipeline

Import an exact upstream release instead of a moving branch:

HLA_COMPASS_ENV=dev hla-compass pipeline import-nfcore nf-core/rnaseq --version 3.14.0

The upstream release here illustrates the command shape, not a recommended or verified scientific configuration. Check its parameters, references, images, and license before use. Retain the returned Pipeline UUID and resolved revision.

For another repository, pipeline import-github OWNER/REPOSITORY --ref COMMIT_SHA uses the organization's GitHub App integration. The App must be installed and have access to a private repository. Do not put a personal access token in parameters or a manifest.

For an existing custom Nextflow ZIP:

HLA_COMPASS_ENV=dev hla-compass pipeline register workflow.zip \
  --name research-workflow --version 1.0.0 \
  --parameter-template parameter-template.json

This uploads the ZIP and registers the definition in one command. Supply your actual workflow and parameter template; registration is not a run. Inspect the exact returned record before launching:

from hla_compass import APIClient

client = APIClient(environment="dev")
pipeline = client.get_pipeline("PIPELINE_UUID")
print(pipeline)

Inputs and launch

Prepare the workflow's required samplesheet, reference files, parameters, and Nextflow configuration. All referenced S3 locations must be authorized for the organization and readable by the runtime. A samplesheet's row identifiers and columns must match the workflow's schema; a generic table export is not necessarily a valid scientific samplesheet.

A params file supplies the base configuration; inline parameters override matching top-level values. params_file (local YAML/JSON) and params_file_uri (existing S3 object) are mutually exclusive. Either can be combined with inline overrides. samplesheet accepts a local file or S3 URI and sets parameters.input. Local files are uploaded before the run request.

After reviewing the inputs and selected profile's ACT cost, a basic launch is:

HLA_COMPASS_ENV=dev hla-compass pipeline run PIPELINE_UUID \
  --params-file params.yaml --samplesheet samplesheet.csv \
  --resource-profile YOUR_GRANTED_PROFILE

Use the actual case-sensitive profile name available to your organization. The CLI does not expose Nextflow configuration profiles, on-demand selection, or resume flags. Use the run form or SDK when those are needed:

run = client.start_pipeline_run(
    "PIPELINE_UUID",
    params_file="params.yaml",
    samplesheet="samplesheet.csv",
    resource_profile="YOUR_GRANTED_PROFILE",
    nextflow_profile="YOUR_WORKFLOW_PROFILE",
)
run_id = run.get("runId") or run["id"]
print(run_id)

The platform supplies a default output location. Choose Nextflow profile names from that workflow's own configuration; do not assume docker, test, or any other profile exists. The SDK also accepts on_demand=True and resume_from_run_id="PRIOR_RUN_UUID". Resume is subject to server validation and requires the previous work directory to remain available; it is not evidence of a fully pinned replay.

Starting a Pipeline Run reserves ACT from its resource profile. A completed run consumes the reservation; a failed or cancelled run refunds it in full. This billing is not per-task metering. Review the profile before submitting; the native Pipeline start is a charged action and is not the prepared Module API.

Via MCP, use the hosted run_pipeline schema and review its inputs and cost before confirmation. A failed dispatch can return HTTP 200 with no top-level JSON-RPC error and isError: true inside the tool result. Check that result before claiming a run started or completed.

Inspect results

Keep the returned Run ID. The run-detail view and these commands inspect the same run:

HLA_COMPASS_ENV=dev hla-compass pipeline status RUN_UUID
HLA_COMPASS_ENV=dev hla-compass pipeline logs RUN_UUID --limit 100
HLA_COMPASS_ENV=dev hla-compass pipeline tasks RUN_UUID

SDK equivalents are get_pipeline_run, get_pipeline_run_logs, and get_pipeline_run_tasks. Follow log pagination tokens. Task details depend on the Nextflow trace being uploaded; an empty task list while running is not a successful empty analysis. Inspect terminal status and error details, then retrieve the recorded report, timeline, trace, logs, and output artifacts when available. A dispatch receipt only proves admission. Cancellation is explicit: hla-compass pipeline cancel RUN_UUID or client.cancel_pipeline_run(run_id).

Pipeline UI

hla-compass init research-pipeline --template pipeline-ui
cd research-pipeline
python -m pip install -r backend/requirements.txt
hla-compass lock
hla-compass validate --strict
python -m pytest
hla-compass test --input examples/sample_input.json

The manifest uses computeType: pipeline, async execution, and backend.main:configure. Its @pipeline_module.handler function translates module inputs into a parameters object for Nextflow and may return a run name or output_uri. It is a short configure Lambda, not a Module subclass or the scientific worker. Replace pipeline/main.nf with your actual workflow: the generated file only checks and prints parameters and performs no analysis.

The local test exercises configure only. The standalone UI previews the form; it does not submit cloud compute. Use the module publishing flow and wait for its terminal result. The platform loads the published, verified custom bundle.js only when its integrity hash is available. Test inputs, submission, failure display, and returned results in the actual hosted UI.

Pipeline-backed Modules use prepare_module_run and start_prepared_module_run, or MCP prepare_selected_module_run, for the reviewed version, inputs, resource profile, and reservation. An accepted linked Module Run and Pipeline Run still need a successful scientific execution check.

Reproducibility and existing external workflows

Retain the resolved workflow revision/package, declared worker image digests, parameters, samplesheet, references, resource profile, Run IDs, and output artifacts. The platform records workflow and run metadata, but a plain S3 URI does not pin the bytes consumed by every downstream Nextflow task. Exact object version/checksum consumption for new named-source pipeline selections is an open release decision; do not describe that path as verified reproducible execution. Existing explicit samplesheet/parameter URI launches remain distinct from this unfinished guarantee.

The existing external AB-MHCQuant workflow remains separate and unchanged. Normal-UI replacement of its whole-dataset execution is deferred. Keep its pinned runner definition, image, configuration, roles, and artifacts; do not refresh or disable it as part of this release. It should be replaced only after its normal-UI execution and results have been verified.