Skip to content

Build modules

A module is a versioned analysis unit with a manifest, runtime entrypoint, tests, and an input/output contract. HLA-Compass supports Lambda, Fargate, and Batch compute in this release. Pipeline-backed Module definitions can be authored, published, and run. The legacy docker manifest value is only a publish alias and is normalized to Fargate.

These instructions use SDK 5.9.2. Confirm the target environment supports the operations you need in its live OpenAPI document. Existing scientific modules need a reviewed dependency/runtime migration, republication, and domain tests.

Start with the publishing prerequisite ladder. The source-upload path is the default and needs no registry or signing setup; custom images use the advanced tier.

Scaffold

For dev/staging, download the exact SDK candidate from TestPyPI; install its dependencies from PyPI:

python -m venv .venv
source .venv/bin/activate
python -m pip download --index-url https://test.pypi.org/simple/ --no-deps --only-binary=:all: --dest sdk-wheel "hla-compass==5.9.2"
python -m pip install "hla-compass[authoring]==5.9.2" --index-url https://pypi.org/simple/ --find-links sdk-wheel
hla-compass init peptide-summary --template no-ui
cd peptide-summary

The authoring extra installs the test and MCP tooling used by the generated module's local authoring workflow. Runtime module requirements remain separate in backend/requirements.txt.

Supported scaffolds include lambda, no-ui, ui, batch, pipeline-ui, and mcp. Run hla-compass init --help for the templates provided by your installed SDK release.

Every scaffold includes:

  • manifest.json, the platform contract;
  • a backend entrypoint and focused tests;
  • SKILL.md, the canonical version-aligned development reference;
  • DATA_ACCESS.md, the linked typed-data and storage contract;
  • INTEGRATIONS.md, the linked catalog, API, and MCP integration reference;
  • AGENTS.md, a bounded Codex loader that points to the managed guidance; and
  • CLAUDE.md, a short orientation that points to that documentation set.

Composite manifests also receive UI_EXTENSION.md and/or MCP_CAPABILITIES.md when the base compute template needs an additional feature reference: React UI authoring, or native MCP tools.

Together those generated files are self-contained within the scaffold and must remain useful outside the platform repository. Do not replace their commands or contracts with private repository links.

Refresh SDK guidance in an existing module

Installing a new SDK does not silently rewrite an existing module. Check and then update its generated guidance explicitly:

hla-compass skill check .
hla-compass skill update .

The managed update is limited to SKILL.md, DATA_ACCESS.md, INTEGRATIONS.md, the concise CLAUDE.md loader, and any required composite feature references. It also creates or refreshes one bounded block in AGENTS.md while preserving existing project-specific text. It does not rerun hla-compass init or replace module source. Instructions between the hla-compass-user-guidance:start and hla-compass-user-guidance:end markers are preserved. Other custom edits and unrecognized legacy guidance fail closed. After reviewing those files, use --force only for an intentional migration; the previous files are backed up by content hash under .hla-compass/backups/sdk-guidance/ by default and ignored from Git and SDK Docker/source build contexts.

Python automation can use the equivalent inspect_module_skill() and update_module_skill() functions from hla_compass.

skill check also verifies that backend/requirements.txt pins the exact SDK release supplying the guidance. A runtime-gap result is intentionally non-zero: update the dependency pin deliberately, regenerate the module lock file when present, and rerun validation/tests. Guidance updates never silently rewrite executable dependencies.

API and MCP integrations

APIClient supports both an interactive bearer session and a headless developer API key. A developer key needs the applicable module scopes and read/write permissions; cancellation also requires delete. See API authentication for issuance and scope requirements. The SDK selects the route family for the active credential automatically.

Authenticate with hla-compass auth login --env dev for an interactive session. Prepare the exact version, inputs, execution mode, and profile before starting paid compute:

from hla_compass import APIClient

client = APIClient(environment="dev")
plan = client.prepare_module_run(
    "MODULE_UUID",
    version="1.0.0",  # An accessible, deployed version of your module.
    parameters={"name": "world"},  # Must match that version's input schema.
    mode="async",  # Must be supported by the module.
)
print(plan)

Review the returned cost and resolved compute profile. Only after approving the plan, use the complete unchanged response in the same Python session:

run = client.start_prepared_module_run(plan)
result = client.wait_for_module_run(run["run_id"])
print(result)

Persist the prepared plan and returned Run ID. After an uncertain start, resend the same plan; do not prepare a replacement merely because a response was lost. An unused expired plan or changed price requires a fresh plan and review. The waiter fetches the result after successful completion and raises APIError on failure, cancellation, or timeout. A wait timeout does not cancel the run.

Set HLA_API_KEY for a headless process. Developer keys use /v1/api/module-runs; bearer sessions use /v1/module-runs. Publish-only keys and short-lived module-run keys cannot use the run control plane.

For a complete raw-HTTP walkthrough against a published module, including the 202 detached-run contract and the poll loop, see Run Off Target Prediction over the API.

Accessible, eligible deployed modules are discoverable through the hosted Platform MCP server. Run hla-compass mcp gateway --env dev for a local MCP client, or connect a supported chat application's remote connector as described in MCP. Local hla-compass mcp schema output is a manifest-derived preview; hosted names, inputSchema, and annotations from tools/list are authoritative.

Use prepare_selected_module_run to obtain the selected module's costed plan, then review its returned run_selected_module next action. Keep admissionId, requestFingerprint, frozen dataContext, and execution controls unchanged when confirming or retrying that action. Hosted module parameters live under inputs; platform controls are siblings, not scientific inputs. Set confirm to true only after approval. The hosted transport consumes confirm before dispatch. Local module stdio tools accept the manifest payload directly; they are not the hosted run-admission interface.

Native MCP authoring

A module may expose additional MCP tools by declaring mcp.tools in manifest.json. Native MCP tools are supported only by the Tier 1 managed source-upload path; custom-image publication rejects a non-empty mcp.tools array.

Declare each tool with a unique name, an object-root Draft-07 input schema, and an entrypoint in a dedicated backend source file:

{
  "mcp": {
    "tools": [
      {
        "name": "profile_terms",
        "description": "Profile normalized query terms",
        "entrypoint": "backend.mcp_handlers:profile_terms",
        "inputSchema": {
          "type": "object",
          "required": ["query"],
          "properties": {
            "query": {"type": "string", "minLength": 1}
          },
          "additionalProperties": false
        }
      }
    ]
  }
}

Keep backend/mcp_handlers.py finite and statically identifiable. Its module-level body may contain only an optional docstring and plain synchronous function definitions:

"""Native MCP handlers."""


def profile_terms(input_data, context):
    import re

    del context
    terms = re.findall(r"\S+", input_data["query"])
    return {"terms": terms, "total": len(terms)}

Use the exact def tool_name(input_data, context): header: two required positional parameters and no defaults or annotations. Do not use decorators, async, yield, positional or keyword variadics, or keyword-only parameters. Do not place imports, constants, classes, assignments, aliases, executable statements, or helper dependencies at module scope; put required imports inside the handler and keep each handler self-contained.

Managed intake statically binds the exact handler definition and source hash to the canonical manifest, source archive, and final image. Runtime invokes a fresh callable recovered from that attested definition. This is source-binding evidence, not a hostile-code sandbox; handler code still executes within the normal module container boundary.

For an existing native-MCP module, hla-compass skill update refreshes managed guidance only. It does not move or rewrite executable Python or change manifest.json. Migrate the handler into the finite source shape above, update every mcp.tools[].entrypoint, add focused tests, and then run:

hla-compass skill update .
hla-compass validate --strict
pytest
hla-compass mcp schema
hla-compass test --input examples/sample_input.json

Publish the migrated module through managed source upload. A local validation or custom-image signature cannot substitute for the platform-owned native-MCP source attestation.

Implement the runtime contract

Keep the backend entrypoint deterministic and explicit about input validation. Return structured output that matches the manifest. Lambda, Fargate, and Batch scaffolds use a Module subclass at an entrypoint such as backend.main:MyModule. Pipeline-backed Modules use a configure callable decorated with @pipeline_module.handler; see Pipeline UI. Use the SDK's runtime data and storage helpers instead of assembling tenant bucket names or database schemas.

Modules may see only the catalog, Catalog Version, organization, and storage capabilities granted to the run. They must not infer or broaden that scope. Prefer a canonical Catalog UUID input and use self.bind_catalog_id(catalog_id) before typed reads; native MCP and external API code uses api.for_catalog_id(catalog_id). Exact provider/catalog inputs may use bind_catalog(provider, catalog) and for_catalog(provider, catalog). All four fail rather than selecting another visible or configured Catalog. Catalog binding does not select a Catalog Version, so require a separate Version UUID for every reproducible version-sensitive read.

For admitted module inputs, use self.context.data_selection.load_manifest(self.storage) for a named-input manifest and self.context.data_selection.load_source(name, self.storage) for one recorded source when the run supplies those capabilities. These read through the run's recorded input lineage. Do not replace them with raw S3 reads that select whichever object happens to be current. General Catalog storage credentials are unavailable in this release; Catalog binding does not grant arbitrary bucket access. See Data and ingestion for typed reads and the separate governed ingestion process.

New Fargate and Batch module publications run as UID/GID 1000:1000 with a read-only root filesystem and a writable /tmp. Use tempfile or HLA_COMPASS_WORKDIR (default /tmp/hla-compass) for scratch and local result files. HOME, temporary directories, and supported language/model caches are redirected below /tmp. Do not write beside installed code, install dependencies at runtime, or assume the working directory is writable. Scratch is ephemeral; return artifacts through the SDK result contract so they are persisted before the task exits. This module policy does not claim strict filesystem hardening of existing Nextflow worker images.

Keep output files in Results

Upload files with self.storage.save_csv("scores.csv", dataframe), self.storage.save_figure("plot.pdf", figure, format="pdf"), or self.storage.save_file("report.pdf", pdf_bytes, "application/pdf"). These helpers upload into the admitted run's private files/ directory. Writing a file to container scratch alone does not upload it.

In an organization bucket with storage-event projection deployed, successful uploads appear in Runs → Outputs shortly afterward; use Refresh output files if needed. Downloads use the recorded S3 object version and fail visibly if that version is no longer available. A later upload to the same path updates that file's registered output; use distinct filenames for outputs you want to retain separately. Files from failed runs can also appear; their presence does not mean the analysis succeeded.

This automatic registration covers new run-file uploads in governed organization buckets. It does not scan historical files or legacy shared-results buckets. Historical files and existing explicit artifact registrations remain available through their existing paths; an older registration without a recorded version does not provide a version-pinned download.

Manifest inputs

The root inputs member of manifest.json is the JSON Schema used to validate module parameters at runtime. Declare an object schema with properties and, when applicable, required; the submitted input names and values must satisfy that schema before the module entrypoint runs. Keep it aligned with the module's Pydantic Input model and rerun hla-compass validate --strict after changing either contract.

Module contracts use JSON Schema Draft-07. $schema may be omitted or must be exactly http://json-schema.org/draft-07/schema#. inputs must accept an object root; a native MCP inputSchema, when supplied, must explicitly declare "type": "object". outputs must also accept an object root because the SDK runtime and MCP structured output expose the Module result envelope. External $ref values, unresolved local references, nested $id values, and post-Draft-07 keywords such as prefixItems, dependentRequired, and unevaluatedProperties are rejected at validation, publish, discovery, and runtime boundaries. Local fragment references remain supported.

Every default and every entry in examples must satisfy the schema where it is declared. Supply --input examples/sample_input.json explicitly for local runtime tests. The test command otherwise tries the scaffold sample file before schema defaults/examples; synthesized values are only a convenience, not representative scientific test data. Required inputs still need valid values.

Module.sync_manifest() updates only a complete, valid existing manifest and replaces it atomically. For Pydantic v2 models it translates the supported 2020-12 forms ($defs, tuple prefixItems, and dependency keywords) to their Draft-07 equivalents and preserves discriminator metadata. Unsupported or lossy schema constructs fail without changing the original file; always review the diff and rerun strict validation and tests.

Validate and test

From the module directory, install its backend dependencies into your authoring virtual environment, generate the Linux dependency lock, and check the contract:

python -m pip install -r backend/requirements.txt
hla-compass lock
hla-compass validate --strict
python -m pytest
hla-compass check --json

hla-compass lock requires Docker and resolves a complete artifact-hashed backend/requirements.lock.txt in the managed Linux build image. Commit that file with backend/requirements.txt; rerun locking after changing dependencies. check is Docker-free and reports skipped checks and notVerified explicitly. It does not prove that a container, UI bundle, or deployed run works.

For Lambda, Fargate, and Batch modules, also test the packaged runtime:

hla-compass test --input examples/sample_input.json

That command requires Docker and verified managed build images, including the UI builder when applicable. For pipeline-backed modules, hla-compass test runs the configure handler in-process, without Docker. Also run the generated tests/test_configure.py with pytest. A passing configure test proves parameter translation, not Nextflow execution. Native MCP authors should also run hla-compass mcp schema.

UI modules

SDK 5.9.2 includes UI kit 0.2.0. Its UI scaffolds include a versioned frontend/vendor/ui-kit package and UI_STYLE.md. The normal npm build installs that local package, so no separate npm publication or account is required. Inspect the example under Documentation → UI kit. Read UI_STYLE.md for components, agent instructions and explicit adoption steps for existing modules. Refreshing guidance does not upgrade executable UI code. The kit inherits the Compass theme and keeps the existing module windows and execution contract.

The current UI host loads an approved module's bundle.js into the host page. Keep the scaffold's production bundle build and React exports intact. The platform supplies input, onInputChange, onExecute, executionStatus, result, error, dataContext, and hostContext; use these to render inputs, submit through the host, and show results. onExecute returns the host result envelope, while the result prop contains the result payload. Do not embed platform credentials or create a second run-submission implementation.

For a follow-up module, feature-detect hostContext.openModule and pass {moduleId, input, dataContext} using an accessible target ID and inputs that match its manifest. hostContext.listModules, when provided, supplies visible targets. Opening another module prefills a new window; it does not start or approve that module's run. These callbacks may be absent in a local preview.

Display dataContext alone does not attach governed run inputs. With SDK 5.8 and a host exposing supportsRunDataContextHandoff, pass the original hostContext.launchDataContext explicitly to openModule or the kit's ModuleHandoff when the follow-up should consume that same saved-view selection. The target prepares a new authorized, costed plan; a previous approval is never carried over. For independent inline inputs, omit the run binding.

A custom UI must be published, verified, and carry an integrity hash before it can render. A blocked or missing custom interface is not proof that its backend is unavailable; inspect the module's standard run surface and deployment state. Local previews only test the form. Verify the published UI, submission, error state, and returned artifacts in the target environment.

The managed UI builder uses npm 12. Dependency lifecycle scripts are skipped unless frontend/package.json contains a reviewed, version-pinned allowScripts approval. When a required native/build dependency reports a skipped script, inspect it with npm install-scripts ls and record the narrow approval with npm install-scripts approve <package>; never enable every dependency script as a blanket workaround.

Publish

Every current scaffold includes a self-contained .github/workflows/publish.yml for CI source publishing. It installs the exact SDK release named by the scaffold from PyPI and does not call or check out the private platform repository. Keep that exact pin until an intentional scaffold upgrade; do not replace the workflow with a reusable @main reference.

hla-compass publish \
  --env dev \
  --scope org \
  --idempotency-key peptide-summary-1.0.0-release-1 \
  --wait

Source publishing sends the module source to the platform. The platform builds the image, scans it, verifies policy, and registers the version. A local build or upload receipt is not proof that a version is deployable; wait for the terminal publish status and address typed validation or security failures. The source archive is uploaded directly to platform storage with a presigned PUT, with a maximum compressed source archive of 2 GiB. Other intake, image, and runtime limits still apply. Keep generated datasets and large reference data out of the source tree; use explicitly authorized runtime inputs rather than baking another copy into every release. The custom-image publication path remains for modules that need a base image the platform builder cannot produce. Managed intake always generates the Dockerfile; root Dockerfile and Dockerfile.hla files are omitted by the SDK and rejected if supplied through a custom client. Build-image releases are also fail-closed: a compute runtime or UI builder that has not passed its release gates cannot silently fall back to a mutable tag.

Persist the idempotency key with the release attempt. Reuse it only after an uncertain response for the same organization, manifest, scope, and source bytes. An exact replay returns the original build without a second upload or rate-limit charge; changing any of that evidence while reusing the key returns 409 IDEMPOTENCY_CONFLICT. Re-signing the unchanged manifest is still an exact replay: after successful verification the platform normalizes only randomized RSA-PSS signature bytes and continues to bind the signed content and signer. Canonicalization omits signature metadata only at the manifest root and omits the platform-owned root integrity value. A reserved signature-metadata key below the root is rejected at both publication intake gates, including for an otherwise-unsigned org manifest. Nested integrity fields stay signed, so changing them is a verification failure.

The reserved names are signature, publicKey, public_key, signatureAlgorithm, signature_algorithm, hashAlgorithm, hash_algorithm, keyFingerprint, and key_fingerprint. Do not reuse them as input/output property names, environment variables, or extension metadata.

Redeploy an existing version

Source publishing already deploys an accepted version. Use the explicit deployment API only for an operational redeploy of an existing governed version, and only from an interactive bearer session:

from hla_compass import APIClient

client = APIClient(environment="dev")  # run `hla-compass auth login` first
receipt = client.deploy_module(
    "MODULE_UUID",
    version="1.2.3",
    deployment_target="auto",
)
deployment = client.wait_for_module_deployment(
    receipt["module_id"],
    receipt["deployment_id"],
    timeout=900,
    poll_interval=5,
)

The receipt proves durable admission, not successful deployment. Its dispatch_state is submitted, pending, or unchanged; use get_module_deployment() for one status read or the waiter above to poll. The waiter returns only for deployed. It raises APIError for failed, cancelled, or superseded, preserving the platform error message. Developer API keys, publish-only keys, and short-lived module-run credentials cannot use these methods. Callers may select only the existing version, deployment target, and runtime configuration. They cannot replace the governed package, object path, container reference, manifest, or persisted scan/schema evidence during redeployment.

Before a production publish:

  • pin dependencies and test the built artifact;
  • keep secrets outside the module tree and run a repository secret scan—the SDK excludes common credential filenames and rejects included symlinks, but that bounded filename policy is not content-aware secret detection;
  • confirm the requested data and network permissions are minimal;
  • review credit and compute implications; and
  • publish an immutable semantic version with release notes.