CI/CD

Gate a pull request on a Roverly run. Your pipeline uploads a build, starts a run, and turns the verdict into a check.

The integration is an HTTP API — GitHub Actions, GitLab CI, Jenkins, Bitbucket and CircleCI all work the same way, with nothing to install.

Prerequisites

Two things a person sets up once. CI cannot do either, and a pipeline wired before they exist fails every time.

A paired desktop, online. Runs execute on your machine. Install the desktop app, pair it, and connect the device or simulator. iOS on real hardware needs provisioning in place first.

If every paired machine is asleep, the run never starts and your step times out.

A pinned production run, for regression. Do a learn run from the desktop app, then mark it Production run on the runs page.

CI cannot do this: pinning needs a learn run, and learn is refused over the API. Without it, POST /v1/runs returns 400.

Get an API key

Dashboard → Settings → API. Give the key both scopes:

Scope	Allows
`trigger`	Start runs, upload builds
`read`	Poll a run, read findings

Store it as ROVERLY_API_KEY in your CI secret store. It is a bearer token — anyone holding it can spend your run allowance.

Your desktop's key is a different kind and cannot call this API.

Quickstart

curl -fsSL https://roverly.ai/cli/roverly -o roverly && chmod +x roverly

ROVERLY_API_KEY="$ROVERLY_KEY" ./roverly test \
  --project-id "$ROVERLY_PROJECT" \
  --run-type feature \
  --pr-body "$PR_BODY" --pr-title "$PR_TITLE" --pr-number "$PR_NUMBER" \
  --branch "$BRANCH" --commit "$COMMIT_SHA" \
  --build ./app/build/outputs/apk/release/app-release.apk \
  --summary-file roverly-comment.md

Exit code	Meaning
`0`	Check passes
`1`	Check fails
`2`	The tool was invoked wrong

--summary-file writes a pull-request comment you can post as-is.

For a copy-paste version, roverly.ai/cli/snippet.sh is the same gate in about fifteen lines of curl and jq. It skips the build upload and the PR comment.

API reference

Base URL https://api.roverly.ai. Every call needs Authorization: Bearer <key>.

Upload a build

Mobile only. Web projects test a URL and install nothing.

POST /v1/artifacts               → { id, upload_url, upload_content_length, expires_at_iso }
PUT  <upload_url>                → the bytes, direct to storage
POST /v1/artifacts/{id}/complete

Bytes go straight to storage, never through the API. Send exactly the Content-Length returned as upload_content_length — the signature binds it.

Declare size_bytes, sha256, and package_id (Android package or iOS bundle id). The package id is checked against the project's: a mismatch would install one app and launch another.

Builds expire after 7 days. Max 200 MB.

For an iOS simulator build, zip the .app and upload the zip. The archive must contain exactly one .app.

Start a run

POST /v1/runs

Field	Notes
`project_id`	Required
`run_type`	`regression` or `feature`. `learn` is refused.
`goal`	Required for `feature` — what to exercise. Map your PR description to it.
`artifact_id`	The build to install
`target_url`	Web only — a preview deployment to test instead of the project URL
`branch`, `commit_sha`, `environment`	Free-text traceability
`pr_number`, `pr_url`, `pr_title`	Optional PR context

Preview deployments. Previews live off your domain (*.vercel.app, *.netlify.app), so add those hosts to the project's preview-domain allowlist first. Otherwise the run is refused — a stray URL must not be able to aim a run at someone else's site.

Read the verdict

GET /v1/runs/{id}          → { status, ... }
GET /v1/runs/{id}/summary  → { status, reasons, blocking, markdown, superseded }

Poll until status is one of succeeded, failed, cancelled.

processing is not finished. Findings are produced during processing. A client that stops when the status is no longer running reads the verdict before the findings exist and reports a pass on a broken change. Check membership of the three terminal statuses — never "not running".

Then read summary. Branch on status (pass / fail) and post markdown as a comment. It is always HTTP 200; an HTTP error means auth or infrastructure, never a verdict.

Stop polling before 60 minutes. An unclaimed run is cleaned up at 60. Past that you are polling an abandoned run and reporting it as your app's verdict. 45 minutes is a sensible ceiling.

Concurrent pushes

Push six times and you get the in-flight run plus one run for the sixth commit. Not six runs.

A run an agent has started always finishes. A queued run nobody has picked up is replaced by the newer one. Replaced runs do not count against your plan.

When your job is the replaced one, summary returns superseded: true with status: "pass". Exit 0 — a newer run is about to answer the same question, and a red check here is noise.

Failure modes

Cause	What you see
No paired desktop online	Run stays queued; your step times out
Another run in progress	Queued behind it; same timeout
No production run pinned	`400` at trigger, naming the fix
Over your plan's allowance	`402` at trigger
Build missing, expired or wrong package	`400` at trigger; no run starts
Run errored	`summary` fails, naming the error
Replaced by a newer commit	`superseded: true`, exit 0

Anything that is not a pass renders as a failed check. The summary text says which.

Fork pull requests are not special-cased. The API cannot tell a fork PR from any other, so a fork PR starts a real run and spends a real slot. On a public repo that is usually wrong — both for cost and because a contributor's token generally cannot post the comment back. Gate it in your pipeline. On GitHub Actions:

if: github.event.pull_request.head.repo.full_name == github.repository

Plans and limits

No plan gates this API and there is no separate CI quota. A CI-triggered run draws on the same monthly pool as one started from the desktop.

Plan	Runs / month	Past the pool
Starter	8	`402` until the month rolls over
Pro	40	Metered as overage; runs continue
Enterprise	Per contract	Per contract

On Starter that means eight gated pull requests a month; the ninth POST /v1/runs returns 402 rather than queueing.

On Pro, overage is funded by prepaid credits. A run needs $2.00 available to start once the pool is used up — add credits under Settings → AI.