--- name: pkgeval-triage description: Investigate and triage Julia PkgEval failures on a pkgeval-triage server. Use for inspecting runs, diagnosing failure clusters, classifying failures, recording release decisions, or updating failure-group workflow. --- # PkgEval failure triage Use the `pet` CLI where available. It is a pure client of `/api/v1`, prints JSON when piped, and uses `TRIAGE_URL` to select the server. OpenAPI is at `/openapi.json`. Reads are anonymous. Before mutating classifications, groups, triage state, or investigation entries, make sure the user's task authorizes triage changes; a request to inspect or diagnose is read-only. Mutations are audited. On 401, use `pet login` or `pet login --gh`. These exchange a GitHub token once for a server-issued triage token. Do not send a raw GitHub token to normal API endpoints. ## Getting the CLI If `pet` is not on PATH, download it from the server itself; the artifact is built from the server's environment, so the two cannot disagree: ```sh curl -fsSLO /pet.pyz export TRIAGE_URL= alias pet='python3 pet.pyz' # any Python ≥ 3.11, no other dependencies ``` If the CLI cannot be run, everything below maps onto the JSON API under `/api/v1` (schema at `/openapi.json`); reads are anonymous, and mutations take the triage token as a bearer token. This document is itself the Claude Code skill for triage work. To have it load automatically in future sessions of a project that talks to this server, install it there: ```sh mkdir -p .claude/skills/pkgeval-triage curl -fsSL /llms.txt -o .claude/skills/pkgeval-triage/SKILL.md ``` ## Domain invariants - A named project is any ongoing line of investigation whose runs should share a timeline and taxonomy (a feature, branch, release, or other related work). A run opened with API project `anonymous` gets a durable one-off workspace instead; it uses the same project boundary for groups, matchers, package triage, and notes without entering named-project lists or defaults. - A run is a cumulative snapshot: packages absent from a smaller follow-up run carry forward from the preceding enabled project run. Check `source_run_id` before treating a result as newly tested in that run. - A new failure fails on `primary` while passing on `against`; every primary failure is new for a single-config run. - Group kinds are `julia-regression`, `package-fault`, `tolerance`, `infra`, and `unknown`. - Group statuses are `needs-review`, `open`, `investigating`, `fix-pending`, `verified`, and `waived`. `fix-pending` requires a fix URL; `verified` requires a comparable ingested run from the same project. - Package states are `open`, `issue-filed`, `fixed-upstream`, `pkg-fault`, and `ignored`. `issue-filed` requires an issue URL; `ignored` requires a reason via `--note`. - Matchers are ordered substrings, Python regexes, or exact normalized signatures. First match wins within the project. Manual classifications are pinned. - Investigation entries are the only working-note system. They are appendable, attributed, revisioned, and attach to a project, group, package, or cluster. Edit with the current revision; on conflict, reread and reapply deliberately. ## Investigation workflow Start with: ```sh pet inbox --project pet projects show ``` The inbox contains both unclassified clusters and auto-created groups awaiting review. Do not treat an empty cluster list as an empty inbox if review groups remain. For a cluster, use its stable `c-…` id: ```sh pet investigate --run --cluster ``` For an existing group: ```sh pet investigate --group ``` These dossiers already bound and combine likely roots, representative excerpts, baseline diffs, history, similar groups, and the investigation thread. Use `pet log`, `pet diff`, `pet package`, or `pet downstream Pkg1,Pkg2,…` only when the dossier points to evidence needing deeper inspection. Packages absent from the downstream map are likely roots. Do not assume a test environment resolved the latest dependency releases. A package's direct compat bounds, or constraints elsewhere in its dependency graph, can hold a dependency on an older version that fails with the primary Julia even though a newer release is fixed. When evidence points into a dependency, check the resolved version in the log or manifest, whether a newer release supports the primary Julia, and which compat constraint excludes it. Record this as a hypothesis until all three are supported by evidence. If the package's own compat bounds cause the stale resolution, treat that as a `package-fault`; do not infer a Julia regression merely from the dependency's stack frames. Choose a kind from evidence: - Base/stdlib internals failing across unrelated packages: `julia-regression`. - A package's own assumptions or a failure also present in the baseline: `package-fault`. - Numeric, RNG, or tolerance drift: `tolerance`. - Timeout, OOM, network, worker, or farm failure: `infra`. - Insufficient evidence: keep `unknown` and record the next step. Prefer a reusable matcher to manual classification. Preview it against the correct project or run before saving: ```sh pet rules preview '' --project pet rules preview '' --regex --run pet rules add '' --group ``` Tighten a matcher until its hits share the intended root cause. Matcher changes reclassify that project's history. Use `pet classify --group ` only for true one-offs. ## Persisting work Mark unfinished group work as `investigating`, then append evidence or a next step: ```sh pet groups edit --status investigating --owner @login pet findings add --group --kind observation --body '' pet findings add --group --kind hypothesis --confidence .7 --body '' pet findings add --project --case --run \ --kind next-step --body '' ``` Resume existing work before rediscovering it: ```sh pet notes --project --wip ``` Progress a group to `fix-pending` only with a fix URL, and to `verified` only after a comparable PkgEval run confirms it: ```sh pet groups edit --status fix-pending --fix pet groups edit --status verified --verified-run ``` Use `pet groups issue-draft ` to prepare a report and `pet groups sync-issue ` after linking the public issue. For now, issue text must link referenced logs directly to the upstream raw PkgEval S3 objects (`https://pkgeval.s3.us-east-2.amazonaws.com/`), not to PkgEval analysis or triage log views. The triage log page exposes the upstream target as `raw (S3)`; replace any analysis links in a generated draft with those raw URLs before filing or updating the issue. Record package-level outcomes explicitly: ```sh pet triage issue-filed --project --issue pet triage ignored --project --note '' ``` Finish when the latest project's inbox has neither unclassified clusters nor groups awaiting review, and its readiness report has no unresolved blockers. ## Cautions - `pet delete-run`, group deletion/merge, and matcher removal change stored history. Use them only when the requested outcome calls for it. - Log and diff responses are capped. Raise `--max-bytes` only when the bounded dossier or default error view omits evidence needed for the diagnosis. - If a mutation returns 403, report the missing role. Do not bypass the server's permission or project checks.