Commit b873742
Eric Bower
·
2026-08-15 09:42:28 -0400 EDT
parent a5aaf30
docs: props
1 files changed,
+156,
-138
+156,
-138
1@@ -2,177 +2,195 @@
2
3 ## 1. Executive Summary
4
5-Migrating existing projects away from GitHub Actions to a dedicated self-hosted CI system is difficult due to ecosystem inertia and configuration friction.
6-
7-This proposal outlines a strategy to **integrate `pici` with GitHub Actions as a drop-in execution engine**. Developers write standard, parallel bash workflows (`pico.sh`) using `zmx`, execute them identically on both local workstations and GitHub Actions runners, and gain **live, interactive terminal debugging on failure** powered by `pipe.pico.sh`.
8-
9-```mermaid
10-flowchart TD
11- subgraph Local Environment
12- Dev[Developer] -->|Runs locally| LocalScript["./pico.sh (or pici run)"]
13- LocalScript -->|PTY Sessions| LocalZMX["zmx (Parallel Tasks)"]
14- end
15-
16- subgraph GitHub Actions Runner
17- GHA[GitHub Actions Workflow] --> InstallZMX["Install zmx"]
18- InstallZMX --> RunScript["Run ./pici-debug.sh"]
19- RunScript --> RunnerZMX["zmx sessions (Host & Docker)"]
20-
21- RunnerZMX -->|Success| GHASuccess["Finish Job (0)"]
22- RunnerZMX -->|Failure| CatchError["Catch non-zero exit code"]
23-
24- CatchError --> StartBridge["Open bidirectional pipe to pipe.pico.sh"]
25- StartBridge --> AllocPTY["Allocate interactive PTY (script / bash -i)"]
26- end
27-
28- subgraph Pico Cloud Relay
29- PipeRelay["ssh pipe.pico.sh pipe {topic}"]
30- end
31-
32- StartBridge <-->|SSH stdin/stdout| PipeRelay
33- Dev -.->|ssh -t pipe.pico.sh pipe {topic}| PipeRelay
34-```
35+Migrating existing projects away from GitHub Actions to a dedicated self-hosted CI system is difficult due to ecosystem inertia and configuration friction.
36+
37+This proposal outlines a strategy to provide an **official custom GitHub Action (`pico-sh/pici-action`)** powered by `pici run`. Developers write standard, parallel bash workflows (`pico.sh`) using `zmx`, execute them with 1:1 parity on both local workstations and GitHub Actions runners, and gain **live, interactive terminal debugging on failure** powered by native `pipe.pico.sh` bridging in Go.
38
39 ---
40
41 ## 2. Core Design Goals
42
43-1. **1:1 Local & Remote Parity**: The exact same `./pico.sh` runs locally and on CI without emulation layers or proprietary YAML DSLs.
44-2. **Native Docker Compatibility**: Seamless support for `docker run ...` steps in `pico.sh` within standard GitHub-hosted runners (`ubuntu-latest`).
45-3. **Interactive Debug on Failure ("Attach")**: When a task fails on GitHub Actions, the runner pauses and establishes an outbound reverse PTY bridge via `pipe.pico.sh`.
46-4. **Zero-Config Pico Authentication**: Key-based authentication directly through the developer's SSH private key—no usernames or external tunnel tokens required.
47+1. **1-Step GitHub Action Experience**: A custom composite action (`pico-sh/pici-action@v1`) that installs `pici` + `zmx`, configures SSH keys, executes the workflow in-place, and uploads artifacts with zero boilerplate.
48+2. **1:1 Local & Remote Parity**: `pici run` provides identical execution semantics across environments: metadata environment variable injection (`PICI_JOB`, `PICI_REPO`, `PICI_BRANCH`, `PICI_COMMIT`), `pico.sh` execution via `zmx`, live progress streaming, artifact rendering, and exit code propagation.
49+3. **Explicit Workspace Semantics**: By default, `pici run` rsyncs to an isolated `/tmp` workspace to keep local working trees clean. In CI environments where the runner is already an ephemeral container/VM, an explicit `--in-place` flag instructs `pici run` to execute directly without rsyncing.
50+4. **Native Docker Compatibility**: Mounts like `$(pwd):/app` work seamlessly in both local and CI environments.
51+5. **Native Go Failure Hook (`pici debug`)**: When a job fails, `pici` directly opens a secure, authenticated PTY bridge to `pipe.pico.sh` in Go, avoiding brittle bash FIFOs or background SSH wrappers.
52+6. **Security & Secret Scrubbing**: `pici` controls the debug shell environment and scrubs `GITHUB_TOKEN` and repository secrets before spawning interactive shells.
53+
54+---
55+
56+## 3. The Custom GitHub Action: `pico-sh/pici-action`
57+
58+We are creating and publishing an official custom action—`pico-sh/pici-action`—that serves as the turnkey entrypoint for GitHub Actions users.
59+
60+### Standard Workflow (`.github/workflows/ci.yml`)
61+
62+With the custom action, adding Pici CI with remote debugging to any repo is a single step:
63+
64+```yaml
65+name: CI
66+on: [push, pull_request]
67+
68+jobs:
69+ ci:
70+ runs-on: ubuntu-latest
71+ steps:
72+ - uses: actions/checkout@v4
73+
74+ - name: Run CI with Pici
75+ uses: picosh/pici-action@v1
76+ with:
77+ pico_ssh_key: ${{ secrets.PICO_SSH_KEY }}
78+```
79+
80+### Action Configuration & Inputs
81+
82+| Input | Description | Default |
83+| :--- | :--- | :--- |
84+| `pico_ssh_key` | SSH private key registered with pico.sh for authenticated debug relay access. | `""` (optional, required for debug relay) |
85+| `debug_on_fail` | Automatically open an interactive `pipe.pico.sh` debug bridge if a job fails. | `"false"` |
86+
87+### What `pico-sh/pici-action` Does Under the Hood
88+1. **Tool Installation**: Installs the static `pici` and pinned `zmx` binaries (with SHA256 checksum verification) to `/usr/local/bin`.
89+2. **Credential Setup**: Writes `pico_ssh_key` to a secure temp keyfile or SSH agent if provided.
90+3. **Environment Propagation**: Passes GitHub Actions run metadata to `pici run`:
91+ - `PICI_JOB="${GITHUB_RUN_ID}"`
92+ - `PICI_EVENT="git.${GITHUB_EVENT_NAME}"`
93+ - `PICI_BRANCH="${GITHUB_REF_NAME}"`
94+ - `PICI_COMMIT="${GITHUB_SHA}"`
95+4. **Execution (`pici run`)**: Invokes `pici run --in-place` (with `--debug-on-fail` if enabled).
96+5. **Job Summaries & Artifact Upload**:
97+ - `pici` formats the run status table and failed logs into `$GITHUB_STEP_SUMMARY`.
98+ - If `upload_artifacts` is enabled, the action uploads `/tmp/pici-artifacts/<repo>/<jobid>/` as a workflow artifact.
99+
100+---
101+
102+## 4. Direct CLI Execution (Without the Action)
103+
104+For users who prefer manual step-by-step control or non-GitHub CI environments, `pici run` can be invoked directly:
105+
106+```yaml
107+ - name: Run CI
108+ run: |
109+ pici run --in-place \
110+ -e PICI_JOB="$GITHUB_RUN_ID" \
111+ -e PICI_EVENT=git.push \
112+ -e PICI_BRANCH="$GITHUB_REF_NAME" \
113+ -e PICI_COMMIT="$GITHUB_SHA"
114+```
115
116 ---
117
118-## 3. Remote Debugging Architecture (`pipe.pico.sh`)
119+## 5. Native Go Debug Bridge (`pipe.pico.sh`)
120
121-### Why `pipe.pico.sh`?
122-`pipe.pico.sh` provides authenticated, bidirectional byte streaming (`io.ReadWriter`) across topics over SSH. Because connections are outbound from the runner, no inbound ports or complex firewall configurations are necessary.
123+Rather than relying on bash scripts with named FIFOs (`mkfifo`) and background `ssh -N` commands, the relay bridge is implemented directly in Go inside `pici`:
124
125-1. **Authentication & Topic Privacy**:
126- - The GitHub Actions runner authenticates using the developer's `PICO_SSH_KEY` secret.
127- - Topics created by an authenticated SSH key are automatically private and isolated to that account.
128-2. **PTY Bridging**:
129- - Standard shell redirection (`A | B`) does not allocate a pseudo-terminal (PTY) and is unidirectional.
130- - By creating named FIFOs (`mkfifo`) and wrapping an interactive shell with `script -q -f -c "bash -i"`, the bridge forwards full PTY escape sequences, window sizes, and raw terminal input.
131+### How it works
132+1. **Hook into Failure Path**: When `resolveJobExitCode()` detects a non-zero exit in `waitAndReport()` and debug mode is enabled (e.g. `--debug-on-fail` or `-e PICI_DEBUG=1`), `pici` initializes the debug session.
133+2. **Native SSH Dialing**: `pici` dials `pipe.pico.sh` using the existing `-pk` / `-ck` SSH key infrastructure.
134+3. **PTY Allocation (`creack/pty`)**: Spawns an interactive shell (`bash -i` or `$SHELL`) connected to the bidirectional SSH stream.
135+4. **Attach Instructions & Notifications**: Prints the `ssh -t pipe.pico.sh pipe <topic>` command to stdout and appends it to `$GITHUB_STEP_SUMMARY`.
136+5. **Idle Timeout & Clean Exit**: Waits for a connection with an idle timeout (e.g., 15 minutes). Once closed or timed out, `pici` terminates and exits with the original non-zero job exit code.
137+6. **Standalone `pici debug`**: `pici debug <repo>/<jobid>` attaches to running or stalled jobs by targeting the deterministic session prefix (`ci.<repo>.<jobid>.`).
138+
139+### Security Advantages
140+- **Environment Scrubbing**: Sensitive CI environment variables (`GITHUB_TOKEN`, `ACTIONS_RUNTIME_TOKEN`, injected secrets) are stripped from the debug shell process environment before launching.
141+- **Host Key Pinning**: The `pipe.pico.sh` host key is verified natively in Go without passing `-o StrictHostKeyChecking=no`.
142
143 ---
144
145-## 4. Initial Implementation: `pici-debug.sh`
146+## 6. Implementation & Verification Plan (Test Repository)
147+
148+To validate the complete flow end-to-end before public release, we will use a dedicated test repository (e.g., `picosh/pici-test`) to confirm local parity, GitHub Actions execution, and interactive failure recovery.
149+
150+### Step 1: Create the Test Repository Structure
151+
152+```
153+pici-test/
154+├── .github/
155+│ └── workflows/
156+│ ├── ci-action.yml # Tests the custom picosh/pici-action
157+│ └── ci-direct.yml # Tests direct pici binary execution
158+├── src/
159+│ └── main_test.go
160+├── pico.sh # Core CI script with configurable test modes
161+└── README.md
162+```
163
164-A minimal, pure-bash wrapper that can be placed in any repository to execute CI and open the debug bridge on failure.
165+### Step 2: Implement the Configurable `pico.sh`
166+
167+The test script supports environment variables to deterministically simulate success, failure, Docker steps, and hangs:
168
169 ```bash
170 #!/usr/bin/env bash
171-set -e
172-
173-TOPIC="pici-${GITHUB_RUN_ID:-$RANDOM}"
174-FIFO_IN=$(mktemp -u /tmp/pici-in.XXXXXX)
175-FIFO_OUT=$(mktemp -u /tmp/pici-out.XXXXXX)
176-
177-cleanup() {
178- kill "$SSH_PID" 2>/dev/null || true
179- rm -f "$FIFO_IN" "$FIFO_OUT"
180-}
181-trap cleanup EXIT
182-
183-# 1. Run the CI script
184-echo "==> Running pico.sh..."
185-if ./pico.sh; then
186- echo "==> CI succeeded!"
187- exit 0
188-fi
189+set -euo pipefail
190
191-CI_EXIT_CODE=$?
192+JOB_ID="${PICI_JOB:-local}"
193+REPO="${PICI_REPO:-pici-test}"
194+EVENT="${PICI_EVENT:-local}"
195+ZMX_SESSION_PREFIX="${ZMX_SESSION_PREFIX:-local.}"
196
197-# 2. Check if debug mode is enabled
198-if [ "${DEBUG_ON_FAIL:-1}" != "1" ]; then
199- exit "$CI_EXIT_CODE"
200+echo "==> Running CI for $REPO (job: $JOB_ID, event: $EVENT)"
201+
202+# 1. Parallel test steps
203+zmx run unit-tests -d bash -c 'echo "Running unit tests..."; sleep 2; echo "Tests passed!"'
204+zmx run lint -d bash -c 'echo "Running linter..."; sleep 1; echo "Lint clean!"'
205+
206+# 2. Simulated failure scenario (when TEST_FAILURE=1)
207+if [ "${TEST_FAILURE:-0}" = "1" ]; then
208+ zmx run failing-step -d bash -c 'echo "Simulating fatal compilation error..."; sleep 2; exit 42'
209 fi
210
211-echo "================================================================"
212-echo "🔴 CI FAILED! Debug bridge active on pipe.pico.sh"
213-echo "Connect from your terminal using:"
214-echo " ssh -t pipe.pico.sh pipe $TOPIC"
215-echo "================================================================"
216-
217-# 3. Create FIFOs for bidirectional PTY stream
218-mkfifo "$FIFO_IN" "$FIFO_OUT"
219-
220-# 4. Connect background bridge to pipe.pico.sh using SSH key
221-ssh -N -o StrictHostKeyChecking=no \
222- -o ServerAliveInterval=15 \
223- pipe.pico.sh "pipe $TOPIC" < "$FIFO_OUT" > "$FIFO_IN" &
224-SSH_PID=$!
225-
226-# 5. Spawn interactive PTY bash shell connected to the pipe
227-script -q -f -c "
228- echo '=== Connected to GitHub Actions Debug Shell ==='
229- echo ''
230- echo 'Current zmx sessions:'
231- zmx list || true
232- echo ''
233- echo 'Type exit to finish CI run.'
234- bash -i
235-" /dev/null < "$FIFO_IN" > "$FIFO_OUT"
236-
237-exit "$CI_EXIT_CODE"
238-```
239+# 3. Simulated hang scenario (when TEST_HANG=1)
240+if [ "${TEST_HANG:-0}" = "1" ]; then
241+ zmx run hung-test -d bash -c 'echo "Simulating hung test process..."; sleep 3600'
242+fi
243
244----
245+# 4. Docker container step
246+zmx run docker-step -d docker run --rm alpine:latest sh -c "echo 'Docker step executed inside alpine container'"
247+
248+# 5. Wait for all steps
249+zmx wait "*"
250+
251+printf "\x1b[32mAll test steps succeeded!\x1b[0m\n"
252+```
253
254-## 5. GitHub Actions Workflow Integration
255+### Step 3: Configure GitHub Actions Workflows
256
257-### Workflow Example (`.github/workflows/ci.yml`)
258+#### Test Workflow 1: Using `picosh/pici-action` (`.github/workflows/ci-action.yml`)
259
260 ```yaml
261-name: CI
262-on: [push, pull_request]
263+name: Pici Action Validation
264+on:
265+ push:
266+ workflow_dispatch:
267+ inputs:
268+ test_failure:
269+ description: "Simulate step failure (exit 42)"
270+ type: boolean
271+ default: false
272
273 jobs:
274- pici:
275+ test-ci:
276 runs-on: ubuntu-latest
277 steps:
278 - uses: actions/checkout@v4
279
280- - name: Install zmx
281- run: |
282- # Fetch and install zmx binary
283- sudo curl -fsSL https://github.com/picosh/zmx/releases/latest/download/zmx_linux_amd64 -o /usr/local/bin/zmx
284- sudo chmod +x /usr/local/bin/zmx
285-
286- - name: Setup Pico SSH Key
287- if: always()
288- env:
289- PICO_SSH_KEY: ${{ secrets.PICO_SSH_KEY }}
290- run: |
291- mkdir -p ~/.ssh
292- echo "$PICO_SSH_KEY" > ~/.ssh/id_ed25519
293- chmod 600 ~/.ssh/id_ed25519
294-
295- - name: Run CI with Pici Debug Bridge
296+ - name: Run Pici CI
297+ uses: picosh/pici-action@main
298+ with:
299+ pico_ssh_key: ${{ secrets.PICO_SSH_KEY }}
300+ debug_on_fail: "true"
301 env:
302- DEBUG_ON_FAIL: "1"
303- run: ./pici-debug.sh
304+ TEST_FAILURE: ${{ github.event.inputs.test_failure && '1' || '0' }}
305 ```
306
307----
308-
309-## 6. Docker Considerations
310+### Step 4: Verification Matrix & Acceptance Criteria
311
312-- **Out of the box**: GitHub Actions `ubuntu-latest` runners come with Docker pre-installed and running. `docker run` commands inside `pico.sh` execute directly against the host Docker daemon.
313-- **Debugging Docker Containers**: When attached via the debug shell:
314- - Users can inspect running/stopped containers via `docker ps -a`.
315- - Users can enter failed containers using `docker exec -it <container_id> /bin/bash`.
316- - For steps using `--rm`, users can rerun the command without `--rm` inside the debug shell to inspect intermediate layers.
317-
318----
319-
320-## 7. Adoption & Evolution Roadmap
321-
322-| Phase | Deliverable | Description |
323-| :--- | :--- | :--- |
324-| **Phase 1 (Immediate)** | `pici-debug.sh` Bash Script | Pure script wrapper requiring zero changes to `pici` or Go codebases. |
325-| **Phase 2 (Packaging)** | `pico-sh/pici-action` | Dedicated GitHub Action packaging `zmx` installation, key setup, and step summary outputs. |
326-| **Phase 3 (Native Go Bridge)** | `pici debug` Subcommand | Built-in `pici` CLI command implementing native SSH dialing to `pipe.pico.sh` with `creack/pty`. |
327-| **Phase 4 (Reporting)** | Rich Step Summaries | Exporting `zmx` session history (`zmx history --html`) directly into `$GITHUB_STEP_SUMMARY`. |
328+| Test Scenario | Trigger | Expected GHA Behavior | Acceptance Criteria |
329+| :--- | :--- | :--- | :--- |
330+| **1. Happy Path** | Push commit with `TEST_FAILURE=0` | All `zmx` sessions pass. | • Workflow exits with `0` (green).<br>• GHA Step Summary displays full table with durations.<br>• HTML report uploaded as workflow artifact. |
331+| **2. Failure & Remote Attach** | Dispatch workflow with `test_failure=true` | `failing-step` exits `42`. | • `waitAndReport` catches exit `42`.<br>• Relay bridge opens on `pipe.pico.sh`.<br>• `ssh -t pipe.pico.sh pipe <topic>` printed to logs and step summary.<br>• Connecting via terminal drops into interactive debug shell with active `zmx` sessions.<br>• Exiting debug shell ends GHA job with code `42` (red). |
332+| **3. Secret Scrubbing Audit** | During interactive debug session | Inspect environment variables inside attached shell. | • `GITHUB_TOKEN`, `ACTIONS_RUNTIME_TOKEN`, and `PICO_SSH_KEY` are absent from `env`. |
333+| **4. Docker Step Execution** | Standard run | Alpine container runs via `docker run --rm`. | • Docker mounts `$(pwd)` correctly in-place.<br>• Container logs captured in `zmx history docker-step`. |
334+| **5. Idle Timeout Cleanup** | Failure triggered without developer attaching | Wait for configured idle timeout (e.g. 15m). | • Bridge cleanly closes upon timeout.<br>• GHA runner exits with the original step failure code (does not hang indefinitely). |