gitshark

Clone repository

git clone https://gitshark.de/git/workaround/Gitshark.git
git clone git@gitshark.de:workaround/Gitshark.git

← Commits

✨ (ci): Flow job outputs through needs

345afebec772f6eda5d20d225894128ec9681dfb · Michael Hainz · 2026-07-23T06:48:46Z

Changes

13 files changed, +267 -37

MODIFY README.md +2 -2
diff --git a/README.md b/README.md
index 0debbf1..d4a60ad 100644
--- a/README.md
+++ b/README.md
@@ -90,8 +90,8 @@
90 90 which a runner claims, executes, and streams logs for — visible on the repository's **Actions** tab;
91 91 a vanished runner's task is reclaimed after a timeout. Jobs are matched to runners by `runs-on`
92 92 labels, ordered by `needs` dependencies, and repository owners manage encrypted secrets and
93 - variables that are delivered to runners. Non-push events, `needs` outputs, `matrix`, and artifacts
94 - are follow-up phases. Guides: [for users](docs/users/ci-runners.md), [for admins](docs/admins/ci-runners.md),
93 + variables that are delivered to runners. Non-push events, `matrix`, and artifacts are follow-up
94 + phases. Guides: [for users](docs/users/ci-runners.md), [for admins](docs/admins/ci-runners.md),
95 95 [architecture](docs/maintainers/ci-runners.md)
96 96 activities from; local users can in turn follow a remote repository — or a whole remote user, whose
97 97 public repositories are then followed and shown grouped — and read their pushes (see below)
MODIFY docs/admins/ci-runners.md +3 -2
diff --git a/docs/admins/ci-runners.md b/docs/admins/ci-runners.md
index 028d884..fbd2cb6 100644
--- a/docs/admins/ci-runners.md
+++ b/docs/admins/ci-runners.md
@@ -89,7 +89,7 @@
89 89 | `ci_runner_registration_token` | Reusable registration tokens: `token_hash`, `created_by_id`, `created_at`, `last_used`. |
90 90 | `ci_runner` | Registered runners: `uuid` (the `x-runner-uuid` value), `token_hash`, `name`, `labels` (comma-joined), `version`, `status` (`IDLE`/`ACTIVE`/`OFFLINE`/`UNSPECIFIED`), `ephemeral`, `last_seen`, `created_at`. |
91 91 | `action_run` | One workflow run per repository: `number` (per-repo sequential), `workflow_name`, `workflow_file`, `event`, `ref`, `commit_sha`, `triggered_by_id`, `status` (`PENDING`/`RUNNING`/`SUCCESS`/`FAILURE`/`CANCELLED`), timestamps. Deleted with its repository. |
92 -| `action_task` | One job within a run: `seq` (surrogate int64 id handed to runners), `run_id`, `name`, `runs_on` (comma-joined labels for runner matching, empty = any), `needs` (comma-joined dependency job names), `payload`, `runner_id` (the claiming runner, null while pending), `status`, `log_length` (durable log-row count = UpdateLog resume offset), `deadline` (zombie timeout), timestamps. Deleted with its run. |
92 +| `action_task` | One job within a run: `seq` (surrogate int64 id handed to runners), `run_id`, `name`, `runs_on` (comma-joined labels for runner matching, empty = any), `needs` (comma-joined dependency job names), `outputs` (JSON of the job's reported outputs), `payload`, `runner_id` (the claiming runner, null while pending), `status`, `log_length` (durable log-row count = UpdateLog resume offset), `deadline` (zombie timeout), timestamps. Deleted with its run. |
93 93 | `action_log` | One log row of a task: `task_id`, `line_index` (0-based), `content`, `timestamp`. Deleted with its task. |
94 94 | `action_secret` | Per-repo CI secret: `repository_id`, `name`, `value_encrypted` (SecretCrypto envelope), decrypted only when delivered to a runner. Deleted with its repository. |
95 95 | `action_variable` | Per-repo CI variable: `repository_id`, `name`, `value` (plaintext config). Deleted with its repository. |
@@ -97,7 +97,8 @@
97 97 `ci_runner*` are introduced by migration `V19__ci_runners.sql`; `action_*` by `V23__action_runs.sql`
98 98 (`V24__action_task_seq.sql` adds `action_task.seq`, `V25__action_task_runs_on.sql` adds
99 99 `action_task.runs_on`, `V26__action_secrets_variables.sql` adds `action_secret`/`action_variable`,
100 -`V27__action_task_needs.sql` adds `action_task.needs`).
100 +`V27__action_task_needs.sql` adds `action_task.needs`, `V28__action_task_outputs.sql` adds
101 +`action_task.outputs`).
101 102 Secrets are stored encrypted and require `GITSHARK_SECRET_KEY` to be set (same key as push mirrors);
102 103 without it, secrets cannot be decrypted and are omitted from what a runner receives.
103 104 The `ci_runner*` tables hold no repository data (losing them only forces re-registration); the
MODIFY docs/maintainers/ci-runners.md +10 -6
diff --git a/docs/maintainers/ci-runners.md b/docs/maintainers/ci-runners.md
index 2096d4d..09da34c 100644
--- a/docs/maintainers/ci-runners.md
+++ b/docs/maintainers/ci-runners.md
@@ -18,7 +18,7 @@
18 18 | Actions UI | `web/ActionResource.java` + `templates/ActionResource/` | Read-only per-repo run list + run detail (jobs and their log rows); sidebar `Actions` tab. |
19 19 | Secrets/variables UI | `web/ActionSettingsResource.java` + `ci/ActionSecretService.java` + `templates/ActionSettingsResource/` | Owner-only CRUD for CI secrets (write-only, encrypted) and variables at `settings/actions`. |
20 20 | Entities | `model/CiRunner.java`, `model/CiRunnerRegistrationToken.java` | Runner state (migration `V19`). |
21 -| Run entities | `model/ActionRun.java`, `model/ActionTask.java`, `model/ActionLog.java` | Run/job/log-row persistence (migrations `V23`–`V27`). `ActionTask.seq` (`bigserial`) is the surrogate int64 `Task.id`; `runs_on` holds the job's labels for matching; `needs` holds its job dependencies. |
21 +| Run entities | `model/ActionRun.java`, `model/ActionTask.java`, `model/ActionLog.java` | Run/job/log-row persistence (migrations `V23`–`V28`). `ActionTask.seq` (`bigserial`) is the surrogate int64 `Task.id`; `runs_on` = matching labels; `needs` = job dependencies; `outputs` = reported job outputs (JSON). |
22 22 | Secret/variable entities | `model/ActionSecret.java`, `model/ActionVariable.java` | Per-repo CI secrets (encrypted) and variables (migration `V26`), delivered to runners in FetchTask. |
23 23 | Workflow ingest | `ci/WorkflowIngestService.java`, `ci/WorkflowRunFactory.java` | Post-receive hook: parse `.forgejo`/`.gitea` workflows at the pushed head, evaluate `on: push`, persist a run + its PENDING tasks (drained by FetchTask). |
24 24 | Admin UI | `ci/AdminRunnerResource.java` + `templates/AdminRunnerResource/` | Token generation, runner list, deletion. |
@@ -87,7 +87,11 @@
87 87 ingest). Dispatch will not hand out a task until every needed job in the run has succeeded; when a
88 88 needed job ends FAILURE/CANCELLED, `rollUpRun` cancels the dependents (to a fixpoint, so the
89 89 cancellation cascades) and the run reaches a terminal state instead of hanging. A dispatched task
90 - carries its needs' results in `Task.needs` (result only — `needs.*.outputs` are not passed yet).
90 + carries its needs' results **and outputs** in `Task.needs`.
91 +- **Job outputs:** a job's outputs (`UpdateTaskRequest.outputs`, sent incrementally by the runner) are
92 + accumulated into `action_task.outputs` (JSON) and echoed back as `sent_outputs`; dispatch delivers a
93 + needed job's outputs to its dependents as `needs.<job>.outputs`. `ActionOutputs` (de)serializes the
94 + JSON, fail-safe to an empty map on a bad value.
91 95 - **Label matching:** a task carries its job's `runs-on` labels (`action_task.runs_on`, parsed at
92 96 ingest). Dispatch scans PENDING tasks oldest-first and claims the first whose labels are all
93 97 advertised by the fetching runner (empty `runs-on` = any runner); an incompatible task is left for a
@@ -115,7 +119,8 @@
115 119 none), `SecretsSettingsTest` (owner adds a secret stored encrypted and never shown, adds/deletes a
116 120 variable, duplicate-name rejected, stranger/anonymous get 404),
117 121 `NeedsOrderingTest` (dependent waits for its need then receives its result; a failed need cancels
118 - the dependent and ends the run).
122 + the dependent and ends the run), `NeedsOutputsTest` (dependent receives an upstream job's outputs;
123 + outputs accumulate across incremental UpdateTask calls).
119 124 - **Zombie reclaim (`ZombieReclaimService`):** a scheduled sweep
120 125 (`gitshark.ci.zombie-reclaim-interval`, default 1m) fails any RUNNING task whose
121 126 `action_task.deadline` has passed — the runner is presumed gone — rolls its run up, and flags the
@@ -142,9 +147,8 @@
142 147 isolated/expanded into its own payload.
143 148 - **Non-push events:** only `push` is evaluated; `pull_request`, scheduled and manual triggers are
144 149 not. (`!`-negation within a single pattern list is also not supported.)
145 -- **`needs` outputs & `matrix`:** `needs` ordering works, but a job's `outputs` are not captured from
146 - UpdateTask or passed to dependents (`Task.needs[*].outputs` is empty); `matrix` expansion is not
147 - implemented.
150 +- **`matrix`:** expansion is not implemented — a job with `strategy.matrix` runs once, not once per
151 + cell (needs a per-job/per-cell payload expander).
148 152 - **Later phases:** concurrency/cancellation, artifacts (`ACTIONS_RESULTS_URL`), repo/org-scoped and
149 153 ephemeral runners, commit/MR status, non-push events.
150 154
MODIFY docs/users/ci-runners.md +5 -4
diff --git a/docs/users/ci-runners.md b/docs/users/ci-runners.md
index b473368..451bb64 100644
--- a/docs/users/ci-runners.md
+++ b/docs/users/ci-runners.md
@@ -8,7 +8,7 @@
8 8 > push that adds a workflow to `.forgejo/workflows/` (or `.gitea/workflows/`) starts a run that a
9 9 > connected runner picks up and executes, with logs and results shown on the repository's **Actions**
10 10 > tab. Runs are triggered by `push` (with branch/tag/path filters) and jobs can be ordered with
11 -> `needs`; other events, `needs` outputs and `matrix` arrive in later phases.
11 +> `needs` (results and outputs included); other events and `matrix` arrive in later phases.
12 12
13 13 ## Running a workflow
14 14
@@ -82,11 +82,12 @@
82 82 steps: [{ run: make deploy }]
83 83 ```
84 84
85 -The result of each needed job is available to the runner. Passing a job's `outputs` to its dependents
86 -(`needs.build.outputs.*`) and `matrix` expansion are not implemented yet.
85 +The result **and outputs** of each needed job are available to dependents — set outputs in the
86 +upstream job and read them with `${{ needs.build.outputs.* }}`. `matrix` expansion is not implemented
87 +yet.
87 88
88 89 ## What's coming
89 90
90 -- Non-push events (`pull_request`, scheduled, manual), `matrix`, `needs` outputs.
91 +- Non-push events (`pull_request`, scheduled, manual), `matrix`.
91 92 - Run cancellation / re-run.
92 93 - Artifacts and commit/merge-request status integration.
ADD src/main/java/de/workaround/ci/ActionOutputs.java +53 -0
diff --git a/src/main/java/de/workaround/ci/ActionOutputs.java b/src/main/java/de/workaround/ci/ActionOutputs.java
new file mode 100644
index 0000000..3d6dd41
--- /dev/null
+++ b/src/main/java/de/workaround/ci/ActionOutputs.java
@@ -0,0 +1,53 @@
1 +package de.workaround.ci;
2 +
3 +import java.util.LinkedHashMap;
4 +import java.util.Map;
5 +
6 +import com.fasterxml.jackson.core.type.TypeReference;
7 +import com.fasterxml.jackson.databind.ObjectMapper;
8 +
9 +/**
10 + * (De)serializes a job's outputs to/from the JSON stored in {@code action_task.outputs}. A shared,
11 + * thread-safe {@link ObjectMapper}; a malformed or empty value reads back as an empty map so a bad row
12 + * never breaks dispatch.
13 + */
14 +final class ActionOutputs
15 +{
16 + private static final ObjectMapper MAPPER = new ObjectMapper();
17 +
18 + private static final TypeReference<LinkedHashMap<String, String>> MAP_TYPE = new TypeReference<>()
19 + {
20 + };
21 +
22 + private ActionOutputs()
23 + {
24 + }
25 +
26 + static Map<String, String> parse(String json)
27 + {
28 + if (json == null || json.isBlank())
29 + {
30 + return new LinkedHashMap<>();
31 + }
32 + try
33 + {
34 + return MAPPER.readValue(json, MAP_TYPE);
35 + }
36 + catch (Exception malformed)
37 + {
38 + return new LinkedHashMap<>();
39 + }
40 + }
41 +
42 + static String write(Map<String, String> outputs)
43 + {
44 + try
45 + {
46 + return MAPPER.writeValueAsString(outputs);
47 + }
48 + catch (Exception e)
49 + {
50 + throw new IllegalStateException("Could not serialize task outputs", e);
51 + }
52 + }
53 +}
MODIFY src/main/java/de/workaround/ci/ConnectRunnerResource.java +11 -5
diff --git a/src/main/java/de/workaround/ci/ConnectRunnerResource.java b/src/main/java/de/workaround/ci/ConnectRunnerResource.java
index 7a95909..ed6aef4 100644
--- a/src/main/java/de/workaround/ci/ConnectRunnerResource.java
+++ b/src/main/java/de/workaround/ci/ConnectRunnerResource.java
@@ -145,8 +145,12 @@
145 145 : null;
146 146 try
147 147 {
148 - progressService.updateTask(uuid, token, state.getId(), state.getResult(), stoppedAt);
149 - return ok(UpdateTaskResponse.newBuilder().setState(state).build().toByteArray());
148 + ActionTask task = progressService.updateTask(uuid, token, state.getId(), state.getResult(), stoppedAt,
149 + request.getOutputsMap());
150 + return ok(UpdateTaskResponse.newBuilder()
151 + .setState(state)
152 + .addAllSentOutputs(ActionOutputs.parse(task.outputs).keySet())
153 + .build().toByteArray());
150 154 }
151 155 catch (RunnerAuthenticationException e)
152 156 {
@@ -181,7 +185,7 @@
181 185 }
182 186
183 187 private static Task toProto(ActionTask task, Map<String, String> secrets, Map<String, String> vars,
184 - Map<String, ActionRun.Status> needs)
188 + Map<String, TaskDispatchService.NeedInfo> needs)
185 189 {
186 190 Task.Builder builder = Task.newBuilder().setId(task.seq);
187 191 if (task.payload != null)
@@ -191,8 +195,10 @@
191 195 builder.setContext(githubContext(task));
192 196 builder.putAllSecrets(secrets);
193 197 builder.putAllVars(vars);
194 - needs.forEach((job, status) -> builder.putNeeds(job,
195 - TaskNeed.newBuilder().setResult(toResult(status)).build()));
198 + needs.forEach((job, info) -> builder.putNeeds(job, TaskNeed.newBuilder()
199 + .setResult(toResult(info.result()))
200 + .putAllOutputs(info.outputs())
201 + .build()));
196 202 return builder.build();
197 203 }
198 204
MODIFY src/main/java/de/workaround/ci/TaskDispatchService.java +23 -13
diff --git a/src/main/java/de/workaround/ci/TaskDispatchService.java b/src/main/java/de/workaround/ci/TaskDispatchService.java
index 24b8d1f..bd96f98 100644
--- a/src/main/java/de/workaround/ci/TaskDispatchService.java
+++ b/src/main/java/de/workaround/ci/TaskDispatchService.java
@@ -65,7 +65,12 @@
65 65 EntityManager em;
66 66
67 67 public record Fetched(Optional<ActionTask> task, long tasksVersion, Map<String, String> secrets,
68 - Map<String, String> vars, Map<String, ActionRun.Status> needs)
68 + Map<String, String> vars, Map<String, NeedInfo> needs)
69 + {
70 + }
71 +
72 + /** A needed job's result and outputs, for the runner's {@code needs} context. */
73 + public record NeedInfo(ActionRun.Status result, Map<String, String> outputs)
69 74 {
70 75 }
71 76
@@ -89,7 +94,7 @@
89 94 });
90 95 Map<String, String> secretMap = next.map(task -> secretsFor(task.run.repository)).orElse(Map.of());
91 96 Map<String, String> varMap = next.map(task -> variablesFor(task.run.repository)).orElse(Map.of());
92 - Map<String, ActionRun.Status> needsMap = next.map(this::needsResults).orElse(Map.of());
97 + Map<String, NeedInfo> needsMap = next.map(this::needsResults).orElse(Map.of());
93 98 return new Fetched(next, tasks.maxSeq(), secretMap, varMap, needsMap);
94 99 }
95 100
@@ -189,35 +194,40 @@
189 194 {
190 195 return true;
191 196 }
192 - Map<String, ActionRun.Status> siblings = statusByJob(task.run);
193 - return needed.stream().allMatch(name -> siblings.get(name) == ActionRun.Status.SUCCESS);
197 + Map<String, ActionTask> siblings = taskByJob(task.run);
198 + return needed.stream().allMatch(name ->
199 + {
200 + ActionTask dep = siblings.get(name);
201 + return dep != null && dep.status == ActionRun.Status.SUCCESS;
202 + });
194 203 }
195 204
196 - private Map<String, ActionRun.Status> statusByJob(ActionRun run)
205 + private Map<String, ActionTask> taskByJob(ActionRun run)
197 206 {
198 - Map<String, ActionRun.Status> byJob = new HashMap<>();
207 + Map<String, ActionTask> byJob = new HashMap<>();
199 208 for (ActionTask sibling : tasks.findByRun(run))
200 209 {
201 - byJob.put(sibling.name, sibling.status);
210 + byJob.put(sibling.name, sibling);
202 211 }
203 212 return byJob;
204 213 }
205 214
206 - /** The results of the jobs a task needs, for the runner's {@code needs} context. */
207 - private Map<String, ActionRun.Status> needsResults(ActionTask task)
215 + /** The result and outputs of the jobs a task needs, for the runner's {@code needs} context. */
216 + private Map<String, NeedInfo> needsResults(ActionTask task)
208 217 {
209 218 Set<String> needed = splitLabels(task.needs);
210 219 if (needed.isEmpty())
211 220 {
212 221 return Map.of();
213 222 }
214 - Map<String, ActionRun.Status> siblings = statusByJob(task.run);
215 - Map<String, ActionRun.Status> results = new HashMap<>();
223 + Map<String, ActionTask> siblings = taskByJob(task.run);
224 + Map<String, NeedInfo> results = new HashMap<>();
216 225 for (String name : needed)
217 226 {
218 - if (siblings.containsKey(name))
227 + ActionTask dep = siblings.get(name);
228 + if (dep != null)
219 229 {
220 - results.put(name, siblings.get(name));
230 + results.put(name, new NeedInfo(dep.status, ActionOutputs.parse(dep.outputs)));
221 231 }
222 232 }
223 233 return results;
MODIFY src/main/java/de/workaround/ci/TaskProgressService.java +9 -1
diff --git a/src/main/java/de/workaround/ci/TaskProgressService.java b/src/main/java/de/workaround/ci/TaskProgressService.java
index 3904fe8..4c24e28 100644
--- a/src/main/java/de/workaround/ci/TaskProgressService.java
+++ b/src/main/java/de/workaround/ci/TaskProgressService.java
@@ -43,7 +43,8 @@
43 43 * @throws TaskNotFoundException if no task has the given surrogate id
44 44 */
45 45 @Transactional
46 - public ActionTask updateTask(String uuid, String token, long taskSeq, Result result, Instant stoppedAt)
46 + public ActionTask updateTask(String uuid, String token, long taskSeq, Result result, Instant stoppedAt,
47 + Map<String, String> outputs)
47 48 {
48 49 CiRunner runner = authenticate(uuid, token);
49 50 ActionTask task = ownedTask(taskSeq, runner);
@@ -53,6 +54,13 @@
53 54 // Already settled (e.g. reclaimed as a zombie); a late runner update must not resurrect it.
54 55 return task;
55 56 }
57 + if (outputs != null && !outputs.isEmpty())
58 + {
59 + // runners send outputs incrementally (only the unsent ones); accumulate them
60 + Map<String, String> merged = ActionOutputs.parse(task.outputs);
61 + merged.putAll(outputs);
62 + task.outputs = ActionOutputs.write(merged);
63 + }
56 64 ActionRun.Status status = map(result);
57 65 task.status = status;
58 66 if (status.isTerminal())
MODIFY src/main/java/de/workaround/model/ActionTask.java +3 -0
diff --git a/src/main/java/de/workaround/model/ActionTask.java b/src/main/java/de/workaround/model/ActionTask.java
index 8ca92da..4e536fc 100644
--- a/src/main/java/de/workaround/model/ActionTask.java
+++ b/src/main/java/de/workaround/model/ActionTask.java
@@ -57,6 +57,9 @@
57 57 /** Names of the jobs this task depends on (comma-joined); empty means none. */
58 58 public String needs = "";
59 59
60 + /** Outputs this job reported (JSON object), accumulated across UpdateTask calls; delivered to dependents. */
61 + public String outputs = "{}";
62 +
60 63 /** The expanded single-job workflow payload delivered to the runner in FetchTask; null until materialized. */
61 64 public String payload;
62 65
ADD src/main/resources/db/migration/V28__action_task_outputs.sql +7 -0
diff --git a/src/main/resources/db/migration/V28__action_task_outputs.sql b/src/main/resources/db/migration/V28__action_task_outputs.sql
new file mode 100644
index 0000000..8f0bbf9
--- /dev/null
+++ b/src/main/resources/db/migration/V28__action_task_outputs.sql
@@ -0,0 +1,7 @@
1 +-- Job outputs for the `needs` context (issue #2, phase 2).
2 +--
3 +-- A JSON object of the outputs a job reported via UpdateTask (accumulated across updates), delivered
4 +-- to dependent jobs as `needs.<job>.outputs`.
5 +
6 +alter table action_task
7 + add column outputs text not null default '{}';
MODIFY src/test/java/de/workaround/ci/NeedsOrderingTest.java +3 -3
diff --git a/src/test/java/de/workaround/ci/NeedsOrderingTest.java b/src/test/java/de/workaround/ci/NeedsOrderingTest.java
index 5a4f5a9..1a163c8 100644
--- a/src/test/java/de/workaround/ci/NeedsOrderingTest.java
+++ b/src/test/java/de/workaround/ci/NeedsOrderingTest.java
@@ -60,11 +60,11 @@
60 60 "deploy stays blocked while build is running");
61 61
62 62 progress.updateTask(reg.runner().uuid, reg.plaintext(), buildSeq,
63 - de.workaround.ci.proto.runner.v1.Result.RESULT_SUCCESS, null);
63 + de.workaround.ci.proto.runner.v1.Result.RESULT_SUCCESS, null, java.util.Map.of());
64 64
65 65 TaskDispatchService.Fetched second = dispatch.fetch(reg.runner().uuid, reg.plaintext());
66 66 assertEquals("deploy", second.task().orElseThrow().name);
67 - assertEquals(ActionRun.Status.SUCCESS, second.needs().get("build"), "needs result delivered");
67 + assertEquals(ActionRun.Status.SUCCESS, second.needs().get("build").result(), "needs result delivered");
68 68
69 69 assertTrue(runIsRunning(runId));
70 70 }
@@ -77,7 +77,7 @@
77 77
78 78 ActionTask build = dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().orElseThrow();
79 79 progress.updateTask(reg.runner().uuid, reg.plaintext(), build.seq,
80 - de.workaround.ci.proto.runner.v1.Result.RESULT_FAILURE, null);
80 + de.workaround.ci.proto.runner.v1.Result.RESULT_FAILURE, null, java.util.Map.of());
81 81
82 82 assertEquals(ActionRun.Status.CANCELLED, deployStatus(runId), "dependent of a failed job is cancelled");
83 83 assertEquals(ActionRun.Status.FAILURE, runStatus(runId), "run finishes rather than hanging");
ADD src/test/java/de/workaround/ci/NeedsOutputsTest.java +137 -0
diff --git a/src/test/java/de/workaround/ci/NeedsOutputsTest.java b/src/test/java/de/workaround/ci/NeedsOutputsTest.java
new file mode 100644
index 0000000..b9fe4a4
--- /dev/null
+++ b/src/test/java/de/workaround/ci/NeedsOutputsTest.java
@@ -0,0 +1,137 @@
1 +package de.workaround.ci;
2 +
3 +import java.time.Instant;
4 +import java.util.List;
5 +import java.util.Map;
6 +import java.util.UUID;
7 +
8 +import org.junit.jupiter.api.Test;
9 +
10 +import de.workaround.ci.proto.runner.v1.Result;
11 +import de.workaround.git.GitRepositoryService;
12 +import de.workaround.model.ActionRun;
13 +import de.workaround.model.ActionTask;
14 +import de.workaround.model.Repository;
15 +import de.workaround.model.User;
16 +import io.quarkus.test.junit.QuarkusTest;
17 +import jakarta.inject.Inject;
18 +import jakarta.transaction.Transactional;
19 +
20 +import static org.junit.jupiter.api.Assertions.assertEquals;
21 +
22 +/**
23 + * Job outputs flowing through {@code needs} (issue #2, phase 2): a job reports outputs via UpdateTask;
24 + * a dependent job then receives them in its needs context when it is dispatched.
25 + */
26 +@QuarkusTest
27 +class NeedsOutputsTest
28 +{
29 + @Inject
30 + RunnerRegistrationService runnerService;
31 +
32 + @Inject
33 + TaskDispatchService dispatch;
34 +
35 + @Inject
36 + TaskProgressService progress;
37 +
38 + @Inject
39 + GitRepositoryService repositories;
40 +
41 + @Inject
42 + ActionRun.Repo runs;
43 +
44 + @Inject
45 + ActionTask.Repo tasks;
46 +
47 + @Test
48 + void dependentReceivesUpstreamJobOutputs()
49 + {
50 + RunnerRegistrationService.RegisteredRunner reg = registerRunner();
51 + seed("nof-a");
52 +
53 + ActionTask build = dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().orElseThrow();
54 + assertEquals("build", build.name);
55 +
56 + progress.updateTask(reg.runner().uuid, reg.plaintext(), build.seq, Result.RESULT_SUCCESS, null,
57 + Map.of("image", "app:1.2.3", "digest", "sha256:abc"));
58 +
59 + TaskDispatchService.Fetched deploy = dispatch.fetch(reg.runner().uuid, reg.plaintext());
60 + assertEquals("deploy", deploy.task().orElseThrow().name);
61 + TaskDispatchService.NeedInfo buildNeed = deploy.needs().get("build");
62 + assertEquals(ActionRun.Status.SUCCESS, buildNeed.result());
63 + assertEquals("app:1.2.3", buildNeed.outputs().get("image"));
64 + assertEquals("sha256:abc", buildNeed.outputs().get("digest"));
65 + }
66 +
67 + @Test
68 + void outputsAccumulateAcrossUpdates()
69 + {
70 + RunnerRegistrationService.RegisteredRunner reg = registerRunner();
71 + seed("nof-b");
72 +
73 + ActionTask build = dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().orElseThrow();
74 + // runner sends outputs incrementally, only the unsent ones each time
75 + progress.updateTask(reg.runner().uuid, reg.plaintext(), build.seq, Result.RESULT_UNSPECIFIED, null,
76 + Map.of("image", "app:1.2.3"));
77 + progress.updateTask(reg.runner().uuid, reg.plaintext(), build.seq, Result.RESULT_SUCCESS, null,
78 + Map.of("digest", "sha256:abc"));
79 +
80 + TaskDispatchService.NeedInfo buildNeed =
81 + dispatch.fetch(reg.runner().uuid, reg.plaintext()).needs().get("build");
82 + assertEquals("app:1.2.3", buildNeed.outputs().get("image"), "earlier output retained");
83 + assertEquals("sha256:abc", buildNeed.outputs().get("digest"), "later output merged");
84 + }
85 +
86 + private RunnerRegistrationService.RegisteredRunner registerRunner()
87 + {
88 + String token = runnerService.createRegistrationToken(persistUser("nof-admin-" + shortId())).plaintext();
89 + return runnerService.register(token, "nof-runner", List.of(), "v4.0.0", false);
90 + }
91 +
92 + @Transactional
93 + void seed(String repoName)
94 + {
95 + User owner = persistUser(repoName + "-" + shortId());
96 + Repository repo = repositories.create(owner, repoName, Repository.Visibility.PUBLIC, null);
97 +
98 + ActionRun run = new ActionRun();
99 + run.repository = repo;
100 + run.number = runs.maxNumber(repo) + 1;
101 + run.workflowName = "CI";
102 + run.workflowFile = ".forgejo/workflows/ci.yml";
103 + run.event = "push";
104 + run.ref = "refs/heads/main";
105 + run.commitSha = "0000000000000000000000000000000000000000";
106 + run.persist();
107 +
108 + newTask(run, "build", "", 10);
109 + newTask(run, "deploy", "build", 5);
110 + }
111 +
112 + private void newTask(ActionRun run, String name, String needs, int secondsAgo)
113 + {
114 + ActionTask task = new ActionTask();
115 + task.run = run;
116 + task.name = name;
117 + task.needs = needs;
118 + task.payload = "on: push";
119 + task.createdAt = Instant.now().minusSeconds(secondsAgo);
120 + task.persist();
121 + }
122 +
123 + @Transactional
124 + User persistUser(String name)
125 + {
126 + User user = new User();
127 + user.oidcSub = name;
128 + user.username = name;
129 + user.persist();
130 + return user;
131 + }
132 +
133 + private static String shortId()
134 + {
135 + return UUID.randomUUID().toString().substring(0, 8);
136 + }
137 +}
MODIFY src/test/java/de/workaround/ci/ZombieReclaimTest.java +1 -1
diff --git a/src/test/java/de/workaround/ci/ZombieReclaimTest.java b/src/test/java/de/workaround/ci/ZombieReclaimTest.java
index cf16e2c..0e334a5 100644
--- a/src/test/java/de/workaround/ci/ZombieReclaimTest.java
+++ b/src/test/java/de/workaround/ci/ZombieReclaimTest.java
@@ -83,7 +83,7 @@
83 83 long seq = tasks.findById(id).seq;
84 84
85 85 // The presumed-dead runner reconnects and reports success — the task must stay FAILURE.
86 - progress.updateTask(reg.runner().uuid, reg.plaintext(), seq, Result.RESULT_SUCCESS, null);
86 + progress.updateTask(reg.runner().uuid, reg.plaintext(), seq, Result.RESULT_SUCCESS, null, java.util.Map.of());
87 87
88 88 ActionTask task = tasks.findById(id);
89 89 assertEquals(ActionRun.Status.FAILURE, task.status);

Keyboard shortcuts

?Show this help
g hGo home
EscClose dialog