✨ (ci): Order jobs by their needs dependencies
Changes
13 files changed, +369 -33
MODIFY
README.md
+3 -2
@@ -89,8 +89,9 @@
89
89
to `.forgejo/workflows/` (`on: push`, with `branches`/`tags`/`paths` glob filters) creates a run,
90
90
which a runner claims, executes, and streams logs for — visible on the repository's **Actions** tab;
91
91
a vanished runner's task is reclaimed after a timeout. Jobs are matched to runners by `runs-on`
92
- labels, and repository owners manage encrypted secrets and variables that are delivered to runners.
93
- Non-push events, `needs`/`matrix`, and artifacts are follow-up phases. Guides: [for users](docs/users/ci-runners.md), [for admins](docs/admins/ci-runners.md),
92
+ labels, ordered by `needs` dependencies, and repository owners manage encrypted secrets and
93
+ variables that are delivered to runners. Non-push events, `needs` outputs, `matrix`, and artifacts
94
+ are follow-up phases. Guides: [for users](docs/users/ci-runners.md), [for admins](docs/admins/ci-runners.md),
94
95
[architecture](docs/maintainers/ci-runners.md)
95
96
activities from; local users can in turn follow a remote repository — or a whole remote user, whose
96
97
public repositories are then followed and shown grouped — and read their pushes (see below)
MODIFY
docs/admins/ci-runners.md
+3 -2
@@ -89,14 +89,15 @@
89
89
| `ci_runner_registration_token` | Reusable registration tokens: `token_hash`, `created_by_id`, `created_at`, `last_used`. |
90
90
| `ci_runner` | Registered runners: `uuid` (the `x-runner-uuid` value), `token_hash`, `name`, `labels` (comma-joined), `version`, `status` (`IDLE`/`ACTIVE`/`OFFLINE`/`UNSPECIFIED`), `ephemeral`, `last_seen`, `created_at`. |
91
91
| `action_run` | One workflow run per repository: `number` (per-repo sequential), `workflow_name`, `workflow_file`, `event`, `ref`, `commit_sha`, `triggered_by_id`, `status` (`PENDING`/`RUNNING`/`SUCCESS`/`FAILURE`/`CANCELLED`), timestamps. Deleted with its repository. |
92
-| `action_task` | One job within a run: `seq` (surrogate int64 id handed to runners), `run_id`, `name`, `runs_on` (comma-joined labels for runner matching, empty = any), `payload`, `runner_id` (the claiming runner, null while pending), `status`, `log_length` (durable log-row count = UpdateLog resume offset), `deadline` (zombie timeout), timestamps. Deleted with its run. |
92
+| `action_task` | One job within a run: `seq` (surrogate int64 id handed to runners), `run_id`, `name`, `runs_on` (comma-joined labels for runner matching, empty = any), `needs` (comma-joined dependency job names), `payload`, `runner_id` (the claiming runner, null while pending), `status`, `log_length` (durable log-row count = UpdateLog resume offset), `deadline` (zombie timeout), timestamps. Deleted with its run. |
93
93
| `action_log` | One log row of a task: `task_id`, `line_index` (0-based), `content`, `timestamp`. Deleted with its task. |
94
94
| `action_secret` | Per-repo CI secret: `repository_id`, `name`, `value_encrypted` (SecretCrypto envelope), decrypted only when delivered to a runner. Deleted with its repository. |
95
95
| `action_variable` | Per-repo CI variable: `repository_id`, `name`, `value` (plaintext config). Deleted with its repository. |
96
96
97
97
`ci_runner*` are introduced by migration `V19__ci_runners.sql`; `action_*` by `V23__action_runs.sql`
98
98
(`V24__action_task_seq.sql` adds `action_task.seq`, `V25__action_task_runs_on.sql` adds
99
-`action_task.runs_on`, `V26__action_secrets_variables.sql` adds `action_secret`/`action_variable`).
99
+`action_task.runs_on`, `V26__action_secrets_variables.sql` adds `action_secret`/`action_variable`,
100
+`V27__action_task_needs.sql` adds `action_task.needs`).
100
101
Secrets are stored encrypted and require `GITSHARK_SECRET_KEY` to be set (same key as push mirrors);
101
102
without it, secrets cannot be decrypted and are omitted from what a runner receives.
102
103
The `ci_runner*` tables hold no repository data (losing them only forces re-registration); the
MODIFY
docs/maintainers/ci-runners.md
+15 -5
@@ -18,7 +18,7 @@
18
18
| Actions UI | `web/ActionResource.java` + `templates/ActionResource/` | Read-only per-repo run list + run detail (jobs and their log rows); sidebar `Actions` tab. |
19
19
| Secrets/variables UI | `web/ActionSettingsResource.java` + `ci/ActionSecretService.java` + `templates/ActionSettingsResource/` | Owner-only CRUD for CI secrets (write-only, encrypted) and variables at `settings/actions`. |
20
20
| Entities | `model/CiRunner.java`, `model/CiRunnerRegistrationToken.java` | Runner state (migration `V19`). |
21
-| Run entities | `model/ActionRun.java`, `model/ActionTask.java`, `model/ActionLog.java` | Run/job/log-row persistence (migrations `V23`–`V25`). `ActionTask.seq` (`bigserial`) is the surrogate int64 `Task.id` for the wire; `ActionTask.runs_on` holds the job's labels for matching. |
21
+| Run entities | `model/ActionRun.java`, `model/ActionTask.java`, `model/ActionLog.java` | Run/job/log-row persistence (migrations `V23`–`V27`). `ActionTask.seq` (`bigserial`) is the surrogate int64 `Task.id`; `runs_on` holds the job's labels for matching; `needs` holds its job dependencies. |
22
22
| Secret/variable entities | `model/ActionSecret.java`, `model/ActionVariable.java` | Per-repo CI secrets (encrypted) and variables (migration `V26`), delivered to runners in FetchTask. |
23
23
| Workflow ingest | `ci/WorkflowIngestService.java`, `ci/WorkflowRunFactory.java` | Post-receive hook: parse `.forgejo`/`.gitea` workflows at the pushed head, evaluate `on: push`, persist a run + its PENDING tasks (drained by FetchTask). |
24
24
| Admin UI | `ci/AdminRunnerResource.java` + `templates/AdminRunnerResource/` | Token generation, runner list, deletion. |
@@ -83,6 +83,11 @@
83
83
that fails to decrypt is dropped, never sent as ciphertext) in the FetchTask `Task.secrets`/`vars`
84
84
maps. Same trust model as GitHub self-hosted runners: secrets go to whatever runner claims the task
85
85
(over TLS). No repo/org scoping of secrets and no fork-PR guard yet (no PR triggers exist).
86
+- **`needs` ordering:** each task records the jobs it depends on (`action_task.needs`, parsed at
87
+ ingest). Dispatch will not hand out a task until every needed job in the run has succeeded; when a
88
+ needed job ends FAILURE/CANCELLED, `rollUpRun` cancels the dependents (to a fixpoint, so the
89
+ cancellation cascades) and the run reaches a terminal state instead of hanging. A dispatched task
90
+ carries its needs' results in `Task.needs` (result only — `needs.*.outputs` are not passed yet).
86
91
- **Label matching:** a task carries its job's `runs-on` labels (`action_task.runs_on`, parsed at
87
92
ingest). Dispatch scans PENDING tasks oldest-first and claims the first whose labels are all
88
93
advertised by the fetching runner (empty `runs-on` = any runner); an incompatible task is left for a
@@ -108,7 +113,9 @@
108
113
nothing when none match, unconstrained task runs anywhere),
109
114
`SecretDeliveryTest` (claimed task receives repo secrets decrypted + variables; empty fetch carries
110
115
none), `SecretsSettingsTest` (owner adds a secret stored encrypted and never shown, adds/deletes a
111
- variable, duplicate-name rejected, stranger/anonymous get 404).
116
+ variable, duplicate-name rejected, stranger/anonymous get 404),
117
+ `NeedsOrderingTest` (dependent waits for its need then receives its result; a failed need cancels
118
+ the dependent and ends the run).
112
119
- **Zombie reclaim (`ZombieReclaimService`):** a scheduled sweep
113
120
(`gitshark.ci.zombie-reclaim-interval`, default 1m) fails any RUNNING task whose
114
121
`action_task.deadline` has passed — the runner is presumed gone — rolls its run up, and flags the
@@ -132,11 +139,14 @@
132
139
runner may under-poll. Add server-side long-poll and a state-driven version counter.
133
140
- **Per-job payload expansion:** `workflow_payload` is the raw workflow YAML (fine while a workflow
134
141
has a single job, which the `github.job` context selects); a multi-job workflow needs each job
135
- isolated/expanded into its own payload. No `needs`/`matrix` yet.
142
+ isolated/expanded into its own payload.
136
143
- **Non-push events:** only `push` is evaluated; `pull_request`, scheduled and manual triggers are
137
144
not. (`!`-negation within a single pattern list is also not supported.)
138
-- **Later phases:** `needs`/`matrix`, concurrency/cancellation, artifacts (`ACTIONS_RESULTS_URL`),
139
- repo/org-scoped and ephemeral runners, commit/MR status, non-push events.
145
+- **`needs` outputs & `matrix`:** `needs` ordering works, but a job's `outputs` are not captured from
146
+ UpdateTask or passed to dependents (`Task.needs[*].outputs` is empty); `matrix` expansion is not
147
+ implemented.
148
+- **Later phases:** concurrency/cancellation, artifacts (`ACTIONS_RESULTS_URL`), repo/org-scoped and
149
+ ephemeral runners, commit/MR status, non-push events.
140
150
141
151
## References
142
152
MODIFY
docs/users/ci-runners.md
+22 -3
@@ -7,8 +7,8 @@
7
7
> **Available today:** an **instance administrator** registers runners against this instance, and a
8
8
> push that adds a workflow to `.forgejo/workflows/` (or `.gitea/workflows/`) starts a run that a
9
9
> connected runner picks up and executes, with logs and results shown on the repository's **Actions**
10
-> tab. Runs are triggered by `push` (with branch/tag/path filters); other events and `needs`/`matrix`
11
-> arrive in later phases.
10
+> tab. Runs are triggered by `push` (with branch/tag/path filters) and jobs can be ordered with
11
+> `needs`; other events, `needs` outputs and `matrix` arrive in later phases.
12
12
13
13
## Running a workflow
14
14
@@ -66,8 +66,27 @@
66
66
an encryption key configured.
67
67
- **Variables** are plain configuration and their values are visible on the settings page.
68
68
69
+## Job ordering with `needs`
70
+
71
+A job can depend on others with `needs`. A dependent job runs only after every job it needs has
72
+succeeded; if one fails, the dependent (and anything downstream of it) is cancelled.
73
+
74
+```yaml
75
+jobs:
76
+ build:
77
+ runs-on: ubuntu-latest
78
+ steps: [{ run: make }]
79
+ deploy:
80
+ needs: build
81
+ runs-on: ubuntu-latest
82
+ steps: [{ run: make deploy }]
83
+```
84
+
85
+The result of each needed job is available to the runner. Passing a job's `outputs` to its dependents
86
+(`needs.build.outputs.*`) and `matrix` expansion are not implemented yet.
87
+
69
88
## What's coming
70
89
71
-- Non-push events (`pull_request`, scheduled, manual), `needs`/`matrix`.
90
+- Non-push events (`pull_request`, scheduled, manual), `matrix`, `needs` outputs.
72
91
- Run cancellation / re-run.
73
92
- Artifacts and commit/merge-request status integration.
MODIFY
src/main/java/de/workaround/ci/ConnectRunnerResource.java
+18 -2
@@ -21,6 +21,7 @@
21
21
import de.workaround.ci.proto.runner.v1.Runner;
22
22
import de.workaround.ci.proto.runner.v1.RunnerStatus;
23
23
import de.workaround.ci.proto.runner.v1.Task;
24
+import de.workaround.ci.proto.runner.v1.TaskNeed;
24
25
import de.workaround.ci.proto.runner.v1.TaskState;
25
26
import de.workaround.ci.proto.runner.v1.UpdateLogRequest;
26
27
import de.workaround.ci.proto.runner.v1.UpdateLogResponse;
@@ -122,7 +123,8 @@
122
123
TaskDispatchService.Fetched fetched = dispatchService.fetch(uuid, token);
123
124
FetchTaskResponse.Builder response = FetchTaskResponse.newBuilder()
124
125
.setTasksVersion(fetched.tasksVersion());
125
- fetched.task().ifPresent(task -> response.setTask(toProto(task, fetched.secrets(), fetched.vars())));
126
+ fetched.task().ifPresent(task ->
127
+ response.setTask(toProto(task, fetched.secrets(), fetched.vars(), fetched.needs())));
126
128
return ok(response.build().toByteArray());
127
129
}
128
130
catch (RunnerAuthenticationException e)
@@ -178,7 +180,8 @@
178
180
}
179
181
}
180
182
181
- private static Task toProto(ActionTask task, Map<String, String> secrets, Map<String, String> vars)
183
+ private static Task toProto(ActionTask task, Map<String, String> secrets, Map<String, String> vars,
184
+ Map<String, ActionRun.Status> needs)
182
185
{
183
186
Task.Builder builder = Task.newBuilder().setId(task.seq);
184
187
if (task.payload != null)
@@ -188,9 +191,22 @@
188
191
builder.setContext(githubContext(task));
189
192
builder.putAllSecrets(secrets);
190
193
builder.putAllVars(vars);
194
+ needs.forEach((job, status) -> builder.putNeeds(job,
195
+ TaskNeed.newBuilder().setResult(toResult(status)).build()));
191
196
return builder.build();
192
197
}
193
198
199
+ private static de.workaround.ci.proto.runner.v1.Result toResult(ActionRun.Status status)
200
+ {
201
+ return switch (status)
202
+ {
203
+ case SUCCESS -> de.workaround.ci.proto.runner.v1.Result.RESULT_SUCCESS;
204
+ case FAILURE -> de.workaround.ci.proto.runner.v1.Result.RESULT_FAILURE;
205
+ case CANCELLED -> de.workaround.ci.proto.runner.v1.Result.RESULT_CANCELLED;
206
+ default -> de.workaround.ci.proto.runner.v1.Result.RESULT_UNSPECIFIED;
207
+ };
208
+ }
209
+
194
210
/**
195
211
* The GitHub-Actions {@code github.*} context the runner needs to execute the task. Without it the
196
212
* runner cannot pick the job from the workflow ({@code github.job}) and dereferences a nil context.
MODIFY
src/main/java/de/workaround/ci/TaskDispatchService.java
+49 -2
@@ -65,7 +65,7 @@
65
65
EntityManager em;
66
66
67
67
public record Fetched(Optional<ActionTask> task, long tasksVersion, Map<String, String> secrets,
68
- Map<String, String> vars)
68
+ Map<String, String> vars, Map<String, ActionRun.Status> needs)
69
69
{
70
70
}
71
71
@@ -89,7 +89,8 @@
89
89
});
90
90
Map<String, String> secretMap = next.map(task -> secretsFor(task.run.repository)).orElse(Map.of());
91
91
Map<String, String> varMap = next.map(task -> variablesFor(task.run.repository)).orElse(Map.of());
92
- return new Fetched(next, tasks.maxSeq(), secretMap, varMap);
92
+ Map<String, ActionRun.Status> needsMap = next.map(this::needsResults).orElse(Map.of());
93
+ return new Fetched(next, tasks.maxSeq(), secretMap, varMap, needsMap);
93
94
}
94
95
95
96
/** Decrypted repository secrets by name; a secret that cannot be decrypted is dropped, never leaked as ciphertext. */
@@ -163,6 +164,10 @@
163
164
continue;
164
165
}
165
166
UUID id = (UUID) candidate[0];
167
+ if (!needsSatisfied(tasks.findById(id)))
168
+ {
169
+ continue;
170
+ }
166
171
@SuppressWarnings("unchecked")
167
172
List<UUID> locked = em.createNativeQuery(
168
173
"select id from action_task where id = :id and status = 'PENDING' for update skip locked")
@@ -176,6 +181,48 @@
176
181
return Optional.empty();
177
182
}
178
183
184
+ /** A task is dispatchable only once every job it needs (in the same run) has succeeded. */
185
+ private boolean needsSatisfied(ActionTask task)
186
+ {
187
+ Set<String> needed = splitLabels(task.needs);
188
+ if (needed.isEmpty())
189
+ {
190
+ return true;
191
+ }
192
+ Map<String, ActionRun.Status> siblings = statusByJob(task.run);
193
+ return needed.stream().allMatch(name -> siblings.get(name) == ActionRun.Status.SUCCESS);
194
+ }
195
+
196
+ private Map<String, ActionRun.Status> statusByJob(ActionRun run)
197
+ {
198
+ Map<String, ActionRun.Status> byJob = new HashMap<>();
199
+ for (ActionTask sibling : tasks.findByRun(run))
200
+ {
201
+ byJob.put(sibling.name, sibling.status);
202
+ }
203
+ return byJob;
204
+ }
205
+
206
+ /** The results of the jobs a task needs, for the runner's {@code needs} context. */
207
+ private Map<String, ActionRun.Status> needsResults(ActionTask task)
208
+ {
209
+ Set<String> needed = splitLabels(task.needs);
210
+ if (needed.isEmpty())
211
+ {
212
+ return Map.of();
213
+ }
214
+ Map<String, ActionRun.Status> siblings = statusByJob(task.run);
215
+ Map<String, ActionRun.Status> results = new HashMap<>();
216
+ for (String name : needed)
217
+ {
218
+ if (siblings.containsKey(name))
219
+ {
220
+ results.put(name, siblings.get(name));
221
+ }
222
+ }
223
+ return results;
224
+ }
225
+
179
226
/** Whether the runner advertises every label the task's {@code runs-on} requires (empty = any). */
180
227
private static boolean labelsSatisfied(String runsOn, Set<String> runnerLabels)
181
228
{
MODIFY
src/main/java/de/workaround/ci/TaskProgressService.java
+51 -0
@@ -1,7 +1,10 @@
1
1
package de.workaround.ci;
2
2
3
3
import java.time.Instant;
4
+import java.util.Arrays;
5
+import java.util.HashMap;
4
6
import java.util.List;
7
+import java.util.Map;
5
8
6
9
import com.google.protobuf.Timestamp;
7
10
@@ -119,6 +122,7 @@
119
122
public void rollUpRun(ActionRun run)
120
123
{
121
124
List<ActionTask> all = tasks.findByRun(run);
125
+ cancelUnsatisfiableDependents(all);
122
126
boolean anyRunning = all.stream().anyMatch(t -> !t.status.isTerminal());
123
127
if (anyRunning)
124
128
{
@@ -133,6 +137,53 @@
133
137
run.finishedAt = Instant.now();
134
138
}
135
139
140
+ /**
141
+ * Cancel any PENDING task whose {@code needs} can no longer all succeed — a needed job finished
142
+ * FAILURE or CANCELLED. Iterates to a fixpoint so a cancellation cascades down the dependency chain,
143
+ * ensuring the run reaches a terminal state instead of hanging on unreachable tasks.
144
+ */
145
+ private static void cancelUnsatisfiableDependents(List<ActionTask> all)
146
+ {
147
+ Map<String, ActionRun.Status> byJob = new HashMap<>();
148
+ for (ActionTask task : all)
149
+ {
150
+ byJob.put(task.name, task.status);
151
+ }
152
+ boolean changed = true;
153
+ while (changed)
154
+ {
155
+ changed = false;
156
+ for (ActionTask task : all)
157
+ {
158
+ if (task.status != ActionRun.Status.PENDING)
159
+ {
160
+ continue;
161
+ }
162
+ boolean blocked = needsOf(task).stream().anyMatch(name ->
163
+ {
164
+ ActionRun.Status dep = byJob.get(name);
165
+ return dep == ActionRun.Status.FAILURE || dep == ActionRun.Status.CANCELLED;
166
+ });
167
+ if (blocked)
168
+ {
169
+ task.status = ActionRun.Status.CANCELLED;
170
+ task.finishedAt = Instant.now();
171
+ byJob.put(task.name, task.status);
172
+ changed = true;
173
+ }
174
+ }
175
+ }
176
+ }
177
+
178
+ private static List<String> needsOf(ActionTask task)
179
+ {
180
+ if (task.needs == null || task.needs.isBlank())
181
+ {
182
+ return List.of();
183
+ }
184
+ return Arrays.stream(task.needs.split(",")).map(String::trim).filter(s -> !s.isEmpty()).toList();
185
+ }
186
+
136
187
private static ActionRun.Status map(Result result)
137
188
{
138
189
return switch (result)
MODIFY
src/main/java/de/workaround/ci/WorkflowIngestService.java
+24 -11
@@ -401,7 +401,8 @@
401
401
for (Iterator<String> it = jobs.fieldNames(); it.hasNext();)
402
402
{
403
403
String name = it.next();
404
- specs.add(new WorkflowRunFactory.JobSpec(name, runsOn(jobs.get(name))));
404
+ JsonNode job = jobs.get(name);
405
+ specs.add(new WorkflowRunFactory.JobSpec(name, runsOn(job), needs(job)));
405
406
}
406
407
}
407
408
return specs;
@@ -410,26 +411,38 @@
410
411
/** The job's {@code runs-on} as comma-joined labels; string or list, empty when absent. */
411
412
private static String runsOn(JsonNode job)
412
413
{
413
- JsonNode runsOn = job == null ? null : job.get("runs-on");
414
- if (runsOn == null)
414
+ return scalarOrList(job, "runs-on");
415
+ }
416
+
417
+ /** The job's {@code needs} as comma-joined job names; string or list, empty when absent. */
418
+ private static String needs(JsonNode job)
419
+ {
420
+ return scalarOrList(job, "needs");
421
+ }
422
+
423
+ /** A workflow field that may be a single string or a list of strings, returned comma-joined. */
424
+ private static String scalarOrList(JsonNode job, String field)
425
+ {
426
+ JsonNode node = job == null ? null : job.get(field);
427
+ if (node == null)
415
428
{
416
429
return "";
417
430
}
418
- if (runsOn.isTextual())
431
+ if (node.isTextual())
419
432
{
420
- return runsOn.asText();
433
+ return node.asText();
421
434
}
422
- if (runsOn.isArray())
435
+ if (node.isArray())
423
436
{
424
- List<String> labels = new ArrayList<>();
425
- for (JsonNode label : runsOn)
437
+ List<String> values = new ArrayList<>();
438
+ for (JsonNode value : node)
426
439
{
427
- if (label.isTextual())
440
+ if (value.isTextual())
428
441
{
429
- labels.add(label.asText());
442
+ values.add(value.asText());
430
443
}
431
444
}
432
- return String.join(",", labels);
445
+ return String.join(",", values);
433
446
}
434
447
return "";
435
448
}
MODIFY
src/main/java/de/workaround/ci/WorkflowRunFactory.java
+3 -2
@@ -56,14 +56,15 @@
56
56
task.run = run;
57
57
task.name = job.name();
58
58
task.runsOn = job.runsOn();
59
+ task.needs = job.needs();
59
60
task.payload = payload;
60
61
task.persist();
61
62
}
62
63
return run;
63
64
}
64
65
65
- /** A job discovered in a workflow: its id and comma-joined {@code runs-on} labels. */
66
- public record JobSpec(String name, String runsOn)
66
+ /** A job discovered in a workflow: its id, comma-joined {@code runs-on} labels and {@code needs}. */
67
+ public record JobSpec(String name, String runsOn, String needs)
67
68
{
68
69
}
69
70
MODIFY
src/main/java/de/workaround/model/ActionTask.java
+3 -0
@@ -54,6 +54,9 @@
54
54
/** The job's {@code runs-on} labels, comma-joined; empty means no constraint (any runner). */
55
55
public String runsOn = "";
56
56
57
+ /** Names of the jobs this task depends on (comma-joined); empty means none. */
58
+ public String needs = "";
59
+
57
60
/** The expanded single-job workflow payload delivered to the runner in FetchTask; null until materialized. */
58
61
public String payload;
59
62
ADD
src/main/resources/db/migration/V27__action_task_needs.sql
+7 -0
@@ -0,0 +1,7 @@
1
+-- Job dependencies for ordering (issue #2, phase 2, `needs`).
2
+--
3
+-- Comma-joined names of the jobs this task depends on (empty = none). A task is dispatched only once
4
+-- every needed job in the same run has succeeded; if one fails, dependents are cancelled.
5
+
6
+alter table action_task
7
+ add column needs text not null default '';
ADD
src/test/java/de/workaround/ci/NeedsOrderingTest.java
+158 -0
@@ -0,0 +1,158 @@
1
+package de.workaround.ci;
2
+
3
+import java.time.Instant;
4
+import java.time.temporal.ChronoUnit;
5
+import java.util.List;
6
+import java.util.UUID;
7
+
8
+import org.junit.jupiter.api.Test;
9
+
10
+import de.workaround.git.GitRepositoryService;
11
+import de.workaround.model.ActionRun;
12
+import de.workaround.model.ActionTask;
13
+import de.workaround.model.Repository;
14
+import de.workaround.model.User;
15
+import io.quarkus.test.junit.QuarkusTest;
16
+import jakarta.inject.Inject;
17
+import jakarta.transaction.Transactional;
18
+
19
+import static org.junit.jupiter.api.Assertions.assertEquals;
20
+import static org.junit.jupiter.api.Assertions.assertTrue;
21
+
22
+/**
23
+ * Job dependency ordering (issue #2, phase 2, {@code needs}): a task is not dispatched until every
24
+ * job it needs has succeeded; when a needed job fails, dependents are cancelled so the run finishes;
25
+ * a dispatched task carries its needs' results.
26
+ */
27
+@QuarkusTest
28
+class NeedsOrderingTest
29
+{
30
+ @Inject
31
+ RunnerRegistrationService runnerService;
32
+
33
+ @Inject
34
+ TaskDispatchService dispatch;
35
+
36
+ @Inject
37
+ TaskProgressService progress;
38
+
39
+ @Inject
40
+ GitRepositoryService repositories;
41
+
42
+ @Inject
43
+ ActionRun.Repo runs;
44
+
45
+ @Inject
46
+ ActionTask.Repo tasks;
47
+
48
+ @Test
49
+ void dependentTaskWaitsForItsNeedThenReceivesItsResult()
50
+ {
51
+ RunnerRegistrationService.RegisteredRunner reg = registerRunner();
52
+ UUID runId = seed("no-a");
53
+
54
+ // only "build" is ready; "deploy" needs "build"
55
+ ActionTask first = dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().orElseThrow();
56
+ assertEquals("build", first.name);
57
+ long buildSeq = first.seq;
58
+
59
+ assertTrue(dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().isEmpty(),
60
+ "deploy stays blocked while build is running");
61
+
62
+ progress.updateTask(reg.runner().uuid, reg.plaintext(), buildSeq,
63
+ de.workaround.ci.proto.runner.v1.Result.RESULT_SUCCESS, null);
64
+
65
+ TaskDispatchService.Fetched second = dispatch.fetch(reg.runner().uuid, reg.plaintext());
66
+ assertEquals("deploy", second.task().orElseThrow().name);
67
+ assertEquals(ActionRun.Status.SUCCESS, second.needs().get("build"), "needs result delivered");
68
+
69
+ assertTrue(runIsRunning(runId));
70
+ }
71
+
72
+ @Test
73
+ void failedNeedCancelsDependentAndEndsRun()
74
+ {
75
+ RunnerRegistrationService.RegisteredRunner reg = registerRunner();
76
+ UUID runId = seed("no-b");
77
+
78
+ ActionTask build = dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().orElseThrow();
79
+ progress.updateTask(reg.runner().uuid, reg.plaintext(), build.seq,
80
+ de.workaround.ci.proto.runner.v1.Result.RESULT_FAILURE, null);
81
+
82
+ assertEquals(ActionRun.Status.CANCELLED, deployStatus(runId), "dependent of a failed job is cancelled");
83
+ assertEquals(ActionRun.Status.FAILURE, runStatus(runId), "run finishes rather than hanging");
84
+ assertTrue(dispatch.fetch(reg.runner().uuid, reg.plaintext()).task().isEmpty());
85
+ }
86
+
87
+ private RunnerRegistrationService.RegisteredRunner registerRunner()
88
+ {
89
+ String token = runnerService.createRegistrationToken(persistUser("no-admin-" + shortId())).plaintext();
90
+ return runnerService.register(token, "no-runner", List.of(), "v4.0.0", false);
91
+ }
92
+
93
+ @Transactional
94
+ UUID seed(String repoName)
95
+ {
96
+ User owner = persistUser(repoName + "-" + shortId());
97
+ Repository repo = repositories.create(owner, repoName, Repository.Visibility.PUBLIC, null);
98
+
99
+ ActionRun run = new ActionRun();
100
+ run.repository = repo;
101
+ run.number = runs.maxNumber(repo) + 1;
102
+ run.workflowName = "CI";
103
+ run.workflowFile = ".forgejo/workflows/ci.yml";
104
+ run.event = "push";
105
+ run.ref = "refs/heads/main";
106
+ run.commitSha = "0000000000000000000000000000000000000000";
107
+ run.persist();
108
+
109
+ newTask(run, "build", "", 10);
110
+ newTask(run, "deploy", "build", 5);
111
+ return run.id;
112
+ }
113
+
114
+ private void newTask(ActionRun run, String name, String needs, int secondsAgo)
115
+ {
116
+ ActionTask task = new ActionTask();
117
+ task.run = run;
118
+ task.name = name;
119
+ task.needs = needs;
120
+ task.payload = "on: push";
121
+ task.createdAt = Instant.now().minusSeconds(secondsAgo);
122
+ task.persist();
123
+ }
124
+
125
+ @Transactional
126
+ boolean runIsRunning(UUID runId)
127
+ {
128
+ return runs.findById(runId).status == ActionRun.Status.RUNNING;
129
+ }
130
+
131
+ @Transactional
132
+ ActionRun.Status runStatus(UUID runId)
133
+ {
134
+ return runs.findById(runId).status;
135
+ }
136
+
137
+ @Transactional
138
+ ActionRun.Status deployStatus(UUID runId)
139
+ {
140
+ ActionRun run = runs.findById(runId);
141
+ return tasks.findByRun(run).stream().filter(t -> t.name.equals("deploy")).findFirst().orElseThrow().status;
142
+ }
143
+
144
+ @Transactional
145
+ User persistUser(String name)
146
+ {
147
+ User user = new User();
148
+ user.oidcSub = name;
149
+ user.username = name;
150
+ user.persist();
151
+ return user;
152
+ }
153
+
154
+ private static String shortId()
155
+ {
156
+ return UUID.randomUUID().toString().substring(0, 8);
157
+ }
158
+}
MODIFY
src/test/java/de/workaround/ci/WorkflowIngestServiceTest.java
+13 -4
@@ -94,18 +94,27 @@
94
94
steps: [{ run: echo hi }]
95
95
multi:
96
96
runs-on: [self-hosted, linux]
97
+ needs: single
97
98
steps: [{ run: echo hi }]
98
99
anywhere:
100
+ needs: [single, multi]
99
101
steps: [{ run: echo hi }]
100
102
""";
101
103
pushWorkflows(repo, Map.of(".forgejo/workflows/ci.yml", yaml));
102
104
103
105
ActionRun run = runs.findByRepository(repo).get(0);
104
- java.util.Map<String, String> byJob = tasks.findByRun(run).stream()
106
+ List<ActionTask> jobs = tasks.findByRun(run);
107
+ java.util.Map<String, String> runsOnByJob = jobs.stream()
105
108
.collect(java.util.stream.Collectors.toMap(t -> t.name, t -> t.runsOn));
106
- assertEquals("ubuntu-latest", byJob.get("single"));
107
- assertEquals("self-hosted,linux", byJob.get("multi"), "list runs-on is comma-joined");
108
- assertEquals("", byJob.get("anywhere"), "absent runs-on means no constraint");
109
+ assertEquals("ubuntu-latest", runsOnByJob.get("single"));
110
+ assertEquals("self-hosted,linux", runsOnByJob.get("multi"), "list runs-on is comma-joined");
111
+ assertEquals("", runsOnByJob.get("anywhere"), "absent runs-on means no constraint");
112
+
113
+ java.util.Map<String, String> needsByJob = jobs.stream()
114
+ .collect(java.util.stream.Collectors.toMap(t -> t.name, t -> t.needs));
115
+ assertEquals("", needsByJob.get("single"), "no needs");
116
+ assertEquals("single", needsByJob.get("multi"), "string needs");
117
+ assertEquals("single,multi", needsByJob.get("anywhere"), "list needs is comma-joined");
109
118
}
110
119
111
120
@Test