Repository navigation
Expand file tree
/
Copy pathMakefile
More file actions
3218 lines (2937 loc) · 191 KB
/
Copy pathMakefile
File metadata and controls
3218 lines (2937 loc) · 191 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Copyright (c) 2026 José M. Requena Plens
# SPDX-License-Identifier: MIT
.PHONY: build build-all build-linux-amd64 build-linux-arm64 build-windows-amd64 build-windows-arm64 build-darwin-amd64 build-darwin-arm64 \
run brand brand-check brand-rasters ensure-mcp-publisher mcp-publisher-version test test-short test-race coverage-conditions coverage-conditions-tree coverage-mutants test-pkg test-integration test-e2e test-e2e-harness test-e2e-http test-e2e-cloudflare-tunnel test-e2e-stdio test-e2e-collector ensure-gotestsum test-e2e-docker test-e2e-docker-enterprise test-e2e-gitlab-com \
e2e-server-binary test-e2e-ce test-e2e-ee test-e2e-gitlab e2e-clean-orphans \
orbit-setup-fixtures orbit-wait-indexer orbit-run-live-tests orbit-ensure-token gen-orbit-record check-orbit-record \
coverage \
modeleval-ce modeleval-ee modeleval-probe \
lint fmt clean version release release-check checksum \
golangci-lint govulncheck check-binary-vulns check-vex image-vulns-images check-image-vulns check-dependency-vulns check-repeatable check-reproducible check-site-headers sonar sonar-status \
mdlint mdlint-fix audit-docs check-doc-links check-md-tables \
analyze analyze-fix analyze-report install-tools \
audit-output audit-tokens audit-tools audit-surface-quality check-surface-quality check-spec-conditions audit-metrics audit-dynamic-aliases audit-test-names audit-godocs audit-godocs-check check-godocs fix-godocs \
audit-catalog-first \
audit-struct-completeness audit-action-coverage audit-1to1 audit-1to1-sdk audit-1to1-enums audit-1to1-paths audit-1to1-paths-report audit-1to1-paths-e2e audit-1to1-grants audit-1to1-grants-report audit-1to1-required audit-1to1-required-report audit-1to1-params audit-1to1-params-report audit-1to1-routes audit-1to1-routes-report audit-1to1-validate-docs gen-audit-1to1-baseline check-audit-1to1-baseline audit-1to1-issues audit-edition-tier \
audit-discovery audit-discovery-check audit-e2e-gaps audit-e2e-coverage e2e-go-coverage e2e-support-coverage e2e-support-coverage-record check-e2e-support-coverage check-e2e-static audit-gateway-chars check-gateway-chars check-test-file-names audit-test-subtests check-test-subtests check-supply-chain \
e2e-coverage-record e2e-coverage-record-ce e2e-coverage-record-ee e2e-coverage-record-render check-e2e-coverage-record check-e2e-coverage-page \
audit-md-escaping check-md-escaping \
check-em-dash check-pr-description check-plan-untracked check-dco check-file-headers fix-file-headers \
audit-action-ids check-action-ids \
audit-i18n check-i18n \
audit-dead-consts check-dead-consts \
audit-sdk-context check-sdk-context \
audit-closed-servers check-closed-servers \
audit-tenancy check-tenancy tenancy-code-identity \
check-readonly-graphql audit-readonly-graphql \
audit-meta-descriptions check-meta-descriptions \
gen-graphql-schema gen-graphql-release-schema check-graphql-schema check-graphql-documents audit-graphql-documents check-graphql-documents-live check-graphql-shapes audit-graphql-shapes audit-graphql-sent check-graphql-sdk-shapes audit-graphql-sdk-sent \
gen-api-live check-api-live gen-api-doc-endpoints check-meta-descriptions \
gen-action-grants check-action-grants check-action-grants-derivation \
record-request-inventory gen-request-inventory check-request-inventory audit-request-inventory \
gen-tool-reference check-tool-reference \
gen-action-catalog-manifest check-action-catalog-manifest gen-llms check-llms gen-lhm-manifest check-lhm-manifest gen-model-corpus check-model-corpus gen-model-results model-results-record model-results-refold model-results-dry-run check-model-results gen-icon-webp check-icon-webp check-server-json check-server-json-packages check-openplugin audit-doc-tool-names check-doc-tool-names check-install-buttons check-mcpb mcpb gen-npm sync-npm-version validate-npm validate-npm-local publish-npm-dry publish-npm gen-pypi validate-pypi validate-pypi-local publish-pypi-dry publish-pypi gen-nuget validate-nuget validate-nuget-local publish-nuget-dry publish-nuget publish-lobehub gen-footprint check-footprint gen-site-stats check-site-stats gen-testing-docs check-testing-docs update-all \
bench-resources bench-resources-render check-bench-resources bench-fairness bench-held bench-sessions \
docs-local-go \
docker-build docker-push docker-run \
inspector inspector-stop help
BINARY_NAME=gitlab-mcp-server
CMD_PATH=./cmd/server
PKGS=./cmd/... ./internal/...
# Keep in lockstep with MCP_PUBLISHER_VERSION in .github/workflows/release.yml:
# this one validates the manifest, that one publishes it. check-server-json
# refuses to run when the two disagree.
MCP_PUBLISHER_VERSION=v1.8.6
# The publisher is installed once per pinned version into a directory named
# after that version, so a stale binary can never validate with another
# release, and CI caches that directory by version instead of resolving the
# module on every push. It is not a go.mod tool on purpose: the registry
# module carries 77 direct dependencies that are not this project's.
MCP_PUBLISHER_DIR=$(HOME)/.cache/gitlab-mcp-server/mcp-publisher/$(MCP_PUBLISHER_VERSION)
MCP_PUBLISHER=$(MCP_PUBLISHER_DIR)/publisher
GO_ANALYSIS_PKGS=./...
# Every e2e build tag, and it must stay every one. A tagged file is invisible
# to go vet and to golangci-lint unless its tag is listed here, so a suite
# added behind a new tag is analysed by nothing until it is added. That has
# happened three times: `httpe2e` was missing until the HTTP suite broke CI,
# `stdioe2e` until the stdio suite did, and `orbitlive` had never been listed
# at all, so test/e2e/orbit had gone unlinted since it was written. The same
# list lives in cmd/gen_testing_docs as e2eTags, for the same reason, and in
# cmd/audit_test_closed_servers as analysisTags, which a test of that command
# holds to this line. Every
# file under test/e2e/gitlab and test/e2e/internal carries
# `e2e` and nothing else, so this one list is the whole of what the analysis
# has to know: no package under test/e2e selects between two file sets any
# more, and one run sees all of them.
GO_ANALYSIS_TAGS=e2e,collectore2e,httpe2e,orbitlive,stdioe2e
# The arguments the Markdown gate is run with, named once because it is run
# from two places. `all` expands to the six escaping contexts and to neither
# staged rule, so a target that spells `--check` without this list gates the
# escaping verdict and silently lets a hand-written card row and a raw flag or
# timestamp through. `make analyze` was in exactly that state while
# `check-md-escaping` gated all three, which is two gates disagreeing about
# what the rule is.
MD_ESCAPING_ARGS=-contexts all,card,bool-time -fail-unresolved-in internal/toolutil
PROJECT_GO_VERSION := $(shell awk '/^go / {print $$2; exit}' go.mod)
GO_TOOLCHAIN ?= go$(PROJECT_GO_VERSION)
export GOTOOLCHAIN := $(GO_TOOLCHAIN)
# E2E test report directory (inside dist/, gitignored)
E2E_REPORT_DIR=dist/e2e-reports
# Where the e2e suite records what it asked the server to do and what the
# server dispatched, one shard per test process, under one directory per
# Docker target (dist/e2e-calls/ce, dist/e2e-calls/ee). The shards are a
# byproduct of one run and are never committed; cmd/audit_e2e_coverage reads
# them back. The e2e targets export it as an absolute path, since the recorder
# refuses a relative one for the same reason the request inventory does, and
# clear the target's directory first, so a shard of an earlier run never
# folds into a later one's record. Every one of them runs with -count=1: a
# package-list run can answer from the test cache, and a cached PASS records
# nothing.
E2E_CALLS_DIR=dist/e2e-calls
# COVER=1 measures how much of the server binary an e2e run executed, which is
# a different question from the one the calls directory answers: that one says
# which catalog actions dispatched, and this one says which statements of the
# program ran at all: startup, the catalog build, the middleware chain, the
# error branches no scenario reaches. It is off by default because it costs a
# build of its own and slows every child, and because the merged profile is a
# report rather than a gate: only a live GitLab produces the input, so no
# committed artifact could be verified offline and no floor could be moved
# without booting one.
#
# With it on, the shared build carries -cover and each run target hands its
# children a GOCOVERDIR under here, one directory per target, cleared where the
# calls directory is cleared: in run-docker-e2e.sh for the two Docker targets,
# since CI invokes that script rather than the target, and in the recipe for
# the self-hosted one, for the three transport modules and for the HTTP
# module's Cloudflare Tunnel target. `make e2e-go-coverage` merges what the run
# left, so a GitLab run and a transport run fold into one figure. dist/ is
# gitignored, so `make clean` removes all of it.
#
# Two names for two things, the way E2E_CALLS_DIR and the exported
# GITLAB_MCP_TEST_E2E_CALLS_DIR are two names: E2E_COVER_ROOT is the parent
# every target writes a leaf under and the one `make e2e-go-coverage` searches,
# and E2E_COVER_DIR, exported to the harness and to run-docker-e2e.sh, is the
# one leaf a single run is handed. They were one name until the collision bit:
# CI set it to an absolute leaf for the run and passed it to make as a relative
# parent for the merge, and the two readings only happened to agree.
#
# E2E_COVER_ROOT is repository-relative and must stay so, since the line below
# joins $(CURDIR) onto it: an absolute override would be appended to the
# repository path rather than replacing it, and the counters would land under
# dist/ while the merge looked where the override said. E2E_CALLS_DIR is joined
# the same way, so this is the convention here rather than a property of
# coverage.
COVER ?=
E2E_COVER_ROOT=dist/e2e-cover
E2E_COVER_PROFILE=$(E2E_REPORT_DIR)/e2e-go-coverage.out
# Empty unless COVER is set, so an ordinary run is unaffected: nothing is built
# with -cover and the children carry no GOCOVERDIR at all. Absent rather than
# empty, because the runtime of a child given an empty one prints "GOCOVERDIR
# not set, no coverage data emitted" on its stderr the moment it starts, which
# would head every child's log for the whole run.
e2e_cover_build = $(if $(COVER),-cover)
e2e_cover_env = $(if $(COVER),E2E_COVER_DIR="$(CURDIR)/$(E2E_COVER_ROOT)/$(1)")
# The support packages of the e2e suite, test/e2e/internal/fixture and
# test/e2e/modeleval, are measured where their code runs (issue 1461): most of
# it is the part only a GitLab reaches, the fixture builders and the sessions
# that start the real binary, so their offline tests alone say little about
# it. The two complete runs, test-e2e-ce and test-e2e-ee, hand the list to
# run-docker-e2e.sh as E2E_COVERPKG, which builds every test binary of the run
# with -coverpkg naming them and writes the statements they reached to
# <stem>-coverprofile.out in the report directory, and `make
# e2e-support-coverage` merges those profiles with the one the packages'
# offline tests write (cmd/merge_coverage).
#
# Unlike COVER this is always on for those two targets: it instruments the two
# packages in the test binaries and nothing of the server, costs no build of
# its own, and a complete run is the only measurement there is of the code only
# a GitLab reaches. It is spelled once, comma-separated as -coverpkg takes it;
# .github/workflows/e2e.yml hands the same list to the script it runs directly,
# and cmd/merge_coverage's tests hold the two to each other. No package the
# complete runs test imports test/e2e/modeleval, so go test warns that nothing
# being tested depends on it and the run half reads absent for it: its online
# code runs under the modeleval targets, which do not measure it.
E2E_SUPPORT_COVERPKG = ./test/e2e/internal/fixture,./test/e2e/modeleval
e2e_comma := ,
E2E_SUPPORT_PACKAGES = $(subst $(e2e_comma), ,$(E2E_SUPPORT_COVERPKG))
# What e2e-support-coverage writes beside the runs' profiles, named so its
# pattern for those, *-coverprofile.out, never reads one of them back.
E2E_SUPPORT_OFFLINE_PROFILE = $(E2E_REPORT_DIR)/e2e-support-offline.out
E2E_SUPPORT_PROFILE = $(E2E_REPORT_DIR)/e2e-support-coverage.out
# The offline half, spelled once for the report target and the gate: the two
# packages' own tests on this tree, with no GitLab, and with the model
# evaluation's two switches set empty in the environment, which outranks
# every dotenv file the harness reads, so a .env that names a model or asks
# for the provider probe cannot turn it into a paid run. The on-demand sweep's
# two settings are set empty as well: TestSweepOrphans_Leftovers_OnDemand is
# one of the fixture package's tests, and with E2E_SWEEP_MIN_AGE or
# E2E_SWEEP_PREFIX exported in the shell, or given on the command line, it
# would delete what it matches on the instance GITLAB_URL names and add what
# that reaches to the offline half, which CI's never reaches.
e2e_support_offline = MODELEVAL_MODELS= MODELEVAL_PROBE= E2E_SWEEP_MIN_AGE= E2E_SWEEP_PREFIX= go test -tags e2e -count=1 -coverpkg=$(E2E_SUPPORT_COVERPKG) -coverprofile="$(E2E_SUPPORT_OFFLINE_PROFILE)" $(E2E_SUPPORT_PACKAGES)
# The committed record of what the complete runs reached in the two packages,
# which is the half CI cannot take: make e2e-support-coverage-record writes it
# after a complete run, and check-e2e-support-coverage, which the coverage job
# of ci.yml runs, holds each package to it (issue 1461).
E2E_SUPPORT_RECORD = docs/development/e2e-support-coverage.json
# How the gate judges a record that predates the tree: checked, as main and a
# stack's top are, or deferred, below a stack's top, where an excess the
# stale record explains is a note. CI hands it the FRESHNESS it decides.
E2E_SUPPORT_FRESHNESS ?= checked
# The revision the e2e run records on its run line, so a recorded baseline
# says which tree produced it. Overridable for a run of a tree git cannot see.
E2E_COMMIT ?= $(shell git rev-parse HEAD 2>/dev/null)
export E2E_COMMIT
# Where the unit suite records the requests it issues, one shard per test
# process. A shard is a byproduct of one run and only the merged inventory is
# committed, so this lives in the gitignored build directory. The path handed
# to the recorder is absolute because a test binary runs in its own package
# directory.
REQUEST_INVENTORY_SHARDS=dist/request-inventory
# GitLab.com Orbit live-test fixtures. All overridable on the command line
# or via .env. Defaults are designed for the canonical fixture group,
# jmrp-oss/orbit-fixtures: the subgroup rather than jmrp-oss itself, which
# holds other projects the path-bounded queries must not count.
ORBIT_FIXTURES_NAMESPACE ?= jmrp-oss/orbit-fixtures
ORBIT_FIXTURES_GITLAB_URL ?= https://gitlab.com
# How long to wait for the indexer to catch up after provisioning fixtures
# (in seconds, polled every 15s). When this elapses the make target
# proceeds with a warning; the live test assertions are tolerant of
# partial indexing (row_count > 0 rather than strict equality).
ORBIT_FIXTURES_INDEXER_TIMEOUT ?= 600
# When set to "true", additionally mirror gitlab-org/cli for realistic
# cross-entity CI/MR data. Adds ~5 min of mirror time on first run.
ORBIT_FIXTURES_MIRROR ?= false
# The -timeout handed to go test by the licensed Docker run, test-e2e-ee. go
# test applies it to each test binary on its own, so with the run naming two
# packages, common and ee, each package gets the whole budget rather than the
# two sharing it. E2E_GITLAB_TIMEOUT below is the same flag for the CE run.
#
# E2E_SWEEP_MIN_AGE, at e2e-clean-orphans, must stay above this timeout plus
# the exit hooks' budgets: it is the only thing that keeps the orphan sweep off
# a package still running.
# TestSweepMinAge_MakefileDefaults_OutlastEveryPackage fails once this default
# plus those budgets' 8m reaches it, a timeout of 112 minutes or more at the 2h
# default; a run given a longer timeout on the command line is one that test
# cannot see, and e2e-clean-orphans below says what to do then.
E2E_DOCKER_ENTERPRISE_TIMEOUT ?= 3600s
# Where the e2e Docker fixture is reached from the machine running the suite.
# Docker itself follows DOCKER_HOST or the active context, so the fixture can
# run on another host: `DOCKER_HOST=ssh://truenas
# E2E_DOCKER_GITLAB_URL=http://192.168.0.40:8929 make test-e2e-docker`. GitLab's
# own idea of its URL (external_url) follows the same value, so the web_url
# fields it answers with are the ones the tests reach. The registry and
# Bitbucket follow the same host, and test/e2e/scripts/run-docker-e2e.sh
# derives them there rather than here, reading the host out of the GitLab URL
# whatever port or path it carries: the registry's external URL, http:// and
# that host on port 5050, Bitbucket's URL, the same on port 7990, and
# Bitbucket's bind, chosen from the Bitbucket URL (the derived one, or
# E2E_DOCKER_BITBUCKET_URL when set): the loopback it names, 127.0.0.1 for
# localhost, whatever its case, the address itself for 127.x.y.z and for
# [::1], the loopback a name resolves to on this machine when it resolves to
# no other address, and 0.0.0.0 for any other host, since the setup script on
# this machine has to reach a remote container's port. Overriding
# E2E_DOCKER_BITBUCKET_URL therefore moves the
# bind with it unless E2E_BITBUCKET_BIND is set too. The
# import test never dials Bitbucket; GitLab does, over the compose network.
# E2E_REGISTRY_EXTERNAL_URL, E2E_DOCKER_BITBUCKET_URL and E2E_BITBUCKET_BIND,
# set in the environment or on the make command line, override each. A
# default here would reach the script as a value someone chose and switch the
# derivation off.
E2E_DOCKER_GITLAB_URL ?= http://localhost:8929
export E2E_GITLAB_EXTERNAL_URL = $(E2E_DOCKER_GITLAB_URL)
# Read version from VERSION file (single source of truth)
VERSION := $(strip $(file < VERSION))
COMMIT := $(shell git rev-parse --short HEAD 2>/dev/null || echo none)
LDFLAGS := -s -w -X main.version=$(VERSION) -X main.commit=$(COMMIT)
# OS detection for portable commands. Each helper takes one path and quotes
# it, so a directory an override names with a space is one argument: with the
# path unquoted, a coverage root of "dist/e2e cover" made RM_RF remove
# dist/e2e and the staged server binary in it. Every recipe that hands a tool
# a path in such a directory, or a file an override names, quotes it as well:
# a mkdir that makes the whole directory turns an unquoted path from a loud
# failure into a quiet one, as audit-e2e-coverage showed when its -o, given a
# report directory of "my reports", wrote the report to a file named my and
# exited 0. Such a path never goes through make's own path functions, which
# split it at every blank and read its glob characters as a pattern; a helper
# that carried each space through $(abspath) as a | turned every | the path
# already held into a space as well. A recipe asks the shell instead: cd and
# pwd for the absolute path a validate target mounts, test -e for the calls
# directory audit-1to1-grants-report reads.
ifeq ($(OS),Windows_NT)
BINARY_EXT=.exe
MKDIR_P=if not exist "$(subst /,\,$1)" mkdir "$(subst /,\,$1)"
RM_RF=if exist "$(subst /,\,$1)" rmdir /s /q "$(subst /,\,$1)"
RM_F=if exist "$(subst /,\,$1)" del /q "$(subst /,\,$1)"
else
BINARY_EXT=
MKDIR_P=mkdir -p "$1"
RM_RF=rm -rf "$1"
RM_F=rm -f "$1"
endif
# Analysis output directory
ANALYSIS_DIR=dist/analysis
PKGSITE ?= $(shell command -v pkgsite 2>/dev/null || printf "%s/bin/pkgsite" "$$(go env GOPATH 2>/dev/null)")
version: build
dist/$(BINARY_NAME)$(BINARY_EXT) --version
build:
go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)$(BINARY_EXT) $(CMD_PATH)
build-all: build-linux-amd64 build-linux-arm64 build-windows-amd64 build-windows-arm64 build-darwin-amd64 build-darwin-arm64
build-linux-amd64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-linux-amd64 $(CMD_PATH)
build-linux-arm64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-linux-arm64 $(CMD_PATH)
build-windows-amd64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-windows-amd64.exe $(CMD_PATH)
build-windows-arm64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=windows GOARCH=arm64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-windows-arm64.exe $(CMD_PATH)
build-darwin-amd64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-darwin-amd64 $(CMD_PATH)
build-darwin-arm64:
$(call MKDIR_P,dist)
CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -trimpath -buildmode=pie -ldflags="$(LDFLAGS)" -o dist/$(BINARY_NAME)-darwin-arm64 $(CMD_PATH)
run:
go run $(CMD_PATH)
## test: run all unit tests with verbose output and coverage profile
test:
go test -v -coverprofile=coverage.out $(PKGS)
## test-short: run all unit tests (fast, no verbose, no coverage)
test-short:
go test -count=1 $(PKGS)
## test-race: run all unit tests with race detector enabled
# The timeout is explicit because go test's default is ten minutes per package
# binary and internal/tools alone takes about 976 s under the detector, against
# 113 s without it, so this target could never finish and reported a timeout
# rather than a race (issue 536). 60m is the same bound the Race Detector
# workflow carries, and it is a bound rather than an expectation: it exists to
# end a deadlock with every goroutine stack printed.
RACE_TIMEOUT ?= 60m
test-race:
go test -v -race -timeout $(RACE_TIMEOUT) -coverprofile=coverage.out $(PKGS)
## test-pkg: run tests for a specific package domain (usage: make test-pkg PKG=branches)
test-pkg:
go test -v -count=1 ./internal/tools/$(PKG)/
## test-integration: run integration tests (build tag: integration)
test-integration:
go test -v -tags integration -coverprofile=coverage.out $(PKGS)
## test-e2e: run end-to-end tests against a real GitLab instance (reads GITLAB_URL, GITLAB_TOKEN from .env); an alias of test-e2e-gitlab.
# The self-hosted run under its shortest name. It used to run the suite the
# rebuilt one supersedes, and it points at the rebuilt one now so the name a
# developer already types runs the suite that gates releases.
test-e2e: test-e2e-gitlab
# ensure-gotestsum installs gotestsum on demand, so the e2e targets work on
# a fresh checkout without a separate install-tools step. The tool was an
# unstated assumption of the developer machine before this — the first run
# on a clean Linux box provisioned an entire GitLab stack and then died on
# "gotestsum: command not found" after several minutes.
# GOTESTSUM resolves to the PATH binary when present, else to the location
# go install writes to (GOBIN, falling back to GOPATH/bin). Recursive (=) on
# purpose: the fallback must be re-evaluated after ensure-gotestsum runs,
# because a prerequisite shell cannot extend the PATH of later recipes.
# Every recipe that runs it or hands it on quotes it, as it does PKGSITE and
# MCP_PUBLISHER, since the environment decides each path and a GOBIN, a GOPATH,
# a PATH directory or a HOME can hold a space.
GOTESTSUM_INSTALL_DIR = $(or $(shell go env GOBIN),$(shell go env GOPATH)/bin)
GOTESTSUM = $(or $(shell command -v gotestsum 2>/dev/null),$(GOTESTSUM_INSTALL_DIR)/gotestsum)
ensure-gotestsum:
@command -v gotestsum >/dev/null 2>&1 || test -x "$(GOTESTSUM_INSTALL_DIR)/gotestsum" || { \
echo "gotestsum not found; installing with go install..."; \
go install gotest.tools/gotestsum; \
}
## test-e2e-harness: run the e2e harness library's own tests (no GitLab needed).
# The harness is the only route from an e2e test to a server, so a defect in it
# would surface as a failure in whichever domain test happened to run first.
# These tests hold it on its own terms: the ledger, the waits, the names, the
# runtime guard's message, the environment a child is given, and one run of the
# real binary against a stub GitLab.
test-e2e-harness: ensure-gotestsum
$(call MKDIR_P,$(E2E_REPORT_DIR))
bash -o pipefail -c '"$(GOTESTSUM)" \
--format testdox \
--junitfile "$(E2E_REPORT_DIR)/e2e-harness-junit.xml" \
-- -tags e2e -count=1 -timeout 600s ./test/e2e/internal/...'
# The three transport modules below stage the server the way the rebuilt
# suite's targets do, through e2e-server-binary and E2E_SERVER_BINARY. Each of
# them builds cmd/server once per test binary when the variable is unset, so a
# developer running all three used to pay for three compiles of the same tree;
# with it set the build happens once here and every module drives that file.
#
# Two properties of it are the modules' own and are worth knowing before
# setting it by hand. A path naming nothing, a directory, or a file that cannot
# be executed is refused rather than built around, so a stale dist/e2e fails the
# run instead of being silently replaced by a build of its own. And a
# `go test -race` run refuses a staged binary outright, because the detector
# reaches only a binary compiled with -race and this target compiles plainly;
# nothing here passes -race, and .github/workflows/race.yml runs the modules
# directly and stages nothing, which is what keeps that refusal theoretical
# there.
#
# COVER=1 measures these three like every other run target: the shared build
# carries -cover and each of them hands its children a GOCOVERDIR under its own
# leaf. They used to stage a plain build instead, because an instrumented
# binary given no GOCOVERDIR prints "warning: GOCOVERDIR not set, no coverage
# data emitted" from the meta-data emit its init runs, before the server writes
# anything, and the stdio module requires every stderr line to parse as JSON
# (test/e2e/stdio/transport_test.go). Each module reads E2E_COVER_DIR itself
# now (test/e2e/{stdio,http,collector}/coverage_test.go), so the key is there
# and the line never appears. What that buys is the only measurement of the
# transport there is: pipes, process lifetime, the handler chain, the listener,
# and every flag a scenario against a GitLab cannot reach.
## test-e2e-stdio: run the stdio transport end-to-end module (no GitLab needed; drives the real binary over pipes).
test-e2e-stdio: ensure-gotestsum e2e-server-binary
$(call MKDIR_P,$(E2E_REPORT_DIR))
$(if $(COVER),$(call RM_RF,$(E2E_COVER_ROOT)/stdio))
bash -o pipefail -c 'E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
$(call e2e_cover_env,stdio) \
"$(GOTESTSUM)" \
--format testdox \
--junitfile "$(E2E_REPORT_DIR)/e2e-stdio-junit.xml" \
-- -tags stdioe2e -count=1 -timeout 900s ./test/e2e/stdio/'
## test-e2e-http: run the HTTP transport end-to-end module (no GitLab needed; the documented proxies and balancers skip without Docker, and fail without it under E2E_HTTP_DOCKER=required, as ci.yml's e2e-http job sets).
test-e2e-http: ensure-gotestsum e2e-server-binary
$(call MKDIR_P,$(E2E_REPORT_DIR))
$(if $(COVER),$(call RM_RF,$(E2E_COVER_ROOT)/http))
bash -o pipefail -c 'E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" E2E_CLOUDFLARE_QUICK_TUNNEL= \
$(call e2e_cover_env,http) \
"$(GOTESTSUM)" \
--format testdox \
--junitfile "$(E2E_REPORT_DIR)/e2e-http-junit.xml" \
-- -tags httpe2e -count=1 -timeout 900s ./test/e2e/http/'
## test-e2e-cloudflare-tunnel: run the HTTP module's Cloudflare Tunnel cases, a real quick tunnel through Cloudflare's edge included (Docker and Internet needed; opens a public trycloudflare.com address while it runs; refuses to run on CI).
# Run by hand, when a change touches the tunnel or the headers the edge passes
# on. The quick tunnel case opens a public address to a server on this
# machine, so it never runs on CI (decided on issue 1112, 31 October 2026):
# with CI set the target stops before it builds or starts anything, and the
# case refuses on its own when it is run some other way.
#
# An interrupted test binary runs no cleanup, and go test does not pass an
# interrupt on to it, so an interrupt would leave the cloudflared container
# and its public address behind. The run is therefore started in a process
# group of its own (set -m, POSIX job control: on Windows, run the target under
# WSL), and the recipe's shell, which exec makes make's own child so that the
# TERM make forwards reaches it, passes an INT, a TERM or a HUP (a closed
# terminal or a dropped SSH session, which signals make's group and not the
# run's) on to that group as an interrupt, and a second one kills the group.
# Once the run has ended, each container the case named in its log line ("if
# this run is interrupted: docker rm -f <name>") is removed by that exact name,
# never by a pattern, as it is after any run that ends with a nonzero status:
# a timeout, a panic off the test's goroutine or a killed process runs no
# cleanup either, and a container the case's own cleanup removed is no longer
# there to remove. The removal ignores SIGPIPE, so a reader of the output that
# went away (ssh without a terminal) does not end it at its first message, and
# a closed terminal under which make itself leads the session (ssh -t host
# make ...), whose HUP make does not pass on, ends the run at its next line of
# output, which gotestsum cannot write. The names are read from the run's own
# JSON log, named after the recipe shell's process ID so that two runs in one
# checkout never share it (a second run used to delete the first one's log and
# write its own in its place, and the first then removed the second's
# container and left its own), and deleted before the run starts, so a name
# an earlier run with that ID printed is never taken for one of this run's.
# test-e2e-http clears the quick tunnel's switch, so this is the only target
# that opens the tunnel. COVER=1 hands the run a leaf of its own, as every
# other run target does.
ifneq ($(filter test-e2e-cloudflare-tunnel,$(MAKECMDGOALS)),)
ifneq ($(CI),)
$(error test-e2e-cloudflare-tunnel never runs on CI (CI=$(CI)): its quick tunnel opens a public trycloudflare.com address to a server on this machine, see test/e2e/README.md)
endif
endif
test-e2e-cloudflare-tunnel: ensure-gotestsum e2e-server-binary
$(call MKDIR_P,$(E2E_REPORT_DIR))
$(if $(COVER),$(call RM_RF,$(E2E_COVER_ROOT)/cloudflare-tunnel))
exec bash -o pipefail -c 'log="$(E2E_REPORT_DIR)/e2e-cloudflare-tunnel-log-$$$$.json"; pid=; \
rm -f "$$log"; \
remove_named() { \
trap "" PIPE; \
for name in $$(sed -n "s/.*if this run is interrupted: docker rm -f \([A-Za-z0-9][A-Za-z0-9_.-]*\).*/\1/p" "$$log" 2>/dev/null | sort -u); do \
if docker container inspect "$$name" >/dev/null 2>&1; then \
echo "test-e2e-cloudflare-tunnel: the run $$1; removing $$name, which the case named" >&2; \
docker rm -f "$$name" >/dev/null; \
fi; \
done; \
}; \
stop() { \
pid=$${pid:-$$!}; \
trap "kill -KILL -- -$$pid 2>/dev/null" INT TERM HUP; \
if [ -n "$$pid" ]; then kill -INT -- "-$$pid" 2>/dev/null; wait "$$pid" 2>/dev/null; fi; \
remove_named "was interrupted by $$1"; \
exit "$$2"; \
}; \
trap "stop INT 130" INT; \
trap "stop TERM 143" TERM; \
trap "stop HUP 129" HUP; \
set -m; \
E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" E2E_CLOUDFLARE_QUICK_TUNNEL=true E2E_HTTP_DOCKER=required \
$(call e2e_cover_env,cloudflare-tunnel) \
"$(GOTESTSUM)" \
--format standard-verbose \
--junitfile "$(E2E_REPORT_DIR)/e2e-cloudflare-tunnel-junit.xml" \
--jsonfile "$$log" \
-- -tags httpe2e -count=1 -v -timeout 20m -run TestProxyGuide_Cloudflare ./test/e2e/http/ & \
pid=$$!; set +m; \
wait "$$pid"; status=$$?; \
if [ "$$status" -ne 0 ]; then remove_named "ended with status $$status"; fi; \
exit "$$status"'
## test-e2e-collector: run this server's telemetry into a real OpenTelemetry Collector (Docker required; skips cleanly without it).
# Deliberately absent from the push-triggered CI jobs, which run the httpe2e and
# stdioe2e tags and never this one. Those two need no daemon; this one pulls a
# container image, which belongs with the Docker-mode targets rather than in the
# path every commit waits on.
test-e2e-collector: ensure-gotestsum e2e-server-binary
$(call MKDIR_P,$(E2E_REPORT_DIR))
$(if $(COVER),$(call RM_RF,$(E2E_COVER_ROOT)/collector))
bash -o pipefail -c 'E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
$(call e2e_cover_env,collector) \
"$(GOTESTSUM)" \
--format testdox \
--junitfile "$(E2E_REPORT_DIR)/e2e-collector-junit.xml" \
-- -tags collectore2e -count=1 -timeout 900s ./test/e2e/collector/'
## test-e2e-docker: the ephemeral GitLab CE run under its older name; an alias of test-e2e-ce.
# It used to carry a Docker lifecycle of its own, written out here, that ran
# the suite this one replaced. That lifecycle now lives once in
# test/e2e/scripts/run-docker-e2e.sh and the suite it started is gone,
# so this name points at the rebuilt CE run and keeps working for anything
# that still spells it: the same GitLab CE, the same Bitbucket fixture, the
# same runner.
test-e2e-docker: test-e2e-ce
# The rebuilt suite: three packages under test/e2e/gitlab that drive the real
# binary over stdio. Each declares the runtime it needs, so the two Docker
# targets below name packages rather than build tags, and both run `common`.
# The Docker lifecycle lives in test/e2e/scripts/run-docker-e2e.sh rather than
# here, once, and every run carries -p 1 (capability locks are process-local)
# and -count=1 (a cached PASS records no calls), which the script adds.
#
# The server binary is built once and handed to every package through
# E2E_SERVER_BINARY, so the three test binaries do not each build cmd/server.
E2E_SERVER_BINARY=dist/e2e/$(BINARY_NAME)$(BINARY_EXT)
# Per package binary, like E2E_DOCKER_ENTERPRISE_TIMEOUT above: common and ce
# each get the whole of it.
#
# It was 1800s and that was not a budget, it was the wall the run hit. A CE
# run of common measures ~1800s on a developer machine, so the margin was two
# seconds, and CI, on a slower runner, panicked with "test timed out after
# 30m0s" after 1800.077s. The figure is the same 3600s the licensed run uses:
# common grows with every scenario, and a timeout is meant to stop a hang
# rather than to cap a suite that is doing its work.
#
# Raising it has a second consequence: E2E_SWEEP_MIN_AGE, at
# e2e-clean-orphans, must stay above this timeout plus the exit hooks'
# budgets, or make e2e-clean-orphans could delete the fixtures of a package
# still running. TestSweepMinAge_MakefileDefaults_OutlastEveryPackage fails
# once this default plus those budgets' 8m reaches it, a timeout of 112 minutes
# or more at the 2h default; raising it for one run on the command line
# (E2E_GITLAB_TIMEOUT=3h) is invisible to that test, and e2e-clean-orphans
# below says what to do then.
E2E_GITLAB_TIMEOUT ?= 3600s
# What makes the two Docker runs of the rebuilt suite, test-e2e-ce and
# test-e2e-ee, the complete runs: the ones whose record says what the suite
# covers, and so the ones nothing may quietly leave out.
#
# E2E_GATE_SKIPS asks run-docker-e2e.sh for a complete run. It holds the run's
# skips to the ones cmd/audit_e2e_coverage/skip_declarations.go declares for
# its runtime, so a run whose tests all passed still fails on a scenario that
# skipped for a reason nobody declared, and on a declaration no skip of it
# matched; a green run used to read as full coverage whatever it had skipped
# (issue 1014). On CE it runs the throttle probe of test/e2e/gitlab/ce, which
# changes an instance-wide setting, last and alone, in a final pass on the same
# stack before the one teardown, and the gate holds that pass to having passed
# it. It also starts the run with E2E_EXTERNAL_NETWORK=true, so the
# GitHub, Gists and Bitbucket Cloud importers, which call a public URL, run:
# the harness keeps that opt-in, since such a test fails for reasons that have
# nothing to do with this server, but the host running a complete run has
# Internet. The script sets it only when the caller did not, in the
# environment, on the make command line, in the file E2E_ENV_FILE names or in
# the repository .env, and refuses to start when the caller turned it off,
# since four importer scenarios would then skip and the gate fail the run an
# hour later. An offline host says E2E_GATE_SKIPS=false, which is not a
# complete run. The modeleval targets share the script and set neither: a
# model evaluation is not a coverage claim, and its cases skip by design where
# a tier lacks them.
e2e_complete_run_env = E2E_GATE_SKIPS="$${E2E_GATE_SKIPS:-true}"
## e2e-server-binary: build the server the rebuilt e2e suite drives, once for every package.
# Instrumented under COVER=1: the binary every documented run drives is the one
# built here, so this is where -cover has to enter. The target is .PHONY, so an
# uninstrumented build left by an earlier run can never be reused.
e2e-server-binary:
$(call MKDIR_P,dist/e2e)
go build $(e2e_cover_build) -o "$(E2E_SERVER_BINARY)" $(CMD_PATH)
## test-e2e-ce: start ephemeral GitLab CE (+ Bitbucket fixture), run the common and ce packages of the rebuilt suite, then the throttle probe alone on the same stack, tear down.
test-e2e-ce: ensure-gotestsum e2e-server-binary
$(e2e_complete_run_env) \
E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
E2E_REPORT_DIR="$(CURDIR)/$(E2E_REPORT_DIR)" \
GITLAB_MCP_TEST_E2E_CALLS_DIR="$(CURDIR)/$(E2E_CALLS_DIR)/ce" \
$(call e2e_cover_env,ce) \
E2E_COVERPKG=$(E2E_SUPPORT_COVERPKG) \
GOTESTSUM="$(GOTESTSUM)" \
./test/e2e/scripts/run-docker-e2e.sh ce -- -timeout $(E2E_GITLAB_TIMEOUT) ./test/e2e/gitlab/common/ ./test/e2e/gitlab/ce/
## test-e2e-ee: start ephemeral GitLab EE with the cached license or the activation code (+ Bitbucket fixture), run the common and ee packages of the rebuilt suite, tear down.
test-e2e-ee: ensure-gotestsum e2e-server-binary
$(e2e_complete_run_env) \
E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
E2E_REPORT_DIR="$(CURDIR)/$(E2E_REPORT_DIR)" \
GITLAB_MCP_TEST_E2E_CALLS_DIR="$(CURDIR)/$(E2E_CALLS_DIR)/ee" \
$(call e2e_cover_env,ee) \
E2E_COVERPKG=$(E2E_SUPPORT_COVERPKG) \
GOTESTSUM="$(GOTESTSUM)" \
./test/e2e/scripts/run-docker-e2e.sh ee -- -timeout $(E2E_DOCKER_ENTERPRISE_TIMEOUT) ./test/e2e/gitlab/common/ ./test/e2e/gitlab/ee/
## test-e2e-docker-enterprise: the licensed Docker run under its older name; an alias of test-e2e-ee.
# The files it used to run, the Enterprise half of the retired suite behind a
# build tag of its own, were deleted once every one of their tests had a
# successor under test/e2e/gitlab/ee, so the licensed run is the rebuilt
# suite's and this name keeps working for anything that still spells it.
test-e2e-docker-enterprise: test-e2e-ee
## test-e2e-gitlab: run the rebuilt suite against a self-hosted GitLab (reads GITLAB_URL, GITLAB_TOKEN from .env); a package the instance cannot serve skips.
test-e2e-gitlab: ensure-gotestsum e2e-server-binary
$(call MKDIR_P,$(E2E_REPORT_DIR))
$(call RM_RF,$(E2E_CALLS_DIR)/self-hosted)
$(call MKDIR_P,$(E2E_CALLS_DIR)/self-hosted)
$(if $(COVER),$(call RM_RF,$(E2E_COVER_ROOT)/self-hosted))
bash -o pipefail -c 'E2E_RUNTIME_MISMATCH=skip E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
$(call e2e_cover_env,self-hosted) \
GITLAB_MCP_TEST_E2E_CALLS_DIR="$(CURDIR)/$(E2E_CALLS_DIR)/self-hosted" "$(GOTESTSUM)" \
--format testdox \
--junitfile "$(E2E_REPORT_DIR)/e2e-gitlab-junit.xml" \
--jsonfile "$(E2E_REPORT_DIR)/e2e-gitlab-log.json" \
-- -tags e2e -p 1 -count=1 -timeout $(E2E_GITLAB_TIMEOUT) ./test/e2e/gitlab/...'
## e2e-clean-orphans: delete what earlier runs left on a GitLab (reads GITLAB_URL, GITLAB_TOKEN from .env): every project and group the token owns, every personal snippet of its user and, with an administrator's token, every user, whose name carries the ID of a run that started more than E2E_SWEEP_MIN_AGE (default 2h) ago; E2E_SWEEP_PREFIX=<prefix> sweeps by name prefix instead, whatever run and however recent.
# Run by hand and by nothing else. A run sweeps only what carries its own run
# ID, at its own exit, and a run that was killed never gets there; this is the
# sweep for what such a run left. It knows no run ID, so it reads the one a name
# carries and the second that run started, and takes it only once that is
# E2E_SWEEP_MIN_AGE ago.
#
# The default, 2h, outlasts every package of a run started with the default
# timeouts above. Each package stamps its own start, and its tests end by its
# timeout, E2E_GITLAB_TIMEOUT or E2E_DOCKER_ENTERPRISE_TIMEOUT (3600s each),
# since the binary's own alarm at that moment is a fatal panic. The exit hooks
# therefore run only after tests that ended in time, and then on the exit
# sweep's 5m and the World teardown's 3m budgets, which bound how long the
# package's fixtures are in use: its timeout plus 8m. go test's own backstop
# only shortens that: it sends SIGQUIT max(1m, timeout/10) past the timeout,
# 6m at 3600s, and kills the binary up to timeout/10 after that.
# TestSweepMinAge_MakefileDefaults_OutlastEveryPackage fails when a default
# timeout plus the exit hooks' 8m reaches E2E_SWEEP_MIN_AGE, a timeout of 112
# minutes or more at the 2h default. It cannot see a run started with a longer
# timeout on the command line or through a go test of its own: after such a
# run, set E2E_SWEEP_MIN_AGE above that timeout plus 8m, and while a run
# started with -timeout 0 is going, do not run this at all. Set it to 0 only
# when no run is going. It is read from the make command line
# (make e2e-clean-orphans E2E_SWEEP_MIN_AGE=3h10m), then from the environment,
# then from .env, and only then is the 2h default used: the recipe sources .env
# before it runs the sweep, so a floor raised there has to survive the recipe
# rather than be replaced by the default below.
#
# A self-hosted run going at the same time is not failed by this either. Such
# a run fails at its end when a group or project it saw at its start is gone
# or renamed, and it leaves out of that check every object whose name carries
# a run identifier the harness minted (not one an E2E_RUN_ID override named),
# so an older run's leftover deleted here does not count against it.
#
# A run started with E2E_RUN_ID carries no date in its names, so this default
# never reaches it. E2E_SWEEP_PREFIX sweeps by name prefix instead: it applies
# one prefix to all four kinds, reaches the token user's own objects as
# readily as the suite's, and would reach a run still going. It cannot pick
# one run either. A name opens with the prefix its test or its builder chose
# (a test's own such as adm-alert- or member-, a builder's such as snippet-
# and mergetrain-, proj- and grp- when a test names none, and the World's
# e2e-world- and world-snippet-), never with the run ID, so a prefix picks
# what some tests named across every run; reaching an E2E_RUN_ID run that way
# takes one sweep per prefix its tests used, and each also reaches other runs,
# live ones included. The prefix is read from the make command line or the
# environment only, never from .env: the recipe hands make's value on even
# when it is empty, so one left in .env is dropped rather than turning every
# later clean into a sweep by name that reaches live runs. It is a test of
# the fixture package because that library is importable only from
# test/e2e. The test skips when neither
# variable is set, which guards a bare go test run; this target always passes
# the age.
E2E_SWEEP_MIN_AGE ?= 2h
E2E_SWEEP_PREFIX ?=
e2e-clean-orphans:
bash -o pipefail -c 'if [ -f .env ]; then set -a; . ./.env; set +a; fi; \
E2E_SWEEP_MIN_AGE="$(if $(filter command line environment,$(origin E2E_SWEEP_MIN_AGE)),$(E2E_SWEEP_MIN_AGE),$${E2E_SWEEP_MIN_AGE:-$(E2E_SWEEP_MIN_AGE)})" E2E_SWEEP_PREFIX=$(E2E_SWEEP_PREFIX) go test -tags e2e -count=1 -v -run "^TestSweepOrphans_Leftovers_OnDemand$$" ./test/e2e/internal/fixture/'
## test-e2e-gitlab-com: end-to-end live test of the Orbit knowledge graph
## handlers against https://gitlab.com. Reads GITLAB_COM_TOKEN from .env,
## provisions the kg-fixtures and security-fixtures projects in
## $(ORBIT_FIXTURES_NAMESPACE) (default: jmrp-oss/orbit-fixtures),
## polls the indexer until it has caught up, runs the orbitlive-tagged
## live tests, and re-records what GitLab.com answers the six Orbit actions
## (gen-orbit-record), which fails every run while the recorded key tree
## differs from the one committed at HEAD. Idempotent: the setup script
## reconciles an existing project (its visibility and the fixture files it
## lacks) and leaves other existing resources alone, so re-running is safe.
##
## Overridable variables (command line or .env):
## ORBIT_FIXTURES_NAMESPACE default: jmrp-oss/orbit-fixtures
## ORBIT_FIXTURES_GITLAB_URL default: https://gitlab.com
## ORBIT_FIXTURES_INDEXER_TIMEOUT default: 600s
## ORBIT_FIXTURES_MIRROR default: false (set to "true" to
## mirror gitlab-org/cli
## as <namespace>/glab-mirror)
## ORBIT_FIXTURES_VISIBILITY default: public (set to "private" for
## a private group of
## your own)
##
## Examples:
## make test-e2e-gitlab-com # default group
## make test-e2e-gitlab-com ORBIT_FIXTURES_NAMESPACE=acme
## make test-e2e-gitlab-com ORBIT_FIXTURES_MIRROR=true # also mirror
test-e2e-gitlab-com: orbit-ensure-token orbit-setup-fixtures orbit-wait-indexer orbit-run-live-tests gen-orbit-record check-graphql-documents-live
@echo ""
@echo "=== test-e2e-gitlab-com complete ==="
@echo "Reports and timings printed above. To re-run later without"
@echo "re-provisioning fixtures, just call: make orbit-run-live-tests"
## orbit-ensure-token: validate that .env exists and exports GITLAB_COM_TOKEN.
## Note: Make runs each @-prefixed recipe line in a fresh subshell,
## so . ./.env and the GITLAB_COM_TOKEN check must share a shell
## invocation (joined with \ and wrapped in {}).
orbit-ensure-token:
@if [ ! -f .env ]; then \
echo "" 1>&2; \
echo "ERROR: .env not found in repo root" 1>&2; \
echo " cp .env.example .env # then add: GITLAB_COM_TOKEN=glpat-..." 1>&2; \
echo "" 1>&2; \
exit 1; \
fi
@. ./.env && { \
if [ -z "$$GITLAB_COM_TOKEN" ]; then \
echo "ERROR: GITLAB_COM_TOKEN is not set in .env" 1>&2; \
exit 1; \
fi; \
printf "✓ GITLAB_COM_TOKEN is set (length=%d)\n" "$${#GITLAB_COM_TOKEN}"; \
}
## orbit-setup-fixtures: idempotently provision the fixture projects.
## Passes the resolved namespace and optional --mirror-cli through to
## the script. An existing project is reconciled (its visibility and the
## fixture files it lacks) and other existing resources are left alone.
## ORBIT_FIXTURES_VISIBILITY is read from the make command line, then the
## environment, then .env: the recipe sources .env after make has handed
## the shell its value, so make's value is passed on explicitly, and an
## empty one leaves the script its public default.
orbit-setup-fixtures: orbit-ensure-token
@. ./.env && { \
mirror_flag=""; \
if [ "$(ORBIT_FIXTURES_MIRROR)" = "true" ]; then mirror_flag="--mirror-cli"; fi; \
echo ""; \
echo "=== Provisioning Orbit fixtures in $(ORBIT_FIXTURES_NAMESPACE) on $(ORBIT_FIXTURES_GITLAB_URL) ==="; \
export GITLAB_COM_TOKEN \
ORBIT_FIXTURES_VISIBILITY="$(if $(filter command line environment,$(origin ORBIT_FIXTURES_VISIBILITY)),$(ORBIT_FIXTURES_VISIBILITY),$${ORBIT_FIXTURES_VISIBILITY:-})" \
ORBIT_FIXTURES_NAMESPACE=$(ORBIT_FIXTURES_NAMESPACE) \
ORBIT_FIXTURES_GITLAB_URL=$(ORBIT_FIXTURES_GITLAB_URL); \
./scripts/setup-orbit-fixtures.sh $$mirror_flag; \
}
## orbit-wait-indexer: poll the Orbit index of $(ORBIT_FIXTURES_NAMESPACE)
## until it has caught up, or until ORBIT_FIXTURES_INDEXER_TIMEOUT elapses:
## caught up is GitLab's own answer, indexing.state READY with no project left
## out (projects.notIndexed 0) and at least the two fixture projects indexed,
## so a namespace indexed on an earlier run is done on the first poll rather
## than waiting the whole timeout for a count it already holds (issue 1534).
## It reads Group.orbit.indexing through GitLab's GraphQL API, which replaced
## GET /api/v4/orbit/graph_status when GitLab.com removed that route on
## 2026-10-28 (issue 1516). A null index, GitLab's answer when Orbit is not
## served to the token there, is said and polled like any other state.
## Proceeds with a warning in the timeout case: the live test skips what it
## cannot ask and is tolerant of partial indexing.
orbit-wait-indexer: orbit-ensure-token
@. ./.env && { \
echo ""; \
echo "=== Waiting for Orbit indexer to catch up (timeout: $(ORBIT_FIXTURES_INDEXER_TIMEOUT)s) ==="; \
attempts=$$(( $(ORBIT_FIXTURES_INDEXER_TIMEOUT) / 15 )); \
query='{"query":"query OrbitWaitIndexer($$fullPath: ID!) { group(fullPath: $$fullPath) { orbit { indexing { state projects { indexed notIndexed } } } } }","variables":{"fullPath":"$(ORBIT_FIXTURES_NAMESPACE)"}}'; \
for i in $$(seq 1 $$attempts); do \
progress=$$(curl -sS "$(ORBIT_FIXTURES_GITLAB_URL)/api/graphql" -H "Authorization: Bearer $$GITLAB_COM_TOKEN" -H "Content-Type: application/json" -d "$$query" 2>/dev/null \
| python3 -I -c "import sys,json; d=json.load(sys.stdin); x=(((d.get('data') or {}).get('group') or {}).get('orbit') or {}).get('indexing'); p=(x or {}).get('projects') or {}; print((x or {}).get('state') or 'null', p.get('indexed') or 0, p.get('notIndexed') or 0)" 2>/dev/null || echo "? 0 0"); \
set -- $$progress; \
printf " [%d/%d] state=%s indexed=%s not_indexed=%s\n" "$$i" "$$attempts" "$$1" "$$2" "$$3"; \
if [ "$$1" = "READY" ] && [ "$$3" -eq 0 ] && [ "$$2" -ge 2 ]; then \
printf " ✓ indexer caught up (%s projects indexed)\n" "$$2"; \
exit 0; \
fi; \
if [ "$$1" = "null" ]; then \
echo " GitLab answered the index null: Orbit is not served for $(ORBIT_FIXTURES_NAMESPACE) to this token" 1>&2; \
fi; \
sleep 15; \
done; \
echo " ⚠ indexer did not catch up within $(ORBIT_FIXTURES_INDEXER_TIMEOUT)s" 1>&2; \
echo " Proceeding anyway — live test assertions are tolerant of" 1>&2; \
echo " partial indexing (row_count > 0 rather than strict equality)" 1>&2; \
}
## orbit-run-live-tests: run the orbitlive-tagged live tests against
## gitlab.com. Standalone — useful for re-running the tests after
## manual setup or after the indexer has had more time to settle.
##
## The live suite lives at test/e2e/orbit/live_test.go (external
## `orbit_test` package, build tag `orbitlive`), not in
## internal/tools/orbit/ — pointing go test at the unit-test package
## would run only the mock-based tests and silently skip the live
## integration coverage.
orbit-run-live-tests: orbit-ensure-token
@. ./.env && { \
echo ""; \
echo "=== Running live tests (build tag: orbitlive) ==="; \
export GITLAB_COM_TOKEN \
ORBIT_FIXTURES_NAMESPACE=$(ORBIT_FIXTURES_NAMESPACE); \
go test -tags orbitlive -count=1 -v -timeout 300s ./test/e2e/orbit/; \
}
## gen-orbit-record: record what GitLab.com answers the requests the six Orbit
## handlers build (GraphQL, the Orbit MCP endpoint and POST /orbit/query), and
## write the key tree of every answer, never a value, to
## docs/development/orbit-responses.json. Reads only, through a proxy on the
## loopback interface, with GITLAB_COM_TOKEN from .env and the fixture
## namespace $(ORBIT_FIXTURES_NAMESPACE), and refuses to record over a
## namespace the indexer has not reached. It prints every key added, dropped
## or changed against the record committed at HEAD (read through git, not
## from the file it has just rewritten) and exits 1 when the tree differs,
## after writing it, so every run fails until the record is committed;
## `make audit-1to1-paths` then says what the change means for the Orbit
## output types (cmd/audit_1to1, shapes.orbit).
## ORBIT_RECORD_FLAGS=-allow-unanswered records past a call GitLab refuses the
## account (403, null where the handler reads, or a top-level group's Orbit
## access with no index, as since Orbit's general availability for a
## namespace without the orbit_promo add-on or a trial):
## it carries the answer HEAD's record holds for the same request, or lists
## the call as unrecorded with GitLab's answer.
ORBIT_RECORD_FLAGS ?=
gen-orbit-record: orbit-ensure-token
@. ./.env && { \
echo ""; \
echo "=== Recording the Orbit answers from GitLab.com ==="; \
export GITLAB_COM_TOKEN \
ORBIT_FIXTURES_NAMESPACE=$(ORBIT_FIXTURES_NAMESPACE); \
go run ./cmd/gen_orbit_record/ $(ORBIT_RECORD_FLAGS); \
}
## check-orbit-record: fail when the committed Orbit response record is not one
## this build can read, was taken from anywhere but GitLab.com, names no Orbit
## version or fixture namespace, lacks an expected call or holds one twice or
## refused, carries a key tree that does not hang together or a path that is
## not made of key names, is past the shared staleness window
## (cmd/internal/provenance), or is not in the form the generator writes. No
## token and no network, so it is a gate.
check-orbit-record:
go run ./cmd/gen_orbit_record/ -check
# A prompt that hands a case its own answer is refused by the unit suite now
# rather than by a target nobody remembers to run: TestContract_NoStimulusNamesItsOwnAnswer
# in internal/testutil/modelcorpus renders every case's stimulus and fails on
# one that names that case's own tool, action or parameter, and
# TestContract_FindsAPlantedAnswer proves the rule can fail. Both run under
# `go test ./internal/...`, so there is nothing to schedule and nothing to
# forget.
# The rebuilt model evaluation: the corpus put to a model against the real
# binary and a real GitLab, recorded as observation and scored afterwards.
#
# The two targets below are the e2e Docker lifecycle with one package swapped
# in, so a model run boots the same GitLab, the same runner and the same
# fixture service the end-to-end suite does. Nothing about them is scheduled:
# a run with a real model costs money at a provider and refuses to start
# without MODELEVAL_SPEND=yes, and a run with no MODELEVAL_MODELS at all skips.
#
# The two record directories are deliberately both under dist/modeleval and
# neither under $(E2E_CALLS_DIR). The coverage audit reads a calls directory's
# parent when it is given one, so a model run recording beside the end-to-end
# suite's shards would fold into the coverage record, and a model call is
# credited to no action by design.
MODELEVAL_DIR=dist/modeleval
## modeleval-ce: put the corpus to the configured models against an ephemeral GitLab CE (usage: MODELEVAL_MODELS=fake:perfect make modeleval-ce)
modeleval-ce: ensure-gotestsum e2e-server-binary
$(call RM_RF,$(MODELEVAL_DIR)/ce)
$(call MKDIR_P,$(MODELEVAL_DIR)/ce)
E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
E2E_REPORT_DIR="$(CURDIR)/$(E2E_REPORT_DIR)" \
E2E_REPORT_NAME=modeleval-ce \
GITLAB_MCP_TEST_MODELEVAL_DIR="$(CURDIR)/$(MODELEVAL_DIR)/ce" \
GITLAB_MCP_TEST_E2E_CALLS_DIR="$(CURDIR)/$(MODELEVAL_DIR)/ce/e2e-calls" \
GOTESTSUM="$(GOTESTSUM)" \
./test/e2e/scripts/run-docker-e2e.sh ce -- -timeout $(E2E_GITLAB_TIMEOUT) ./test/e2e/modeleval/
## modeleval-ee: the same against an ephemeral licensed GitLab EE, which is what the licensed cases need.
modeleval-ee: ensure-gotestsum e2e-server-binary
$(call RM_RF,$(MODELEVAL_DIR)/ee)
$(call MKDIR_P,$(MODELEVAL_DIR)/ee)
E2E_SERVER_BINARY="$(CURDIR)/$(E2E_SERVER_BINARY)" \
E2E_REPORT_DIR="$(CURDIR)/$(E2E_REPORT_DIR)" \
E2E_REPORT_NAME=modeleval-ee \
GITLAB_MCP_TEST_MODELEVAL_DIR="$(CURDIR)/$(MODELEVAL_DIR)/ee" \
GITLAB_MCP_TEST_E2E_CALLS_DIR="$(CURDIR)/$(MODELEVAL_DIR)/ee/e2e-calls" \
GOTESTSUM="$(GOTESTSUM)" \
./test/e2e/scripts/run-docker-e2e.sh ee -- -timeout $(E2E_DOCKER_ENTERPRISE_TIMEOUT) ./test/e2e/modeleval/
## modeleval-probe: ask each configured provider whether it accepts the request this repository builds (PAID; needs MODELEVAL_PROBE=yes).
# It needs no GitLab: it never opens an Env, so the bootstrap that probes an
# instance never runs. It is the only paid thing here that is not a run, and it
# is two requests per model plus one slice request, which is a contract check.
modeleval-probe:
MODELEVAL_PROBE=$${MODELEVAL_PROBE:-yes} \
go test -tags e2e -count=1 -v -run TestProviderContract ./test/e2e/modeleval/
## coverage-conditions: report the boolean conditions of PKG never evaluated both ways (gobco). A line reported is a missing test case; `&&`, `||` and `!` operands count separately.
# The recipe is scripts/coverage-conditions.sh. gobco parses and type-checks
# every .go file of a directory together, whatever its build constraint says,
# so a package declaring one function per platform (cmd/server,
# internal/toolutil, test/e2e/internal/harness) died with a redeclaration
# panic before measuring anything (issue 1017). The script stages such a
# package in a copy of the module under the temporary directory, holding only
# the files the go command builds here, and blanks the constraint lines of the
# kept files whose constraint names more than the platform (a tag, a release,
# cgo, the compiler), which gobco's own narrower build context would decline
# to instrument; it runs gobco there, names the files it left out, whose
# conditions this run does not measure, and removes the copy whatever
# happens. The copy is of the module so that import paths, relative paths and
# directory names stay the original's. A package whose external test package
# gobco cannot read is staged too. gobco type-checks that package against the
# package's non-test files alone, so the copy makes each test file of the
# package that declares no test (an export_test.go) a file of the package
# behind //go:build gc, which go builds and gobco does not instrument; and
# gobco bridges to it through an import of the package it may not find, so an
# external test package that imports nothing of the package is given one,
# until a gobco release fixes both (upstream-bugs entries 97 and 140). A
# package gobco could already read, a constraint on the platform alone
# included, is run where it is, as before.
# TAGS names build tags for a package behind one (TAGS=e2e), passed to go list
# and to gobco's go test alike, and a report that measured no condition (0/0)
# is refused. GOBCO_GATE=all turns the report into the gate a touched package
# is held to (it fails on any condition not evaluated both ways), and
# GOBCO_GATE=beyond:linux/amd64 into the one the Windows and macOS legs of CI
# run, which holds only the files linux/amd64 does not build. The script
# exits 3 when the gate fails a run, which no other failure of it uses, so the
# on-demand measurement (.github/workflows/conditions.yml), which runs the
# script itself, can tell a held figure from no figure; make reports any
# failure of this recipe as its own, with status 2.
coverage-conditions:
@test -n "$(PKG)" || { echo "usage: make coverage-conditions PKG=./cmd/gen_llms [TAGS=e2e]"; exit 2; }
@scripts/coverage-conditions.sh $(PKG) $(TAGS)
# Where coverage-conditions-tree leaves each package's record and raw output,
# how many packages it measures at once, and what gobco counts.
CONDITIONS_RECORDS ?= dist/conditions-tree
CONDITIONS_PARALLEL ?= 1
CONDITIONS_MODE ?= condition
## coverage-conditions-tree: condition or branch coverage (gobco) of every package under cmd/ and internal/, one row each and the total, written as the summary conditions.yml's tree dispatch writes. CONDITIONS_RECORDS keeps the records (dist/conditions-tree); CONDITIONS_PARALLEL packages run at once (1); CONDITIONS_MODE is condition or branch (condition).
# Each package is measured by scripts/coverage-conditions.sh, the recipe
# behind coverage-conditions, with GOBCO_GATE=all as the workflow's default,
# which decides only which conditions the summary marks as held and moves no
# figure, and GOBCO_MODE=$(CONDITIONS_MODE): condition is gobco's default,
# every operand of && and || and every comparison, and branch is gobco's
# -branch, the whole condition of each if and for and each switch case. The
# summary goes to stdout and to summary.md beside the records. It is the
# command docs/development/testing/README.md takes the recorded figures with,
# so they can be taken again on any machine without a dispatch; that page
# says how long each run behind them took, on how many cores and with how
# many packages at a time.
coverage-conditions-tree:
@mkdir -p "$(CONDITIONS_RECORDS)"
@bash -o pipefail -c 'go run ./cmd/measure_conditions tree -records "$(CONDITIONS_RECORDS)" -parallel "$(CONDITIONS_PARALLEL)" -mode "$(CONDITIONS_MODE)" -commit "$$(git rev-parse HEAD)" | tee "$(CONDITIONS_RECORDS)/summary.md"'
## coverage-mutants: mutation-test PKG with gremlins, INVERT_LOGICAL on so `&&`/`||` independence is checked. The gate on a changed package is Lived 0 and Not covered 0.
# The recipe itself is scripts/coverage-mutants.sh. It moved out of this file
# when it gained the staging step that makes a `package main` measurable at all
# (issue 872): that needs a copy of the package inside the module, a run against
# the copy and a removal that happens whatever the outcome, and a cleanup
# guaranteed by a trap is not something a one-line make recipe can express. The
# reasons below still describe what it does, and the reason for the staging is
# written beside it.
#
# The per-mutant timeout is derived from PKG's own baseline, because gremlins
# computes it as that baseline times a coefficient and applies no floor. On a
# fast package that product is smaller than the fixed cost of starting `go
# test` at all, so every mutant is reported TIMED OUT having never run: measured
# here, internal/tools/surfaces (0.015s of tests) reported 1 killed and 12 timed
# out, and the same package under a budget that clears the startup cost reports