Skip to content

Commit bfce150

Browse files
rogpepperogpeppe
committed
encoding/jsonschema: encode oneOf etc using matchN
This CL updates JSON Schema generation to use the newly added `matchN` primitive for `allOf`, `anyOf` and `oneOf`. This has a much closer correlation with the JSON Schema primitives than the current approach of using CUE's `&` and `|` operators. Specifically: - the schema arguments should not affect the final result other than to validate it, but both `&` and `|` can affect the result. - the result could become non-concrete due to `|` ambiguity in `anyOf`. Although this does fix a bunch of issues, there are some regressions. See issues #3418, #3420, #3422 for details. Comparative test stats (pre-matchN before post-matchN): ``` v2: schema extract (pass / total): 971 / 1637 = 59.3% schema extract (pass / total): 975 / 1637 = 59.6% tests (pass / total): 3081 / 7175 = 42.9% tests (pass / total): 3140 / 7175 = 43.8% tests on extracted schemas (pass / total): 3081 / 3542 = 87.0% tests on extracted schemas (pass / total): 3140 / 3546 = 88.6% v3: schema extract (pass / total): 971 / 1637 = 59.3% schema extract (pass / total): 967 / 1637 = 59.1% tests (pass / total): 3063 / 7175 = 42.7% tests (pass / total): 3074 / 7175 = 42.8% tests on extracted schemas (pass / total): 3063 / 3542 = 86.5% tests on extracted schemas (pass / total): 3074 / 3538 = 86.9% ``` This change also requires that we update the CI generated code and remove the workaround for the previous `oneOf` limitation. Also add a `brokenInV2` tag to the encoding/jsonschema tests to work around the fact that some V2 tests are now broken. For #3380 For #3165 Signed-off-by: Roger Peppe <[email protected]> Change-Id: I2630a6d2b1614b2479802e788c16249d2cf4aa6b Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1200526 Reviewed-by: Daniel Martí <[email protected]> TryBot-Result: CUEcueckoo <[email protected]>
1 parent 2e19c1f commit bfce150

40 files changed

+503
-434
lines changed

‎.github/workflows/evict_caches.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ jobs:
2121
run: touch -t 202211302355 $(find * -type d)
2222
- name: Restore git file modification times
2323
uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
24-
- id: DispatchTrailer
25-
name: Try to extract Dispatch-Trailer
24+
- name: Try to extract Dispatch-Trailer
25+
id: DispatchTrailer
2626
run: |-
2727
x="$(git log -1 --pretty='%(trailers:key=Dispatch-Trailer,valueonly)')"
2828
if [[ "$x" == "" ]]
@@ -38,11 +38,11 @@ jobs:
3838
echo "value<<EOD" >> $GITHUB_OUTPUT
3939
echo "$x" >> $GITHUB_OUTPUT
4040
echo "EOD" >> $GITHUB_OUTPUT
41-
- if: |-
41+
- name: Check we don't have Dispatch-Trailer on a protected branch
42+
if: |-
4243
((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
4344
Dispatch-Trailer: {"type":"')))) && (contains(github.event.head_commit.message, '
4445
Dispatch-Trailer: {"type":"'))
45-
name: Check we don't have Dispatch-Trailer on a protected branch
4646
run: |-
4747
echo "github.event.head_commit.message contains Dispatch-Trailer but we are on a protected branch"
4848
false

‎.github/workflows/push_tip_to_trybot.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,6 @@ concurrency: push_tip_to_trybot
1010
jobs:
1111
push:
1212
runs-on: ubuntu-22.04
13-
defaults:
14-
run:
15-
shell: bash
16-
if: ${{github.repository == 'cue-lang/cue'}}
1713
steps:
1814
- name: Write netrc file for cueckoo Gerrithub
1915
run: |-
@@ -49,3 +45,7 @@ jobs:
4945
echo "Giving up"
5046
exit 1
5147
fi
48+
defaults:
49+
run:
50+
shell: bash
51+
if: ${{github.repository == 'cue-lang/cue'}}

‎.github/workflows/release.yml

+11-11
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ concurrency: release
1414
jobs:
1515
goreleaser:
1616
runs-on: ubuntu-22.04
17+
if: ${{github.repository == 'cue-lang/cue'}}
1718
defaults:
1819
run:
1920
shell: bash
20-
if: ${{github.repository == 'cue-lang/cue'}}
2121
steps:
2222
- name: Checkout code
2323
uses: actions/checkout@v4
@@ -28,8 +28,8 @@ jobs:
2828
run: touch -t 202211302355 $(find * -type d)
2929
- name: Restore git file modification times
3030
uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
31-
- id: DispatchTrailer
32-
name: Try to extract Dispatch-Trailer
31+
- name: Try to extract Dispatch-Trailer
32+
id: DispatchTrailer
3333
run: |-
3434
x="$(git log -1 --pretty='%(trailers:key=Dispatch-Trailer,valueonly)')"
3535
if [[ "$x" == "" ]]
@@ -45,11 +45,11 @@ jobs:
4545
echo "value<<EOD" >> $GITHUB_OUTPUT
4646
echo "$x" >> $GITHUB_OUTPUT
4747
echo "EOD" >> $GITHUB_OUTPUT
48-
- if: |-
48+
- name: Check we don't have Dispatch-Trailer on a protected branch
49+
if: |-
4950
((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
5051
Dispatch-Trailer: {"type":"')))) && (contains(github.event.head_commit.message, '
5152
Dispatch-Trailer: {"type":"'))
52-
name: Check we don't have Dispatch-Trailer on a protected branch
5353
run: |-
5454
echo "github.event.head_commit.message contains Dispatch-Trailer but we are on a protected branch"
5555
false
@@ -76,13 +76,13 @@ jobs:
7676
install-only: true
7777
version: v2.2.0
7878
- name: Run GoReleaser with CUE
79-
run: cue cmd release
80-
working-directory: ./internal/ci/goreleaser
8179
env:
8280
GITHUB_TOKEN: ${{ secrets.CUECKOO_GITHUB_PAT }}
83-
- if: startsWith(github.ref, 'refs/tags/v')
84-
name: Re-test cuelang.org
81+
run: cue cmd release
82+
working-directory: ./internal/ci/goreleaser
83+
- name: Re-test cuelang.org
84+
if: startsWith(github.ref, 'refs/tags/v')
8585
run: 'curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ${{ secrets.CUECKOO_GITHUB_PAT }}" -H "X-GitHub-Api-Version: 2022-11-28" --fail --request POST --data-binary "{\"event_type\":\"Re-test post release of ${GITHUB_REF##refs/tags/}\"}" https://api.github.com/repos/cue-lang/cuelang.org/dispatches'
86-
- if: startsWith(github.ref, 'refs/tags/v')
87-
name: Trigger unity build
86+
- name: Trigger unity build
87+
if: startsWith(github.ref, 'refs/tags/v')
8888
run: 'curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ${{ secrets.PORCUEPINE_GITHUB_PAT }}" -H "X-GitHub-Api-Version: 2022-11-28" --fail --request POST --data-binary "{\"event_type\":\"Check against CUE ${GITHUB_REF##refs/tags/}\",\"client_payload\":{\"type\":\"unity\",\"payload\":{\"versions\":\"\\\"${GITHUB_REF##refs/tags/}\\\"\"}}}" https://api.github.com/repos/cue-unity/unity-private/dispatches'

‎.github/workflows/tip_triggers.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@ name: Triggers on push to tip
88
jobs:
99
push:
1010
runs-on: ubuntu-22.04
11+
if: ${{github.repository == 'cue-lang/cue'}}
1112
defaults:
1213
run:
1314
shell: bash
14-
if: ${{github.repository == 'cue-lang/cue'}}
1515
steps:
1616
- name: Trigger unity build
1717
run: 'curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ${{ secrets.PORCUEPINE_GITHUB_PAT }}" -H "X-GitHub-Api-Version: 2022-11-28" --fail --request POST --data-binary "{\"event_type\":\"Check against ${GITHUB_SHA}\",\"client_payload\":{\"type\":\"unity\",\"payload\":{\"versions\":\"\\\"commit:${GITHUB_SHA}\\\"\"}}}" https://api.github.com/repos/cue-unity/unity-private/dispatches'

‎.github/workflows/trybot.yml

+39-39
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@ name: TryBot
99
- release-branch.*
1010
tags-ignore:
1111
- v*
12-
pull_request: {}
1312
workflow_dispatch: {}
13+
pull_request: {}
1414
jobs:
1515
test:
1616
strategy:
@@ -24,13 +24,13 @@ jobs:
2424
- macos-14
2525
- windows-2022
2626
runs-on: ${{ matrix.runner }}
27-
defaults:
28-
run:
29-
shell: bash
3027
if: |-
3128
(contains(github.event.head_commit.message, '
3229
Dispatch-Trailer: {"type":"trybot"')) || ! (contains(github.event.head_commit.message, '
3330
Dispatch-Trailer: {"type":"'))
31+
defaults:
32+
run:
33+
shell: bash
3434
steps:
3535
- name: Checkout code
3636
uses: actions/checkout@v4
@@ -41,8 +41,8 @@ jobs:
4141
run: touch -t 202211302355 $(find * -type d)
4242
- name: Restore git file modification times
4343
uses: chetan/git-restore-mtime-action@075f9bc9d159805603419d50f794bd9f33252ebe
44-
- id: DispatchTrailer
45-
name: Try to extract Dispatch-Trailer
44+
- name: Try to extract Dispatch-Trailer
45+
id: DispatchTrailer
4646
run: |-
4747
x="$(git log -1 --pretty='%(trailers:key=Dispatch-Trailer,valueonly)')"
4848
if [[ "$x" == "" ]]
@@ -58,11 +58,11 @@ jobs:
5858
echo "value<<EOD" >> $GITHUB_OUTPUT
5959
echo "$x" >> $GITHUB_OUTPUT
6060
echo "EOD" >> $GITHUB_OUTPUT
61-
- if: |-
61+
- name: Check we don't have Dispatch-Trailer on a protected branch
62+
if: |-
6263
((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
6364
Dispatch-Trailer: {"type":"')))) && (contains(github.event.head_commit.message, '
6465
Dispatch-Trailer: {"type":"'))
65-
name: Check we don't have Dispatch-Trailer on a protected branch
6666
run: |-
6767
echo "github.event.head_commit.message contains Dispatch-Trailer but we are on a protected branch"
6868
false
@@ -71,73 +71,73 @@ jobs:
7171
with:
7272
cache: false
7373
go-version: ${{ matrix.go-version }}
74-
- id: go-mod-cache-dir
75-
name: Get go mod cache directory
74+
- name: Get go mod cache directory
75+
id: go-mod-cache-dir
7676
run: echo "dir=$(go env GOMODCACHE)" >> ${GITHUB_OUTPUT}
77-
- id: go-cache-dir
78-
name: Get go build/test cache directory
77+
- name: Get go build/test cache directory
78+
id: go-cache-dir
7979
run: echo "dir=$(go env GOCACHE)" >> ${GITHUB_OUTPUT}
80-
- if: |-
81-
(((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
82-
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
83-
uses: actions/cache@v4
84-
with:
80+
- with:
8581
path: |-
8682
${{ steps.go-mod-cache-dir.outputs.dir }}/cache/download
8783
${{ steps.go-cache-dir.outputs.dir }}
8884
key: ${{ runner.os }}-${{ matrix.go-version }}-${{ github.run_id }}
8985
restore-keys: ${{ runner.os }}-${{ matrix.go-version }}
90-
- if: |-
91-
! (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
86+
if: |-
87+
(((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
9288
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
93-
uses: actions/cache/restore@v4
94-
with:
89+
uses: actions/cache@v4
90+
- with:
9591
path: |-
9692
${{ steps.go-mod-cache-dir.outputs.dir }}/cache/download
9793
${{ steps.go-cache-dir.outputs.dir }}
9894
key: ${{ runner.os }}-${{ matrix.go-version }}-${{ github.run_id }}
9995
restore-keys: ${{ runner.os }}-${{ matrix.go-version }}
96+
uses: actions/cache/restore@v4
97+
if: |-
98+
! (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
99+
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test'))
100100
- if: |-
101101
github.repository == 'cue-lang/cue' && (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
102102
Dispatch-Trailer: {"type":"')))) || github.ref == 'refs/heads/ci/test')
103103
run: go clean -testcache
104-
- if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
105-
name: Early git and code sanity checks
104+
- name: Early git and code sanity checks
105+
if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
106106
run: go run ./internal/ci/checks
107-
- if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
108-
name: Generate
107+
- name: Generate
109108
run: go generate ./...
110-
- if: |-
109+
if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
110+
- name: Test
111+
if: |-
111112
((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
112113
Dispatch-Trailer: {"type":"')))) || !(matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
113-
name: Test
114114
run: go test ./...
115-
- if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
116-
name: Test with -race
117-
run: go test -race ./...
115+
- name: Test with -race
118116
env:
119117
GORACE: atexit_sleep_ms=10
118+
if: (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
119+
run: go test -race ./...
120120
- name: Test with -tags=cuewasm
121121
run: go test -tags cuewasm ./cmd/cue/cmd ./cue/interpreter/wasm
122122
- name: gcloud auth for end-to-end tests
123123
id: auth
124+
uses: google-github-actions/auth@v2
124125
if: |-
125126
github.repository == 'cue-lang/cue' && (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
126127
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test')) && (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
127-
uses: google-github-actions/auth@v2
128128
with:
129129
credentials_json: ${{ secrets.E2E_GCLOUD_KEY }}
130-
- if: |-
130+
- name: gcloud setup for end-to-end tests
131+
if: |-
131132
github.repository == 'cue-lang/cue' && (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
132133
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test')) && (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
133-
name: gcloud setup for end-to-end tests
134134
uses: google-github-actions/setup-gcloud@v2
135-
- if: |-
136-
github.repository == 'cue-lang/cue' && (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
137-
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test')) && (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
138-
name: End-to-end test
135+
- name: End-to-end test
139136
env:
140137
CUE_TEST_LOGINS: ${{ secrets.E2E_CUE_LOGINS }}
138+
if: |-
139+
github.repository == 'cue-lang/cue' && (((github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-branch.')) && (! (contains(github.event.head_commit.message, '
140+
Dispatch-Trailer: {"type":"')))) || (github.ref == 'refs/heads/ci/test')) && (matrix.go-version == '1.23.x' && matrix.runner == 'ubuntu-22.04')
141141
run: |-
142142
cd internal/_e2e
143143
go test -race
@@ -147,6 +147,6 @@ jobs:
147147
go vet ./...
148148
go mod tidy
149149
(cd internal/_e2e && go test -run=-)
150-
- if: always()
151-
name: Check that git is clean at the end of the job
150+
- name: Check that git is clean at the end of the job
151+
if: always()
152152
run: test -z "$(git status --porcelain)" || (git status; git diff; false)

‎.github/workflows/trybot_dispatch.yml

+7-7
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@ name: Dispatch trybot
99
jobs:
1010
trybot:
1111
runs-on: ubuntu-22.04
12+
if: ${{ ((github.ref == 'refs/heads/ci/test') && false) || github.event.client_payload.type == 'trybot' }}
1213
defaults:
1314
run:
1415
shell: bash
15-
if: ${{ ((github.ref == 'refs/heads/ci/test') && false) || github.event.client_payload.type == 'trybot' }}
1616
steps:
1717
- name: Write netrc file for cueckoo Gerrithub
1818
run: |-
@@ -22,17 +22,17 @@ jobs:
2222
password ${{ secrets.CUECKOO_GERRITHUB_PASSWORD }}
2323
EOD
2424
chmod 600 ~/.netrc
25-
- id: payload
25+
- name: Write fake payload
26+
id: payload
2627
if: github.repository == 'cue-lang/cue' && (github.ref == 'refs/heads/ci/test')
27-
name: Write fake payload
2828
run: |-
2929
cat <<EOD >> $GITHUB_OUTPUT
3030
value<<DOE
3131
null
3232
DOE
3333
EOD
34-
- if: github.event.client_payload.type != 'trybot'
35-
name: Trigger TryBot (fake data)
34+
- name: Trigger TryBot (fake data)
35+
if: github.event.client_payload.type != 'trybot'
3636
run: |-
3737
mkdir tmpgit
3838
cd tmpgit
@@ -81,8 +81,8 @@ jobs:
8181
echo "Giving up"
8282
exit 1
8383
fi
84-
- if: github.event.client_payload.type == 'trybot'
85-
name: Trigger TryBot (repository_dispatch payload)
84+
- name: Trigger TryBot (repository_dispatch payload)
85+
if: github.event.client_payload.type == 'trybot'
8686
run: |-
8787
mkdir tmpgit
8888
cd tmpgit

0 commit comments

Comments
 (0)