Skip to content

Commit b3aeb67

Browse files
arkqarkq
authored
Convert CI python test arguments to YAML format (project-chip#35988)
* Convert CI python test arguments to YAML format * Drop support for non-YAML CI arguments format Fixes project-chip#35787 * Rename factoryreset to factory-reset
1 parent d4c2eaf commit b3aeb67

File tree

169 files changed

+2603
-1354
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

169 files changed

+2603
-1354
lines changed

‎.github/workflows/tests.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -595,7 +595,7 @@ jobs:
595595
"
596596
- name: Run Tests
597597
run: |
598-
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/darwin-x64-all-clusters-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --quiet --app-args "--discriminator 3840 --interface-id -1" --script-args "-t 3600 --disable-test ClusterObjectTests.TestTimedRequestTimeout"'
598+
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/darwin-x64-all-clusters-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factory-reset --quiet --app-args "--discriminator 3840 --interface-id -1" --script-args "-t 3600 --disable-test ClusterObjectTests.TestTimedRequestTimeout"'
599599
- name: Uploading core files
600600
uses: actions/upload-artifact@v4
601601
if: ${{ failure() && !env.ACT }}

‎docs/guides/matter-repl.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -244,8 +244,8 @@ Some tests provides the option to exclude them. For example, you can use
244244
It is recommended to use the test wrapper to run mobile-device-test.py, for
245245
example, you can run:
246246
247-
```
248-
./scripts/tests/run_python_test.py --app chip-all-clusters-app --factoryreset
247+
```shell
248+
./scripts/tests/run_python_test.py --app chip-all-clusters-app --factory-reset
249249
```
250250

251251
It provides some extra options, for example:
@@ -254,7 +254,7 @@ It provides some extra options, for example:
254254
optional arguments:
255255
-h, --help show this help message and exit
256256
--app APP Path to local application to use, omit to use external apps.
257-
--factoryreset Remove app config and repl configs (/tmp/chip* and /tmp/repl*) before running the tests.
257+
--factory-reset Remove app config and repl configs (/tmp/chip* and /tmp/repl*) before running the tests.
258258
--app-args APP_ARGS The extra parameters passed to the device side app.
259259
--script SCRIPT Path to the test script to use, omit to use the default test script (mobile-device-test.py).
260260
--script-args SCRIPT_ARGS Arguments for the REPL test script
@@ -263,8 +263,8 @@ optional arguments:
263263
You can pass your own flags for mobile-device-test.py by appending them to the
264264
command line with two dashes, for example:
265265

266-
```
267-
./scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-no-ble-no-wifi-tsan-clang/chip-all-clusters-app --factoryreset --script-args "-t 90 --disable-test ClusterObjectTests.TestTimedRequestTimeout"
266+
```shell
267+
./scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-no-ble-no-wifi-tsan-clang/chip-all-clusters-app --factory-reset --script-args "-t 90 --disable-test ClusterObjectTests.TestTimedRequestTimeout"
268268
```
269269

270270
will pass `-t 90 --disable-test ClusterObjectTests.TestTimedRequestTimeout` to

‎docs/testing/python.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ Python tests located in src/python_testing
6161
# --passcode 20202021
6262
# --trace-to json:${TRACE_TEST_JSON}.json
6363
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
64-
# factoryreset: true
64+
# factory-reset: true
6565
# quiet: true
6666
# === END CI TEST ARGUMENTS ===
6767
@@ -627,7 +627,9 @@ for the CI). These arguments can be passed as sets of key/value pairs using the
627627
`./scripts/tests/run_python_test.py` is a convenient script that starts an
628628
example DUT on the host and includes factory reset support
629629

630-
`./scripts/tests/run_python_test.py --factoryreset --app <your_app> --app-args "whatever" --script <your_script> --script-args "whatever"`
630+
```shell
631+
./scripts/tests/run_python_test.py --factory-reset --app <your_app> --app-args "whatever" --script <your_script> --script-args "whatever"
632+
```
631633

632634
# Running tests in CI
633635

@@ -678,7 +680,7 @@ for that run, e.g.:
678680
# app: ${TYPE_OF_APP}
679681
# app-args: <app_arguments>
680682
# script-args: <script_arguments>
681-
# factoryreset: <true|false>
683+
# factory-reset: <true|false>
682684
# quiet: <true|false>
683685
# === END CI TEST ARGUMENTS ===
684686
```
@@ -692,7 +694,7 @@ for that run, e.g.:
692694

693695
- Example: `${TYPE_OF_APP}`
694696

695-
- `factoryreset`: Determines whether a factory reset should be performed
697+
- `factory-reset`: Determines whether a factory reset should be performed
696698
before the test.
697699

698700
- Example: `true`

‎scripts/tests/TestTimeSyncTrustedTimeSourceRunner.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -57,9 +57,9 @@ def run_test_section(self, app_args: str, script_args: str, factory_reset_all: b
5757
cmd = self.get_base_run_python_cmd(self.run_python_test_path, self.app_path, app_args,
5858
self.script_path, script_args)
5959
if factory_reset_all:
60-
cmd = cmd + ' --factoryreset'
60+
cmd = cmd + ' --factory-reset'
6161
if factory_reset_app:
62-
cmd = cmd + ' --factoryreset-app-only'
62+
cmd = cmd + ' --factory-reset-app-only'
6363

6464
logging.info(f'Running cmd {cmd}')
6565

‎src/controller/python/test/test_scripts/mobile-device-test.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
# --disable-test ClusterObjectTests.TestTimedRequestTimeout
3434
# --trace-to json:${TRACE_TEST_JSON}.json
3535
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
36-
# factoryreset: true
36+
# factory-reset: true
3737
# quiet: true
3838
# === END CI TEST ARGUMENTS ===
3939

‎src/python_testing/TCP_Tests.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
# --passcode 20202021
2727
# --trace-to json:${TRACE_TEST_JSON}.json
2828
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
29-
# factoryreset: true
29+
# factory-reset: true
3030
# quiet: true
3131
# === END CI TEST ARGUMENTS ===
3232
#

‎src/python_testing/TC_ACE_1_2.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
# test-runner-runs:
2323
# run1:
2424
# app: ${ALL_CLUSTERS_APP}
25-
# factoryreset: true
25+
# factory-reset: true
2626
# quiet: true
2727
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
2828
# script-args: >

‎src/python_testing/TC_ACE_1_3.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
# test-runner-runs:
2323
# run1:
2424
# app: ${ALL_CLUSTERS_APP}
25-
# factoryreset: true
25+
# factory-reset: true
2626
# quiet: true
2727
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
2828
# script-args: >

‎src/python_testing/TC_ACE_1_4.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
# --string-arg PIXIT.ACE.APPATTRIBUTE:OnOff
3535
# --trace-to json:${TRACE_TEST_JSON}.json
3636
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
37-
# factoryreset: true
37+
# factory-reset: true
3838
# quiet: true
3939
# === END CI TEST ARGUMENTS ===
4040

‎src/python_testing/TC_ACE_1_5.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
# test-runner-runs:
2323
# run1:
2424
# app: ${ALL_CLUSTERS_APP}
25-
# factoryreset: true
25+
# factory-reset: true
2626
# quiet: true
2727
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
2828
# script-args: >

‎src/python_testing/TC_ACL_2_11.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
# test-runner-runs:
2323
# run1:
2424
# app: ${NETWORK_MANAGEMENT_APP}
25-
# factoryreset: true
25+
# factory-reset: true
2626
# quiet: true
2727
# app-args: >
2828
# --discriminator 1234 --KVS kvs1

‎src/python_testing/TC_ACL_2_2.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
# test-runner-runs:
2020
# run1:
2121
# app: ${ALL_CLUSTERS_APP}
22-
# factoryreset: true
22+
# factory-reset: true
2323
# quiet: true
2424
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
2525
# script-args: >

‎src/python_testing/TC_AccessChecker.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
# test-runner-runs:
66
# run1:
77
# app: ${ALL_CLUSTERS_APP}
8-
# factoryreset: true
8+
# factory-reset: true
99
# quiet: true
1010
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
1111
# script-args: >

‎src/python_testing/TC_BOOLCFG_2_1.py

+13-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,19 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
23+
# script-args: >
24+
# --storage-path admin_storage.json
25+
# --commissioning-method on-network
26+
# --discriminator 1234
27+
# --passcode 20202021
28+
# --trace-to json:${TRACE_TEST_JSON}.json
29+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
30+
# factory-reset: true
31+
# quiet: true
2532
# === END CI TEST ARGUMENTS ===
2633

2734
import functools

‎src/python_testing/TC_BOOLCFG_3_1.py

+13-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,19 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
23+
# script-args: >
24+
# --storage-path admin_storage.json
25+
# --commissioning-method on-network
26+
# --discriminator 1234
27+
# --passcode 20202021
28+
# --trace-to json:${TRACE_TEST_JSON}.json
29+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
30+
# factory-reset: true
31+
# quiet: true
2532
# === END CI TEST ARGUMENTS ===
2633

2734
import logging

‎src/python_testing/TC_BOOLCFG_4_1.py

+13-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,19 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: --discriminator 1234 --KVS kvs1 --trace-to json:${TRACE_APP}.json
23+
# script-args: >
24+
# --storage-path admin_storage.json
25+
# --commissioning-method on-network
26+
# --discriminator 1234
27+
# --passcode 20202021
28+
# --trace-to json:${TRACE_TEST_JSON}.json
29+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
30+
# factory-reset: true
31+
# quiet: true
2532
# === END CI TEST ARGUMENTS ===
2633

2734
import logging

‎src/python_testing/TC_BOOLCFG_4_2.py

+18-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,24 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --enable-key 000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: >
23+
# --discriminator 1234
24+
# --KVS kvs1
25+
# --enable-key 000102030405060708090a0b0c0d0e0f
26+
# --trace-to json:${TRACE_APP}.json
27+
# script-args: >
28+
# --storage-path admin_storage.json
29+
# --commissioning-method on-network
30+
# --discriminator 1234
31+
# --passcode 20202021
32+
# --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f
33+
# --trace-to json:${TRACE_TEST_JSON}.json
34+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
35+
# factory-reset: true
36+
# quiet: true
2537
# === END CI TEST ARGUMENTS ===
2638

2739
import logging

‎src/python_testing/TC_BOOLCFG_4_3.py

+18-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,24 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --enable-key 000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: >
23+
# --discriminator 1234
24+
# --KVS kvs1
25+
# --enable-key 000102030405060708090a0b0c0d0e0f
26+
# --trace-to json:${TRACE_APP}.json
27+
# script-args: >
28+
# --storage-path admin_storage.json
29+
# --commissioning-method on-network
30+
# --discriminator 1234
31+
# --passcode 20202021
32+
# --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f
33+
# --trace-to json:${TRACE_TEST_JSON}.json
34+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
35+
# factory-reset: true
36+
# quiet: true
2537
# === END CI TEST ARGUMENTS ===
2638

2739
import logging

‎src/python_testing/TC_BOOLCFG_4_4.py

+18-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,24 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --enable-key 000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: >
23+
# --discriminator 1234
24+
# --KVS kvs1
25+
# --enable-key 000102030405060708090a0b0c0d0e0f
26+
# --trace-to json:${TRACE_APP}.json
27+
# script-args: >
28+
# --storage-path admin_storage.json
29+
# --commissioning-method on-network
30+
# --discriminator 1234
31+
# --passcode 20202021
32+
# --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f
33+
# --trace-to json:${TRACE_TEST_JSON}.json
34+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
35+
# factory-reset: true
36+
# quiet: true
2537
# === END CI TEST ARGUMENTS ===
2638

2739
import logging

‎src/python_testing/TC_BOOLCFG_5_1.py

+18-6
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,24 @@
1616
#
1717

1818
# === BEGIN CI TEST ARGUMENTS ===
19-
# test-runner-runs: run1
20-
# test-runner-run/run1/app: ${ALL_CLUSTERS_APP}
21-
# test-runner-run/run1/factoryreset: True
22-
# test-runner-run/run1/quiet: True
23-
# test-runner-run/run1/app-args: --discriminator 1234 --KVS kvs1 --enable-key 000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_APP}.json
24-
# test-runner-run/run1/script-args: --storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f --trace-to json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
19+
# test-runner-runs:
20+
# run1:
21+
# app: ${ALL_CLUSTERS_APP}
22+
# app-args: >
23+
# --discriminator 1234
24+
# --KVS kvs1
25+
# --enable-key 000102030405060708090a0b0c0d0e0f
26+
# --trace-to json:${TRACE_APP}.json
27+
# script-args: >
28+
# --storage-path admin_storage.json
29+
# --commissioning-method on-network
30+
# --discriminator 1234
31+
# --passcode 20202021
32+
# --hex-arg PIXIT.BOOLCFG.TEST_EVENT_TRIGGER_KEY:000102030405060708090a0b0c0d0e0f
33+
# --trace-to json:${TRACE_TEST_JSON}.json
34+
# --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
35+
# factory-reset: true
36+
# quiet: true
2537
# === END CI TEST ARGUMENTS ===
2638

2739
import logging

0 commit comments

Comments
 (0)