Skip to content

Commit 5af4e44

Browse files
cecilleaustina-csa
authored andcommitted
Docs: move CI python testing args into CI section (project-chip#34264)
1 parent 97222c8 commit 5af4e44

File tree

1 file changed

+62
-52
lines changed

1 file changed

+62
-52
lines changed

docs/testing/python.md

+62-52
Original file line numberDiff line numberDiff line change
@@ -85,58 +85,8 @@ The default_matter_test_main() function is used to run the test on the command
8585
line. These two lines should appear verbatim at the bottom of every python test
8686
file.
8787

88-
## Defining the test arguments
89-
90-
Below is the format:
91-
92-
```
93-
# test-runner-runs: <run_identifier>
94-
# test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
95-
# test-runner-run/<run_identifier>/factoryreset: <True|False>
96-
# test-runner-run/<run_identifier>/quiet: <True|False>
97-
# test-runner-run/<run_identifier>/app-args: <app_arguments>
98-
# test-runner-run/<run_identifier>/script-args: <script_arguments>
99-
```
100-
101-
### Description of Parameters
102-
103-
- test-runner-runs: Specifies the identifier for the run. This can be any
104-
unique identifier.
105-
106-
- Example: run1
107-
108-
- test-runner-run/<run_identifier>/app: Indicates the application to be used
109-
in the test. Different app types as needed could be referenced from section
110-
[name: Generate an argument environment file ] of the file
111-
[.github/workflows/tests.yaml](https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml)
112-
113-
- Example: \${TYPE_OF_APP}
114-
115-
- test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
116-
reset should be performed before the test.
117-
118-
- Example: True
119-
120-
- test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
121-
run. When set to True, the test run will be quieter.
122-
123-
- Example: True
124-
125-
- test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
126-
passed to the application during the test.
127-
128-
- Example: --discriminator 1234 --KVS kvs1 --trace-to
129-
json:\${TRACE_APP}.json
130-
131-
- test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
132-
passed to the test script.
133-
- Example: --storage-path admin_storage.json --commissioning-method
134-
on-network --discriminator 1234 --passcode 20202021 --trace-to
135-
json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
136-
137-
This structured format ensures that all necessary configurations are clearly
138-
defined and easily understood, allowing for consistent and reliable test
139-
execution.
88+
The structured comments above the class definition are used to set up the CI for
89+
the tests. Please see [Running tests in CI](#running-tests-in-ci).
14090

14191
## Cluster Codegen
14292

@@ -610,3 +560,63 @@ example DUT on the host and includes factory reset support
610560
- if there are things in your test that will fail on CI (ex. test vendor
611561
checks), gate them on the PICS_SDK_CI_ONLY
612562
- is_ci = self.check_pics('PICS_SDK_CI_ONLY')
563+
564+
The CI test runner uses a structured environment setup that can be declared
565+
using structured comments at the top of the test file. To use this structured
566+
format, use the --load-from-env flag with the run_python_tests.py runner.
567+
568+
Ex:
569+
`scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --load-from-env /tmp/test_env.yaml --script src/python_testing/TC_ICDM_2_1.py'`
570+
571+
## Defining the CI test arguments
572+
573+
Below is the format:
574+
575+
```
576+
# test-runner-runs: <run_identifier>
577+
# test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
578+
# test-runner-run/<run_identifier>/factoryreset: <True|False>
579+
# test-runner-run/<run_identifier>/quiet: <True|False>
580+
# test-runner-run/<run_identifier>/app-args: <app_arguments>
581+
# test-runner-run/<run_identifier>/script-args: <script_arguments>
582+
```
583+
584+
### Description of Parameters
585+
586+
- test-runner-runs: Specifies the identifier for the run. This can be any
587+
unique identifier.
588+
589+
- Example: run1
590+
591+
- test-runner-run/<run_identifier>/app: Indicates the application to be used
592+
in the test. Different app types as needed could be referenced from section
593+
[name: Generate an argument environment file ] of the file
594+
[.github/workflows/tests.yaml](https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml)
595+
596+
- Example: \${TYPE_OF_APP}
597+
598+
- test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
599+
reset should be performed before the test.
600+
601+
- Example: True
602+
603+
- test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
604+
run. When set to True, the test run will be quieter.
605+
606+
- Example: True
607+
608+
- test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
609+
passed to the application during the test.
610+
611+
- Example: --discriminator 1234 --KVS kvs1 --trace-to
612+
json:\${TRACE_APP}.json
613+
614+
- test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
615+
passed to the test script.
616+
- Example: --storage-path admin_storage.json --commissioning-method
617+
on-network --discriminator 1234 --passcode 20202021 --trace-to
618+
json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
619+
620+
This structured format ensures that all necessary configurations are clearly
621+
defined and easily understood, allowing for consistent and reliable test
622+
execution.

0 commit comments

Comments
 (0)