@@ -85,58 +85,8 @@ The default_matter_test_main() function is used to run the test on the command
8585line. These two lines should appear verbatim at the bottom of every python test
8686file.
8787
88- ## Defining the test arguments
89-
90- Below is the format:
91-
92- ```
93- # test-runner-runs: <run_identifier>
94- # test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
95- # test-runner-run/<run_identifier>/factoryreset: <True|False>
96- # test-runner-run/<run_identifier>/quiet: <True|False>
97- # test-runner-run/<run_identifier>/app-args: <app_arguments>
98- # test-runner-run/<run_identifier>/script-args: <script_arguments>
99- ```
100-
101- ### Description of Parameters
102-
103- - test-runner-runs: Specifies the identifier for the run. This can be any
104- unique identifier.
105-
106- - Example: run1
107-
108- - test-runner-run/<run_identifier>/app: Indicates the application to be used
109- in the test. Different app types as needed could be referenced from section
110- [ name: Generate an argument environment file ] of the file
111- [ .github/workflows/tests.yaml] ( https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml )
112-
113- - Example: \${TYPE_OF_APP}
114-
115- - test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
116- reset should be performed before the test.
117-
118- - Example: True
119-
120- - test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
121- run. When set to True, the test run will be quieter.
122-
123- - Example: True
124-
125- - test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
126- passed to the application during the test.
127-
128- - Example: --discriminator 1234 --KVS kvs1 --trace-to
129- json:\$ {TRACE_APP}.json
130-
131- - test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
132- passed to the test script.
133- - Example: --storage-path admin_storage.json --commissioning-method
134- on-network --discriminator 1234 --passcode 20202021 --trace-to
135- json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
136-
137- This structured format ensures that all necessary configurations are clearly
138- defined and easily understood, allowing for consistent and reliable test
139- execution.
88+ The structured comments above the class definition are used to set up the CI for
89+ the tests. Please see [ Running tests in CI] ( #running-tests-in-ci ) .
14090
14191## Cluster Codegen
14292
@@ -610,3 +560,63 @@ example DUT on the host and includes factory reset support
610560- if there are things in your test that will fail on CI (ex. test vendor
611561 checks), gate them on the PICS_SDK_CI_ONLY
612562 - is_ci = self.check_pics('PICS_SDK_CI_ONLY')
563+
564+ The CI test runner uses a structured environment setup that can be declared
565+ using structured comments at the top of the test file. To use this structured
566+ format, use the --load-from-env flag with the run_python_tests.py runner.
567+
568+ Ex:
569+ ` scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --load-from-env /tmp/test_env.yaml --script src/python_testing/TC_ICDM_2_1.py' `
570+
571+ ## Defining the CI test arguments
572+
573+ Below is the format:
574+
575+ ```
576+ # test-runner-runs: <run_identifier>
577+ # test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
578+ # test-runner-run/<run_identifier>/factoryreset: <True|False>
579+ # test-runner-run/<run_identifier>/quiet: <True|False>
580+ # test-runner-run/<run_identifier>/app-args: <app_arguments>
581+ # test-runner-run/<run_identifier>/script-args: <script_arguments>
582+ ```
583+
584+ ### Description of Parameters
585+
586+ - test-runner-runs: Specifies the identifier for the run. This can be any
587+ unique identifier.
588+
589+ - Example: run1
590+
591+ - test-runner-run/<run_identifier>/app: Indicates the application to be used
592+ in the test. Different app types as needed could be referenced from section
593+ [ name: Generate an argument environment file ] of the file
594+ [ .github/workflows/tests.yaml] ( https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml )
595+
596+ - Example: \${TYPE_OF_APP}
597+
598+ - test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
599+ reset should be performed before the test.
600+
601+ - Example: True
602+
603+ - test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
604+ run. When set to True, the test run will be quieter.
605+
606+ - Example: True
607+
608+ - test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
609+ passed to the application during the test.
610+
611+ - Example: --discriminator 1234 --KVS kvs1 --trace-to
612+ json:\$ {TRACE_APP}.json
613+
614+ - test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
615+ passed to the test script.
616+ - Example: --storage-path admin_storage.json --commissioning-method
617+ on-network --discriminator 1234 --passcode 20202021 --trace-to
618+ json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
619+
620+ This structured format ensures that all necessary configurations are clearly
621+ defined and easily understood, allowing for consistent and reliable test
622+ execution.
0 commit comments