1
1
# Python framework tests
2
2
3
- This file is a placeholder for python framework test information.
4
-
5
- NOTE: be sure to include information about how you need to commission with the
6
- python controller, not chip-tool and how to do that in the scripts
7
-
8
- Note - go through the docs in the readme and move them over here, and take out
9
- the deprecated stuff
10
-
11
- Note - add notes about record error and fail test vs. asserts
12
-
13
- ---
14
-
15
3
The python test framework is built on top of the ChipDeviceCtrl.py python
16
4
controller API and the mobly test framework. Python tests are interaction tests,
17
5
and can be used for certification testing, and / or integration testing in the
@@ -25,7 +13,7 @@ Python tests located in src/python_testing
25
13
sample test showing test setup and test harness integration
26
14
- [ https://github.com/google/mobly/blob/master/docs/tutorial.md ] ( https://github.com/google/mobly/blob/master/docs/tutorial.md )
27
15
- [ ChipDeviceCtrl.py] ( https://github.com/project-chip/connectedhomeip/blob/master/src/controller/python/chip/ChipDeviceCtrl.py ) -
28
- Controller implementation with API documentation
16
+ Controller implementation ( [ API documentation] ( ./ChipDeviceCtrlAPI.md ) )
29
17
- [ scripts/tests/run_python_test.py] ( https://github.com/project-chip/connectedhomeip/blob/master/scripts/tests/run_python_test.py )
30
18
to easily set up app and script for testing - used in CI
31
19
@@ -408,6 +396,25 @@ reference.
408
396
- if the test is exceptionally long running, define a property function
409
397
` default_timeout ` to adjust the timeout. The default is 90 seconds
410
398
399
+ Deferred failures: For some tests, it makes sense to perform the entire test
400
+ before failing and collect all the errors so the developers can address all the
401
+ failures without needing to re-run the test multiple times. For example, tests
402
+ that look at every attribute on the cluster and perform independent operations
403
+ on them etc.
404
+
405
+ For such tests, use the ProblemNotice format and the convenience functions:
406
+
407
+ - self.record_error
408
+ - self.record_warning
409
+
410
+ These functions keep track of the problems, and will print them at the end of
411
+ the test. The test will not be failed until the assert is called.
412
+
413
+ A good example of this type of test can be found in the device basic composition
414
+ tests, where all the test steps are independent and performed on a single read.
415
+ See
416
+ [ Device Basic Composition tests] ( https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_DeviceBasicComposition.py )
417
+
411
418
## Command line arguments
412
419
413
420
- Use help to get a full list
@@ -476,11 +483,14 @@ second_ctrl = fa.new_fabric_admin.NewController(nodeId=node_id)
476
483
You can run the python script as-is for local testing against an already-running
477
484
DUT
478
485
479
- ./scripts/tests/run_python_test.py is a convenient script to fire up an example
480
- DUT on the host, with factory reset support
486
+ ` ./scripts/tests/run_python_test.py ` is a convenient script to fire up an
487
+ example DUT on the host, with factory reset support
481
488
482
489
` ./scripts/tests/run_python_test.py --factoryreset --app <your_app> --app-args "whatever" --script <your_script> --script-args "whatever" `
483
490
491
+ Note that devices must be commissioned by the python test harness to run tests.
492
+ chip-tool and the python test harness DO NOT share a fabric.
493
+
484
494
# Running tests in CI
485
495
486
496
- add to .github/workflows/tests.yaml repl_tests_linux
0 commit comments