|
| 1 | +# Integration and Certification Tests |
| 2 | + |
| 3 | +Integration tests use a server and a controller or controllers to test the |
| 4 | +behavior of a device. Certification tests are all integration tests. For |
| 5 | +certified products, the device under test (DUT) is tested against one of the SDK |
| 6 | +controller implementations (either chip-tool or the python-based controller, |
| 7 | +depending on the test type). For software component certification, the software |
| 8 | +component is tested against a sample device built from the SDK. |
| 9 | + |
| 10 | +Certification tests require an accompanying certification test plan in order to |
| 11 | +be used in the certification testing process. More information about test plans |
| 12 | +can be found in the |
| 13 | +[test plans repository](https://github.com/CHIP-Specifications/chip-test-plans/tree/master/docs). |
| 14 | +Integration testing can also be used outside of the certification testing |
| 15 | +program to test device behavior in the SDK. Certification tests are all run in |
| 16 | +the [CI](./ci_testing). |
| 17 | + |
| 18 | +There are two main integration test types: |
| 19 | + |
| 20 | +- [YAML](./yaml.md) |
| 21 | +- [Python framework](./python.md) |
| 22 | + |
| 23 | +YAML is a human-readable serialization language that uses structured tags to |
| 24 | +define test steps. Tests are defined in YAML, and parsed and run through a |
| 25 | +runner that is backed by the chip-tool controller. |
| 26 | + |
| 27 | +The Python framework tests are written in python and use the |
| 28 | +[Mobly](https://github.com/google/mobly) test framework to execute tests. |
| 29 | + |
| 30 | +## Which test framework to use |
| 31 | + |
| 32 | +Both types of tests can be run through the Test Harness for certification |
| 33 | +testing, locally for the purposes of development and in the CI for the SDK. The |
| 34 | +appropriate test framework to use is whatever lets you automate your tests in a |
| 35 | +way that is understandable, readable, and has the features you need |
| 36 | + |
| 37 | +- YAML |
| 38 | + - pros: more readable, simpler to write, easy for ATLs to parse and |
| 39 | + understand |
| 40 | + - cons: conditionals are harder (not all supported), no branch control, |
| 41 | + schema not well documented |
| 42 | +- python |
| 43 | + - pros: full programming language, full control API with support for core |
| 44 | + (certs, commissioning, etc), less plumbing if you need to add features, |
| 45 | + can use python libraries |
| 46 | + - cons: more complex, can be harder to read |
0 commit comments