|
| 1 | +# Integration Test utilities |
| 2 | + |
| 3 | +There are several test utilities that can be used to simulate or force behavior |
| 4 | +on devices for the purposes of testing. |
| 5 | + |
| 6 | +When using any of these utilties it is important to inject the errors at the |
| 7 | +point where they are running through the MOST code that they can. |
| 8 | + |
| 9 | +If the cluster uses the [ClusterLogic](./unit_testing_clusters.md) pattern, this |
| 10 | +means injecting errors as close as possible to the driver layer, rather than |
| 11 | +catching errors in the server. |
| 12 | + |
| 13 | +## TestEventTriggers |
| 14 | + |
| 15 | +TestEventTriggers are used to test interactions on the DUT that are difficult to |
| 16 | +perform during certification testing (ex. triggering a smoke alarm) |
| 17 | + |
| 18 | +**These should be used sparingly!** |
| 19 | + |
| 20 | +TestEventTriggers are started though a command in the General Diagnostics |
| 21 | +cluster. The command takes a “test key” and a “trigger code” to request that a |
| 22 | +device to perform a specific action. Currently most devices use a default key, |
| 23 | +but it can be overridden by a specific device if required. |
| 24 | + |
| 25 | +**TestEventTriggers need to be turned off outside of certification tests** |
| 26 | + |
| 27 | +To use these triggers: |
| 28 | + |
| 29 | +- Derive from |
| 30 | + [TestEventTriggerHandler](https://github.com/project-chip/connectedhomeip/blob/master/src/app/TestEventTriggerDelegate.h) |
| 31 | +- Implement HandleEventTrigger function |
| 32 | +- Register with TestEventTriggerDelegate::AddHandler |
| 33 | + |
| 34 | +Please see |
| 35 | +[EnergyEvseTestEventTriggerHandler](https://github.com/project-chip/connectedhomeip/blob/master/src/app/clusters/energy-evse-server/EnergyEvseTestEventTriggerHandler.h) |
| 36 | +for a good example. |
| 37 | + |
| 38 | +## NamedPipes |
| 39 | + |
| 40 | +NamedPipes are used to trigger actions on Linux applications. These can be used |
| 41 | +in the CI, and are normally used to simulate manual actions for CI integration. |
| 42 | +Any required manual action in a test (ex. push a button) should have a |
| 43 | +corresponding NamedPipe action to allow the test to run in the CI. |
| 44 | + |
| 45 | +In python tests, the app-pid required to access the named pipe can be passed in |
| 46 | +as a flag (--app-pid). |
| 47 | + |
| 48 | +NamedPipes are implemented in |
| 49 | +[NamedPipeCommands.h](https://github.com/project-chip/connectedhomeip/blob/master/examples/platform/linux/NamedPipeCommands.h) |
| 50 | + |
| 51 | +To use NamedPipes |
| 52 | + |
| 53 | +- Derive from NamedPipeCommandDelegate |
| 54 | +- Implement the OnEventCommandReceived(const char \* json) function |
| 55 | +- Instantiate and start a NamedPipeCommands object to receive commands and |
| 56 | + pass in the NamedPipeCommandDelegate and a file path basename |
| 57 | +- (while running) Write to the file (basename_pid) to trigger the actions |
| 58 | + |
| 59 | +For a good example, see Air Quality: |
| 60 | + |
| 61 | +- [Delegate](https://github.com/project-chip/connectedhomeip/blob/master/examples/air-quality-sensor-app/linux/AirQualitySensorAppAttrUpdateDelegate.cpp) |
| 62 | +- [main](https://github.com/project-chip/connectedhomeip/blob/master/examples/air-quality-sensor-app/linux/main.cpp) |
| 63 | +- [README](https://github.com/project-chip/connectedhomeip/blob/master/examples/air-quality-sensor-app/linux/README.md) |
| 64 | + |
| 65 | +[RVC Clean Mode](https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_RVCCLEANM_2_1.py) |
| 66 | +gives an example of how to use named pipes in testing. |
| 67 | + |
| 68 | +## Fault Injection |
| 69 | + |
| 70 | +Fault injection is used to inject conditional code paths at runtime, e.g. |
| 71 | +errors. This is very useful for things like client testing, to check error |
| 72 | +handling for paths that are difficult to hit under normal operating conditions. |
| 73 | + |
| 74 | +The fault injection framework we are currently using is nlFaultInjection.The |
| 75 | +framework uses a macro for injecting the code paths, and the macro is set to a |
| 76 | +no-op if the option is turned off at compile time. The build option to turn on |
| 77 | +fault inject is `chip_with_nlfaultinjection`. |
| 78 | + |
| 79 | +Fault injection has been plumbed out through a manufacturer-specific |
| 80 | +[fault injection cluster](#fault-injection-cluster) that is available in the |
| 81 | +SDK. This allows fault injection to be turned on and off using standard cluster |
| 82 | +mechanisms during testing. For certification, operating these using a secondary, |
| 83 | +non-DUT controller is recommended. For a good example of this, please see |
| 84 | +[TC-IDM-1.3](https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/interactiondatamodel.adoc#tc-idm-1-3-batched-commands-invoke-request-action-from-dut-to-th-dut_client). |
| 85 | + |
| 86 | +The nlFaultInjection allows the application to define multiple managers. In the |
| 87 | +SDK, we have managers for System, inet and CHIP. CHIP should be used for |
| 88 | +anything above the system layer (basically all new cluster development). The |
| 89 | +CHIP fault manager is available at |
| 90 | +[lib/support/CHIPFaultInjection.h](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/CHIPFaultInjection.h). |
| 91 | + |
| 92 | +To add new fault injection code paths: |
| 93 | + |
| 94 | +- Add new IDs (aFaultID) to the enum in |
| 95 | + [CHIPFaultInjection](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/CHIPFaultInjection.h) |
| 96 | +- add CHIP_FAULT_INJECT(aFaultID, aStatements) at the point where the fault |
| 97 | + injection should occur |
| 98 | + |
| 99 | +### Fault Injection example |
| 100 | + |
| 101 | +```c++ |
| 102 | +CHIP_ERROR CASEServer::OnMessageReceived(Messaging::ExchangeContext * ec, |
| 103 | + const PayloadHeader & payloadHeader, |
| 104 | + System::PacketBufferHandle && payload) |
| 105 | +{ |
| 106 | + MATTER_TRACE_SCOPE("OnMessageReceived", "CASEServer"); |
| 107 | + |
| 108 | + bool busy = GetSession().GetState() != CASESession::State::kInitialized; |
| 109 | + CHIP_FAULT_INJECT(FaultInjection::kFault_CASEServerBusy, busy = true); |
| 110 | + if (busy) |
| 111 | + { |
| 112 | +… |
| 113 | +``` |
| 114 | +
|
| 115 | +### Fault Injection cluster |
| 116 | +
|
| 117 | +The Fault injection cluster is a manufacturer-specific cluster, available in the |
| 118 | +SDK (0xFFF1FC06). |
| 119 | +
|
| 120 | +- [server](https://github.com/project-chip/connectedhomeip/blob/master/src/app/clusters/fault-injection-server/fault-injection-server.cpp) |
| 121 | +- [xml](https://github.com/project-chip/connectedhomeip/blob/master/src/app/zap-templates/zcl/data-model/chip/fault-injection-cluster.xml) |
| 122 | +
|
| 123 | +Example apps can be compiled with this cluster for client-side certification and |
| 124 | +integration tests. |
| 125 | +
|
| 126 | +To use this cluster to turn on a fault, use the FailAtFault command: |
| 127 | +
|
| 128 | +- Type: FaultType - use FaultType::kChipFault (0x03) |
| 129 | +- Id: int32u - match the ID you set up for your fault |
| 130 | +- NumCallsToSkip: int32u - number of times to run normally |
| 131 | +- NumCallsToFail: int32u - number of times to hit the fault injection |
| 132 | + condition after NumCallsToSkip |
| 133 | +- TakeMutex: bool - controls access to the fault injection manager for |
| 134 | + multi-threaded systems. False is fine. |
0 commit comments