Skip to content

Commit 91dd530

Browse files
committed
Start of moving to real files
1 parent 6316f56 commit 91dd530

17 files changed

+476
-617
lines changed
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

docs/temp/out.md

+118-613
Large diffs are not rendered by default.
+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
#
2+
# Copyright (c) 2023 Project CHIP Authors
3+
# All rights reserved.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
import chip
19+
import os
20+
from pydoc_markdown import PydocMarkdown
21+
from pydoc_markdown.interfaces import Context
22+
from pydoc_markdown.contrib.loaders.python import PythonLoader
23+
from pydoc_markdown.contrib.renderers.markdown import MarkdownRenderer
24+
25+
SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
26+
WARNING = ("<!---\n"
27+
"This file is automatically generated by a script.\n"
28+
"DO NOT HAND-EDIT THIS FILE.\n"
29+
f"Script: {os.path.basename(__file__)}\n"
30+
"-->\n\n")
31+
32+
33+
def main():
34+
35+
session = PydocMarkdown()
36+
session.loaders[0].modules = chip.ChipDeviceCtrl
37+
38+
session.renderer.render_to_string(session.process(session.load_modules()))
39+
40+
md_file = os.path.abspath(os.path.join(SCRIPT_DIR, 'ChipDeviceCtrlAPI.md'))
41+
with open(md_file, "w") as f:
42+
pydoc.doc(ChipDeviceCtrl, output=f)
43+
44+
45+
if __name__ == "__main__":
46+
main()
109 KB
Loading
70.4 KB
Loading
Loading
Loading
Loading
Loading
Loading

docs/testing/img/unit_tests.png

28.3 KB
Loading

docs/testing/index.md

+21-3
Original file line numberDiff line numberDiff line change
@@ -11,16 +11,34 @@ in the SDK.
1111
*
1212
```
1313

14-
## Unit testing
14+
## Integration and Certification tests
1515

16-
- [Unit tests](./unit_testing.md)
16+
Integration tests test the entire software stack using the same mechanisms as a
17+
standard controller.
1718

18-
## Integration and Certification tests
19+
![](./img/integration_tests.png)
20+
21+
The certification tests are all integration tests, since they run against the
22+
product as a black box.
1923

2024
- [Integration and Certification tests](./integration_tests.md)
2125
- [YAML](./yaml.md)
2226
- [Python testing framework](./python.md)
2327
- [Enabling tests in the CI](./ci_testing.md)
28+
- [Test Event Triggers](./test_event_triggers.md)
29+
- [Named pipes](./named_pipes.md)
30+
- [Fault injection](./fault_injection.md)
31+
32+
## Unit testing
33+
34+
Unit tests run on small pieces (“units”) of business logic. They do not use an
35+
external controller and instead test at the public interface of the class or
36+
function. For cluster, this requires an API that separates the cluster logic
37+
from the global ember and message delivery layers.
38+
39+
![](./img/unit_tests.png)
40+
41+
- [Unit tests](./unit_testing.md)
2442

2543
## PICS and PIXIT
2644

docs/testing/unit_testing.md

+120-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,122 @@
11
# Unit testing
22

3-
This doc is a placeholder for an guide around unit tests.
3+
## Why?
4+
5+
- MUCH faster than integration tests.
6+
- Runs as part of build process.
7+
- Allows testing specific error conditions that are difficult to trigger under
8+
normal operating conditions.
9+
- e.g. out of memory errors etc.
10+
- Allows testing different device compositions without defining multiple
11+
example applications.
12+
- e.g. feature combinations not in example apps.
13+
14+
## Unit testing in the SDK - nlUnitTest
15+
16+
The following example gives a small demonstration of how to use nlUnitTest to
17+
write a unit test
18+
19+
```
20+
#include <lib/support/UnitTestContext.h>
21+
#include <lib/support/UnitTestRegistration.h>
22+
#include <nlunit-test.h>
23+
24+
class YourTestContext : public Test::AppContext {
25+
...
26+
};
27+
28+
29+
static void TestName(nlTestSuite * apSuite, void * apContext) {
30+
// If you register the test suite with a context, cast
31+
// apContext as appropriate
32+
YourTestContext * ctx = static_cast<YourTestContext *>(aContext);
33+
34+
// Do some test things here, then check the results using NL_TEST_ASSERT
35+
NL_TEST_ASSERT(apSuite, <boolean condition>)
36+
}
37+
38+
static const nlTest sTests[] =
39+
{
40+
NL_TEST_DEF("TestName", TestName), // Can have multiple of these
41+
NL_TEST_SENTINEL() // If you forget this, you’re going to have a bad time
42+
};
43+
44+
nlTestSuite sSuite =
45+
{
46+
"TheNameOfYourTestSuite", // Test name
47+
&sTests[0], // The list of tests to run
48+
TestContext::Initialize, // Runs before all the tests (can be nullptr)
49+
TestContext::Finalize // Runs after all the tests (can be nullptr)
50+
};
51+
52+
int YourTestSuiteName()
53+
{
54+
return chip::ExecuteTestsWithContext<YourTestContext>(&sSuite); // or “without”
55+
}
56+
57+
CHIP_REGISTER_TEST_SUITE(YourTestSuiteName)
58+
```
59+
60+
Each test gets an nlTestSuite object (apSuite) that is passed into the test
61+
assertions, and a void\* context (apContext) that is yours to do with as you
62+
require.
63+
64+
The apContext should be derived from
65+
[Test::AppContext](https://github.com/project-chip/connectedhomeip/blob/master/src/app/tests/AppTestContext.h)
66+
67+
See
68+
[TestSpan.cpp](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/tests/TestSpan.cpp)
69+
for a great example of a good unit test.
70+
71+
## nlUnitTest - Compiling and running
72+
73+
- Add to src/some_directory/tests/BUILD.gn
74+
- chip_test_suite_using_nltest("tests")
75+
- See for example
76+
[src/lib/support/tests/BUILD.gn](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/tests/BUILD.gn)
77+
- [./gn_build.sh](https://github.com/project-chip/connectedhomeip/blob/master/gn_build.sh)
78+
will build and run all tests
79+
- CI runs this, so any unit tests that get added will automatically be
80+
added to the CI
81+
- Test binaries are compiled into:
82+
- out/debug/<host_compiler>/tests
83+
- e.g. out/debug/linux_x64_clang/tests
84+
- Tests are run when ./gn_build.sh runs, but you can run them individually in
85+
a debugger from their location.
86+
87+
## Debugging unit tests
88+
89+
- After running ./gn_build.sh, test binaries are compiled into
90+
- out/debug/<host_compiler>/tests
91+
- e.g. out/debug/linux_x64_clang/tests
92+
- Individual binaries, can be run through regular tools:
93+
- gdb
94+
- valgrind
95+
- Your favorite tool that you tell everyone about.
96+
97+
## Utilities
98+
99+
We have a small number of unit testing utilities that should be used in unit
100+
tests.
101+
102+
Consider adding more utilities for general use if you require them for your
103+
tests.
104+
105+
### Mock clock
106+
107+
The mock clock is located in
108+
[src/system/SystemClock.h](https://github.com/project-chip/connectedhomeip/blob/master/src/system/SystemClock.h)
109+
as `System::Clock::Internal::MockClock`.
110+
111+
To use the mock clock, use the `chip::System::SystemClock()` function as normal.
112+
In the test, instantiate a MockClock and use the `SetSystemClockForTesting` to
113+
inject the clock. The Set and Advance functions in the MockClock can then be
114+
used to set exact times for testing. This allows testing specific edge
115+
conditions in tests, helps reduce test flakiness due to race conditions, and
116+
reduces the time required for testing as tests no long require real-time waits.
117+
118+
### TestPersistentStorageDelegate
119+
120+
The TestPersistentStorageDelegate is an in-memory version of storage that easily
121+
allows removal of keys, presence checks, etc. It is available at
122+
[src/lib/support/TestPersistentStorageDelegate.h](https://github.com/project-chip/connectedhomeip/blob/master/src/lib/support/TestPersistentStorageDelegate.h)

docs/testing/unit_testing_clusters.md

+171
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Designing Clusters for Testing and Portability
2+
3+
## Unit Testable, Modular Cluster Design
4+
5+
When designing new clusters, consider the following approach:
6+
7+
- Separate the cluster logic from the on-the-wire data model
8+
- Server vs. ClusterLogic
9+
- Makes the cluster logic unit-testable without generating TLV.
10+
- Separate the basic cluster logic from code that is platform- or
11+
device-specific.
12+
- ClusterLogic uses a ClusterDriver
13+
- Makes the cluster logic portable between platforms / manufacturers
14+
- Removes necessity of overriding global singleton functions like
15+
PostAttributeChangeCallback.
16+
17+
General approach:
18+
19+
![](./img/unit_testable_clusters.png)
20+
21+
Class proposal:
22+
23+
![](./img/unit_testable_clusters_all_classes.png)
24+
25+
### ClusterServer
26+
27+
![](./img/unit_testable_clusters_server.png)
28+
29+
The ClusterServerClass is a **Very** light wrapper over ClusterLogic. It
30+
translates Interaction Model wire format handling into API calls for cluster
31+
logic methods.
32+
33+
This class iImplements both the AttributeAccessInterface and the CommandHandler
34+
interfaces so ClusterLogic properly handles data dependencies between command
35+
and attributes.
36+
37+
An example code snippet showing the translation of the TLV into API calls to the
38+
ClusterLogic class:
39+
40+
```c++
41+
CHIP_ERROR DiscoBallServer::Read(const ConcreteReadAttributePath & aPath,
42+
AttributeValueEncoder & aEncoder)
43+
{
44+
DiscoBallClusterLogic * cluster = FindEndpoint(aPath.mEndpointId);
45+
VerifyOrReturnError(cluster != nullptr, CHIP_IM_GLOBAL_STATUS(UnsupportedEndpoint));
46+
47+
switch (aPath.mAttributeId)
48+
{
49+
case Clusters::DiscoBall::Attributes::Run::Id:
50+
return aEncoder.Encode(cluster->GetRunAttribute());
51+
52+
}
53+
}
54+
```
55+
56+
### ClusterLogic
57+
58+
![](./img/unit_testable_clusters_logic.png)
59+
60+
The ClusterLogic class is for all the code that is SHARED between platforms. It
61+
does NOT include any TLV parsing or direct calls to Ember/IM/LogEvent etc.
62+
63+
The class should include attribute getter/setters and handlers for all commands.
64+
65+
The class receive “plain data” Matter requests from ClusterServer class,
66+
performs required common actions, and calls driver class to perform platform- or
67+
hardware-specific actions. It also receives driver updates (e.g.
68+
application-driven value changes) from the ClusterDriver class and updates state
69+
as appropriate.
70+
71+
The class should handle spec-requirements for:
72+
73+
- Range checking (CONSTRAINT_ERROR)
74+
- Attribute and metadata storage (persistent or in-memory)
75+
- Data dependencies between commands and attributes
76+
- Event generation / dirty attributes (callback to server)
77+
- Calling driver when platform or hardware interactions are required
78+
79+
API recommendation:
80+
81+
- Maintain all cluster state in a separate data-only class.
82+
- Provider getters/setters for application logic to use.
83+
- Implement handlers for all commands, conditional on features.
84+
- Let the caller provide (inject) dependencies. Avoid explicit memory
85+
management.
86+
87+
### ClusterDriver
88+
89+
Implements hardware or platform-specific actions required on cluster
90+
interactions or when application wants to report state changes.
91+
92+
![](./img/unit_testable_clusters_driver.png)
93+
94+
### MatterContext
95+
96+
The ClusterLogic class must not directly use global resource because they cannot
97+
be isolated for testing. Instead, the MatterContext holds pointers to Matter
98+
stack objects, which can be be injected / faked for testing. This includes -
99+
Wrapper over IM Engine interface functions for marking attributes dirty, and
100+
logging events. - Storage - Anything you would normally access with
101+
Server::GetInstance()
102+
103+
![](./img/unit_testable_clusters_context.png)
104+
105+
### ClusterDriver
106+
107+
The ClusterDriver is called by the ClusterLogic class and is used to translate
108+
attribute changes and commands into application actions. It also reports
109+
external changes back to the ClusterLogic class.
110+
111+
The API design for this class with vary by the cluster, but it si generally
112+
recommended to use a generic API where possible, so the API ports easily to
113+
other platforms. For example an attribute changed callback with the changes
114+
listed. It is important to be careful about the design and revisit this early if
115+
issues arise.
116+
117+
## Unit testing with the modular cluster design
118+
119+
### ClusterLogic class
120+
121+
The unit test instantiates the ClusterLogic, provides MatterContext and
122+
ClusterDriver instance with fakes/mocks for testing.
123+
124+
Unit test against the API, and check the fakes/mocks to ensure they are being
125+
called as appropriate.
126+
127+
As with all unit tests, prefer testing for behavior rather than implementation
128+
details.
129+
130+
Important tests to consider:
131+
132+
- Initialization and initial attribute correctness.
133+
- Errors for out-of-range on all attribute setters and command handlers.
134+
- All spec-defined error conditions, especially ones that are difficult to
135+
trigger.
136+
- Data dependencies between commands and attributes.
137+
- Incoming actions in different states (stopped, running, etc).
138+
- Calls out to storage for persistent attributes.
139+
- Calls out to driver for changes as appropriate.
140+
- Driver error reporting.
141+
- Event generation and dirty attribute marking, including attributes that are
142+
changed from the driver side.
143+
- Others - very dependent on the cluster.
144+
145+
# Unit testing ClusterServer
146+
147+
- Best to have the lightest wrapping possible
148+
- If the wrapper is light, the code can be covered by integration or unit
149+
tests, or a combination.
150+
- Correctness can mostly be validated by inspection if it’s trivial.
151+
- Important tests
152+
- Errors when ClusterLogic instances aren’t properly registered.
153+
- Flow through to ClusterLogic for all reads/writes/command.
154+
- Can unit test this class by generating the TLV / path for input, parsing the
155+
TLV output.
156+
157+
# Unit testing existing clusters
158+
159+
- Important for clusters where there are multiple configurations that cannot
160+
easily be represented with example apps
161+
- Option 1
162+
- Refactor the cluster logic to be unit-testable.
163+
- Option 2
164+
- Test at AttributeAccessInterface boundary.
165+
- Instantiate or access the cluster server instance in the test.
166+
- Read / Write TLV and use TLV encode/decode functions to verify
167+
correctness.
168+
- See TestPowerSourceCluster for an example of how to do this
169+
- Additional test coverage on clusters, especially for hard to trigger
170+
conditions, is important. However, **don’t let perfection be the enemy of
171+
progress** .

0 commit comments

Comments
 (0)