Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TC-CNET-4.13: Update cnet 4.13 test as per spec #33024

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

TC-CNET-4.13: Update cnet 4.13 test as per spec #33024

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0a70418
send a commission complete command after adding networks
Kshitijjain21 Apr 17, 2024
8a0ec9a
Merge branch 'project-chip:master' into update-CNET-4.13-test
Kshitijjain21 Apr 17, 2024
aaa8a20
Restyled by prettier-yaml
restyled-commits Apr 17, 2024
ebccaae
Merge branch 'master' into update-CNET-4.13-test
Kshitijjain21 Apr 17, 2024
2207e90
arm fail safe after commissioning complete command
Kshitijjain21 Apr 18, 2024
3886fb2
Merge branch 'update-CNET-4.13-test' of github.com:Kshitijjain21/conn…
Kshitijjain21 Apr 18, 2024
1164d6a
Merge branch 'project-chip:master' into update-CNET-4.13-test
Kshitijjain21 Apr 18, 2024
e6a2f46
arm fail safe after commissioning complete command
Kshitijjain21 Apr 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 70 additions & 50 deletions src/app/tests/suites/certification/Test_TC_CNET_4_13.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ tests:
disabled: true

- label:
"Step 1: TH sendsArmFailSafecommand to theDUT withExpiryLengthSeconds
set to 900"
"Step 1: TH sends ArmFailSafecommand to the DUT with
ExpiryLengthSeconds set to 900"
verification: |
./chip-tool generalcommissioning arm-fail-safe 900 0 1 0

Expand All @@ -50,19 +50,20 @@ tests:
disabled: true

- label:
"Step 2: TH readsNetworksattribute from theDUT and saves thenumber of
entries as 'NumNetworks'"
"Step 2: TH reads Networks attribute from the DUT and saves the number
of entries as 'NumNetworks'"
PICS: CNET.S.A0001
verification: |
./chip-tool network-commissioning read networks 1 0
./chip-tool networkcommissioning read networks 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 3: TH calculates thenumber ofremainingnetwork slots
as'MaxNetworksValue' -'NumNetworks'and saves as'RemainingNetworkSlots'"
"Step 3: TH calculates the number ofremaining network slots as
'MaxNetworksValue' - 'NumNetworks' and saves as
'RemainingNetworkSlots'"
verification: |
The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
Expand All @@ -81,15 +82,15 @@ tests:

- label:
"Step 4: TH calculates the midpoint of the network list as
floor(('MaxNetwor ksValue' + 1)/2) and saves as 'Midpoint'"
floor(('MaxNetworksValue' + 1)/2) and saves as 'Midpoint'"
verification: |
The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 5: TH sends ArmFailSafe command to the DUT with the
ExpiryLengthSeco nds field set to 900"
ExpiryLengthSeconds field set to 900"
verification: |
./chip-tool generalcommissioning arm-fail-safe 900 0 1 0

Expand All @@ -99,10 +100,10 @@ tests:
disabled: true

- label:
"Step 2: TH sendsAddOrUpdateWiFiNetworkcommand to theDUT. This
stepshould be repeated'RemainingNetworkSlots' times usingDIFFERENT
SSIDand credentialvalues and theBreadcrumb fieldset to 1. Note
thatthese credentialsare NOT requiredto be connectable."
"Step 2: TH sends AddOrUpdateWiFiNetworkcommand to the DUT. This step
should be repeated 'RemainingNetworkSlots' times using DIFFERENT SSID
and credentialvalues and the Breadcrumb fieldset to 1. Note that these
credentials are NOT required to be connectable."
PICS: CNET.S.C02.Rsp && CNET.S.C05.Tx
verification: |
./chip-tool networkcommissioning add-or-update-wi-fi-network hex:63686970736574757035 matter123 1 0 --Breadcrumb 1
Expand All @@ -112,8 +113,8 @@ tests:
disabled: true

- label:
"Step 6: TH readsNetworksattribute from theDUT and saves thelist
as'OriginalNetworkList'"
"Step 6: TH readsNetworks attribute from the DUT and saves the list as
'OriginalNetworkList'"
PICS: CNET.S.A0001
verification: |
./chip-tool networkcommissioning read networks 1 0
Expand All @@ -122,20 +123,39 @@ tests:
( Pre-Condition)
disabled: true

- label: "Step 7: TH sends the CommissioningComplete command to the DUT"
verification: |
./chip-tool generalcommissioning CommissioningComplete 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new disabled (i.e. manual) test step. We generally do not allow new manual test steps anymore.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback. The change done here is allow this test to align with the matter spec. Since the test was a manual test and has not been changed to an automated/semi automated test what would be the correct course of action to get it to align with the spec?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andy31415 since this is a old cert test, we can make updates to the manual test.
@Kshitijjain21 But we can not merge this change unless test plan PR is approved.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds several new steps that are manual ... so it kind of makes the problem worse. I understand the justification of "we add a test step in a long list of test steps that is already manual" however I also am concerned that this does increase the count of manual stuff invocation. We should not do that.


- label:
"Step 7: TH sends ReorderNetwo rk Command to the DUT with the
following fields: 1. NetworkID is PIXIT.CNET .WIFI_1ST_ ACCESSPOI
NT_SSID 2. NetworkIn dex is 'MaxNetwo rksValue' 3. Breadcrum b is 2"
"Step 8: TH sends ArmFailSafe command to the DUT with the
ExpiryLengthSeconds field set to 900"
verification: |
./chip-tool generalcommissioning arm-fail-safe 900 0 1 0


The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 9: TH sends ReorderNetwork Command to the DUT with the following
fields: 1. NetworkID is PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID 2.
NetworkIndex is 'MaxNetworksValue' 3. Breadcrumb is 2"
PICS: CNET.S.C08.Rsp && CNET.S.C05.Tx
verification: |
./chip-tool network-commissioning read networks 1 0
./chip-tool networkcommissioning reorder-network hex:47524C50726976617465 1 0 --Breadcrumb 2

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 8: TH reads Breadcumb attribute from the General Commissioning
"Step 10: TH reads Breadcumb attribute from the General Commissioning
Cluster"
verification: |
./chip-tool generalcommissioning read breadcrumb 1 0
Expand All @@ -145,17 +165,17 @@ tests:
disabled: true

- label:
"Step 9: TH sendsReorderNetwork Commandto the DUTwith
thefollowingfields: 1. NetworkIDis aNetworkIDvalue NOTpresent
in'OriginalNetworkList'2.NetworkIn dex is'Midpoint'3.Breadcrumb is 2"
"Step 11: TH sends ReorderNetwork Command to the DUT with the
following fields: 1. NetworkID is a NetworkID value NOT present in
'OriginalNetworkList' 2.NetworkIndex is'Midpoint' 3.Breadcrumb is 2"
PICS: CNET.S.C08.Rsp && CNET.S.C05.Tx
verification: |
The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 10: TH reads Breadcumb attribute from the General Commissioning
"Step 12: TH reads Breadcumb attribute from the General Commissioning
Cluster"
verification: |
./chip-tool generalcommissioning read breadcrumb 1 0
Expand All @@ -165,58 +185,58 @@ tests:
disabled: true

- label:
"Step 11: TH sends ReorderNetwo rk Command to the DUT with the
following fields: 1. NetworkID is PIXIT.CNET .WIFI_1ST_ ACCESSPOI
NT_SSID 2. NetworkIn dex is 'Midpoint' 3. Breadcrum b is 2"
"Step 13: TH sends ReorderNetwork Command to the DUT with the
following fields: 1. NetworkID is PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID
2. NetworkIndex is 'Midpoint' 3. Breadcrumb is 2"
PICS: CNET.S.C08.Rsp && CNET.S.C05.Tx
verification: |
./chip-tool networkcommissioning reorder-network hex:47524C50726976617465 1 0 --Breadcrumb 1
./chip-tool networkcommissioning reorder-network hex:47524C50726976617465 1 0 --Breadcrumb 2

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 12: TH readsBreadcumbattribute from
theGeneralCommissioningCluster"
"Step 14: TH reads Breadcrumb attribute from the GeneralCommissioning
Cluster"
verification: |
./chip-tool generalcommissioning read breadcrumb 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label: "Step 13: TH readsNetworksattribute list fromthe DUT"
- label: "Step 15: TH reads Networks attribute list from the DUT"
PICS: CNET.S.A0001
verification: |
./chip-tool network-commissioning read networks 1 0
./chip-tool networkcommissioning read networks 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 14: TH sendsArmFailSafecommand to theDUT withExpiryLengthSeconds
set to 0"
"Step 16: TH sends ArmFailSafe command to the DUT with
ExpiryLengthSeconds set to 0"
verification: |
./chip-tool generalcommissioning arm-fail-safe 0 0 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label: "Step 15: TH readsNetworksattribute list fromthe DUT"
- label: "Step 17: TH reads Networks attribute list from the DUT"
PICS: CNET.S.A0001
verification: |
./chip-tool network-commissioning read networks 1 0
./chip-tool networkcommissioning read networks 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 16: TH sends ArmFailSafe command to the DUT with
ExpiryLengthSeco nds set to 900"
"Step 18: TH sends ArmFailSafe command to the DUT with
ExpiryLengthSeconds set to 900"
verification: |
./chip-tool generalcommissioning arm-fail-safe 900 0 1 0

Expand All @@ -226,28 +246,28 @@ tests:
disabled: true

- label:
"Step 17: TH sendsAddOrUpdateWiFiNetworkcommand to theDUT. This
stepshould be repeated'RemainingNetworkSlots' times usingDIFFERENT
SSIDand credentialvalues and Breadcrumb fieldset to 1. Note thatthese
credentialsare NOT requiredto be connectable."
"Step 19: TH sends AddOrUpdateWiFiNetwork command to the DUT. This
step should be repeated 'RemainingNetworkSlots' times using DIFFERENT
SSID and credential values and Breadcrumb field set to 1. Note that
these credentials are NOT required to be connectable."
PICS: CNET.S.C02.Rsp && CNET.S.C05.Tx
verification: |
The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label:
"Step 18: TH sendsReorderNetwork Commandto the DUTwith
thefollowingfields: 1.
NetworkIDisPIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID2.NetworkIndex is
"Step 20: TH sends ReorderNetwork Command to the DUT with the
following fields: 1. NetworkID is
PIXIT.CNET.WIFI_1ST_ACCESSPOINT_SSID2.NetworkIndex is
'Midpoint'3.Breadcrumb is 2"
PICS: CNET.S.C08.Rsp && CNET.S.C05.Tx
verification: |
The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label: "Step 19: TH sends the CommissioningCo mplete command to the DUT"
- label: "Step 21: TH sends the CommissioningComplete command to the DUT"
verification: |
./chip-tool generalcommissioning CommissioningComplete 1 0

Expand All @@ -256,19 +276,19 @@ tests:
disabled: true

- label:
"Step 20: TH sendsArmFailSafecommand to theDUT withExpiryLengthSeconds
set to 0"
"Step 22: TH sends ArmFailSafe command to the DUT with
ExpiryLengthSeconds set to 0"
verification: |
./chip-tool generalcommissioning arm-fail-safe 0 0 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
disabled: true

- label: "Step 21: TH readsNetworksattribute list fromthe DUT"
- label: "Step 23: TH reads Networks attribute list from the DUT"
PICS: CNET.S.A0001
verification: |
./chip-tool network-commissioning read networks 1 0
./chip-tool networkcommissioning read networks 1 0

The test case is not verifiable in RPI platform. As MaxNetworks value is 1 but expected is 4
( Pre-Condition)
Expand Down
Loading