Simplify the implementation of run_tv_casting_test.py for easier addition of future test sequences that verifies that Linux tv-casting-app continues to work with Linux tv-app.
#33855
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pullapprove
bot
requested review from
andy31415,
andyg-apple,
anush-apple,
arkq,
bauerschwan,
bzbarsky-apple,
carol-apple,
cecille,
chrisdecenzo,
chshu,
chulspro,
cliffamzn,
Damian-Nordic,
dhrishi,
fessehaeve,
harimau-qirex,
harsha-rajendran,
hawk248,
hicklin,
jepenven-silabs,
jmartinez-silabs,
jmeg-sfy,
joonhaengHeo,
jtung-apple,
kiel-apple,
kkasperczyk-no,
kpschoedel,
ksperling-apple and
lazarkov
|
PR #33855: Size comparison from 3058c4b to 46871fc Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…dition of future test sequences that verifies that Linux tv-casting-app continues to work with Linux tv-app.
shaoltan-amazon
force-pushed
the
Linux-test-casting-simplified
branch
from
46871fc to
2de194a
Compare
|
PR #33855: Size comparison from bfa3e6f to 2de194a Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
shaoltan-amazon
force-pushed
the
Linux-test-casting-simplified
branch
from
3bee6d7 to
8ebfb19
Compare
|
PR #33855: Size comparison from b05165e to 8ebfb19 Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
…prevent circular import issues.
shaoltan-amazon
force-pushed
the
Linux-test-casting-simplified
branch
from
b88cdf1 to
6d66186
Compare
sharadb-amazon
approved these changes
Jun 18, 2024
|
PR #33855: Size comparison from a39c62e to 6d66186 Full report (8 builds for cc32xx, mbed, qpg, stm32, tizen)
|
|
PR #33855: Size comparison from a39c62e to f8da857 Full report (85 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, mbed, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
|
chrisdecenzo
approved these changes
Jun 18, 2024
hawk248
approved these changes
Jun 20, 2024
austina-csa
pushed a commit
to austina-csa/connectedhomeip
that referenced
this pull request
Jul 10, 2024
…dition of future test sequences that verifies that Linux tv-casting-app continues to work with Linux tv-app. (project-chip#33855) * Simplify the implementation of `run_tv_casting_test.py` for easier addition of future test sequences that verifies that Linux tv-casting-app continues to work with Linux tv-app. * Addressed PR comments from @sharadb-amazon. * Addressed @sharadb-amazon PR comments. Also restructured the code to prevent circular import issues. * Run restyle. * Fixed typo.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
There is currently a CI check implementation that verifies that the Linux tv-casting-app continues to work with the Linux tv-app whenever a PR is created. However, adding new test sequences is not that simple as we would need to keep defining new functions for parsing output messages of interest and/or sending input commands. Therefore, we want to improve the design flow to make it easier to add new test cases in the future.
Solution
The new design will allow users to define a new test
Sequenceconsisting of a test sequence name and a list ofStepobjects. EachStepcontains the relevant information needed to either parse for output string(s) of interest or send input command. Essentially, to add a new flow, we need to define a test sequence list ofStepobjects containing:subprocessto parse for output_msg or send input_cmdtimeout_secspecified the timeout duration for parsing theoutput_msg(optional, defaults to DEFAULT_TIMEOUT_SEC)output_msgorinput_cmd(mutually exclusive)Note that there could be multiple instances of a particular output message block so we would need to parse them block-by-block. For this reason, we would need to define the start line, relevant lines we are looking for within the message block, and the last line. If the last line contains trivial closing characters (e.g., closing brackets, braces, or commas), we also need to include the line before it with actual log content. For example:
Step(subprocess_='tv-casting-app', output_msg=['InvokeResponseMessage =', 'exampleData', 'InteractionModelRevision =', '},'])For input commands, we need to define the command as a string followed by
\n. For example:Step(subprocess_='tv-casting-app', input_cmd='cast request 0\n')In short, this PR aims to improve the existing test script implementation to follow a new design flow where user can provide test sequence(s) consisting of test steps of output strings or input commands that the test should validate for.
Testing
Tested the
run_tv_casting_test.pyscript locally to ensure that it still works as expected.