-
Notifications
You must be signed in to change notification settings - Fork 16
Improve fungible token coverage #113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
name: "test coverage" | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
paths-ignore: | ||
- "**.md" | ||
pull_request: | ||
paths-ignore: | ||
- "**.md" | ||
workflow_dispatch: | ||
|
||
# If new code is pushed to a PR branch, then cancel in progress workflows for | ||
# that PR. Ensures that we don't waste CI time, and returns results quicker. | ||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
cancel-in-progress: true | ||
|
||
env: | ||
# Not needed in CI, should make things a bit faster | ||
CARGO_INCREMENTAL: 0 | ||
CARGO_TERM_COLOR: always | ||
# Remove unnecessary WASM build artifacts | ||
WASM_BUILD_CLEAN_TARGET: 1 | ||
# stripping symbols and optimizing for binary size | ||
RUSTFLAGS: -C strip=symbols -C opt-level=s | ||
# Enable sscache | ||
RUSTC_WRAPPER: "sccache" | ||
SCCACHE_GHA_ENABLED: "true" | ||
SCCACHE_CACHE_SIZE: "50GB" | ||
|
||
jobs: | ||
test-coverage: | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
os: | ||
- ubuntu-latest | ||
|
||
runs-on: ${{ matrix.os }} | ||
steps: | ||
- name: git checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Run sccache | ||
uses: mozilla-actions/sccache-action@v0.0.4 | ||
|
||
- name: install rust toolchain | ||
uses: actions-rust-lang/setup-rust-toolchain@v1 | ||
with: | ||
toolchain: stable | ||
components: llvm-tools-preview | ||
target: wasm32-unknown-unknown | ||
|
||
- name: Install cargo-llvm-cov | ||
uses: taiki-e/install-action@v2 | ||
with: | ||
tool: cargo-llvm-cov | ||
|
||
- name: Generate code coverage report | ||
run: | | ||
cargo llvm-cov --workspace --lcov --output-path lcov.info --ignore-filename-regex "test\.rs" --fail-under-lines 90 | ||
|
||
- name: Generate detailed text coverage report | ||
run: | | ||
cargo llvm-cov --text --ignore-filename-regex "test\.rs" --package stellar-fungible > coverage-report.txt | ||
echo "Coverage report generated at coverage-report.txt" | ||
|
||
# Extract branch coverage and verify threshold | ||
if grep -q "TOTAL.*branches" coverage-report.txt; then | ||
BRANCH_COVERAGE=$(grep "TOTAL.*branches" coverage-report.txt | awk '{print $NF}' | sed 's/%//') | ||
echo "Branch coverage: $BRANCH_COVERAGE%" | ||
if (( $(echo "$BRANCH_COVERAGE < 85" | bc -l) )); then | ||
echo "::error::Branch coverage $BRANCH_COVERAGE% is below the target of 85%" | ||
echo "Uncovered branches:" | ||
cat coverage-report.txt | grep -B 1 "\^" || echo "No specific uncovered branches found!" | ||
exit 1 | ||
else | ||
echo "Branch coverage meets target: $BRANCH_COVERAGE% ≥ 85%" | ||
fi | ||
else | ||
echo "Branch coverage information not available in this version of cargo-llvm-cov" | ||
BRANCH_COVERAGE="Not available" | ||
fi | ||
|
||
# Create a job summary with coverage stats | ||
echo "## Coverage Summary" >> $GITHUB_STEP_SUMMARY | ||
echo "📊 **Coverage Report for stellar-fungible**" >> $GITHUB_STEP_SUMMARY | ||
echo "" >> $GITHUB_STEP_SUMMARY | ||
LINE_COVERAGE=$(grep "TOTAL.*lines" coverage-report.txt | awk '{print $NF}') | ||
echo "- Line coverage: $LINE_COVERAGE" >> $GITHUB_STEP_SUMMARY | ||
echo "- Branch coverage: $BRANCH_COVERAGE%" >> $GITHUB_STEP_SUMMARY | ||
echo "" >> $GITHUB_STEP_SUMMARY | ||
echo "Files with uncovered lines:" >> $GITHUB_STEP_SUMMARY | ||
echo '```' >> $GITHUB_STEP_SUMMARY | ||
grep -B 1 "\^" coverage-report.txt | grep -v -- "^--$" | head -20 >> $GITHUB_STEP_SUMMARY || echo "No uncovered lines found!" >> $GITHUB_STEP_SUMMARY | ||
echo '```' >> $GITHUB_STEP_SUMMARY | ||
|
||
- name: Upload coverage report artifact | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: coverage-report | ||
path: | | ||
coverage-report.txt | ||
lcov.info | ||
retention-days: 14 | ||
|
||
- name: Upload coverage report to Codecov | ||
# Only run if the Codecov token is configured | ||
if: ${{ secrets.CODECOV_TOKEN != '' }} | ||
continue-on-error: true # Don't fail the workflow if Codecov upload fails | ||
uses: codecov/codecov-action@v3 | ||
with: | ||
token: ${{ secrets.CODECOV_TOKEN }} | ||
files: lcov.info | ||
fail_ci_if_error: false # Don't fail even if the Codecov upload fails | ||
Comment on lines
+109
to
+117
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I've added an optional Codecov integration that only runs if you have the CODECOV_TOKEN secret set up. It gives nicer visualizations and keeps history longer than GitHub's 14-day artifact storage. (Totally optional though - the core coverage validation works fine without it.) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. tsk tsk the workflow infact did not work fine without it. still working on this one! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hey @tinkerer-shubh
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
# Code Coverage Strategy | ||
|
||
This document outlines our approach to test coverage for the `stellar-fungible` package, addressing GitHub Issue [#96](https://github.com/OpenZeppelin/stellar-contracts/issues/96). | ||
|
||
## Coverage Goals | ||
|
||
- **Target**: Aim for 100% line coverage where practical | ||
- **Minimum Threshold**: 90% line coverage enforced in CI | ||
|
||
## Currently Uncovered Areas | ||
|
||
### 1. Generated Code | ||
|
||
The following code is generated by macros and is difficult to directly test: | ||
|
||
- `#[contracttype]` definitions in `storage.rs` (lines 7-27) | ||
- `#[contracttype]` for `Metadata` struct in `metadata/storage.rs` (line 9) | ||
- `#[contractclient]` for trait definitions in `fungible.rs` and `mintable/mod.rs` | ||
|
||
**Justification**: Generated code is compiled and validated by the Rust compiler. Testing this code would essentially test the macro implementation rather than our business logic. | ||
|
||
### 2. Error Paths in Extensions | ||
|
||
Some error handling paths have partial coverage: | ||
|
||
- `query_cap` error path in `capped/storage.rs` (line 45) | ||
- `check_cap` error path in `capped/storage.rs` (line 65) | ||
|
||
**Strategy**: These represent panic paths that are challenging to test because they occur in unwrap_or_else closures. We have tests that verify the errors are triggered under expected conditions. | ||
|
||
### 3. TTL Extension Logic | ||
|
||
Specific branches in TTL extension logic have partial coverage: | ||
|
||
- `set_allowance` conditional branch in `storage.rs` (line 180) | ||
- `extend_ttl` in `storage.rs` (line 197-198) | ||
|
||
**Improvement Plan**: We have added a new test `test_set_allowance_positive_amount_with_expired_ledger` to specifically target one of these branches. | ||
|
||
## Future Improvement Opportunities | ||
|
||
1. **Branch Coverage Metrics**: Consider adding branch coverage in addition to line coverage measurements | ||
2. **Specialized Testing Techniques**: Explore techniques for testing generated code without duplicating the macro implementation | ||
3. **Documentation Updates**: Update this document as coverage improves or new uncoverable areas are identified | ||
|
||
## Why 100% Line Coverage is Challenging | ||
|
||
100% line coverage is an aspirational goal that may not be practically achievable due to: | ||
|
||
1. The nature of Rust's macro system | ||
2. The design of traits and interfaces | ||
3. Some error paths that require complex setup to trigger | ||
|
||
We believe our current approach balances thorough testing with practical considerations, ensuring high quality while acknowledging the limitations of coverage metrics. | ||
|
||
## Changelog of Coverage Improvements | ||
|
||
### v0.1.0 - Coverage Enhancement Initiative | ||
|
||
This changelog documents the improvements made to increase code coverage as part of issue #96. | ||
|
||
#### New Tests Added | ||
|
||
1. `test_set_allowance_positive_amount_with_expired_ledger` - Added to test the branch condition where `amount > 0 && live_until_ledger < current_ledger` in the `set_allowance` function, which ensures proper validation of expired allowances. | ||
|
||
2. `test_set_allowance_expired_to_valid` - Added to validate the behavior when transitioning from an expired allowance to a valid one, ensuring proper TTL extension. | ||
|
||
3. `test_query_cap_direct_error_path` - Added to directly test the storage access pattern in the `query_cap` function, ensuring branch coverage in the capped extension. | ||
|
||
#### CI Improvements | ||
|
||
1. Added branch coverage reporting to the GitHub Actions workflow. | ||
2. Enhanced the workflow with a summary report that highlights: | ||
- Line coverage percentage | ||
- Files with uncovered lines | ||
- Clear pass/fail indicators | ||
|
||
3. Added artifact retention for coverage reports to maintain historical data. | ||
|
||
#### Documentation | ||
|
||
Created this `COVERAGE.md` file to document: | ||
- Coverage strategy | ||
- Uncovered areas with justifications | ||
- Future improvement opportunities | ||
|
||
#### Results | ||
|
||
The coverage report indicates that we have successfully: | ||
- Achieved >90% line coverage across the codebase | ||
- Improved branch coverage in critical areas like allowance management and extensions | ||
- Reduced uncovered code in error paths | ||
|
||
Some areas remain uncovered (documented in the "Uncovered Areas" section), but these are primarily in generated code or extreme edge cases that would require specialized testing techniques. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added coverage.md mainly to help reviewers understand the reasoning behind the test coverage choices in this PR. I think we can safely remove this file in the end before the merge - it's just helpful context for the review process. Also, I'd love feedback on how I’ve set up the test placements in the project structure—just checking to see if they feel right with the rest of it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Heads up about how this implementation works with cargo-llvm-cov's text reports:
This works fine now, but if cargo-llvm-cov changes its output format, our parsing might break. We should probably:
For now though, this approach keeps things simple while getting the job done.
(OR we can just keep the line coverage for now and add branch coverage when it is natively supported in
cargo-llvm-cov
)