Skip to content

perf: Remove unnecessary caches #5439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: develop
Choose a base branch
from
Open

Conversation

bthomee
Copy link
Collaborator

@bthomee bthomee commented May 20, 2025

High Level Overview of Change

NuDB and RocksDB internally already use caches, making the additional caches in the code not very valuable or even unnecessary. Moreover, some caches are defined but are not even used, which can also be removed.

Context of Change

The codebase contains many caches, and this PR is one of several to refactor the codebase with as goal to improve performance, such as reducing memory use, to support future account, trust line, and transaction growth.

This PR removes unnecessary caches in DatabaseNodeImp and SHAMapStoreImp; in the former we will instead rely on the caches built into the underlying NuDB or RocksDB stores, while the latter merely instantiated the caches but never used them. A performance analysis demonstrated that memory use reduced by 7%, CPU use increased by 2%, while all other metrics did not change significantly - a reasonable trade-off between memory and CPU. A more extensive performance analysis will be performed before this change is to be released.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (non-breaking change that only restructures code)
  • Performance (increase or change in throughput and/or latency)
  • Tests (you added tests for code that already exists, or your new feature included in this PR)
  • Documentation update
  • Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
  • Release

@bthomee bthomee added the Perf Attn Needed Attention needed from RippleX Performance Team label May 20, 2025
@bthomee bthomee marked this pull request as ready for review May 20, 2025 14:15
@bthomee bthomee requested a review from a team as a code owner May 20, 2025 14:15
@bthomee bthomee requested a review from vlntb May 20, 2025 14:42
Copy link

codecov bot commented May 20, 2025

Codecov Report

Attention: Patch coverage is 23.07692% with 20 lines in your changes missing coverage. Please review.

Project coverage is 78.6%. Comparing base (70371a4) to head (d96c416).

Files with missing lines Patch % Lines
src/xrpld/nodestore/detail/DatabaseNodeImp.cpp 20.0% 20 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           develop   #5439   +/-   ##
=======================================
  Coverage     78.5%   78.6%           
=======================================
  Files          813     813           
  Lines        70084   69999   -85     
  Branches      8283    8262   -21     
=======================================
- Hits         55031   54985   -46     
+ Misses       15053   15014   -39     
Files with missing lines Coverage Δ
src/xrpld/app/misc/SHAMapStoreImp.cpp 76.4% <100.0%> (-0.2%) ⬇️
src/xrpld/app/misc/SHAMapStoreImp.h 96.6% <ø> (ø)
src/xrpld/nodestore/detail/DatabaseNodeImp.h 73.9% <ø> (-5.0%) ⬇️
src/xrpld/nodestore/detail/DatabaseNodeImp.cpp 35.4% <20.0%> (+0.4%) ⬆️

... and 5 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@PeterChen13579
Copy link
Contributor

Just looping in @kuznetsss before approving, I don’t believe we do any node-level DB caching in Clio, right? So no changes should be needed on our side?

@kuznetsss
Copy link
Contributor

As I understand this is rippled's internal database. So no changes on Clio's side are required.

@bthomee bthomee added this to the 2.6.0 (Q3 2025) milestone May 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Perf Attn Needed Attention needed from RippleX Performance Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants