You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rust SQLX lib provides an iterator-like interface fetch(...) usually used with while let Some(row) = rows.try_next().await? {...} construct. In my case, each row may take some time to process, so I would like to use Rayon's par_iter(), but that requires a real iterator. Using fetch_all is not an option because all rows may simply not fit into memory.
How can I use Rayon's par_iter to process millions of rows produced by the sqlx's fetch stream?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Crossposting with stackoverflow:
Rust SQLX lib provides an iterator-like interface
fetch(...)
usually used withwhile let Some(row) = rows.try_next().await? {...}
construct. In my case, each row may take some time to process, so I would like to use Rayon'spar_iter()
, but that requires a real iterator. Usingfetch_all
is not an option because all rows may simply not fit into memory.How can I use Rayon's
par_iter
to process millions of rows produced by the sqlx'sfetch
stream?Beta Was this translation helpful? Give feedback.
All reactions