Since the process_csv scripts are effectively making SQL queries and then calling subprocess commands, I would like to explore re-writing them in Rust to improve serialization performance of the data (degauss operates through JSON), more efficient parallelization, and binary portability. I can take ownership of these changes and revisions. Not saying we should pursue this path but wanting to open a discussion.
Since the
process_csvscripts are effectively making SQL queries and then calling subprocess commands, I would like to explore re-writing them in Rust to improve serialization performance of the data (degauss operates through JSON), more efficient parallelization, and binary portability. I can take ownership of these changes and revisions. Not saying we should pursue this path but wanting to open a discussion.