Currently the backend supports only .csv file export. That is because, as it turns out, it is not possible to split .xlsx files into chunks, which then can be sent to the user using generator and streams.
If such method of splitting exists, a librarly called XlsxWriter could be used to write into a file without eating up entire memory for very large files, as per documentation:
https://xlsxwriter.readthedocs.io/working_with_memory.html (especially the part about constant-memory mode)
https://flask.palletsprojects.com/en/2.3.x/patterns/streaming/
It is not something that is a must, as .csv files can be imported into Excel just fine, but it is worth considering for the future.
As for the detailed reason why this is impossible to achieve, I'll copy and paste ChatGPT's answer, because why not:
Unfortunately, streaming XLSX files in chunks directly from an Excel file isn't as straightforward as it is with plain text formats like CSV. The XLSX format is a ZIP archive of XML files, and the structure of an Excel file relies on having the correct relationships and dependencies, making it challenging to stream incrementally.
However, you can consider a workaround where you generate the XLSX file in chunks and then stream each chunk individually. This involves creating multiple XLSX files, each containing a portion of the data, and then sending them one by one. The client can then concatenate the received files.
Keep in mind that this is a workaround, and the client will need to combine the received chunks to reconstruct the complete Excel file. Additionally, you may need to adjust the delay and chunking logic based on your specific use case and requirements.
Currently the backend supports only .csv file export. That is because, as it turns out, it is not possible to split .xlsx files into chunks, which then can be sent to the user using generator and streams.
If such method of splitting exists, a librarly called
XlsxWritercould be used to write into a file without eating up entire memory for very large files, as per documentation:https://xlsxwriter.readthedocs.io/working_with_memory.html (especially the part about
constant-memorymode)https://flask.palletsprojects.com/en/2.3.x/patterns/streaming/
It is not something that is a must, as .csv files can be imported into Excel just fine, but it is worth considering for the future.
As for the detailed reason why this is impossible to achieve, I'll copy and paste ChatGPT's answer, because why not: