Dear Josh,
I recognize that we are currently using the crate-python driver together with the BatchSizeAutomator in order to balance throughput saturation vs. overload situations.
https://github.com/crate/ts-data-generator/blob/9c64608d2cd9618e9d3b8d9be387c199b1fe4a4e/requirements.txt#L3
https://github.com/crate/ts-data-generator/blob/67330181e262d069f09e27bc64eb18951384b2f4/src/data_generator/data_generator.py#L60-L62
On this matter, I would also like to point out asyncpg's upcoming MagicStack/asyncpg#295, which improves executemany() to batch args for performance. It will probably be available with asyncpg>=0.22.0. Thanks to @mfussenegger for sharing this.
With kind regards,
Andreas.
Dear Josh,
I recognize that we are currently using the
crate-pythondriver together with theBatchSizeAutomatorin order to balance throughput saturation vs. overload situations.https://github.com/crate/ts-data-generator/blob/9c64608d2cd9618e9d3b8d9be387c199b1fe4a4e/requirements.txt#L3
https://github.com/crate/ts-data-generator/blob/67330181e262d069f09e27bc64eb18951384b2f4/src/data_generator/data_generator.py#L60-L62
On this matter, I would also like to point out
asyncpg's upcoming MagicStack/asyncpg#295, which improvesexecutemany()to batch args for performance. It will probably be available withasyncpg>=0.22.0. Thanks to @mfussenegger for sharing this.With kind regards,
Andreas.