feat: make AsyncAws batch size configurable
feat: make AsyncAws batch size configurable: a task in bird-critic-1.5-full-300 (Harbor dataset). When files are written to S3 in bulk, the AsyncAws write-batch adapter copies them in fixed-size batches so that it does not hold too many sockets open at once and hit "Too many open files". That…
The task
When files are written to S3 in bulk, the AsyncAws write-batch adapter copies them in fixed-size batches so that it does not hold too many sockets open at once and hit "Too many open files". That batch size is currently hardcoded, which we believe causes trouble on cloud deploys where the practical socket limit…
Part of kochsnow/bird-critic-1.5-full-300.