Using a large number of workers can result in many tiny bloblogs being created for small operations #46

Open
opened 2021-07-02 20:40:00 +00:00 by reivilibre · 0 comments
Owner

e.g. using yama push.

Because we want a high level of concurrency here to hide latency from the network, this can result in e.g. 12 bloblogs less than a few kilobytes being created for a small push operation.

This could be cleaned up in a vacuum pass but it'd be nice to avoid that happening for small operations in the first place.

It's not sure how we could do that since a perfect solution involves predicting the future. But it might be possible to apply a heuristic or two.

e.g. using `yama push`. Because we want a high level of concurrency here to hide latency from the network, this can result in e.g. 12 bloblogs less than a few kilobytes being created for a small push operation. This *could* be cleaned up in a vacuum pass but it'd be nice to avoid that happening for small operations in the first place. It's not sure how we could do that since a perfect solution involves predicting the future. But it might be possible to apply a heuristic or two.
reivilibre changed title from Using a large number of workerscan result in many tiny bloblogs being created for small operations to Using a large number of workers can result in many tiny bloblogs being created for small operations 2021-07-02 20:40:07 +00:00
reivilibre added this to the Dogfood with Friends milestone 2021-09-02 22:02:46 +00:00
Sign in to join this conversation.
No Label
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: reivilibre/yama#46
No description provided.