Thresholds for new cmsearch tasks needed

[ChristianB]

Hello,

there is a new Batch of tasks for cmsearch in preparation. This Batch contains a lot of very short cmsearch calls with only three calls that are estimated with more than 12 hours. We restructured our Job Generator and can now assign tasks to the corresponding app (S, XXL, VM) according to the forecast. We would like to get some feedback on the thresholds to choose for each application. This is my suggestion I want to discuss:
cmsearch S: total task runtime up to 12 hours but a single call not more than 4 hours (this would give us a pseudo-checkpointing)
cmsearch XXL: more than 4 but less than 48 hours estimated runtime (only one call, no checkpointing)
cmsearch VM: everything above 48 hours runtime (checkpointing via VM)
The speciality with S is that we can bundle several cmsearch calls into one task until 1000 calls are reached or the estimated runtime is 12 hours.

The loss of computation for S and XXL is manageable in my opinion. The thresholds are configurable and are to be discussed. Do you want shorter S or longer XXL or ...?

The German thread already mentioned that we should rename XXL to L as the real XXL tasks are now taken care of by VM. Additionally there are two users who want a single call within S to be not more than 1 to 2 hours.

[Jacob Klein]

I like to think of things in terms of "what might be lost" when BOINC is restarted.
For the non-VM applications, it sounds like you lose all the work on the "current call", while retaining completed work on the "previously completed calls".
For the VM application, I believe we lose "up to the last 10 minutes" of work that was not saved within the last snapshot.

As for structure, I'd personally want "completed data to be reported" in a frequency as low as "20 minutes". ie: If a project could create tasks of 4-hour-duration, 2-hour-duration, 1-hour-duration, or 20-minute-duration... I'd prefer 20-minute-duration.

I like your proposal:
S: multiple <4hr calls, up to 12-hour total
XXL: 1 call, <48-hour total
VM: 48+hr tasks

But I might like this one better, if it is doable:
S: 1 call, 20min-4hr total
L: 1 call, 4hr-48hr total
VM: Begin with 48+hr tasks, but when those tasks run out, assign additional shorter-length tasks

I require checkpointing, because I absolutely detest the idea of losing progress. Thus, I will probably only do VM tasks, and would request that even "shorter than 48-hr" tasks be put on the VM queue when the big 48+hr tasks are completed/assigned.

Regards,
Jacob

[ChristianB]

But I might like this one better, if it is doable:
S: 1 call, 20min-4hr total
L: 1 call, 4hr-48hr total
VM: Begin with 48+hr tasks, but when those tasks run out, assign additional shorter-length tasks

The current batch contains a lot of calls with a runtime <20min. Where whould they fit in your proposal? We could solve this by doing it like this:
S: multiple <0.5hr calls, up to 12hr total
XXL: 1 call, 0.5hr-48hr total
VM: 48+hr tasks

We would still have pseudo checkpointing with S and a loss of the last 30 min at most. What we don't want is a lot of very short tasks because this means a lot of traffic and load on the server. The S tasks should at least run 4 to 8 hours with multiple calls.

[Jacob Klein]

I see. So, you know up front that several of the calls will be <20min. And I understand the request to not overload with server traffic. You want S to last 4-8 hours? Okay, let's try 4 hours.

So, I now propose:
S: Multiple <4hr calls, up to 4 hr total (so, could be 12 20-min calls, or 1 4-hr call, so long as not to exceed 4 hr total)
L: 1 call, 4hr-48hr
VM: 48+hr tasks

[Rebelion]

Hello. Wanna ask - why did I get the task today, which needs 4046 hours?
It's not possible to calculate this task till 31.01.2014...

[Jacob Klein]

Rebelion,
VM tasks will automatically extend their deadlines on the server, as you make progress on them.