ecm resource exceptions

[JohnMD]

https://www.rechenkraft.net/yoyo/workun ... d=75156706
This ordinary ecm task is currently using about 2½ GB RAM, which is well over the normal max of 1 GB.

https://www.rechenkraft.net/yoyo/workun ... d=75164906
This ecm P2 task will run for about 2 weeks which is well over the normal limit of 5 days.


I expect you will receive crashes and/or cancellations for current ecm WU's which is a waste of everyone's time - especially yours.
I expect better things from Rechenkraft

[yoyo]

Hi John,
I do not expect many crashes.

The first RU workunit has memory requirements of 4 GB RAM. Since a long time already, ecm workunits have RAM requirements up to 4 GB.

For your second CN workunit, I got already valid results back. The runtimes were between 9 and 24 hours. Seams that estimation of your Boinc is wrong.

kind regards,
yoyo

[JohnMD]

Thanks Yoyo for your explanation.

I had not seen that ordinary ecm tasks had escalated from 1GB to 4GB. That is a big jump for my hard-wired 8GB machines !

The ecm P2 WU estimate is based on actual progress which is now more than 15 days (BOINC estimat - 1 week). This is not the first time for me, but I've also had "sensible" P2 WU's. This cpu is running at a very modest 1 GHz - but there must be significant waiting time when several GB ram need access. Do you recommend I let it run to completion ?

[yoyo]

David Anderson claims, that RAM requirements of workunits are properly handled by BOINC.If a workunit requires 4GB RAM, it is only downloaded or started if 4 GB RAM is free. But in some cases this seems not to work properly. I have the impression it doesn't work, especially when many workunits are started at the same time and at the same time all allocate much RAM.

Regarding the P2, they do not have a progress indicator. So what BOINC shows is some BOINC magic. The only thing I can tell is that the ecm cn P2 is estimated to need 1.13722159465142e15 floating point operations. Based on the FLOPS measurement of your computer it would need 15 days. And it has no checkpoints.

[Michael H.W. Weber]

To my experience, BOINC indeed can't handle the RAM requirements properly. It seems that it just checks the free RAM at the time of WU startup, not taking into account that some projects don't have fixed RAM per WU requirements but may change RAM comsumption dynamically. One such example is RNA World. Moreover, it appears that starting multiple tasks at the same time may also fail in proper RAM allocation leading to swapping and unresponsive systems. I solve this issue by clearly limiting the number of tasks for such projects per CPU.

Regarding non-checkpointed long running tasks: 15 days may indeed sound challenging but there are hundreds of successfully completed non-checkpointed BOINC tasks reported for RNA World with total runtime requirements of way above 100 days - a BOINC community performance rarely matched even by most high performance compute clusters.

Michael.

[JohnMD]

Thanks everyone for your input :
The ecm P2 task prophesied for at least 2 weeks suddenly finished after about 2½ days. the task document said nothing.
Knowing the old ecm requirements, my machines are restricted to just one (1) occurrence. My 2 older (W10) machines with slots have been extended to process ecm P2 with up to 10GB.
It is one thing for BOINC to give wild estimates.
It is completely different when the application itself supplies incorrect numbers of progress.

[gemini8]

The ecm P2 task prophesied for at least 2 weeks suddenly finished after about 2½ days. the task document said nothing.

The only thing I can tell is that the ecm cn P2 is estimated to need 1.13722159465142e15 floating point operations. Based on the FLOPS measurement of your computer it would need 15 days.

This is the info yoyo provided in his most recent posting.
As BOINC clearly tests some irrelevant stuff when doing its benchmarks (newer instructions aren't tested) the number could only be a worst estimate.

It is one thing for BOINC to give wild estimates.
It is completely different when the application itself supplies incorrect numbers of progress.

Regarding the P2, they do not have a progress indicator. So what BOINC shows is some BOINC magic.

This coupled with the benchmark issues is what lets things look disturbing.
AFAIK yoyo takes other people's applications and lets them run in BOINC through the BOINC wrapper. I know he hasn't access to the source code of every application, so it could or couldn't be he'd be able to help the situation by compiling them himself to include checkpoints.