Massive cmcsearch RAM requirements with some WUs

[nenym]

I have got super cms task cms_6S7[e]_Monodelphis-domestica-(gray-short-tailed-opossum)_CM000375.lin.EMBL_f_1268067422_0_3 http://www.rnaworld.de/rnaworld/result. ... id=1296446. Top memory load of that task was 7GB RAM. After 9hours crunching core switched to other application, after restarting search task it started from beginning, so I have killed it.
I mean it is not good application for Boinc if needs 7GB RAM for one task and for 10 hour task (4CPU Xeon 3,0 GHz, 8 GB RAM) has no checkpointing. If memory load were stable, it would be horrible, but acceptable. If memory load is between 0,5G - 7G RAM, it is not acceptable.

[Michael H.W. Weber]

I mean it is not good application for Boinc if needs 7GB RAM for one task and for 10 hour task (4CPU Xeon 3,0 GHz, 8 GB RAM) has no checkpointing. If memory load were stable, it would be horrible, but acceptable. If memory load is between 0,5G - 7G RAM, it is not acceptable.

It is acceptable for BOINC provided such WUs can be properly addressed to the appropriately equipped machines. And that is what we currently have to further optimize.

Michael.

[nenym]

I am pleased by the way you are doing RNA user friendly. I hope the RNA tasks will not need baby-sitting at short time.

I little reflection from my point of view.

Let us take as example machine ID 2653 - 4CPU Xeon 2.83GHz, 8GB RAM, 1x GPU GTX 260.

1. Simple extreme
Running only RNA + Seti CUDA.
- 4x supertask RNA as that machine tried to crunch = 4x 7GB RAM top memory load,
- 1 Seti CUDA = 0,1 GB RAM.
28GB RAM > 8GB RAM.

2. My typical
Running 25 FreeHal tasks, 2 yoyo_ecm_mp tasks, 2 RNA tasks, 1x Seti CUDA task.
- when FreeHal tasks starts (the first 10 minutes), top memory load of one task is 85 MB RAM, but top is short (3 minutes) and tasks doesn't start at the same time. Top memory load use about 1/3 tasks at the same time = 0,7 GB RAM,
- yoyo_ecm_task takes typically 300M RAM, top memory load (5x during 2,5 hours for 5 minutes) is 1,5 GB RAM = 3GB RAM,
- 1 Seti CUDA = 0,1 GB RAM,
At that moment is used cca 4GB RAM by Boinc for other apps than RNA. System takes about 0,5 GB RAM and for RNA is less than 3,5 GB RAM free . From my point of wiev, it is enough only for "short" tasks < 3 hours, not for 10 hours tasks.

I am glad than project staff is communicating with crunchers. Nice, thanks.

EDIT: grammar

[nico]

What are your preferences for "Use at most xx of memory when computer is not in use"?
Unfortunately we cannot determine the amount of memory needed by one WU in advance at the moment.
But we are looking for possible solutions.

To prevent the restarting of WUs set "Leave applications in memory while suspended" in your Boinc manager preferences - this however will be no solution for big WUs.... we are working on it.

[nenym]

In use 85%, 90% when idle (ID 2653 - workstation running AutoCAD); 95/95 (ID 2654 - it is server station). Both machines have 8GB RAM and 2GB swap file. Leave in memory while suspended - not allowed as I have about 5 - 6 active projects and Boinc core is switching on checkpoints and time to time running panic mode (when long task with short deadline from exotic project is received - e.g. LHC, or tasks with extremely short deadline - Almere, Virtual Prairie benchmark tasks, Pirates, Wanless2, Mopac...).

[Matthias Lehmkuhl]

First: the good thing, you are using a Win XP 64 Bit, and so there are not the memory limits that are the problem for running this results on a Win 32 Bit. No chance to run this result on a 32 Bit computer.

The second: I learned, that under Windows the maximum pagefile/swapfile size at least should be 1.5 x-times of the installed RAM when you are using manual settings. In your case it should be 12 GB.

[nenym]

The second: I learned, that under Windows the maximum pagefile/swapfile size at least should be 1.5 x-times of the installed RAM when you are using manual settings. In your case it should be 12 GB.

I had had no swap file before started to parcitipate on DC (no problem occured), then I have created a little one "to have no problem". Thanks for reminder, I know that rule and WinXP x_86 systems I have set by that way. Going to set it for XP x_64 now. Hoever it is an emergency solution as crunching using the swap file makes machines laggy and unusable.

[Christopher Herr]

Hello everyone,

i have also a problem with a cmsearch task on my new rig (Intel i7 880@3.07 GHz, 12 GB RAM, but you can look for yourself at the task details):

Problem task with enormous RAM demands

It has been running for four seconds and then waiting for memory the last three hours and the task properties say, it wants to hog 10.11 GB of virtual memory and the task package has 10.15GB! With a maximum setting of 8 GB for BOINC the latter simply is impossible, isn't it?

Now with a swap file of approximately 18GB i shouldn' have problems to accommodate those extreme requirements, methinks.
But why the heck is it waiting for memory, when the average memory load is just about 17 percent?!

I recently switched my heaviest project weight over to RNAWorld to seriously crunch some WUs here.

Am I supposed to cancel this WU, is this a known problem of cmsearch 0.14 and is there any other advice?

Cheers and thx for any advice,
Christopher

[Norman]

normally such a short WU is under 100MB memory.
strange !

[Christopher Herr]

normally such a short WU is under 100MB memory.
strange !

Well, it isn't and in the meantime, I tried to stop the task, quit and restart BOINC and restart the task again with little or no success...
Hence I made a screenshot of the task properties dialog:

[Michael H.W. Weber]

Well, it would surprise me if that WU would take that much RAM (in fact, I would expect something in the order of 100 MB). I think there is a problem with you system. Please restart your machine to put everything back to order. The problem is that we cannot make a RAM forecast on CMSEARCH work units to inspect this more in detail but the RAM requirements correlate quite well with the run time and I think that task does not have an extraordinarily long run time.

Michael.

[Christopher Herr]

Well, it would surprise me if that WU would take that much RAM (in fact, I would expect something in the order of 100 MB). I think there is a problem with you system. Please restart your machine to put everything back to order. The problem is that we cannot make a RAM forecast on CMSEARCH work units to inspect this more in detail but the RAM requirements correlate quite well with the run time and I think that task does not have an extraordinarily long run time.

Michael.

Well, it seems to. I have restarted that rig once now, but I have to do it another time, because I deinstalled a system software just now. If that issue isn't resolved by then and it isn't by now, I regret, I will have to cancel it. No sense in me keeping this task until its deadline passed and I have to cancel it anyway. Perhaps somebody else has more luck and can crunch it successfully.

Cheers and Thx Michael!
Christopher