linux_AMD64 cmsearch memory requirements

Everything about the project RNA World
Nachricht
Autor
cisf
Mikrocruncher
Mikrocruncher
Beiträge: 20
Registriert: 07.03.2010 21:55

linux_AMD64 cmsearch memory requirements

#1 Ungelesener Beitrag von cisf » 07.03.2010 22:06

what are the memory requirements for cmsearch ?

in the F.A.Q it says : usually below 100 MB of RAM

but i notice it requires close to 2G ! (http://www.rnaworld.de/rnaworld/result. ... id=1290834)

also i see there are a few "Maximum memory exceeded" errors to other people.

my system starts to swap on such memory requiremnts, and so gets unusable for me, so do i need to check out cmsearch from my crunching options, or it is just an error ?

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8170
Registriert: 17.12.2002 14:09
Wohnort: Berlin

Re: linux_AMD64 cmsearch memory requirements

#2 Ungelesener Beitrag von yoyo » 07.03.2010 22:20

cisf hat geschrieben: also i see there are a few "Maximum memory exceeded" errors to other people.
Can you give an reference for this?
I do not see it in the results yet.
yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

cisf
Mikrocruncher
Mikrocruncher
Beiträge: 20
Registriert: 07.03.2010 21:55

Re: linux_AMD64 cmsearch memory requirements

#3 Ungelesener Beitrag von cisf » 07.03.2010 22:28

http://www.rnaworld.de/rnaworld/result. ... id=1290556

http://www.rnaworld.de/rnaworld/result. ... id=1289905 - this computer has about 5 tasks witch errord with memory related issues (to little memory perhaps, but shuldn't the aplicatin test if the computer can handle the jobs ?)

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8170
Registriert: 17.12.2002 14:09
Wohnort: Berlin

Re: linux_AMD64 cmsearch memory requirements

#4 Ungelesener Beitrag von yoyo » 07.03.2010 22:37

cisf hat geschrieben: http://www.rnaworld.de/rnaworld/result. ... id=1289905 - this computer has about 5 tasks witch errord with memory related issues (to little memory perhaps, but shuldn't the aplicatin test if the computer can handle the jobs ?)
This one is strange, because it doesn't start the wu, means not even the unzip at the beginning wasn't done. I think this is not an to less ram problem.
The other wus aborted because of disc usage. The wu used more disc space than allowed.
I assigned more disc space beginning with wuID=511915.
yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Ananas
WU-Schieber
WU-Schieber
Beiträge: 1184
Registriert: 27.04.2008 18:37
Wohnort: Nordlichter Köln

Re: linux_AMD64 cmsearch memory requirements

#5 Ungelesener Beitrag von Ananas » 07.03.2010 22:51

cisf hat geschrieben:http://www.rnaworld.de/rnaworld/result. ... id=1290556

http://www.rnaworld.de/rnaworld/result. ... id=1289905 - this computer has about 5 tasks witch errord with memory related issues (to little memory perhaps, but shuldn't the aplicatin test if the computer can handle the jobs ?)
@Yoyo : Some results on this host complain about RAM too

I wonder which setting it is - rsc_memory_bound in the workunit or ram_max_used_busy_pct / vm_max_used_pct in the global preferences.

The Athlon XP might not be allowed to use enough memory for those results - the BOINC default setting for Use at most xx% of memory when computer is in use and Use at most xx% of page file are quite low.


We already had the problem that the WUs went to hosts with insufficient physical RAM when the project has been closed Beta, I haven't seen a report anymore after Yoyo has increased the requirements though.


Especially with results that don't checkpoint (requiring "Leave applications in memory"), the BOINC strategy of pausing results when the free memory gets low is quite destructive. It cannot get any better when all the stuff has to be kept in memory.
vi BOINC/checkin_notes
:1,$s/bug/feature/g
:wq!

Erzaehlen sich Biologen eigentlich Klein-RNA-Witze?

Benutzeravatar
Michael H.W. Weber
Vereinsvorstand
Vereinsvorstand
Beiträge: 23174
Registriert: 07.01.2002 01:00
Wohnort: Marpurk

Re: linux_AMD64 cmsearch memory requirements

#6 Ungelesener Beitrag von Michael H.W. Weber » 07.03.2010 23:07

cisf hat geschrieben:what are the memory requirements for cmsearch ?
At present, there is no way to exactly determine the RAM requirements except for running the WU - although developments are underway to incorporate a RAM forecast function like the run time forecast we already have available. So it is very useful for us if you report a higher memory requirement than that what we measured as the current maximum with previous WUs (as noted in the FAQ). Please note that these on purpose are indeed very large CMS WUs and the one you have picked ranges amongst the most demanding ones.

Michael.
Fördern, kooperieren und konstruieren statt fordern, konkurrieren und konsumieren.

Bild Bild

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8170
Registriert: 17.12.2002 14:09
Wohnort: Berlin

Re: linux_AMD64 cmsearch memory requirements

#7 Ungelesener Beitrag von yoyo » 08.03.2010 06:14

I increased
rsc_memory_bound from 500MB -> 1GB
and
rsc_disk_bound from 100MB 1GB

yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Benutzeravatar
Michael H.W. Weber
Vereinsvorstand
Vereinsvorstand
Beiträge: 23174
Registriert: 07.01.2002 01:00
Wohnort: Marpurk

Re: linux_AMD64 cmsearch memory requirements

#8 Ungelesener Beitrag von Michael H.W. Weber » 08.03.2010 09:51

It seems that some people expercience RAM requirements of above 1.5 GB, so I think we need to refine --mxsize and increase the aforementioned boundaries even further for these complex CMS WUs.

Michael.
Fördern, kooperieren und konstruieren statt fordern, konkurrieren und konsumieren.

Bild Bild

Ananas
WU-Schieber
WU-Schieber
Beiträge: 1184
Registriert: 27.04.2008 18:37
Wohnort: Nordlichter Köln

Re: linux_AMD64 cmsearch memory requirements

#9 Ungelesener Beitrag von Ananas » 08.03.2010 09:59

The major problem with RAM requirements above 1GB is, that nearly all hosts that have quite a a lot of RAM are set to run several tasks. So even even with 4GB installed, a quadcore might run into a RAM shortage with 4 CMS results running.


p.s.: An important task on the todo list should be to catch the stderr output of cmsearch. As long as stderr always gets lost, it is hard to figure out what caused the program to abort the calculation (if it is a program controlled exit and not a crash).
vi BOINC/checkin_notes
:1,$s/bug/feature/g
:wq!

Erzaehlen sich Biologen eigentlich Klein-RNA-Witze?

Zurück zu „RNA World Discussions (english)“