Mem use . Monstruous

Fehler und Wünsche zum Projekt yoyo@home
Bugs and wishes for the project yoyo@home
Nachricht
Autor
marsinph
PDA-Benutzer
PDA-Benutzer
Beiträge: 31
Registriert: 05.04.2018 09:08

Mem use . Monstruous

#1 Ungelesener Beitrag von marsinph » 15.05.2019 22:04

Hello,
I think I have not a bad config : i7-3930K 12 cores with two RX580 and 32Gb DDR3 under Win7 x64.
The problem : ECM_UC......P2_x takes after a few minutes about 7Gb RAM ! Yes, seven giga RAM by WU !!!
So unable ti run more than 3 WU (with 24RAM.)

They start with about 100Mb, then increase to 200Mb. And after about five minuts increase to 7Gb !
Result : computer unable to manage.
Running time : about 1 hour. No problem.
The biggest problem is the deadline : three days, PRJ send WU according running/cores.
Because only tthree WU can run togheter, impossible to return the WU inside the deadline.
On a 8Gb RAM : only one WU can run. But because deadline andrunning time, also considering boinc "accept task for one day"

Project send 12 (cores) x24 (1hour/wu/day) = 296WU.
But because the monstruous required RAM, only 48 can be crunched (with manual control)

No other choice that to always cancel. All WU will be too late

Perhaps I do something wrong ? Then explain !

Or please, admin , reconsider your WU for normal users config !!!

See :

Application
ecm 705.02
Name
ecm_uc_1557857447_np_157_2900e6_02_206_P2
State
Running
Received
14/05/2019 22:43:41
Report deadline
19/05/2019 22:43:42
Estimated computation size
19,530 GFLOPs
CPU time
00:18:57
CPU time since checkpoint
00:18:57
Elapsed time
00:19:00
Estimated time remaining
00:48:07
Fraction done
22.893%
Virtual memory size
7.06 GB
Working set size
7.24 GB

Directory
slots/3
Process ID
6600
Progress rate
72.360% per hour
Executable
ecmwrapper_705.02_windows_x86_64.exe

Benutzeravatar
Michael H.W. Weber
Vereinsvorstand
Vereinsvorstand
Beiträge: 22419
Registriert: 07.01.2002 01:00
Wohnort: Marpurk
Kontaktdaten:

Re: Mem use . Monstruous

#2 Ungelesener Beitrag von Michael H.W. Weber » 16.05.2019 09:55

If you adjust your settings to work on only 3 ECM tasks simultaneously per machine you should be safe with your 32 GB of RAM. You can do this by placing a plain text file called app_config.xml in the Yoyo@home projects directory on your machine(s):

Code: Alles auswählen

<app_config>
  <app>
    <name>ecm</name>
    <max_concurrent>3</max_concurrent>
    <report_results_immediately/>
  </app>
</app_config>
There are at least three types of ECM tasks. The P2 ones (check file name) require up to 10 GB of RAM. These are the "trouble makers".
The tasks with P1 require very little RAM, all others may require up to 1.8 GB of RAM.

You may also download work for a couple of days, check the file names, delete all of the P2 and just run Yoyo@home without any config file.

Michael.
Fördern, kooperieren und konstruieren statt fordern, konkurrieren und konsumieren.

http://signature.statseb.fr I: Kaputte Seite A
http://signature.statseb.fr II: Kaputte Seite B

Bild Bild Bild

marsinph
PDA-Benutzer
PDA-Benutzer
Beiträge: 31
Registriert: 05.04.2018 09:08

Re: Mem use . Monstruous

#3 Ungelesener Beitrag von marsinph » 16.05.2019 17:29

Hello Michael,
Thank you for help, it is a solution, but but is not THE solution.
To prevent memory, it is the solution. I can let run other project on the others cores.
Nice, but swapping between project.
I will not change all the projects because one project for one kind of WU cause problems !

Your nice solution nothing solve about "ecm" WU. Thank you. but then all ECM will be reduced, also the normal WU (xy,........
My cache is set on a half day (x12 cores, running one hour) So I get 144WU.
Nice, but deadline too short (five days). Impossible to crunch.

Now, about P2 and P1, I have see the difference in RAM. But then the P1 runs about 8 hours for much more less "relative" credits.
I compare credite by running time (or reverse) and on the same host of course.
P2 : one hour for 100CR = 100CR/hr/wu
P1 : 8 hours for 600CR = 75CR/hr/wu

And always cancelling P2 is not very fair.
All the WU I have ecm_uc_......P2_x x vary between 3 and 5. So amready 3-5times in error or cancelled.
Is it the goal of Yoyo ?
A lot of neverending WU, resend, and resent,...and staying in DB ! (not our problem)

Best regards
Philippe

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: Mem use . Monstruous

#4 Ungelesener Beitrag von yoyo » 16.05.2019 21:08

Some facts:
1. You get such P2 only if you have 10 GB free RAM. If a host has less free RAM it will not get P2.
2. BOINC will start a P2 only if you have 10 GB free RAM. Otherwise a P2 is not started
3. If running workunits occupy more RAM as you have configured in BOINC it suspends such a workunit. Means you should reserve some RAM for your operating system and other programs you run. You should not give 90% of your RAM to BOINC.
Deadline:
4. You will get also credits up to 5 days after the deadline.
5. If a workunit wasn't started and reaches the deadline, BOINC will abort this workunit. So no loss of credits because of deadline.

Since I know that 3. is not working properly in all cases and you will have some troubles with I give more credits for P2.
If you don't want to run P2, give only 40% of your RAM to BOINC.
If you want to earn the increased credits, handle the P2, e.g. as Michael proposed or other ways or let BOINC do it.
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Benutzeravatar
[P3D] Crashtest
PDA-Benutzer
PDA-Benutzer
Beiträge: 45
Registriert: 31.07.2011 09:00

Re: Mem use . Monstruous

#5 Ungelesener Beitrag von [P3D] Crashtest » 20.05.2019 19:15

Once again only problems with the fu**ing P2 Wu - please put them in a sub project everyone could unselect them if needed ASAP !!!!

Mal wieder nur Ärger mit den Schei* P2 Wu - könnt ihr die endlich in ein Unterprojekt packen damit die und nur die Problemwuzen verhindern kann?!
Systeme mit 12GB erfüllen die Anforderungen für die WUs aber BOINC ist einfach zu doof dafür - Beispiel:
Rechner mit 12GB RAM und 8 Kernen
- beim Start sind mind. 10GB frei daher startet BOINC natürlich 8 davon
- kurz danach krachen die 8 WU da diese sich viel zu viel RAM schnappen
- teilweise stürzt dadurch Windows ab
….
Es ist keine Lösung ECM auf 2-3 WU zu begrenzen nur eine Unterart von ECM - halt Schei* P2 nur Ärger macht ?!


Links with more of ECM P2 Only Problems:
viewtopic.php?f=56&t=17013 (aus 2018)
viewtopic.php?f=56&t=16670 (aus 2017)
viewtopic.php?f=56&t=16587

PS:
Was ist "M Queens" ?
EPYC ?!? Irgend sowas mit EPYC hab ich auch, so mit 64 Kernen, 128 Threads und so :roll2:
Intel? Dat ist doch ein Hersteller von Netzwerkcontrollern :evil2: und auf der Platine ist so ein Intel Ding mit 10 Gigabit
Bild

marsinph
PDA-Benutzer
PDA-Benutzer
Beiträge: 31
Registriert: 05.04.2018 09:08

Re: Mem use . Monstruous

#6 Ungelesener Beitrag von marsinph » 20.05.2019 20:48

Hello everyone

Only one thing
Why need all the world to make change for one kind of WU ???

I think it is to project to change the WU. Not the reverse (opposite).
Latest year there were also problem but about not respect Boinc setting running time.

So, I am sorry, like Crashtest" very wise and clever write, suggest, make a subproject !!!

I will not remove my RAM to be able to NOT get/receive the P2 !!!

And also, please, build a normal credit system. Depending what you want (running time / real Gflops / CPU time,)
Two or 3 WU, same App, same host, started and running togheter, ...
Pure theory :
One run 100sec for 100CR (CR ratio 1/1)
the second run 200 for 100 CR (ratio 2/1)
The third run 50 for 200CR (ratio 1/4)

So , so long it is impossible to not select (unselect P2), All I receive will be automatically deleted.
Sorry. I will not change computer settings only because one kind of WU

Best regards

Benutzeravatar
Beyond
Prozessor-Polier
Prozessor-Polier
Beiträge: 111
Registriert: 02.02.2008 01:48
Wohnort: Rum River watershed, MN, USA

Re: Mem use . Monstruous

#7 Ungelesener Beitrag von Beyond » 30.05.2019 17:18

yoyo hat geschrieben:
16.05.2019 21:08
Some facts:
1. You get such P2 only if you have 10 GB free RAM. If a host has less free RAM it will not get P2.
2. BOINC will start a P2 only if you have 10 GB free RAM. Otherwise a P2 is not started
3. If running workunits occupy more RAM as you have configured in BOINC it suspends such a workunit. Means you should reserve some RAM for your operating system and other programs you run. You should not give 90% of your RAM to BOINC.
Deadline:
4. You will get also credits up to 5 days after the deadline.
5. If a workunit wasn't started and reaches the deadline, BOINC will abort this workunit. So no loss of credits because of deadline.

Since I know that 3. is not working properly in all cases and you will have some troubles with I give more credits for P2.
If you don't want to run P2, give only 40% of your RAM to BOINC.
If you want to earn the increased credits, handle the P2, e.g. as Michael proposed or other ways or let BOINC do it.
The problem is that there seem to be little but the monstrous WUs being sent anymore. Thus for instance my "perpetual Yoyo machine" (that's been running mostly ECM since 2009) becomes starved of ECM WUs and ends up running other projects like Universe@home.

https://www.rechenkraft.net/yoyo/top_ho ... tal_credit

If I set the preferences to allow other Yoyo subprojects like Siever then it never gets any ECM WUs. How to regularly get the normal sized ECM WUs? There's not even the usual choice that pretty much every other BOINC project has to "If no work for selected applications is available, accept work from other applications". What to do to get ECM but not the P2?

marsinph
PDA-Benutzer
PDA-Benutzer
Beiträge: 31
Registriert: 05.04.2018 09:08

Re: Mem use . Monstruous

#8 Ungelesener Beitrag von marsinph » 30.05.2019 18:22

Hello Beyond.
To not get P2, it is very easy : remove RAM from your host (maximum 8Gb) !!!
No it is not a joke !!! Downgrade your host. like explained and Indirectly suggested from project admin.
But not a solution of course.
Always cancelling WU is also not a solution.
The best is the solution like "Crashtest" wrote. Therefore, project admin need to do it.
With results that more crunchers will come back. He forget that some are crunching to get credit in some competitions.
And here credits are full random. On the same host , running 24/24, OGR WU give 10% CR to compare with other WU.
No any logic.
CR are not adequate wtih running time, nor CPU time.... I compare ONLY on the same physical host.
Running time of P2, is also not to compare. For the same batch, some run 1hr, some 3hrs using 8Gb RAM,
and sometimes the longest WU give more CR than short WU, sometimes, always for the same batch, it is opposite.
It is not proportional ! It seem a real lottery !
Very short example, all on the same host (I7-2600K), host ID 426043 , all P2 WU, running once at the time), Win7 x64 Ultimate.
Si I can fully compare.
CPU time (sec) / CR .....avg CR/sec*1000
4512 / 119.....26.3
3042 / 119 ....39.1
2677 / 80......29.8
1957 / 80......40.8
29600 / 580...19.5
I have very carefully checked the numbers. All are full correct.
I already post about it, answer : you got credit. Of course, I got. But how ???

I really think the best solution is the one suggested by [P3D] Crashtest !

Best regards

marsinph
PDA-Benutzer
PDA-Benutzer
Beiträge: 31
Registriert: 05.04.2018 09:08

Re: Mem use . Monstruous

#9 Ungelesener Beitrag von marsinph » 30.05.2019 20:21

Again, I have do a down grade to 8Gb. To test.
Project refuse tio send ECM !!!
I have accept "Siever" Computing size (given by PRJà about 14 Gflops.
The PRJ says I have a computin capacity of 3 Gflops.
Estimated running time 1h25 . !!!!!!!!!!!!!
The "s" mean second, like everyone known.
So normally the Siever WU need should need a few seconds to run
After 20 minuts running, only 3% done..... It means 666 mnuts to run.
Very far from what the project estimate.
Two possibbbbbbbilties, all of us can not manage our host, or data given are totally false.

ECM are bullshit, Siever the same, On OGR, no any running problem. Woaw.
But total ridiculous credit/

I not undersrtand the philosophu of this PRJ !! (By the way Uafu the same !!!!
Of course, same research, apps and problems.
The difference, on Yafu, PRJ do not respect our computing preference and take all CPU power at 100%.
You set computing at 80%, Yafu dont respect it....

Yoyo, sorry, but please consider all of it.

Best regards

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: Mem use . Monstruous

#10 Ungelesener Beitrag von yoyo » 30.05.2019 20:48

Beyond hat geschrieben:
30.05.2019 17:18

The problem is that there seem to be little but the monstrous WUs being sent anymore. Thus for instance my "perpetual Yoyo machine" (that's been running mostly ECM since 2009) becomes starved of ECM WUs and ends up running other projects like Universe@home.

https://www.rechenkraft.net/yoyo/top_ho ... tal_credit

If I set the preferences to allow other Yoyo subprojects like Siever then it never gets any ECM WUs. How to regularly get the normal sized ECM WUs? There's not even the usual choice that pretty much every other BOINC project has to "If no work for selected applications is available, accept work from other applications". What to do to get ECM but not the P2?
In the last month I had a bug in my wu generator this leads to miss the phase 2 of 160k ecm wus. For them only phase 1 was computed. I detected it during the Pentathlon. Therefore now I'm creating ecm workunit only from this missed p2 queue, 17k left currently. In some days (I assume not more than a week) I'm done with this p2 queue. Afterwards normal processing with creating normal ecm wus from the different ecm projects will continue.

kr,
yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Benutzeravatar
Beyond
Prozessor-Polier
Prozessor-Polier
Beiträge: 111
Registriert: 02.02.2008 01:48
Wohnort: Rum River watershed, MN, USA

Re: Mem use . Monstruous

#11 Ungelesener Beitrag von Beyond » 30.05.2019 21:32

yoyo hat geschrieben:
30.05.2019 20:48
Beyond hat geschrieben:
30.05.2019 17:18

The problem is that there seem to be little but the monstrous WUs being sent anymore. Thus for instance my "perpetual Yoyo machine" (that's been running mostly ECM since 2009) becomes starved of ECM WUs and ends up running other projects like Universe@home.

https://www.rechenkraft.net/yoyo/top_ho ... tal_credit

If I set the preferences to allow other Yoyo subprojects like Siever then it never gets any ECM WUs. How to regularly get the normal sized ECM WUs? There's not even the usual choice that pretty much every other BOINC project has to "If no work for selected applications is available, accept work from other applications". What to do to get ECM but not the P2?
In the last month I had a bug in my wu generator this leads to miss the phase 2 of 160k ecm wus. For them only phase 1 was computed. I detected it during the Pentathlon. Therefore now I'm creating ecm workunit only from this missed p2 queue, 17k left currently. In some days (I assume not more than a week) I'm done with this p2 queue. Afterwards normal processing with creating normal ecm wus from the different ecm projects will continue
Thanks for the update yoyo.

Benutzeravatar
Michael H.W. Weber
Vereinsvorstand
Vereinsvorstand
Beiträge: 22419
Registriert: 07.01.2002 01:00
Wohnort: Marpurk
Kontaktdaten:

Re: Mem use . Monstruous

#12 Ungelesener Beitrag von Michael H.W. Weber » 31.05.2019 08:39

yoyo hat geschrieben:
30.05.2019 20:48
...now I'm creating ecm workunit only from this missed p2 queue, 17k left currently. In some days (I assume not more than a week) I'm done with this p2 queue. Afterwards normal processing with creating normal ecm wus from the different ecm projects will continue.
Probably, a brief note on the project website and in the BOINC client would be useful...

Michael.
Fördern, kooperieren und konstruieren statt fordern, konkurrieren und konsumieren.

http://signature.statseb.fr I: Kaputte Seite A
http://signature.statseb.fr II: Kaputte Seite B

Bild Bild Bild

Antworten

Zurück zu „Fehler, Wünsche / Bugs, Wishes“