ECM memory usage calculation too low

Alles zum Projekt yoyo@home
Everything about the project yoyo@home
Nachricht
Autor
Benutzeravatar
Norman
Klimawolke
Klimawolke
Beiträge: 2188
Registriert: 20.03.2003 14:34
Wohnort: Saarland
Kontaktdaten:

Re: ECM memory usage calculation too low

#61 Ungelesener Beitrag von Norman » 23.01.2011 16:01

yoyo hat geschrieben:ATH im mersenneforum hat ecm mal mit MPIR statt GMP übersetzt. Diese Version ist wesentlich schneller. Wäre schön mal ein Test auf Intel und AMD auf Win64 zu bekommen.
Hier ist die Version http://www.hoegge.dk/mersenne/ecm63mpir ... smredc.zip
yoyo
alte version von ATH:

Code: Alles auswählen

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Input number is 2486977010...7938509483 (65050 digits)
[Sun Jan 23 12:47:02 2011]
Using special division for factor of 2^216091+1
Using B1=250000, B2=126038650, polynomial Dickson(3), sigma=3076591651
dF=1344, k=7, d=12180, d2=11, i0=10
Expected number of curves to find a factor of n digits:
35	40	45	50	55	60	65	70	75	80
4954	70922	1226642	2.5e+007	5.8e+008	1.6e+010	2.7e+013	4e+018	5.3e+023	Inf
Step 1 took 7216794ms
Estimated memory usage: 1698M
Initializing tables of differences for F took 3572ms
Computing roots of F took 83866ms
Building F from its roots took 127687ms
Computing 1/F took 47580ms
Initializing table of differences for G took 24336ms
Computing roots of G took 59640ms
Building G from its roots took 126953ms
Computing roots of G took 59702ms
Building G from its roots took 126954ms
Computing G * H took 32776ms
Reducing  G * H mod F took 61682ms
Computing roots of G took 59686ms
Building G from its roots took 126782ms
Computing G * H took 32573ms
Reducing  G * H mod F took 61480ms
Computing roots of G took 59483ms
Building G from its roots took 128903ms
Computing G * H took 32916ms
Reducing  G * H mod F took 62962ms
Computing roots of G took 61090ms
Building G from its roots took 128155ms
Computing G * H took 32822ms
Reducing  G * H mod F took 62136ms
Computing roots of G took 61292ms
Building G from its roots took 127828ms
Computing G * H took 32417ms
Reducing  G * H mod F took 59904ms
Computing roots of G took 60029ms
Building G from its roots took 128124ms
Computing G * H took 32760ms
Reducing  G * H mod F took 62042ms
Computing polyeval(F,G) took 229509ms
Computing product of all F(g_i) took 10639ms
Step 2 took 2408546ms
Expected time to find a factor of n digits:
35	40	45	50	55	60	65	70	75	80
1.51y	21.65y	374.40y	7642y	177479y	5e+006y	8e+009y	1e+015y	2e+020y	Inf
neue version von ATH:

Code: Alles auswählen

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Input number is 2486977010...7938509483 (65050 digits)
[Sun Jan 23 09:54:08 2011]
Using special division for factor of 2^216091+1
Using B1=250000, B2=126038650, polynomial Dickson(3), sigma=1333584536
dF=1344, k=7, d=12180, d2=11, i0=10
Expected number of curves to find a factor of n digits:
35	40	45	50	55	60	65	70	75	80
4954	70922	1226642	2.5e+007	5.8e+008	1.6e+010	2.7e+013	4e+018	5.3e+023	Inf
Step 1 took 3117898ms
Estimated memory usage: 1698M
Initializing tables of differences for F took 1856ms
Computing roots of F took 40904ms
Building F from its roots took 93226ms
Computing 1/F took 27909ms
Initializing table of differences for G took 11185ms
Computing roots of G took 26489ms
Building G from its roots took 93008ms
Computing roots of G took 26442ms
Building G from its roots took 92789ms
Computing G * H took 21840ms
Reducing  G * H mod F took 33821ms
Computing roots of G took 26473ms
Building G from its roots took 92977ms
Computing G * H took 21933ms
Reducing  G * H mod F took 34258ms
Computing roots of G took 26692ms
Building G from its roots took 94443ms
Computing G * H took 21871ms
Reducing  G * H mod F took 33961ms
Computing roots of G took 27191ms
Building G from its roots took 92712ms
Computing G * H took 22074ms
Reducing  G * H mod F took 34133ms
Computing roots of G took 26677ms
Building G from its roots took 93288ms
Computing G * H took 21840ms
Reducing  G * H mod F took 34289ms
Computing roots of G took 26458ms
Building G from its roots took 93023ms
Computing G * H took 21871ms
Reducing  G * H mod F took 34242ms
Computing polyeval(F,G) took 142726ms
Computing product of all F(g_i) took 8720ms
Step 2 took 1501509ms
Expected time to find a factor of n digits:
35	40	45	50	55	60	65	70	75	80
264.89d	10.39y	179.68y	3668y	85176y	2e+006y	4e+009y	6e+014y	8e+019y	Inf

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: ECM memory usage calculation too low

#62 Ungelesener Beitrag von yoyo » 23.01.2011 16:30

Das sieht doch gut aus, oder?
yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Crystal Pellet

Re: ECM memory usage calculation too low

#63 Ungelesener Beitrag von Crystal Pellet » 23.01.2011 16:59

yoyo hat geschrieben:Das sieht doch gut aus, oder?
yoyo
Das sieht schon besser aus. Was sagst Du zu dem configured with GMP 5.0.1 im output-file?

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: ECM memory usage calculation too low

#64 Ungelesener Beitrag von yoyo » 23.01.2011 17:12

Ich versteh das so, dass MPIR für GMP konfiguriert wurde, also das gleich Interface wie GMP bietet. Damit kann man GMP transparent mit MPIR austauschen.
yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Benutzeravatar
Norman
Klimawolke
Klimawolke
Beiträge: 2188
Registriert: 20.03.2003 14:34
Wohnort: Saarland
Kontaktdaten:

Re: ECM memory usage calculation too low

#65 Ungelesener Beitrag von Norman » 23.01.2011 18:33

ja, das kann man sagen. datt geht schon um einiges flotter mit dieser version.

Crystal Pellet

Re: ECM memory usage calculation too low

#66 Ungelesener Beitrag von Crystal Pellet » 26.01.2011 15:45

I tested the first curve of job ecm_mp_1296011711_P2281_1115_0 with the current ecm application on stock and with ATH's
last new ecm63mpircore2-64-asmredc with MPIR compiled application to compare the speed and the results with the same input.
Tested on AMD Phenom I @2.1GHz with Windows Vista64 4GB RAM.

Current application:

Code: Alles auswählen

ecm.exe -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 110000000 > out
Finished the 1st curve in 1 hour 15 minutes. Peak physical memory usage: 1589000K.

Output:
GMP-ECM 6.2.3 [powered by GMP 4.2.1_MPIR_1.1.1] [ECM]
Input number is
680417416157416239152074255389742230567486020997264272316163310035269998483008885799
837628518530348108432246239159011314343447955207828719397024400853102676556373882318
511000182589726834410813210136391822097042296036924803569996395032234537102888243805
372223813818490623343767008916940746672496726262911341320488422941475154875929922552
352184637585086556680543717631402885260760651553387234979903055199925464684158745102
121923115932718150942791364725066743147812344450691607208155228647746145082537313893
651098225810647075961018540512934745070187771038728113530206698315838028179797963992
42735752201655664299872455218130581340058214411457698515851354238904187139422050546188547 (677 digits)
[Wed Jan 26 12:47:11 2011]
Using special division for factor of 2^2281+1
Using B1=110000000, B2=754319786236, polynomial Dickson(30), sigma=3431016731
dF=103680, k=6, d=1141140, d2=17, i0=80
Expected number of curves to find a factor of n digits:
20 25 30 35 40 45 50 55 60 65
2 4 10 34 136 616 3166 17966 111838 756246
Step 1 took 3045124ms
Estimated memory usage: 1653M
Initializing tables of differences for F took 1935ms
Computing roots of F took 116829ms
Building F from its roots took 48423ms
Computing 1/F took 21965ms
Initializing table of differences for G took 2574ms
Computing roots of G took 113584ms
Building G from its roots took 46535ms
Computing roots of G took 115426ms
Building G from its roots took 48594ms
Computing G * H took 12574ms
Reducing G * H mod F took 17691ms
Computing roots of G took 116985ms
Building G from its roots took 47970ms
Computing G * H took 12246ms
Reducing G * H mod F took 19189ms
Computing roots of G took 117734ms
Building G from its roots took 48734ms
Computing G * H took 12246ms
Reducing G * H mod F took 18517ms
Computing roots of G took 118436ms
Building G from its roots took 45802ms
Computing G * H took 12995ms
Reducing G * H mod F took 17550ms
Computing roots of G took 119871ms
Building G from its roots took 46738ms
Computing G * H took 12808ms
Reducing G * H mod F took 18439ms
Computing polyeval(F,G) took 104505ms
Computing product of all F(g_i) took 609ms
Step 2 took 1438594ms
Expected time to find a factor of n digits:
20 25 30 35 40 45 50 55 60 65
2.18h 4.69h 12.82h 1.78d 7.05d 31.98d 164.31d 2.55y 15.90y 107.52y
--------------------------------------------------------------------------------------------------
Test with the new ecm63mpircore2-64-asmredc application:

Code: Alles auswählen

ecm.exe -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 110000000 > out
This single curve finished in 1 hour and 18 minutes. Peak physical memory usage: 1588144K.

out:
GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Input number is
680417416157416239152074255389742230567486020997264272316163310035269998483008885799
837628518530348108432246239159011314343447955207828719397024400853102676556373882318
511000182589726834410813210136391822097042296036924803569996395032234537102888243805
372223813818490623343767008916940746672496726262911341320488422941475154875929922552
352184637585086556680543717631402885260760651553387234979903055199925464684158745102
121923115932718150942791364725066743147812344450691607208155228647746145082537313893
651098225810647075961018540512934745070187771038728113530206698315838028179797963992
42735752201655664299872455218130581340058214411457698515851354238904187139422050546188547 (677 digits)
[Wed Jan 26 12:14:49 2011]
Using special division for factor of 2^2281+1
Using B1=110000000, B2=754319786236, polynomial Dickson(30), sigma=1887370993
dF=103680, k=6, d=1141140, d2=17, i0=80
Expected number of curves to find a factor of n digits:
35 40 45 50 55 60 65 70 75 80
34 136 620 3169 17982 111949 757094 5550239 4.3e+007 3.6e+008
Step 1 took 3162172ms
Estimated memory usage: 1653M
Initializing tables of differences for F took 1919ms
Computing roots of F took 114926ms
Building F from its roots took 59467ms
Computing 1/F took 29406ms
Initializing table of differences for G took 3245ms
Computing roots of G took 109326ms
Building G from its roots took 60310ms
Computing roots of G took 107719ms
Building G from its roots took 58079ms
Computing G * H took 14321ms
Reducing G * H mod F took 23603ms
Computing roots of G took 102509ms
Building G from its roots took 57096ms
Computing G * H took 12137ms
Reducing G * H mod F took 22479ms
Computing roots of G took 101495ms
Building G from its roots took 56675ms
Computing G * H took 15132ms
Reducing G * H mod F took 22230ms
Computing roots of G took 100168ms
Building G from its roots took 54772ms
Computing G * H took 14835ms
Reducing G * H mod F took 22153ms
Computing roots of G took 98467ms
Building G from its roots took 56020ms
Computing G * H took 14867ms
Reducing G * H mod F took 20748ms
Computing polyeval(F,G) took 125456ms
Computing product of all F(g_i) took 562ms
Step 2 took 1481120ms
Expected time to find a factor of n digits:
35 40 45 50 55 60 65 70 75 80
1.85d 7.34d 33.31d 170.29d 2.65y 16.48y 111.47y 817.21y 6372y 53732y

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: ECM memory usage calculation too low

#67 Ungelesener Beitrag von yoyo » 29.01.2011 19:21

Hello,

I deployed the new ecm binaries for Windows 32 and 64 bit and recreated the failed mp workunits. Lets see how it is working now.

yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Crystal Pellet

Re: ECM memory usage calculation too low

#68 Ungelesener Beitrag von Crystal Pellet » 29.01.2011 19:33

I'm very honoured that i got the 1st one of the retried batch:

yoyo@home 29-Jan 19:18 Started download of ecm63_win64_core2.exe
yoyo@home 29-Jan 19:18 Started download of ecm_mp_1296324954_P216091_1000_info

Benutzeravatar
Conan
XBOX360-Installer
XBOX360-Installer
Beiträge: 80
Registriert: 11.02.2010 11:09

Re: ECM memory usage calculation too low

#69 Ungelesener Beitrag von Conan » 26.02.2011 08:15

I have just started running the ECM work units and was wondering how much memory they should take?
I have had all my ECM work units fail on my Windows XP 32 machine with 4GB RAM.
The error messages indicate that I have run out of Virtual memory and when I OK for Windows to increase the memory size the WUs then error out.

The WUs start off only needing at most 35 MB of RAM which is no problem but some time later this jumps to well over 470 MB and more, and with 4 running I run out of memory.

I might have to stop running them I think.

Conan
Bild

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: ECM memory usage calculation too low

#70 Ungelesener Beitrag von yoyo » 26.02.2011 08:39

Can you give me the name or id of such an workunit?

Usually I assign each wu the amount of ram which is needed. It depends also on how many ecm you have run in parallel. The biggest ones needs 1.8 GB RAM and I assign the wu a bit more to be sure. It seems that Boinc doesn't take properly care of these requirements if more than one wu is running in parallel. In Boinc is a mechanism implemented to stop wus (waiting for memory) if not enough RAM is available. But I do not know how exactly it works.

yoyo
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Benutzeravatar
Conan
XBOX360-Installer
XBOX360-Installer
Beiträge: 80
Registriert: 11.02.2010 11:09

Re: ECM memory usage calculation too low

#71 Ungelesener Beitrag von Conan » 26.02.2011 14:15

Thanks yoyo,
The failed work units are as follows
9589944
9589945
9589946
9589952
9589954

Yes the Boinc Manager had at least 2 at a time running when I have struck the problems.

It is possible that the memory requirements for the work units went well past 1 GB for each work unit with both Boinc and Windows not knowing how to handle the situation so it locked up and then made the WUs error out.

Conan
Bild

Kara Jade

Re: ECM memory usage calculation too low

#72 Ungelesener Beitrag von Kara Jade » 13.04.2011 20:32

Wed 13 Apr 12:10:04 2011 yoyo@home Aborting task ecm_op_1302639624_193_97M.C176_2965_0: exceeded memory limit 1036.45MB > 1024.00MB
Wed 13 Apr 12:10:05 2011 yoyo@home Computation for task ecm_op_1302639624_193_97M.C176_2965_0 finished
:( I only have 1GB of RAM. Well, time to put these tasks on suspension and stop procrastinating the RAM upgrade.

Antworten

Zurück zu „Number crunching“