ECM memory usage calculation too low

[Crystal Pellet]

AMD Phenom I 2.1GHz 4GB RAM Vista64

Both versions (6.2.3 and 6.3) crashed.
The second task did not claim high resources at the moment where the 1st already crashed.

Process properties at the crash:


ecm623.exe -v -v -v -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 250000 > out

out 683,614kB:
075559596698003674054087452920899877824225446530052260591825406521708444780735017092321797364819395075618975086795749840829579144274150257956846021638225367720158814846847892852
424832899344355930794497552496679769553116425263127224936327512184515206632627405319062433958446609625850342269681987033546789498815369200313418338617888864921843369312338166109
328398874562593977435710152487710463449933848377843115379941600625499082100829021559286291903670719161216954475382893962076332057028632498089984571364213176668946054526030826210
998480965528480974344777849200024922893693534453837627623496968700560169115539669201882635473529200039700354578061449200814053579193514940929845009155066929701004041478518548707
407793739631007379067358870034577190613922195106323017686628404734382717720872639203385419432487119485661342594293109804608420108967413662765345905273770160606745263122940017223
044713024761620714035281186500980675864498350291127278632105000934217585464459805807448283422486700880467897631086593352632182211267771764108848003538784511508186506021390452112
866365008795327691624703946210076879328299504632095630170727861577421015915887758282254463929905770370738821042111186538161683040920982506325824302009532937137420621226617809598
884265870401340007230356028379764057212171375861768330459873422317384428670784310167508358223210360065940165951265303602304587224884917852305420867886509444180371507384006995186
677412812760885684566921588546756288147107815093284578922595920910866569178756853540211562924232529980763049499908292156731080117535773643961497937684070924099645014941457996474
677470705238444111634030687562310736229287162710549967544739960952427438222153035152164687808462780560092968783571135025810935193442774562117813470192223193421006720046040276858
535137975713443178342671420751679677785591080448420603026288692586250821629017335336869433381626237496751628638165118550011412056973247487716698367384297399202216903077206419071
011383300162303472328573754193944351589188115876329541230151260536866964618072924211133183142243841059744646509027829066834818079818520967486957383294843106473506761859872500781
457741720588041679655770514971580865661070492640994578990416637613566673825692068807359811768123095947331923625127753231631508359100494070450247248649896864005971230487590754063
563169408403594129289575216267412738364549429487929622315449960901786732874836441236415333592312629704695871238822622389819731116745325238267913100659373646862891954048663832133
579555723896300969046660223276229941896323762985110176970440736539818430704973020441434464997089683376315724 * x^1343)
Building G from its roots took 130558ms
Computing G * H took 30576ms


ecm63.exe -v -v -v -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 250000 > out
out 85,777kB:
650075954296360958097471547174752220497520184394533019888510444787252683082226421367780614801259861009562606372248613877325959532789351648357434590807035695708396611645998650481
630524495400324119413061399999290681834594799154782722827136183956534751177314517978976270174869236320697099185317830087655882250166988489730958964564156987755996793988246167978
492459803121814678609242170575108374523366953047576270976691866013990427118621807089313661253293665613426272561743718803133261318386523518386525129014781868123267031418856869990
38185090734634098708565204475082615611111563943857490766033278965036261749972941298

[yoyo]

Thanks for the test.
I raised it already in the gmp-ecm forum and hope to get help from the developer.
yoyo

[Crystal Pellet]

As promised, if fails with maxmem parameter second test done without -maxmem on Intel Q8300 2.5GHz 8GB RAM Vista64:

Both tasks crashed. Process properties at the crash moment:


ecm623.exe -v -v -v -v -nn -timestamp -chkpnt checkpnt -inp in 250000 > out

out 1,464,453kB:

539379532140028152320372649085162716830754023361586739781985569202863634443152640382840
341960370434496309049409693017042814099367474494432325333813184456734041182408694176588
717822348869521761965355157860357560188720109593681008040965117651753149266932818637299
256232026334561547018240903366529438535805571951163760950952071319558883704518436035913
093362440219329354286678027483212974200863565185443614523034846769029343656048598159464
772723191682005274311714911101927470868277274039059193566456098749631907351516457624560
897738910242166268139866919566632067762501336628603845404098082797899027184056749493487
117685286468622569838388012262552139114962865731636595716927454618172737737531220759547
227339347896295093620376799721650349468364231820776688454867869336513126456505239481211
636114192905894511198900412923006150134869521865106291156913022921484729736037123807694
867811700278797501927390593127539091210043892394707849689057166257234749334459482461154
687759013112494362479127769579930385765725893772470374799357617367928037078510483197538
220186827815908603652192481587237269792204366689652771945494737151448590913450468990698
054281950245893187730753389229770939029299110395008813264959058766453473549243744202288
837960158305730682165680768870399696000926649427226084368736416961387605078093108238970
523340497966298934350573251416025231801765998300343482411990264309033574940676510497522
526379867663994365955359805619879514945791968746237453465217060053833225775973888964957
7700732319389089043154334233420562084003864139429450053179632055608529261062732237 * x^2879)
Building G from its roots took 297900ms


ecm63.exe -v -v -v -v -nn -timestamp -chkpnt checkpnt -inp in 250000 > out

out 183,371kB:

941453680618061761093055229611409386876079224689085213140395820141752358794543099372770
299922753362805323283563861114967041623913562419322255543666995010596761039915899295444
994913248344115799956633348446123398317566935195268172650100747635881150177939346092255
836042934981827894633161493711696374872552641832126288155148303858864019946707911357072
9140454258495982190976342948673514354586079227292

[Bogusz]

Hi the same problem here, I have a machine with Windows 7 64bit, intel 930 CPU and 6GB RAM.

[Grubix]

Ich habe den Test ausgeführt, mit einem Phenom II X6 1090T / Win7-64 / 8GB. Es kommt zu einem Crash. Die Ausgabe der "out"-Datei schreibe ich später, gibt noch ein kleines Problem damit, weil die Datei 700MB groß ist...

Bis denn dann, Grubix.

[Norman]

out ca. 1kb
win7 x64
i7 920
12gb ram
getestet mit:

Code: Alles auswählen

ecm.exe -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 250000 > out

Code: Alles auswählen

ecm 6.3.0

 Problemereignisname:	APPCRASH
  Anwendungsname:	ecm.exe
  Anwendungsversion:	0.0.0.0
  Anwendungszeitstempel:	4c5c3146
  Fehlermodulname:	ecm.exe
  Fehlermodulversion:	0.0.0.0
  Fehlermodulzeitstempel:	4c5c3146
  Ausnahmecode:	c00000fd
  Ausnahmeoffset:	0000000000090147
  Betriebsystemversion:	6.1.7600.2.0.0.256.1
  Gebietsschema-ID:	1031
  Zusatzinformation 1:	4873
  Zusatzinformation 2:	4873992456b93327d69e13f44ba7612f
  Zusatzinformation 3:	9d7c
  Zusatzinformation 4:	9d7ca467dde92e9bed399d1db8935ad0

GMP-ECM 6.3 [configured with GMP 5.0.1 and --enable-asm-redc] [ECM]
Input number is 2486977010...7938509483 (65050 digits)
[Thu Jan 20 11:04:18 2011]
Using special division for factor of 2^216091+1
Using B1=250000, B2=126038650, polynomial Dickson(3), sigma=399884835
dF=1344, k=7, d=12180, d2=11, i0=10
Expected number of curves to find a factor of n digits:
35	40	45	50	55	60	65	70	75	80
4954	70922	1226642	2.5e+007	5.8e+008	1.6e+010	2.7e+013	4e+018	5.3e+023	Inf
Step 1 took 3202467ms
Estimated memory usage: 1698M
Initializing tables of differences for F took 1857ms
Computing roots of F took 40716ms


ecm 6.2.3

Problemsignatur:
  Problemereignisname:	APPCRASH
  Anwendungsname:	ecm.exe
  Anwendungsversion:	0.0.0.0
  Anwendungszeitstempel:	49f3ad2f
  Fehlermodulname:	ecm.exe
  Fehlermodulversion:	0.0.0.0
  Fehlermodulzeitstempel:	49f3ad2f
  Ausnahmecode:	c00000fd
  Ausnahmeoffset:	0000000000080d97
  Betriebsystemversion:	6.1.7600.2.0.0.256.1
  Gebietsschema-ID:	1031
  Zusatzinformation 1:	7b45
  Zusatzinformation 2:	7b45cabe5f8b27f0f4af1b9e1ad348d3
  Zusatzinformation 3:	85ae
  Zusatzinformation 4:	85aef35fad171fcd364831d89d29f3a4

GMP-ECM 6.2.3 [powered by GMP 4.2.1_MPIR_1.1.1] [ECM]
Input number is 2486977010...7938509483 (65050 digits)
[Thu Jan 20 11:08:41 2011]
Using special division for factor of 2^216091+1
Using B1=250000, B2=126038650, polynomial Dickson(3), sigma=2641729487
dF=1344, k=7, d=12180, d2=11, i0=10
Expected number of curves to find a factor of n digits:
20	25	30	35	40	45	50	55	60	65
8	50	430	4958	70955	1226930	2.5e+007	5.9e+008	1.6e+010	2.6e+013
Step 1 took 4056213ms
Estimated memory usage: 1698M
Initializing tables of differences for F took 3027ms
Computing roots of F took 61714ms
Building F from its roots took 75941ms
Computing 1/F took 26692ms
Initializing table of differences for G took 15631ms
Computing roots of G took 35522ms
Building G from its roots took 76003ms
Computing roots of G took 35615ms
Building G from its roots took 76316ms
Computing G * H took 19282ms

[yoyo]

That all is basically bad, that both versions with and without -maxmem is crashing.
I'm just waiting for some feedback of the developers.
But seems that I have to avoid for the future to run such large numbers with small B1.
yoyo

[Sysadm@Nbg]

Verluste gibts überall - dieserist besonders ärgerlich (da Linux 64-bit Kiste)

[yoyo]

Ja, sorry, ich hab einfach den ganzen Batch gekillt. Notbremse gezogen nachdem fast keine WU durchlief.

Hätte ich wohl das "Beta" bei yoyo@home stehen lassen sollen

yoyo

[yoyo]

Hello,

can you try this version http://www.mersenneforum.org/showpost.p ... tcount=213 and state if it works and which version you used?

yoyo

[Crystal Pellet]

The first test I'll do on Vista64 Phenom I 4GB RAM:
ecm63core2-64.exe -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 250000 > out
Result later.

[Crystal Pellet]

The first test I'll do on Vista64 Phenom I 4GB RAM:
ecm63core2-64.exe -v -nn -timestamp -chkpnt checkpnt -inp in -maxmem 1800 250000 > out
Result later.

The good news: "The task didn't crash."
The bad news: "The taskduration (1 curve) now 4 hours and 50 minutes in stead of 2 hours and 5 minutes"
Peak working set: 2,082,804K
Max Working Set: 1,866,284K
Peak Virtual Memory: 2,216,080K

Contents out-file:

GMP-ECM 6.3 [configured with GMP 5.0.1] [ECM]
Input number is 2486977010...7938509483 (65050 digits)
[Fri Jan 21 08:30:04 2011]
Using special division for factor of 2^216091+1
Using B1=250000, B2=126038650, polynomial Dickson(3), sigma=2456448850
dF=1344, k=7, d=12180, d2=11, i0=10
Expected number of curves to find a factor of n digits:
35 40 45 50 55 60 65 70 75 80
4954 70922 1226642 2.5e+007 5.8e+008 1.6e+010 2.7e+013 4e+018 5.3e+023 Inf
Step 1 took 13154706ms
Estimated memory usage: 1698M
Initializing tables of differences for F took 6537ms
Computing roots of F took 159090ms
Building F from its roots took 236155ms
Computing 1/F took 84817ms
Initializing table of differences for G took 46083ms
Computing roots of G took 110901ms
Building G from its roots took 236684ms
Computing roots of G took 112321ms
Building G from its roots took 236575ms
Computing G * H took 60247ms
Reducing G * H mod F took 112165ms
Computing roots of G took 110589ms
Building G from its roots took 237434ms
Computing G * H took 60497ms
Reducing G * H mod F took 111322ms
Computing roots of G took 112461ms
Building G from its roots took 236950ms
Computing G * H took 60700ms
Reducing G * H mod F took 111245ms
Computing roots of G took 109419ms
Building G from its roots took 220913ms
Computing G * H took 55723ms
Reducing G * H mod F took 101073ms
Computing roots of G took 103834ms
Building G from its roots took 207263ms
Computing G * H took 55193ms
Reducing G * H mod F took 99591ms
Computing roots of G took 98936ms
Building G from its roots took 211101ms
Computing G * H took 52370ms
Reducing G * H mod F took 99201ms
Computing polyeval(F,G) took 374277ms
Computing product of all F(g_i) took 19017ms
Step 2 took 4251168ms
Expected time to find a factor of n digits:
35 40 45 50 55 60 65 70 75 80
2.73y 39.15y 677.04y 13819y 320944y 9e+006y 1e+010y 2e+015y 3e+020y Inf

I'm doing some other tests with ecm63core2-64-asmredc.exe and ecm63rc4-core2-64.exe.
They are still busy. If they succeed (get a valid result), they also will run much longer.
It looks like all these versions are not optimized for running on AMD's and will take 2½ the
runtime compared with the current Windows-application, which is optimized for AMD's.

Your choice:
1. Keep the old application and have much faster completion of all kinds of ecm's (as, es, mp, nr, op, ru, xy) and keep your fingers from these high mp-numbers or
2. Your ecm-project can crunch big mp-numbers, but all batches will take 3 times longer (perhaps even longer, because some AMD-crunchers could move on....
3. Find the developer of the current ecm-application and find the bug.
4. Distribute those monsters only to Linux clients.