Long running work unit

[Beyond]

Hello,
I extendet all deadlines by 30 additional days. The task which stays at 100%, let them run.
yoyo

Thanks yoyo!

[Beyond]

cms_GA-p[e30-50MB_Lin64s]_1_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4144_2
(15% done after 110 hours)

Since I posted the above, this WU has apparently "restarted" twice. The 1st time it got to about 4% done and now it's restarted again and is at 3.3% done. The machine uptime is at 23 days, BOINC has not been restarted. Here's the WU link:

The stderr from 9/9/11 has this caution:
3 archives were successfully processed.
caution: filename not matched: common/*

The stderr.txt from 9/15/11:

RNA World wrapper v0.04
wrapper: no checkpoint file found
wrapper: running unzip (-o *.zip)
wrapper: no checkpoint file found
wrapper: running unzip_cpufeat (cmsearch.zip)
wrapper: no checkpoint file found
wrapper: running cmsearch (--forecast 1 -T 0.0 --fil-T-hmm 0.0 --fil-T-qdb 0.0
RF00028_Intron_gpI.cm Bos-taurus-(cattle)_CM000203.lin.EMBL.fasta)
forecast.txt found.
wrapper: no checkpoint file found
wrapper: running cmsearch (-o cms_GA-p[e30-50MB_Lin64s]_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4143_0 -T 0.0 --fil-T-hmm 0.0 --fil-T-qdb 0.0
RF00028_Intron_gpI.cm Bos-taurus-(cattle)_CM000203.lin.EMBL.fasta)
forecast.txt found.
wrapper: windows. no checkpoint image

... hundreds of "wrapper: windows. no checkpoint image" deleted...

wrapper: windows. no checkpoint image
17:35:23 (3132): No heartbeat from core client for 30 sec - exiting
17:35:24 (3132): No heartbeat from core client for 30 sec - exiting
RNA World wrapper v0.04
wrapper: no checkpoint file found
wrapper: running cmsearch (-o cms_GA-p[e30-50MB_Lin64s]_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4143_0 -T 0.0 --fil-T-hmm 0.0 --fil-T-qdb 0.0
RF00028_Intron_gpI.cm Bos-taurus-(cattle)_CM000203.lin.EMBL.fasta)
forecast.txt found.
wrapper: windows. no checkpoint image

... hundreds of "wrapper: windows. no checkpoint image" deleted...

wrapper: windows. no checkpoint image
21:23:36 (1296): No heartbeat from core client for 30 sec - exiting
RNA World wrapper v0.04
wrapper: no checkpoint file found
wrapper: running cmsearch (-o cms_GA-p[e30-50MB_Lin64s]_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4143_0 -T 0.0 --fil-T-hmm 0.0 --fil-T-qdb 0.0
RF00028_Intron_gpI.cm Bos-taurus-(cattle)_CM000203.lin.EMBL.fasta)
forecast.txt found.
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
05:09:23 (4112): No heartbeat from core client for 30 sec - exiting
RNA World wrapper v0.04
wrapper: no checkpoint file found
wrapper: running cmsearch (-o cms_GA-p[e30-50MB_Lin64s]_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4143_0 -T 0.0 --fil-T-hmm 0.0 --fil-T-qdb 0.0
RF00028_Intron_gpI.cm Bos-taurus-(cattle)_CM000203.lin.EMBL.fasta)
forecast.txt found.
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image
wrapper: windows. no checkpoint image

I'm suspending it, waiting for advice...

Edit: notice the filename says "4144_2" and the wrapper says it's running "4143_0"?

[Norman]

I hope you have "leave application in memory" selected in boinc settings otherwhise i could be that the task make a restart from scratch after a benmark or a app-switch.

[Beyond]

I hope you have "leave application in memory" selected in boinc settings otherwhise i could be that the task make a restart from scratch after a benmark or a app-switch.

Yes, it's been left in memory when I suspended it. There also has been no app switches and no benchmark runs. It's been running at "High Priority" since it started.

[Michael H.W. Weber]

That is a bit strange, maybe you should not suspend it?

Michael.

[Beyond]

That is a bit strange, maybe you should not suspend it?
Michael.

It looks like it keeps starting over from the beginning. Is there any way to tell if it's continuing on instead of restarting?

[yoyo]

Hello,
I think the problem is this:

05:09:23 (4112): No heartbeat from core client for 30 sec - exiting

You computer is somehow hanging for some time or the system time changes. Than Boinc doesn't get a heartbeat within 30s and terminates.

yoyo

[shauge]

Hi, I have some wus that seems to have a long time until finish and are starting to reach deadline. I think all of these needs a deadline extension, although some have a few days left:
Task IDs:
13141412
13130739
13126123
13122656
13122081
13114113
13114109
13079484

Regards,
shauge

[yoyo]

Extensions granted.
yoyo

[ftpd]

Hej YoYo,

I have one itron-wu task-id = 13121683 host-id = 9125.
Deadline is 24/09 and is still running for more then 320 hrs.

Extend please deadline.

Thx,

Ton

PS Good to have small-wu's again. Thx!

[yoyo]

done.
yoyo

[Beyond]

Just tell us the taskID and we extent the deadline.
yoyo

I have 4 WUs that are reporting that they will exceed the deadline. All are on fast machines:

cms_GA-p[e30-50MB_Lin64s]_1_Canis-lupus-familiaris-(dog)_CM000028.lin.EMBL_RF00028_Intron_gpI_1314906424_9632_0
(23.5% done after 149 hours)

cms_GA-p[e30-50MB_Lin64s]_1_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00028_Intron_gpI_1314906424_4144_2
(15% done after 110 hours)

cms_GA-p[e30-50MB_Lin64s]_1_Bos-taurus-(cattle)_CM000203.lin.EMBL_RF00030_RNase_MRP_1314906424_4146_3
(12.3% done after 67 hours)

cms_GA-p[e30-50MB_Lin64s]_1_Anopheles-gambiae-str.-PEST_CM000356.lin.EMBL_RF00030_RNase_MRP_1315448223_30_1
(9.9% done after 66 hours)

Hello, I think the problem is this:

05:09:23 (4112): No heartbeat from core client for 30 sec - exiting

You computer is somehow hanging for some time or the system time changes. Than Boinc doesn't get a heartbeat within 30s and terminates.
yoyo

All 4 of the above WUs have since restarted with the same message in stderr.txt: "No heartbeat from core client for 30 sec - exiting". That's well over 800 hours of wasted CPU time. These WUs are on 4 different machines. All are Win7-64, BOINC 6.12.34, 3 are X6, 1 is an X4. All 4 machines show over a month uptime since last reboot. BOINC task duration in Process Lasso shows that it has not restarted. System times are set to NIST automatically via Win7 but never vary more than a second between updates. No problems whatsoever with any other projects. These huge tasks with no checkpointing are unacceptable IMO.


Also as requested I've left this task running (on a 5th machine BTW):

In addition I have one WU that has been sitting at 100% completion for the last few days. It's still using a full core of CPU and is at 179 hours. What to do with it?:

cms_GA[e20-30MB_Lin64f]_1_Oryza-sativa-Japonica-Group_AP008216.lin.EMBL_RF00028_Intron_gpI_1307967123_54908_18

It's now at 323 hours and still showing 100%. Looking at the WU link shows it has been handed out 21 times with no successful completions. Keep running it or abort? If keep running, for how long?

Thanks/Beyond