open Beta Test for new cmsearch VM application

[ChristianB]

I just double checked to make sure. The actual scientific result file WAS copied into out.zip but was empty. The file that has the output of cmsearch (and any error messages) was not copied to the out.zip (because of a bug when converting those XXL tasks to VM). So either cmsearch found nothing or it terminated because it needs more memory is unknown because v1.02 of the app doesn't have the memory detection feature. I corrected the bug and can repair the converted tasks but I still need to see if the newest vboxwrapper will fix our other problems. Thanks to Crystal Pellet and Jacob Klein it got fixed but I haden't have time to review and prepare a new app version.

[zombie67]

What is the expected run times? I think I saw something in an earlier post from the admin about 10 days. But I am over 12 days now with 35 tasks, and none have completed (yet). And this is with the fastest intel CPUs. So what is "normal" for these tasks?

[Crystal Pellet]

Computation error of my task of workunit http://www.rnaworld.de/rnaworld/workuni ... id=6330531

RNA World 04 Nov 11:08:57 Computation for task cmsvm_GA-p[e30-50MB_Lin64f]_1_Rattus-norvegicus_1330438623_109790_0 finished
RNA World 04 Nov 11:08:57 Output file cmsvm_GA-p[e30-50MB_Lin64f]_1_Rattus-norvegicus_1330438623_109790_0_0 for task cmsvm_GA-p[e30-50MB_Lin64f]_1_Rattus-norvegicus_1330438623_109790_0 absent
RNA World 04 Nov 11:08:57 Output file cmsvm_GA-p[e30-50MB_Lin64f]_1_Rattus-norvegicus_1330438623_109790_0_1 for task cmsvm_GA-p[e30-50MB_Lin64f]_1_Rattus-norvegicus_1330438623_109790_0 absent

[Crystal Pellet]

Computation error of my task of workunit http://www.rnaworld.de/rnaworld/workuni ... id=6330531

OK, that was the bad news (>4,000,000 seconds CPU-time wasted?)

The good news is that I've been able to save the VM with the latest snapshot. I got it running now outside of BOINC with a new shared directory.
Of course the directory at first was empty, cause BOINC cleaned almost everything, but progress.txt is created and updated in the new shared folder.
I suppose the other shared files during BOINC-life were just input files for the task, but read into the VM at the very first start.

Christian you probably can confirm that. I hope, if, when, whether the task ends, there will appear some usable output files into the shared folder. Please advice.

[ChristianB]

Yes, the output files will be there in the end. You can keep it running and send the files. I can integrate this into the Science DB but not in BOINC.

[Michael H.W. Weber]

So either cmsearch found nothing...

With the -p WUs that probability is virtually zero. To my expectation, in any of these WUs there should be at least one "hit", which then likely is just a scientifically non-relevant RNA fragment found in many sequences. Also, the truely empty files contain some characters to indicate successful completion of processing, if I remember correctly.

Michael.

[zombie67]

What is a normal run time for these VM tasks?

[Beorn]

For each work package you will find an estimated run time on a reference system. For example for the package 6330742 we both share, this estimation is 14 weeks, 5 days and a bit.

For the small test work units, which were sent out beginning of October, that seemed to be quite reliable. I did 52 of those and my system was between 50% and 80% of the ref. system with most at around 60%. However, I could not finish any of the larger work units from 20th of October yet and even reached 138% of the est. time of the ref. system here for one WU on the same computer as above. So IMHO it remains to be seen if the estimation is still reliable for the WU out it the field now.

Regards

EDIT: OMG, one was ripe and just fell from the tree!! http://www.rnaworld.de/rnaworld/result. ... d=14920843

Let's see what we can see. That's 839,064.95 sec. which would be 104% of the ref. system. Hope my wingman will make it back to the airfield now.

@ChristianB: Could you do me a big favour and check if you received something useful here?

[ChristianB]

EDIT: OMG, one was ripe and just fell from the tree!! http://www.rnaworld.de/rnaworld/result. ... d=14920843

Let's see what we can see. That's 839,064.95 sec. which would be 104% of the ref. system. Hope my wingman will make it back to the airfield now.

@ChristianB: Could you do me a big favour and check if you received something useful here?

I can confirm that the result is valid (as in contains scientific data)! Congratulations.

[zombie67]

For each work package you will find an estimated run time on a reference system.

Ah! I see now. You have to look at each WU to see the individual estimates. Got it.

But something is odd. I could swear that they all had due dates of ~Nov 9th, which is impossible for some of the very long tasks. I'll have to double check when I get back to my machines later today.

[Beorn]

Thank you, Christian. Good news, indeed. For the records: the upload consisted of two files, which took only 7 and 3 seconds. There are no unusual leftovers (slot is occupied by another workunit now, no 'zombie' processes), three more VM were running concurrently, all are still alive.

05.11.2013 17:33:43 | RNA World | Computation for task cmsvm_GA-p[b-Lin64f-2]_1_Klebsiella-pneumoniae-342_CP000964.cir.EMBL_RF00177_SSU_rRNA_5_1358679723_1030_1 finished
05.11.2013 17:33:46 | RNA World | Started upload of cmsvm_GA-p[b-Lin64f-2]_1_Klebsiella-pneumoniae-342_CP000964.cir.EMBL_RF00177_SSU_rRNA_5_1358679723_1030_1_0
05.11.2013 17:33:46 | RNA World | Started upload of cmsvm_GA-p[b-Lin64f-2]_1_Klebsiella-pneumoniae-342_CP000964.cir.EMBL_RF00177_SSU_rRNA_5_1358679723_1030_1_1
05.11.2013 17:33:49 | RNA World | Finished upload of cmsvm_GA-p[b-Lin64f-2]_1_Klebsiella-pneumoniae-342_CP000964.cir.EMBL_RF00177_SSU_rRNA_5_1358679723_1030_1_1
05.11.2013 17:33:53 | RNA World | Finished upload of cmsvm_GA-p[b-Lin64f-2]_1_Klebsiella-pneumoniae-342_CP000964.cir.EMBL_RF00177_SSU_rRNA_5_1358679723_1030_1_0
05.11.2013 17:33:55 | RNA World | Sending scheduler request: To report completed tasks.
05.11.2013 17:33:55 | RNA World | Reporting 1 completed tasks

[zombie67]

But something is odd. I could swear that they all had due dates of ~Nov 9th, which is impossible for some of the very long tasks. I'll have to double check when I get back to my machines later today.

Yep, they are all due 11/09 and 11/10 according to BOINCmanager. And some of these are very long tasks, over 14 weeks. The tasks page on the project site now shows due dates in December, which don't match BOINCmanager.. But even so, those December due dates are still way too soon. This is all very confusing.