Too many work units

Alles zum Projekt yoyo@home
Everything about the project yoyo@home
Nachricht
Autor
orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Too many work units

#1 Ungelesener Beitrag von orpy » 31.10.2020 14:24

My computer has BOINC set to store 0.1 days of work and 0.1 additional days.

I've just had to abort 524 WUs that were each estimated to take more than 2 days each and a further 235 WUs that were set to take more than one day each -- and that still leaves hundreds more to process that require less than a day.

Why is the server ignoring preferences and sending so many WUs that will just end up exceeding their deadlines?
Bild

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: Too many work units

#2 Ungelesener Beitrag von yoyo » 31.10.2020 15:57

It is not the server. You boinc client request work for an amount of seconds and the server delivers it. Those settings are not sent to the server.
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Re: Too many work units

#3 Ungelesener Beitrag von orpy » 31.10.2020 18:13

I posted the settings for that host's BOINC client: store 0.1 days + an additional 0.1 days.

All other projects honour those settings, apart from Yoyo.

According to BAM that host had 45147:30:23 of work left to process!

Did my computer really ask for 162531023s work?

I note that there is also a max 16 processor limit set in the Yoyo preferences -- but this doesn't work either on this PC's 20 processor CPU.

[Removed a line saying other hosts were also receiving too many tasks] Just checked all hosts (Mac and Windows) and it looks as if they are all honouring the set preferences -- apart from this one host...
Bild

Benutzeravatar
gemini8
Vereinsvorstand
Vereinsvorstand
Beiträge: 5898
Registriert: 31.05.2011 10:30
Wohnort: Hannover

Re: Too many work units

#4 Ungelesener Beitrag von gemini8 » 31.10.2020 19:39

Your machine asked for about five hours per thread.
Did it do any Yoyo work before?
If not, it might have calculated the length of the work-units as too short on incoming, and then asked for more work.
As yoyo already mentioned it's not the server that sends work on its behalf - it's Boinc running on the machine asking for enough work for a specified amount of time.

*edit*
Fixed those hours above.
Must have been too tired yesterday after too short a night.
*end edit*
Gruß, Jens
- - - - - -
Lowend-User und Teilzeit-Cruncher

Bild Bild Bild
Bild

orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Re: Too many work units

#5 Ungelesener Beitrag von orpy » 01.11.2020 15:14

This machine has been crunching Yoyo since 2018 without any issues.

I noticed last week that there were hundreds and hundreds of one particular WU type -- can't remember which right now -- so changed the storage settings from 0.5 days to 0.1 days and then aborted most of the WUs.

Looked again yesterday and there were even more WUs of all types.

I haven't downloaded a new task since, but still have hundreds of hours of WUs to process even after aborting more than 800.

I'll restart it once it has finished the current batch and see what happens when I select "Allow new tasks" again.
Bild

Benutzeravatar
gemini8
Vereinsvorstand
Vereinsvorstand
Beiträge: 5898
Registriert: 31.05.2011 10:30
Wohnort: Hannover

Re: Too many work units

#6 Ungelesener Beitrag von gemini8 » 01.11.2020 15:29

That's a curious behaviour then.
Don't have any further ideas.
Gruß, Jens
- - - - - -
Lowend-User und Teilzeit-Cruncher

Bild Bild Bild
Bild

orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Re: Too many work units

#7 Ungelesener Beitrag von orpy » 01.11.2020 16:07

Very curious :)

I can't see how it could miscalculate the processing times so badly that it had downloaded 524 WUs with estimated processing times greater than 2 days -- and a deadline of November 4th!

And, at the same time, not miscalculate the time it takes to crunch SRBase GPU tasks...

Unless it is using the 4:40 WU estimates for the GPU tasks and applying them to the CPU WUs?
Bild

Benutzeravatar
gemini8
Vereinsvorstand
Vereinsvorstand
Beiträge: 5898
Registriert: 31.05.2011 10:30
Wohnort: Hannover

Re: Too many work units

#8 Ungelesener Beitrag von gemini8 » 02.11.2020 06:39

This might happen with a project that has both.
Yoyo hasn't, so I'm not sure it's possible to mismatch Yoyo because of SRBase.

On VGTU I had once work-units that ran too long and got canceled.
That was due to a lot of short work-units, which made the scheduler think the long ones were over some limit.
It helped to adjust some line in cc_config.xml iirc.
So, I know, Boinc can do silly stuff sometimes. ;-)

Is it possible that your machine used some other preferences?
I happened to run into a long-persisting issue which made some machines switch between different web preferences (BAM! and another) twice an hour.
That was a behavioural Boinc bug. :biggrin2:
Gruß, Jens
- - - - - -
Lowend-User und Teilzeit-Cruncher

Bild Bild Bild
Bild

orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Re: Too many work units

#9 Ungelesener Beitrag von orpy » 02.11.2020 10:07

At midnight last night the machine had about 40 WUs estimated at under 3hrs each.

I enabled new tasks and watched two uploads that each resulted in the download of one new WU.

Got up this morning and there are another stupid number of WUs that have been downloaded.

414 Cruncher OGR with a deadline of 2/12 -- estimated at 1 day 16 hours each

205 ECM 705-02 with a deadline of 7/11 -- estimated at 1 day 9 hours each.

185 M Queens with a deadline of 7/11 -- estimated at 16 hours each.

177 ECM and Siever with a deadline of 7/11 with estimates under 9 hours.

Logs only go back as far as 4 am, but show that no more than 3 new tasks were requested with each upload of a completed task -- so this bulk download must have happened between midnight and 4am...

GPU tasks still behaving as they should...

BOINC reports that it is using Local prefs, and that I can chose to use the web preferences at rechenkraft.net
Bild

Benutzeravatar
gemini8
Vereinsvorstand
Vereinsvorstand
Beiträge: 5898
Registriert: 31.05.2011 10:30
Wohnort: Hannover

Re: Too many work units

#10 Ungelesener Beitrag von gemini8 » 02.11.2020 11:03

Local preferences should be safe.
This is quite the interesting behaviour.
To me it seems that Boinc isn't able to calculate the estimated work-hours properly, or that there's something wrong with communication between client and server.
Someone who's more into the technical side might be able to advise the use of special logging flags in order to pinpoint the problem.

But, in case this goes on, the tasks should be killed when arriving at the deadline without having started.

Apart from challenges I usually crunch a mix of projects on my machines, which works out nicely.
And I don't remember problems like this, except for cases in which a server reset my prefs and gave me cpu work instead of gpu work.
Gruß, Jens
- - - - - -
Lowend-User und Teilzeit-Cruncher

Bild Bild Bild
Bild

orpy
Mikrocruncher
Mikrocruncher
Beiträge: 26
Registriert: 12.09.2020 10:11

Re: Too many work units

#11 Ungelesener Beitrag von orpy » 02.11.2020 11:23

This machine's RAC has been poor for the last few months -- performing no better than 12 core i7 PCs when it has 20 cores. I think that it must have been clogged with really long WUs and many of them going over deadline.

I've aborted all work that isn't in progress and have switched projects so that the freed cores work on something else.

First core has switched to SRBase and is behaving as I'd expect -- 1 WU processing and two more queued up.

I'll watch and see what happens and then reset the Yoyo project tomorrow once the remaining work has finished.

The server should have a realistic maximum set for work that has been delivered.

Delivering more than 50 days work to a single host over a 4 hour period is excessive.
Bild

Benutzeravatar
yoyo
Vereinsvorstand
Vereinsvorstand
Beiträge: 8045
Registriert: 17.12.2002 14:09
Wohnort: Berlin
Kontaktdaten:

Re: Too many work units

#12 Ungelesener Beitrag von yoyo » 02.11.2020 14:01

Which hostid is it?
HILF mit im Rechenkraft-WiKi, dies gibts zu tun.
Wiki - FAQ - Verein - Chat

Bild Bild

Antworten

Zurück zu „Number crunching“