Benutzer Diskussion:Yoyo/yoyo@home

Aus Rechenkraft
Zur Navigation springen Zur Suche springen
TODO for server upgrade
  • purge DB
  • backup svn working directory
  • backup DB
  • backup yoyo server
  • update svn, configure, rebuild
  • upgrade yoyo server
  • watch logfiles for errors ;)

phases

  1. Phase I: setup of Boinc server
  2. Phase II: wrapping of Distributed.net OGR
  3. Phase III: wrapping of Evolution@home
  4. Phase IV: wrapping of Muon
  5. Phase V: wrapping of ecm for various projects

nice projects ;)

  • sr1/2sieve

Then those factor are removed from sieve file. After that you can start LLR. Syntax is - srfile name of sieve file -f factors.txt ( or any other name where factors are stored) If you need more info I will be glad to help So it looks something like this srfile sr_332.abcd -k factors.txt -a sieve file can be on abcd or in npg extension ( it depends is it multi or just one sequence, npg is just one sequence) and last -a say that outuput is also in abcd format, if you put -g will be in npg format (edited) 

svn checkout svn://scm.gforge.inria.fr/svnroot/ecm/trunk ecm
cd ecm
libtoolize
autoheader
aclocal
automake -c -a
autoconf
# Linux 64 bit
./configure ABI=64 CC="gcc -std=gnu99" CFLAGS="-W -Wall -Wundef -O3 -pedantic -fomit-frame-pointer -m64 -mtune=core2 -march=core2"
# aarch64
./configure CC="gcc -std=gnu99" CFLAGS="-W -Wall -Wundef -O3 -pedantic -fomit-frame-pointer -march=armv8-a+fp+simd"
# arm32
./configure CC="gcc -std=gnu99" CFLAGS="-W -Wall -Wundef -O3 -pedantic -fomit-frame-pointer"
make LDFLAGS="--static -static-libgcc"
strip ecm
make check

# Mac
glibtoolize; autoheader; aclocal; automake -c -a; autoconf
./configure ABI=64 CC="gcc -std=gnu99" CFLAGS="-W -Wall -Wundef -O3 -pedantic -fomit-frame-pointer -arch x86_64 -mtune=core2 -march=core2" --with-gmp=./gmp/
make -j 2; strip ecm; make check
mv ecm ecm64
./configure ABI=32 CC="gcc -std=gnu99" CFLAGS="-W -Wall -Wundef -O3 -pedantic -fomit-frame-pointer -arch i386 -mtune=core2 -march=core2" --with-gmp=./gmp/
make -j 2; strip ecm; make check
mv ecm ecm32

odlk9

Questions Answers
How to call the application, commandline? generator_kf_odlk9 /a
How many and which Input-, Outputfiles ? Inputfiles: start.txt, end.txt, config.txt, Outputfiles: output.txt
What is a the workunit? start.txt, end.txt, config.txt
What is in the result? output.txt
Project description, what does the project do?
Is source code available? -> make a Boinc app, include Boinc API calls Source code is available, wrapper or Boinc app possible
progress indicator, e.g. write a progress.txt with values 0..1 ? no
checkpoints available? yes, kind of, last found square is written to start.txt, so start.txt is the checkpoint file
runtime prediction, best between 1h - 5h unpredictable, up to weeks
memory prediction (everything below 1M is fine) some KB
How do we validate results? quorum=2
How do we generate workunits?
What should be done with the results?
badges
found solutions in db

new subproject

Project description, what does the project do? The project generates sieve files for sequences k*b^n+/-1 up to b=1030. This project works for CRUS and SRBase.
Is source code available? -> make a Boinc app, include Boinc API calls Es gibt Anwendungen für Win und Linux (jeweils 32/64 bit) Anwendungen sind hier zu finden: http://www.rieselprime.de/dl/CRUS_pack.zip (nur dort), src: https://sites.google.com/site/geoffreywalterreynolds/programs/sr1sieve
How to call the application, commandline? Die Anwendung ist command line only, es werden keine Adminrechte zur Ausführung benötigt.
./sr1sieve-x86_64-linux -P 96199999999999 -i sieve-sierp-base17-2M-5M.txt -o out -f factor
How many and which Input-, Outputfiles ? Es wird 1 Inputdatei geben sowie eine Output. Mein Konzept sieht es vor die Outputdatei sofort als Input für die nächte Range zu nutzen.
progress indicator, e.g. write a progress.txt with values 0..1 ? Progress indicator gibts es, aber nicht in dieser Boinc Form.
checkpoints available? Nein
What is in the workunit? Die erste Zeile enthält das pmin (untere Sieblimit) sowie Angaben zur Base. Die weiteren Zeilen sind Primzahlkanidaten. Beispiel: http://www.noprimeleftbehind.net/crus/sieve-sierp-base17-2M-5M.txt
What is in the result? Das result ist Inputfile-Factors found=output-File.
runtime prediction, best between 1h - 5h 4-5h
memory prediction (everything below 1M is fine) 1,4 MB
How do we validate results? quorum=2?
How do we generate workunits?
What should be done with the results?
badges
found solutions in db

hints

iptables -A INPUT -m tcp -p tcp -m recent --dport 22 --rcheck --seconds 30 --hitcount 2 --name sshlist --rsource -j LOG --log-prefix "SSH ATTACK: "
iptables -A INPUT -m tcp -p tcp -m recent --dport 22 --rcheck --seconds 30 --hitcount 2 --name sshlist --rsource -j DROP
iptables -A INPUT -m state --state NEW -m tcp -p tcp -m recent --dport 22 --set --name sshlist --rsource -j ACCEPT

mount /mnt/sda2
mount /mnt/sdb2
mount /mnt/sdb3
rsync -xavHD --delete /mnt/sda2/ /mnt/sdb2
/boot/grub/menu.cfg

android adt bundle
./android
./emulator -avd yoyo
./avd push <local file> <path on android>
./avd shell

yoyo ToDo

I develop the project in my spare time, means some hours a day. Therefore I restrict me to the following topics:

ToDo Result
ECM for Android
ecm p1 and p2 app for higher B1

ecm for arm & arm64 wrapper ogr for arm (Debian Boinc App Examples)
ARM Hard Float (armhf)
little-endian ARM (arm) = ARM EABI = armel
http://www.memetic.org/raspbian-benchmarking-armel-vs-armhf/

done
wrapper: progress bar improvement for ogr and ecm

wrapper ogr send checkpoint info to boinc

muon screensaver: show something useful Projekt gestoppt
mail bei erreichen eines badges
ecm screensaver
evo - screensaver schreibfehler behoben, neu kompilieren Projekt gestoppt
Have a look into ecmclient source to see the protocol to ecmserver. With this we can connect automated to more ecmserver very easy.

Other option would be to modify ecmclient, so ecm is no more automated started, instead the iformation should be written into a file. Also the result of ecm should be read from file and transfered to ecmserver.

stats for each application 14.04.10
ecm andere b1 etablieren 14.04.10
additional badges 04.02.10
cancel remaining ecm wus if factor was found 27.09.09: update workunit set assimilate_state=2

<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255) </message> <stderr_txt> ecmwrapper_0.01_i686-pc-linux-gnu: /lib/libgcc_s.so.1: version `GCC_4.2.0' not found (required by ecmwrapper_0.01_i686-pc-linux-gnu)

</stderr_txt> ]]>

28.12.08: use -static-libgcc

<core_client_version>5.10.28</core_client_version> <![CDATA[ <message> process exited with code 195 (0xc3, -61) </message> <stderr_txt> wrapper: starting wrapper: running ecm (-v -nn -timestamp -chkpnt checkpnt -inp in 3000000) ecm: error while loading shared libraries: libgmp.so.3: cannot open shared object file: No such file or directory app exit status: 0x7f00 called boinc_finish

</stderr_txt> ]]>

28.12.08: needs to compile static with libgmp.a instead with libgmp.so

./configure --enable-fat --with-gmp=/usr/

page about the crew who helped the project 29.08.08: The yoyo@home crew
  • Restart der evo crashed, sollte von vorne anfangen
  • Mac version
  • Linux version
  • 64 bit versions
 26.05.08: solved since some weeks
validator for quorum=1 27.12.07: credits now based on calculated GigaIndividuals
send work back to author 21.12.07: now done automatic once a day
stats for OGR and Evolution@home 25.11.07: I added information on the server status page to see for which application work units are available.
evo: wrapper with progress indicator 23.11.07: implemented in 1.02 by cody. Since estimated runtime is very rough, progress indicator maybe also wrong and wu finishes already at 50%.
dnet account nutzbar machen, so dass wus auf diesem verrechnet werden. 26.05.08: on dnet stats each yoyo@home user is now listed individuell
<stderr_txt>

wrapper_1.04_i686-pc-linux-gnu: /lib/libgcc_s.so.1: version `GCC_4.2.0' not found (required by wrapper_1.04_i686-pc-linux-gnu)

13.08.07: Version 1.05 für Linux jetzt statisch gelinkt mit "-static-libgcc"
server umzug, in welchen Schritten was? 13.08.07: erledigt
legacy Application einbinden
work_generator 13.08.07: läuft jede Stunde und erzeugt z.Z. 2x20 Boinc WUs
wrapper + client 03.08.07: läuft
assimilator 03.08.07: sample_assimilator läuft gut.
validator -> simple_bitwise reicht evtl. 03.08.07: Läuft prima.
Wie erzeugt man eine neue Applikation? 01.08.07: project.xml + xadd
Wie führt man fixed Credits ein? 21.07.07: Der WU kann man fixed credits mitgeben. Dem Validator muss man dann noch sagen, dass er die Credits von der WU nehmen soll (-credit_from_wu).
Wie erzeugt man eine neue Applikationsversion? Evtl. mit statisch gelinkten Libs verbinden. 17.07.07: Windows Applikation eingebunden und Linux Applikation neu übersetzt.
robots.txt 17.07.07: Ist jetzt drin, da müßten jetzt die google/yahoo bots nicht mehr alles lesen.
gui_urls.xml 17.07.07: Im BOINC Manager sind jetzt zusätzliche Buttons.
Wie sorgt man dafür dass das Quorum von verschiedenen Nutzern berechnet wird und nicht alles bei Einem? 17.07.07: Ein Schalter in der Projekt Konfig erledigt das.

Mithilfe ToDo

Wer gern helfen möchte ist bei folgenden Themen herzlich eingeladen, ich kann mich aus Zeitgründen darum zunächst nicht kümmern.

ToDo Ergebnis
  • Mac/win98 Wrapper meldet keine CPU Zeit zurück.
Solaris version 26.05.08: Thanks to Dotsch for the Solaris version
project bilder für simple gui, wie bindet man die ein? 26.05.08: Thanks to Rebirther for the pictures
Assimilator, der credits gemäß GNodes vergibt. 16.09.07: Validator ruft jetzt dnetc auf um die stats units (GNodes) zu ermitteln die werden zur Creditberechnung benutzt. credit=GNodes/5.
Suspend, Boinc beenden funktioniert nicht richtig und damit wahscheinlich auch keine Checkpoints. dnet mit -ckpoint <fname> -pausefile <fn> -exitfile <fn> aufrufen und das pause-/exitfile aus dem Wrapper erzeugen bei den entsprechenden Aktionen. Wenn der Wrapper startet diese löschen. 31.08.07 1.07 macht das.
Fortschrittsanzeige: Wenn man in die job.xml 4 Tasks packt dann sollte die Fortshrittsanzeige zumindest 0% - 25% - 50% - 75% anzeigen. Also statt 4 OGR in eine Boinc WU, 1 OGR = 1 Task in job.xml erzeugen. 29.08.07 1.06: 8 OGR sind jetzt in einer Boinc WU und Fortschrittsanzeige geht 0% - 12,5% - 25% -....
Eigenen Server aufsetzen mit Debian + Boinc Server Software. 13.08.07: jetzt auf rkn/yoyo
Ich bräuchte jemanden, der unter Win übersetzen kann! Der Boinc wrapper wird für mein Projekt benötigt. 03.08.07: Danke Cody.
Uppercase für Mac 21.07.07: Danke an scsimodo, der für Mac compiliert hat. Den wrapper brauch ich später auch noch.
Uppercase für Windows 17.07.07: Dank tutta von boinc.be ist jetzt auch Uppercase unter Windows verfügbar.

dnetc for Android

  • Setup the toolchain
    • download the NDK, extract it
    • set $NDKROOT to the directory where you extracted it
    • create the Android tool chain with
export ANDROIDTC="$HOME/android-tc"
$NDKROOT/build/tools/make-standalone-toolchain.sh --platform=android-16 --arch=arm --install-dir=$ANDROIDTC
  • set environment for dnetc compilation
export ANDROIDTC="$HOME/android-tc"
export TCBINARIES="$ANDROIDTC/bin"
export TCINCLUDES="$ANDROIDTC/arm-linux-androideabi"
export TCSYSROOT="$ANDROIDTC/sysroot"
export STDCPPTC="$TCINCLUDES/lib/libstdc++.a"
export PATH="$PATH:$TCBINARIES:$TCINCLUDES/bin"
  • apply the patches (bug#4450) to the dnetc sources
cd dnetc
patch -p1 ../patch2
  • configure & make dnetc
./configure android-arm-eabi
make
  • to run the emulator:
    • download the Android SDK
    • download the Android versions with sdk/tools/android
    • create a Android Virtual Device (AVD) with sdk/tools/android avd
    • run the emulator for this device
sdk/tools/emulator -avd <your avd name>
    • copy files to android
sdk/platform-tools/adb push /<dir>/<file> /data/data
    • run a shell in Android emulator
sdk/platform-tools/adb shell
    • run dnetc in this shell
cd /data/data; 
./dnetc -cpuinfo
dnetc -stress

race prep

  • optimze large tables: done
  • reduce ogr deadline: 1728000 -> 432000
  • reduce max wus in progress
  • delete results fast: 1 day

Boinc for QNAP

  • mit ipkg optware-devel installieren, dann sind auch sed und gawk dabei
  • /opt/share/automake-1.12/Automake/ $perl_threads=0 (automake --version muss funktionieren)
export AUTOCONF=`which autoconf`
export PATH=/opt/bin:$PATH
./_autosetup
./configure --disable-manager --disable-server --with-boinc-platform=arm-unknown-linux-gnueabi --with-boinc-alt-platform=arm-linux-gnu
make
  • Fehler in zlib.h (zlib 1.2.5: Einfach erstes des doppelten Auftretens auskommentiert)

SQL Stuff

Delete a user

https://boinc.berkeley.edu/trac/wiki/RightToErasure 

 set @id=342092;
 --    user_submit - ( delete from user_submit where user_id = ? )
 --    user_submit_app - ( delete from user_submit_app where user_id = ? ) * badge_user (delete from badge_user where user_id = ?)
 --    banishment_vote (delete from banishment_vote where userid = ?)
 --    credit_user (delete from credit_user where userid = ?)
   delete from credited_job where userid = @id;
   delete from donation_paypal where userid = @id;
   delete from forum_logging where userid = @id;
   delete from forum_preferences where userid = @id;
 --    friend (delete from friend where user_src = ? or user_dest = ?)
 --    host_app_version (delete from host_app_version where host_id in (select id from host where userid = ?) )
   delete from msg_from_host where hostid in (select id from host where userid = @id);
   delete from msg_to_host where hostid in (select id from host where userid = @id);
   delete from host where userid = @id;
 --    notify ( delete from notfiy where userid = ? )
   delete from post_ratings where post in ( select id from post where user = @id );
   delete from post_ratings where user = @id;
   update post set parent_post = 0 where parent_post in (select id from post where user = @id );
   delete from post where user = @id;
   update thread set owner = 0 where owner = @id;
   delete from private_messages where userid = @id or senderid = @id;
   delete from sent_email where userid = @id;
   delete from subscriptions where userid = @id;
 --    team_admin ( delete from team_admin where userid = ? )
 --    team_delta ( delete from team_delta where userid = ? )
   update team set userid = 0 where userid = @id;
 --    token ( delete from token where userid = ? )
   delete from user where id = @id;

Change appid for unsent results

set @from=12;
set @to=11;
update `result` set appid=@to WHERE appid=@from and server_state=2 limit 1000;
update `workunit` set appid=@to WHERE appid=@from and id in ( select workunitid from result where appid=@to );

Extend deadline for some results

update result set server_state=2, outcome=0, report_deadline=report_deadline+5*24*3600 where id in (37609410, 37609495)

XML Replace

UPDATE workunit SET xml_doc = REPLACE(xml_doc, "<credit>4</credit>", "<credit>600</credit>") where appid=55;
update workunit SET xml_doc = REPLACE(xml_doc, "</file_ref>", "</file_ref>\n<credit>401</credit>") where appid=55 and assimilate_state=0 and xml_doc not like '%credit%';

tbc

Abgerufen von „https://www.rechenkraft.net/wiki/index.php?title=Benutzer_Diskussion:Yoyo/yoyo@home&oldid=37013“