Spectre and Meltdown Protection - Performance

Everything about the project RNA World

Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 03.03.2018 22:54

I recently picked up Intel's new CPU Microcode update, that provides Spectre protection for my PC. And I found a way to test its performance, on an RNA World task! You can easily test performance for both the Spectre and the Meltdown protections.

Basically, I did the following steps on a backup clone of a VM.
Do NOT try to do these on a running task! Use a clone, if you decide to test!

RESULTS:
Spectre Protection costs 20% in performance, for a VirtualBox VM!
Meltdown Protection doesn't cost anything in performance.

METHODOLOGY:

On Host:
[*]Set the host protection settings, by running InSpectre as Administrator, and setting Enable/Disable, then restarting Windows whenever you make a settings change.
https://www.grc.com/inspectre.htm

On Guest (Backup clone of a VM):
[*]Kill the PID of the cmsearch process
[*]Shutdown the Guest
[*]Edit the VM settings, to Enable network adapter 1, for NAT
[*]Start the Guest
[*]Hit Ctrl+C when it starts the BOINC script.
[*]Update the apt-get packages with:
apt-get update
[*]Install sysbench with:
apt-get install sysbench
[*]Run sysbench on the Guest, with:
sysbench --test=cpu --cpu-max-prime=20000 run

Useful keys:
= is shift+0
- is only on numpad
y is z

DETAILS:
Code: Select all
20180303: Meltdown Protection: ON, Spectre Protection: ON
test 1 total time: 26.4573
test 2 total time: 26.4192
test 3 total time: 26.4090

20180303: Meltdown Protection: OFF, Spectre Protection: ON
test 1 total time: 26.4010
test 2 total time: 26.5037
test 3 total time: 26.4018

20180303: Meltdown Protection: ON, Spectre Protection: OFF
test 1 total time: 21.2456
test 2 total time: 21.3365
test 3 total time: 21.2588

20180303: Meltdown Protection: OFF, Spectre Protection: OFF
test 1 total time: 21.2296
test 2 total time: 21.2345
test 3 total time: 21.3814


I will be running with both protections ON, on this machine... even though tasks will take 20-25% more time to complete. It is what it is.

Regards,
Jacob Klein
Last edited by Jacob Klein on 04.03.2018 12:54, edited 1 time in total.
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby gemini8 » 04.03.2018 08:40

Hi Jacob.
Thanks for testing this!
I think this might be something spoiling your fun (although the fun will be prolonged... :biggrin5: ).
Gruß, Jens
- - - - - -
Lowend-User und Teilzeitcruncher

Image Image
Image
User avatar
gemini8
Vereinsmitglied
Vereinsmitglied
 
Posts: 2487
Joined: 31.05.2011 10:30
Location: Hannover

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 04.03.2018 12:55

It was a fun test that I had been wanting to run, just had to find out how.
And yes, tasks will take longer, but even that is "prolonged fun" as you put it! :)
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 05.03.2018 01:13

Damn. I have some bad news for RNA World VM tasks, and Spectre Protection Performance. :(

RESULTS:
Spectre Protection may make an RNA World VM task take 3 times as long to complete!
Details below.

METHODOLOGY:
Use InSpectre to set protection settings, then restart.
Start BOINC VMs after Poweroff, in order to get forecast estimates.

MY PLAN:
I think I'm going to run with Spectre protection off, while I have BOINC VMs in progress, unless serious exploits are in the wild. I will be continually monitoring that.
Side note: One of my favorite games, Hearthstone, which makes tons of kernel-mode CPU calls, also has a huge performance hit while Spectre Protection is enabled. So, that game will run better too.

SUMMARY:
16200000 / 5450000 = 2.97
20880000 / 7100000 = 2.94

DETAILS:
Code: Select all
Use InSpectre to set protection settings, then restart
Start 2 BOINC VMs after Poweroff, in order to get forecast estimates

Results:

20180304: Meltdown Protection: ON, Spectre Protection: ON
VM 1 test 1: 16256600
VM 1 test 2: 16295118
VM 1 test 3: 16180155
VM 2 test 1: 22070772
VM 2 test 2: 21329382
VM 2 test 3: 21131576

20180304: Meltdown Protection: OFF, Spectre Protection: ON
VM 1 test 1: 16218376
VM 1 test 2: 16218376
VM 1 test 3: 16179857
VM 2 test 1: 20834482
VM 2 test 2: 20884316
VM 2 test 3: 20883932

20180304: Meltdown Protection: ON, Spectre Protection: OFF
VM 1 test 1: 5584102
VM 1 test 2: 5392988
VM 1 test 3: 5545880
VM 2 test 1: 7224541
VM 2 test 2: 6928215
VM 2 test 3: 7224925

20180304: Meltdown Protection: OFF, Spectre Protection: OFF
VM 1 test 1: 5431211
VM 1 test 2: 5392988
VM 1 test 3: 5431211
VM 2 test 1: 6977282
VM 2 test 2: 7323443
VM 2 test 3: 7175088
Last edited by Jacob Klein on 05.03.2018 13:13, edited 1 time in total.
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby gemini8 » 05.03.2018 10:22

That's quite evil!
I wonder if other vm-projects will take a similar heavy blow.
Won't stop me from crunching them.
Btw.: Still waiting for my first monster. Maybe I should step down there... ^^
Gruß, Jens
- - - - - -
Lowend-User und Teilzeitcruncher

Image Image
Image
User avatar
gemini8
Vereinsmitglied
Vereinsmitglied
 
Posts: 2487
Joined: 31.05.2011 10:30
Location: Hannover

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 05.03.2018 13:34

I'd call it unfortunate instead of evil. Who knew branch prediction and speculative execution would be so very important for a CPU-intense VM? :)
Regarding monsters ... Keep searching, you'll eventually find one! :)
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby gemini8 » 06.03.2018 10:03

That's what I'm doing.
Gruß, Jens
- - - - - -
Lowend-User und Teilzeitcruncher

Image Image
Image
User avatar
gemini8
Vereinsmitglied
Vereinsmitglied
 
Posts: 2487
Joined: 31.05.2011 10:30
Location: Hannover

Re: Spectre and Meltdown Protection - Performance

Unread postby Kolossus » 06.03.2018 10:49

I think the easiest way is to remove the weighting patch from the computer.....
Gruß Harald

Wer die Ärmsten dieser Welt gesehen hat, fühlt sich reich genug zu helfen. (Dr. Albert Schweitzer)

Image Image
User avatar
Kolossus
Gridmaster
Gridmaster
 
Posts: 2032
Joined: 26.10.2014 14:51
Location: Köln

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 06.03.2018 13:25

Kolossus wrote:I think the easiest way is to remove the weighting patch from the computer.....

Nope.


Things that won't work:
- I cannot roll back the BIOS. On my Area 51 R2, here, if I try to run older BIOS executables, it detects it's an older version, and halts the installation.
- I will not undo my Windows 10 support for the Microcode solution for Spectre protection. Essentially, that would mean "not taking any more cumulative updates ever", which is not an option. This is my daily driver, I use this PC for everything.
- I cannot force the CPU to use an older microcode, via the VMWare CPU Microcode Update Driver, here: https://labs.vmware.com/flings/vmware-cpu-microcode-update-driver .... when applied, using the older Microcode files from Intel, it simply says in Event Viewer that my microcode is already up to date.

The only option available is:
- Use InSpectre to disable Spectre Protection, if you are sure you want to take the risk in order to regain performance. https://www.grc.com/inspectre.htm

We are lucky that the InSpectre tool exists. Even if it's just changing registry keys, it makes things super clear and easy!

Regards,
Jacob
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby Hesilman » 29.05.2018 12:24

I have no idea how this didn't go into class action lawsuit against Intel and AMD. We just didn't get what we paid for after that patch was applied.
Hesilman
Fingerzähler
Fingerzähler
 
Posts: 1
Joined: 22.05.2018 10:17

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 02.06.2018 17:32

At least InSpectre allows us to disable the protection and keep the performance, if we want !
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Re: Spectre and Meltdown Protection - Performance

Unread postby Jacob Klein » 20.10.2018 12:59

Microsoft recently announced that their Windows 10 19H1 builds, in the Fast Ring, use retpoline and "import optimization", which should negate Spectre v2 mitigation performance degradation.

https://twitter.com/mamyun/status/1052715532883877888
Mehmet Iyigun, Windows/Azure Kernel Team wrote:Yes, we have enabled retpoline by default in our 19H1 flights along with what we call "import optimization" to further reduce perf impact due to indirect calls in kernel-mode. Combined, these reduce the perf impact of Spectre v2 mitigations to noise-level for most scenarios.

https://twitter.com/mamyun/status/1053142866359287813
Mehmet Iyigun, Windows/Azure Kernel Team wrote:It's possible to enable retpoline on stock RS5 since the necessary binary metadata is present in the binaries. However, we also have made a few fixes and perf improvements after RS5 forked, so it's better as a complete package...


I spent some time testing this. But I have bad news.

Conclusion:
Microsoft's Windows 10 19H1 retpoline and "import optimization" changes, DO NOT IMPROVE Spectre v2 mitigation performance degradation, for RNA World VM Tasks.

My testing data:

Code: Select all
Speed, Microcode 3D, VirtualBox 5.2.21 Test Build 125885, Win10 RS5 v1809 (Build 17763.55)
Meltdown Protection ON, Spectre Protection ON
VM 1 test 1: 16333640
VM 1 test 2: 16486235
Meltdown Protection OFF, Spectre Protection ON
VM 1 test 1: 16180748
VM 1 test 2: 16104598
Meltdown Protection ON, Spectre Protection OFF
VM 1 test 1: 5354766
VM 1 test 2: 5354766
Meltdown Protection OFF, Spectre Protection OFF
VM 1 test 1: 5469731
VM 1 test 2: 5354766
====================================================================================
Speed, Microcode 3D, VirtualBox 5.2.21 Test Build 125885, Win10 19H1 (Build 18262)
Meltdown Protection ON, Spectre Protection ON
VM 1 test 1: 16333640
VM 1 test 2: 16142227
Meltdown Protection OFF, Spectre Protection ON
VM 1 test 1: 16371565
VM 1 test 2: 16409788
Meltdown Protection ON, Spectre Protection OFF
VM 1 test 1: 5354766
VM 1 test 2: 5354766
Meltdown Protection OFF, Spectre Protection OFF
VM 1 test 1: 5354766
VM 1 test 2: 5431508
Jacob Klein
Oberfalter
Oberfalter
 
Posts: 427
Joined: 26.07.2013 15:41

Next

Return to RNA World Discussions (english)

Who is online

Users browsing this forum: No registered users and 10 guests