Re: PROMETHEUS
Posted: Mon Nov 27, 2017 3:21 pm
PROMETHEUS is getting an upgrade. Today I ordered some new hardware, we're maxing out our CPU's.
Current Hardware:
1 x E5-1650 Sandy Bridge-E XEON / 3.2 GHz to 3.8 GHz Turbo / 6 cores / 12 threads / 12MB L3 Cache
1 x Supermicro 4U 48db cooler
New Hardware:
2 x E5-2667 v2 / Ivy Bridge-E XEON's / 3.3 GHz to 4.0 GHz Turbo / 8 Cores / 12 Threads / 25MB L3 Cache each
2 x Noctua 4U 18db coolers / http://noctua.at/en/nh-u9dx-i4
Part Pictures:
This new setup gets us 10 more cores so we'll have 16 cores and 32 threads at our disposal. It also boosts memory bandwidth from 52GB/s to 104GB/s due to moving from Quad Channel to a Octo-Channel memory configuration. It nets us a whopping 50MB of L3 cache up from the 12MB we have right now.
It boosts both our single-thread performance by 5% minimum (likely will be around 7-10% in reality). And it boosts our multithreaded performance by 181% - Yes you read that correctly, 181%. From 11768 points to 33092 points on CPU benchmark tests.
This drastic multithreaded increase is not just due to the 10 extra cores but the extra 38MB of L3 cache which really assists heavily in multithreaded workloads as it's used in core to core data exchange so the more cache you have the more efficiently the cores can work together before needing to access the much slower RAM outside of the CPU's.
This upgrade does come at a cost, it'll be about $809 USD, possibly $909 USD after import duty / taxes as I've had to import these CPU's from America to get them so cheaply.
Usually the CPU's cost £500 each but I managed to get them for £220 each before shipping and import taxes making it about £500 total for both, possibly £600 with import duty and then another £100 for the heatsinks and fans.
I'm really excited about this upgrade. I did consider upgrading us instead to a 1950X 16 core CPU from AMD. However not only does that CPU have lower Single Thread and Multithread performance than two E5-2667 v2's but it also would cost a lot more. £700 for the 1950X on its own + £700 for 64GB of ECC DDR4 Memory + £370 for a motherboard + £60 for a cooler, all that would have come out to £1830 instead of the £600-£700 this will end up costing. So by comparison this is not just an absolute bargain but higher performing too.
I did not do this upgrade for Minecraft, although that is part of it. On our server we have regularly been hitting 100% CPU Usage throughout the day. We mostly sit around 60% CPU usage averaged out with peaks to 100% lasting a few minutes at a time, over a full day that's about an hour or two of 100% CPU load. These high load scenarios are not caused by Minecraft alone, that's only about 25-35% of the problem.
The main thing taking up the rest is all the other stuff the server is doing. Virtual Machines, backing up, downloading, serving my business needs, I run several websites from it one of which serves millions of requests per day. All of this combined has been why the server needed to be upgraded. Due to this mostly being for my personal / business needs I am not going to be placing the cost of this upgrade on our Minecraft donation page, I do not consider this a Minecraft server expense.
I still may break out Minecraft into its own server next year but we'll see how the new dual processor system upgrade handles first. By having two processors and so many cores I am expecting our CPU to turbo up to 3.8-4GHz more often than our current 6 core is able to, thus resulting in higher single thread performance which is what Minecraft craves.
So I hope you found this post interesting, the new processors will be here between December 1st and December 5th.
Current Hardware:
1 x E5-1650 Sandy Bridge-E XEON / 3.2 GHz to 3.8 GHz Turbo / 6 cores / 12 threads / 12MB L3 Cache
1 x Supermicro 4U 48db cooler
New Hardware:
2 x E5-2667 v2 / Ivy Bridge-E XEON's / 3.3 GHz to 4.0 GHz Turbo / 8 Cores / 12 Threads / 25MB L3 Cache each
2 x Noctua 4U 18db coolers / http://noctua.at/en/nh-u9dx-i4
Part Pictures:
This new setup gets us 10 more cores so we'll have 16 cores and 32 threads at our disposal. It also boosts memory bandwidth from 52GB/s to 104GB/s due to moving from Quad Channel to a Octo-Channel memory configuration. It nets us a whopping 50MB of L3 cache up from the 12MB we have right now.
It boosts both our single-thread performance by 5% minimum (likely will be around 7-10% in reality). And it boosts our multithreaded performance by 181% - Yes you read that correctly, 181%. From 11768 points to 33092 points on CPU benchmark tests.
This drastic multithreaded increase is not just due to the 10 extra cores but the extra 38MB of L3 cache which really assists heavily in multithreaded workloads as it's used in core to core data exchange so the more cache you have the more efficiently the cores can work together before needing to access the much slower RAM outside of the CPU's.
This upgrade does come at a cost, it'll be about $809 USD, possibly $909 USD after import duty / taxes as I've had to import these CPU's from America to get them so cheaply.
Usually the CPU's cost £500 each but I managed to get them for £220 each before shipping and import taxes making it about £500 total for both, possibly £600 with import duty and then another £100 for the heatsinks and fans.
I'm really excited about this upgrade. I did consider upgrading us instead to a 1950X 16 core CPU from AMD. However not only does that CPU have lower Single Thread and Multithread performance than two E5-2667 v2's but it also would cost a lot more. £700 for the 1950X on its own + £700 for 64GB of ECC DDR4 Memory + £370 for a motherboard + £60 for a cooler, all that would have come out to £1830 instead of the £600-£700 this will end up costing. So by comparison this is not just an absolute bargain but higher performing too.
I did not do this upgrade for Minecraft, although that is part of it. On our server we have regularly been hitting 100% CPU Usage throughout the day. We mostly sit around 60% CPU usage averaged out with peaks to 100% lasting a few minutes at a time, over a full day that's about an hour or two of 100% CPU load. These high load scenarios are not caused by Minecraft alone, that's only about 25-35% of the problem.
The main thing taking up the rest is all the other stuff the server is doing. Virtual Machines, backing up, downloading, serving my business needs, I run several websites from it one of which serves millions of requests per day. All of this combined has been why the server needed to be upgraded. Due to this mostly being for my personal / business needs I am not going to be placing the cost of this upgrade on our Minecraft donation page, I do not consider this a Minecraft server expense.
I still may break out Minecraft into its own server next year but we'll see how the new dual processor system upgrade handles first. By having two processors and so many cores I am expecting our CPU to turbo up to 3.8-4GHz more often than our current 6 core is able to, thus resulting in higher single thread performance which is what Minecraft craves.
So I hope you found this post interesting, the new processors will be here between December 1st and December 5th.