A baffling spot of localised cooling

How do you keep your cool in this sort of weather? Well, there's various options, but I'll bet one you've not tried is wrapping up in lots of insuating foam.

And yet, that's been just the ticket for some worker nodes up here; despite it being one of the warmer days (23° C outside). Have a look at the temperature graph, and see if you can spot when something changed:
(The peak at midnight was due to a sneak attack Hammercloud; it was just before 12 when I put in the insulation.)

I'd discovered that there's some empty head space at the top of the racks. In those racks were there's a network switch at the top, this wasn't doing much, but where there were worker nodes, the top node was a lot hotter than the node two down from it. That's a lot sharper change than I'd expected - it was noticeable by touching the metal cover on the front of the nodes. The theory was that hot air out the back of the nodes was being sucked forward over the top of the highest node (through the headspace), and then recirculated round, getting hotter, until the steady state of it was about 5 K hotter that the others.

So, it was time to do something about that. First couple attempts at stopping up the gap didn't have much effect, until I dug out a few bits of packing foam (that the nodes were shipped in). Being, of course, the correct width, and jut a bit taller than 1U, they fit snugly into the headspace.


And that foam baffle reduced the temperature; to the point that the node at the top of the racks are now at the lowest temperature since records began! (i.e. they were installed.) Counter intuitive, but that's the way air/heat flow goes sometimes.

Although these worker nodes are due for replacement, we're going to be reusing the racks themselves, so little things like this are good to know. It may be that this won't be a problem with the new worker nodes - or it might be the case that it'd be worse. Either way, forewarns is fore armed (and cooler).