The only alternative I see, is to use a traditional server on a Desktop computer with all its downsides.
A plug computer or a "traditional" server are two extreme cases. There is a lot in between. There are dozens of so called SBC (Singe-Board-Computer) with low power consumption and x86 compatible CPU (and FPU). They are rather expensive compared to a plug computer, but still cheaper than a desktop (depends on the configuration, of course).