AMD CPU Unable to Boot after CPU Microcode Update

Symptoms

• The server with AMD EPYC CPU  unable to boot. We've seen only 2nd generation of EPYC affected (also known under the name Rome). Most likely, you'll have a black screen with no errors right after the grub menu as a sign that something went wrong at the early boot stage. To check your CPU family and model run:

[root@server]$ egrep 'family|model|stepping' /proc/cpuinfo | head -4 
cpu family : 23 
model : 49 
model name : AMD EPYC 7282 16-Core Processor 
stepping : 0

• You have recently updated your CloudLinux machine including linux-firmware package (check /var/log/yum.log):

Jun 02 12:34:54 Updated: linux-firmware-20191203-76.gite8a0f4c.el7.noarch

Root Cause

It is a bug in the Linux kernel microcode driver that causes system to freeze during boot when loading the CPU microcode very early at boot (known as early_microcode). So, the recommended way to handle this is to load microcode updates at boot by disabling early_microcode. We also have a case raised on our side to investigate this behavior.

Solution

Create /etc/dracut.conf.d/microcode.conf with the contents

early_microcode=no

Rebuild the initramfs images with:

[root@server ~]# dracut -f

After this verify that initramfs image doesn't contain Early Microcode section at the beginning:

[root@server]$ lsinitrd |head -10
Image: /boot/initramfs-3.10.0-962.3.2.lve1.5.35.el7.x86_64.img: 21M
========================================================================
Version: dracut-033-568.el7

Arguments: -f --add-drivers 'raid1'

dracut modules:
bash
nss-softokn
i18n

 And reboot the server.