Google cloud instances run on top of KVM hypervisor. In newer KVM versions, such as the one used in Google Cloud, virtio-scsi is used instead of an older virtio-blk storage backend. The virtio-scsi disk devices looks like:
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
In RHEL6, a module is added in 2.6.32-244.el6 kernel. But virtio-scsi is not shipped in RHEL5 or Centos5.
* Tue Feb 28 2012 Aristeu Rozanski <[email protected]> [2.6.32-244.el6] - [virt] virtio-scsi: add power management (Paolo Bonzini) [782027] - [virt] virtio-scsi: add driver (Paolo Bonzini) [782027]
Legacy Centos5/RHEL5 servers moved using Lift and Shift to Google Cloud will fail to boot with an error similar to the following:
Unable to access resume device (LABEL=ROOT) Creating root device. Mounting root filesystem. mount: could not find filesystem '/dev/root' Setting up other filesystems. Setting up new root fs setuproot: Kernel panic - not syncing: Attempted to kill init! moving /dev failed: No such file
Google has graciously backported the kernel code for 2.6.18* kernels. To fix the issue, we need to build and use virtio_scsi driver from backported code.
Build kernel module
Get the backported kernel code to a RHEL5/Centos5 system.
git clone https://github.com/GoogleCloudPlatform/compute-centos5-drivers.git
Get the src package for the kernel version (this might need .redhat.com access)
The following are other locations:
https://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/os/SRPMS/
https://vault.centos.org/5.5/centosplus/x86_64/RPMS/
Install the package.
rpm -ivh kernel-2.6.18-194.el5.src.rpm
Build the kernel source code.
cd /usr/src/redhat/SPECS rpmbuild -bp --target=x86_64 kernel.spec cd /usr/src/redhat/BUILD/kernel-2.6.18/linux-2.6.18.x86_64 make oldconfig make prepare make scripts make modules
Build the virtio_scsi module.
make M=/root/compute-centos5-drivers-master/third_party/virtio_scsi
Install module in the source system
The easiest way to install the module is to upload the module to Google Cloud Storage and use it to build initrd in the source system. Then the source system could be synced to Google Cloud.
Create a new bucket and upload the module:
# gsutil cp 2.6.18-194.el5-virtio_scsi.ko gs://rhel5test/
In the source system, download and copy the module to the relevant kernel module path:
wget https://storage.googleapis.com/rhel5test/`uname -r`-virtio_scsi.ko cp `uname -r`-virtio_scsi.ko /lib/modules/`uname -r`/extra/virtio_scsi.ko
Run depmod to generate modules.dep and map files:
depmod -a
Create initrd file using the virtio modules:
mkinitrd -f --with=virtio_pci --with=virtio_scsi --preload virtio --preload scsi_mod /boot/initrd-gcp-$(uname -r).img $(uname -r)
Update grub.conf with new initrd. For example,
title GCP Red Hat Enterprise Linux Server (2.6.18-194.el5) root (hd0,0) kernel /vmlinuz-2.6.18-194.el5 ro root=UUID=eff36034-b712-468f-8e6e-0816973a866d rhgb edd=off quiet console=ttyS0 initrd /initrd-gcp-2.6.18-194.el5.img
Update the default parameter to boot from the new kernel:
default=1
Once the initrd is built and the grub configuration is made, the system can be synced to Google Cloud.
Both RHEL5 and Centos5 have reached End of Life. It is strongly suggested to move them to later operating system releases. The above workaround is a good method to migrate the systems to the cloud until the application is refactored to run in later operating system releases.
8 Comments. Leave new
Wonderful info.. Helped us migrate EL5 to GCP. Do you have similar idea for EL4?
Sean,
I am glad that this helped you.
RHEL4 uses kernel version 2.6.9* which is very old. AFAIK there is no virtio_scsi backport available for 2.6.9* kernel. It will not be an easy task. May be it is time to re-architect/port the applications running in rhel4 to later operating system versions.
Well, the RHEL5 VM now boots in GCP, but does not take the SSH keys from GCP.
How did you guys solve that?
I guess you are talking about editing public SSH key metadata, I do not remember having any issues.
You can also provision ssh key in the source system and use it after the migration.
Hi, I have build the virtio-scsi drivers 2.6.18-419.el5 , as it is the latest one on my centos 5.4, used link https://github.com/GoogleCloudPlatform/compute-centos5-drivers.
cmd> sudo find /lib/modules/ -iname virtio_scsi*
/lib/modules/2.6.18-419.el5/extra/virtio_scsi
/lib/modules/2.6.18-419.el5/extra/virtio_scsi/virtio_scsi.ko
built the drivers on source itself, i migrated this server on GCP, but it failed to boot.
1. ACPI: Core revision 20060707
ACPI Error (psloop-0196): Found unknown opcode 15 at AML address ffffc20000004ae8 offset 4, ignoring [20060707]
ACPI Error (psloop-0196): Found unknown opcode 15 at AML address ffffc20000004aef offset B, ignoring [20060707]…
2. ACPI: Core revision 20060707
ACPI Error (psloop-0196): Found unknown opcode 15 at AML address ffffc20000004ae8 offset 4, ignoring [20060707]
ACPI Error (psloop-0196): Found unknown opcode 15 at AML address ffffc20000004aef offset B, ignoring [20060707]….
3. system went in hung state
any idea, what is missing here?
Hello Smita.
I haven’t tested with 2.6.18-419.el5.
These errors seems related to ACPI mappings.. I found following bugzilla which fixed the issue in hypervisor level in qemu-kvm package.
https://bugzilla.redhat.com/show_bug.cgi?id=1377087
As in this case host system is managed by Google, you might need to file a ticket with them.
You can also try with a kernel older than kernel-2.6.18-398.
On boot:
virtio_scsi: no version for “struct_module” found :(
This message does not seem to be complete. The error probably happens when module is not built current against kernel. Not sure this check for Red Hat hash check.
Make sure that you are building against right kernel versions and following the instructions correctly.