Using bhyve (BSD hypervisor)

BHyVe is a type-2 hypervisor developed on FreeBSD. Its similar to KVM, and a different approach to jail/lxc containers.

Requires Intel VT-x and Extended Page Tables (EPT). BIOS/UEFI support is in progress. Minimal device emulation support (virtio-blk, virtio-net).

Guest OS supported are FreeBSD/amd64 using bhyveload@man and OpenBSD/Linux using grub2-bhyve.

It uses bhyve@man a userland part of hypervisor that emulates devices, bhyvectl as a management tool, libvmmapi a userland api (wrapper library of /dev/vmm operations) and vmm.ko a kernel part of hypervisor.

Preparing host

# detect vmx
$ dmesg | grep vmx
vmx_init: processor does not support VMX operation

# load kernel module
$ kldload vmm
# create tap device and bridge network interface
$ ifconfig tap0 create
$ sysctl
$ ifconfig bridge0 create
# add lan 'em0' and tap interfaces to the bridge
$ ifconfig bridge0 addm em0 addm tap0
$ ifconfig bridge0 up

## persistent configuration: start bhyve guests at boot time
$ vi /etc/sysctl.conf
$ vi /boot/loader.conf
$ vi /etc/rc.conf
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="addm igb0 addm tap0"

## delete/destroy network interface
$ ifconfig bridge0 down
$ ifconfig bridge0 deletem em0 deletem tap0
$ ifconfig bridge0 destroy

Creating guests

# load a guest
bhyveload -m ${mem} -d ${disk} ${name}
# run it
bhyve -c ${cpus} -m ${mem} -s 0,hostbridge -s 2,virtio-blk,${disk} 
  -s 3,virtio-net,${tap} -s 31,lpc -l com1,stdio vm0
  • FreeBSD
$ fetch

# create virtual disk
$ truncate -s 16G guest.img
# '' shell wrapper that loads VM and starts it in a loop
$ /usr/share/examples/bhyve/ -c 4 -m 1024M -t tap0 
  -d guest.img -i -I FreeBSD-10.1-RELEASE-amd64-bootonly.iso guestname
# install guest and in the end enter shell
# (prior to 10.0 only) edit '/etc/ttys' and 
  ttyu0 "/usr/libexec/getty 3wire" xterm on secure
# reboot

# to start guest from virtual disk do
$ /usr/share/examples/bhyve/ -c 4 -m 1024M -t tap0 
  -d guest.img guestname
  • OpenBSD
$ fetch
$ bunzip2 flashimg.amd64-20131014.bz2

# install port grub2-bhyve
$ cd /usr/ports/sysutils/grub2-bhyve
$ make install clean
# create tap device and add-it to bridge
$ ifconfig tap1 create
$ ifconfig bridge0 addm tap1

# create a grub device map
$ vi
(hd0) ./obsd.img
# boot/load image
$ grub-bhyve -m -r hd0 -M 512 obsd
# on boot prompt type
kopenbsd -h com0 (hd0,openbsd1)/bsd
# start guest vm, use root:test123
$ bhyve -c 2 -m 512M -A -H -P -s 0:0,amd_hostbridge -s \
  1:0,lpc -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,obsd.img \
  -l com1,stdio -W obsd
# cleanup
$ bhyvectl --destroy --vm=obsd

see create USB flash installer for OpenBSD

  • Linux
$ fetch \
  -o somelinux.iso

# create virtual disk
$ truncate -s 16G linux.img
# create grub device map
$ vi
(hd0) ./linux.img
(cd0) ./somelinux.iso
# boot/load image
$ grub-bhyve -m -r cd0 -M 1024M linuxguest
# install guest and reboot
$ bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap1 \
  -s 3:0,virtio-blk,./linux.img -s 4:0,ahci-cd,./somelinux.iso \
  -l com1,stdio -c 4 -m 1024M linuxguest
# after install stop guest
$ bhyvectl --destroy --vm=linuxguest

# now can start from virtual disk, and boot
$ grub-bhyve -m -r hd0,msdos1 -M 1024M linuxguest
$ bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap1 \
  -s 3:0,virtio-blk,./linux.img -l com1,stdio -c 4 -m 1024M linuxguest

see FreeBSD as a Host with bhyve and Virtualization with bhyve @ BSDNow.

Virtual Machine Consoles

# use null modem device *nmdm* kernel module to wrap the bhyve console
$ kldload nmdm
$ bhyve -AI -H -P -s 0:0,hostbridge -s 1:0,lpc \
  -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./linux.img \
  -l com1,/dev/nmdm0A -c 4 -m 1024M linuxguest

# use 'cu' to attach/reattach to the console
$ cu -l /dev/nmdm0B -s 9600

Managing Virtual Machines

# device node is created in /dev/vmm for each virtual machine
$ ls /dev/vmm
guestname linuxguest

# destroy a VM
$ bhyvectl --destroy --vm=guestname

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s