Skip to content

Personal Site

Bhyve VM Creation

Type to start searching

Personal Site

Home

Lean

Modern Python

Old Posts

Old Posts

FreeBSD Jail Creation

Bhyve VM Creation

Bhyve VM Creation

Table of contents

Bhyve Initial Setup

New VM Creation Script

Creating a Linux guest

pfSense in a VM

Multiple VMs using bhyve

vm-bhyve

Compiling NGINX with ChaCha20 support

NSD and Unbound config

Saltstack install and config

PostgreSQL 10.1 with replication

Building Go with Bazel

RQLite

Signify

NGINX

Quick Multi-Node Kubernetes Cluster

Serverless

Caching freebsd-update and pkg files

Pi Bitwarden

Gemini

Puppet Bolt

Microserver NAS

LXC/LXD Containers

Find a New Programming Language

Firecracker

Blue/Green deploymetns with Puppet, NGINX and FreeBSD Jails

Bonded NICs on Ubuntu

Basecamp books

Table of contents

Bhyve Initial Setup

New VM Creation Script

Creating a Linux guest

pfSense in a VM

Multiple VMs using bhyve

vm-bhyve

Bhyve VM Creation¶

Bhyve Initial Setup¶

Enable the tap interface in /etc/sysctl.conf and load it on the currently running system

net.link.tap.up_on_open=1
sysctl -f /etc/sysctl.conf

Enable bhyve, serial console and bridge/tap interface kernel modules in /boot/loader.conf. Reboot to apply changes or use kldload.

vmm_load="YES"
nmdm_load="YES"
if_bridge_load="YES"
if_tap_load="YES"

Set up the network interfaces in /etc/rc.conf

cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="addm re0 addm tap0"

Create a ZFS volume

zfs create -V16G -o volmode=dev zroot/testvm

Download the installation image

fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-disc1.iso

Start the VM

sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 512M -t tap0 -d /dev/zvol/zroot/testvm -i -I FreeBSD-11.1-RELEASE-amd64-disc1.iso testvm

Install as normal, following the menu options

New VM Creation Script¶

#! /bin/sh
read -p "Enter hostname: " hostname
zfs create -V16G -o volmode=dev zroot/$hostname
sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 512M -t tap0 -d /dev/zvol/zroot/$hostname -i -I ~/FreeBSD-11.1-RELEASE-amd64-disc1.iso $hostname

Creating a Linux guest¶

Create a file for the hard disk

truncate -s 16G linux.img

Create the file to map the virtual devices for kernel load

~/device.map

(hd0) /root/linux.img
(cd0) /root/linux.iso

Load the kernel

grub-bhyve -m ~/device.map -r cd0 -M 1024M linuxguest

Grub should start, choose install as normal

Start the VM

bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,/root/linux.img -l com1,/dev/nmdm0A -c 1 -m 512M linuxguest

Access through the serial console

cu -l /dev/nmdm0B

pfSense in a VM¶

Download the pfSense disk image from the website using fetch

fetch https://frafiles.pfsense.org/mirror/downloads/pfSense-CE-2.3.1-RELEASE-2g-amd64-nanobsd.img.gz -o ~/pfSense.img.gz

Create the storage

zfs create -V2G -o volmode=dev zroot/pfsense

Unzip the file, and redirect output to the storage via dd

gzip -dc pfSense.img.gz | dd of=/dev/zvol/zroot/pfsense obs=64k

Load the kernel and start the boot process

bhyveload -c /dev/nmdm0A -d /dev/zvol/zroot/pfsense -m 256MB pfsense

Start the VM

/usr/sbin/bhyve -c 1 -m 256 -A -H -P -s 0:0,hostbridge -s 1:0,virtio-net,tap0 -s 3:0,ahci-hd,/dev/zvol/zroot/pfsense -s 4:1,lpc -l com1,/dev/nmdm0A pfsense

Connect to the VM via the serial connection with nmdm

cu -l /dev/nmdm0B

Perform initial configuration through the shell to assign the network interfaces

Once done, use the IP address to access through the web console

When finished, you can shutdown/reboot

To de-allocate the resources, you need to destroy the VM

bhyvectl --destroy --vm=pfsense

Multiple VMs using bhyve¶

To allow networking on multiple vms, there should be a tap assigned to each vm, connected to the same bridge.

cloned_interfaces="bridge0 tap0 tap1 tap2"
ifconfig_bridge0="addm re0 addm tap0 addm tap1 addm tap2"

Then when you provision vms, assign one of the tap interfaces to them.

vm-bhyve¶

A better way for managing a bhyve hypervisor.

Follow the instructions on the repo.

When adding the switch to a network interface, it doesn't work with re0. tap1 works, but then internet doesnt work in the VMs. Needs sorting.

zfs

bsd-cloud-init should be tested, it sets hostname based on openstack image name.

otherwise, if we figure out how to make a template VM, you could set the hostname as part of transferring over the rc.conf file

create template VM, start it, zfs send/recv?

Made with

Material for MkDocs