Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
BHYVE(8)		  BSD System Manager's Manual		      BHYVE(8)

     bhyve -- run a guest operating system inside a virtual machine

     bhyve [-AaCDeHhPSuWwxY]
	   [-c [[cpus=]numcpus][,sockets=n][,cores=n][,threads=n]] [-G port]
	   [-l help|lpcdev[,conf]] [-m memsize[K|k|M|m|G|g|T|t]]
	   [-p vcpu:hostcpu] [-r file] [-s help|slot,emulation[,conf]]
	   [-U uuid] vmname

     bhyve is a	hypervisor that	runs guest operating systems inside a virtual

     Parameters	such as	the number of virtual CPUs, amount of guest memory,
     and I/O connectivity can be specified with	command-line parameters.

     If	not using a boot ROM, the guest	operating system must be loaded	with
     bhyveload(8) or a similar boot loader before running bhyve, otherwise, it
     is	enough to run bhyve with a boot	ROM of choice.

     bhyve runs	until the guest	operating system reboots or an unhandled hy-
     pervisor exit is detected.

     -a		 The guest's local APIC	is configured in xAPIC mode.  The
		 xAPIC mode is the default setting so this option is redun-
		 dant.	It will	be deprecated in a future version.

     -A		 Generate ACPI tables.	Required for FreeBSD/amd64 guests.

     -c	[setting ...]
		 Number	of guest virtual CPUs and/or the CPU topology.	The
		 default value for each	of numcpus, sockets, cores, and
		 threads is 1.	The current maximum number of guest virtual
		 CPUs is 16.  If numcpus is not	specified then it will be cal-
		 culated from the other	arguments.  The	topology must be con-
		 sistent in that the numcpus must equal	the product of
		 sockets, cores, and threads.  If a setting is specified more
		 than once the last one	has precedence.

     -C		 Include guest memory in core file.

     -D		 Destroy the VM	on guest initiated power-off.

     -e		 Force bhyve to	exit when a guest issues an access to an I/O
		 port that is not emulated.  This is intended for debug	pur-

     -G	port	 Start a debug server that uses	the GDB	protocol to export
		 guest state to	a debugger.  An	IPv4 TCP socket	will be	bound
		 to the	supplied port to listen	for debugger connections.
		 Only a	single debugger	may be attached	to the debug server at
		 a time.  If port begins with `w', bhyve will pause execution
		 at the	first instruction waiting for a	debugger to attach.

     -h		 Print help message and	exit.

     -H		 Yield the virtual CPU thread when a HLT instruction is	de-
		 tected.  If this option is not	specified, virtual CPUs	will
		 use 100% of a host CPU.

     -l	[help|lpcdev[,conf]]
		 Allow devices behind the LPC PCI-ISA bridge to	be configured.
		 The only supported devices are	the TTY-class devices com1
		 through com4, the boot	ROM device bootrom, and	the debug/test
		 device	pc-testdev.

		 help print a list of supported	LPC devices.

     -m	memsize[K|k|M|m|G|g|T|t]
		 Guest physical	memory size in bytes.  This must be the	same
		 size that was given to	bhyveload(8).

		 The size argument may be suffixed with	one of K, M, G or T
		 (either upper or lower	case) to indicate a multiple of	kilo-
		 bytes,	megabytes, gigabytes, or terabytes.  If	no suffix is
		 given,	the value is assumed to	be in megabytes.

		 memsize defaults to 256M.

     -p	vcpu:hostcpu
		 Pin guest's virtual CPU vcpu to hostcpu.

     -P		 Force the guest virtual CPU to	exit when a PAUSE instruction
		 is detected.

     -r	file	 Resume	a guest	from a snapshot.  The guest memory contents
		 are restored from file, and the guest device and vCPU state
		 are restored from the file "file.kern".

		 Note that the current snapshot	file format requires that the
		 configuration of devices in the new VM	match the VM from
		 which the snapshot was	taken by specifying the	same [-s] and
		 [-l] options.	The count of vCPUs and memory configuration
		 are read from the snapshot.

     -s	[help|slot,emulation[,conf]]
		 Configure a virtual PCI slot and function.

		 bhyve provides	PCI bus	emulation and virtual devices that can
		 be attached to	slots on the bus.  There are 32	available
		 slots,	with the option	of providing up	to 8 functions per

		 help	     print a list of supported PCI devices.

		 slot	     pcislot[:function]	bus:pcislot:function

			     The pcislot value is 0 to 31.  The	optional
			     function value is 0 to 7.	The optional bus value
			     is	0 to 255.  If not specified, the function
			     value defaults to 0.  If not specified, the bus
			     value defaults to 0.


			     hostbridge	| amd_hostbridge

					 Provide a simple host bridge.	This
					 is usually configured at slot 0, and
					 is required by	most guest operating
					 systems.  The amd_hostbridge emula-
					 tion is identical but uses a PCI ven-
					 dor ID	of AMD.

			     passthru	 PCI pass-through device.

			     virtio-net	 Virtio	network	interface.

			     virtio-blk	 Virtio	block storage interface.

					 Virtio	SCSI interface.

			     virtio-9p	 Virtio	9p (VirtFS) interface.

			     virtio-rnd	 Virtio	RNG interface.

					 Virtio	console	interface, which ex-
					 poses multiple	ports to the guest in
					 the form of simple char devices for
					 simple	IO between the guest and host

			     ahci	 AHCI controller attached to arbitrary

			     ahci-cd	 AHCI controller attached to an	ATAPI

			     ahci-hd	 AHCI controller attached to a SATA

			     e1000	 Intel e82545 network interface.

			     uart	 PCI 16550 serial device.

			     lpc	 LPC PCI-ISA bridge with COM1 and COM2
					 16550 serial ports, a boot ROM, and,
					 optionally, the debug/test device.
					 The LPC bridge	emulation can only be
					 configured on bus 0.

			     fbuf	 Raw framebuffer device	attached to
					 VNC server.

			     xhci	 eXtensible Host Controller Interface
					 (xHCI)	USB controller.

			     nvme	 NVM Express (NVMe) controller.

			     hda	 High Definition Audio Controller.

		 [conf]	     This optional parameter describes the backend for
			     device emulations.	 If conf is not	specified, the
			     device emulation has no backend and can be	con-
			     sidered unconnected.

			     Network backends:




					 If mac	is not specified, the MAC ad-
					 dress is derived from a fixed OUI and
					 the remaining bytes from an MD5 hash
					 of the	slot and function numbers and
					 the device name.

					 The MAC address is an ASCII string in
					 ethers(5) format.

					 With virtio-net devices, the mtu pa-
					 rameter can be	specified to inform
					 the guest about the largest MTU that
					 should	be allowed, expressed in

					 With netgraph backend,	the path and
					 peerhook parameters must be specified
					 to set	the destination	node and cor-
					 responding hook.  The optional	param-
					 eters socket and hook may be used to
					 set the ng_socket(4) node name	and
					 source	hook.  The ADDRESS, HOOK and
					 NAME must comply with netgraph(4) ad-
					 dressing rules.

			     Block storage devices:



			     The block-device-options are:

			     nocache   Open the	file with O_DIRECT.

			     direct    Open the	file using O_SYNC.

			     ro	       Force the file to be opened read-only.

				       Specify the logical and physical	sector
				       sizes of	the emulated disk.  The	physi-
				       cal sector size is optional and is
				       equal to	the logical sector size	if not
				       explicitly specified.

			     nodelete  Disable emulation of guest trim re-
				       quests via DIOCGDELETE requests.

			     SCSI devices:


			     The scsi-device-options are:

			     iid=IID	 Initiator ID to use when sending re-
					 quests	to specified CTL port.	The
					 default value is 0.

			     9P	devices:


			     The 9p-device-options are:

			     ro		 Expose	the share in read-only mode.

			     TTY devices:

			     stdio	 Connect the serial port to the	stan-
					 dard input and	output of the bhyve

			     /dev/xxx	 Use the host TTY device for serial
					 port I/O.

			     Boot ROM device:

			     romfile	 Map romfile in	the guest address
					 space reserved	for boot firmware.

			     Pass-through devices:

					 Connect to a PCI device on the	host
					 at the	selector described by slot,
					 bus, and function numbers.

			     Guest memory must be wired	using the -S option
			     when a pass-through device	is configured.

			     The host device must have been reserved at	boot-
			     time using	the pptdevs loader variable as de-
			     scribed in	vmm(4).

			     Virtio console devices:

					 A maximum of 16 ports per device can
					 be created.  Every port is named and
					 corresponds to	a Unix domain socket
					 created by bhyve.  bhyve accepts at
					 most one connection per port at a


					   +o   Due to lack of destructors in
					       bhyve, sockets on the filesys-
					       tem must	be cleaned up manually
					       after bhyve exits.

					   +o   There is	no way to use the
					       "console	port" feature, nor the
					       console port resize at present.

					   +o   Emergency write is advertised,
					       but no-op at present.

			     Framebuffer devices:


					 IPv4:port or [IPv6%zone]:port
						   An IP address and a port
						   VNC should listen on.  The
						   default is to listen	on lo-
						   calhost IPv4	address	and
						   default VNC port 5900.  An
						   IPv6	address	must be	en-
						   closed in square brackets
						   and may contain an optional
						   zone	identifier.

					 width and height
						   A display resolution, width
						   and height, respectively.
						   If not specified, a default
						   resolution of 1024x768 pix-
						   els will be used.  Minimal
						   supported resolution	is
						   640x480 pixels, and maximum
						   is 1920x1200	pixels.

					 vgaconf   Possible values for this
						   option are "io" (default),
						   "on"	, and "off".  PCI
						   graphics cards have a dual
						   personality in that they
						   are standard	PCI devices
						   with	BAR addressing,	but
						   may also implicitly decode
						   legacy VGA I/O space
						   (0x3c0-3df) and memory
						   space (64KB at 0xA0000).
						   The default "io" option
						   should be used for guests
						   that	attempt	to issue BIOS
						   calls which result in I/O
						   port	queries, and fail to
						   boot	if I/O decode is dis-

						   The "on" option should be
						   used	along with the CSM
						   BIOS	capability in UEFI to
						   boot	traditional BIOS
						   guests that require the
						   legacy VGA I/O and memory
						   regions to be available.

						   The "off" option should be
						   used	for the	UEFI guests
						   that	assume that VGA
						   adapter is present if they
						   detect the I/O ports.  An
						   example of such a guest is
						   OpenBSD in UEFI mode.

						   Please refer	to the bhyve
						   FreeBSD wiki	page
						   for configuration notes of
						   particular guests.

					 wait	   Instruct bhyve to only boot
						   upon	the initiation of a
						   VNC connection, simplifying
						   the installation of operat-
						   ing systems that require
						   immediate keyboard input.
						   This	can be removed for
						   post-installation use.

					 password  This	type of	authentication
						   is known to be cryptograph-
						   ically weak and is not in-
						   tended for use on untrusted
						   networks.  Many implementa-
						   tions will want to use
						   stronger security, such as
						   running the session over an
						   encrypted channel provided
						   by IPsec or SSH.

			     xHCI USB devices:

			     tablet	 A USB tablet device which provides
					 precise cursor	synchronization	when
					 using VNC.

			     NVMe devices:

			     devpath	 Accepted device paths are:
					 /dev/blockdev or /path/to/image or

			     maxq	 Max number of queues.

			     qsz	 Max elements in each queue.

			     ioslots	 Max number of concurrent I/O re-

			     sectsz	 Sector	size (defaults to blockif sec-
					 tor size).

			     ser	 Serial	number with maximum 20 charac-

			     AHCI devices:

			     nmrr	 Nominal Media Rotation	Rate, known as
					 RPM. value 1 will indicate device as
					 Solid State Disk. default value is 0,
					 not report.

			     ser	 Serial	Number with maximum 20 charac-

			     rev	 Revision Number with maximum 8	char-

			     model	 Model Number with maximum 40 charac-

			     HD	Audio devices:

			     play	 Playback device, typically /dev/dsp0.

			     rec	 Recording device, typically

     -S		 Wire guest memory.

     -u		 RTC keeps UTC time.

     -U	uuid	 Set the universally unique identifier (UUID) in the guest's
		 System	Management BIOS	System Information structure.  By de-
		 fault a UUID is generated from	the host's hostname and

     -w		 Ignore	accesses to unimplemented Model	Specific Registers
		 (MSRs).  This is intended for debug purposes.

     -W		 Force virtio PCI device emulations to use MSI interrupts in-
		 stead of MSI-X	interrupts.

     -x		 The guest's local APIC	is configured in x2APIC	mode.

     -Y		 Disable MPtable generation.

     vmname	 Alphanumeric name of the guest.  This should be the same as
		 that created by bhyveload(8).

     The current debug server provides limited support for debuggers.

     Each virtual CPU is exposed to the	debugger as a thread.

     General purpose registers can be queried for each virtual CPU, but	other
     registers such as floating-point and system registers cannot be queried.

     Memory (including memory mapped I/O regions) can be read and written by
     the debugger.  Memory operations use virtual addresses that are resolved
     to	physical addresses via the current virtual CPU's active	address	trans-

     The running guest can be interrupted by the debugger at any time (for
     example, by pressing Ctrl-C in the	debugger).

     Single stepping is	only supported on Intel	CPUs supporting	the MTRAP VM

     Breakpoints are supported on Intel	CPUs that support single stepping.
     Note that continuing from a breakpoint while interrupts are enabled in
     the guest may not work as expected	due to timer interrupts	firing while
     single stepping over the breakpoint.

     bhyve deals with the following signals:

	     Trigger ACPI poweroff for a VM

     Exit status indicates how the VM was terminated:

     0	     rebooted
     1	     powered off
     2	     halted
     3	     triple fault
     4	     exited due	to an error

     If	not using a boot ROM, the guest	operating system must have been	loaded
     with bhyveload(8) or a similar boot loader	before bhyve(4)	can be run.
     Otherwise,	the boot loader	is not needed.

     To	run a virtual machine with 1GB of memory, two virtual CPUs, a virtio
     block device backed by the	/my/image filesystem image, and	a serial port
     for the console:

	   bhyve -c 2 -s 0,hostbridge -s 1,lpc -s 2,virtio-blk,/my/image \
	     -l	com1,stdio -A -H -P -m 1G vm1

     Run a 24GB	single-CPU virtual machine with	three network ports, one of
     which has a MAC address specified:

	   bhyve -s 0,hostbridge -s 1,lpc -s 2:0,virtio-net,tap0 \
	     -s	2:1,virtio-net,tap1 \
	     -s	2:2,virtio-net,tap2,mac=00:be:fa:76:45:00 \
	     -s	3,virtio-blk,/my/image -l com1,stdio \
	     -A	-H -P -m 24G bigvm

     Run an 8GB	quad-CPU virtual machine with 8	AHCI SATA disks, an AHCI ATAPI
     CD-ROM, a single virtio network port, an AMD hostbridge, and the console
     port connected to an nmdm(4) null-modem device.

	   bhyve -c 4 \
	     -s	0,amd_hostbridge -s 1,lpc \
	     -s	1:0,ahci,hd:/images/disk.1,hd:/images/disk.2,\
	   cd:/images/install.iso \
	     -s	3,virtio-net,tap0 \
	     -l	com1,/dev/nmdm0A \
	     -A	-H -P -m 8G

     Run a UEFI	virtual	machine	with a display resolution of 800 by 600	pixels
     that can be accessed via VNC at:

	   bhyve -c 2 -m 4G -w -H \
	     -s	0,hostbridge \
	     -s	3,ahci-cd,/path/to/uefi-OS-install.iso \
	     -s	4,ahci-hd,disk.img \
	     -s	5,virtio-net,tap0 \
	     -s	29,fbuf,tcp=,w=800,h=600,wait \
	     -s	30,xhci,tablet \
	     -s	31,lpc -l com1,stdio \
	     -l	bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \

     Run a UEFI	virtual	machine	with a VNC display that	is bound to all	IPv6
     addresses on port 5900.

	   bhyve -c 2 -m 4G -w -H \
	     -s	0,hostbridge \
	     -s	4,ahci-hd,disk.img \
	     -s	5,virtio-net,tap0 \
	     -s	29,fbuf,tcp=[::]:5900,w=800,h=600 \
	     -s	30,xhci,tablet \
	     -s	31,lpc -l com1,stdio \
	     -l	bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \

     bhyve(4), netgraph(4), ng_socket(4), nmdm(4), vmm(4), ethers(5),
     bhyvectl(8), bhyveload(8)

     Intel, 64 and IA-32 Architectures Software	Developer_as Manual, Volume 3.

     bhyve first appeared in FreeBSD 10.0.

     Neel Natu <>
     Peter Grehan <>

BSD			       January 18, 2021				   BSD


Want to link to this manual page? Use this URL:

home | help