Mastering Proxmox - Second Edition

Table of Contents

Mastering Proxmox - Second Edition

Credits

About the Author

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the color images of this book

Errata

Piracy

Questions

1. Understanding Proxmox VE and Advanced Installation

Understanding Proxmox features

It is free!

Built-in firewall

Open vSwitch

The graphical user interface

KVM

Linux containers or LXC

Storage plugins

Vibrant culture

The basic installation of Proxmox

The advanced installation option

Debugging the Proxmox installation

Customizing the Proxmox splash screen

Summary

2. Exploring the Proxmox GUI

The GUI menu system

The Datacenter menu

Datacenter | Search

Datacenter | Summary

Datacenter | Options

Datacenter | Storage

Datacenter | Backup

Datacenter | Users

Datacenter | Groups

Datacenter | Pools

Datacenter | Permissions

Datacenter | Roles

Datacenter | Authentication

Datacenter | HA

Datacenter | Firewall

Datacenter | Support

Node menu

Node | Search

Node | Summary

Node | Services

Node | Network

Node | DNS

Node | Time

Node | Syslog

Node | Task History

Node | Firewall

Node | Updates

Node | Console

Node | Ceph

Node | Subscription

The KVM VM menu

KVM VM | Summary

KVM VM | Hardware

KVM VM | Options

KVM VM | Task History

KVM VM | Monitor

KVM VM | Backup

KVM VM | Snapshot

KVM VM | Console

KVM VM | Firewall

KVM VM | Permissions

The LXC Container menu

The LXC Container | Summary

LXC Container | Resources

LXC Container | Network

LXC Container | DNS

LXC Container | Options

LXC Container | Task History

LXC Container | Backup

LXC Container | Console

LXC Container | Snapshots

LXC Container | Firewall

LXC Container | Permissions

Pool menu

Pool | Summary

Pool | Members

Pool | Permissions

Cluster view mode

The Server View

The Folder View

Storage view

The Pool View

Summary

3. Proxmox under the Hood

The Proxmox cluster filesystem

Proxmox directory structure

Dissecting the configuration files

The cluster configuration file

logging { }

nodelist { }

nodeid

quorum_votes

ring0_addr

quorum { }

totem { }

interface { }

Storage configuration file

User configuration files

The password configuration file

KVM virtual machine configuration file

Arguments in the KVM configuration file

LXC container configuration file

Version configuration file

Member nodes

Virtual machine list file

The cluster log file

Ceph configuration files

Firewall configuration file

Summary

4. Storage Systems

Local storage versus shared storage

Live migration of a virtual machine

Seamless expansion of multinode storage space

Centralized backup

Multilevel data tiering

Central storage management

Local and shared storage comparison

A virtual disk image

Supported image formats

The .qcow2 image type

The .raw image type

The .vmdk image type

Managing disk images

Resizing a virtual disk image

Moving a virtual disk image

Throttling a virtual disk image

Caching a virtual disk image

Storage types in Proxmox

Directory

iSCSI

LVM

NFS

ZFS

Ceph RBD

The Ceph components

A physical node

Maps

A cluster map

A CRUSH Map

Monitor

OSD

OSD Journal

MDS

Placement Group

Pools

The Ceph components summary

Virtual Ceph for training

Ceph cluster

Ceph on Proxmox

Preparing the Proxmox node for Ceph

Installing Ceph

Creating MONs from Proxmox GUI

Creating OSD from Proxmox GUI

Creating a new Ceph Pool using Proxmox GUI

Connecting RBD to Proxmox

Ceph command list

GlusterFS

Noncommercial and commercial storage options

Summary

5. KVM Virtual Machines

Exploring a KVM

Creating a KVM

Creating a VM from scratch

General tab

OS tab

CD/DVD tab

Hard Disk tab

Bus/Device

Storage

Disk size

Format

Cache

No backup

Discard

Iothread

CPU tab

Sockets

Cores

Enabling NUMA

Type

Memory tab

Network

Bridged mode

Firewall

NAT mode

No network device

Model

MAC address

Rate limit

Multiqueues

Disconnect

Creating a VM by cloning

Mode

Creating VMs from templates

Advanced configuration options for VM

Configuring a sound device

Configuring PCI passthrough

Configuring GPU Passthrough

Configuring Hotplug

Migrating a KVM

Nested virtual cluster

Summary

6. LXC Virtual Machines

Exploring an LXC virtual machine

Understanding container templates

Creating an LXC container

General tab

Node

VM ID

Hostname

Resource Pool

The Template tab

The Root Disk tab

The CPU tab

CPU limits

CPU units

The Memory tab

The Network tab

ID

Name

MAC address

Bridge

The VLAN Tag

Firewall

IPv4/IPv6

The DNS tab

The Confirm tab

Managing an LXC container

Adjusting resources using the GUI

Adjusting resources using the CLI

Adjusting resources using direct modification

Migrating an LXC container

Accessing an LXC container

Unprivileged versus privileged containers

Creating an unprivileged container as root

Checking container processes

Converting OpenVZ to LXC

Summary

7. Network of Virtual Networks

Exploring a virtual network

A physical network versus a virtual network

A physical network

A virtual network

Networking components in Proxmox

Virtual Network Interface Card (vNIC)

Adding vNIC

A virtual bridge

Adding a virtual bridge

Extra bridge options

bridge_stp

bridge_fd

Virtual LAN

Adding a VLAN

Network Address Translation/Translator

Adding NAT/masquerading

Network bonding

Adding a bonding interface

The layer 2 hash policy

The layer 2+3 hash policy

The layer 3+4 hash policy

Multicast

Configuring multicast on Netgear

Open vSwitch

Features of Open vSwitch

Adding the Open vSwitch bridge

Adding the Open vSwitch bond

Adding Open vSwitch IntPort

CLI for Open vSwitch

Practicing Open vSwitch

Configuration requirements

Solutions

Sample virtual networks

Network #1 – Proxmox in its simplest form

Network #2 – the multitenant environment

Network #3 – academic institution

A multitenant virtual environment

A multitenant network diagram

Summary

8. The Proxmox Firewall

Exploring the Proxmox VE firewall

Components of the Proxmox firewall

Zones

Security Groups

IPSet

Rules

Protocols

Macros

The pve-firewall and pvefw-logger services

Configuration files of a firewall

Configuring the Datacenter-specific firewall

Configuring the Datacenter firewall through the GUI

Creating the Datacenter firewall rules

Creating the Datacenter IPSet

Creating aliases

Configuring the Datacenter firewall through the CLI

[OPTIONS]

[ALIASES]

[IPSET ]

[RULES]

[group ]

Configuring a host-specific firewall

Creating host firewall rules

Options for the host zone firewall

Enable a firewall

The SMURFS filter

The TCP flags filter

nf_conntrack_max

nf_conntrack_tcp_timeout_established

log_level_in/out

tcp_flags_log_level

smurf_log_level

Configuring the host firewall through the CLI

Configuring a VM-specific firewall

Creating VM firewall rules

Creating Aliases

Creating IPSet

Options for a VM zone firewall

Enable DHCP

The MAC filter

Configuring a VM-specific firewall through the CLI

Integrating a Suricata IDS/IPS

Installing/configuring Suricata

Limitation of Suricata in Proxmox

Summary

9. Proxmox High Availability

Understanding High Availability

High Availability in Proxmox

How Proxmox HA works

Requirements for HA setup

Minimum three nodes

Shared storage

Fencing

BIOS power on feature

Configuring Proxmox HA

Status menu

Groups menu

ID

Node

Restricted

Nofailback

Resources menu

Fencing menu

Testing Proxmox HA configuration

The Proxmox HA simulator

Configuring the Proxmox HA simulator

Summary

10. Backup/Restore VMs

Proxmox backup options

A full backup

Full backup modes

Snapshot

Suspend

Stop

Backup compression

None

LZO

GZIP

Snapshots

Configuring backup storage

Configuring full backup

Creating a schedule for Backup

Node

Storage

Day of week

Start Time

Selection mode

Send e-mail to

E-mail notification

Compression

Mode

Enable

Creating a manual backup

Creating snapshots

Restoring a VM

Backup/restore through the CLI

Backup using the CLI

Restore using the CLI

Unlocking a VM after a backup error

Back up a configuration file

#bwlimit

#lockwait

#stopwait

#script

#exclude-path

#pigz

Summary

11. Updating and Upgrading Proxmox

Introducing Proxmox update

Updating Proxmox through the GUI

Updating Proxmox through the CLI

Difference between upgrade and dist-upgrade

Recovering from the grub2 update issue

Updating after a subscription change

Rebooting dilemma after Proxmox updates

Summary

12. Monitoring a Proxmox Cluster

An introduction to monitoring

Proxmox built-in monitoring

Zabbix as a monitoring solution

Installing Zabbix

Configuring Zabbix

Configuring a host to monitor

Displaying data using a graph

Configuring the Disk Health notification

Installing smart monitor tools

Configuring the Zabbix agent

Creating a Zabbix item in the GUI

Creating a trigger in the GUI

Creating graphs in the GUI

Configuring SNMP in Proxmox

OIDs

MIB

Adding an SNMP device in Zabbix

Monitoring the Ceph cluster with the Ceph dashboard

Summary

13. Proxmox Production-Level Setup

Defining production level

Key components

Stable and scalable hardware

Redundancy

Node level

Utility level

Network level

HVAC level

Storage level

Current load versus future growth

Budget

Simplicity

Tracking hardware inventory

Hardware selection

Sizing CPU and memory

Single socket versus multi-socket

Hyper-threading – enable versus disable

Start small with VM resources

Balancing node resources

Production Ceph cluster

Forget about hardware RAID

SSD for Ceph Journal

Network bandwidth

Liquid cooling

Total immersion in oil

Total immersion in 3M Novec

Direct contact liquid cooling

Real-world Proxmox scenarios

Scenario 1 – an academic institution

Scenario 2 – multi-tier storage cluster with a Proxmox cluster

Scenario 3 – virtual infrastructure for a multi-tenant cloud service provider

Scenario 4 – nested virtual environment for a software development company

Scenario 5 – virtual infrastructure for a public library

Scenario 6 – multi-floor office virtual infrastructure with virtual desktops

Scenario 7 – virtual infrastructure for the hotel industry

Scenario 8 – virtual infrastructure for a geological survey organization

Summary

14. Proxmox Troubleshooting

Proxmox node issues

Issue – Fresh Proxmox install stuck with /dev to be fully populated error during node reboot

Issue – Rejoining a node to a Proxmox node with the same old IP address

Issue – Booting with a USB stick fails in Proxmox

Issue – Proxmox installation completed but GRUB is in an endless loop after reboot

Issue – LSI MegaRAID 9240-8i/9240-4i causes an error during booting on the Proxmox node

Downloading and updating the LSI driver

Updating the Supermicro BIOS

Main cluster issues

Issue – Proxmox virtual machines are running, but the Proxmox GUI shows that everything is offline

Issue – Kernel panic when disconnecting USB devices, such as a keyboard, mouse, or UPS

Issue – Virtual machines on Proxmox will not shut down if shutdown is initiated from the Proxmox GUI

Issue – Kernel panic with HP NC360T (Intel 82571EB Chipset) only in Proxmox VE 3.2

Issue – The Proxmox cluster is out of Quorum and cluster filesystem is in read-only mode

Issue – Proxmox cannot start due to the getpwnam error

Issue – Cannot log in to the GUI as root after reinstalling Proxmox on the same node

Issue – The Upgrade button is disabled on the Proxmox GUI, which prevents the node upgrade

Issue – VM will not respond to shutdown, restart

Issue – The Proxmox GUI not showing RRD graphs

Storage issues

Issue – Deleting a damaged LVM from Proxmox with the error read failed from 0 to 4096

Issue – Proxmox cannot mount NFS Share due to the timing out error

Issue: How to delete leftover NFS shares in Proxmox or what to do when the NFS stale file handles error occurs?

Issue – Proxmox issues—mode session exit code 21 errors while trying to access the iSCSI target

Issue – Cannot read an iSCSI target even after it has been deleted from Proxmox storage

Issue – A Ceph node is removed from the Proxmox cluster, but OSDs still show up in PVE

Issue – The 'No Such Block Device' error during creation of an OSD through the Proxmox GUI

Issue – The fstrim command does not trim unused blocks for the Ceph storage

Issue – The RBD Couldn't Connect To Cluster (500) error when connecting Ceph with Proxmox

Issue – Changing the storage type from ide to virtio after the VM has been set up and the OS has been installed

Issue – The pveceph configuration not initialized (500) error when you click on the Ceph tab in the Proxmox GUI

Issue – The CephFS storage disappears after a Proxmox node reboots

Issue – VM Cloning does not parse in the Ceph storage

Network connectivity issues

Issue – No connectivity on Realtek RTL8111/8411 Rev. 06 network interfaces

Issue – Network performance is slower with the e1000 virtual network interface

Issue – Patch port for Openvswitch in Proxmox not working

Issue – Trying to add a node to a newly created Proxmox cluster when nodes do not form quorum

Issue – Implemented IPv6 but firewall rules do not get applied

KVM virtual machine issues

Issue – Windows 7 VM does not reboot, instead it shuts down requiring manual boot from Proxmox

Issue – The qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 4.1

Issue – Online migration of a virtual machine fails with a 'Failed to sync data' error

Issue – Adjusting RAM through the GUI and rebooting the VM does not change allocated memory

Issue – No audio in Windows KVM

Issue – The virtio virtual disk is not available during the Windows Server installation

LXC container issues

Issue – A Proxmox node hangs when trying to stop or restart an LXC container

Issue – The noVNC console only shows a cursor for LXC containers

Backup/restore issues

Issue – A Proxmox VM is locked after backup crashes unexpectedly

Issue – How can Proxmox backup only the primary OS virtual disk instead of all the virtual disks for a VM?

Issue – Backup of virtual machines stops prematurely with an Operation Not Permitted error

Issue – A backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage

Issue – Backup of virtual machines aborts a backup task prematurely

Issue – Backup storage has a lot of .dat files and .tmp folders using the storage space

VNC/SPICE console issues

Issue – The mouse pointer is not shared with SPICE-VIEWER on Windows 8 VM

Issue – The SPICE console is unstable or nonfunctioning after updating to Proxmox VE 4.1

Issue – Remote Viewer is unable to connect to a SPICE-enabled virtual machine on the Windows OS

Firewall issues

Issue – Rules are created and a firewall is enabled for vNIC, but rules do not get applied

Issue – A firewall is enabled for a VM and the necessary rules are created, but nothing is being filtered for that VM. All other VM firewall rules in the same node work properly

Summary

Index

Mastering Proxmox - Second Edition

Mastering Proxmox - Second Edition

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: July 2014

Second edition: May 2016