RHEL 8 + libvirt + docker + portainer

Build minimal install box of RHEL 8. You can choose Virtual Host and Headless administration and it covers most of this right off the bat, it will just skip them in the yum commands.

Watch your disk partitions! Changing storage after the fact isn’t terrible, but if you need to change storage, follow this: https://linuxconfig.org/configure-default-kvm-virtual-storage-on-redhat-linux#h6-1-create-new-virtual-storage-directory

This is RHEL, so register it with your free developer account (16 Free RHEL servers? Yes, thank you!)

SSH should be enabled and running:

sudo yum install openssh-server
sudo systemctl enable sshd
sudo systemctl start sshd
sudo systemctl status sshd
ssh to box:
yum -y update
yum -y install cockpit cockpit-machines
yum -y install qemu-kvm libvirt libguestfs-tools virt-install
yum -y install mc iperf3 iptraf-ng
yum -y install virt-install virt-viewer
systemctl start cockpit.socket
systemctl enable cockpit.socket
systemctl status cockpit.socket
firewall-cmd –add-service=cockpit –permanent
firewall-cmd –reload
# Cockpit is on: https://SERVER_IP:9090/

modprobe fuse
virt-host-validate
systemctl start libvirtd.service
systemctl enable libvirtd.service
systemctl status libvirtd.service

If needed, set up network bridge:
Cockpit – Networking – Add Bridge
Select your ethernet card (em1)

Should now be able to build virtual machines from within cockpit. You might need to reload the page or go out and back in for it to figure out libvirt is enabled.

If you can’t build a new VM, reboot the box at least once before freaking out.
ISO’s to install from:
ISOs should live in /var/lib/libvirt/images/ probably want to mount that as a read only nfs. If you mount iso’s somewhere else you can have permission issues that are tough to get around. Even when mounted in the above dir, I had issues selecting the OS when choosing the ISO, try picking the OS first if it won’t let you pick it after the iso selection.
mount:
go to cockpit – storage and add the NFS or SMB share, stupid easy.

Netdata:

yum update
bash <(curl -Ss https://my-netdata.io/kickstart.sh)
#(wait) (answer any prompts)
systemctl start netdata
systemctl enable netdata

# Add to IPTables via firewall-cmd:
firewall-cmd --zone=public --permanent --add-port=19999/tcpservice firewalld reload
http://SERVER_IP:19999/
Done!

Docker:

yum install -y yum-utils
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum makecache
yum remove buildah podman containerd runc
yum -y install docker-ce
systemctl enable --now docker
systemctl status docker
usermod -aG docker $USER
docker version
docker pull alpine
docker images
docker run -it --rm alpine /bin/sh

exit


Docker compose:

curl -s https://api.github.com/repos/docker/compose/releases/latest \
  | grep browser_download_url \
  | grep docker-compose-Linux-x86_64 \
  | cut -d '"' -f 4 \
  | wget -qi -

chmod +x docker-compose-Linux-x86_64
mv docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
docker-compose version


CTop

export VER="0.7.3"
wget https://github.com/bcicen/ctop/releases/download/v${VER}/ctop-${VER}-linux-amd64 -O ctop
chmod +x ctop
sudo mv ctop /usr/local/bin/ctop


Portainer
docker pull portainer/portainer:latest
docker volume create portainer_data
docker run -d -p 8000:8000 -p 9000:9000 –name=portainer –restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce

Go to http://SERVER_IP:9000/

Set a strong password

Choose Local and click Connect

Portainer: add custom repo for templates:

https://raw.githubusercontent.com/SelfhostedPro/selfhosted_templates/master/Template/template.json
and click Save

Ubuntu 20.04.1 LTS Libvirt + Cockpit + Docker + Portainer

Build minimal install box, add OpenSSH during installation
Watch your disk partitions! You will be using a lot (eventually)

Base install:

ssh to box:

# I'm bad, I do it all as root
sudo su - 
apt update
apt upgrade

libVirt install:

# libVirt install
apt install cpu-checker
apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virtinst virt-manager
systemctl is-active libvirtd
# should output "active"
usermod -aG libvirt $USER
usermod -aG kvm $USER
exit
# Do this as your user also
sudo usermod -aG libvirt $USER
sudo usermod -aG kvm $USER
sudo brctl show

Cockpit:

sudo su - 
apt install cockpit -y
systemctl start cockpit
ss -tunlp | grep 9090
ufw allow 9090/tcp
apt install cockpit-machines cockpit-storaged cockpit-packagekit cockpit-networkmanager cockpit-dashboard cockpit-bridge
# Cockpit should now be available on https://ip:9090

Docker:

sudo apt-get update

sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent 

sudo apt-get install software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository  "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

sudo apt -y install mc iperf3 iptraf-ng

sudo docker run hello-world

Portainer

docker run -d --name portainer --restart unless-stopped -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

NetData

apt install curl -y
bash <(curl -Ss https://my-netdata.io/kickstart.sh)

Links

# Cockpit is on:
https://SERVER_IP:9090/

# Portainer is on:
http://SERVER_IP:9000/

# Netdata is on
http://SERVER_IP:19999

This was needed in CentOS 8, not sure about Ubuntu 20.04 yet:

Set up network bridge:
Cockpit - Networking - Add Bridge
Select your ethernet card (em1) *(ymmv)
Name - bridge0
Reboot. Really. This may save some headaches later.


For desktop OS type guests, you should be good to go, for servers you have to use a bridge to get on the right network.

Assuming you added the bridge above (Cockpit – Network – Add Bridge)

Virtual machines – VM – Networking – Interface Type – Bridge to LAN, Source – bridge0, e1000e or virt, whatever.

Restart the VM, it should appear on your network. You may have to reboot the host if this is the first time through!

Beware that firewalld and iptables are all in this mess! You may have to enable ports or disable the firewall on the LAN side to get your server to work.

If you can’t build a new VM or can’t connect, reboot the box at least once before freaking out.

ISO’s to install from

ISOs should live in /var/lib/libvirt/images/ probably want to mount that as a read only nfs mount to your NAS:

Go to cockpit – storage and add the NFS share to that path, stupid easy.

Portainer templates

  • Open Portainer (http://SERVER_IP:9000/)
  • Go to Settings
  • Go to App Templates
  • Select “Use External Templates”
  • Paste in:
  • https://raw.githubusercontent.com/SelfhostedPro/selfhosted_templates/master/Template/template.json
  • Click “Save Settings”
  • Go to “App Templates” in the blue bar menu
  • Turn on “Show Container Templates”

Storage changes

If you need to change storage locations for libVirt, follow this: https://linuxconfig.org/configure-default-kvm-virtual-storage-on-redhat-linux#h6-1-create-new-virtual-storage-directory