Ultimate Server Guide

The Proxmox / Docker Server Information

Create ZFS "folders" aka dataset

Run this command to create a ZFS Dataset

zfs create rpool/config/<containername>

Run this command to destroy a ZFS Dataset

zfs destroy rpool/config/<containername>

New Proxmox / Docker Server Deployment guide

Deployment Summary

Here, we will document the general steps to deploy a Proxmox / Docker setup.

nano /etc/apt/sources.list.d/pve-enterprise.list

Comment out the enterprise entry with # and save

/etc/apt/soruces.list.d/ceph.list

Comment out the entry here as well.

nano /etc/apt/sources.list

Check one of my running servers to see what it has, but my newer one has this

deb http://ftp.debian.org/debian bookworm main contrib
deb http://ftp.debian.org/debian bookworm-updates main contrib

# Proxmox VE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

# security updates
deb http://security.debian.org/debian-security bookworm-security main contrib

 

deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription
apt update && apt upgrade
Possibly important, maybe not

This below command- I'm not sure it makes sense, so I wouldn't run it, but Allyn gave it to me so it may be necessary. I'm confused because it says apt-get install when we never use apt-get.

apt update && apt upgrade -y && reboot apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties common
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
zfs create -o mountpoint=/var/lib/docker rpool/docker
zfs create -o mountpoint=/config rpool/config
mkdir /etc/systemd/system/docker.service.d
nano /etc/systemd/system/docker.service.d/storage-driver.conf
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --storage-driver=zfs -H fd://
apt update && apt install docker-ce docker-ce-cli containerd.io -y
docker info | grep Storage

Installing Portainer

zfs create rpool/config/portainer
docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /config/portainer:/data portainer/portainer-ce:latest

Installing Portainer App Templates

https://raw.githubusercontent.com/ntv-one/portainer/main/template.json

https://raw.githubusercontent.com/portainer/templates/master/templates-2.0.json - this was mine idk

PROXMOX Nag Screen Removal

Run the following command in the proxmox terminal

sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js && systemctl restart pveproxy.service


PROXMOX VM Tips

Help! My VM won't shutdown!

Help! My VM randomly suspends!

Help! My game crashes without warning!

Increase the hard drive space of a PROXMOX VM:

  1. Shut down the VM
  2. Run the command: (Example: qm resize 100 ide0 +150G)
  3. qm resize [VM_ID] [DISK_NAME] +[SIZE_INCREASE]G
  4. Start the VM
  5. Go into Disk Management, right-click the drive you want to add the additional storage to and press extend volume
  6. As a general rule. Windows only allows you to extend partitions that are next to each other. If you your new unallocated volume is NOT physically next to your main partition, you will have to use a third party tool to "move" it next to it. I recommend > https://www.diskpart.com/download-home.html < The free version of this software has a solid and tested feature to re-arrange partitions on a physical drive. After using this tool to move your partition structure, you should now be able to complete 5.
  7. Send it.

Setting up SAMBA

VFIO "GPU Passthrough" Guide for Proxmox

BIOS Settings:

Proxmox Kernel Version:

Proxmox Config:

nano /etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs
Change to:
For Intel:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on
For AMD:
root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet amd_iommu=on

Virtual Machine Configuration:

Updating Portainer

From Proxmox:

docker stop portainer
docker remove portainer
docker pull portainer/portainer-ce:latest
docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /config/portainer:/data portainer/portainer-ce:latest

IF YOU ACCIDENTALLY UPGRADE TO EE (Business Edition):

docker stop portainer
docker run -it --name portainer-database-rollback -v /config/portainer:/data portainer/portainer-ee:latest --rollback-to-ce
docker run -d -p 8000:8000 -p 9000:9000 -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /config/portainer:/data portainer/portainer-ce:latest

Postgresql

 

\l < list databases

create database:

CREATE DATABASE database;

delete database:

DROP DATABASE database;

grant admin to user on new db:

GRANT ALL PRIVILEGES ON DATABASE database TO user;

Checkpoints and Snapshots for ZFS

Zpool checkpoint rpool

image.png

Backups

Backups are accomplished differently depending on the server.

Proxmox1 uses Duplicati
Proxmox2 uses rsync

Rsync & crontab

Using the below command, you can see the rsync commands which backup docker.

crontab -e

Below is an example configuration.

# Palworld Hourly Backup
0 * * * * rsync -aAX --delete --exclude '*.recycle' --exclude 'rsync' /config/palworld/Pal/Saved /config/rsync/backups/palworld/palworld_$(date +\%FT\%H\%M\%z)

# Daily Rolling Backup at 1am
00 01 * * * rsync -aAX --delete --exclude '*.recycle' --exclude 'rsync' /config/ /config/rsync/backups/daily/daily

# Weekly Backup at 2pm on Friday
00 02 * * 5 rsync -aAX --delete --exclude '*.recycle' --exclude 'rsync' /config/ /config/rsync/backups/weekly/weekly_$(date +\%FT\%H\%M\%z)

# Monthly Backup at 3am on 1st of the month
00 03 1 * * rsync -aAX --delete --exclude '*.recycle' --exclude 'rsync' /config/ /config/rsync/backups/monthly/monthly_$(date +\%FT\%H\%M\%z)

This website can help setup various timings for when to run rsync.

Note that cron jobs require a leading slash ahead of % symbols. Example: date +\%FT\%H\%M\%z

Note that an empty line is required below the crontab -e file

Directory Tree Scanner - qdirstat - windirstat alternative

Directory Tree Scanner - qdirstat - windirstat alternative

Docker Network Limit fix

nano edit the below file

/etc/docker/daemon.json

{
  "log-level": "warn",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "5"
  },
  "default-address-pools": [
    {
      "base": "172.16.0.0/12",
      "size": 24
    }
  ],
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

Save, restart docker

service docker restart

then maybe restart containers and stacks

Add Pfetch and fastfetch to proxmox along with making it run at start

apt install unzip && wget https://github.com/dylanaraps/pfetch/archive/master.zip && unzip master.zip && install pfetch-master/pfetch /usr/local/bin/ && ls -l /usr/local/bin/pfetch && echo -e "\n# Add pfetch command\npfetch" >> ~/.bashrc

 

cd ~/ && wget https://github.com/fastfetch-cli/fastfetch/releases/download/2.20.0/fastfetch-linux-amd64.tar.gz && tar -xf fastfetch-linux-amd64.tar.gz && mkdir /root/bin/ && cp ~/fastfetch-linux-amd64/usr/bin/* /bin/

Proxmox Power Savings / CPU Governor

https://tteck.github.io/Proxmox/#proxmox-ve-cpu-scaling-governor
https://community.home-assistant.io/t/psa-how-to-configure-proxmox-for-lower-power-usage/323731/27 

Proxmox ships with Performance Governor by default, and you may want to change it to save power.

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/scaling-governor.sh)"

ZSH customize without OhMyZSH

git clone --depth=1 https://gitee.com/romkatv/powerlevel10k.git ~/powerlevel10k && echo 'source ~/powerlevel10k/powerlevel10k.zsh-theme' >>~/.zshrc