zaki work log

作業ログやら生活ログやらなんやら

Proxmox VEの8から9へのアップグレード

今月に入ってPVEのver9がリリースされていたので、個人環境をアップグレード。

www.proxmox.com

pve.proxmox.com

7から使ってると対象外だが、8を新規インストールしたノードだと事前チェックでエラーがあったりしたので、ひとまず作業内容のまとめ。
基本的には7から8へのアップグレードの手順と同一。

zaki-hmkc.hatenablog.com

また、クラスタ構成の場合の注意事項は特には見当たらないため、オーソドックスに「アップグレードするPVE上のVMは他ノードへ寄せてアップグレード」を全ノードで繰り返す。

zaki-hmkc.hatenablog.com

作業前状態

root@pve02:~# pveversion 
pve-manager/8.4.11/14a32011146091ed (running kernel: 6.8.12-13-pve)

root@pve02:~# pvecm status
Cluster information
-------------------
Name:             pve-cluster
Config Version:   2
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Sun Aug 17 11:04:06 2025
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000002
Ring ID:          1.2d
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   2
Highest expected: 2
Total votes:      2
Quorum:           2  
Flags:            Quorate 

Membership information
----------------------
    Nodeid      Votes Name
0x00000001          1 192.168.0.4
0x00000002          1 192.168.0.5 (local)

pve8to9

pve8to9を実行すると、HAクラスタを組むために後から追加したノードで以下の通りの結果。

= SUMMARY =

TOTAL:    52
PASSED:   39
SKIPPED:  7
WARNINGS: 3
FAILURES: 1

ATTENTION: Please check the output for detailed information!
Try to solve the problems one at a time and then run this checklist tool again.

今回は警告だけでなくエラーも出てるんで、ちゃんとチェックしていく。

エラー

FAIL: systemd-boot meta-package installed.

FAIL: systemd-boot meta-package installed. This will cause problems on upgrades of other boot-related packages. Remove 'systemd-boot' See https://pve.proxmox.com/wiki/Upgrade_from_8_to_9#sd-boot-warning for more information.

https://pve.proxmox.com/wiki/Upgrade_from_8_to_9#sd-boot-warning

リンク先を読むと「PVE8.1 - 8.4」をISOからインストールしてると出る模様。確かにver7から使ってる方では出てない。 特に必要としていないので削除する。

root@pve02:~# dpkg -l | grep systemd-boot
ii  systemd-boot                         252.38-1~deb12u1                    amd64        simple UEFI boot manager - tools and services
ii  systemd-boot-efi:amd64               252.38-1~deb12u1                    amd64        simple UEFI boot manager - EFI binaries

root@pve02:~# apt-get remove systemd-boot
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  systemd-boot
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 250 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 54327 files and directories currently installed.)
Removing systemd-boot (252.38-1~deb12u1) ...
Processing triggers for man-db (2.11.2-2) ...

警告

警告は以下

WARN: cluster consists of less than three quorum-providing nodes!

3ノード必要なところ、2ノードのクラスタを組んでるので仕方ない。
分かっていて構成してる状態なので今回は無視

WARN: Removable bootloader found at '/boot/efi/EFI/BOOT/BOOTX64.efi'

ブートローダーの設定不備

WARN: Removable bootloader found at '/boot/efi/EFI/BOOT/BOOTX64.efi', but GRUB packages not set up to update it!
Run the following command:
echo 'grub-efi-amd64 grub2/force_efi_extra_removable boolean true' | debconf-set-selections -v -u
Then reinstall GRUB with 'apt install --reinstall grub-efi-amd64'

メッセージの通り、以下を実行

root@pve02:~# echo 'grub-efi-amd64 grub2/force_efi_extra_removable boolean true' | debconf-set-selections -v -u
info: Trying to set 'grub2/force_efi_extra_removable' [boolean] to 'true'
info: Loading answer for 'grub2/force_efi_extra_removable'

root@pve02:~# apt install --reinstall grub-efi-amd64
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/45.7 kB of archives.
After this operation, 0 B of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 54309 files and directories currently installed.)
Preparing to unpack .../grub-efi-amd64_2.06-13+pmx7_amd64.deb ...
Unpacking grub-efi-amd64 (2.06-13+pmx7) over (2.06-13+pmx7) ...
Setting up grub-efi-amd64 (2.06-13+pmx7) ...
Installing for x86_64-efi platform.
File descriptor 3 (pipe:[1153572]) leaked on vgs invocation. Parent PID 179937: grub-install.real
File descriptor 3 (pipe:[1153572]) leaked on vgs invocation. Parent PID 179937: grub-install.real
Installation finished. No error reported.
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-13-pve
Found initrd image: /boot/initrd.img-6.8.12-13-pve
Found linux image: /boot/vmlinuz-6.8.12-4-pve
Found initrd image: /boot/initrd.img-6.8.12-4-pve
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Adding boot menu entry for UEFI Firmware Settings ...
done
Processing triggers for shim-signed:amd64 (1.44+pmx1+15.8-1+pmx1) ...

WARN: The matching CPU microcode package 'intel-microcode' could not be found!

intel-microcode」パッケージを入れた方が良いよ、という内容。

WARN: The matching CPU microcode package 'intel-microcode' could not be found! Consider installing it to receive the latest security and bug fixes for your CPU.
        Ensure you enable the 'non-free-firmware' component in the apt sources and run:
        apt install intel-microcode

ただしintel-microcodeはデフォルトで有効になってるcontribコンポーネントにはないので、メッセージの通りnon-free-firmwareを追加する。(non-freeじゃないのね)
/etc/apt/sources.listの変更点は以下の通り。

root@pve02:~# diff -u /etc/apt/sources.list.org /etc/apt/sources.list
--- /etc/apt/sources.list.org   2025-01-08 21:27:37.713420715 +0900
+++ /etc/apt/sources.list       2025-08-17 10:56:46.972880545 +0900
@@ -1,6 +1,6 @@
-deb http://ftp.jp.debian.org/debian bookworm main contrib
+deb http://ftp.jp.debian.org/debian bookworm main contrib non-free-firmware
 
-deb http://ftp.jp.debian.org/debian bookworm-updates main contrib
+deb http://ftp.jp.debian.org/debian bookworm-updates main contrib non-free-firmware
 
 # security updates
-deb http://security.debian.org bookworm-security main contrib
+deb http://security.debian.org bookworm-security main contrib non-free-firmware

インストール

root@pve02:~# apt-get update

root@pve02:~# apt-get install intel-microcode
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  iucode-tool
The following NEW packages will be installed:
  intel-microcode iucode-tool
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 11.2 MB of archives.
After this operation, 19.9 MB of additional disk space will be used.
Do you want to continue? [Y/n]

これで(ノード数不足以外の)エラーと警告が全て解消された。

= SUMMARY =

TOTAL:    51
PASSED:   41
SKIPPED:  7
WARNINGS: 1
FAILURES: 0

PVE8の最新のアップグレード

更新が他にないことを確認

root@pve02:~# apt-get update
root@pve02:~# apt-get upgrade
root@pve02:~# apt-get dist-upgrade

APTラインの更新

PVE8(debian 12 bookworm)用の設定をPVE9(debian 13 trixie)に更新

sed -i 's/bookworm/trixie/g' /etc/apt/sources.list
sed -i 's/bookworm/trixie/g' /etc/apt/sources.list.d/pve-no-subscription.list

Ceph、エンタープライズ版パッケージは使ってないのでスキップ。
無料版は上記の/etc/apt/sources.list.d/pve-no-subscription.listに定義してる。

リポジトリ情報の更新は特に問題なし。

root@pve02:~# apt-get update
Get:1 http://ftp.jp.debian.org/debian trixie InRelease [138 kB]
Get:2 http://ftp.jp.debian.org/debian trixie-updates InRelease [47.1 kB]
Get:3 http://ftp.jp.debian.org/debian trixie/main amd64 Packages [9,668 kB]
Get:4 http://ftp.jp.debian.org/debian trixie/main Translation-en [6,484 kB]
Get:5 http://security.debian.org trixie-security InRelease [43.4 kB]
Get:6 http://ftp.jp.debian.org/debian trixie/contrib amd64 Packages [53.8 kB]
Get:7 http://ftp.jp.debian.org/debian trixie/contrib Translation-en [49.6 kB]
Get:8 http://ftp.jp.debian.org/debian trixie/non-free-firmware amd64 Packages [6,868 B]
Get:9 http://ftp.jp.debian.org/debian trixie/non-free-firmware Translation-en [4,704 B]
Get:10 http://ftp.jp.debian.org/debian trixie-updates/main amd64 Packages [2,432 B]
Get:11 http://ftp.jp.debian.org/debian trixie-updates/main Translation-en [396 B]
Get:12 http://security.debian.org trixie-security/main amd64 Packages [8,780 B]
Get:13 http://security.debian.org trixie-security/main Translation-en [9,164 B]
Get:14 http://download.proxmox.com/debian/pve trixie InRelease [2,771 B]
Get:15 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages [170 kB]
Fetched 16.7 MB in 2s (8,282 kB/s)
Reading package lists... Done
root@pve02:~#

アップグレード

ここからが本番

root@pve02:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages will be REMOVED:
  [...]
The following NEW packages will be installed:
  [...]
The following packages will be upgraded:
  [...]
586 upgraded, 158 newly installed, 59 to remove and 0 not upgraded.
Need to get 579 MB of archives.
After this operation, 1,181 MB of additional disk space will be used.
Do you want to continue? [Y/n]

アップグレード中、例によっていくつかのパッケージは個別に確認が求められるが、ドキュメント記載の推奨値を参考に設定していく。

https://pve.proxmox.com/wiki/Upgrade_from_8_to_9#Upgrade_the_system_to_Debian_Trixie_and_Proxmox_VE_9.0

アップグレードのチェック

root@pve02:~# pve8to9
= CHECKING VERSION INFORMATION FOR PVE PACKAGES =

Checking for package updates..
PASS: all packages up-to-date

Checking proxmox-ve package version..
PASS: already upgraded to Proxmox VE 9

Checking running kernel version..
WARN: a suitable kernel (proxmox-kernel-6.14) is installed, but an unsuitable (6.8.12-13-pve) is booted, missing reboot?!

...

他は問題ないのでリブート。

root@pve02:~# pveversion
pve-manager/9.0.5/9c5600b249dbfd2f (running kernel: 6.14.8-2-pve)
root@pve02:~#

起動後に再度apt-get updateすると、無効化してたはずのエンタープライズ版のリポジトリ設定が再度有効化されているため、オフにしておく。

root@pve02:~# apt-get update
Hit:1 http://security.debian.org trixie-security InRelease
Hit:2 http://ftp.jp.debian.org/debian trixie InRelease
Hit:3 http://ftp.jp.debian.org/debian trixie-updates InRelease
Hit:4 http://download.proxmox.com/debian/pve trixie InRelease
Err:5 https://enterprise.proxmox.com/debian/pve trixie InRelease
  401  Unauthorized [IP: 117.120.5.24 443]
Reading package lists... Done
E: Failed to fetch https://enterprise.proxmox.com/debian/pve/dists/trixie/InRelease  401  Unauthorized [IP: 117.120.5.24 443]
E: The repository 'https://enterprise.proxmox.com/debian/pve trixie InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@pve02:~# ls -l /etc/apt/sources.list.d/
total 16
-rw-r--r-- 1 root root  74 Nov 21  2024 ceph.list.disabled
-rw-r--r-- 1 root root  70 Nov 20  2024 pve-enterprise.list.disabled
-rw-r--r-- 1 root root 160 Aug 13 20:00 pve-enterprise.sources
-rw-r--r-- 1 root root 170 Aug 17 11:43 pve-no-subscription.list

root@pve02:~# mv /etc/apt/sources.list.d/pve-enterprise.sources /etc/apt/sources.list.d/pve-enterprise.sources.disabled

root@pve02:~# apt-get update
Hit:1 http://ftp.jp.debian.org/debian trixie InRelease
Hit:2 http://ftp.jp.debian.org/debian trixie-updates InRelease
Hit:3 http://security.debian.org trixie-security InRelease
Hit:4 http://download.proxmox.com/debian/pve trixie InRelease
Reading package lists... Done
root@pve02:~#

別ノードのアップグレード

1台目ノードのアップグレードが完了したため、完了したノードへVMマイグレーション(あるいは停止)し、もう片方のノードも同様の作業を行う。