Compare commits

..
Author SHA1 Message Date
MickLesk bc2c53358c ArchiveBox: install libatomic1 for the Playwright driver
Playwright's bundled node now links libatomic.so.1, which Debian 12 does not pull in; playwright install-deps failed with exit 127 on every install.
2026-11-14 19:06:10 +01:00
47 changed files with 3812 additions and 1379 deletions

No files matched your search

-221
View File
@@ -1,224 +1,3 @@
## 2026-11-14
### 🆕 New Scripts
- LibreDB Studio ([#18699](https://github.com/community-scripts/ProxmoxVE/pull/18699))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- GoatCounter: let the singlefile download unpack the binary [@MickLesk](https://github.com/MickLesk) ([#18721](https://github.com/community-scripts/ProxmoxVE/pull/18721))
- Zerobyte: let the singlefile download unpack restic [@MickLesk](https://github.com/MickLesk) ([#18720](https://github.com/community-scripts/ProxmoxVE/pull/18720))
- LibreNMS: default to an 8 GB disk [@MickLesk](https://github.com/MickLesk) ([#18723](https://github.com/community-scripts/ProxmoxVE/pull/18723))
## 2026-11-13
### 🆕 New Scripts
- marimo ([#18672](https://github.com/community-scripts/ProxmoxVE/pull/18672))
- GoatCounter ([#18662](https://github.com/community-scripts/ProxmoxVE/pull/18662))
- Unsloth ([#18697](https://github.com/community-scripts/ProxmoxVE/pull/18697))
- colanode ([#18685](https://github.com/community-scripts/ProxmoxVE/pull/18685))
- SimpleLogin ([#18680](https://github.com/community-scripts/ProxmoxVE/pull/18680))
- Nightingale ([#18674](https://github.com/community-scripts/ProxmoxVE/pull/18674))
- Temporal ([#18670](https://github.com/community-scripts/ProxmoxVE/pull/18670))
- TrailBase ([#18669](https://github.com/community-scripts/ProxmoxVE/pull/18669))
- Weaviate ([#18671](https://github.com/community-scripts/ProxmoxVE/pull/18671))
- Spacebot ([#18668](https://github.com/community-scripts/ProxmoxVE/pull/18668))
- New-API ([#18667](https://github.com/community-scripts/ProxmoxVE/pull/18667))
- Martin ([#18666](https://github.com/community-scripts/ProxmoxVE/pull/18666))
- Langflow ([#18665](https://github.com/community-scripts/ProxmoxVE/pull/18665))
- Inngest ([#18664](https://github.com/community-scripts/ProxmoxVE/pull/18664))
- Hatchet ([#18663](https://github.com/community-scripts/ProxmoxVE/pull/18663))
- Engram ([#18661](https://github.com/community-scripts/ProxmoxVE/pull/18661))
- Euro-Office ([#18660](https://github.com/community-scripts/ProxmoxVE/pull/18660))
- Cypht ([#18659](https://github.com/community-scripts/ProxmoxVE/pull/18659))
- AirConnect ([#18653](https://github.com/community-scripts/ProxmoxVE/pull/18653))
- Bulwark ([#18656](https://github.com/community-scripts/ProxmoxVE/pull/18656))
- Convex ([#18658](https://github.com/community-scripts/ProxmoxVE/pull/18658))
- Chroma ([#18657](https://github.com/community-scripts/ProxmoxVE/pull/18657))
- Bugsink ([#18655](https://github.com/community-scripts/ProxmoxVE/pull/18655))
- 9Router ([#18654](https://github.com/community-scripts/ProxmoxVE/pull/18654))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- wikijs: fix missing backend deps on amd64 [@CrazyWolf13](https://github.com/CrazyWolf13) ([#18715](https://github.com/community-scripts/ProxmoxVE/pull/18715))
### 💾 Core
- tools: Issue self-signed certificates for 825 days and renew expiring ones [@MickLesk](https://github.com/MickLesk) ([core#251](https://github.com/community-scripts/core/pull/251))
- core: start and restart a .deb's services after binary mode installs it [@MickLesk](https://github.com/MickLesk) ([core#250](https://github.com/community-scripts/core/pull/250))
- forge: find a pinned tag whether or not it is written with its v [@MickLesk](https://github.com/MickLesk) ([core#249](https://github.com/community-scripts/core/pull/249))
- core: unpack a compressed single binary in singlefile mode [@MickLesk](https://github.com/MickLesk) ([core#248](https://github.com/community-scripts/core/pull/248))
- forge: skip pre-release tags in get_latest_gh_tag [@MickLesk](https://github.com/MickLesk) ([core#263](https://github.com/community-scripts/core/pull/263))
- tools: use MySQL's current signing key and the trixie suite [@MickLesk](https://github.com/MickLesk) ([core#262](https://github.com/community-scripts/core/pull/262))
- tools: install a bare executable from a URL [@MickLesk](https://github.com/MickLesk) ([core#260](https://github.com/community-scripts/core/pull/260))
- tools: match packageManager pins without their hash [@MickLesk](https://github.com/MickLesk) ([core#258](https://github.com/community-scripts/core/pull/258))
- tools: build Ruby on disk and without docs [@MickLesk](https://github.com/MickLesk) ([core#257](https://github.com/community-scripts/core/pull/257))
- forge: rank a trailing letter as a revision in the version key [@MickLesk](https://github.com/MickLesk) ([core#256](https://github.com/community-scripts/core/pull/256))
- fix: pin tag-only apps with var_appversion too [@MickLesk](https://github.com/MickLesk) ([core#255](https://github.com/community-scripts/core/pull/255))
- tools: read the mounts once when emptying a target for a clean install [@MickLesk](https://github.com/MickLesk) ([core#254](https://github.com/community-scripts/core/pull/254))
- forge: use the designated release for a tag prefix when it carries the prefix [@MickLesk](https://github.com/MickLesk) ([core#252](https://github.com/community-scripts/core/pull/252))
- gpu / hwaccell: type GPUs by PCI ID and install the right NVIDIA packages on Ubuntu [@MickLesk](https://github.com/MickLesk) ([core#247](https://github.com/community-scripts/core/pull/247))
- Clear yarn 1's root cache in cleanup_lxc [@MickLesk](https://github.com/MickLesk) ([core#246](https://github.com/community-scripts/core/pull/246))
- fix: run Meilisearch --upgrade-db on patch updates too [@CrazyWolf13](https://github.com/CrazyWolf13) ([core#259](https://github.com/community-scripts/core/pull/259))
- db: only chmod a database credentials file that is a regular file [@MickLesk](https://github.com/MickLesk) ([core#253](https://github.com/community-scripts/core/pull/253))
- fix: take a Debian cloud image from the newest build that has it [@MickLesk](https://github.com/MickLesk) ([core#261](https://github.com/community-scripts/core/pull/261))
- telemetry: always on for DevScripts, with a one-time notice [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([core#245](https://github.com/community-scripts/core/pull/245))
- Shield pct start and the template check from a closed console [@MickLesk](https://github.com/MickLesk) ([core#239](https://github.com/community-scripts/core/pull/239))
- core: say why a container stopped by itself during its setup [@MickLesk](https://github.com/MickLesk) ([core#234](https://github.com/community-scripts/core/pull/234))
- Download a corrupt local template again instead of keeping it [@MickLesk](https://github.com/MickLesk) ([core#235](https://github.com/community-scripts/core/pull/235))
- fetch the install script before the container is created [@MickLesk](https://github.com/MickLesk) ([core#236](https://github.com/community-scripts/core/pull/236))
## 2026-11-12
### 🆕 New Scripts
- Pixelfed ([#18523](https://github.com/community-scripts/ProxmoxVE/pull/18523))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- OnlyOffice: change ct port to 8083 [@MickLesk](https://github.com/MickLesk) ([#18642](https://github.com/community-scripts/ProxmoxVE/pull/18642))
- dotnetaspwebapi: start the service only once an app is deployed [@MickLesk](https://github.com/MickLesk) ([#18639](https://github.com/community-scripts/ProxmoxVE/pull/18639))
- Whisparr-eros: set the first login and save it [@MickLesk](https://github.com/MickLesk) ([#18617](https://github.com/community-scripts/ProxmoxVE/pull/18617))
- #### 💥 Breaking Changes
- Updated ASP.NET Web App script to Debian and adds a .Net selection [@kris701](https://github.com/kris701) ([#18628](https://github.com/community-scripts/ProxmoxVE/pull/18628))
### 💾 Core
- github: bump workflow actions to their current majors [@MickLesk](https://github.com/MickLesk) ([core#240](https://github.com/community-scripts/core/pull/240))
- Say what the host sees when a container gets no address [@MickLesk](https://github.com/MickLesk) ([core#230](https://github.com/community-scripts/core/pull/230))
- Fall back to public DNS for Alpine containers too [@MickLesk](https://github.com/MickLesk) ([core#231](https://github.com/community-scripts/core/pull/231))
- Do not fail a finished install over the monitoring service [@MickLesk](https://github.com/MickLesk) ([core#232](https://github.com/community-scripts/core/pull/232))
- Deploy a release by moving it, and stop on a full disk² [@MickLesk](https://github.com/MickLesk) ([core#233](https://github.com/community-scripts/core/pull/233))
- core: say when pct create fails because timedatectl does not answer [@MickLesk](https://github.com/MickLesk) ([core#237](https://github.com/community-scripts/core/pull/237))
- ref: stop calling every exit 255 a dpkg error [@MickLesk](https://github.com/MickLesk) ([core#238](https://github.com/community-scripts/core/pull/238))
- Write uv's shell completions without STD [@MickLesk](https://github.com/MickLesk) ([core#241](https://github.com/community-scripts/core/pull/241))
- Generate database passwords with random_password / Keep the database helpers' credentials files private [@MickLesk](https://github.com/MickLesk) ([core#242](https://github.com/community-scripts/core/pull/242))
### 📚 Documentation
- github: bump workflow actions to their current majors [@MickLesk](https://github.com/MickLesk) ([#18637](https://github.com/community-scripts/ProxmoxVE/pull/18637))
- pocketbase-bot: apply website metadata requests from Accept/Decline boxes [@MickLesk](https://github.com/MickLesk) ([#18631](https://github.com/community-scripts/ProxmoxVE/pull/18631))
## 2026-11-11
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- TrueNas: Fixing isolookup and dialog [@juronja](https://github.com/juronja) ([#18578](https://github.com/community-scripts/ProxmoxVE/pull/18578))
- #### ✨ New Features
- Immich: bump to 3.4.1 [@MickLesk](https://github.com/MickLesk) ([#18619](https://github.com/community-scripts/ProxmoxVE/pull/18619))
- Immich: bump to 3.4.0 [@MickLesk](https://github.com/MickLesk) ([#18600](https://github.com/community-scripts/ProxmoxVE/pull/18600))
- #### 💥 Breaking Changes
- Pocketbase: install and update from the new host (removed in github) [@MickLesk](https://github.com/MickLesk) ([#18609](https://github.com/community-scripts/ProxmoxVE/pull/18609))
- #### 🔧 Refactor
- Paperless-AI: install and update to 4.0 [@MickLesk](https://github.com/MickLesk) ([#18620](https://github.com/community-scripts/ProxmoxVE/pull/18620))
### 💾 Core
- core: drop the SUDO_ variables an update through sudo carries [@MickLesk](https://github.com/MickLesk) ([core#229](https://github.com/community-scripts/core/pull/229))
- feat: let setup_deb822_repo use a keyring already on the system [@MickLesk](https://github.com/MickLesk) ([core#227](https://github.com/community-scripts/core/pull/227))
### 📚 Documentation
- Fill platforms when a script moves from DevScripts [@MickLesk](https://github.com/MickLesk) ([#18611](https://github.com/community-scripts/ProxmoxVE/pull/18611))
## 2026-11-10
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- writefreely: Force symbolic link update [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([#18588](https://github.com/community-scripts/ProxmoxVE/pull/18588))
- openproject: start memcached when IPv6 is disabled [@MickLesk](https://github.com/MickLesk) ([#18584](https://github.com/community-scripts/ProxmoxVE/pull/18584))
- homeassistant.sh: Update pip install command for runlike [@Zajsoft](https://github.com/Zajsoft) ([#18583](https://github.com/community-scripts/ProxmoxVE/pull/18583))
- Tunarr: fix install and update for the new release layout and latest ErsatzTV-ffmpeg [@chrisbenincasa](https://github.com/chrisbenincasa) ([#18577](https://github.com/community-scripts/ProxmoxVE/pull/18577))
### 💾 Core
- fix: keep a script's config file when a package installs it [@MickLesk](https://github.com/MickLesk) ([core#226](https://github.com/community-scripts/core/pull/226))
- fix: flush the image before importing a VM disk [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([core#223](https://github.com/community-scripts/core/pull/223))
### 📂 Github
- pr-script-test: pre-answer prompt variables built in a loop [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([#18596](https://github.com/community-scripts/ProxmoxVE/pull/18596))
- Build PR scripts on a Proxmox test host via /test [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([#18594](https://github.com/community-scripts/ProxmoxVE/pull/18594))
## 2026-11-09
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- zipline: Update RAM to 3GB [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18569](https://github.com/community-scripts/ProxmoxVE/pull/18569))
- OpenObserve: record installed version, drop killall ExecStop [@BadFlo](https://github.com/BadFlo) ([#18566](https://github.com/community-scripts/ProxmoxVE/pull/18566))
- OpenObserve: extract release tarball with --no-same-owner [@BadFlo](https://github.com/BadFlo) ([#18564](https://github.com/community-scripts/ProxmoxVE/pull/18564))
- Immich: download before stopping, and clear maintenance mode after an interrupted update [@angusmaul](https://github.com/angusmaul) ([#18553](https://github.com/community-scripts/ProxmoxVE/pull/18553))
- cinephage: bump Node.js from 24 to 26 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18374](https://github.com/community-scripts/ProxmoxVE/pull/18374))
- kan: bump Node.js from 20 to 24 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18549](https://github.com/community-scripts/ProxmoxVE/pull/18549))
- myip: bump Node.js from 24 to 26 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18550](https://github.com/community-scripts/ProxmoxVE/pull/18550))
- #### 🔧 Refactor
- security: remove hardcoded credentials in several scripts [@MickLesk](https://github.com/MickLesk) ([#18572](https://github.com/community-scripts/ProxmoxVE/pull/18572))
- yuvomi: install from the attested release asset [@ulsklyc](https://github.com/ulsklyc) ([#18554](https://github.com/community-scripts/ProxmoxVE/pull/18554))
### 🗑️ Deleted Scripts
- coolify: delete old ct / its an addon [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([#18562](https://github.com/community-scripts/ProxmoxVE/pull/18562))
### 💾 Core
- core: detect GPUs by their PCI class, not by text anywhere in the line [@MickLesk](https://github.com/MickLesk) ([core#221](https://github.com/community-scripts/core/pull/221))
- core: load an app's own settings from its .vars file [@MickLesk](https://github.com/MickLesk) ([core#216](https://github.com/community-scripts/core/pull/216))
### 🧰 Tools
- #### 🐞 Bug Fixes
- IP-Tag: filter guest-agent IPs by CIDR before limiting to 3 [@mazelado](https://github.com/mazelado) ([#18537](https://github.com/community-scripts/ProxmoxVE/pull/18537))
## 2026-11-08
### 🆕 New Scripts
- Chatto ([#18521](https://github.com/community-scripts/ProxmoxVE/pull/18521))
- Firefox-Syncserver ([#18520](https://github.com/community-scripts/ProxmoxVE/pull/18520))
### 🚀 Updated Scripts
- #### 🔧 Refactor
- Immich: do not clone libjxl's testdata [@MickLesk](https://github.com/MickLesk) ([#18539](https://github.com/community-scripts/ProxmoxVE/pull/18539))
- Databasus: build 3.65 and ship the verification agent [@MickLesk](https://github.com/MickLesk) ([#18500](https://github.com/community-scripts/ProxmoxVE/pull/18500))
### 💾 Core
- fix: a closed console no longer ends a running setup | ERR trap fixes | check VM ID before creating [@MickLesk](https://github.com/MickLesk) ([core#219](https://github.com/community-scripts/core/pull/219))
### 🧰 Tools
- #### 🐞 Bug Fixes
- update-apps: keep the backup alive on a closed console, stop before restoring [@MickLesk](https://github.com/MickLesk) ([#18540](https://github.com/community-scripts/ProxmoxVE/pull/18540))
- clean-lxcs: run with sh and keep going when one container fails [@MickLesk](https://github.com/MickLesk) ([#18538](https://github.com/community-scripts/ProxmoxVE/pull/18538))
- update-lxcs: keep going when one container fails [@MickLesk](https://github.com/MickLesk) ([#18541](https://github.com/community-scripts/ProxmoxVE/pull/18541))
## 2026-11-07
### 🆕 New Scripts
+96 -12
View File
@@ -1,9 +1,21 @@
name: ARM script test (manual)
# Builds one ct/ script on GitHub's free arm64 runner, inside Incus. Started
# by hand from the Actions tab; the result is in the job summary and the
# artifact. The workflow lives in community-scripts/ci, shared with
# DevScripts/ProxmoxVE; this file is identical in both repositories.
# Builds one ct/ script on GitHub's free arm64 runner (ubuntu-24.04-arm),
# inside Incus, with the same harness as pr-script-test.yml. Started by hand
# from the Actions tab; it posts nothing anywhere: the result is in the job
# summary and the artifact (with the screenshots).
#
# pr-script-test.yml runs the same ARM build for every /test, as information
# only. This one is for trying a script or branch on ARM without a PR.
#
# Public repositories get the standard hosted runners, arm64 included, at no
# cost. Each job is a fresh VM that is thrown away afterwards, so the script
# under test never touches the lab.
#
# The engine refuses to guess on arm64: a script that leaves var_arm64 unset
# stops with "refusing to guess". Here var_arm64=no means "not built", yes is
# built as is, and unset is built with var_arm64=yes to see whether it would
# work.
on:
workflow_dispatch:
@@ -18,13 +30,85 @@ on:
default: main
type: string
permissions: {}
permissions:
contents: read
jobs:
arm-script-test:
permissions:
contents: read
uses: community-scripts/ci/.github/workflows/arm-script-test.yml@main
with:
script: ${{ inputs.script }}
ref: ${{ inputs.ref }}
arm:
runs-on: ubuntu-24.04-arm
timeout-minutes: 120
steps:
# The harness from the default branch, as in pr-script-test.yml.
- uses: actions/checkout@v7
with:
sparse-checkout: .github/workflows/scripts
persist-credentials: false
- name: Decide var_arm64
id: plan
env:
SCRIPT: ${{ inputs.script }}
REF: ${{ inputs.ref }}
GH_TOKEN: ${{ github.token }}
run: |
[[ "$SCRIPT" =~ ^ct/[A-Za-z0-9._-]+\.sh$ ]] || { echo "::error::script must be ct/<name>.sh"; exit 1; }
[[ "$REF" =~ ^[A-Za-z0-9._/-]+$ ]] || { echo "::error::bad ref"; exit 1; }
sha=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${REF}" --jq .sha)
text=$(gh api "repos/${GITHUB_REPOSITORY}/contents/${SCRIPT}?ref=${sha}" --jq .content | base64 -d)
# A commented-out line ("#var_arm64=...") counts as unset.
arm=$(grep -m1 -oE '^[[:space:]]*var_arm64="[$][{]var_arm64:-(yes|no)[}]"' <<<"$text" | grep -oE ':-(yes|no)' | cut -c3- || true)
echo "sha=$sha" >>"$GITHUB_OUTPUT"
echo "arm64=${arm:-unset}" >>"$GITHUB_OUTPUT"
case "$arm" in
no) echo "::notice::${SCRIPT} sets var_arm64=no: not built on ARM" ;;
yes) echo "${SCRIPT} sets var_arm64=yes" ;;
*) echo "::notice::${SCRIPT} leaves var_arm64 unset: building with var_arm64=yes to see whether it works" ;;
esac
echo "Building ${SCRIPT} at ${sha:0:7} on $(uname -m), $(nproc) cores, $(free -g | awk '/^Mem/ {print $2}') GB RAM, $(df -h / | awk 'NR==2 {print $4}') free"
- name: Set up Incus
if: steps.plan.outputs.arm64 != 'no'
timeout-minutes: 20 # about 2 when healthy; a hang must not hold the report
run: sudo bash .github/workflows/scripts/arm-runner-setup.sh
- name: Build and check ${{ inputs.script }}
if: steps.plan.outputs.arm64 != 'no'
env:
PLATFORM: incus
VAR_ARM64: "yes"
PR_REPO: ${{ github.repository }}
PR_SHA: ${{ steps.plan.outputs.sha }}
SCRIPT: ${{ inputs.script }}
UPSTREAM: ${{ github.repository }}
CTID_BASE: "9000"
GH_API_TOKEN: ${{ github.token }}
OUT_DIR: ${{ runner.temp }}/result
BUILD_TIMEOUT: 90m
run: sudo --preserve-env=PLATFORM,VAR_ARM64,PR_REPO,PR_SHA,SCRIPT,UPSTREAM,CTID_BASE,GH_API_TOKEN,OUT_DIR,BUILD_TIMEOUT,GITHUB_RUN_ID,GITHUB_RUN_ATTEMPT bash .github/workflows/scripts/pve-run-script.sh
- name: Summary
if: always() && steps.plan.outputs.arm64 != 'no'
env:
OUT_DIR: ${{ runner.temp }}/result
ARM64: ${{ steps.plan.outputs.arm64 }}
run: |
sudo chmod -R a+rX "$OUT_DIR" 2>/dev/null || true
r="$OUT_DIR/result.json"
[[ -f "$r" ]] || { echo "No result.json" >>"$GITHUB_STEP_SUMMARY"; exit 0; }
{
echo "### ARM (arm64, Incus): $(jq -r .script "$r")"
echo
echo "| Status | Install | Update | Duration | var_arm64 | Detail |"
echo "|---|---|---|---|---|---|"
jq -r --arg arm "$ARM64" '"| \(.status) | \(if .update_status != "" then "pass" else .status end) | \(if .update_status == "" then "-" else .update_status end) | \(.duration)s | \($arm) | \(if .status == "pass" then (.url // "" | if . == "" then "no URL" else . + " → " + $ARGS.named.c end) else .step end) |"' --arg c "$(jq -r .http_code "$r")" "$r"
echo
ls "$OUT_DIR"/screenshot*.png &>/dev/null && echo "Screenshots: in the \`result-arm\` artifact." || echo "No screenshot."
} >>"$GITHUB_STEP_SUMMARY"
- if: always()
uses: actions/upload-artifact@v7
with:
name: result-arm
path: ${{ runner.temp }}/result
if-no-files-found: ignore
retention-days: 14
+831 -21
View File
@@ -1,14 +1,35 @@
name: PR script test
# Builds the ct/ and vm/ scripts a pull request touches on real Proxmox and
# Incus hosts and reports the outcome on the PR, as a check run and as one
# comment kept up to date. Comment `/test` (optionally a full SHA, app names
# and platforms) to build again.
# Builds the ct/ and vm/ scripts a pull request touches on real Proxmox and Incus hosts and
# reports the outcome on the PR, as a check run and as one comment kept up to
# date.
#
# The workflow itself lives in community-scripts/ci and is shared with
# DevScripts/ProxmoxVE; this file only holds the triggers and the queue, and
# is identical in both repositories. It runs in this repository's context:
# its token, its runners, its PRs.
# Every push to a non-draft PR is built automatically, forks included: the
# maintainers decided (2026-11-12) that the lab hosts run contributors' code
# unattended. They are isolated, hold no secret, and are treated as
# compromised anyway; the sweeper and fresh guests per build keep them usable.
# Someone with write access can also comment `/test` (optionally followed by
# a full SHA to pin, the scripts to build, and the platforms to build on) to
# build again or pick what to build. Only a commit that existed when the
# comment was written gets built: a push after it is never covered, not even
# one that lands in the seconds before this job runs.
#
# Approval stays narrow: only PRs the push-app-to-main app opens on a branch
# of this repository (a script moving over from DevScripts, reviewed and
# tested there) are approved when every script passes. Every other PR gets
# the report, the check, and "changes requested" on a real failure.
#
# gate ubuntu-latest who asked, which scripts, which commit
# test proxmox-test / build + checks, one matrix job per script, platform
# incus-test / (pve, incus, arm) and OS; arm runs on GitHub's
# ubuntu-24.04-arm arm64 runner and is information only
# report ubuntu-latest check run + PR comment
#
# The test job gets a read-only token and never sees a secret: the host it runs
# on executes untrusted code and has to be treated as compromised.
#
# This file is identical in DevScripts and ProxmoxVE; github.repository decides
# which URLs get rewritten and which container ID range is used.
on:
issue_comment:
@@ -17,8 +38,8 @@ on:
# default branch, as `/test <app>` in the issue would.
issues:
types: [labeled]
# pull_request_target runs this file from main, never the PR's copy of it,
# and does not wait for "approve and run" on forks.
# Every PR, see above. pull_request_target runs this file from main, never
# the PR's copy of it, and does not wait for "approve and run" on forks.
pull_request_target:
types: [opened, reopened, synchronize, ready_for_review]
paths:
@@ -57,19 +78,808 @@ concurrency:
cancel-in-progress: false
jobs:
pr-script-test:
# The most any job in the shared workflow gets; each job there asks for
# less (the test job: contents read only).
gate:
if: >-
github.event_name == 'workflow_dispatch' ||
(github.event_name == 'issue_comment' && startsWith(github.event.comment.body, '/test')) ||
(github.event_name == 'issues' && github.event.label.name == 'Ready For Testing' && !github.event.issue.pull_request) ||
(github.event_name == 'pull_request_target' && !github.event.pull_request.draft)
runs-on: ubuntu-latest
permissions:
actions: read
actions: read # when the run was dispatched, for a /test without SHA
checks: write
contents: read
issues: write
pull-requests: write
uses: community-scripts/ci/.github/workflows/pr-script-test.yml@main
with:
screenshots_client_id: ${{ vars.GHAPP_SCREENSHOTS_CLIENT_ID }}
# Only the screenshot App, and only the report job uses it: never
# `secrets: inherit`, the test jobs run PR code.
secrets:
screenshots_app_key: ${{ secrets.GHAPP_SCREENSHOTS_PRIVATE_KEY }}
outputs:
matrix: ${{ steps.plan.outputs.matrix }}
pr: ${{ steps.plan.outputs.pr }}
repo: ${{ steps.plan.outputs.repo }}
sha: ${{ steps.plan.outputs.sha }}
check_id: ${{ steps.plan.outputs.check_id }}
actor: ${{ steps.plan.outputs.actor }}
not_built: ${{ steps.plan.outputs.not_built }}
trusted: ${{ steps.plan.outputs.trusted }}
kind: ${{ steps.plan.outputs.kind }}
platforms: ${{ steps.plan.outputs.platforms }}
base_sha: ${{ steps.plan.outputs.base_sha }}
steps:
- id: plan
uses: actions/github-script@v9
with:
script: |
const { owner, repo } = context.repo;
const MAX = 10;
core.setOutput('matrix', '[]');
// Trusted: a PR the push-app-to-main app opened on a branch of this
// repository, i.e. a script moving over from DevScripts.
const TRUSTED_AUTHOR = 'push-app-to-main[bot]';
const isTrusted = p => !!p && p.user && p.user.login === TRUSTED_AUTHOR && p.user.type === 'Bot' &&
!!p.head.repo && p.head.repo.full_name === `${owner}/${repo}`;
let prNumber, requested = '';
if (context.eventName === 'pull_request_target') {
const p = context.payload.pull_request;
prNumber = p.number;
// Exactly the pushed commit; a later push starts its own run.
requested = p.head.sha;
} else if (context.eventName === 'workflow_dispatch') {
prNumber = Number(context.payload.inputs.pr);
requested = `${context.payload.inputs.sha || ''} ${context.payload.inputs.scripts || ''}`;
} else if (context.eventName === 'issues') {
// "Ready For Testing" on a script issue: the app the issue form
// names, as `/test <app>` would. Only people who can triage set a
// label, and the default branch is what gets built.
const issue = context.payload.issue;
prNumber = issue.number;
// Triage can label too; building needs write, as for /test. The
// merge workflow labels through an app, i.e. a bot.
const sender = context.payload.sender;
if (sender.type !== 'Bot') {
const { data: perm } = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username: sender.login });
if (!['admin', 'maintain', 'write'].includes(perm.permission)) {
core.notice(`${sender.login} has ${perm.permission} access; a test needs write`);
return;
}
}
const body = (issue.body || '').replace(/\r/g, '');
const field = name => {
const m = body.match(new RegExp(`^###[ \t]*${name}[ \t]*\n+([^\n#][^\n]*)`, 'm'));
return m && m[1].trim() !== '_No response_' ? m[1].trim() : '';
};
const type = field('Script Type');
if (type && !/^(CT|VM)\b/.test(type)) { core.notice(`${type}: only ct/ and vm/ scripts are built`); return; }
const path = field('Script path').replace(/^\/+/, '');
// Same slug rule as pr-merged-ready-for-testing.yml: the "Name of
// the Script" field, else the title, lowercased, spaces -> dashes.
const slug = (field('Name of the Script') || issue.title).trim().toLowerCase().replace(/\s+/g, '-');
requested = /^(ct|vm)\/[A-Za-z0-9._-]+\.sh$/.test(path) ? path : slug;
core.info(`Ready For Testing: building ${requested}`);
} else {
prNumber = context.payload.issue.number;
const body = context.payload.comment.body.trim();
if (!/^\/test(\s|$)/.test(body)) return;
// author_association hides private org members, so ask for the permission itself.
const user = context.payload.comment.user.login;
const { data: perm } = await github.rest.repos.getCollaboratorPermissionLevel({ owner, repo, username: user });
if (!['admin', 'maintain', 'write'].includes(perm.permission)) {
core.notice(`${user} has ${perm.permission} access; /test needs write`);
return;
}
await github.rest.reactions.createForIssueComment({
owner, repo, comment_id: context.payload.comment.id, content: 'rocket',
});
requested = body.split('\n')[0].slice('/test'.length);
}
// `/test <app>` in an issue (not a PR) builds that app from the
// default branch as it is right now, e.g. to see whether a reported
// error reproduces. No PR, so no check run, no review, no approval.
const isIssue = ['issue_comment', 'issues'].includes(context.eventName) && !context.payload.issue.pull_request;
let pr = null, head, headSha;
if (isIssue) {
const { data: info } = await github.rest.repos.get({ owner, repo });
const { data: branch } = await github.rest.repos.getBranch({ owner, repo, branch: info.default_branch });
headSha = branch.commit.sha;
head = { owner: { login: owner }, name: repo, full_name: `${owner}/${repo}` };
} else {
({ data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber }));
if (pr.state !== 'open' || !pr.head.repo) { core.notice('PR is closed or its fork is gone'); return; }
head = pr.head.repo;
headSha = pr.head.sha;
if (context.eventName === 'pull_request_target' && headSha !== requested) {
core.notice('Pushed to again since; the newer push has its own run');
return;
}
}
core.setOutput('kind', isIssue ? 'issue' : 'pr');
core.setOutput('trusted', !isIssue && isTrusted(pr) ? 'true' : 'false');
const MARK = '<!-- pr-script-test -->';
const sticky = async body => {
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number: prNumber });
// Only our own comment: anyone can paste the marker into theirs.
const mine = comments.find(c => c.body.includes(MARK) && c.user.type === 'Bot');
if (mine) await github.rest.issues.updateComment({ owner, repo, comment_id: mine.id, body });
else await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body });
};
// Which commit the approval covers. Reading pr.head.sha alone would
// build whatever was pushed between the comment and this job
// starting, i.e. code nobody looked at. Two ways to be sure:
//
// /test <sha> that exact commit, and only while it is the head. A
// full SHA: the author writes both commits, so a short
// prefix is a quick birthday collision away.
// /test the latest commit, if nothing reached the fork after
// the comment. pushed_at is set by GitHub on every push
// and read after the PR, so a push in between shows up.
// Branches of this repository belong to people with
// write access, who could build anything anyway.
const words = requested.split(/\s+/).filter(Boolean);
// Platform words pick where to build: `/test arm`, `/test pve incus
// bark-server`. None means every platform. A run on only some of
// them is reported under its own check name and never approves.
const PLATFORM = { pve: 'pve', proxmox: 'pve', incus: 'incus', arm: 'arm', arm64: 'arm', aarch64: 'arm' };
const platformOf = t => PLATFORM[t.toLowerCase()];
const only = new Set(words.map(platformOf).filter(Boolean));
const onlyList = ['pve', 'incus', 'arm'].filter(p => only.has(p));
const tokens = words.filter(t => !platformOf(t));
const isSha = t => /^[0-9a-f]{7,40}$/.test(t);
// App names (bark-server) become ct/ paths; such a name without a
// ct/ script falls back to vm/ below.
const bare = new Set();
const toPath = t => {
if (/^(ct|vm)\/[A-Za-z0-9._-]+\.sh$/.test(t) || !/^[A-Za-z0-9._-]+$/.test(t)) return t;
const p = `ct/${t.replace(/\.sh$/, '')}.sh`;
bare.add(p);
return p;
};
const approved = isIssue ? [] : tokens.filter(isSha);
let entries = (isIssue ? tokens : tokens.filter(t => !isSha(t))).map(toPath);
const refuse = async (why, retry) => {
await sticky([MARK, '### Proxmox script test', '', `⚠️ ${why} Nothing was built.`, '', retry].join('\n'));
core.notice(why);
};
if (isIssue) {
// The default branch is trusted; only which app is needed.
if (entries.length === 0) {
await refuse('In an issue, `/test` needs to know which app.', 'Comment `/test <app>`, for example `/test bark-server` (the name of its `ct/` script).');
return;
}
} else if (approved.length > 1) {
await refuse('`/test` takes at most one commit.', 'Comment `/test` to build the latest commit.');
return;
}
if (approved.length === 1 && approved[0] !== headSha) {
await refuse(
approved[0].length !== 40 && headSha.startsWith(approved[0])
? 'A short SHA is not enough, it has to be all 40 characters.'
: `\`${approved[0]}\` is not the head of this PR (any more).`,
'Comment `/test` to build the latest commit.');
return;
}
if (!isIssue && approved.length === 0 && head.full_name !== `${owner}/${repo}`) {
const askedAt = context.eventName === 'workflow_dispatch'
? (await github.rest.actions.getWorkflowRun({ owner, repo, run_id: context.runId })).data.created_at
: context.payload.comment.created_at;
const { data: fork } = await github.rest.repos.get({ owner: head.owner.login, repo: head.name });
if (new Date(fork.pushed_at) >= new Date(askedAt)) {
await refuse(
`Something was pushed to ${head.full_name} after the \`/test\`, so it is not certain which commit was meant.`,
'Comment `/test` again once you have looked at the latest commit.');
return;
}
}
if (entries.some(e => !/^(ct|vm)\/[A-Za-z0-9._-]+\.sh$/.test(e))) {
await refuse(
isIssue ? 'Only app names (as in `ct/<app>.sh` or `vm/<name>.sh`) and platforms (`pve`, `incus`, `arm`) can be requested.' : 'Only app names, `ct/<name>.sh` and `vm/<name>.sh` paths, platforms (`pve`, `incus`, `arm`) and the commit can be requested.',
isIssue ? 'For example: `/test bark-server`, `/test arm bark-server` or `/test debian-vm`' : 'For example: `/test`, `/test arm`, or `/test pve incus bark-server`.');
return;
}
// Everything the PR changes that this test cannot build is named in
// the comment, so a /test never ends in silence.
const notBuilt = [];
const code = s => '`' + String(s).replace(/[`\r\n]/g, '') + '`';
if (entries.length === 0) {
const files = await github.paginate(github.rest.pulls.listFiles, { owner, repo, pull_number: prNumber });
const set = new Set();
for (const f of files) {
if (f.status === 'removed') continue;
let m;
if ((m = f.filename.match(/^install\/([A-Za-z0-9._-]+)-install\.sh$/))) set.add(`ct/${m[1]}.sh`);
else if (/^(ct|vm)\/[A-Za-z0-9._-]+\.sh$/.test(f.filename)) set.add(f.filename);
else if (/^(tools|turnkey)\/.*\.sh$/.test(f.filename)) notBuilt.push(`${code(f.filename)}: only \`ct/\` and \`vm/\` scripts are built`);
}
entries = [...set].sort();
}
const include = [];
// Upgrade test: install the ct/ script from the PR's base branch,
// then update with the PR, as a user who installed before the merge
// would. Only in ProxmoxVE, where users have installed it, and only
// when the PR changes update_script(): with the same update code an
// existing install updates exactly as it does today. New scripts
// have nothing to upgrade from. Proxmox only, once per script.
const UPGRADE_REPO = 'community-scripts/ProxmoxVE';
let baseSha = '';
if (!isIssue && `${owner}/${repo}` === UPGRADE_REPO) {
const { data: b } = await github.rest.repos.getBranch({ owner, repo, branch: pr.base.ref });
baseSha = b.commit.sha;
}
// From `function update_script() {` to the first `}` at the start of
// a line; null without one.
const updateCode = t => ((t || '').match(/^(function[ \t]+)?update_script[ \t]*\(\)[ \t]*\{[\s\S]*?^\}/m) || [null])[0];
const updateChanged = async (p, headText) => {
if (!baseSha) return false;
let baseText;
try {
const res = await github.rest.repos.getContent({ owner, repo, path: p, ref: baseSha });
baseText = Buffer.from(res.data.content || '', 'base64').toString('utf8');
} catch (e) { return false; }
const before = updateCode(baseText), after = updateCode(headText);
return before !== null && after !== null && before !== after;
};
const fetchScript = async p => {
try {
const res = await github.rest.repos.getContent({ owner: head.owner.login, repo: head.name, path: p, ref: headSha });
return Buffer.from(res.data.content || '', 'base64').toString('utf8');
} catch (e) { return null; }
};
for (let script of entries.slice(0, MAX)) {
let text = await fetchScript(script);
// /test debian-vm in an issue: there is no ct/debian-vm.sh, but a vm/ one.
const wasBare = isIssue || bare.has(script);
if (text === null && wasBare && script.startsWith('ct/')) {
const vm = `vm/${script.slice(3)}`;
if ((text = await fetchScript(vm)) !== null) script = vm;
}
if (text === null) {
const where = isIssue ? 'on the default branch' : 'in this PR';
notBuilt.push(wasBare
? (script.startsWith('ct/') ? `${code(script.slice(3, -3))}: neither \`ct/\` nor \`vm/\` has it ${where}` : `${code(script)} does not exist ${where}`)
: `\`${script}\` does not exist in this PR (an \`install/\` script is built through the \`ct/\` script of the same name)`);
continue;
}
const isVm = script.startsWith('vm/');
// The test hosts are amd64; an ARM-only VM script refuses to run there.
if (isVm && /^\s*vm_require_arch\s+arm64\b/m.test(text)) {
notBuilt.push(`${code(script)}: ARM64 only, the test hosts are amd64`);
continue;
}
// Scripts that ask for the OS get built once per OS.
const oses = !isVm && /msg_menu\s+"Choose the container OS"/.test(text) ? ['debian', 'alpine'] : [''];
const name = isVm ? `vm-${script.slice(3, -3)}` : script.slice(3, -3);
// Every ct/ script on every platform: the same files run on Proxmox
// and, through community-scripts/core's Incus backend, on Incus.
// VMs are Proxmox only. ARM (Incus on GitHub's arm64 runner) is
// information only and never decides the result; a script that
// says var_arm64=no is not built there. Unset is built with
// var_arm64=yes, to see whether it would work.
const arm = (text.match(/^[ \t]*var_arm64="\$\{var_arm64:-(yes|no)\}"/m) || [])[1] || 'unset';
let platforms = isVm ? ['pve'] : arm === 'no' ? ['pve', 'incus'] : ['pve', 'incus', 'arm'];
if (!isVm && arm === 'no' && (!only.size || only.has('arm'))) notBuilt.push(`${code(script)} on ARM64: the script sets \`var_arm64=no\``);
if (only.size) {
if (isVm && !only.has('pve')) notBuilt.push(`${code(script)}: VM scripts are built on \`pve\` only`);
platforms = platforms.filter(p => only.has(p));
}
for (const platform of platforms) {
for (const os of oses) include.push({ script, os, platform, arm64: platform === 'arm' ? arm : '', mode: '', name: `${os ? `${name}-${os}` : name}-${platform}` });
}
if (!isVm && platforms.includes('pve') && await updateChanged(script, text)) {
const os = oses[0];
include.push({ script, os, platform: 'pve', arm64: '', mode: 'upgrade', name: `${os ? `${name}-${os}` : name}-pve-upgrade` });
}
}
for (const script of entries.slice(MAX)) notBuilt.push(`\`${script}\`: at most ${MAX} scripts per run, name it with \`/test ${script}\``);
if (include.length === 0) {
await sticky([
MARK, '### Proxmox script test', '',
isIssue || onlyList.length
? `ℹ️ Nothing to build${onlyList.length ? ` on ${onlyList.join(', ')}` : ''}:`
: 'ℹ️ Nothing to build: this test builds `ct/` and `vm/` scripts (and changed `install/` scripts through their `ct/` script).', '',
...(notBuilt.length ? notBuilt.map(n => `- ${n}`) : ['- this PR changes no `ct/`, `vm/` or `install/` script']),
].join('\n'));
core.notice('No ct/ or vm/ script to build');
return;
}
const runUrl = `${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`;
// A check run belongs to a PR's commit; an issue has none.
let checkId = '';
if (!isIssue) {
const { data: check } = await github.rest.checks.create({
// A partial run gets its own name: it must not stand in for the
// full test, e.g. as a required check.
owner, repo, name: `Proxmox script test${onlyList.length ? ` (${onlyList.join(', ')} only)` : ''}`, head_sha: headSha, status: 'in_progress',
details_url: runUrl,
output: { title: `Building ${include.length}`, summary: include.map(i => `- \`${i.script}\` ${[i.platform, i.os, i.mode].filter(Boolean).join(' · ')}`).join('\n') },
});
checkId = String(check.id);
}
core.setOutput('matrix', JSON.stringify(include));
core.setOutput('pr', String(prNumber));
core.setOutput('repo', head.full_name);
core.setOutput('sha', headSha);
core.setOutput('check_id', checkId);
core.setOutput('not_built', JSON.stringify(notBuilt));
core.setOutput('platforms', onlyList.join(' '));
core.setOutput('base_sha', baseSha);
// Who asked: the /test commenter, or whoever dispatched the run.
core.setOutput('actor', context.actor);
await sticky([
MARK, '### Proxmox script test', '',
`⏳ Building ${include.length} at ${headSha.slice(0, 7)}${isIssue ? ' (default branch)' : ''}${onlyList.length ? `, only on ${onlyList.join(', ')}` : ''} — [watch it live](${runUrl}) (open the \`test\` job)`, '',
...include.map(i => `- \`${i.script}\` (${[i.platform, i.os, i.mode && 'upgrade from the base branch'].filter(Boolean).join(' · ')})`),
...(notBuilt.length ? ['', '**Not built:**', ...notBuilt.map(n => `- ${n}`)] : []),
].join('\n'));
test:
needs: gate
if: needs.gate.outputs.matrix != '[]' && needs.gate.outputs.matrix != ''
# Only `proxmox-test` / `incus-test`, never `self-hosted`: the runners are
# registered without default labels so that no other workflow can be
# scheduled onto them. ARM runs on GitHub's own arm64 runner, a fresh VM
# per job (free for public repositories).
runs-on: ${{ matrix.platform == 'arm' && 'ubuntu-24.04-arm' || matrix.platform == 'incus' && 'incus-test' || 'proxmox-test' }}
# ARM is information only: a failed or missing ARM job (no runner, no
# minutes left, ...) never turns the run red. The report leaves it out of
# the result as well.
continue-on-error: ${{ matrix.platform == 'arm' }}
permissions:
contents: read
timeout-minutes: 120 # install (45) + update (30) + second update + checks
strategy:
fail-fast: false
matrix:
include: ${{ fromJSON(needs.gate.outputs.matrix) }}
name: test (${{ matrix.name }})
steps:
# The default branch, never the PR: the harness itself must be trusted.
- uses: actions/checkout@v7
with:
sparse-checkout: .github/workflows/scripts
persist-credentials: false
# The arm64 runner is a plain Ubuntu VM: Incus and the screenshot
# browser are set up on it for this one job.
- name: Set up the ARM runner
if: matrix.platform == 'arm'
timeout-minutes: 20 # about 2 when healthy; a hang must not hold the report
run: sudo bash .github/workflows/scripts/arm-runner-setup.sh
- name: Build and check ${{ matrix.script }}
env:
PR_REPO: ${{ needs.gate.outputs.repo }}
PR_SHA: ${{ needs.gate.outputs.sha }}
SCRIPT: ${{ matrix.script }}
OS: ${{ matrix.os }}
# ARM is the Incus harness on another machine.
PLATFORM: ${{ matrix.platform == 'arm' && 'incus' || matrix.platform }}
VAR_ARM64: ${{ matrix.platform == 'arm' && 'yes' || '' }}
# upgrade: install from the base branch, update with the PR.
TEST_MODE: ${{ matrix.mode || '' }}
BASE_SHA: ${{ needs.gate.outputs.base_sha }}
UPSTREAM: ${{ github.repository }}
CTID_BASE: ${{ github.repository == 'community-scripts/ProxmoxVE' && '9050' || '9000' }}
# Only so release lookups are not rate limited: unauthenticated, all
# test containers share the lab's one public IP and 60 calls an hour.
# This job's token can read public contents and expires with the job.
GH_API_TOKEN: ${{ github.token }}
OUT_DIR: ${{ runner.temp }}/result
# The self-hosted runners run as root; the GitHub one needs sudo (pct
# and incus do).
run: >-
${{ matrix.platform == 'arm' && 'sudo --preserve-env=PR_REPO,PR_SHA,SCRIPT,OS,PLATFORM,VAR_ARM64,TEST_MODE,BASE_SHA,UPSTREAM,CTID_BASE,GH_API_TOKEN,OUT_DIR,GITHUB_RUN_ID,GITHUB_RUN_ATTEMPT' || '' }}
bash .github/workflows/scripts/pve-run-script.sh
- if: always() && matrix.platform == 'arm'
run: sudo chmod -R a+rX "$RUNNER_TEMP/result" 2>/dev/null || true
- if: always()
uses: actions/upload-artifact@v7
with:
name: result-${{ matrix.name }}
path: ${{ runner.temp }}/result
if-no-files-found: ignore
retention-days: 14
report:
needs: [gate, test]
if: always() && needs.gate.result == 'success' && needs.gate.outputs.matrix != '[]' && needs.gate.outputs.matrix != ''
runs-on: ubuntu-latest
permissions:
actions: read
checks: write
contents: write # the ci-screenshots branch only; main is protected
issues: write
pull-requests: write
steps:
- uses: actions/download-artifact@v8
with:
pattern: result-*
path: results
- uses: actions/github-script@v9
env:
MATRIX: ${{ needs.gate.outputs.matrix }}
PR: ${{ needs.gate.outputs.pr }}
SHA: ${{ needs.gate.outputs.sha }}
CHECK_ID: ${{ needs.gate.outputs.check_id }}
TEST_RESULT: ${{ needs.test.result }}
ACTOR: ${{ needs.gate.outputs.actor }}
NOT_BUILT: ${{ needs.gate.outputs.not_built }}
TRUSTED: ${{ needs.gate.outputs.trusted }}
KIND: ${{ needs.gate.outputs.kind }}
PLATFORMS: ${{ needs.gate.outputs.platforms }}
with:
script: |
const fs = require('fs');
const { owner, repo } = context.repo;
const runUrl = `${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`;
const matrix = JSON.parse(process.env.MATRIX);
// download-artifact unpacks a single artifact straight into results/
// and several into results/<name>/, so look for every result.json
// and match it by what it says it is.
const path = require('path');
const found = new Map();
const walk = dir => {
if (!fs.existsSync(dir)) return;
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
const p = path.join(dir, e.name);
if (e.isDirectory()) walk(p);
else if (e.name === 'result.json') {
try {
const r = JSON.parse(fs.readFileSync(p, 'utf8'));
// Written on the host that ran the PR's code: anything but a
// known status is an error, never something new.
if (!['pass', 'fail', 'unsupported', 'error'].includes(r.status)) r.status = 'error';
const log = path.join(dir, 'install.log');
r.install_log = fs.existsSync(log) ? fs.readFileSync(log, 'utf8') : '';
const shot = path.join(dir, 'screenshot.png');
r.shot_file = fs.existsSync(shot) ? shot : '';
if (!['updated', 'no-update', 'fail', 'error', ''].includes(r.update_status || '')) r.update_status = '';
// Installed fine, but the update could not be tested (e.g. a
// GitHub rate limit): not a pass, and not the PR's fault.
if (r.status === 'pass' && r.update_status === 'error') { r.status = 'error'; r.step = `update ${r.update_step}`; }
const ulog = path.join(dir, 'update.log');
r.update_log = fs.existsSync(ulog) ? fs.readFileSync(ulog, 'utf8') : '';
const u2log = path.join(dir, 'update2.log');
r.update2_log = fs.existsSync(u2log) ? fs.readFileSync(u2log, 'utf8') : '';
if (!['upgrade', ''].includes(r.mode || '')) r.mode = '';
r.mode = r.mode || '';
r.notes = Array.isArray(r.notes) ? r.notes.filter(n => typeof n === 'string').slice(0, 8) : [];
r.resources = typeof r.resources === 'string' ? r.resources : '';
const ushot = path.join(dir, 'screenshot-update.png');
r.shot_update_file = fs.existsSync(ushot) ? ushot : '';
// ARM runs the Incus harness on an arm64 machine; result.json
// says so in arch. Only the platforms this workflow knows: an
// unknown one would never match a matrix row anyway.
if (r.arch === 'aarch64') r.platform = 'arm';
else if (!['pve', 'incus'].includes(r.platform)) r.platform = 'pve';
found.set(`${r.script}|${r.platform}|${r.os}|${r.mode}`, r);
} catch (err) { core.warning(`unreadable ${p}: ${err.message}`); }
}
}
};
walk('results');
// A job that died before writing result.json still gets a row.
const rows = matrix.map(m => ({ ...(found.get(`${m.script}|${m.platform || 'pve'}|${m.os || 'default'}|${m.mode || ''}`) || {
script: m.script, os: m.os || 'default', platform: m.platform || 'pve', mode: m.mode || '', notes: [], resources: '', update2_log: '',
status: process.env.TEST_RESULT === 'cancelled' ? 'cancelled' : 'error',
step: 'no result (runner or job failure)', duration: 0, log_tail: '', install_log: '', shot_file: '',
update_status: '', update_log: '', shot_update_file: '',
}), name: m.name }));
// Where a row ran: "pve", "incus · alpine", "arm64", "pve · upgrade", or "VM" (Proxmox only).
const isVm = r => r.script.startsWith('vm/');
const isArm = r => r.platform === 'arm';
const where = r => {
if (isVm(r)) return 'VM';
const p = isArm(r) ? 'arm64' : r.platform;
const w = r.os && r.os !== 'default' ? `${p} · ${r.os}` : p;
return r.mode === 'upgrade' ? `${w} · upgrade` : w;
};
// ARM is information only: shown, never counted. The result, the
// check, the review and the approval come from Proxmox and Incus.
const coreRows = rows.filter(r => !isArm(r));
const armRows = rows.filter(isArm);
// Screenshots go to an orphan branch, because a comment can only show
// an image that has a URL. One file per PR and script, overwritten on
// every run; the link names the commit, so no cache shows an old one.
// The PNG comes from the host that ran the PR's code: check it again.
const BRANCH = 'ci-screenshots';
const prNum = Number(process.env.PR);
const isPng = buf => buf.length > 8 && buf.length < 3000000 &&
buf.subarray(0, 8).equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]));
const ensureBranch = async () => {
try { await github.rest.git.getRef({ owner, repo, ref: `heads/${BRANCH}` }); return; }
catch (e) { if (e.status !== 404) throw e; }
const { data: blob } = await github.rest.git.createBlob({ owner, repo, encoding: 'utf-8',
content: 'Screenshots taken by pr-script-test.yml for the PR comments. Not code; never merge this branch.\n' });
const { data: tree } = await github.rest.git.createTree({ owner, repo, tree: [{ path: 'README.md', mode: '100644', type: 'blob', sha: blob.sha }] });
const { data: commit } = await github.rest.git.createCommit({ owner, repo, message: 'Start ci-screenshots', tree: tree.sha, parents: [] });
await github.rest.git.createRef({ owner, repo, ref: `refs/heads/${BRANCH}`, sha: commit.sha });
};
const uploadShot = async (r, local, suffix) => {
const buf = fs.readFileSync(local);
if (!isPng(buf)) { core.warning(`${r.script}: screenshot is not a plausible PNG, skipped`); return ''; }
const file = `${process.env.KIND === 'issue' ? 'issue' : 'pr'}-${prNum}/${r.name}${suffix}.png`;
for (let attempt = 1; attempt <= 3; attempt++) {
try {
await ensureBranch();
let sha;
try { sha = (await github.rest.repos.getContent({ owner, repo, path: file, ref: BRANCH })).data.sha; }
catch (e) { if (e.status !== 404) throw e; }
const { data } = await github.rest.repos.createOrUpdateFileContents({
owner, repo, branch: BRANCH, path: file, sha, content: buf.toString('base64'),
message: `Screenshot: ${r.script} (${where(r)})${suffix} for #${prNum} at ${process.env.SHA.slice(0, 7)}`,
});
return `https://raw.githubusercontent.com/${owner}/${repo}/${data.commit.sha}/${file}`;
} catch (e) {
// Two reports at once race on the branch head; try again.
core.warning(`${r.script}: screenshot upload attempt ${attempt} failed: ${e.message}`);
await new Promise(res => setTimeout(res, 2000 * attempt));
}
}
return '';
};
for (const r of rows) {
if (r.shot_file) r.shot_url = await uploadShot(r, r.shot_file, '');
if (r.shot_update_file) r.shot_update_url = await uploadShot(r, r.shot_update_file, '-after-update');
}
// unsupported: the script needs a CPU feature the test host lacks
// (AVX2 and the like). Not the PR's fault, so not a failure.
const icon = { pass: '✅', fail: '❌', error: '⚠️', cancelled: '⏹️', unsupported: '🖥️' };
const dur = s => `${Math.floor(s / 60)}m${String(s % 60).padStart(2, '0')}s`;
// Text from the build lands outside code blocks here: no table breaks,
// no code spans, and no @mention of a stranger (zero-width space).
const clean = s => String(s || '').replace(/[|\r\n]/g, ' ').replace(/`/g, "'").replace(/@/g, '@​').slice(0, 200);
const fence = s => String(s || '').replace(/~~~~/g, '~~~');
// fail: the app broke, the PR's business. error/cancelled: our test
// setup did, not the PR's. unsupported: the host's CPU is too old.
const count = s => coreRows.filter(r => r.status === s).length;
const passed = count('pass');
const untestable = count('unsupported');
const broken = count('fail');
const setupTrouble = count('error') + count('cancelled');
const allPass = passed === coreRows.length;
// `/test arm`: nothing that counts was built, so the result is
// information only as a whole.
const armOnly = coreRows.length === 0;
// Built on only some platforms (`/test pve`): a pass there says
// nothing about the others, so it neither approves nor lifts an
// earlier "changes requested". A failure is still a failure.
const partial = !!process.env.PLATFORMS;
const cancelled = (armOnly ? rows : coreRows).every(r => r.status === 'cancelled');
const of = `**${passed}/${coreRows.length} passed**`;
const armPassed = armRows.filter(r => r.status === 'pass').length;
const armLine = armRows.length && !armOnly
? `ARM64 (information only, not counted): ${armPassed}/${armRows.length} passed${armPassed < armRows.length ? `; failures there do not block this PR` : ''}.`
: '';
const headline = armOnly
? `ℹ️ ARM64 only, information only: **${armPassed}/${armRows.length} passed**`
: broken
? `❌ ${of}`
: setupTrouble
? `⚠️ ${of}, ${setupTrouble} could not be tested because of a problem with the test setup (not the PR)`
: untestable
? `🖥️ ${of}, ${untestable} need${untestable === 1 ? 's' : ''} a newer CPU than the test host has (nothing failed)`
: `✅ ${of}`;
// Update: the installed app is set back a version and updated with
// the PR's update_script. ➖ = it ran but had nothing to update.
const updateIcon = { updated: '✅', 'no-update': '➖', fail: '❌', error: '⚠️' };
const detail = r => {
if (r.status !== 'pass') return clean(r.step);
const ip = /^[0-9.]{7,15}$/.test(r.ip || '') ? r.ip : '';
if (isVm(r) && !r.url) return ip ? `booted, got ${ip}` : 'booted, no DHCP lease (an installer ISO, or no network)';
const reach = r.url ? `${clean(r.url)} → ${r.http_code}` : 'no URL to check';
return r.update_status === 'no-update' ? `${reach}; update: nothing to update${r.update_step ? ` (${clean(r.update_step)})` : ''}` : reach;
};
const table = [
'| Script | Platform | Install | Update | Duration | Detail |',
'|---|---|---|---|---|---|',
...rows.map(r => `| \`${r.script}\` | ${where(r)} | ${r.update_status ? '✅' : icon[r.status] || '❔'} | ${updateIcon[r.update_status] || '–'} | ${dur(r.duration || 0)} | ${detail(r)} |`),
].join('\n');
// A comment holds 65536 characters. Each script gets an equal share
// and a log that does not fit loses its beginning: the end is where
// a failure shows.
const budget = Math.floor(52000 / rows.length);
// A check run summary holds 65535 BYTES, not characters, and log
// emoji take 3-4 bytes each. Cut at a character boundary and close
// a ~~~~ block left open.
const fitBytes = (text, max) => {
if (Buffer.byteLength(text) <= max) return text;
let cut = Buffer.from(text).subarray(0, max - 300).toString('utf8').replace(/�+$/, '');
if ((cut.match(/^~~~~/gm) || []).length % 2) cut += '\n~~~~';
return `${cut}\n\n… cut to fit GitHub's size limit; the full report is in the PR comment and the logs in the run artifacts.`;
};
const tail = (text, max) => {
text = fence(text).trimEnd();
if (text.length <= max) return text;
const kept = text.slice(text.length - max);
const cut = text.slice(0, text.length - max).split('\n').length;
return `… ${cut} earlier lines cut, the full log is in the run artifacts\n${kept.slice(kept.indexOf('\n') + 1)}`;
};
const details = (summaryLine, text, max) => [
`<details><summary>${summaryLine}</summary>`, '',
'~~~~text', tail(text, max), '~~~~', '</details>',
].join('\n');
const logs = rows.map(r => {
const parts = [];
const failed = r.status !== 'pass';
// A VM script's own output is all there is: no install log inside.
const buildShare = (failed || isVm(r)) && r.log_tail ? Math.min(r.log_tail.length, Math.floor(budget / 3)) : 0;
if (buildShare) parts.push(details(`${icon[r.status] || '❔'} ${r.script} (${where(r)}) — build log`, r.log_tail, buildShare));
if (isVm(r)) return parts.join('\n\n');
// The update log gets up to a third of what is left, the install log the rest.
const updateShare = r.update_log ? Math.min(r.update_log.length, Math.floor((budget - buildShare) / 3)) : 0;
if (r.install_log) {
const lines = r.install_log.trimEnd().split('\n').length;
parts.push(details(`📄 ${r.script} (${where(r)}) — install log (${lines} lines)`, r.install_log, budget - buildShare - updateShare - 400));
} else if (!buildShare) {
parts.push(`<sub>${r.script} (${where(r)}): no install log was captured</sub>`);
}
if (updateShare) {
const lines = r.update_log.trimEnd().split('\n').length;
parts.push(details(`🔄 ${r.script} (${where(r)}) — update log (${lines} lines)`, r.update_log, updateShare));
}
if (r.update2_log) {
parts.push(details(`🔁 ${r.script} (${where(r)}) — second update log`, r.update2_log, Math.min(r.update2_log.length, Math.floor(budget / 8))));
}
return parts.join('\n\n');
});
const shots = rows.filter(r => r.shot_url || r.shot_update_url).map(r => [
`<details open><summary>📸 ${r.script} (${where(r)}) — ${r.url ? clean(r.url) : 'console'}${r.shot_url && r.shot_update_url ? ': after install / after update' : r.shot_update_url ? ': after update' : ''}</summary>`, '',
[r.shot_url, r.shot_update_url].filter(Boolean)
.map((u, i, all) => `<img src="${u}" width="${all.length > 1 ? 420 : 640}" alt="Screenshot of ${r.script}">`).join(' '),
'', '</details>',
].join('\n'));
const summary = [
`${headline} at \`${process.env.SHA.slice(0, 7)}\``, '',
table, '',
...(armLine ? [`<sub>${armLine}</sub>`, ''] : []),
...(rows.some(r => r.mode === 'upgrade')
? ['<sub>`upgrade`: installed from the base branch as it is now, then updated with this PR, as a user who installed before the merge would.</sub>', '']
: []),
...(() => {
const noted = rows.filter(r => r.notes.length);
return noted.length
? ['**Notes** (information only, not counted):', ...noted.flatMap(r => r.notes.map(n => `- \`${r.script}\` (${where(r)}): ${clean(n)}`)), '']
: [];
})(),
...(() => {
const measured = rows.filter(r => r.resources);
return measured.length
? ['<details><summary>📊 Resources</summary>', '', '| Script | Platform | Usage |', '|---|---|---|',
...measured.map(r => `| \`${r.script}\` | ${where(r)} | ${clean(r.resources)} |`), '', '</details>', '']
: [];
})(),
...(partial ? [`<sub>Built only on ${process.env.PLATFORMS.split(' ').join(', ')}; comment \`/test\` for every platform.</sub>`, ''] : []),
...shots, '',
...(() => { try { const n = JSON.parse(process.env.NOT_BUILT || '[]'); return n.length ? ['**Not built:**', ...n.map(x => `- ${x}`), ''] : []; } catch (e) { return []; } })(),
...logs, '',
`Full logs: [workflow run](${runUrl}) (artifacts \`result-*\`). Comment \`/test${partial ? ` ${process.env.PLATFORMS}` : ''}${process.env.KIND === 'issue' ? ' ' + [...new Set(rows.map(r => r.script.slice(3, -3)))].join(' ') : ''}\` to build again.`,
].join('\n');
// Issues have no check run (there is no PR commit to attach it to).
if (process.env.CHECK_ID) {
await github.rest.checks.update({
owner, repo, check_run_id: Number(process.env.CHECK_ID), status: 'completed',
// Untestable here is "test it by hand", not a pass: action_required,
// because a required check would count neutral as passed.
conclusion: cancelled ? 'cancelled' : armOnly ? 'neutral' : allPass ? 'success' : broken ? 'failure' : 'action_required',
output: {
title: armOnly
? `ARM64 ${armPassed}/${armRows.length} passed (information only)`
: `${passed}/${coreRows.length} passed${armRows.length ? ` (+ ARM ${armPassed}/${armRows.length}, not counted)` : ''}`,
summary: fitBytes(summary, 65000),
},
});
}
// The result pings whoever started the test. A mention added by
// editing a comment does not reliably notify, so the old comment
// goes and the result is posted as a new one, at the end of the
// thread where it is looked for.
// Whoever started the test and the PR's author, once each; no bots.
const pr = Number(process.env.PR);
const isIssue = process.env.KIND === 'issue';
const { data: prInfo } = isIssue
? await github.rest.issues.get({ owner, repo, issue_number: pr })
: await github.rest.pulls.get({ owner, repo, pull_number: pr });
const author = prInfo.user && prInfo.user.type === 'User' ? prInfo.user.login : '';
const login = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})$/;
const ping = [...new Set([process.env.ACTOR || '', author])].filter(l => login.test(l)).map(l => `@${l} `).join('');
const MARK = '<!-- pr-script-test -->';
const body = `${MARK}\n### Proxmox script test\n\n${ping}${summary}`.slice(0, 65000);
const comments = await github.paginate(github.rest.issues.listComments, { owner, repo, issue_number: pr });
for (const c of comments.filter(c => c.body.includes(MARK) && c.user.type === 'Bot')) {
await github.rest.issues.deleteComment({ owner, repo, comment_id: c.id });
}
const { data: posted } = await github.rest.issues.createComment({ owner, repo, issue_number: pr, body });
// An issue has nothing to review or approve: the comment is all.
if (isIssue) return;
// A real failure requests changes on the tested commit. Untestable
// (CPU), runner errors and cancelled runs are not the PR's fault and
// leave the review state alone. Our previous "changes requested" is
// always dismissed first: a pass lifts it, a new failure replaces it.
const REVIEW_MARK = '<!-- pr-script-test-review -->';
const failedRows = cancelled ? [] : coreRows.filter(r => r.status === 'fail');
const sha7 = process.env.SHA.slice(0, 7);
// Trusted PRs (scripts moving over from DevScripts) are approved when
// every script passes -- not on untestable or setup trouble, which
// means "not tested", not "good". That approval counts toward the
// ruleset's required review; a push dismisses it as stale, and a
// later run that does not fully pass withdraws it.
// Anyone with write access can push to a migration branch, and the
// approval counts toward the required review. So it only covers
// what a migration contains: ct/ and install/ scripts. A PR that
// also changes anything else (workflows, misc/, ...) is tested but
// never approved.
let approve = process.env.TRUSTED === 'true' && allPass && !partial;
// A partial run that passed leaves every review as it is.
const leaveReviews = partial && !failedRows.length;
let noApproveWhy = '';
if (approve) {
const changed = await github.paginate(github.rest.pulls.listFiles, { owner, repo, pull_number: pr });
const other = changed.map(f => f.filename).filter(n => !/^(ct|install)\/[A-Za-z0-9._-]+\.sh$/.test(n));
if (other.length) {
approve = false;
noApproveWhy = `Not approved automatically: besides \`ct/\` and \`install/\` scripts this PR changes ${other.slice(0, 5).map(n => '`' + n.replace(/[`\r\n]/g, '') + '`').join(', ')}${other.length > 5 ? ', …' : ''}. That needs a human review.`;
}
}
if (noApproveWhy) {
await github.rest.issues.updateComment({ owner, repo, comment_id: posted.id, body: `${body}\n\n> ${noApproveWhy}`.slice(0, 65000) });
}
try {
const reviews = await github.paginate(github.rest.pulls.listReviews, { owner, repo, pull_number: pr });
const ours = state => reviews.filter(v => v.user && v.user.type === 'Bot' && v.state === state && (v.body || '').includes(REVIEW_MARK));
for (const v of leaveReviews ? [] : ours('CHANGES_REQUESTED')) {
await github.rest.pulls.dismissReview({
owner, repo, pull_number: pr, review_id: v.id,
message: failedRows.length ? 'Replaced by a newer Proxmox script test.' : `Proxmox script test passed at ${sha7}.`,
});
}
if (!approve && !cancelled && !leaveReviews) {
for (const v of ours('APPROVED')) {
await github.rest.pulls.dismissReview({
owner, repo, pull_number: pr, review_id: v.id,
message: `Withdrawn: the Proxmox script test at ${sha7} did not fully pass.`,
});
}
}
if (approve && !ours('APPROVED').some(v => v.commit_id === process.env.SHA)) {
await github.rest.pulls.createReview({
owner, repo, pull_number: pr, commit_id: process.env.SHA, event: 'APPROVE',
body: [
REVIEW_MARK, `**Proxmox script test passed** at \`${sha7}\`: ${rows.map(r => `\`${r.script}\``).join(', ')}.`, '',
`Script moved over from DevScripts, built and checked on a real Proxmox host. [Test result](${posted.html_url}).`,
].join('\n'),
});
}
if (failedRows.length) {
await github.rest.pulls.createReview({
owner, repo, pull_number: pr, commit_id: process.env.SHA, event: 'REQUEST_CHANGES',
body: [
REVIEW_MARK, `**Proxmox script test failed** at \`${sha7}\`:`, '',
...failedRows.map(r => `- \`${r.script}\` (${where(r)}): ${clean(r.step)}`), '',
`Logs and details: [test result](${posted.html_url}). Push a fix, then ask for \`/test\` again.`,
].join('\n'),
});
}
} catch (e) {
core.warning(`Could not update the review state: ${e.message}`);
}
+209 -7
View File
@@ -1,8 +1,12 @@
name: PR test command
# Posts a ready-to-run test command for the scripts a PR touches. The workflow
# lives in community-scripts/ci (shared with DevScripts and ProxmoxVE); this
# file only holds the trigger. Nothing from the PR is checked out or run.
# Posts a ready-to-run test command for reviewers.
# It uses this PR’s script branch with the production engine, since both resolve
# independently. ct/ scripts need community-scripts/core; vm/, tools/ and
# turnkey/ scripts run straight from the branch.
#
# pull_request_target allows comments on fork PRs. No PR code is checked out or
# executed; API inputs are validated and the comment is assembled in JavaScript.
on:
pull_request_target:
@@ -15,11 +19,209 @@ on:
- "tools/**"
- "turnkey/**"
permissions: {}
jobs:
pr-test-command:
comment:
if: github.repository == 'community-scripts/ProxmoxVE'
runs-on: self-hosted
permissions:
pull-requests: write
contents: read
uses: community-scripts/ci/.github/workflows/pr-test-command.yml@main
steps:
- uses: actions/github-script@v9
with:
script: |
const MARKER = '<!-- pr-test-command -->';
const MAX_APPS = 10;
const pr = context.payload.pull_request;
const head = pr.head.repo; // null when the fork is gone
if (!head) return;
const owner = context.repo.owner;
const repo = context.repo.repo;
// Git allows backticks in a ref name, and this one ends up inside a
// fenced block. Anything outside the ordinary set is not worth
// rendering, so bail rather than escape.
const ref = pr.head.ref;
if (!/^[A-Za-z0-9._\/-]+$/.test(ref)) return;
const base = `https://raw.githubusercontent.com/${head.full_name}/${ref}`;
const files = await github.paginate(github.rest.pulls.listFiles, {
owner, repo, pull_number: pr.number, per_page: 100,
});
// ct/foo.sh and install/foo-install.sh are the same app. A removed
// file has nothing left to run.
const apps = new Map(); // slug -> {ct, install}
const others = [];
for (const f of files) {
if (f.status === 'removed') continue;
let m = f.filename.match(/^ct\/([a-z0-9][a-z0-9._-]*)\.sh$/);
if (m) { apps.set(m[1], { ...apps.get(m[1]), ct: true }); continue; }
m = f.filename.match(/^install\/([a-z0-9][a-z0-9._-]*)-install\.sh$/);
if (m) { apps.set(m[1], { ...apps.get(m[1]), install: true }); continue; }
if (/^(vm|tools|turnkey)\/[A-Za-z0-9._\/-]+\.sh$/.test(f.filename)) others.push(f.filename);
}
if (apps.size === 0 && others.length === 0) return;
others.sort();
// Where a tool runs is read from the script itself; it is never run.
const HOST = /(^|[\s;&|(`$])(pct|qm|pveam|pvesm|pvesh|pveversion)\s/m;
async function runsOn(path) {
if (path.startsWith('vm/')) return 'pve';
const appliance = (path.split('/').pop().match(/(?:^|-)(pbs|pmg|pdm)(?=[0-9-]|\.sh$)/) || [])[1];
if (path.startsWith('tools/pve/')) return appliance || 'pve';
let text = '';
try {
const res = await github.rest.repos.getContent({
owner: head.owner.login, repo: head.name, path, ref: pr.head.sha,
});
text = Buffer.from(res.data.content || '', 'base64').toString('utf8');
} catch (e) {}
if (HOST.test(text.replace(/^\s*#.*$/gm, '').replace(/command -v \S+/g, ''))) return 'pve';
if (appliance) return appliance;
return path.startsWith('tools/addon/') ? 'guest' : 'unknown';
}
const WHERE = {
pve: 'in the Proxmox VE shell',
guest: 'inside the LXC or VM that should get it',
pbs: 'in the Proxmox Backup Server shell',
pmg: 'in the Proxmox Mail Gateway shell',
pdm: 'in the Proxmox Datacenter Manager shell',
unknown: 'wherever the script is meant to run',
};
// Read the ct script at the PR head to see which engine it loads.
// Read only -- it is never sourced or run.
async function bootstrapOf(slug) {
try {
const res = await github.rest.repos.getContent({
owner: head.owner.login, repo: head.name,
path: `ct/${slug}.sh`, ref: pr.head.sha,
});
if (!res.data.content) return 'unknown';
const text = Buffer.from(res.data.content, 'base64').toString('utf8');
const firstLines = text.split('\n').slice(0, 12).join('\n');
return /_cs_boot=/.test(firstLines) ? 'core' : 'legacy';
} catch (e) {
return e.status === 404 ? 'missing' : 'unknown';
}
}
const ready = [], legacy = [], missing = [];
for (const slug of [...apps.keys()].sort()) {
const kind = await bootstrapOf(slug);
if (kind === 'core') ready.push(slug);
else if (kind === 'legacy') legacy.push(slug);
else if (kind === 'missing') missing.push(slug);
}
const lines = [MARKER];
if (ready.length > 0) {
const shown = ready.slice(0, MAX_APPS);
lines.push(
'### Try this branch',
'',
'The engine and the scripts resolve independently, so this runs the changed',
'`ct/` and `install/` scripts against the **production** engine:',
'',
);
for (const slug of shown) {
lines.push(
'```bash',
`export COMMUNITY_SCRIPTS_URL=${base}`,
`bash -c "$(curl -fsSL "$COMMUNITY_SCRIPTS_URL/ct/${slug}.sh")"`,
'```',
'',
);
}
if (ready.length > shown.length) {
lines.push(
`${ready.length - shown.length} more script(s) changed; same command, different slug.`,
'',
);
}
lines.push(
'Both lines are needed. Without `COMMUNITY_SCRIPTS_URL` the engine falls back to',
'`main`, so curling the branch URL on its own gives you the `ct/` script from',
'this PR and the `install/` script from `main`. Frequently the one you meant to test.',
'',
'The same command works on an Incus host: the engine detects the platform and',
'loads the matching backend, while the scripts still come from this branch.',
'',
'<details><summary>Useful while testing</summary>',
'',
'`dev_mode=net` logs every fetch with status and URL, which is the quickest way',
'to confirm the branch is really being used. `dev_mode=keep` stops a failed',
'build from deleting the container along with the evidence.',
'',
'```bash',
`export COMMUNITY_SCRIPTS_URL=${base}`,
`dev_mode=net,keep bash -c "$(curl -fsSL "$COMMUNITY_SCRIPTS_URL/ct/${shown[0]}.sh")"`,
'```',
'</details>',
);
}
if (others.length > 0) {
const shown = others.slice(0, MAX_APPS);
lines.push(...(ready.length > 0 ? ['', '---', ''] : ['### Try this branch', '']));
for (const path of shown) {
lines.push(
`\`${path}\`, run ${WHERE[await runsOn(path)]}:`,
'```bash',
`bash -c "$(curl -fsSL "${base}/${path}")"`,
'```',
'',
);
}
if (others.length > shown.length) {
lines.push(
`${others.length - shown.length} more script(s) changed; same command, different path.`,
'',
);
}
lines.push(
'Only the script itself comes from this branch. The engine it loads still',
'comes from community-scripts/core on `main`.',
);
}
if (legacy.length > 0) {
lines.push(
'',
ready.length > 0 ? '---' : '### Not testable this way yet',
'',
`\`${legacy.join('`, `')}\` still uses the older one-liner bootstrap, which`,
'resolves everything from `ProxmoxVE/main` and ignores `COMMUNITY_SCRIPTS_URL`.',
'There is no way to point it at this branch — test it from a checkout on the',
'host instead, or migrate the script to the `_cs_boot` bootstrap first.',
);
}
if (missing.length > 0) {
lines.push(
'',
`No \`ct/\` script found for \`${missing.join('`, `')}\`, so there is nothing to`,
'run. If the install script was renamed, its `ct/` counterpart needs the same',
'name.',
);
}
if (lines.length === 1) return; // marker only, nothing worth saying
const body = lines.join('\n');
// Update in place rather than posting again on every push.
const comments = await github.paginate(github.rest.issues.listComments, {
owner, repo, issue_number: pr.number, per_page: 100,
});
const mine = comments.find(c => c.body.includes(MARKER));
if (mine) {
if (mine.body !== body) {
await github.rest.issues.updateComment({ owner, repo, comment_id: mine.id, body });
}
} else {
await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body });
}
+192
View File
@@ -0,0 +1,192 @@
#!/usr/bin/env bash
# Turns a GitHub-hosted arm64 runner (ubuntu-24.04-arm) into a throwaway Incus
# test host for pve-run-script.sh (PLATFORM=incus): the same layout as
# incus-ci-host-setup.sh, minus what a VM that lives for one job does not need
# (services, sweeper, runner registration). Run with sudo.
#
# The runner is a fresh VM per job and is discarded afterwards; nothing here
# has to be undone.
set -euo pipefail
NET_ADDR=10.98.0.1
NET_CIDR=10.98.0.0/24
SERVE_ROOT=/srv/ci
SERVE_PORT=18080
start=$(date +%s)
# --- Incus ----------------------------------------------------------------------
# Ubuntu 24.04's own incus is 6.0.0, which lacks `-f csv,noheader`; the engine
# then finds no storage pool. Zabbly (the Incus maintainers' packages) has the
# current 6.0 LTS, as Debian 13 does.
mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc
# The key Zabbly publishes in its install instructions, not whatever the
# server returns today.
gpg --show-keys --with-colons /etc/apt/keyrings/zabbly.asc 2>/dev/null |
awk -F: '/^fpr/ {print $10; exit}' | grep -qx 4EFC590696CB15B87C73A3AD82CC8797C838DCFD ||
{ echo "::error::unexpected Zabbly signing key"; exit 1; }
# Only the Incus packages may come from there; everything else stays Ubuntu's.
printf '%s\n' 'Package: *' 'Pin: origin pkgs.zabbly.com' 'Pin-Priority: 1' '' \
'Package: incus* liblxc* lxcfs* qemu* edk2*' 'Pin: origin pkgs.zabbly.com' 'Pin-Priority: 600' \
>/etc/apt/preferences.d/zabbly
printf '%s\n' 'Enabled: yes' 'Types: deb' 'URIs: https://pkgs.zabbly.com/incus/lts-6.0' \
"Suites: $(. /etc/os-release && echo "$VERSION_CODENAME")" 'Components: main' \
"Architectures: $(dpkg --print-architecture)" 'Signed-By: /etc/apt/keyrings/zabbly.asc' \
>/etc/apt/sources.list.d/zabbly-incus-lts-6.0.sources
# Each slow step has its own limit, so a hang names itself in the log instead
# of holding the PR report until the job times out.
step() { local t=$1; shift; timeout "$t" "$@" || { echo "::error::'$*' failed or hung (limit ${t})"; return 1; }; }
# What the runner's network looks like when apt can't get through.
netdiag() {
echo "::group::Network on this runner"
resolvectl status 2>&1 | grep -E 'DNS Servers|Current DNS|resolv.conf mode' || cat /etc/resolv.conf
local h
for h in ports.ubuntu.com pkgs.zabbly.com github.com; do
echo "$h -> $(timeout 10 getent ahostsv4 "$h" | awk '{print $1; exit}')"
timeout 15 curl -s -o /dev/null -w " http %{http_code} in %{time_total}s\n" "http://$h/" || echo " no answer"
done
echo "::endgroup::"
}
# The runner's resolver (Azure's 168.63.129.16) stopped resolving the arm64
# mirror on 2026-11-11 while every other name worked. Then the name is
# looked up over DNS-over-HTTPS and pinned in /etc/hosts for this job.
for h in ports.ubuntu.com; do
timeout 10 getent ahostsv4 "$h" >/dev/null && continue
ip=$(curl -fsS --max-time 10 -H 'accept: application/dns-json' "https://1.1.1.1/dns-query?name=$h&type=A" |
jq -r '[.Answer[]? | select(.type == 1) | .data][0] // empty')
if [[ "$ip" =~ ^[0-9]+(\.[0-9]+){3}$ ]]; then
echo "$ip $h" >>/etc/hosts
echo "::warning::the runner's DNS can't resolve $h; using $ip from 1.1.1.1"
fi
done
step 5m apt-get -o Acquire::Retries=3 -o Acquire::http::Timeout=30 update -qq || { netdiag; exit 1; }
DEBIAN_FRONTEND=noninteractive step 10m apt-get install -y -qq incus jq >/dev/null
# Docker on the runner sets the FORWARD policy to DROP, which cuts the Incus
# containers off from the internet.
iptables -P FORWARD ACCEPT
step 2m incus admin init --preseed <<EOF
networks:
- name: incusbr0
type: bridge
config:
ipv4.address: ${NET_ADDR}/24
ipv4.nat: "true"
ipv6.address: none
storage_pools:
- name: default
driver: dir
profiles:
- name: default
devices:
root:
path: /
pool: default
type: disk
eth0:
name: eth0
network: incusbr0
type: nic
EOF
# The PR tree server the harness expects (a service on the lab hosts).
mkdir -p "$SERVE_ROOT"
nohup python3 -m http.server "$SERVE_PORT" --bind "$NET_ADDR" --directory "$SERVE_ROOT" >/tmp/ci-serve.log 2>&1 &
echo "Incus $(incus --version) ready after $(($(date +%s) - start))s"
# --- Screenshots ----------------------------------------------------------------
# Optional: without them the build is still tested. The harness takes one only
# when `chromium` exists, the user ci-shot exists and the CI_SHOT chain keeps
# that user off private addresses (here: the cloud metadata service at
# 169.254.169.254 and the runner itself).
shots() {
useradd --system --create-home --shell /usr/sbin/nologin ci-shot
# Ubuntu's apt chromium is a snap wrapper; Playwright ships a plain arm64
# build. Started directly with --screenshot, that build crashes on this
# runner (sandbox or not), so `chromium` here is a stand-in that takes the
# harness's arguments and drives the browser through Playwright instead.
mkdir -p /opt/pw
(cd /opt/pw && step 5m npm install --silent --no-audit --no-fund playwright-core@1.49.1 >/dev/null) || return 1
PLAYWRIGHT_BROWSERS_PATH=/opt/pw-browsers DEBIAN_FRONTEND=noninteractive \
step 10m /opt/pw/node_modules/.bin/playwright-core install --with-deps chromium >/dev/null || return 1
chmod -R a+rX /opt/pw /opt/pw-browsers
cat >/opt/pw/shot.js <<'EOF'
// node shot.js <url> <out.png> <width> <height>
const { chromium } = require('/opt/pw/node_modules/playwright-core');
(async () => {
const [url, out, w, h] = process.argv.slice(2);
const browser = await chromium.launch({ chromiumSandbox: true });
try {
const page = await browser.newPage({ viewport: { width: Number(w), height: Number(h) }, ignoreHTTPSErrors: true });
await page.goto(url, { waitUntil: 'load', timeout: 30000 }).catch(() => {});
await page.waitForTimeout(5000);
await page.screenshot({ path: out });
} finally {
await browser.close();
}
})().catch(e => { console.error(e.message); process.exit(1); });
EOF
cat >/usr/local/bin/chromium <<'EOF'
#!/usr/bin/env bash
# Stand-in for `chromium --headless ... --screenshot=<file> --window-size=W,H <url>`
# on the ARM runner (see arm-runner-setup.sh). Other flags are ignored.
url="" out="" w=1280 h=800
for a in "$@"; do
case "$a" in
--version) echo "Chromium (Playwright $(node -p "require('/opt/pw/node_modules/playwright-core/package.json').version"))"; exit 0 ;;
--screenshot=*) out="${a#*=}" ;;
--window-size=*) s="${a#*=}"; w="${s%,*}"; h="${s#*,}" ;;
-*) ;;
*) url="$a" ;;
esac
done
[[ -n "$url" && -n "$out" ]] || { echo "usage: chromium --screenshot=<file> <url>" >&2; exit 2; }
PLAYWRIGHT_BROWSERS_PATH=/opt/pw-browsers exec node /opt/pw/shot.js "$url" "$out" "$w" "$h"
EOF
chmod 755 /usr/local/bin/chromium
# Ubuntu 24.04 forbids unprivileged user namespaces through AppArmor, and
# Chromium's sandbox needs them. The sandbox stays on; the restriction goes,
# on a VM that exists for this one job.
sysctl -qw kernel.apparmor_restrict_unprivileged_userns=0 2>/dev/null || true
local uid fw net ns
uid=$(id -u ci-shot)
for fw in iptables ip6tables; do
$fw -w -N CI_SHOT
$fw -w -I OUTPUT -m owner --uid-owner "$uid" -j CI_SHOT
done
iptables -w -A CI_SHOT -d "$NET_ADDR" -j REJECT
iptables -w -A CI_SHOT -d "$NET_CIDR" -j ACCEPT
for ns in 127.0.0.53 $(awk '/^nameserver/ && $2 !~ /:/ {print $2}' /run/systemd/resolve/resolv.conf 2>/dev/null); do
iptables -w -A CI_SHOT -d "$ns" -p udp --dport 53 -j ACCEPT
iptables -w -A CI_SHOT -d "$ns" -p tcp --dport 53 -j ACCEPT
done
for net in 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16 172.16.0.0/12 192.168.0.0/16 224.0.0.0/3; do
iptables -w -A CI_SHOT -d "$net" -j REJECT
done
for net in ::1/128 fc00::/7 fe80::/10 ff00::/8; do
ip6tables -w -A CI_SHOT -d "$net" -j REJECT
done
# The same call the harness makes, on a page that needs no network.
local dir
dir=$(mktemp -d)
chown ci-shot "$dir"
# HOME: runuser keeps root's, which ci-shot cannot write (the harness does the same).
runuser -u ci-shot -- env HOME="$dir" timeout 60 chromium --headless=new --disable-gpu --hide-scrollbars \
--no-first-run --window-size=320,200 --user-data-dir="$dir/profile" \
--screenshot="$dir/shot.png" 'data:text/html,<h1>ci</h1>' &>"$dir/chromium.log" || true
if [[ ! -s "$dir/shot.png" ]]; then
echo "Chromium could not take a test screenshot:"
head -n 15 "$dir/chromium.log"
echo "userns: $(sysctl -n kernel.apparmor_restrict_unprivileged_userns 2>&1)"
rm -rf -- "$dir"
return 1
fi
rm -rf -- "$dir"
echo "$(chromium --version) ready for screenshots"
}
shots || echo "::warning::no screenshots on this ARM run (the build is still tested)"
echo "Runner ready after $(($(date +%s) - start))s"
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# Installs ci-vm-controller.sh on the hypervisor (prox1): the script, the
# token reader, a systemd unit per CI VM, and each VM's config. Run as root
# on prox1 from a directory that also holds ci-vm-controller.sh:
#
# scp ci-controller-setup.sh ci-vm-controller.sh root@prox1:/root/ci-controller/
# ssh root@prox1 'cd /root/ci-controller && bash ci-controller-setup.sh'
#
# It does not start anything: a VM goes into service with
# systemctl enable --now ci-vm-controller@<vmid>
# once it has a 'clean' snapshot without any runner registered (see
# docs/guides/pr-script-testing.md, "Single-use hosts").
#
# The token (/etc/ci-controller/token) is put there by hand: a fine-grained
# token for the two repositories with Administration: read and write only.
set -euo pipefail
cd "$(dirname "$0")"
[[ -f ci-vm-controller.sh ]] || { echo "ci-vm-controller.sh missing next to this script" >&2; exit 1; }
command -v jq >/dev/null || DEBIAN_FRONTEND=noninteractive apt-get install -y -qq jq >/dev/null
install -m 755 ci-vm-controller.sh /usr/local/sbin/ci-vm-controller.sh
install -d -m 700 /etc/ci-controller
cat >/usr/local/sbin/ci-gh-token <<'EOF'
#!/bin/sh
# Prints the token ci-vm-controller.sh uses (Administration: write on the CI
# repositories). Swap this for a GitHub App token later without touching
# the controller.
exec cat /etc/ci-controller/token
EOF
chmod 755 /usr/local/sbin/ci-gh-token
cat >/etc/systemd/system/ci-vm-controller@.service <<'EOF'
[Unit]
Description=Single-use CI runner host, VM %i
After=network-online.target pve-cluster.service pvedaemon.service
Wants=network-online.target
# A missing token is a setup error, not something to retry every 30 s.
ConditionPathExists=/etc/ci-controller/token
[Service]
ExecStart=/usr/local/sbin/ci-vm-controller.sh %i
Restart=always
RestartSec=30
# Stopping the controller leaves the VM as it is; the next start resets it,
# which also ends a job that may still be running there.
KillMode=mixed
[Install]
WantedBy=multi-user.target
EOF
# One repository per host: see ci-vm-controller.sh.
conf() { # conf <vmid> <ip> <name> <label> <repo>
cat >"/etc/ci-controller/$1.conf" <<EOF
VM_IP=$2
HOST_NAME=$3
LABEL=$4
REPOS=$5
EOF
}
conf 200 10.201.146.232 pve-ci proxmox-test community-scripts/DevScripts
conf 201 10.201.146.231 pve-ci-2 proxmox-test community-scripts/ProxmoxVE
conf 204 10.201.146.235 pve-ci-3 proxmox-test community-scripts/ProxmoxVE
conf 202 10.201.146.233 incus-ci incus-test community-scripts/DevScripts
conf 203 10.201.146.234 incus-ci-2 incus-test community-scripts/ProxmoxVE
systemctl daemon-reload
echo "Installed. Token: $([[ -s /etc/ci-controller/token ]] && echo present || echo 'MISSING (/etc/ci-controller/token)')"
+201
View File
@@ -0,0 +1,201 @@
#!/usr/bin/env bash
# Keeps one CI VM (pve-ci, incus-ci, ...) as a single-use GitHub runner host.
# Runs on the hypervisor (prox1), which PR code never touches.
#
# Every cycle:
# 1. roll the VM back to its clean snapshot and boot it;
# 2. register a just-in-time runner on it (generate-jitconfig) for the
# host's repository. Such a runner takes exactly one job and is deleted
# by GitHub afterwards, so its credentials are worthless once that job
# is over;
# 3. (with several repositories in REPOS: as soon as one runner is busy,
# delete the others that are still idle, so no second job starts on a
# host that is about to be reset);
# 4. once every runner of this cycle is gone, start over at 1.
#
# PR code runs as root on the VM, so nothing the VM says is trusted: when a
# job is over is read from GitHub's API, never from the VM.
#
# Config: /etc/ci-controller/<vmid>.conf (or the environment), see below.
# Usage: ci-vm-controller.sh <vmid> run forever (the systemd unit)
# ONE_CYCLE=1 ci-vm-controller.sh <vmid> one job, then stop (testing)
set -euo pipefail
VMID=${1:?usage: ci-vm-controller.sh <vmid>}
CONF=${CONF:-/etc/ci-controller/$VMID.conf}
# shellcheck source=/dev/null
[[ -f "$CONF" ]] && . "$CONF"
: "${VM_IP:?VM_IP}" # address of the CI VM
: "${HOST_NAME:?HOST_NAME}" # runner name prefix, e.g. pve-ci
: "${LABEL:?LABEL}" # proxmox-test or incus-test
# One repository per host. Several would each have a runner waiting on the
# VM, and a job of one could read the idle runner's JIT config of the other
# and use it elsewhere before it is withdrawn: one repository's PR code
# faking the other's results. (Several are fine within one trust domain.)
: "${REPOS:?REPOS, e.g. community-scripts/DevScripts}"
EXTRA_LABELS=${EXTRA_LABELS:-} # space-separated, e.g. for a test
SNAPSHOT=${SNAPSHOT:-clean}
TOKEN_CMD=${TOKEN_CMD:-/usr/local/sbin/ci-gh-token} # prints a token with repo Administration: write
PVE=${PVE:-} # prefix for qm, e.g. "ssh root@prox1" when not run on it
GUEST_SSH=${GUEST_SSH:-"ssh -o BatchMode=yes -o ConnectTimeout=5 -o StrictHostKeyChecking=accept-new root@$VM_IP"}
POLL=${POLL:-20}
MAX_BUSY_MIN=${MAX_BUSY_MIN:-180} # the test job's own limit is 90
MAX_IDLE_MIN=${MAX_IDLE_MIN:-720} # recycle an idle host twice a day
ONE_CYCLE=${ONE_CYCLE:-}
log() { echo "[$(date -u +%FT%TZ)] vm $VMID: $*"; }
qm() { if [[ -n "$PVE" ]]; then $PVE qm "$@"; else command qm "$@"; fi; }
# Not in a subshell, so the status and the cached token survive the call:
# api METHOD PATH [JSON] -> body in $API_BODY, HTTP status in $API_STATUS.
API_FILE=$(mktemp)
trap 'rm -f "$API_FILE"' EXIT
TOKEN="" TOKEN_AT=0
api() {
local method=$1 path=$2 data=${3:-}
# Cached for 30 minutes (an App installation token lives 60).
if [[ -z "$TOKEN" || $(($(date +%s) - TOKEN_AT)) -gt 1800 ]]; then
TOKEN=$($TOKEN_CMD) || { log "no GitHub token"; API_STATUS=000 API_BODY=""; return 1; }
TOKEN_AT=$(date +%s)
fi
API_STATUS=$(curl -sS -o "$API_FILE" -w '%{http_code}' -X "$method" \
-H "Authorization: Bearer $TOKEN" -H 'Accept: application/vnd.github+json' \
-H 'X-GitHub-Api-Version: 2022-11-28' ${data:+-d "$data"} "https://api.github.com$path") || API_STATUS=000
API_BODY=$(<"$API_FILE")
[[ "$API_STATUS" != 000 ]]
}
reset_vm() {
log "rolling back to '$SNAPSHOT'"
[[ "$(qm status "$VMID")" == *running* ]] && qm stop "$VMID" --timeout 60 >/dev/null
qm rollback "$VMID" "$SNAPSHOT" >/dev/null
qm start "$VMID" >/dev/null
local i
for i in $(seq 1 60); do
$GUEST_SSH true 2>/dev/null && { log "booted"; return 0; }
sleep 5
done
log "no SSH after 5 minutes"
return 1
}
# Parallel arrays: one entry per runner registered in this cycle.
R_REPO=() R_ID=() R_GONE=()
register() { # register <owner/repo>
local repo=$1 name dir labels body cfg id
name="$HOST_NAME-${repo#*/}-$(date +%s)"
dir=/opt/actions-runner/${repo#*/}
labels=$(printf '%s\n' "$LABEL" $EXTRA_LABELS | jq -R . | jq -sc .)
api POST "/repos/$repo/actions/runners/generate-jitconfig" \
"$(jq -nc --arg n "$name" --argjson l "$labels" '{name: $n, runner_group_id: 1, labels: $l, work_folder: "_work"}')" || return 1
body=$API_BODY
[[ "$API_STATUS" == 201 ]] || { log "$repo: generate-jitconfig failed ($API_STATUS): $(jq -r .message <<<"$body" 2>/dev/null)"; return 1; }
id=$(jq -r .runner.id <<<"$body")
cfg=$(jq -r .encoded_jit_config <<<"$body")
# The config goes in through stdin, not the command line.
if ! $GUEST_SSH "umask 077; cat >/run/ci-jit-${repo#*/} && systemd-run --quiet --collect --unit=ci-jit-${repo#*/} \
-p WorkingDirectory=$dir -E RUNNER_ALLOW_RUNASROOT=1 \
/bin/sh -c 'cfg=\$(cat /run/ci-jit-${repo#*/}); rm -f /run/ci-jit-${repo#*/}; exec ./run.sh --jitconfig \"\$cfg\"'" <<<"$cfg"; then
log "$repo: could not start the runner on the VM"
api DELETE "/repos/$repo/actions/runners/$id" >/dev/null || true
return 1
fi
R_REPO+=("$repo") R_ID+=("$id") R_GONE+=(0)
log "$repo: runner $name (id $id) started"
}
drop() { # drop <index>: delete an idle runner; GitHub refuses a busy one
local i=$1
api DELETE "/repos/${R_REPO[$i]}/actions/runners/${R_ID[$i]}" >/dev/null || return 1
if [[ "$API_STATUS" == 204 || "$API_STATUS" == 404 ]]; then R_GONE[$i]=1; return 0; fi
return 1
}
watch_cycle() {
local start busy_since="" i body status busy any_busy left offline_since=""
start=$(date +%s)
while :; do
sleep "$POLL"
any_busy=0 left=0 offline=0
for i in "${!R_ID[@]}"; do
[[ "${R_GONE[$i]}" == 1 ]] && continue
api GET "/repos/${R_REPO[$i]}/actions/runners/${R_ID[$i]}" || { left=1; continue; }
body=$API_BODY
if [[ "$API_STATUS" == 404 ]]; then
R_GONE[$i]=1
log "${R_REPO[$i]}: runner ${R_ID[$i]} is gone (job done or removed)"
continue
fi
left=1
busy=$(jq -r .busy <<<"$body"); status=$(jq -r .status <<<"$body")
[[ "$busy" == true ]] && any_busy=1
[[ "$status" == offline ]] && offline=1
done
if [[ "$any_busy" == 1 && -z "$busy_since" ]]; then
busy_since=$(date +%s)
log "a job started; withdrawing the idle runners"
fi
# A host that has started a job takes no other one: it will be reset.
if [[ -n "$busy_since" ]]; then
for i in "${!R_ID[@]}"; do
[[ "${R_GONE[$i]}" == 1 ]] && continue
api GET "/repos/${R_REPO[$i]}/actions/runners/${R_ID[$i]}" || continue
body=$API_BODY
[[ "$API_STATUS" == 200 && "$(jq -r .busy <<<"$body")" == false ]] && drop "$i" &&
log "${R_REPO[$i]}: idle runner ${R_ID[$i]} withdrawn"
done
fi
[[ "$left" == 0 ]] && { log "cycle over"; return 0; }
local now; now=$(date +%s)
if [[ -n "$busy_since" && $((now - busy_since)) -gt $((MAX_BUSY_MIN * 60)) ]]; then
log "job still running after $MAX_BUSY_MIN minutes; resetting anyway"
break
fi
if [[ -z "$busy_since" && $((now - start)) -gt $((MAX_IDLE_MIN * 60)) ]]; then
log "idle for $MAX_IDLE_MIN minutes; recycling"
break
fi
# A runner that never comes online (or dies while idle) holds the host
# forever otherwise.
if [[ -z "$busy_since" && "$offline" == 1 ]]; then
[[ -z "$offline_since" ]] && offline_since=$now
if [[ $((now - offline_since)) -gt 300 ]]; then log "a runner is offline for 5 minutes; resetting"; break; fi
else
offline_since=""
fi
done
for i in "${!R_ID[@]}"; do [[ "${R_GONE[$i]}" == 1 ]] || drop "$i" || true; done
}
# Registrations of this host that a stopped or crashed controller left behind:
# their VM was reset, so they are offline and never take a job, but they
# clutter the runner list. Busy or online ones are left alone.
cleanup_stale() {
local repo ids id
for repo in $REPOS; do
api GET "/repos/$repo/actions/runners?per_page=100" || continue
[[ "$API_STATUS" == 200 ]] || continue
ids=$(jq -r --arg p "$HOST_NAME-${repo#*/}-" \
'.runners[] | select((.name | startswith($p)) and .status == "offline" and (.busy | not)) | .id' <<<"$API_BODY")
for id in $ids; do
api DELETE "/repos/$repo/actions/runners/$id" && log "$repo: stale runner $id removed"
done
done
}
while :; do
R_REPO=() R_ID=() R_GONE=()
if reset_vm; then
cleanup_stale
ok=0
for repo in $REPOS; do register "$repo" && ok=1 || true; done
if [[ "$ok" == 1 ]]; then watch_cycle; else log "no runner could be registered"; sleep 60; fi
else
sleep 60
fi
[[ -n "$ONE_CYCLE" ]] && { log "ONE_CYCLE: stopping; the VM is left as the job left it"; break; }
done
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env bash
# One-time setup of an Incus test host behind pr-script-test.yml: the same
# ct/ scripts as on Proxmox, built through community-scripts/core's Incus
# backend. Safe to re-run. Run as root on a disposable Debian 13 machine (it
# builds pull request code as root).
#
# incus-ci-host-setup.sh host only
# incus-ci-host-setup.sh <owner/repo> <token> host + a runner for that repo
# incus-ci-host-setup.sh <owner/repo> jit host + runner software only, for
# ci-vm-controller.sh (single-use host)
#
# The token is a runner *registration* token (valid one hour):
# gh api -X POST repos/<owner/repo>/actions/runners/registration-token --jq .token
set -euo pipefail
RUNNER_VERSION="${RUNNER_VERSION:-2.338.2}"
SERVE_ROOT=/srv/ci
SERVE_PORT=18080
NET_ADDR=10.98.0.1
NET_CIDR=10.98.0.0/24
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq incus jq curl python3 iptables chromium fonts-dejavu fonts-noto-color-emoji >/dev/null
# Containers get a NATed bridge of their own (Incus runs DHCP and DNS on it)
# and a plain directory pool on the root disk.
if ! incus storage show default &>/dev/null; then
incus admin init --preseed <<EOF
networks:
- name: incusbr0
type: bridge
config:
ipv4.address: ${NET_ADDR}/24
ipv4.nat: "true"
ipv6.address: none
storage_pools:
- name: default
driver: dir
profiles:
- name: default
devices:
root:
path: /
pool: default
type: disk
eth0:
name: eth0
network: incusbr0
type: nic
EOF
fi
# Screenshots of the app under test are taken as this user, never as root.
id ci-shot &>/dev/null || useradd --system --create-home --shell /usr/sbin/nologin ci-shot
# Same rules as on the Proxmox hosts: the screenshot browser may reach the
# test containers, DNS and the public internet, nothing private, and not this
# host's own address on the container network (the PR tree server).
cat >/usr/local/sbin/ci-shot-firewall <<EOF
#!/usr/bin/env bash
set -euo pipefail
uid=\$(id -u ci-shot)
for fw in iptables ip6tables; do
\$fw -w -N CI_SHOT 2>/dev/null || \$fw -w -F CI_SHOT
\$fw -w -C OUTPUT -m owner --uid-owner "\$uid" -j CI_SHOT 2>/dev/null ||
\$fw -w -I OUTPUT -m owner --uid-owner "\$uid" -j CI_SHOT
done
iptables -w -A CI_SHOT -d ${NET_ADDR} -j REJECT
iptables -w -A CI_SHOT -d ${NET_CIDR} -j ACCEPT
# systemd-resolved's stub; it forwards to the lab resolver itself.
iptables -w -A CI_SHOT -d 127.0.0.53 -p udp --dport 53 -j ACCEPT
iptables -w -A CI_SHOT -d 127.0.0.53 -p tcp --dport 53 -j ACCEPT
for ns in \$(awk '/^nameserver/ && \$2 !~ /:/ {print \$2}' /etc/resolv.conf /run/systemd/resolve/resolv.conf 2>/dev/null); do
iptables -w -A CI_SHOT -d "\$ns" -p udp --dport 53 -j ACCEPT
iptables -w -A CI_SHOT -d "\$ns" -p tcp --dport 53 -j ACCEPT
done
for net in 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16 172.16.0.0/12 192.168.0.0/16 224.0.0.0/3; do
iptables -w -A CI_SHOT -d "\$net" -j REJECT
done
for net in ::1/128 fc00::/7 fe80::/10 ff00::/8; do
ip6tables -w -A CI_SHOT -d "\$net" -j REJECT
done
EOF
chmod +x /usr/local/sbin/ci-shot-firewall
cat >/etc/systemd/system/ci-shot-firewall.service <<'EOF'
[Unit]
Description=Keep the screenshot browser away from this host and the lab network
After=network-online.target incus.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/ci-shot-firewall
[Install]
WantedBy=multi-user.target
EOF
[[ -s /etc/timezone ]] || {
timedatectl set-timezone Europe/Vienna
echo Europe/Vienna >/etc/timezone
}
# The rewritten PR trees, served to the engine and to the containers. Waits
# for incusbr0, which only exists once Incus is up.
mkdir -p "$SERVE_ROOT"
cat >/etc/systemd/system/ci-serve.service <<EOF
[Unit]
Description=Serve pull request trees to script builds
After=incus.service network-online.target
Wants=incus.service
[Service]
ExecStartPre=/bin/sh -c 'for i in \$(seq 1 60); do ip -4 addr show dev incusbr0 | grep -q ${NET_ADDR} && exit 0; incus network list >/dev/null 2>&1; sleep 1; done; exit 1'
ExecStart=/usr/bin/python3 -m http.server ${SERVE_PORT} --bind ${NET_ADDR} --directory ${SERVE_ROOT}
DynamicUser=yes
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
# Leftovers of a job that was killed before its cleanup trap ran.
cat >/usr/local/sbin/ci-sweep <<'EOF'
#!/usr/bin/env bash
# Delete CI containers (ci-9000 .. ci-9099) that no running job holds, and stale trees.
# A job holds its guest with /run/ci-jobs/<id>. A job killed outright (runner
# restart, timeout) never removes it, so a lock older than any job can run
# (90 minutes) holds nothing.
held() { [[ -e "/run/ci-jobs/$1" && -z "$(find "/run/ci-jobs/$1" -mmin +180)" ]]; }
for name in $(incus list -c n -f csv 2>/dev/null | grep -E '^ci-9[0-9]{3}$'); do
held "$name" && continue
incus delete "$name" --force &>/dev/null && echo "swept $name"
done
find /run/ci-jobs -maxdepth 1 -type f ! -name ".*" -mmin +180 -delete 2>/dev/null
find /srv/ci -mindepth 1 -maxdepth 1 -type d -mmin +180 -exec rm -rf {} +
find /usr/local/community-scripts/logs /var/log/community-scripts -type f -mtime +2 -delete 2>/dev/null
find /opt/actions-runner/*/_diag -type f -mtime +14 -delete 2>/dev/null
true
EOF
chmod +x /usr/local/sbin/ci-sweep
cat >/etc/systemd/system/ci-sweep.service <<'EOF'
[Unit]
Description=Remove leftover CI containers
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ci-sweep
EOF
cat >/etc/systemd/system/ci-sweep.timer <<'EOF'
[Unit]
Description=Remove leftover CI containers every 3 hours
[Timer]
OnCalendar=0/3:30
Persistent=true
[Install]
WantedBy=timers.target
EOF
systemctl daemon-reload
systemctl enable --now ci-serve.service ci-sweep.timer ci-shot-firewall.service >/dev/null
systemctl restart ci-serve.service ci-shot-firewall.service
# --- Runner (one per repository) ----------------------------------------------
if [[ $# -eq 2 ]]; then
repo="$1"
token="$2"
[[ "$repo" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || { echo "bad repo" >&2; exit 2; }
dir="/opt/actions-runner/${repo#*/}"
mkdir -p "$dir"
cd "$dir"
if [[ ! -x ./config.sh ]]; then
curl -fsSL "https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz" | tar -xz
./bin/installdependencies.sh >/dev/null
fi
# `jit`: a single-use host. ci-vm-controller.sh on the hypervisor registers
# a fresh runner for every job and resets the VM afterwards, so only the
# runner software goes on it, and the 'clean' snapshot is taken after this.
if [[ "$token" == jit ]]; then
echo "runner software for ${repo} installed; ci-vm-controller.sh registers a runner per job"
exit 0
fi
if [[ -f .runner ]]; then
./svc.sh stop >/dev/null 2>&1 || true
./svc.sh uninstall >/dev/null 2>&1 || true
RUNNER_ALLOW_RUNASROOT=1 ./config.sh remove --token "$token" >/dev/null 2>&1 || true
fi
# Only `incus-test`: see pve-ci-host-setup.sh for why there are no default labels.
RUNNER_ALLOW_RUNASROOT=1 ./config.sh --unattended --replace --no-default-labels \
--url "https://github.com/${repo}" --token "$token" \
--name "$(hostname -s)-${repo#*/}" --labels incus-test --work _work
./svc.sh install root >/dev/null
./svc.sh start >/dev/null
echo "runner for ${repo} registered"
fi
echo "incus test host ready: ${SERVE_ROOT} on ${NET_ADDR}:${SERVE_PORT}, sweeper every 3h"
+229
View File
@@ -0,0 +1,229 @@
#!/usr/bin/env bash
# One-time setup of the Proxmox test host behind pr-script-test.yml.
# Safe to re-run. Run as root on the test host (a disposable PVE, never a
# production node: it builds pull request code as root).
#
# pve-ci-host-setup.sh host only
# pve-ci-host-setup.sh <owner/repo> <token> host + a runner for that repo
# pve-ci-host-setup.sh <owner/repo> jit host + runner software only, for
# ci-vm-controller.sh (single-use host)
#
# The token is a runner *registration* token (valid one hour):
# gh api -X POST repos/<owner/repo>/actions/runners/registration-token --jq .token
set -euo pipefail
RUNNER_VERSION="${RUNNER_VERSION:-2.338.2}"
SERVE_ROOT=/srv/ci
SERVE_PORT=18080
UPLINK="${UPLINK:-vmbr0}"
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq jq curl python3 dnsmasq iptables chromium fonts-dejavu fonts-noto-color-emoji >/dev/null
# Screenshots of the app under test are taken as this user, never as root.
id ci-shot &>/dev/null || useradd --system --create-home --shell /usr/sbin/nologin ci-shot
# The page Chromium renders is served by code under test, and a redirect or an
# <img> could point it at this host (the PVE UI on :8006) or the lab network;
# the screenshot would then publish that on the PR. So ci-shot may reach the
# test containers, DNS and the public internet (apps load fonts and scripts
# from CDNs), and nothing private.
cat >/usr/local/sbin/ci-shot-firewall <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
uid=$(id -u ci-shot)
for fw in iptables ip6tables; do
$fw -w -N CI_SHOT 2>/dev/null || $fw -w -F CI_SHOT
$fw -w -C OUTPUT -m owner --uid-owner "$uid" -j CI_SHOT 2>/dev/null ||
$fw -w -I OUTPUT -m owner --uid-owner "$uid" -j CI_SHOT
done
iptables -w -A CI_SHOT -d 10.99.0.1 -j REJECT
iptables -w -A CI_SHOT -d 10.99.0.0/24 -j ACCEPT
for ns in $(awk '/^nameserver/ && $2 !~ /:/ {print $2}' /etc/resolv.conf); do
iptables -w -A CI_SHOT -d "$ns" -p udp --dport 53 -j ACCEPT
iptables -w -A CI_SHOT -d "$ns" -p tcp --dport 53 -j ACCEPT
done
for net in 0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16 172.16.0.0/12 192.168.0.0/16 224.0.0.0/3; do
iptables -w -A CI_SHOT -d "$net" -j REJECT
done
for net in ::1/128 fc00::/7 fe80::/10 ff00::/8; do
ip6tables -w -A CI_SHOT -d "$net" -j REJECT
done
EOF
chmod +x /usr/local/sbin/ci-shot-firewall
cat >/etc/systemd/system/ci-shot-firewall.service <<'EOF'
[Unit]
Description=Keep the screenshot browser away from this host and the lab network
After=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/ci-shot-firewall
[Install]
WantedBy=multi-user.target
EOF
# Containers get their own NATed bridge instead of the lab network: no DHCP
# lease surprises (a lab lease once handed a container the host's own address),
# and an app listening on 8080 can never be confused with this host.
cat >/etc/network/interfaces.d/ci-nat <<EOF
auto vmbr1
iface vmbr1 inet static
address 10.99.0.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
post-up sysctl -qw net.ipv4.ip_forward=1
post-up iptables -t nat -C POSTROUTING -s 10.99.0.0/24 -o ${UPLINK} -j MASQUERADE 2>/dev/null || iptables -t nat -A POSTROUTING -s 10.99.0.0/24 -o ${UPLINK} -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s 10.99.0.0/24 -o ${UPLINK} -j MASQUERADE || true
EOF
ifreload -a
cat >/etc/dnsmasq.d/ci.conf <<'EOF'
interface=vmbr1
bind-interfaces
except-interface=lo
dhcp-range=10.99.0.100,10.99.0.250,255.255.255.0,2h
dhcp-option=option:router,10.99.0.1
dhcp-option=option:dns-server,10.99.0.1
no-resolv
EOF
# Upstream = whatever this host resolves with. Debian's dnsmasq would otherwise
# ask resolvconf, which has nothing to offer on a PVE host.
awk '/^nameserver/ {print "server=" $2}' /etc/resolv.conf >>/etc/dnsmasq.d/ci.conf
systemctl enable dnsmasq >/dev/null
systemctl restart dnsmasq
# A PVE ISO install writes /etc/timezone; a cloud-image install does not, and
# the engine's pct_timezone aborts without it.
[[ -s /etc/timezone ]] || {
timedatectl set-timezone Europe/Vienna
echo Europe/Vienna >/etc/timezone
}
# Containers and VM disks live on the root filesystem; there is no LVM in a
# cloud-image PVE. Snippets hold the Cloud-Init configs some vm/ scripts write.
pvesm set local --content iso,vztmpl,backup,rootdir,images,snippets
# Templates are fetched once here rather than on every first build.
pveam update >/dev/null
for t in $(pveam available --section system | awk '/_amd64\./ && /debian-13-standard|alpine-3\.2[0-9]-default|ubuntu-24\.04-standard/ {print $2}'); do
pveam list local | grep -q "$t" || pveam download local "$t" >/dev/null
done
# The rewritten PR trees, served to the engine and to the containers.
mkdir -p "$SERVE_ROOT"
cat >/etc/systemd/system/ci-serve.service <<EOF
[Unit]
Description=Serve pull request trees to script builds
After=network-online.target
[Service]
ExecStart=/usr/bin/python3 -m http.server ${SERVE_PORT} --bind 10.99.0.1 --directory ${SERVE_ROOT}
DynamicUser=yes
Restart=always
[Install]
WantedBy=multi-user.target
EOF
# Leftovers of a job that was killed before its cleanup trap ran.
cat >/usr/local/sbin/ci-sweep <<'EOF'
#!/usr/bin/env bash
# Destroy CI containers (9000-9099) and VMs (9100-9199) that no running job
# holds, and stale trees.
# A job holds its guest with /run/ci-jobs/<id>. A job killed outright (runner
# restart, timeout) never removes it, so a lock older than any job can run
# (90 minutes) holds nothing.
held() { [[ -e "/run/ci-jobs/$1" && -z "$(find "/run/ci-jobs/$1" -mmin +180)" ]]; }
for id in $(pct list | awk 'NR>1 && $1>=9000 && $1<=9099 {print $1}'); do
held "$id" && continue
pct stop "$id" --skiplock 1 &>/dev/null || true
pct destroy "$id" --purge 1 --force 1 &>/dev/null && echo "swept CT $id"
done
for id in $(qm list | awk 'NR>1 && $1>=9100 && $1<=9199 {print $1}'); do
held "$id" && continue
qm stop "$id" --skiplock 1 --timeout 30 &>/dev/null || true
qm destroy "$id" --purge 1 --destroy-unreferenced-disks 1 --skiplock 1 &>/dev/null && echo "swept VM $id"
done
# VM images and ISOs are cached between builds; drop the ones no build used
# for a week. Container templates (*.tar.*) stay.
find /var/lib/vz/template/cache /var/lib/vz/template/iso -maxdepth 1 -type f ! -name '*.tar.*' -atime +7 -delete 2>/dev/null
find /run/ci-jobs -maxdepth 1 -type f ! -name ".*" -mmin +180 -delete 2>/dev/null
find /srv/ci -mindepth 1 -maxdepth 1 -type d -mmin +180 -exec rm -rf {} +
# Engine logs that a killed job never got to remove.
find /tmp -maxdepth 1 -type f \( -name 'create-lxc-*.log' -o -name 'pct_create_*.log' \
-o -regex '/tmp/.*-9[0-9][0-9][0-9]-[0-9a-f]+\.log' \) -mmin +180 -delete
find /var/log/community-scripts -type f -mtime +2 -delete 2>/dev/null
find /opt/actions-runner/*/_diag -type f -mtime +14 -delete 2>/dev/null
true
EOF
chmod +x /usr/local/sbin/ci-sweep
cat >/etc/systemd/system/ci-sweep.service <<'EOF'
[Unit]
Description=Remove leftover CI containers
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ci-sweep
EOF
cat >/etc/systemd/system/ci-sweep.timer <<'EOF'
[Unit]
Description=Remove leftover CI containers every 3 hours
[Timer]
OnCalendar=0/3:30
Persistent=true
[Install]
WantedBy=timers.target
EOF
systemctl daemon-reload
systemctl enable --now ci-serve.service ci-sweep.timer ci-shot-firewall.service >/dev/null
systemctl restart ci-serve.service ci-shot-firewall.service
# --- Runner (one per repository) ----------------------------------------------
if [[ $# -eq 2 ]]; then
repo="$1"
token="$2"
[[ "$repo" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || { echo "bad repo" >&2; exit 2; }
dir="/opt/actions-runner/${repo#*/}"
mkdir -p "$dir"
cd "$dir"
if [[ ! -x ./config.sh ]]; then
curl -fsSL "https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz" | tar -xz
./bin/installdependencies.sh >/dev/null
fi
# `jit`: a single-use host. ci-vm-controller.sh on the hypervisor registers
# a fresh runner for every job and resets the VM afterwards, so only the
# runner software goes on it, and the 'clean' snapshot is taken after this.
if [[ "$token" == jit ]]; then
echo "runner software for ${repo} installed; ci-vm-controller.sh registers a runner per job"
exit 0
fi
# A registered runner is reconfigured from scratch: stop and remove the old
# registration (the token works for removal too).
if [[ -f .runner ]]; then
./svc.sh stop >/dev/null 2>&1 || true
./svc.sh uninstall >/dev/null 2>&1 || true
RUNNER_ALLOW_RUNASROOT=1 ./config.sh remove --token "$token" >/dev/null 2>&1 || true
fi
# Root, because pct needs it; the host is disposable by design.
# --no-default-labels: without it the runner also answers to `self-hosted`,
# and any other workflow with `runs-on: self-hosted` (and its secrets) could
# land on this host.
# Named after this host: with --replace, a fixed name would make a second
# test host take over the first one's registration instead of adding to it.
RUNNER_ALLOW_RUNASROOT=1 ./config.sh --unattended --replace --no-default-labels \
--url "https://github.com/${repo}" --token "$token" \
--name "$(hostname -s)-${repo#*/}" --labels proxmox-test --work _work
./svc.sh install root >/dev/null
./svc.sh start >/dev/null
echo "runner for ${repo} registered"
fi
echo "test host ready: ${SERVE_ROOT} on :${SERVE_PORT}, sweeper every 3h"
+863
View File
@@ -0,0 +1,863 @@
#!/usr/bin/env bash
# Build one ct/ or vm/ script from a pull request on this test host, check
# that the container or VM really works, destroy it, and leave a result.json
# behind. The vm/ part lives in pve-run-vm.sh, sourced from here.
#
# PLATFORM says what this host is: pve (Proxmox VE, pct) or incus. The ct/
# scripts are the same for both; community-scripts/core picks its Incus
# backend by itself when it finds incus instead of pveversion. Only the way
# this harness reaches into the container differs (guest_* below).
#
# Runs on the self-hosted runner, which *is* the disposable test host. The
# harness comes from the default branch; only the script under test comes from
# the pull request, pinned to the commit a maintainer approved.
#
# The engine hardcodes raw.githubusercontent.com/<upstream>/main for every fetch
# (ct -> build.func -> install script -> install.func -> tools.func). Pointing it
# at a fork branch therefore means rewriting those URLs: the PR tree is unpacked
# under SERVE_ROOT, every upstream main URL in it is replaced by this host's own
# HTTP server, and the containers fetch from there. Engine changes in the PR are
# tested along with the script, which is what a reviewer wants anyway.
set -uo pipefail
: "${PR_REPO:?}" "${PR_SHA:?}" "${SCRIPT:?}" "${UPSTREAM:?}" "${CTID_BASE:?}" "${OUT_DIR:?}"
OS="${OS:-}"
PLATFORM="${PLATFORM:-pve}"
CTID_COUNT="${CTID_COUNT:-50}"
SERVE_ROOT="${SERVE_ROOT:-/srv/ci}"
# The containers' own network: vmbr1 + dnsmasq on Proxmox, incusbr0 on Incus.
if [[ "$PLATFORM" == "incus" ]]; then
SERVE_URL="${SERVE_URL:-http://10.98.0.1:18080}"
BRIDGE="${BRIDGE:-incusbr0}"
else
SERVE_URL="${SERVE_URL:-http://10.99.0.1:18080}"
BRIDGE="${BRIDGE:-vmbr1}"
fi
BUILD_TIMEOUT="${BUILD_TIMEOUT:-45m}"
URL_WAIT="${URL_WAIT:-120}"
# Every value below reaches a path, a URL or a command line.
[[ "$PR_REPO" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || { echo "bad PR_REPO" >&2; exit 2; }
[[ "$UPSTREAM" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || { echo "bad UPSTREAM" >&2; exit 2; }
[[ "$PR_SHA" =~ ^[0-9a-f]{40}$ ]] || { echo "bad PR_SHA" >&2; exit 2; }
[[ "$SCRIPT" =~ ^(ct|vm)/[A-Za-z0-9._-]+\.sh$ ]] || { echo "bad SCRIPT" >&2; exit 2; }
[[ -z "$OS" || "$OS" =~ ^(debian|ubuntu|alpine)$ ]] || { echo "bad OS" >&2; exit 2; }
[[ "$PLATFORM" =~ ^(pve|incus)$ ]] || { echo "bad PLATFORM" >&2; exit 2; }
# yes: build with var_arm64=yes. On arm64 the engine refuses to guess for a
# script that leaves var_arm64 unset; the ARM run builds those anyway, to see.
VAR_ARM64="${VAR_ARM64:-}"
[[ -z "$VAR_ARM64" || "$VAR_ARM64" == "yes" ]] || { echo "bad VAR_ARM64" >&2; exit 2; }
# upgrade: install the script as it is on the base branch (BASE_SHA of
# UPSTREAM), then update with the PR. That is what a user does who installed
# before the PR is merged and runs `update` after it. Empty: install and
# update both from the PR.
TEST_MODE="${TEST_MODE:-}"
[[ -z "$TEST_MODE" || "$TEST_MODE" == "upgrade" ]] || { echo "bad TEST_MODE" >&2; exit 2; }
if [[ -n "$TEST_MODE" ]]; then
[[ "${BASE_SHA:-}" =~ ^[0-9a-f]{40}$ ]] || { echo "bad BASE_SHA" >&2; exit 2; }
[[ "$SCRIPT" == ct/* ]] || { echo "upgrade tests are for ct/ scripts" >&2; exit 2; }
fi
[[ "$PLATFORM" == "pve" || "$SCRIPT" == ct/* ]] || { echo "vm/ scripts run on pve only" >&2; exit 2; }
[[ "$CTID_BASE" =~ ^[0-9]+$ ]] || { echo "bad CTID_BASE" >&2; exit 2; }
mkdir -p "$OUT_DIR"
RAW_LOG="$OUT_DIR/build.raw.log"
LOG="$OUT_DIR/build.log"
RESULT="$OUT_DIR/result.json"
INSTALL_LOG_OUT="$OUT_DIR/install.log"
UPDATE_RAW="$OUT_DIR/update.raw.log"
UPDATE_LOG="$OUT_DIR/update.log"
UPDATE_TIMEOUT="${UPDATE_TIMEOUT:-30m}"
UPDATE2_RAW="$OUT_DIR/update2.raw.log"
UPDATE2_LOG="$OUT_DIR/update2.log"
# How long the app has to keep running without a service restarting.
STABLE_WAIT="${STABLE_WAIT:-60}"
kind="${SCRIPT%%/*}" # ct or vm
app=$(basename "$SCRIPT" .sh)
job="${GITHUB_RUN_ID:-local}-${GITHUB_RUN_ATTEMPT:-1}-${PLATFORM}-${kind}-${app}-${OS:-default}${TEST_MODE:+-$TEST_MODE}"
tree="$SERVE_ROOT/$job"
base="$SERVE_URL/$job"
src_tree="$tree" # what gets installed; the base branch's tree in an upgrade test
started=$(date +%s)
status="fail"
step=""
exit_code=""
ctid="" # the guest's ID (pve: 9000, incus: ci-9000), container or VM
# The container, whatever the platform. ctid is set once a slot is reserved.
guest_exec() {
if [[ "$PLATFORM" == "incus" ]]; then incus exec "$ctid" -- "$@"; else pct exec "$ctid" -- "$@"; fi
}
guest_exists() {
if [[ "$PLATFORM" == "incus" ]]; then incus info "$ctid" &>/dev/null; else pct status "$ctid" &>/dev/null; fi
}
guest_running() {
if [[ "$PLATFORM" == "incus" ]]; then
[[ "$(incus list "^${ctid}\$" -c s -f csv 2>/dev/null)" == "RUNNING" ]]
else
pct status "$ctid" 2>/dev/null | grep -q running
fi
}
guest_destroy() {
if [[ "$PLATFORM" == "incus" ]]; then
incus delete "$ctid" --force &>/dev/null || true
else
pct stop "$ctid" --skiplock 1 &>/dev/null || true
pct destroy "$ctid" --purge 1 --force 1 &>/dev/null || true
fi
}
ct_ip=""
url=""
http_code=""
failed_units=""
# updated: a real update ran; no-update: update ran, nothing to update;
# fail: the update broke; empty: not attempted (the install failed).
update_status=""
update_step=""
# install: building and checking what was installed; update: from the first
# update on. In an upgrade test a failure while installing is the base
# branch's, not the PR's.
phase="install"
# Findings that do not fail the build, shown under the table on the PR.
notes=()
# Disk, RAM and OOM kills of the container, measured at the end.
resources=""
# Second `update` right after the first: no-update, updated-again, or empty.
update2_status=""
# Spinner frames: the engine redraws "⠋ Starting VM" a few times a second
# without a tty too. An alternation, not a bracket: these are multibyte and
# the runner's locale may be C.
SPINNER_RE='^\s*(⠋|⠙|⠹|⠸|⠼|⠴|⠦|⠧|⠇|⠏)'
strip_ansi() {
sed -E 's/\x1B\[[0-9;?]*[A-Za-z]//g; s/\x1B[()][A-Z0-9]//g; s/\r/\n/g' | grep -vE '^\s*$' | grep -vE "$SPINNER_RE"
}
# The install script's own output: everything $STD hides from the build log.
# Both engines write it inside the container as /root/.install-<session>.log;
# dev_mode=keep makes sure a failed container is still there to read it from.
# If it is gone anyway, the copy the engine leaves on the host is the fallback.
collect_install_log() {
[[ -n "$ctid" ]] || return 0
# Exactly one build ran in this container, so the newest install log is ours.
if guest_exists &&
guest_exec sh -c 'cat "$(ls -t /root/.install-*.log 2>/dev/null | head -n1)"' 2>/dev/null |
strip_ansi >"$INSTALL_LOG_OUT" && [[ -s "$INSTALL_LOG_OUT" ]]; then
return 0
fi
[[ "$PLATFORM" == "pve" ]] || { rm -f "$INSTALL_LOG_OUT"; return 0; }
# The engine names its host-side files after the session: pct_create_<ctid>_<date>_<session>.log
local sid f
sid=$(find /tmp -maxdepth 1 -name "pct_create_${ctid}_*.log" -newer "$RAW_LOG.start" -printf '%f\n' 2>/dev/null |
sed -E 's/.*_([0-9a-f]+)\.log$/\1/' | head -n1)
[[ -n "$sid" ]] || { rm -f "$INSTALL_LOG_OUT"; return 0; }
for f in /tmp/*-"${ctid}"-"${sid}".log /var/log/community-scripts/install-"${sid}"-combined-*.log; do
[[ -s "$f" ]] && { strip_ansi <"$f" >"$INSTALL_LOG_OUT"; return 0; }
done
rm -f "$INSTALL_LOG_OUT"
}
# "Requires AVX2" and friends: the script may be fine, the test host's CPU is
# not (Xeon E5-2650, Sandy Bridge: AVX, no AVX2). Shown as untestable rather
# than as a failure of the PR -- but the log is written by the code under test,
# so its word is not enough. Only the engine's error lines count, only when the
# build or install itself failed, and the feature they name must really be
# missing from this host's /proc/cpuinfo. Claiming a feature the host has
# changes nothing.
CPU_RE='(requir|need)[a-z]*[^.]{0,60}(AVX(2|-?512)?|SSE4[._]?2|x86-64-v[234])\b|\b(AVX(2|-?512)?|x86-64-v[234])\b[^.]{0,60}(not (supported|available)|does not have|is missing|unsupported)'
classify_cpu_failure() {
local line feature flags
line=$(strip_ansi <"$RAW_LOG" | grep -E '✖|\[ERROR\]' | grep -m1 -iE "$CPU_RE" |
sed -E 's/^\s*(✖️?|\[[^]]*\]|\s)*//' | cut -c1-200)
[[ -n "$line" ]] || return 0
# The CPU flag that would have to be missing for the message to be true.
case "${line,,}" in
*avx-512* | *avx512* | *x86-64-v4*) feature=avx512f ;;
*avx2* | *x86-64-v3*) feature=avx2 ;;
*sse4.2* | *sse4_2* | *x86-64-v2*) feature=sse4_2 ;;
*) return 0 ;;
esac
flags=" $(grep -m1 '^flags' /proc/cpuinfo | cut -d: -f2) "
[[ "$flags" == *" ${feature} "* ]] && return 0
status="unsupported"
step="needs a newer CPU than the test host has (no ${feature}): ${line}"
}
# Disk used against var_disk, RAM in use at rest (anonymous memory, without
# the page cache that fills any limit) against the limit, and how often the
# kernel killed something for lack of memory, over the container's whole life,
# the build included. A dir-backed Incus root has no size of its own: df shows
# the host's disk. Then what the container holds is counted with du, and the
# script's var_disk is the size to compare with.
measure_resources() {
local out disk_size disk_used anon limit oom want_disk want_ram line=()
out=$(guest_exec sh -c 'df -Pm / | awk "NR==2 {print \$2, \$3}"
awk "/^anon / {print int(\$2 / 1048576)}" /sys/fs/cgroup/memory.stat 2>/dev/null || echo -
cat /sys/fs/cgroup/memory.max 2>/dev/null || echo -
awk "/^oom_kill / {print \$2}" /sys/fs/cgroup/memory.events 2>/dev/null || echo -' 2>/dev/null) || return 0
read -r disk_size disk_used <<<"$(sed -n 1p <<<"$out")"
anon=$(sed -n 2p <<<"$out"); limit=$(sed -n 3p <<<"$out"); oom=$(sed -n 4p <<<"$out")
want_disk=$(grep -m1 -oE 'var_disk="\$\{var_disk:-[0-9]+\}"' "$src_tree/$SCRIPT" | grep -oE '[0-9]+' | head -n1)
want_ram=$(grep -m1 -oE 'var_ram="\$\{var_ram:-[0-9]+\}"' "$src_tree/$SCRIPT" | grep -oE '[0-9]+' | head -n1)
[[ "$disk_size" =~ ^[0-9]+$ && "$disk_used" =~ ^[0-9]+$ ]] || return 0
if [[ -n "$want_disk" ]] && ((disk_size > want_disk * 1024 * 3 / 2)); then
disk_size=$((want_disk * 1024))
disk_used=$(guest_exec sh -c 'du -sxm / 2>/dev/null | cut -f1' 2>/dev/null)
[[ "$disk_used" =~ ^[0-9]+$ ]] || return 0
fi
line+=("disk $(awk -v u="$disk_used" 'BEGIN {printf "%.1f", u / 1024}') of $(awk -v s="$disk_size" 'BEGIN {printf "%.0f", s / 1024}') GB")
if ((disk_used * 100 >= disk_size * 90)); then
notes+=("disk ${disk_used} of ${disk_size} MB used after the install: var_disk may be too small")
fi
[[ "$limit" =~ ^[0-9]+$ ]] && limit=$((limit / 1048576)) || limit="${want_ram:-}"
if [[ "$anon" =~ ^[0-9]+$ ]]; then
line+=("RAM at rest ${anon}${limit:+ of ${limit}} MB")
if [[ "$limit" =~ ^[0-9]+$ ]] && ((anon * 100 >= limit * 80)); then
notes+=("${anon} of ${limit} MB RAM in use at rest: var_ram may be too small")
fi
fi
if [[ "$oom" =~ ^[0-9]+$ ]]; then
line+=("OOM kills ${oom}")
((oom > 0)) && notes+=("the kernel killed ${oom} process(es) for lack of memory: var_ram is too small for the build or the app")
fi
resources=$(
IFS='|'
echo "${line[*]}"
)
resources=${resources//|/ · }
}
finish() {
[[ -f "$RAW_LOG" ]] && strip_ansi <"$RAW_LOG" >"$LOG"
[[ -f "$RAW_LOG" ]] && collect_install_log
[[ -f "$UPDATE_RAW" ]] && strip_ansi <"$UPDATE_RAW" >"$UPDATE_LOG"
[[ -f "$UPDATE2_RAW" ]] && strip_ansi <"$UPDATE2_RAW" >"$UPDATE2_LOG"
rm -f "$UPDATE_RAW" "$UPDATE2_RAW"
[[ "$kind" == ct && -n "$ctid" && -z "$resources" ]] && guest_running && measure_resources
jq -n \
--arg script "$SCRIPT" --arg kind "$kind" --arg platform "$PLATFORM" --arg os "${OS:-default}" --arg status "$status" \
--arg mode "$TEST_MODE" --arg resources "$resources" --arg update2_status "$update2_status" \
--arg notes "$(printf '%s\n' "${notes[@]}")" \
--arg step "$step" --arg exit_code "$exit_code" --arg sha "$PR_SHA" \
--arg ctid "$ctid" --arg url "$url" --arg http_code "$http_code" \
--arg failed_units "$failed_units" --arg ip "$ct_ip" --arg arch "$(uname -m)" \
--arg update_status "$update_status" --arg update_step "$update_step" \
--argjson duration "$(($(date +%s) - started))" \
--arg tail "$([[ -f "$LOG" ]] && tail -n 60 "$LOG")" \
'{script: $script, kind: $kind, platform: $platform, arch: $arch, os: $os, status: $status, step: $step, exit_code: $exit_code,
sha: $sha, ctid: $ctid, ip: $ip, url: $url, http_code: $http_code,
failed_units: $failed_units, update_status: $update_status, update_step: $update_step,
mode: $mode, resources: $resources, update2_status: $update2_status,
notes: ($notes | split("\n") | map(select(. != ""))),
duration: $duration, log_tail: $tail}' >"$RESULT"
echo "::group::result"
jq 'del(.log_tail)' "$RESULT"
echo "::endgroup::"
}
# --- Live output ------------------------------------------------------------------
# The job log on GitHub streams while the job runs, so mirroring the build there
# is all "live" takes. Two streams: the engine's own output, and the install
# log inside the container, which holds everything $STD hides (apt, npm, ...).
#
# Both are written by the code under test. A line starting with `::` would be
# read by the runner as a workflow command, so command processing is switched
# off for as long as they run, with a token the build cannot know.
stream_token=""
log_pid=""
install_pid=""
# Colours stay, cursor movement and spinner redraws go.
live_filter() {
sed -u -E 's/\r/\n/g; s/\x1B\[[0-9;?]*[ABCDEFGHJKSTfhilnsu]//g' | sed -u -E "/^\s*\$/d; /${SPINNER_RE}/d"
}
start_streams() {
stream_token=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
echo "::stop-commands::${stream_token}"
tail -n +1 -F --pid="$build_pid" "$RAW_LOG" 2>/dev/null > >(live_filter) &
log_pid=$!
# A VM has no install log to follow: its script runs on this host only.
[[ "$kind" == "ct" ]] || return 0
# setsid: its own process group, so stop_streams can take down the
# pct/incus exec + tail pair inside it in one go.
setsid bash -c '
gx() { if [ "$2" = incus ]; then incus exec "$1" -- "${@:3}"; else pct exec "$1" -- "${@:3}"; fi; }
for _ in $(seq 1 900); do
f=$(gx "$1" "$2" sh -c "ls -t /root/.install-*.log 2>/dev/null | head -n1" 2>/dev/null)
if [ -n "$f" ]; then
echo "──────── install log of CT $1 (live) ────────"
if [ "$2" = incus ]; then exec incus exec "$1" -- tail -n +1 -F "$f"; fi
exec pct exec "$1" -- tail -n +1 -F "$f"
fi
sleep 2
done' _ "$ctid" "$PLATFORM" 2>/dev/null > >(live_filter | sed -u 's/^/ │ /') &
install_pid=$!
}
stop_streams() {
[[ -n "$stream_token" ]] || return 0
[[ -n "$install_pid" ]] && kill -TERM -- "-$install_pid" &>/dev/null
[[ -n "$log_pid" ]] && kill -TERM "$log_pid" &>/dev/null
wait "$log_pid" "$install_pid" &>/dev/null
sleep 1 # let the filters flush
install_pid=""
log_pid=""
echo "::${stream_token}::"
stream_token=""
}
build_pid=""
update_pid=""
cleanup() {
stop_streams
local pid
for pid in "$build_pid" "$update_pid"; do
[[ -n "$pid" ]] || continue
pkill -TERM -P "$pid" &>/dev/null || true
kill -TERM "$pid" &>/dev/null || true
sleep 2
done
if [[ -n "$ctid" && "$kind" == "vm" ]]; then
vm_cleanup
elif [[ -n "$ctid" ]]; then
guest_exists && guest_destroy
rm -f "/run/ci-jobs/$ctid"
# What the engine leaves on the host for this container and its session.
local f sid
[[ "$PLATFORM" == "pve" ]] && for f in /tmp/pct_create_"${ctid}"_*.log; do
[[ -f "$f" && "$f" -nt "$RAW_LOG.start" ]] || continue
sid=$(sed -E 's/.*_([0-9a-f]+)\.log$/\1/' <<<"$f")
rm -f -- "$f" /tmp/create-lxc-"${sid}".log /tmp/*-"${ctid}"-"${sid}".log
done
fi
rm -f "$RAW_LOG.start"
rm -rf -- "$tree" "$tree-base"
}
trap cleanup EXIT
# A cancelled job sends TERM; without this the EXIT trap would not run.
trap 'exit 143' TERM
trap 'exit 130' INT
# fail <reason> [fail|unsupported]: only the build/install branches pass the
# second argument, after classify_cpu_failure has checked the host.
#
# The exit code says whether *this job* worked, not whether the app did. A
# script that fails to build is a result: it is written down, reported on the
# PR (check, comment, review) and the job exits 0 -- a red run per broken app
# would only be notification noise. Only trouble with the test setup itself
# (setup_error) fails the job, because that is ours to look at.
fail() {
step="$1"
status="${2:-fail}"
[[ "$status" == "unsupported" ]] || status="fail"
if [[ "$TEST_MODE" == "upgrade" && "$phase" == "install" && "$status" == "fail" ]]; then
status="error"
step="the install before this PR (${UPSTREAM}@${BASE_SHA:0:7}) failed, so the upgrade was not tested: $1"
fi
finish
echo "::warning title=${SCRIPT}::${step}"
exit 0
}
setup_error() {
step="$1"
status="error"
finish
echo "::error title=${SCRIPT}::test setup: ${step}"
exit 1
}
# Any answer means the app is up, unless it says it isn't: nothing at all,
# 404/410 (the printed URL leads nowhere), 5xx (server error, proxy without
# backend), or a 3xx left after 5 redirects (a loop). 401/403 are a login
# wall (warpgate answers 401), 400 a quirk of the app: both pass.
url_ok() { [[ "$1" =~ ^[1-4][0-9][0-9]$ && ! "$1" =~ ^(3[0-9][0-9]|404|410)$ ]]; }
url_why() {
case "$1" in
404 | 410) echo "the printed URL leads nowhere" ;;
5??) echo "server error" ;;
3??) echo "still redirecting after 5 hops" ;;
*) echo "the app never answered" ;;
esac
}
# --- Reserve a container ID ---------------------------------------------------
# A lock file per ID, not just "first free one", because the engine only creates
# the container minutes later and two jobs would otherwise pick the same ID.
# VMs take the range 100 above the containers' (DevScripts 9100-9149,
# ProxmoxVE 9150-9199).
mkdir -p /run/ci-jobs
if [[ "$kind" == "vm" ]]; then
source "$(dirname "$0")/pve-run-vm.sh"
CTID_BASE=$((CTID_BASE + 100))
vm_wait_turn
fi
exec 9>/run/ci-jobs/.lock
flock 9
# On Incus the ID is an instance name: var_ctid=ci-9000 names the container.
for ((id = CTID_BASE; id < CTID_BASE + CTID_COUNT; id++)); do
name=$id
[[ "$PLATFORM" == "incus" ]] && name="ci-$id"
[[ -e "/run/ci-jobs/$name" ]] && continue
if [[ "$PLATFORM" == "incus" ]]; then
incus info "$name" &>/dev/null && continue
else
pvesh get /cluster/nextid --vmid "$id" &>/dev/null || continue
fi
echo "$job" >"/run/ci-jobs/$name"
ctid=$name
break
done
flock -u 9
exec 9>&-
[[ -n "$ctid" ]] || setup_error "no free ${kind} ID in ${CTID_BASE}-$((CTID_BASE + CTID_COUNT - 1))"
# --- Fetch the approved commit and point it at ourselves ------------------------
mkdir -p "$tree"
curl -fsSL --retry 3 "https://codeload.github.com/${PR_REPO}/tar.gz/${PR_SHA}" |
tar -xz -C "$tree" --strip-components=1 || setup_error "could not download ${PR_REPO}@${PR_SHA:0:7}"
[[ -f "$tree/$SCRIPT" ]] || fail "${SCRIPT} does not exist at ${PR_SHA:0:7}"
upstream_re="https://raw\.githubusercontent\.com/${UPSTREAM//./\\.}/(refs/heads/)?main"
grep -rlE "$upstream_re" "$tree" --include='*.sh' --include='*.func' |
xargs -r sed -i -E "s#${upstream_re}#${base}#g"
# What gets installed: the PR, or in an upgrade test the base branch.
src_tree="$tree"
src_base="$base"
if [[ "$TEST_MODE" == "upgrade" ]]; then
src_tree="$tree-base"
src_base="$base-base"
mkdir -p "$src_tree"
curl -fsSL --retry 3 "https://codeload.github.com/${UPSTREAM}/tar.gz/${BASE_SHA}" |
tar -xz -C "$src_tree" --strip-components=1 || setup_error "could not download ${UPSTREAM}@${BASE_SHA:0:7}"
[[ -f "$src_tree/$SCRIPT" ]] || fail "${SCRIPT} does not exist at ${UPSTREAM}@${BASE_SHA:0:7}"
grep -rlE "$upstream_re" "$src_tree" --include='*.sh' --include='*.func' |
xargs -r sed -i -E "s#${upstream_re}#${src_base}#g"
fi
if [[ "$kind" == "vm" ]]; then
vm_build_and_check
exit 0
fi
# --- Build ----------------------------------------------------------------------
if [[ "$TEST_MODE" == "upgrade" ]]; then
echo "Upgrade test: installing ${SCRIPT} from ${UPSTREAM}@${BASE_SHA:0:7} (before this PR), then updating with ${PR_REPO}@${PR_SHA:0:7}"
fi
echo "Building ${SCRIPT} (${OS:-default OS}) on ${PLATFORM} as CT ${ctid} from $([[ "$TEST_MODE" == upgrade ]] && echo "${UPSTREAM}@${BASE_SHA:0:7}" || echo "${PR_REPO}@${PR_SHA:0:7}")"
hostname_safe="ci-$(tr -c 'a-z0-9-\n' '-' <<<"${app,,}" | cut -c1-50)"
build_env=(
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOME=/root TERM=dumb LANG=C.UTF-8
mode=default MODE=default PHS_SILENT=1 DIAGNOSTICS=no
var_ctid="$ctid" var_hostname="$hostname_safe" var_brg="$BRIDGE"
"dev_mode=timing,keep" COMMUNITY_SCRIPTS_URL="$src_base"
)
if [[ "$PLATFORM" == "incus" ]]; then
build_env+=(var_container_storage=default)
else
build_env+=(var_container_storage=local var_template_storage=local)
fi
touch "$RAW_LOG.start"
[[ -n "$OS" ]] && build_env+=(var_os="$OS")
[[ -n "$VAR_ARM64" ]] && build_env+=(var_arm64=yes)
# A script that hands over to upstream's own installer (run_external_installer)
# asks first; nobody can answer in CI, and testing the script means running it.
build_env+=(var_external_installer=yes)
# Both engines turn var_github_token into GITHUB_TOKEN inside the container.
[[ -n "${GH_API_TOKEN:-}" ]] && build_env+=(var_github_token="$GH_API_TOKEN")
# How much API quota this build starts with, with the token and without (the
# lab's one address). /rate_limit itself costs nothing. A "rate limit" in the
# build is then easy to place.
gh_quota() {
curl -s --max-time 10 ${1:+-H "Authorization: Bearer $1"} https://api.github.com/rate_limit 2>/dev/null |
jq -r '.rate | "\(.remaining) of \(.limit) left, resets in \(((.reset - now) / 60) | floor) min"' 2>/dev/null || echo "unknown"
}
echo "GitHub API quota: token $(gh_quota "${GH_API_TOKEN:-}") · no token $(gh_quota "")"
# --- Answers for the script's questions ------------------------------------------
# Nobody types into CI, so anything the install asks has to be answered up front.
# AGENTS.md rule 24: a script reads var_<x> first and prompts only when it is
# unset (or uses prompt_input_required ... "var_x"). Every such variable gets a
# CI value here. The names come from json/<app>.json app_vars (DevScripts) and
# from the read/prompt_ lines of the install script (both repos). A bare `read`
# that ignores its variable still fails, and the report says why.
ENGINE_VARS=" var_ctid var_hostname var_os var_version var_cpu var_ram var_disk var_brg var_net var_container_storage var_template_storage var_unprivileged var_tags var_arm64 var_testurl var_verbose var_timezone var_http_proxy var_http_no_proxy var_appversion var_github_token "
host_app=$(tr -c 'a-z0-9-\n' '-' <<<"${app,,}")
ci_value() {
case "${1,,}" in
*pass* | *secret* | *token* | *key*) head -c 32 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | cut -c1-20 ;;
*email* | *mail*) echo "ci@example.test" ;;
*url* | *endpoint*) echo "https://ci-${host_app}.example.test" ;;
# One name per variable: var_opencloud_collabora_fqdn ->
# ci-opencloud-collabora.example.test, so several of them cannot collide.
*host* | *domain* | *fqdn*)
local part="${1#var_}"
part=$(sed -E 's/_?(fqdn|hostname|host|domain)$//' <<<"${part,,}" | tr '_' '-')
echo "ci-${part:-$host_app}.example.test"
;;
*user*) echo "ciadmin" ;;
*) echo "ci-test" ;;
esac
}
declare -A answers=()
json="$src_tree/json/${app}.json"
if [[ -f "$json" ]]; then
while IFS=$'\t' read -r name default; do
answers[$name]="$default"
done < <(jq -r '(if type == "array" then .[0] else . end).app_vars // [] | .[] |
select(.name | test("^var_[a-z0-9_]{1,40}$")) | [.name, (.default // "" | tostring)] | @tsv' "$json" 2>/dev/null)
fi
# Names built at run time from a loop, e.g. OpenCloud:
# servers=("opencloud" "collabora" "wopi")
# for server in "${servers[@]}"; do fqdn_var="var_opencloud_${server}_fqdn"
# Each var_...${x}... template is expanded with the words of `for x in ...`
# (or of the array that loop walks). Nothing is executed, only read.
expand_templates() {
local tpl x pre suf words arr w
while read -r tpl; do
x=$(sed -E 's/^[^$]*\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?.*$/\1/' <<<"$tpl")
pre="${tpl%%\$*}"
suf=$(sed -E 's/^[^$]*\$\{?[A-Za-z_][A-Za-z0-9_]*\}?//' <<<"$tpl")
words=$(grep -m1 -oE "for ${x} in [^;]*" "$1" | sed -E "s/^for ${x} in //")
if [[ "$words" =~ \$\{([A-Za-z_][A-Za-z0-9_]*)\[@\]\} ]]; then
arr="${BASH_REMATCH[1]}"
words=$(grep -m1 -oE "^\s*${arr}=\([^)]*\)" "$1" | sed -E "s/^\s*${arr}=\(//; s/\)$//")
fi
for w in $words; do
w="${w//[\"\']/}"
[[ "${pre}${w}${suf}" =~ ^var_[a-z0-9_]{1,40}$ ]] && echo "${pre}${w}${suf}"
done
done < <(grep -oE 'var_[a-z0-9_]*\$\{?[A-Za-z_][A-Za-z0-9_]*\}?[a-z0-9_]*' "$1" | sort -u)
}
# The engine names the install script after APP (lower case, no spaces), which
# need not match the ct/ file name: ct/bluesky-pds.sh -> blueskypds-install.sh.
nsapp=$(grep -m1 -oE '^APP="[^"]+"' "$src_tree/$SCRIPT" | cut -d'"' -f2 | tr -d ' ' | tr '[:upper:]' '[:lower:]')
for f in "$src_tree/install/${app}-install.sh" "$src_tree/install/${nsapp}-install.sh" "$src_tree/$SCRIPT"; do
[[ -f "$f" ]] || continue
while read -r name; do
[[ -v "answers[$name]" ]] || answers[$name]=""
done < <({
grep -E '\bread\b|prompt_(input|input_required|confirm|select|password)\b' "$f" | grep -oE '\bvar_[a-z0-9_]{1,40}\b'
expand_templates "$f"
} | sort -u)
done
for name in "${!answers[@]}"; do
[[ "$ENGINE_VARS" == *" $name "* ]] && continue
value="${answers[$name]}"
[[ -n "$value" ]] || value=$(ci_value "$name")
build_env+=("$name=$value")
case "${name,,}" in
*pass* | *secret* | *token* | *key*) echo "Pre-answered ${name}=(random)" ;;
*) echo "Pre-answered ${name}=${value}" ;;
esac
done
# In the background so that a TERM from a cancelled job is handled at once:
# bash defers traps until a foreground command returns.
env -i "${build_env[@]}" timeout --kill-after=1m "$BUILD_TIMEOUT" \
bash -c 'bash -c "$(curl -fsSL "$1")"' _ "$src_base/$SCRIPT" </dev/null >"$RAW_LOG" 2>&1 &
build_pid=$!
start_streams
wait "$build_pid"
exit_code=$?
build_pid=""
stop_streams
if ((exit_code == 124 || exit_code == 137)); then
fail "build timed out after ${BUILD_TIMEOUT}"
fi
# The engine marks the step that broke with a cross. Its closing "Installation
# failed in container" line is generic, so the cross before it is the useful one.
failure_line() {
local line
line=$(strip_ansi <"$RAW_LOG" | grep -E '✖|\[ERROR\]|error:' | grep -v 'Installation failed in container' |
tail -n 1 | sed -E 's/^\s*(✖️?|\s)*//' | cut -c1-200)
# A prompt nobody can answer: say what to do instead.
# First, because the report keeps 200 characters.
if [[ "$line" == *"read exited with status"* || "$line" == *"executing command read "* ]]; then
local asked
asked=$(grep -oE '\bvar_[a-z0-9_]{1,40}\b' <<<"$line" | tail -n 1)
line="unanswerable prompt (AGENTS.md rule 24): ask only when ${asked:-var_<x>} is unset, or use prompt_input_required. ${line}"
fi
echo "$line"
}
if ((exit_code != 0)); then
last=$(failure_line)
step="build exited ${exit_code}${last:+: $last}"
status="fail"
classify_cpu_failure
fail "$step" "$status"
fi
# With dev_mode=keep (needed to read the install log) the engine carries on
# after a failed install and still prints "Completed Successfully", exit 0.
# The container is a number on Proxmox (9000) and a name on Incus (ci-9000).
install_exit=$(strip_ansi <"$RAW_LOG" | grep -oE 'Installation failed in container [A-Za-z0-9._-]+ \(exit code: [0-9]+\)' |
grep -oE '[0-9]+\)$' | tr -d ')' | tail -n 1)
if [[ -n "$install_exit" ]]; then
last=$(failure_line)
step="install script exited ${install_exit}${last:+: $last}"
status="fail"
classify_cpu_failure
fail "$step" "$status"
fi
grep -qi 'completed successfully' "$RAW_LOG" || fail "build exited 0 but never reported success"
# --- Checks ---------------------------------------------------------------------
guest_running || fail "container ${ctid} is not running"
ct_ip=$(guest_exec sh -c "ip -4 -o addr show dev eth0 | awk '{print \$4}' | cut -d/ -f1" 2>/dev/null | head -n1)
# What the app's page looks like, for the PR comment. Chromium runs as its own
# unprivileged user with its sandbox on: the page is served by code under test.
# ci-shot-firewall (pve-ci-host-setup.sh) keeps that user off this host and the
# lab network; without it there is no screenshot rather than an unguarded one.
# $1: file name in OUT_DIR (screenshot.png after install, -update after update).
take_screenshot() {
command -v chromium >/dev/null && id ci-shot &>/dev/null || return 0
iptables -w -S CI_SHOT 2>/dev/null | grep -q -- '-d 10.0.0.0/8 .*REJECT' || return 0
local dir
dir=$(mktemp -d)
chown ci-shot "$dir"
# HOME: runuser keeps root's, which ci-shot cannot write; Playwright's
# Chromium (the ARM runner) then crashes in its crash handler.
runuser -u ci-shot -- env HOME="$dir" timeout 60 chromium --headless=new --disable-gpu --hide-scrollbars \
--ignore-certificate-errors --no-first-run --window-size=1280,800 --virtual-time-budget=10000 \
--user-data-dir="$dir/profile" --screenshot="$dir/shot.png" "$url" &>/dev/null
# Only a plausible PNG leaves this function.
if [[ -s "$dir/shot.png" && $(stat -c %s "$dir/shot.png") -lt 3000000 ]] &&
[[ "$(head -c 8 "$dir/shot.png" | od -An -tx1 | tr -d ' \n')" == "89504e470d0a1a0a" ]]; then
cp "$dir/shot.png" "$OUT_DIR/${1:-screenshot.png}"
fi
rm -rf -- "$dir"
}
# Is the app up? No failed services, and the URL the script tells the user to
# open, e.g. http://${IP}:1337/admin, answers without an error (see url_ok;
# redirects are followed, so a jump to /login is fine). A script that prints no URL has no web UI to check.
# Run after the install and again after the update. Sets health_problem.
app_url() { # the URL the script at $1 tells the user to open
local pattern
pattern=$(grep -oE 'https?://\$\{IP\}(:[0-9]+)?(/[A-Za-z0-9._~/?=&%-]*)?' "$1" | head -n1)
[[ -n "$pattern" && -n "$ct_ip" ]] && echo "${pattern/\$\{IP\}/$ct_ip}"
}
url=$(app_url "$src_tree/$SCRIPT")
health_problem=""
check_health() {
health_problem=""
failed_units=""
if guest_exec sh -c 'command -v systemctl' &>/dev/null; then
sleep 10
failed_units=$(guest_exec systemctl --failed --no-legend --plain 2>/dev/null | awk '{print $1}' | paste -sd, -)
elif guest_exec sh -c 'command -v rc-status' &>/dev/null; then
failed_units=$(guest_exec rc-status --crashed 2>/dev/null | awk '{print $1}' | paste -sd, -)
fi
[[ -z "$failed_units" ]] || { health_problem="failed services: ${failed_units}"; return 1; }
[[ -n "$url" ]] || return 0
local waited
for ((waited = 0; waited < URL_WAIT; waited += 5)); do
http_code=$(curl -sk -L --max-redirs 5 -o /dev/null -w '%{http_code}' --max-time 10 "$url" || true)
url_ok "$http_code" && break
sleep 5
done
# Also when the check fails: an error page explains more than a status code.
[[ "$http_code" =~ ^[1-5][0-9][0-9]$ ]] && take_screenshot "$1"
url_ok "$http_code" ||
{ health_problem="${url} answered ${http_code/#000/nothing} within ${URL_WAIT}s ($(url_why "$http_code"))"; return 1; }
}
check_health screenshot.png || fail "$health_problem"
# --- Update -------------------------------------------------------------------------
# A fresh install is already on the latest release, so a plain `update` would
# only say "No update available" and never run update_script's real path. The
# version files the install wrote (~/.<app>, one per fetch_and_deploy_gh_release)
# that update_script checks are set back first (see below); check_for_gh_release
# then sees an update and runs it all. Apps deployed from a branch
# (fetch_and_deploy_gh_branch, checked with check_for_gh_branch) store a commit
# hash instead of a version. `update` is the /usr/bin/update the install generated: it pulls
# ct/<app>.sh from the URL this container was built from, i.e. the PR's tree.
UPDATE_FAKE="0.0.0-ci"
update_fail() {
update_status="fail"
update_step="$1"
fail "update: $1"
}
version_files=$(guest_exec sh -c '
for f in /root/.*; do
[ -f "$f" ] || continue
case "$f" in */.install-* | *.pending | *.bak | */.bash* | */.profile | */.hushlogin | */.viminfo | */.lesshst | */.wget-hsts | */.selected_editor) continue ;; esac
[ "$(wc -l <"$f")" -le 1 ] && grep -qE "^(v?[0-9]+(\.[0-9A-Za-z_+-]+)+|[0-9a-f]{7,40})$" "$f" && echo "$f"
done' 2>/dev/null | grep -E '^/root/\.[A-Za-z0-9._-]+$')
# Only set back what update_script checks: the names it passes to
# check_for_*release. The install can also deploy helpers the update never
# touches (immich: ~/.mise); those would stay 0.0.0-ci and fail the run. A
# name that is a variable can't be read here, so then all of them are set back.
tracked=$(grep -oE 'check_for_[a-z]*_?release[[:space:]]+("[^"]*"|[^[:space:]]+)' "$tree/$SCRIPT" |
sed -E 's/^check_for_[a-z]*_?release[[:space:]]+//; s/^"(.*)"$/\1/' | tr -d ' ' | tr '[:upper:]' '[:lower:]' | sort -u)
if [[ -n "$tracked" && "$tracked" != *'$'* ]]; then
version_files=$(grep -xF -f <(sed 's|^|/root/.|' <<<"$tracked") <<<"$version_files" || true)
fi
# A version keeps its digits and gets "-dev" (6.7.16.0 -> 6.7.16.0-dev): the
# engine ranks that below the release, so the update runs, and an
# update_script that reads the installed series (shopware: composer
# "~6.7.0" from ~/.shopware) still gets the real one. 0.0.0-ci broke that.
# A commit hash (or a bare number) has no series and gets 0.0.0-ci.
declare -A update_fake=()
for f in $version_files; do
cur=$(guest_exec cat "$f" 2>/dev/null | tr -d '\r\n')
if [[ "$cur" =~ ^v?[0-9]+(\.[0-9A-Za-z_+-]+)+$ ]]; then
update_fake[$f]="${cur}-dev"
else
update_fake[$f]="$UPDATE_FAKE"
fi
guest_exec sh -c "echo '${update_fake[$f]}' > '$f'; rm -f '$f.pending'" ||
update_fail "could not reset $f"
echo "Update test: set back $f from ${cur} to ${update_fake[$f]}"
done
phase="update"
# Upgrade test: the container pulls its update from where it was installed
# from (COMMUNITY_SCRIPTS_URL in /usr/bin/update); point that at the PR, as
# merging it would.
if [[ "$TEST_MODE" == "upgrade" ]]; then
guest_exec sed -i -E "s#^export COMMUNITY_SCRIPTS_URL=.*#export COMMUNITY_SCRIPTS_URL=\"${base}\"#" /usr/bin/update &&
guest_exec grep -qxF "export COMMUNITY_SCRIPTS_URL=\"${base}\"" /usr/bin/update ||
setup_error "could not point /usr/bin/update at the PR"
echo "Upgrade test: /usr/bin/update now pulls from ${PR_REPO}@${PR_SHA:0:7}"
fi
# run_update <raw log> <title>: runs `update` in the container, live in the
# job log; sets update_rc.
run_update() {
local raw=$1 token
: >"$raw"
token=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')
echo "──────── $2 (live) ────────"
echo "::stop-commands::${token}"
# timeout runs a program, not a shell function, so spell out the exec here.
if [[ "$PLATFORM" == "incus" ]]; then guest_cmd=(incus exec "$ctid" --); else guest_cmd=(pct exec "$ctid" --); fi
timeout --kill-after=1m "$UPDATE_TIMEOUT" "${guest_cmd[@]}" env PHS_SILENT=1 mode=default MODE=default \
TERM=dumb DIAGNOSTICS=no var_ignore_breaking_changes=yes var_external_installer=yes ${GH_API_TOKEN:+GITHUB_TOKEN="$GH_API_TOKEN"} update </dev/null \
> >(tee "$raw" | live_filter) 2>&1 &
update_pid=$!
wait "$update_pid"
update_rc=$?
update_pid=""
sleep 1
echo "::${token}::"
}
run_update "$UPDATE_RAW" "update test"
update_failure_line() {
strip_ansi <"${1:-$UPDATE_RAW}" | grep -E '✖|\[ERROR\]|error:' | tail -n 1 | sed -E 's/^\s*(✖️?|\s)*//' | cut -c1-180
}
if ((update_rc == 124 || update_rc == 137)); then
update_fail "timed out after ${UPDATE_TIMEOUT}"
elif ((update_rc != 0)); then
last=$(update_failure_line)
update_fail "exited ${update_rc}${last:+: $last}"
fi
# The engine can report an error and still exit 0 (bark-server: "github: rate
# limit reached", then "Cleaned"). A rate limit is ours, not the PR's: the
# update could not be tested. Any other error line is a failed update.
update_error=$(strip_ansi <"$UPDATE_RAW" | grep -E '✖|\[ERROR\]' | tail -n 1 | sed -E 's/^\s*(✖️?|\s)*//' | cut -c1-180)
if [[ -n "$update_error" ]]; then
if [[ "${update_error,,}" == *"rate limit"* ]]; then
echo "GitHub API quota now: token $(gh_quota "${GH_API_TOKEN:-}") · no token $(gh_quota "")"
update_status="error"
update_step="could not be tested: ${update_error}"
status="pass"
finish
exit 0
fi
update_fail "$update_error"
fi
if strip_ansi <"$UPDATE_RAW" | grep -qi 'updated successfully'; then
update_status="updated"
for f in $version_files; do
[[ "$(guest_exec cat "$f" 2>/dev/null)" != "${update_fake[$f]}" ]] ||
update_fail "reported success, but $f is still ${update_fake[$f]}"
done
else
# Ran cleanly without updating anything: no version file to fake (apt/npm
# based), or an update_script that decides differently. Not a failure.
update_status="no-update"
update_step=$(strip_ansi <"$UPDATE_RAW" | grep -iE 'no update|up to date|up-to-date|already' | tail -n 1 | sed -E 's/^\s*(✔️?|ℹ️?|\s)*//' | cut -c1-160)
fi
url=$(app_url "$tree/$SCRIPT")
check_health screenshot-update.png || update_fail "after the update: $health_problem"
# --- Second update ------------------------------------------------------------------
# Right after an update, `update` has nothing to do: it must say so and leave
# the app alone. One that breaks on a second run breaks for every user on
# their next update; one that updates again although nothing changed
# rebuilds on every run (often a version file it never writes).
if [[ "$update_status" == "updated" ]]; then
run_update "$UPDATE2_RAW" "second update (must have nothing to do)"
if ((update_rc == 124 || update_rc == 137)); then
update_fail "second run timed out after ${UPDATE_TIMEOUT}"
elif ((update_rc != 0)); then
last=$(update_failure_line "$UPDATE2_RAW")
update_fail "second run exited ${update_rc}${last:+: $last}"
fi
update_error=$(strip_ansi <"$UPDATE2_RAW" | grep -E '✖|\[ERROR\]' | tail -n 1 | sed -E 's/^\s*(✖️?|\s)*//' | cut -c1-180)
if [[ -n "$update_error" && "${update_error,,}" != *"rate limit"* ]]; then
update_fail "second run: $update_error"
fi
if strip_ansi <"$UPDATE2_RAW" | grep -qi 'updated successfully'; then
update2_status="updated-again"
notes+=("a second \`update\` right after the first updated again: update_script does not notice it is current (e.g. a version file it never writes)")
else
update2_status="no-update"
fi
check_health screenshot-update.png || update_fail "after the second update: $health_problem"
fi
# --- Does it stay up? -----------------------------------------------------------
# A service that crash-loops can be "active" at every single look. Count the
# restarts of every service (and Docker container) over STABLE_WAIT seconds:
# any that went up was restarted while nobody touched the container.
restart_counts() {
guest_exec sh -c '
if command -v systemctl >/dev/null; then
systemctl list-units --type=service --all --no-legend --plain | awk "{print \$1}" |
while read -r u; do echo "$u $(systemctl show -p NRestarts --value "$u" 2>/dev/null)"; done
fi
if command -v docker >/dev/null; then
docker ps -aq | xargs -r docker inspect -f "docker:{{.Name}} {{.RestartCount}}" 2>/dev/null
fi' 2>/dev/null | awk '$2 ~ /^[0-9]+$/' | sort
}
before=$(restart_counts)
sleep "$STABLE_WAIT"
after=$(restart_counts)
restarting=$(join <(echo "$before") <(echo "$after") | awk '$3 > $2 {printf "%s%s (+%d)", sep, $1, $3 - $2; sep=", "}')
[[ -z "$restarting" ]] || fail "not stable: restarted within ${STABLE_WAIT}s without anyone touching it: ${restarting}"
check_health screenshot-update.png || fail "not stable: ${STABLE_WAIT}s later, ${health_problem}"
# Error lines in the journal since boot: often harmless, sometimes the reason
# a feature is broken. Information only.
if guest_exec sh -c 'command -v journalctl' &>/dev/null; then
# Without the noise every container produces: udev may not trigger devices
# in an unprivileged container, and logind finds no VTs.
journal_errors=$(guest_exec journalctl -b -p err -q -o cat --no-pager 2>/dev/null | grep -vE '^\s*$' |
grep -vE "Failed to write 'add' to '/sys/.*/uevent'|Seat has no VTs but VT number not 0" | sort | uniq -c | sort -rn)
if [[ -n "$journal_errors" ]]; then
n=$(awk '{s += $1} END {print s}' <<<"$journal_errors")
top=$(head -n 2 <<<"$journal_errors" | sed -E 's/^\s*[0-9]+\s+//' | cut -c1-120 | paste -sd'|' - | sed 's/|/; /g')
notes+=("${n} error line(s) in the journal since boot, e.g. ${top}")
fi
fi
status="pass"
finish
+140
View File
@@ -0,0 +1,140 @@
# shellcheck shell=bash
# The vm/ half of pve-run-script.sh, sourced from it: same inputs, same
# result.json, same live output. Only building and checking differ.
#
# A VM script runs entirely on this host (there is no install script inside
# the guest to follow), unattended through the core engine's VM_UNATTENDED=1:
# the script's defaults, with ID, name, bridge and storage set from here. The
# guest then boots on vmbr1 like the containers do, so its address is whatever
# this host's dnsmasq leased to its MAC; no guest agent needed.
#
# What passes: the script exits 0 and says "Completed successfully", the VM is
# running and keeps running through the boot wait, and where the script prints
# a URL built from the VM's address (http://${VM_IP}:9000) that URL answers 200.
# An installer ISO or an image without DHCP gets no address; that is reported,
# not failed. There is no update test: a VM has no `update`.
VM_BOOT_WAIT="${VM_BOOT_WAIT:-300}"
VM_URL_WAIT="${VM_URL_WAIT:-600}"
LEASES="${LEASES:-/var/lib/misc/dnsmasq.leases}"
# One VM build per host at a time: a VM takes 2-8 GB of RAM and a disk image,
# where a container takes a fraction. Held until the job exits; the build is
# started with this descriptor closed, so a QEMU process can never inherit it.
vm_wait_turn() {
exec 7>/run/ci-jobs/.vm-turn
flock -n 7 && return 0
echo "Another VM build is running on this host; waiting for it (one VM at a time)"
flock -w 3600 7 || setup_error "waited an hour for this host's VM slot"
}
vm_cleanup() {
if qm status "$ctid" &>/dev/null; then
qm stop "$ctid" --skiplock 1 --timeout 30 &>/dev/null || true
qm destroy "$ctid" --purge 1 --destroy-unreferenced-disks 1 --skiplock 1 &>/dev/null || true
fi
rm -f "/run/ci-jobs/$ctid" "/run/ci-jobs/console-${ctid}.png"
# Cloud-Init network snippets the script wrote for this VM.
[[ -f "$RAW_LOG.start" ]] &&
find /var/lib/vz/snippets -maxdepth 1 -type f -name "*${ctid}*" -newer "$RAW_LOG.start" -delete 2>/dev/null
true
}
# The VM's screen, straight from QEMU. Pixels only, but they come from the
# guest, so the same checks as for a browser screenshot apply.
vm_console_shot() {
local f="/run/ci-jobs/console-${ctid}.png"
rm -f "$f"
printf 'screendump %s -f png\n' "$f" | timeout 30 qm monitor "$ctid" &>/dev/null
if [[ -s "$f" && $(stat -c %s "$f") -lt 3000000 ]] &&
[[ "$(head -c 8 "$f" | od -An -tx1 | tr -d ' \n')" == "89504e470d0a1a0a" ]]; then
cp "$f" "$OUT_DIR/screenshot.png"
fi
rm -f "$f"
}
vm_running() {
qm status "$ctid" 2>/dev/null | grep -q 'status: running'
}
vm_build_and_check() {
local hn mac pattern waited last
hn="ci-$(tr -c 'a-z0-9-\n' '-' <<<"${app,,}" | cut -c1-40)-${ctid}"
echo "Building ${SCRIPT} as VM ${ctid} from ${PR_REPO}@${PR_SHA:0:7}"
local vm_env=(
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOME=/root TERM=dumb LANG=C.UTF-8 DIAGNOSTICS=no
VM_UNATTENDED=1 VM_VMID="$ctid" VM_HOSTNAME="$hn" VM_BRIDGE="$BRIDGE"
VM_STORAGE=local VM_ISO_STORAGE=local VM_START=yes
COMMUNITY_SCRIPTS_URL="$base"
)
# Release lookups (vm_latest_from_github and the like).
[[ -n "${GH_API_TOKEN:-}" ]] && vm_env+=(GITHUB_TOKEN="$GH_API_TOKEN")
touch "$RAW_LOG.start"
env -i "${vm_env[@]}" timeout --kill-after=1m "$BUILD_TIMEOUT" \
bash -c 'bash -c "$(curl -fsSL "$1")"' _ "$base/$SCRIPT" </dev/null >"$RAW_LOG" 2>&1 7>&- &
build_pid=$!
start_streams
wait "$build_pid"
exit_code=$?
build_pid=""
stop_streams
((exit_code == 124 || exit_code == 137)) && fail "build timed out after ${BUILD_TIMEOUT}"
if ((exit_code != 0)); then
last=$(failure_line)
step="script exited ${exit_code}${last:+: $last}"
status="fail"
classify_cpu_failure
fail "$step" "$status"
fi
# vm_finish ends on a msg_ok of the script's choosing ("VM created. ..."),
# older scripts on "Completed successfully". exit_script and friends leave
# with 0 too, but on a cross.
last=$(strip_ansi <"$RAW_LOG" | tail -n 1 | cut -c1-200)
if ! grep -qi 'completed successfully' "$RAW_LOG" && ! [[ "$last" =~ ^[[:space:]]*✔ ]]; then
fail "script exited 0 but never reported success${last:+ (last line: ${last})}"
fi
vm_running || fail "VM ${ctid} is not running after the script finished"
# The address dnsmasq handed to this VM's MAC. The VM must stay up while it
# boots: one that powers itself off or crashes is a failure.
mac=$(qm config "$ctid" 2>/dev/null | sed -nE 's/^net0:[^=]*=([0-9A-Fa-f:]{17}).*/\1/p' | tr 'A-F' 'a-f')
echo "Waiting up to ${VM_BOOT_WAIT}s for VM ${ctid} (${mac:-no NIC}) to boot and get an address"
for ((waited = 0; waited < VM_BOOT_WAIT; waited += 5)); do
vm_running || fail "VM ${ctid} stopped by itself while booting"
[[ -n "$mac" ]] && ct_ip=$(awk -v m="$mac" '$2 == m {print $3}' "$LEASES" 2>/dev/null | tail -n 1)
[[ -n "$ct_ip" ]] && break
sleep 5
done
[[ -n "$ct_ip" ]] && echo "VM ${ctid} got ${ct_ip}" || echo "VM ${ctid} got no DHCP lease within ${VM_BOOT_WAIT}s"
# Only a URL the script builds from the address it found; <VM-IP> in a
# summary is a hint for later (after an installer ran), not a promise.
# ${VM_IP}, $VM_IP and ${VM_IP:-<VM-IP>} all count.
local ip_re='\$\{?VM_IP(:-[^}]*)?\}?'
pattern=$(grep -oE "https?://${ip_re}(:[0-9]+)?(/[A-Za-z0-9._~/?=&%-]*)?" "$tree/$SCRIPT" | head -n1)
if [[ -n "$pattern" ]]; then
[[ -n "$ct_ip" ]] || fail "no IP address within ${VM_BOOT_WAIT}s, so ${pattern} could not be checked"
url=$(sed -E "s#${ip_re}#${ct_ip}#" <<<"$pattern")
for ((waited = 0; waited < VM_URL_WAIT; waited += 10)); do
vm_running || fail "VM ${ctid} stopped by itself while its web UI was awaited"
http_code=$(curl -sk -L --max-redirs 5 -o /dev/null -w '%{http_code}' --max-time 10 "$url" || true)
url_ok "$http_code" && break
sleep 10
done
[[ "$http_code" =~ ^[1-5][0-9][0-9]$ ]] && take_screenshot screenshot.png
else
# Let the console settle on a login prompt rather than a boot message.
sleep 20
fi
[[ -s "$OUT_DIR/screenshot.png" ]] || vm_console_shot
if [[ -n "$url" ]] && ! url_ok "$http_code"; then
fail "${url} answered ${http_code/#000/nothing} within ${VM_URL_WAIT}s"
fi
vm_running || fail "VM ${ctid} stopped by itself"
status="pass"
finish
}
+168 -17
View File
@@ -1,9 +1,5 @@
name: Update script timestamp on .sh changes
# After a push to main, stamps the PocketBase record of every script whose .sh
# files changed. The workflow lives in community-scripts/ci (shared with
# DevScripts); this file only holds the trigger and what differs here.
on:
push:
branches:
@@ -15,18 +11,173 @@ on:
- "turnkey/**/*.sh"
- "vm/**/*.sh"
permissions: {}
jobs:
update-script-timestamp:
permissions:
contents: read
uses: community-scripts/ci/.github/workflows/update-script-timestamp.yml@main
with:
mode: main
paths: "ct/ install/ tools/ turnkey/ vm/"
pocketbase_url: ${{ vars.POCKETBASE_URL }}
pocketbase_collection: ${{ vars.POCKETBASE_COLLECTION }}
secrets:
pocketbase_admin_email: ${{ secrets.POCKETBASE_ADMIN_EMAIL }}
pocketbase_admin_password: ${{ secrets.POCKETBASE_ADMIN_PASSWORD }}
runs-on: self-hosted
steps:
- name: Checkout Repository
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Get changed .sh files and derive slugs
id: slugs
run: |
changed=$(git diff --name-only "${{ github.event.before }}" "${{ github.event.after }}" -- ct/ install/ tools/ turnkey/ vm/ | grep '\.sh$' || true)
if [[ -z "$changed" ]]; then
echo "No .sh files changed in ct/, install/, tools/, turnkey/, or vm/."
echo "count=0" >> "$GITHUB_OUTPUT"
exit 0
fi
declare -A seen
slugs=""
for f in $changed; do
[[ -f "$f" ]] || continue
base="${f##*/}"
base="${base%.sh}"
if [[ "$f" == install/* && "$base" == *-install ]]; then
slug="${base%-install}"
else
slug="$base"
fi
if [[ -z "${seen[$slug]:-}" ]]; then
seen[$slug]=1
slugs="$slugs $slug"
fi
done
slugs=$(echo $slugs | xargs -n1 | sort -u)
if [[ -z "$slugs" ]]; then
echo "No slugs to update."
echo "count=0" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "$slugs" > changed_slugs.txt
echo "count=$(echo "$slugs" | wc -w)" >> "$GITHUB_OUTPUT"
- name: Parse PR number from merge commit
id: pr
run: |
re='#([0-9]+)'
if [[ "$COMMIT_MSG" =~ $re ]]; then
echo "number=${BASH_REMATCH[1]}" >> "$GITHUB_OUTPUT"
else
echo "number=" >> "$GITHUB_OUTPUT"
fi
env:
COMMIT_MSG: ${{ github.event.head_commit.message }}
- name: Update script timestamps in PocketBase
if: steps.slugs.outputs.count != '0'
env:
POCKETBASE_URL: ${{ vars.POCKETBASE_URL }}
POCKETBASE_COLLECTION: ${{ vars.POCKETBASE_COLLECTION }}
POCKETBASE_ADMIN_EMAIL: ${{ secrets.POCKETBASE_ADMIN_EMAIL }}
POCKETBASE_ADMIN_PASSWORD: ${{ secrets.POCKETBASE_ADMIN_PASSWORD }}
COMMIT_URL: ${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }}
PR_URL: ${{ steps.pr.outputs.number != '' && format('{0}/{1}/pull/{2}', github.server_url, github.repository, steps.pr.outputs.number) || '' }}
run: |
node << 'ENDSCRIPT'
(async function() {
const fs = require('fs');
const https = require('https');
const http = require('http');
const url = require('url');
function request(fullUrl, opts, redirectCount) {
redirectCount = redirectCount || 0;
return new Promise(function(resolve, reject) {
const u = url.parse(fullUrl);
const isHttps = u.protocol === 'https:';
const body = opts.body;
const options = {
hostname: u.hostname,
port: u.port || (isHttps ? 443 : 80),
path: u.path,
method: opts.method || 'GET',
headers: opts.headers || {}
};
if (body) options.headers['Content-Length'] = Buffer.byteLength(body);
const lib = isHttps ? https : http;
const req = lib.request(options, function(res) {
if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
if (redirectCount >= 5) return reject(new Error('Too many redirects from ' + fullUrl));
const redirectUrl = url.resolve(fullUrl, res.headers.location);
res.resume();
resolve(request(redirectUrl, opts, redirectCount + 1));
return;
}
let data = '';
res.on('data', function(chunk) { data += chunk; });
res.on('end', function() {
resolve({ ok: res.statusCode >= 200 && res.statusCode < 300, statusCode: res.statusCode, body: data });
});
});
req.on('error', reject);
if (body) req.write(body);
req.end();
});
}
const raw = process.env.POCKETBASE_URL.replace(/\/$/, '');
const apiBase = /\/api$/i.test(raw) ? raw : raw + '/api';
const coll = process.env.POCKETBASE_COLLECTION;
const slugsText = fs.readFileSync('changed_slugs.txt', 'utf8').trim();
const slugs = slugsText ? slugsText.split(/\s+/).filter(Boolean) : [];
if (slugs.length === 0) {
console.log('No slugs to update.');
return;
}
const authUrl = apiBase + '/collections/users/auth-with-password';
const authBody = JSON.stringify({
identity: process.env.POCKETBASE_ADMIN_EMAIL,
password: process.env.POCKETBASE_ADMIN_PASSWORD
});
const authRes = await request(authUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: authBody
});
if (!authRes.ok) {
throw new Error('Auth failed: ' + authRes.body);
}
const token = JSON.parse(authRes.body).token;
const recordsUrl = apiBase + '/collections/' + encodeURIComponent(coll) + '/records';
for (const slug of slugs) {
const filter = "(slug='" + slug.replace(/'/g, "''") + "')";
const listRes = await request(recordsUrl + '?filter=' + encodeURIComponent(filter) + '&perPage=1', {
headers: { 'Authorization': token }
});
const list = JSON.parse(listRes.body);
const record = list.items && list.items[0];
if (!record) {
console.log('Slug not in DB, skipping: ' + slug);
continue;
}
const today = new Date().toISOString().split('T')[0];
const patchBody = {
script_updated: today,
last_update_commit: process.env.PR_URL || process.env.COMMIT_URL || ''
};
// When a dev script is merged into main, promote it to production
if (record.is_dev === true) {
patchBody.is_dev = false;
patchBody.script_created = today;
console.log('Promoting dev script to production: ' + slug);
}
const patchRes = await request(recordsUrl + '/' + record.id, {
method: 'PATCH',
headers: { 'Authorization': token, 'Content-Type': 'application/json' },
body: JSON.stringify(patchBody)
});
if (!patchRes.ok) {
console.warn('PATCH failed for slug ' + slug + ': ' + patchRes.body);
continue;
}
console.log('Updated timestamp for slug: ' + slug);
}
console.log('Done.');
})().catch(e => { console.error(e); process.exit(1); });
ENDSCRIPT
shell: bash
+163 -71
View File
@@ -128,9 +128,6 @@ Exercise vigilance regarding copycat or coat-tailing sites that seek to exploit
@@ -144,7 +141,7 @@ Exercise vigilance regarding copycat or coat-tailing sites that seek to exploit
<details>
<summary><h4>November (14 entries)</h4></summary>
<summary><h4>November (7 entries)</h4></summary>
[View November 2026 Changelog](.github/changelogs/2026/11.md)
@@ -584,72 +581,6 @@ Exercise vigilance regarding copycat or coat-tailing sites that seek to exploit
</details>
## 2026-11-16
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- ArchiveBox: install libatomic1 for the Playwright driver [@MickLesk](https://github.com/MickLesk) ([#18722](https://github.com/community-scripts/ProxmoxVE/pull/18722))
- Ollama/OpenWebUI: download update before stopping the service [@jobben-2025](https://github.com/jobben-2025) ([#18755](https://github.com/community-scripts/ProxmoxVE/pull/18755))
- opengym: bump Node.js from 22 to 26 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18756](https://github.com/community-scripts/ProxmoxVE/pull/18756))
- razzia: bump Node.js from 24 to 26 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18757](https://github.com/community-scripts/ProxmoxVE/pull/18757))
- reactive-resume: bump Node.js from 24 to 26 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#18758](https://github.com/community-scripts/ProxmoxVE/pull/18758))
- #### 💥 Breaking Changes
- immich: reuse prebuilt image-processing libraries from the base image [@michelroegl-brunner](https://github.com/michelroegl-brunner) ([#18605](https://github.com/community-scripts/ProxmoxVE/pull/18605))
### 💾 Core
- Treat a closed update menu as cancel [@MickLesk](https://github.com/MickLesk) ([core#270](https://github.com/community-scripts/core/pull/270))
- forge: a .deb without service units is not an error [@MickLesk](https://github.com/MickLesk) ([core#269](https://github.com/community-scripts/core/pull/269))
## 2026-11-15
### 🆕 New Scripts
- DietPi ([#18746](https://github.com/community-scripts/ProxmoxVE/pull/18746))
- openlist ([#18747](https://github.com/community-scripts/ProxmoxVE/pull/18747))
- BlissOS VM ([#18745](https://github.com/community-scripts/ProxmoxVE/pull/18745))
- vLLM ([#18744](https://github.com/community-scripts/ProxmoxVE/pull/18744))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- Trilium: keep the installed version when an update download fails [@s31teg](https://github.com/s31teg) ([#18592](https://github.com/community-scripts/ProxmoxVE/pull/18592))
- several scripts: keep the port in redirects of PHP apps behind nginx on a custom port [@MickLesk](https://github.com/MickLesk) ([#18732](https://github.com/community-scripts/ProxmoxVE/pull/18732))
- NPMplus: offer the container's timezone as the default [@MickLesk](https://github.com/MickLesk) ([#18733](https://github.com/community-scripts/ProxmoxVE/pull/18733))
- #### ✨ New Features
- OpenCloud: bump to v9.0.0, configure the service account through env [@SimKaiLong](https://github.com/SimKaiLong) ([#18586](https://github.com/community-scripts/ProxmoxVE/pull/18586))
- #### 🔧 Refactor
- Nginx Proxy Manager: fall back to Alpine's distfiles for the OpenResty source [@MickLesk](https://github.com/MickLesk) ([#18736](https://github.com/community-scripts/ProxmoxVE/pull/18736))
### 💾 Core
- Keep the certificate CN within 64 characters [@MickLesk](https://github.com/MickLesk) ([core#267](https://github.com/community-scripts/core/pull/267))
- setup_rust: retry the rustup install [@MickLesk](https://github.com/MickLesk) ([core#266](https://github.com/community-scripts/core/pull/266))
- setup_nodejs: probe a module's binaries by its bin field [@MickLesk](https://github.com/MickLesk) ([core#265](https://github.com/community-scripts/core/pull/265))
## 2026-11-14
### 🆕 New Scripts
- LibreDB Studio ([#18699](https://github.com/community-scripts/ProxmoxVE/pull/18699))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- GoatCounter: let the singlefile download unpack the binary [@MickLesk](https://github.com/MickLesk) ([#18721](https://github.com/community-scripts/ProxmoxVE/pull/18721))
- Zerobyte: let the singlefile download unpack restic [@MickLesk](https://github.com/MickLesk) ([#18720](https://github.com/community-scripts/ProxmoxVE/pull/18720))
- LibreNMS: default to an 8 GB disk [@MickLesk](https://github.com/MickLesk) ([#18723](https://github.com/community-scripts/ProxmoxVE/pull/18723))
## 2026-11-13
### 🆕 New Scripts
@@ -1467,4 +1398,165 @@ Exercise vigilance regarding copycat or coat-tailing sites that seek to exploit
### 💾 Core
- Clean up after installs and updates again [@MickLesk](https://github.com/MickLesk) ([core#147](https://github.com/community-scripts/core/pull/147))
- Fix apt_conffile_report exiting 1 when uutils comm sees locale-sorted input [@BadFlo](https://github.com/BadFlo) ([core#146](https://github.com/community-scripts/core/pull/146))
- Fix apt_conffile_report exiting 1 when uutils comm sees locale-sorted input [@BadFlo](https://github.com/BadFlo) ([core#146](https://github.com/community-scripts/core/pull/146))
## 2026-10-15
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- fix(apprise-api): copy static assets into the served directory [@brokentokin](https://github.com/brokentokin) ([#17940](https://github.com/community-scripts/ProxmoxVE/pull/17940))
- Bytestash: modify backup process | Update Resources [@MickLesk](https://github.com/MickLesk) ([#17935](https://github.com/community-scripts/ProxmoxVE/pull/17935))
- Zabbix: Add '-y' flag to apt install commands [@MickLesk](https://github.com/MickLesk) ([#17933](https://github.com/community-scripts/ProxmoxVE/pull/17933))
- karakeep: bump Node.js from 22 to 24 [@github-actions[bot]](https://github.com/github-actions[bot]) ([#17087](https://github.com/community-scripts/ProxmoxVE/pull/17087))
### 💾 Core
- Keep VM release lookups working without system.func [@MickLesk](https://github.com/MickLesk) ([core#144](https://github.com/community-scripts/core/pull/144))
- Stop calling exit 1 "Operation not permitted" at several steps [@MickLesk](https://github.com/MickLesk) ([core#140](https://github.com/community-scripts/core/pull/140))
## 2026-10-14
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- Paperclip: fall back to a non-frozen install when the release lockfile is stale [@MickLesk](https://github.com/MickLesk) ([#17922](https://github.com/community-scripts/ProxmoxVE/pull/17922))
- #### 🔧 Refactor
- Immich: rebuild the image libraries when the revision record is missing [@MickLesk](https://github.com/MickLesk) ([#17919](https://github.com/community-scripts/ProxmoxVE/pull/17919))
### 💾 Core
- Show what the failing command printed, and keep verbose output in the log [@MickLesk](https://github.com/MickLesk) ([core#139](https://github.com/community-scripts/core/pull/139))
- Install jq before the release lookup and show what a non-JSON answer was [@MickLesk](https://github.com/MickLesk) ([core#138](https://github.com/community-scripts/core/pull/138))
- Refresh the NodeSource key before a same-major Node.js update [@MickLesk](https://github.com/MickLesk) ([core#137](https://github.com/community-scripts/core/pull/137))
- github: say why a forge could not be reached instead of 'connection failed' [@MickLesk](https://github.com/MickLesk) ([core#136](https://github.com/community-scripts/core/pull/136))
## 2026-10-13
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- Fix URL for downloading OneDev latest release [@ryansully](https://github.com/ryansully) ([#17915](https://github.com/community-scripts/ProxmoxVE/pull/17915))
- LocalAI: pin the CPU backends when the container has no GPU runtime [@MickLesk](https://github.com/MickLesk) ([#17909](https://github.com/community-scripts/ProxmoxVE/pull/17909))
- [Gluetun] Update GitHub repository source install and update. [@CerberusStyle](https://github.com/CerberusStyle) ([#17899](https://github.com/community-scripts/ProxmoxVE/pull/17899))
- Immich: Bump to 3.3.1 and restore (failed) geodata/countryInfo.txt on update [@MickLesk](https://github.com/MickLesk) ([#17894](https://github.com/community-scripts/ProxmoxVE/pull/17894))
- WikiJS: use v2 Repo until v3 is stable [@MickLesk](https://github.com/MickLesk) ([#17895](https://github.com/community-scripts/ProxmoxVE/pull/17895))
- #### 💥 Breaking Changes
- zammad: Debian 13, new apt repo, Elasticsearch 8 [@durzo](https://github.com/durzo) ([#17836](https://github.com/community-scripts/ProxmoxVE/pull/17836))
### 💾 Core
- Keep the version key parseable by jq 1.6 [@MickLesk](https://github.com/MickLesk) ([core#134](https://github.com/community-scripts/core/pull/134))
## 2026-10-12
### 🆕 New Scripts
- WatchState ([#17881](https://github.com/community-scripts/ProxmoxVE/pull/17881))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- chore(ct): sync wazuh defaults with PocketBase [@github-actions[bot]](https://github.com/github-actions[bot]) ([#17889](https://github.com/community-scripts/ProxmoxVE/pull/17889))
- Wanderer: bring back the wanderer-pb wrapper [@MickLesk](https://github.com/MickLesk) ([#17874](https://github.com/community-scripts/ProxmoxVE/pull/17874))
- Tracearr: install only the server's production dependencies [@MickLesk](https://github.com/MickLesk) ([#17873](https://github.com/community-scripts/ProxmoxVE/pull/17873))
- Paperless-ngx: QoL restart crashed services | patch regex [@MickLesk](https://github.com/MickLesk) ([#17812](https://github.com/community-scripts/ProxmoxVE/pull/17812))
- Splunk Enterprise: start the service as the splunk user [@rcastley](https://github.com/rcastley) ([#17816](https://github.com/community-scripts/ProxmoxVE/pull/17816))
### 💾 Core
- Draw a Community Scripts banner while a script has no header of its own [@MickLesk](https://github.com/MickLesk) ([core#133](https://github.com/community-scripts/core/pull/133))
- Order pre-release labels when comparing versions [@MickLesk](https://github.com/MickLesk) ([core#132](https://github.com/community-scripts/core/pull/132))
- gh: curl_download: wait between attempts [@MickLesk](https://github.com/MickLesk) ([core#131](https://github.com/community-scripts/core/pull/131))
### 🧰 Tools
- #### ✨ New Features
- Immich Public Proxy: optional visitor upload service (v4 Release) [@MickLesk](https://github.com/MickLesk) ([#17875](https://github.com/community-scripts/ProxmoxVE/pull/17875))
## 2026-10-11
### 🆕 New Scripts
- Cinephage ([#17852](https://github.com/community-scripts/ProxmoxVE/pull/17852))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- Immich: build libvips with JPEG 2000 support [@MickLesk](https://github.com/MickLesk) ([#17864](https://github.com/community-scripts/ProxmoxVE/pull/17864))
- hermesagent: match the relaunched root gateway regardless of interpreter [@MickLesk](https://github.com/MickLesk) ([#17847](https://github.com/community-scripts/ProxmoxVE/pull/17847))
- Radicale: install the bcrypt and argon2 extras [@MickLesk](https://github.com/MickLesk) ([#17813](https://github.com/community-scripts/ProxmoxVE/pull/17813))
- fix(zigbee2mqtt): preserve backup stream and prevent filename collisions [@petermnt](https://github.com/petermnt) ([#17751](https://github.com/community-scripts/ProxmoxVE/pull/17751))
- #### ✨ New Features
- QoL: let var_* answer the optional add-on prompts [@MickLesk](https://github.com/MickLesk) ([#17858](https://github.com/community-scripts/ProxmoxVE/pull/17858))
- QoL: Confirm third-party installers through confirm_external_installer [@MickLesk](https://github.com/MickLesk) ([#17857](https://github.com/community-scripts/ProxmoxVE/pull/17857))
- docker: take the lxcfs toggle, TCP socket and Compose choice from app variables [@MickLesk](https://github.com/MickLesk) ([#17850](https://github.com/community-scripts/ProxmoxVE/pull/17850))
- #### 🔧 Refactor
- RustDesk Server: run the official server, make the web UI optional [@MickLesk](https://github.com/MickLesk) ([#17854](https://github.com/community-scripts/ProxmoxVE/pull/17854))
- Caddy: install the .deb from GitHub instead of the Cloudsmith repo [@MickLesk](https://github.com/MickLesk) ([#17862](https://github.com/community-scripts/ProxmoxVE/pull/17862))
### 💾 Core
- Add confirm_external_installer [@MickLesk](https://github.com/MickLesk) ([core#130](https://github.com/community-scripts/core/pull/130))
- setup_docker: make LXC resource limits visible to nested containers [@andrebrait](https://github.com/andrebrait) ([core#9](https://github.com/community-scripts/core/pull/9))
- Incus: fix ENABLE_FUSE / TUN vars [@MickLesk](https://github.com/MickLesk) ([core#127](https://github.com/community-scripts/core/pull/127))
- Resolve a uv required-version range to the newest release inside it [@MickLesk](https://github.com/MickLesk) ([core#126](https://github.com/community-scripts/core/pull/126))
- Upgrade MongoDB in place when an app package depends on it [@MickLesk](https://github.com/MickLesk) ([core#128](https://github.com/community-scripts/core/pull/128))
## 2026-10-10
### 🆕 New Scripts
- Weblate ([#17837](https://github.com/community-scripts/ProxmoxVE/pull/17837))
- CertMate ([#17803](https://github.com/community-scripts/ProxmoxVE/pull/17803))
- AnythingLLM ([#17802](https://github.com/community-scripts/ProxmoxVE/pull/17802))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- update endurain repo from codeberg to gh [@johanngrobe](https://github.com/johanngrobe) ([#17831](https://github.com/community-scripts/ProxmoxVE/pull/17831))
- Immich: download countryInfo.txt into the geodata directory on update [@claytonfaria](https://github.com/claytonfaria) ([#17823](https://github.com/community-scripts/ProxmoxVE/pull/17823))
- paperclip: keep root and skip /opt/paperclip-data chown when PAPERCLIP_HOME is custom [@austinpilz](https://github.com/austinpilz) ([#17825](https://github.com/community-scripts/ProxmoxVE/pull/17825))
## 2026-10-09
### 🆕 New Scripts
- FoldingAtHome ([#17799](https://github.com/community-scripts/ProxmoxVE/pull/17799))
- LocalAI ([#17801](https://github.com/community-scripts/ProxmoxVE/pull/17801))
- Tvheadend ([#17800](https://github.com/community-scripts/ProxmoxVE/pull/17800))
### 🚀 Updated Scripts
- #### 🐞 Bug Fixes
- OpenCloud: fix Collabora embedding with Collabora 26.04.4 [@SimKaiLong](https://github.com/SimKaiLong) ([#17810](https://github.com/community-scripts/ProxmoxVE/pull/17810))
- #### 🔧 Refactor
- Immich: survive an interrupted update and a failing ML build [@MickLesk](https://github.com/MickLesk) ([#17798](https://github.com/community-scripts/ProxmoxVE/pull/17798))
### 💾 Core
- check_for_release: compare against the release marked as latest [@MickLesk](https://github.com/MickLesk) ([core#125](https://github.com/community-scripts/core/pull/125))
### 🧰 Tools
- #### ✨ New Features
- FileBrowser Quantum: migrate the config and database to v2 [@MickLesk](https://github.com/MickLesk) ([#17815](https://github.com/community-scripts/ProxmoxVE/pull/17815))
-56
View File
@@ -1,56 +0,0 @@
#!/usr/bin/env bash
# Engine comes from community-scripts/core; this repo only ships the scripts.
# Local checkout wins (COMMUNITY_SCRIPTS_CORE_DIR, else a sibling ../core), so a
# fork/branch of core can be tested without touching this file.
_cs_boot="${COMMUNITY_SCRIPTS_CORE_DIR:-$(dirname "${BASH_SOURCE[0]}")/../../core}/core/build.func"
source "$_cs_boot" 2>/dev/null || source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/core/build.func")
# Copyright (c) 2021-2026 community-scripts ORG
# Author: mews-se
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://dietpi.com/
APP="DietPi"
var_tags="${var_tags:-os}"
var_cpu="${var_cpu:-1}"
var_ram="${var_ram:-512}"
var_disk="${var_disk:-4}"
var_os="${var_os:-debian}"
var_version="${var_version:-13}"
#var_arm64="${var_arm64:-no}" # unset = ask the user; set yes/no only when verified
var_unprivileged="${var_unprivileged:-1}"
header_info "$APP"
variables
color
catch_errors
function update_script() {
header_info
check_container_storage
check_container_resources
if [[ ! -f /boot/dietpi/.version ]]; then
msg_error "No ${APP} Installation Found!"
exit
fi
msg_info "Updating DietPi LXC"
$STD /boot/dietpi/dietpi-update 1
msg_ok "Updated DietPi LXC"
exit
}
start
build_container
description
msg_info "Restarting ${APP} for DietPi's first boot"
if command -v pct &>/dev/null; then
pct reboot "$CTID"
else
incus restart "$CTID"
fi
msg_ok "Restarted ${APP}"
msg_ok "Completed successfully!"
msg_custom "🚀" "${GN}" "${APP} setup has been successfully initialized!"
echo -e "${INFO}${YW} DietPi's first run setup starts at the next console login.${CL}"
+2
View File
@@ -41,6 +41,8 @@ function update_script() {
msg_ok "Stopped GoatCounter"
fetch_and_deploy_gh_release "goatcounter" "arp242/goatcounter" "singlefile" "latest" "/opt/goatcounter" "goatcounter-v*-linux-$(arch_resolve).gz"
mv /opt/goatcounter/goatcounter /opt/goatcounter/goatcounter.gz
gunzip -f /opt/goatcounter/goatcounter.gz
msg_info "Starting GoatCounter"
systemctl start goatcounter
+204 -69
View File
@@ -38,20 +38,12 @@ function update_script() {
exit
fi
if ! grep -qsE '^[[:space:]]*Suites:.*(^|[[:space:]])testing([[:space:]]|$)' /etc/apt/sources.list.d/*.sources; then
if ! grep -qE '(^|[[:space:]])testing([[:space:]]|$)' /etc/apt/sources.list.d/debian.sources 2>/dev/null; then
msg_info "Adding Debian Testing repo"
if grep -q "trixie-updates" /etc/apt/sources.list.d/debian.sources 2>/dev/null; then
sed -i 's/ trixie-updates/ trixie-updates testing/g' /etc/apt/sources.list.d/debian.sources
elif grep -qsE '^[[:space:]]*Suites:.*trixie' /etc/apt/sources.list.d/debian.sources; then
sed -i '/^[[:space:]]*Suites:.*trixie/ s/$/ testing/' /etc/apt/sources.list.d/debian.sources
else
cat <<EOF >/etc/apt/sources.list.d/testing.sources
Types: deb
URIs: http://deb.debian.org/debian
Suites: testing
Components: main
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
EOF
sed -i '/^[[:space:]]*Suites:.*trixie/ s/$/ testing/' /etc/apt/sources.list.d/debian.sources
fi
cat <<EOF >/etc/apt/preferences.d/preferences
Package: *
@@ -79,6 +71,9 @@ EOF
fetch_and_deploy_gh_release "mise" "jdx/mise" "singlefile" "latest" "/usr/local/bin" "mise-v*-linux-${MISE_ARCH}"
fi
STAGING_DIR=/opt/staging
BASE_DIR=${STAGING_DIR}/base-images
SOURCE_DIR=${STAGING_DIR}/image-source
cd /tmp
RELEASE="v3.4.1"
if [[ -f ~/.intel_version ]]; then
@@ -105,8 +100,21 @@ EOF
msg_ok "Updated Intel OpenVINO dependencies"
fi
fi
# Libraries the prebuilt image-processing libraries link against; pigz compresses database backups.
ensure_dependencies liblcms2-dev libjpeg62-turbo-dev libspng-dev libexif-dev libopenjp2-7-dev libaom-dev pigz
# Without the revision file the refresh was skipped and sharp failed on an old libvips (#17914).
[[ -f ~/.immich_library_revisions ]] || msg_warn "No record of the compiled image-processing libraries, rebuilding all of them"
for lib in imagemagick jpegli libheif libjxl libraw libvips; do
grep -q "^${lib}:" ~/.immich_library_revisions 2>/dev/null || echo "${lib}: unknown" >>~/.immich_library_revisions
done
[[ -d "$BASE_DIR/.git" ]] || $STD git clone -b main https://github.com/immich-app/base-images "$BASE_DIR"
libraries=("libjxl" "jpegli" "libheif" "libraw" "imagemagick" "libvips")
cd "$BASE_DIR"
msg_warn "Checking for updates to custom image-processing libraries (recompile time: 2-15min per library)"
ensure_dependencies liblcms2-dev libjpeg62-turbo-dev libspng-dev libexif-dev libopenjp2-7-dev pigz
$STD git pull
for library in "${libraries[@]}"; do
compile_"$library"
done
msg_ok "Image-processing libraries up to date"
# 3.3.0 reads geodata/countryInfo.txt; the update before #17823 saved it into app/ instead.
if [[ -d /opt/immich/geodata && ! -s /opt/immich/geodata/countryInfo.txt ]]; then
@@ -123,7 +131,6 @@ EOF
if check_for_gh_release "Immich" "immich-app/immich" "${RELEASE}" "each release is tested individually before the version is updated. Please do not open issues for this"; then
msg_warn "The update takes 5-15 minutes, do not interrupt it"
fetch_prebuilt_libs
PREV_IMMICH="${CS_PREVIOUS_VERSION:-$(cat ~/.immich)}"
INSTALL_DIR="/opt/${APP}"
UPLOAD_DIR="$(sed -n '/^IMMICH_MEDIA_LOCATION/s/[^=]*=//p' /opt/immich/.env)"
@@ -150,7 +157,6 @@ EOF
systemctl stop immich-web
systemctl stop immich-ml
msg_ok "Stopped Services"
install_prebuilt_libs
VCHORD_RELEASE="1.1.1"
PG_VERSION=$(ls /etc/postgresql/ 2>/dev/null | sort -V | tail -1)
PG_VERSION=${PG_VERSION:-16}
@@ -168,6 +174,7 @@ EOF
$STD sudo -u postgres psql -d immich -c "REINDEX INDEX face_index;"
$STD sudo -u postgres psql -d immich -c "REINDEX INDEX clip_index;"
fi
ensure_dependencies ccache gcc-13 g++-13
[[ -f "$ML_DIR"/ml_start.sh ]] && cp "$ML_DIR"/ml_start.sh "$INSTALL_DIR"
if grep -qs "set -a" "$APP_DIR"/bin/start.sh && grep -qs "warnings" "$APP_DIR"/bin/start.sh; then
@@ -427,65 +434,193 @@ EOF
exit
}
# Immich ships the custom photo-processing libraries (libjxl, jpegli, libheif,
# libraw, ImageMagick, libvips) prebuilt in its own OCI base image; the release's
# Dockerfile pins it. fetch_prebuilt_libs downloads and unpacks it while Immich
# still runs, so a failure leaves the install as it was. install_prebuilt_libs
# copies it into /usr/local once the services are stopped.
function fetch_prebuilt_libs() {
PREBUILT_BUNDLE=""
BASE_REF="$(curl -fsSL "https://raw.githubusercontent.com/immich-app/immich/${RELEASE}/server/Dockerfile" | awk '/^FROM ghcr.io\/immich-app\/base-server-dev/ {print $2; exit}')" || true
if [[ -z "$BASE_REF" ]]; then
msg_error "Could not read the base image from Immich ${RELEASE}'s Dockerfile. Immich was not updated."
exit 115
function compile_libjxl() {
SOURCE=${SOURCE_DIR}/libjxl
LIBJXL_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libjxl.json)"
if [[ "$LIBJXL_REVISION" != "$(grep 'libjxl' ~/.immich_library_revisions | awk '{print $2}')" ]]; then
msg_info "Recompiling libjxl"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/libjxl/libjxl.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBJXL_REVISION"
$STD git submodule update --init --recursive --depth 1 --recommend-shallow -- third_party
mkdir build
cd build
$STD cmake \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TESTING=OFF \
-DJPEGXL_ENABLE_DOXYGEN=OFF \
-DJPEGXL_ENABLE_MANPAGES=OFF \
-DJPEGXL_ENABLE_BENCHMARK=OFF \
-DJPEGXL_ENABLE_EXAMPLES=OFF \
-DJPEGXL_FORCE_SYSTEM_BROTLI=ON \
-DJPEGXL_FORCE_SYSTEM_HWY=ON \
-DJPEGXL_ENABLE_HWY_AVX3=ON \
-DJPEGXL_ENABLE_HWY_AVX3_ZEN4=ON \
-DJPEGXL_ENABLE_HWY_SVE=OFF \
-DJPEGXL_ENABLE_HWY_SVE2=OFF \
-DJPEGXL_ENABLE_HWY_SVE2_128=ON \
-DJPEGXL_ENABLE_PLUGINS=ON \
..
$STD cmake --build . -- -j"$(nproc)"
$STD cmake --install .
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/{build,third_party}
sed -i "s/libjxl: .*$/libjxl: $LIBJXL_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled libjxl"
fi
[[ "$(cat ~/.immich_base_image 2>/dev/null)" == "$BASE_REF" ]] && return 0
msg_info "Fetching prebuilt image-processing libraries"
# skopeo refuses a reference with both a tag and a digest; the digest is the pin.
local image="$BASE_REF"
[[ "$BASE_REF" == *@sha256:* ]] && image="ghcr.io/immich-app/base-server-dev@${BASE_REF#*@}"
if ! { command -v skopeo && command -v umoci; } >/dev/null 2>&1 &&
! $STD apt install -y --no-install-recommends skopeo umoci; then
msg_error "Could not install skopeo and umoci, which Immich needs for its image-processing libraries. Immich was not updated."
exit 105
fi
local oci_dir
oci_dir="$(mktemp -d)"
PREBUILT_BUNDLE="$(mktemp -d)"
if ! $STD skopeo copy --override-os linux --override-arch "$(arch_resolve amd64 arm64)" "docker://${image}" "oci:${oci_dir}:img"; then
rm -rf "$oci_dir" "$PREBUILT_BUNDLE"
msg_error "Could not download Immich's base image (${image}). Immich was not updated."
exit 115
fi
if ! $STD umoci unpack --image "${oci_dir}:img" "$PREBUILT_BUNDLE"; then
rm -rf "$oci_dir" "$PREBUILT_BUNDLE"
msg_error "Could not unpack Immich's base image. Immich needs Proxmox VE 9.1 or newer; it was not updated."
exit 105
fi
rm -rf "$oci_dir"
msg_ok "Fetched prebuilt image-processing libraries"
}
function install_prebuilt_libs() {
[[ -n "${PREBUILT_BUNDLE:-}" ]] || return 0
msg_info "Installing prebuilt image-processing libraries"
# The libraries, headers and ImageMagick data, without the node runtime, global
# node modules and ffmpeg of the node base image: this script provides those.
tar -C "${PREBUILT_BUNDLE}/rootfs/usr/local" \
--exclude=./bin/node --exclude=./bin/npm --exclude=./bin/npx --exclude=./bin/corepack \
--exclude=./bin/ffmpeg --exclude=./bin/ffprobe \
--exclude=./lib/node_modules --exclude=./include/node \
-cf - . | tar -C /usr/local -xf -
rm -rf "$PREBUILT_BUNDLE"
ldconfig /usr/local/lib
if ! vips --version >/dev/null 2>&1 || ! pkg-config --exists vips; then
msg_error "The prebuilt libvips from ${BASE_REF} does not work in this container."
exit 1
function compile_jpegli() {
SOURCE=${SOURCE_DIR}/jpegli
JPEGLI_LIBJPEG_LIBRARY_SOVERSION="62"
JPEGLI_LIBJPEG_LIBRARY_VERSION="62.3.0"
JPEGLI_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/jpegli.json)"
if [[ "$JPEGLI_REVISION" != "$(grep 'jpegli' ~/.immich_library_revisions | awk '{print $2}')" ]]; then
msg_info "Recompiling jpegli"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/google/jpegli.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$JPEGLI_REVISION"
$STD git submodule update --init --depth 1 --recommend-shallow third_party/libjpeg-turbo
$STD git apply -3 "$BASE_DIR"/server/sources/jpegli-patches/jpegli-empty-dht-marker.patch
$STD git apply -3 "$BASE_DIR"/server/sources/jpegli-patches/jpegli-icc-warning.patch
mkdir build
cd build
$STD cmake \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TESTING=OFF \
-DJPEGLI_ENABLE_DOXYGEN=OFF \
-DJPEGLI_ENABLE_MANPAGES=OFF \
-DJPEGLI_ENABLE_BENCHMARK=OFF \
-DJPEGLI_ENABLE_TOOLS=OFF \
-DJPEGLI_ENABLE_DEVTOOLS=OFF \
-DJPEGLI_ENABLE_FUZZERS=OFF \
-DJPEGLI_ENABLE_JNI=OFF \
-DJPEGLI_ENABLE_OPENEXR=OFF \
-DJPEGLI_ENABLE_SJPEG=OFF \
-DJPEGLI_ENABLE_SKCMS=OFF \
-DJPEGLI_FORCE_SYSTEM_HWY=ON \
-DJPEGLI_FORCE_SYSTEM_LCMS2=ON \
-DJPEGLI_ENABLE_JPEGLI_LIBJPEG=ON \
-DJPEGLI_INSTALL_JPEGLI_LIBJPEG=ON \
-DJPEGLI_ENABLE_HWY_AVX3=ON \
-DJPEGLI_ENABLE_HWY_AVX3_ZEN4=ON \
-DJPEGLI_ENABLE_HWY_SVE=OFF \
-DJPEGLI_ENABLE_HWY_SVE2=OFF \
-DJPEGLI_ENABLE_HWY_SVE2_128=ON \
-DJPEGLI_LIBJPEG_LIBRARY_SOVERSION="$JPEGLI_LIBJPEG_LIBRARY_SOVERSION" \
-DJPEGLI_LIBJPEG_LIBRARY_VERSION="$JPEGLI_LIBJPEG_LIBRARY_VERSION" \
-DLIBJPEG_TURBO_VERSION_NUMBER=2001005 \
..
$STD cmake --build . -- -j"$(nproc)"
$STD cmake --install .
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/{build,third_party}
sed -i "s/jpegli: .*$/jpegli: $JPEGLI_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled jpegli"
fi
}
function compile_libheif() {
SOURCE=${SOURCE_DIR}/libheif
ensure_dependencies libaom-dev
LIBHEIF_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libheif.json)"
if [[ "${update:-}" ]] || [[ "$LIBHEIF_REVISION" != "$(grep 'libheif' ~/.immich_library_revisions | awk '{print $2}')" ]]; then
msg_info "Recompiling libheif"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/strukturag/libheif.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBHEIF_REVISION"
mkdir build
cd build
$STD cmake --preset=release-noplugins \
-DWITH_DAV1D=ON \
-DENABLE_PARALLEL_TILE_DECODING=ON \
-DWITH_LIBSHARPYUV=ON \
-DWITH_LIBDE265=ON \
-DWITH_AOM_DECODER=OFF \
-DWITH_AOM_ENCODER=ON \
-DWITH_X265=OFF \
-DWITH_EXAMPLES=OFF \
..
$STD make install -j"$(nproc)"
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/build
sed -i "s/libheif: .*$/libheif: $LIBHEIF_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled libheif"
fi
}
function compile_libraw() {
SOURCE=${SOURCE_DIR}/libraw
LIBRAW_REVISION="e419de08001de28ae6988ecb22df47e52b9c5eaa"
# : "${LIBRAW_REVISION:=$(jq -cr '.revision' "$BASE_DIR"/server/sources/libraw.json)}"
if [[ "$LIBRAW_REVISION" != "$(grep 'libraw' ~/.immich_library_revisions | awk '{print $2}')" ]]; then
msg_info "Recompiling libraw"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/LibRaw/LibRaw.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBRAW_REVISION"
$STD autoreconf --install
$STD ./configure --disable-examples
$STD make -j"$(nproc)"
$STD make install
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
sed -i "s/libraw: .*$/libraw: $LIBRAW_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled libraw"
fi
}
function compile_imagemagick() {
SOURCE=$SOURCE_DIR/imagemagick
: "${IMAGEMAGICK_REVISION:=$(jq -cr '.revision' "$BASE_DIR"/server/sources/imagemagick.json)}"
if [[ "$IMAGEMAGICK_REVISION" != "$(grep 'imagemagick' ~/.immich_library_revisions | awk '{print $2}')" ]] ||
! grep -q 'DMAGICK_LIBRAW' /usr/local/lib/ImageMagick-7*/config-Q16HDRI/configure.xml; then
msg_info "Recompiling ImageMagick"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/ImageMagick/ImageMagick.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$IMAGEMAGICK_REVISION"
$STD ./configure --with-modules CPPFLAGS="-DMAGICK_LIBRAW_VERSION_TAIL=202502"
$STD make -j"$(nproc)"
$STD make install
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
sed -i "s/imagemagick: .*$/imagemagick: $IMAGEMAGICK_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled ImageMagick"
fi
}
function compile_libvips() {
SOURCE=$SOURCE_DIR/libvips
LIBVIPS_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libvips.json)"
if [[ "$LIBVIPS_REVISION" != "$(grep 'libvips' ~/.immich_library_revisions | awk '{print $2}')" ]] ||
[[ "$(vips -l foreign 2>/dev/null)" != *jp2kload* ]]; then
msg_info "Recompiling libvips"
[[ -d "$SOURCE" ]] && rm -rf "$SOURCE"
$STD git clone https://github.com/libvips/libvips.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBVIPS_REVISION"
$STD git apply "$BASE_DIR"/server/sources/libvips-patches/0001-put-other-loaders-ahead-of-dcrawload.patch
$STD meson setup build --buildtype=release --libdir=lib -Dintrospection=disabled -Dtiff=disabled
cd build
$STD ninja install
ldconfig /usr/local/lib
cd "$STAGING_DIR"
rm -rf "$SOURCE"/build
sed -i "s/libvips: .*$/libvips: $LIBVIPS_REVISION/" ~/.immich_library_revisions
msg_ok "Recompiled libvips"
fi
echo "$BASE_REF" >~/.immich_base_image
# Leftovers of the libraries this script used to compile.
rm -rf /opt/staging/base-images /opt/staging/image-source ~/.immich_library_revisions
msg_ok "Installed prebuilt image-processing libraries"
}
start
-61
View File
@@ -1,61 +0,0 @@
#!/usr/bin/env bash
_cs_boot="${COMMUNITY_SCRIPTS_CORE_DIR:-$(dirname "${BASH_SOURCE[0]}")/../../core}/core/build.func"
source "$_cs_boot" 2>/dev/null || source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/core/build.func")
# Copyright (c) 2021-2026 community-scripts ORG
# Author: Yusuf Gundogdu (yusuf-gundogdu)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://libredb.org
APP="LibreDB Studio"
var_tags="${var_tags:-database;sql}"
var_cpu="${var_cpu:-2}"
var_ram="${var_ram:-2048}"
var_disk="${var_disk:-6}"
var_os="${var_os:-debian}"
var_version="${var_version:-13}"
var_arm64="${var_arm64:-yes}"
var_unprivileged="${var_unprivileged:-1}"
header_info "$APP"
variables
color
catch_errors
function update_script() {
header_info
check_container_storage
check_container_resources
if [[ ! -f /usr/bin/libredb-studio ]]; then
msg_error "No ${APP} Installation Found!"
exit
fi
if check_for_gh_release "libredb-studio" "libredb/libredb-studio"; then
msg_info "Stopping Service"
systemctl stop libredb-studio
msg_ok "Stopped Service"
create_backup /etc/libredb-studio/env
DPKG_FORCE_CONFOLD=1 fetch_and_deploy_gh_release "libredb-studio" "libredb/libredb-studio" "binary" "latest" "" "libredb-studio_*_$(arch_resolve).deb"
restore_backup
msg_info "Starting Service"
systemctl start libredb-studio
msg_ok "Started Service"
msg_ok "Updated successfully!"
fi
exit
}
start
build_container
description
msg_ok "Completed Successfully!\n"
echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}"
echo -e "${INFO}${YW}Access it using the following URL:${CL}"
echo -e "${GATEWAY}${BGN}http://${IP}:3000${CL}"
echo -e "${INFO}${YW}Admin credentials: /etc/libredb-studio/env inside the container${CL}"
+1 -1
View File
@@ -10,7 +10,7 @@ APP="LibreNMS"
var_tags="${var_tags:-monitoring}"
var_cpu="${var_cpu:-2}"
var_ram="${var_ram:-2048}"
var_disk="${var_disk:-8}"
var_disk="${var_disk:-4}"
var_os="${var_os:-debian}"
var_version="${var_version:-13}"
var_arm64="${var_arm64:-yes}"
+1 -2
View File
@@ -63,8 +63,7 @@ function update_script() {
OPENRESTY_VERSION="1.29.2.5"
if [[ "$(cat ~/.openresty 2>/dev/null)" != "$OPENRESTY_VERSION" ]]; then
CLEAN_INSTALL=1 fetch_and_deploy_from_url "https://openresty.org/download/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty" ||
CLEAN_INSTALL=1 fetch_and_deploy_from_url "https://distfiles.alpinelinux.org/distfiles/edge/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty"
CLEAN_INSTALL=1 fetch_and_deploy_from_url "https://openresty.org/download/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty"
msg_info "Building OpenResty"
cd /opt/openresty
+7 -14
View File
@@ -34,25 +34,18 @@ function update_script() {
if check_for_gh_release "ollama-com" "ollama/ollama"; then
ensure_dependencies zstd
OLLAMA_INSTALL_DIR="/usr/local/lib/ollama"
OLLAMA_STAGING_DIR="${OLLAMA_INSTALL_DIR}.new"
rm -rf "$OLLAMA_STAGING_DIR"
mkdir -p "$OLLAMA_STAGING_DIR"
if ! fetch_and_deploy_gh_release "ollama-com" "ollama/ollama" "prebuild" "latest" "$OLLAMA_STAGING_DIR" "ollama-linux-$(arch_resolve).tar.zst"; then
rm -rf "$OLLAMA_STAGING_DIR"
msg_error "Download or deployment failed – check network connectivity and GitHub API availability (installed Ollama left unchanged)"
exit 250
fi
msg_info "Stopping Services"
systemctl stop ollama
msg_ok "Services Stopped"
rm -rf "${OLLAMA_INSTALL_DIR}.old"
mv "$OLLAMA_INSTALL_DIR" "${OLLAMA_INSTALL_DIR}.old"
mv "$OLLAMA_STAGING_DIR" "$OLLAMA_INSTALL_DIR"
OLLAMA_INSTALL_DIR="/usr/local/lib/ollama"
rm -rf "$OLLAMA_INSTALL_DIR" /usr/local/bin/ollama
mkdir -p "$OLLAMA_INSTALL_DIR"
if ! fetch_and_deploy_gh_release "ollama-com" "ollama/ollama" "prebuild" "latest" "$OLLAMA_INSTALL_DIR" "ollama-linux-$(arch_resolve).tar.zst"; then
msg_error "Download or deployment failed – check network connectivity and GitHub API availability"
exit 250
fi
ln -sf "$OLLAMA_INSTALL_DIR/bin/ollama" /usr/local/bin/ollama
rm -rf "${OLLAMA_INSTALL_DIR}.old"
msg_ok "Updated Ollama"
msg_info "Starting Services"
+8 -23
View File
@@ -40,7 +40,7 @@ function update_script() {
msg_ok "Allowed OpenCloud to embed Collabora"
fi
RELEASE="v9.0.0"
RELEASE="v8.1.0"
if check_for_gh_release "OpenCloud" "opencloud-eu/opencloud" "${RELEASE}" "each release is tested individually before the version is updated. Please do not open issues for this"; then
OLD_VERSION="$(cat ~/.opencloud 2>/dev/null)"
msg_info "Stopping services"
@@ -68,30 +68,15 @@ function update_script() {
# STORAGE_USERS_POSIX_WATCH_PATH=<path-to-storage-or-bind-mount>' /etc/opencloud/opencloud.env
fi
# Every OpenCloud service falls back to OC_SERVICE_ACCOUNT_ID/SECRET, so services that newly
# need the service account (sharing in 8.0, auth_guest in 9.0) work without editing opencloud.yaml
if ! grep -q '^OC_SERVICE_ACCOUNT_ID=' /etc/opencloud/opencloud.env; then
ACCOUNT_ID="$(awk '/service_account_id:/ {gsub(/["'\'']/, "", $2); print $2; exit}' /etc/opencloud/opencloud.yaml)"
ACCOUNT_SECRET="$(awk '/service_account_secret:/ {sub(/^[^:]*:[[:space:]]*/, ""); gsub(/^["'\'']|["'\'']$/, ""); print; exit}' /etc/opencloud/opencloud.yaml)"
if [[ -n "$ACCOUNT_ID" && -n "$ACCOUNT_SECRET" && "$ACCOUNT_SECRET" != *"'"* ]]; then
printf "OC_SERVICE_ACCOUNT_ID=%s\nOC_SERVICE_ACCOUNT_SECRET='%s'\n" "$ACCOUNT_ID" "$ACCOUNT_SECRET" |
sed -i '/^STORAGE_SYSTEM_OC_ROOT=/r /dev/stdin' /etc/opencloud/opencloud.env
else
msg_warn "Could not read the service account from opencloud.yaml; set OC_SERVICE_ACCOUNT_ID/SECRET in opencloud.env"
fi
if ! sed -n '/^sharing:/,/^storage_users:/p' /etc/opencloud/opencloud.yaml | grep -q 'service_account'; then
ACCOUNT_ID="$(sed -n '/^activitylog:/,/*.$/p' /etc/opencloud/opencloud.yaml | awk -F'id:' '{print $2}' | tr -d '[:space:]')"
ACCOUNT_SECRET="$(sed -n '/^activitylog:/,/*.$/p' /etc/opencloud/opencloud.yaml | awk -F'secret:' '{print $2}' | tr -d '[:space:]')"
sed -i "/^sharing:/a\\
service_account:\\
service_account_id: $ACCOUNT_ID\\
service_account_secret: $ACCOUNT_SECRET" /etc/opencloud/opencloud.yaml
fi
if ! grep -q 'OC_ENABLE_GUEST_LINKS' /etc/opencloud/opencloud.env; then
sed -i '/^## Start additional services/i ## Guest links - share with external people by email\
## Requires the SMTP settings above and notifications in OC_ADD_RUN_SERVICES\
# OC_ENABLE_GUEST_LINKS=true\
' /etc/opencloud/opencloud.env
fi
# 9.0: the default space quota is read only under its new name, and thumbnails are cached in thumbnails/cache
sed -i 's/^GRAPH_SPACES_DEFAULT_QUOTA=/OC_SPACES_DEFAULT_QUOTA=/' /etc/opencloud/opencloud.env
rm -rf /var/lib/opencloud/thumbnails/files
msg_info "Starting services"
systemctl start opencloud opencloud-wopi
msg_ok "Started services"
+1 -1
View File
@@ -36,7 +36,7 @@ function update_script() {
systemctl stop nginx opengym
msg_ok "Stopped openGym"
NODE_VERSION="26" setup_nodejs
NODE_VERSION="22" setup_nodejs
CLEAN_INSTALL=1 fetch_and_deploy_gh_release "opengym" "DuarteSantos8/openGym" "tarball" "latest" "/opt/opengym" "" "v"
msg_info "Building openGym"
-94
View File
@@ -1,94 +0,0 @@
#!/usr/bin/env bash
_cs_boot="${COMMUNITY_SCRIPTS_CORE_DIR:-$(dirname "${BASH_SOURCE[0]}")/../../core}/core/build.func"
source "$_cs_boot" 2>/dev/null || source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/core/build.func")
# Copyright (c) 2021-2026 community-scripts ORG
# Author: uchouT (uchouT)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://github.com/OpenListTeam/OpenList/
APP="OpenList"
var_tags="${var_tags:-files;storage}"
var_cpu="${var_cpu:-1}"
var_arm64="${var_arm64:-yes}"
var_unprivileged="${var_unprivileged:-1}"
if [[ -z "${var_os:-}" ]] && command -v pveversion >/dev/null 2>&1; then
var_os=$(msg_menu "Choose the container OS" \
"debian" "Debian 13" \
"alpine" "Alpine 3.24 (smaller footprint)")
fi
if [[ "${var_os:-}" == "alpine" ]]; then
var_ram="${var_ram:-512}"
var_disk="${var_disk:-4}"
var_version="${var_version:-3.24}"
else
var_ram="${var_ram:-1024}"
var_disk="${var_disk:-6}"
var_version="${var_version:-13}"
fi
header_info "$APP"
variables
color
catch_errors
update_deb_based() {
if [[ ! -f /opt/openlist/openlist ]]; then
msg_error "No OpenList Installation Found!"
exit 1
fi
if check_for_gh_release "openlist" "OpenListTeam/OpenList"; then
msg_info "Stopping OpenList"
systemctl stop openlist
msg_ok "Stopped OpenList"
CLEAN_INSTALL=1 fetch_and_deploy_gh_release "openlist" "OpenListTeam/OpenList" "prebuild" "latest" "/opt/openlist" "openlist-linux-musl-$(arch_resolve "amd64" "arm64").tar.gz"
msg_info "Starting OpenList"
systemctl start openlist
msg_ok "Started OpenList"
msg_ok "Updated successfully!"
fi
}
update_alpine() {
if [[ ! -f /opt/openlist/openlist ]]; then
msg_error "No OpenList Installation Found!"
exit 1
fi
msg_info "Updating Alpine Packages"
$STD apk -U upgrade
msg_ok "Updated Alpine Packages"
if check_for_gh_release "openlist" "OpenListTeam/OpenList"; then
msg_info "Stopping OpenList"
$STD rc-service openlist stop
msg_ok "Stopped OpenList"
CLEAN_INSTALL=1 fetch_and_deploy_gh_release "openlist" "OpenListTeam/OpenList" "prebuild" "latest" "/opt/openlist" "openlist-linux-musl-$(arch_resolve "amd64" "arm64").tar.gz"
msg_info "Starting OpenList"
$STD rc-service openlist start
msg_ok "Started OpenList"
msg_ok "Updated successfully!"
fi
}
function update_script() {
header_info
check_container_storage
check_container_resources
run_os_update
}
start
build_container
description
msg_ok "Completed Successfully!\n"
echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}"
echo -e "${INFO}${YW}Access it using the following URL:${CL}"
echo -e "${GATEWAY}${BGN}http://${IP}:5244${CL}"
echo -e "${INFO}${YW}Credentials: /root/openlist.creds (inside the container)${CL}"
+11 -19
View File
@@ -101,29 +101,21 @@ EOF
if [ -x "/usr/bin/ollama" ]; then
msg_info "Checking for Ollama Update"
if check_for_gh_release "ollama-com" "ollama/ollama"; then
rm -rf /usr/lib/ollama.new
mkdir -p /usr/lib/ollama.new
if ! fetch_and_deploy_gh_release "ollama-com" "ollama/ollama" "prebuild" "latest" "/usr/lib/ollama.new" "ollama-linux-$(arch_resolve).tar.zst"; then
rm -rf /usr/lib/ollama.new
msg_error "Ollama download or deployment failed – check network connectivity and GitHub API availability (installed Ollama left unchanged)"
msg_info "Stopping Ollama Service"
systemctl stop ollama
msg_ok "Stopped Service"
rm -rf /usr/lib/ollama /usr/bin/ollama
if ! fetch_and_deploy_gh_release "ollama-com" "ollama/ollama" "prebuild" "latest" "/usr/lib/ollama" "ollama-linux-$(arch_resolve).tar.zst"; then
msg_error "Ollama download or deployment failed – check network connectivity and GitHub API availability"
else
msg_info "Stopping Ollama Service"
systemctl stop ollama
msg_ok "Stopped Service"
rm -rf /usr/lib/ollama.old
if [[ -d /usr/lib/ollama ]]; then
mv /usr/lib/ollama /usr/lib/ollama.old
fi
mv /usr/lib/ollama.new /usr/lib/ollama
ln -sf /usr/lib/ollama/bin/ollama /usr/bin/ollama
rm -rf /usr/lib/ollama.old
msg_ok "Updated Ollama to ${CHECK_UPDATE_RELEASE}"
msg_info "Starting Ollama Service"
systemctl start ollama
msg_ok "Started Service"
fi
msg_info "Starting Ollama Service"
systemctl start ollama
msg_ok "Started Service"
fi
fi
+1 -1
View File
@@ -42,7 +42,7 @@ function update_script() {
restore_backup
NODE_VERSION="26" NODE_MODULE="pnpm@11" setup_nodejs
NODE_VERSION="24" NODE_MODULE="pnpm@11" setup_nodejs
msg_info "Building Razzia"
cd /opt/razzia
+1 -1
View File
@@ -38,7 +38,7 @@ function update_script() {
ensure_dependencies git
create_backup /opt/reactive-resume/.env
NODE_VERSION="26" NODE_MODULE="corepack" setup_nodejs
NODE_VERSION="24" NODE_MODULE="corepack" setup_nodejs
CLEAN_INSTALL=1 fetch_and_deploy_gh_release "reactive-resume" "amruthpillai/reactive-resume" "tarball" "latest" "/opt/reactive-resume"
restore_backup
+24 -34
View File
@@ -25,54 +25,44 @@ function update_script() {
header_info
check_container_storage
check_container_resources
if [[ ! -d /opt/trilium && ! -f /etc/systemd/system/trilium.service ]]; then
if [[ ! -d /opt/trilium ]]; then
msg_error "No ${APP} Installation Found!"
exit
fi
if check_for_gh_release "Trilium" "TriliumNext/Trilium" "" "" "v"; then
# Notes live in Trilium's data directory, not in /opt/trilium: TRILIUM_DATA_DIR
# when the service sets it, otherwise ~/trilium-data or ~/.local/share/trilium-data.
DATA_DIR="$(systemctl show trilium -p Environment --value 2>/dev/null | tr ' ' '\n' | sed -n 's/^TRILIUM_DATA_DIR=//p' || true)"
if [[ -z "$DATA_DIR" ]]; then
DATA_DIR="/root/trilium-data"
[[ -d "$DATA_DIR" ]] || DATA_DIR="/root/.local/share/trilium-data"
if [[ -d /opt/trilium/db ]]; then
DB_PATH="/opt/trilium/db"
DB_RESTORE_PATH="/opt/trilium/db"
elif [[ -d /opt/trilium/assets/db ]]; then
DB_PATH="/opt/trilium/assets/db"
DB_RESTORE_PATH="/opt/trilium/assets/db"
else
msg_error "Database not found in either /opt/trilium/db or /opt/trilium/assets/db"
exit
fi
if [[ "${DATA_DIR%/}" == "/opt/trilium" || "$DATA_DIR" == /opt/trilium/* ]]; then
msg_error "Notes are stored in ${DATA_DIR}, which the update replaces. Move TRILIUM_DATA_DIR out of /opt/trilium first."
exit 1
fi
DB_BACKUP="${DATA_DIR}/backup/backup-before-update.db"
msg_info "Stopping Service"
systemctl stop trilium
sleep 1
msg_ok "Stopped Service"
if [[ -f "${DATA_DIR}/document.db" ]]; then
msg_info "Backing up Database"
rm -f "$DB_BACKUP" "${DB_BACKUP}-wal" "${DB_BACKUP}-shm"
# Trilium folds its -wal file into document.db when it stops; a leftover one
# is copied as well, so the backup is complete even if it was killed.
if mkdir -p "${DATA_DIR}/backup" && cp -a "${DATA_DIR}/document.db" "$DB_BACKUP" &&
{ [[ ! -f "${DATA_DIR}/document.db-wal" ]] || cp -a "${DATA_DIR}/document.db-wal" "${DB_BACKUP}-wal"; }; then
msg_ok "Backed up Database to ${DB_BACKUP}"
else
rm -f "$DB_BACKUP" "${DB_BACKUP}-wal"
systemctl start trilium
msg_error "Database backup failed - update not started, Trilium restarted"
exit 1
fi
else
msg_warn "No database found in ${DATA_DIR}, nothing to back up"
fi
msg_info "Backing up Database"
mkdir -p /opt/trilium_backup
cp -r "${DB_PATH}" /opt/trilium_backup/
rm -rf /opt/trilium
msg_ok "Backed up Database"
CLEAN_INSTALL=1 fetch_and_deploy_gh_release "Trilium" "TriliumNext/Trilium" "prebuild" "latest" "/opt/trilium" "TriliumNotes-Server-*linux-$(arch_resolve "x64" "arm64").tar.xz" "v" || {
systemctl start trilium
msg_error "Update failed - the previous version is still installed and was restarted"
exit 1
}
fetch_and_deploy_gh_release "Trilium" "TriliumNext/Trilium" "prebuild" "latest" "/opt/trilium" "TriliumNotes-Server-*linux-$(arch_resolve "x64" "arm64").tar.xz" "v"
msg_info "Restoring Database"
mkdir -p "$(dirname "${DB_RESTORE_PATH}")"
cp -r /opt/trilium_backup/$(basename "${DB_PATH}") "${DB_RESTORE_PATH}"
rm -rf /opt/trilium_backup
msg_ok "Restored Database"
msg_info "Starting Service"
systemctl start trilium
sleep 1
msg_ok "Started Service"
msg_ok "Updated successfully!"
fi
-102
View File
@@ -1,102 +0,0 @@
#!/usr/bin/env bash
_cs_boot="${COMMUNITY_SCRIPTS_CORE_DIR:-$(dirname "${BASH_SOURCE[0]}")/../../core}/core/build.func"
source "$_cs_boot" 2>/dev/null || source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/core/build.func")
# Copyright (c) 2021-2026 community-scripts ORG
# Author: MickLesk (CanbiZ)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://github.com/vllm-project/vllm
APP="vLLM"
var_tags="${var_tags:-ai;llm;inference}"
var_cpu="${var_cpu:-4}"
var_ram="${var_ram:-8192}"
var_disk="${var_disk:-20}"
var_os="${var_os:-debian}"
var_version="${var_version:-13}"
var_gpu="${var_gpu:-yes}"
var_arm64="${var_arm64:-no}"
var_unprivileged="${var_unprivileged:-1}"
header_info "$APP"
variables
color
catch_errors
function update_script() {
header_info
check_container_storage
check_container_resources
if [[ ! -d /opt/vllm ]]; then
msg_error "No ${APP} Installation Found!"
exit 1
fi
if check_for_gh_release "vllm" "vllm-project/vllm"; then
msg_info "Stopping Service"
systemctl stop vllm
msg_ok "Stopped Service"
create_backup /opt/vllm/vllm.env
VLLM_BACKEND="$(cat /opt/vllm/.backend 2>/dev/null || echo cuda)"
VLLM_VERSION="$(get_latest_github_release "vllm-project/vllm")"
msg_info "Updating vLLM (${VLLM_BACKEND}, Patience)"
case "$VLLM_BACKEND" in
rocm)
$STD uv pip install --python /opt/vllm/bin/python --upgrade vllm \
--extra-index-url https://wheels.vllm.ai/rocm/
;;
cpu)
github_api_call "https://api.github.com/repos/vllm-project/vllm/releases/tags/v${VLLM_VERSION}" /tmp/vllm-release.json
VLLM_WHEEL="$(jq -r '.assets[] | select(.name | endswith("+cpu-cp38-abi3-manylinux_2_39_x86_64.whl")) | .browser_download_url' /tmp/vllm-release.json | head -n1)"
rm -f /tmp/vllm-release.json
[[ -n "$VLLM_WHEEL" ]] || {
msg_error "vLLM ${VLLM_VERSION} has no CPU wheel for x86_64"
exit 1
}
$STD uv pip install --python /opt/vllm/bin/python "$VLLM_WHEEL" --torch-backend cpu
;;
*)
$STD uv pip install --python /opt/vllm/bin/python --upgrade vllm
;;
esac
msg_ok "Updated vLLM (${VLLM_BACKEND})"
restore_backup
msg_info "Refreshing CUDA Environment"
CUDA_ROOT="$(find /opt/vllm/lib/python3*/site-packages/nvidia -maxdepth 1 -type d -name 'cu[0-9]*' 2>/dev/null | sort -V | tail -1 || true)"
if [[ -n "$CUDA_ROOT" ]]; then
CUDART="$(find "${CUDA_ROOT}/lib" -maxdepth 1 -name 'libcudart.so.*' 2>/dev/null | sort -V | tail -1 || true)"
if [[ -n "$CUDART" && ! -e "${CUDA_ROOT}/lib/libcudart.so" ]]; then
ln -s "$(basename "$CUDART")" "${CUDA_ROOT}/lib/libcudart.so"
fi
sed -i '/^\(CUDA_HOME\|FLASHINFER_NVCC\|PATH\|LD_LIBRARY_PATH\|LIBRARY_PATH\)=/d' /opt/vllm/vllm.env
cat <<EOF >>/opt/vllm/vllm.env
CUDA_HOME=${CUDA_ROOT}
FLASHINFER_NVCC=${CUDA_ROOT}/bin/nvcc
PATH=${CUDA_ROOT}/bin:/opt/vllm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LD_LIBRARY_PATH=${CUDA_ROOT}/lib
LIBRARY_PATH=${CUDA_ROOT}/lib
EOF
fi
msg_ok "Refreshed CUDA Environment"
msg_info "Starting Service"
systemctl start vllm
msg_ok "Started Service"
msg_ok "Updated successfully!"
fi
cleanup_lxc
exit
}
start
build_container
description
msg_ok "Completed Successfully!\n"
echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}"
echo -e "${INFO}${YW}Access it using the following URL:${CL}"
echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:8000/v1${CL}"
-54
View File
@@ -1,54 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2021-2026 community-scripts ORG
# Author: mews-se
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://dietpi.com/
source /dev/stdin <<<"$FUNCTIONS_FILE_PATH"
color
verb_ip6
catch_errors
setting_up_container
network_check
update_os
msg_warn "WARNING: This script will run an external installer from a third-party source (https://dietpi.com/)."
msg_warn "The following code is NOT maintained or audited by our repository."
msg_warn "If you have any doubts or concerns, please review the installer code before proceeding:"
msg_custom "${TAB3}${GATEWAY}${BGN}${CL}" "\e[1;34m" "→ https://raw.githubusercontent.com/MichaIng/DietPi/master/.build/images/dietpi-installer"
echo
if ! prompt_confirm "${TAB3}Do you want to continue?" "y" 60; then
msg_error "Aborted by user. No changes have been made."
exit 10
fi
motd_ssh
msg_info "Converting Debian to DietPi"
curl -fsSL https://raw.githubusercontent.com/MichaIng/DietPi/master/.build/images/dietpi-installer -o /tmp/dietpi-installer
$STD env G_INTERACTIVE=0 GITBRANCH=master HW_MODEL=75 DISTRO_TARGET=8 GUEST_NETWORK_REQUIRED=1 IMAGE_CREATOR=community-scripts PREIMAGE_INFO='Proxmox Debian LXC template' bash /tmp/dietpi-installer
rm -f /tmp/dietpi-installer
msg_ok "Converted Debian to DietPi"
msg_info "Configuring DietPi"
sed -i -e 's/^AUTO_SETUP_NET_ETHERNET_ENABLED=.*/AUTO_SETUP_NET_ETHERNET_ENABLED=0/' \
-e "s/^AUTO_SETUP_NET_HOSTNAME=.*/AUTO_SETUP_NET_HOSTNAME=$(hostname)/" /boot/dietpi.txt
if [[ -n "${PASSWORD:-}" ]]; then
sed -i '/^AUTO_SETUP_GLOBAL_PASSWORD=/d' /boot/dietpi.txt
echo "AUTO_SETUP_GLOBAL_PASSWORD=$PASSWORD" >>/boot/dietpi.txt
fi
rm -f /etc/apt/sources.list.d/debian.sources
rm -f /etc/sysctl.d/97-dietpi.conf
rm -f /etc/systemd/system/local-fs.target.wants/dietpi-fs_partition_resize.service
msg_ok "Configured DietPi"
customize
# Autologin on tty1 only: DietPi's first run setup runs on the first console that logs in, and the Proxmox UI attaches to tty1
rm -f /etc/systemd/system/console-getty.service.d/override.conf
if [[ -f /etc/systemd/system/container-getty@.service.d/override.conf ]]; then
mkdir -p /etc/systemd/system/container-getty@1.service.d
mv /etc/systemd/system/container-getty@.service.d/override.conf /etc/systemd/system/container-getty@1.service.d/
rmdir /etc/systemd/system/container-getty@.service.d
fi
cleanup_lxc
+2
View File
@@ -14,6 +14,8 @@ network_check
update_os
fetch_and_deploy_gh_release "goatcounter" "arp242/goatcounter" "singlefile" "latest" "/opt/goatcounter" "goatcounter-v*-linux-$(arch_resolve).gz"
mv /opt/goatcounter/goatcounter /opt/goatcounter/goatcounter.gz
gunzip -f /opt/goatcounter/goatcounter.gz
msg_info "Configuring GoatCounter"
ADMIN_EMAIL="${var_admin_email:-admin@goatcounter.local}"
+177 -45
View File
@@ -69,9 +69,13 @@ $STD apt install --no-install-recommends -y \
git \
redis \
pigz \
autoconf \
build-essential \
python3-dev \
automake \
cmake \
jq \
libtool \
libltdl-dev \
libgdk-pixbuf-2.0-dev \
libbrotli-dev \
@@ -92,6 +96,8 @@ $STD apt install --no-install-recommends -y \
libltdl7 \
libopenjp2-7 \
libopenjp2-7-dev \
meson \
ninja-build \
pkg-config \
mesa-utils \
mesa-va-drivers \
@@ -108,7 +114,8 @@ $STD apt install --no-install-recommends -y \
libhwy-dev \
libwebp-dev \
libaom-dev \
unzip
unzip \
ccache
setup_deb822_repo \
"jellyfin" \
@@ -195,54 +202,179 @@ sed -i "s/^#shared_preload.*/shared_preload_libraries = 'vchord.so'/" /etc/postg
systemctl restart postgresql.service
PG_DB_NAME="immich" PG_DB_USER="immich" PG_DB_GRANT_SUPERUSER="true" PG_DB_SKIP_ALTER_ROLE="true" setup_postgresql_db
msg_info "Installing GCC-13 (available as fallback compiler)"
$STD apt install -y gcc-13 g++-13
msg_ok "Installed GCC-13"
msg_warn "Compiling Custom Photo-processing Libraries (can take anywhere from 15min to 2h)"
LD_LIBRARY_PATH=/usr/local/lib
export LD_RUN_PATH=/usr/local/lib
STAGING_DIR=/opt/staging
BASE_REPO="https://github.com/immich-app/base-images"
BASE_DIR=${STAGING_DIR}/base-images
SOURCE_DIR=${STAGING_DIR}/image-source
$STD git clone -b main "$BASE_REPO" "$BASE_DIR"
mkdir -p "$SOURCE_DIR"
# Immich ships the custom photo-processing libraries (libjxl, jpegli, libheif,
# libraw, ImageMagick, libvips) prebuilt in its own OCI base image. Pull the one
# the release's Dockerfile pins (skopeo + umoci, no container runtime) and use its
# /usr/local. There is no compile fallback: this needs Proxmox VE 9.1 or newer.
msg_info "Deploying prebuilt image-processing libraries"
BASE_REF="$(curl -fsSL "https://raw.githubusercontent.com/immich-app/immich/${RELEASE}/server/Dockerfile" | awk '/^FROM ghcr.io\/immich-app\/base-server-dev/ {print $2; exit}')" || true
if [[ -z "$BASE_REF" ]]; then
msg_error "Could not read the base image from Immich ${RELEASE}'s Dockerfile."
exit 115
fi
# skopeo refuses a reference with both a tag and a digest; the digest is the pin.
BASE_IMAGE="$BASE_REF"
[[ "$BASE_REF" == *@sha256:* ]] && BASE_IMAGE="ghcr.io/immich-app/base-server-dev@${BASE_REF#*@}"
if ! $STD apt install -y --no-install-recommends skopeo umoci; then
msg_error "Could not install skopeo and umoci, which Immich needs for its image-processing libraries."
exit 105
fi
OCI_DIR="$(mktemp -d)"
OCI_BUNDLE="$(mktemp -d)"
if ! $STD skopeo copy --override-os linux --override-arch "$(arch_resolve amd64 arm64)" "docker://${BASE_IMAGE}" "oci:${OCI_DIR}:img"; then
rm -rf "$OCI_DIR" "$OCI_BUNDLE"
msg_error "Could not download Immich's base image (${BASE_IMAGE})."
exit 115
fi
if ! $STD umoci unpack --image "${OCI_DIR}:img" "$OCI_BUNDLE"; then
rm -rf "$OCI_DIR" "$OCI_BUNDLE"
msg_error "Could not unpack Immich's base image. Immich needs Proxmox VE 9.1 or newer."
exit 105
fi
# The libraries, headers and ImageMagick data, without the node runtime, global
# node modules and ffmpeg of the node base image: this script provides those.
tar -C "${OCI_BUNDLE}/rootfs/usr/local" \
--exclude=./bin/node --exclude=./bin/npm --exclude=./bin/npx --exclude=./bin/corepack \
--exclude=./bin/ffmpeg --exclude=./bin/ffprobe \
--exclude=./lib/node_modules --exclude=./include/node \
-cf - . | tar -C /usr/local -xf -
rm -rf "$OCI_DIR" "$OCI_BUNDLE"
export LD_LIBRARY_PATH=/usr/local/lib
msg_info "(1/6) Compiling libjxl"
cd "$STAGING_DIR"
SOURCE=${SOURCE_DIR}/libjxl
LIBJXL_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libjxl.json)"
$STD git clone https://github.com/libjxl/libjxl.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBJXL_REVISION"
$STD git submodule update --init --recursive --depth 1 --recommend-shallow -- third_party
mkdir build
cd build
$STD cmake \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TESTING=OFF \
-DJPEGXL_ENABLE_DOXYGEN=OFF \
-DJPEGXL_ENABLE_MANPAGES=OFF \
-DJPEGXL_ENABLE_BENCHMARK=OFF \
-DJPEGXL_ENABLE_EXAMPLES=OFF \
-DJPEGXL_FORCE_SYSTEM_BROTLI=ON \
-DJPEGXL_FORCE_SYSTEM_HWY=ON \
-DJPEGXL_ENABLE_HWY_AVX3=ON \
-DJPEGXL_ENABLE_HWY_AVX3_ZEN4=ON \
-DJPEGXL_ENABLE_HWY_SVE=OFF \
-DJPEGXL_ENABLE_HWY_SVE2=OFF \
-DJPEGXL_ENABLE_HWY_SVE2_128=ON \
-DJPEGXL_ENABLE_PLUGINS=ON \
..
$STD cmake --build . -- -j"$(nproc)"
$STD cmake --install .
ldconfig /usr/local/lib
if ! vips --version >/dev/null 2>&1 || ! pkg-config --exists vips; then
msg_error "The prebuilt libvips from ${BASE_IMAGE} does not work in this container."
exit 1
fi
echo "$BASE_REF" >~/.immich_base_image
msg_ok "Deployed prebuilt image-processing libraries"
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/{build,third_party}
msg_ok "(1/6) Compiled libjxl"
msg_info "(2/6) Compiling jpegli"
SOURCE=${SOURCE_DIR}/jpegli
JPEGLI_LIBJPEG_LIBRARY_SOVERSION="62"
JPEGLI_LIBJPEG_LIBRARY_VERSION="62.3.0"
JPEGLI_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/jpegli.json)"
$STD git clone https://github.com/google/jpegli.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$JPEGLI_REVISION"
$STD git submodule update --init --depth 1 --recommend-shallow third_party/libjpeg-turbo
$STD git apply -3 "$BASE_DIR"/server/sources/jpegli-patches/jpegli-empty-dht-marker.patch
$STD git apply -3 "$BASE_DIR"/server/sources/jpegli-patches/jpegli-icc-warning.patch
mkdir build
cd build
$STD cmake \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_TESTING=OFF \
-DJPEGLI_ENABLE_DOXYGEN=OFF \
-DJPEGLI_ENABLE_MANPAGES=OFF \
-DJPEGLI_ENABLE_BENCHMARK=OFF \
-DJPEGLI_ENABLE_TOOLS=OFF \
-DJPEGLI_ENABLE_DEVTOOLS=OFF \
-DJPEGLI_ENABLE_FUZZERS=OFF \
-DJPEGLI_ENABLE_JNI=OFF \
-DJPEGLI_ENABLE_OPENEXR=OFF \
-DJPEGLI_ENABLE_SJPEG=OFF \
-DJPEGLI_ENABLE_SKCMS=OFF \
-DJPEGLI_FORCE_SYSTEM_HWY=ON \
-DJPEGLI_FORCE_SYSTEM_LCMS2=ON \
-DJPEGLI_ENABLE_JPEGLI_LIBJPEG=ON \
-DJPEGLI_INSTALL_JPEGLI_LIBJPEG=ON \
-DJPEGLI_ENABLE_HWY_AVX3=ON \
-DJPEGLI_ENABLE_HWY_AVX3_ZEN4=ON \
-DJPEGLI_ENABLE_HWY_SVE=OFF \
-DJPEGLI_ENABLE_HWY_SVE2=OFF \
-DJPEGLI_ENABLE_HWY_SVE2_128=ON \
-DJPEGLI_LIBJPEG_LIBRARY_SOVERSION="$JPEGLI_LIBJPEG_LIBRARY_SOVERSION" \
-DJPEGLI_LIBJPEG_LIBRARY_VERSION="$JPEGLI_LIBJPEG_LIBRARY_VERSION" \
-DLIBJPEG_TURBO_VERSION_NUMBER=2001005 \
..
$STD cmake --build . -- -j"$(nproc)"
$STD cmake --install .
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/{build,third_party}
msg_ok "(2/6) Compiled jpegli"
msg_info "(3/6) Compiling libheif"
SOURCE=${SOURCE_DIR}/libheif
LIBHEIF_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libheif.json)"
$STD git clone https://github.com/strukturag/libheif.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBHEIF_REVISION"
mkdir build
cd build
$STD cmake --preset=release-noplugins \
-DWITH_DAV1D=ON \
-DENABLE_PARALLEL_TILE_DECODING=ON \
-DWITH_LIBSHARPYUV=ON \
-DWITH_LIBDE265=ON \
-DWITH_AOM_DECODER=OFF \
-DWITH_AOM_ENCODER=ON \
-DWITH_X265=OFF \
-DWITH_EXAMPLES=OFF \
..
$STD make install -j"$(nproc)"
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
rm -rf "$SOURCE"/build
msg_ok "(3/6) Compiled libheif"
msg_info "(4/6) Compiling libraw"
SOURCE=${SOURCE_DIR}/libraw
LIBRAW_REVISION="e419de08001de28ae6988ecb22df47e52b9c5eaa"
# : "${LIBRAW_REVISION:=$(jq -cr '.revision' $BASE_DIR/server/sources/libraw.json)}"
$STD git clone https://github.com/LibRaw/LibRaw.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBRAW_REVISION"
$STD autoreconf --install
$STD ./configure --disable-examples
$STD make -j"$(nproc)"
$STD make install
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
msg_ok "(4/6) Compiled libraw"
msg_info "(5/6) Compiling imagemagick"
SOURCE=$SOURCE_DIR/imagemagick
: "${IMAGEMAGICK_REVISION:=$(jq -cr '.revision' $BASE_DIR/server/sources/imagemagick.json)}"
$STD git clone https://github.com/ImageMagick/ImageMagick.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$IMAGEMAGICK_REVISION"
$STD ./configure --with-modules CPPFLAGS="-DMAGICK_LIBRAW_VERSION_TAIL=202502"
$STD make -j"$(nproc)"
$STD make install
ldconfig /usr/local/lib
$STD make clean
cd "$STAGING_DIR"
msg_ok "(5/6) Compiled imagemagick"
msg_info "(6/6) Compiling libvips"
SOURCE=$SOURCE_DIR/libvips
LIBVIPS_REVISION="$(jq -cr '.revision' "$BASE_DIR"/server/sources/libvips.json)"
$STD git clone https://github.com/libvips/libvips.git "$SOURCE"
cd "$SOURCE"
$STD git reset --hard "$LIBVIPS_REVISION"
$STD git apply "$BASE_DIR"/server/sources/libvips-patches/0001-put-other-loaders-ahead-of-dcrawload.patch
$STD meson setup build --buildtype=release --libdir=lib -Dintrospection=disabled -Dtiff=disabled
cd build
$STD ninja install
ldconfig /usr/local/lib
cd "$STAGING_DIR"
rm -rf "$SOURCE"/build
msg_ok "(6/6) Compiled libvips"
cat <<EOF >~/.immich_library_revisions
imagemagick: $IMAGEMAGICK_REVISION
jpegli: $JPEGLI_REVISION
libheif: $LIBHEIF_REVISION
libjxl: $LIBJXL_REVISION
libraw: $LIBRAW_REVISION
libvips: $LIBVIPS_REVISION
EOF
msg_ok "Custom Photo-processing Libraries Compiled Successfully"
fetch_and_deploy_gh_release "Immich" "immich-app/immich" "tarball" "${RELEASE}" "$SRC_DIR"
PNPM_VERSION="$(jq -r '.packageManager | split("@")[1] | split("+")[0]' ${SRC_DIR}/package.json)"
-1
View File
@@ -139,7 +139,6 @@ server {
fastcgi_pass unix:${PHP_SOCK};
fastcgi_param SCRIPT_FILENAME \$realpath_root\$fastcgi_script_name;
include fastcgi_params;
fastcgi_param HTTP_HOST \$http_host;
fastcgi_hide_header X-Powered-By;
fastcgi_read_timeout 300;
}
-1
View File
@@ -134,7 +134,6 @@ server {
fastcgi_pass unix:__PHP_SOCK__;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_param HTTP_HOST $http_host;
fastcgi_param HTTP_X_FORWARDED_HOST $http_host;
fastcgi_param HTTP_X_FORWARDED_PROTO $scheme;
fastcgi_read_timeout 300;
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2021-2026 community-scripts ORG
# Author: Yusuf Gundogdu (yusuf-gundogdu)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://libredb.org
source /dev/stdin <<<"$FUNCTIONS_FILE_PATH"
color
verb_ip6
catch_errors
setting_up_container
network_check
update_os
fetch_and_deploy_gh_release "libredb-studio" "libredb/libredb-studio" "binary" "latest" "" "libredb-studio_*_$(arch_resolve).deb"
msg_info "Configuring LibreDB Studio"
JWT_SECRET="$(openssl rand -hex 32)"
ADMIN_PASSWORD="$(openssl rand -base64 18 | tr -d '/+=' | cut -c1-16)"
mkdir -p /etc/libredb-studio
cat <<EOF >/etc/libredb-studio/env
HOSTNAME=0.0.0.0
PORT=3000
# Cookie is marked Secure on non-loopback hosts; plain-HTTP LXC login loops without this.
AUTH_COOKIE_SECURE=false
AUTH_BOOTSTRAP=off
JWT_SECRET=${JWT_SECRET}
ADMIN_EMAIL=admin@libredb.org
ADMIN_PASSWORD=${ADMIN_PASSWORD}
EOF
chmod 600 /etc/libredb-studio/env
systemctl enable -q --now libredb-studio
msg_ok "Configured LibreDB Studio"
motd_ssh
customize
cleanup_lxc
+1 -2
View File
@@ -37,8 +37,7 @@ ln -sf /opt/certbot/bin/certbot /usr/local/bin/certbot
msg_ok "Set up Certbot"
OPENRESTY_VERSION="1.29.2.5"
fetch_and_deploy_from_url "https://openresty.org/download/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty" ||
fetch_and_deploy_from_url "https://distfiles.alpinelinux.org/distfiles/edge/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty"
fetch_and_deploy_from_url "https://openresty.org/download/openresty-${OPENRESTY_VERSION}.tar.gz" "/opt/openresty"
msg_info "Building OpenResty"
cd /opt/openresty
-5
View File
@@ -39,12 +39,7 @@ msg_ok "Fetched NPMplus"
attempts=0
TZ_INPUT="${var_npmplus_tz:-}"
TZ_DEFAULT="$(cat /etc/timezone 2>/dev/null || true)"
while true; do
if [[ -z "$TZ_INPUT" && -n "$TZ_DEFAULT" ]]; then
read -r -p "${TAB3}Enter your TZ Identifier [${TZ_DEFAULT}]: " TZ_INPUT || true
TZ_INPUT="${TZ_INPUT:-$TZ_DEFAULT}"
fi
[[ -n "$TZ_INPUT" ]] || read -r -p "${TAB3}Enter your TZ Identifier (e.g., Europe/Berlin): " TZ_INPUT || { msg_error "No TZ identifier given; set var_npmplus_tz or answer the prompt"; exit 64; }
if validate_tz "$TZ_INPUT"; then
break
+1 -15
View File
@@ -14,8 +14,6 @@ network_check
update_os
MAX_ATTEMPTS=3
# Each FQDN is read from var_opencloud_opencloud_fqdn, var_opencloud_collabora_fqdn and var_opencloud_wopi_fqdn when set.
# Named literally here because the prompt below builds the name from $server, and unattended runs look for the literal names.
servers=("opencloud" "collabora" "wopi")
attempt=0
for server in "${servers[@]}"; do
@@ -70,7 +68,7 @@ $STD sudo -u cool coolconfig set-admin-password --user=admin --password="$COOLPA
echo "$COOLPASS" >~/.coolpass
msg_ok "Installed Collabora Online"
fetch_and_deploy_gh_release "OpenCloud" "opencloud-eu/opencloud" "singlefile" "v9.0.0" "/usr/bin" "opencloud-*-linux-$(arch_resolve)"
fetch_and_deploy_gh_release "OpenCloud" "opencloud-eu/opencloud" "singlefile" "v8.1.0" "/usr/bin" "opencloud-*-linux-$(arch_resolve)"
mv /usr/bin/OpenCloud /usr/bin/opencloud
msg_info "Configuring OpenCloud"
@@ -130,10 +128,6 @@ COLLABORATION_JWT_SECRET=
## Allow insecure connections. Defaults to false.
# NOTIFICATIONS_SMTP_INSECURE=false
## Guest links - share with external people by email
## Requires the SMTP settings above and notifications in OC_ADD_RUN_SERVICES
# OC_ENABLE_GUEST_LINKS=true
## Start additional services at runtime
## Examples: notifications, antivirus etc.
## Do not uncomment unless configured above.
@@ -223,14 +217,6 @@ chown -R opencloud:opencloud "$CONFIG_DIR" "$DATA_DIR"
sudo -u opencloud opencloud init --config-path "$CONFIG_DIR" --insecure no
OPENCLOUD_SECRET="$(sed -n '/jwt/p' "$CONFIG_DIR"/opencloud.yaml | awk '{print $2}')"
sed -i "s/JWT_SECRET=/&${OPENCLOUD_SECRET//&/\\&}/" "$ENV_FILE"
ACCOUNT_ID="$(awk '/service_account_id:/ {gsub(/["'\'']/, "", $2); print $2; exit}' "$CONFIG_DIR"/opencloud.yaml)"
ACCOUNT_SECRET="$(awk '/service_account_secret:/ {sub(/^[^:]*:[[:space:]]*/, ""); gsub(/^["'\'']|["'\'']$/, ""); print; exit}' "$CONFIG_DIR"/opencloud.yaml)"
if [[ -n "$ACCOUNT_ID" && -n "$ACCOUNT_SECRET" && "$ACCOUNT_SECRET" != *"'"* ]]; then
printf "OC_SERVICE_ACCOUNT_ID=%s\nOC_SERVICE_ACCOUNT_SECRET='%s'\n" "$ACCOUNT_ID" "$ACCOUNT_SECRET" |
sed -i '/^STORAGE_SYSTEM_OC_ROOT=/r /dev/stdin' "$ENV_FILE"
else
msg_warn "Could not read the service account from opencloud.yaml; set OC_SERVICE_ACCOUNT_ID/SECRET in opencloud.env"
fi
msg_ok "Configured OpenCloud"
msg_info "Starting services"
+1 -1
View File
@@ -17,7 +17,7 @@ msg_info "Installing Dependencies"
$STD apt install -y nginx
msg_ok "Installed Dependencies"
NODE_VERSION="26" setup_nodejs
NODE_VERSION="22" setup_nodejs
fetch_and_deploy_gh_release "opengym" "DuarteSantos8/openGym" "tarball" "latest" "/opt/opengym" "" "v"
msg_info "Building openGym"
-98
View File
@@ -1,98 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2021-2026 community-scripts ORG
# Author: uchouT (uchouT)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://github.com/OpenListTeam/OpenList/
source /dev/stdin <<<"$FUNCTIONS_FILE_PATH"
color
verb_ip6
catch_errors
setting_up_container
network_check
update_os
setup_deb_based() {
fetch_and_deploy_gh_release "openlist" "OpenListTeam/OpenList" "prebuild" "latest" "/opt/openlist" "openlist-linux-musl-$(arch_resolve "amd64" "arm64").tar.gz"
msg_info "Configuring OpenList"
ADMIN_PASS="$(random_password 16)"
$STD /opt/openlist/openlist --data /opt/openlist_data admin set "$ADMIN_PASS"
if [[ ! -f /opt/openlist_data/data.db ]]; then
msg_error "OpenList did not initialise its database"
exit 1
fi
cat <<EOF >~/openlist.creds
OpenList Credentials
Username: admin
Password: ${ADMIN_PASS}
EOF
chmod 600 ~/openlist.creds
msg_ok "Configured OpenList"
msg_info "Creating Service"
cat <<EOF >/etc/systemd/system/openlist.service
[Unit]
Description=OpenList
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/openlist
ExecStart=/opt/openlist/openlist server --data /opt/openlist_data
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
systemctl enable -q --now openlist
msg_ok "Created Service"
}
setup_alpine() {
fetch_and_deploy_gh_release "openlist" "OpenListTeam/OpenList" "prebuild" "latest" "/opt/openlist" "openlist-linux-musl-$(arch_resolve "amd64" "arm64").tar.gz"
msg_info "Configuring OpenList"
ADMIN_PASS="$(random_password 16)"
$STD /opt/openlist/openlist --data /opt/openlist_data admin set "$ADMIN_PASS"
if [[ ! -f /opt/openlist_data/data.db ]]; then
msg_error "OpenList did not initialise its database"
exit 1
fi
cat <<EOF >~/openlist.creds
OpenList Credentials
Username: admin
Password: ${ADMIN_PASS}
EOF
chmod 600 ~/openlist.creds
msg_ok "Configured OpenList"
msg_info "Creating Service"
cat <<EOF >/etc/init.d/openlist
#!/sbin/openrc-run
name="openlist"
description="OpenList Daemon"
directory="/opt/openlist"
command="/opt/openlist/openlist"
command_args="server --data /opt/openlist_data"
command_background="yes"
pidfile="/run/openlist.pid"
depend() {
need net
}
EOF
chmod +x /etc/init.d/openlist
$STD rc-update add openlist default
$STD rc-service openlist start
msg_ok "Created Service"
}
run_os_setup
motd_ssh
customize
cleanup_lxc
-1
View File
@@ -101,7 +101,6 @@ server {
location ~ \.php$ {
include fastcgi_params;
fastcgi_param HTTP_HOST \$http_host;
fastcgi_pass unix:${PHP_SOCK};
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT \$document_root;
+1 -1
View File
@@ -17,7 +17,7 @@ msg_info "Installing Dependencies"
$STD apt install -y nginx
msg_ok "Installed Dependencies"
NODE_VERSION="26" NODE_MODULE="pnpm@11" setup_nodejs
NODE_VERSION="24" NODE_MODULE="pnpm@11" setup_nodejs
fetch_and_deploy_gh_release "razzia" "Ralex91/Razzia" "tarball"
msg_info "Building Razzia"
+1 -1
View File
@@ -15,7 +15,7 @@ update_os
PG_VERSION="16" setup_postgresql
PG_DB_NAME="reactive_resume" PG_DB_USER="reactive_resume" setup_postgresql_db
NODE_VERSION="26" NODE_MODULE="corepack" setup_nodejs
NODE_VERSION="24" NODE_MODULE="corepack" setup_nodejs
msg_info "Installing Dependencies"
$STD apt install -y \
-135
View File
@@ -1,135 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2021-2026 community-scripts ORG
# Author: MickLesk (CanbiZ)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://github.com/vllm-project/vllm
source /dev/stdin <<<"$FUNCTIONS_FILE_PATH"
color
verb_ip6
catch_errors
setting_up_container
network_check
update_os
var_backend="${var_backend:-auto}"
var_model="${var_model:-Qwen/Qwen2.5-0.5B-Instruct}"
var_port="${var_port:-8000}"
var_dtype="${var_dtype:-auto}"
var_vram_utilization="${var_vram_utilization:-0.90}"
var_max_model_len="${var_max_model_len:-}"
var_hf_token="${var_hf_token:-}"
var_api_key="${var_api_key:-}"
msg_info "Installing Dependencies"
$STD apt install -y \
build-essential \
python3-dev
msg_ok "Installed Dependencies"
PYTHON_VERSION="3.12" setup_uv
setup_hwaccel
if [[ "$var_backend" == "auto" ]]; then
case "${HWACCEL_VENDOR:-none}" in
nvidia) var_backend="cuda" ;;
amd) var_backend="rocm" ;;
*) var_backend="cpu" ;;
esac
fi
VLLM_VERSION="$(get_latest_github_release "vllm-project/vllm")"
msg_info "Installing vLLM (${var_backend}, Patience)"
$STD uv venv --python 3.12 /opt/vllm
case "$var_backend" in
cuda)
$STD uv pip install --python /opt/vllm/bin/python vllm
;;
rocm)
$STD uv pip install --python /opt/vllm/bin/python vllm \
--extra-index-url https://wheels.vllm.ai/rocm/ --upgrade
;;
cpu)
if [[ "$(uname -m)" != "x86_64" ]]; then
msg_error "vLLM publishes no prebuilt CPU wheel for $(uname -m)"
exit 1
fi
github_api_call "https://api.github.com/repos/vllm-project/vllm/releases/tags/v${VLLM_VERSION}" /tmp/vllm-release.json
VLLM_WHEEL="$(jq -r '.assets[] | select(.name | endswith("+cpu-cp38-abi3-manylinux_2_39_x86_64.whl")) | .browser_download_url' /tmp/vllm-release.json | head -n1)"
rm -f /tmp/vllm-release.json
[[ -n "$VLLM_WHEEL" ]] || {
msg_error "vLLM ${VLLM_VERSION} has no CPU wheel for x86_64"
exit 1
}
$STD uv pip install --python /opt/vllm/bin/python "$VLLM_WHEEL" --torch-backend cpu
;;
*)
msg_error "Unknown var_backend '${var_backend}' (expected auto, cuda, rocm or cpu)"
exit 1
;;
esac
echo "$var_backend" >/opt/vllm/.backend
cat <<EOF >~/.vllm
${VLLM_VERSION}
EOF
msg_ok "Installed vLLM (${var_backend})"
msg_info "Configuring vLLM"
mkdir -p /opt/vllm/models
VLLM_SERVE_ARGS="--dtype ${var_dtype} --gpu-memory-utilization ${var_vram_utilization}"
[[ -n "$var_max_model_len" ]] && VLLM_SERVE_ARGS+=" --max-model-len ${var_max_model_len}"
cat <<EOF >/opt/vllm/vllm.env
VLLM_MODEL=${var_model}
VLLM_HOST=0.0.0.0
VLLM_PORT=${var_port}
HF_HOME=/opt/vllm/models
HF_TOKEN=${var_hf_token}
VLLM_API_KEY=${var_api_key}
# Appended to 'vllm serve' verbatim.
VLLM_SERVE_ARGS=${VLLM_SERVE_ARGS}
EOF
CUDA_ROOT="$(find /opt/vllm/lib/python3*/site-packages/nvidia -maxdepth 1 -type d -name 'cu[0-9]*' 2>/dev/null | sort -V | tail -1 || true)"
if [[ -n "$CUDA_ROOT" ]]; then
CUDART="$(find "${CUDA_ROOT}/lib" -maxdepth 1 -name 'libcudart.so.*' 2>/dev/null | sort -V | tail -1 || true)"
if [[ -n "$CUDART" && ! -e "${CUDA_ROOT}/lib/libcudart.so" ]]; then
ln -s "$(basename "$CUDART")" "${CUDA_ROOT}/lib/libcudart.so"
fi
cat <<EOF >>/opt/vllm/vllm.env
CUDA_HOME=${CUDA_ROOT}
FLASHINFER_NVCC=${CUDA_ROOT}/bin/nvcc
PATH=${CUDA_ROOT}/bin:/opt/vllm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LD_LIBRARY_PATH=${CUDA_ROOT}/lib
LIBRARY_PATH=${CUDA_ROOT}/lib
EOF
fi
chmod 600 /opt/vllm/vllm.env
msg_ok "Configured vLLM"
msg_info "Creating Service"
cat <<EOF >/etc/systemd/system/vllm.service
[Unit]
Description=vLLM OpenAI-Compatible Inference Server
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/opt/vllm
EnvironmentFile=/opt/vllm/vllm.env
ExecStart=/opt/vllm/bin/vllm serve \${VLLM_MODEL} --host \${VLLM_HOST} --port \${VLLM_PORT} \$VLLM_SERVE_ARGS
Restart=on-failure
RestartSec=10
TimeoutStartSec=0
[Install]
WantedBy=multi-user.target
EOF
systemctl enable -q --now vllm
msg_ok "Created Service"
motd_ssh
customize
cleanup_lxc
-1
View File
@@ -114,7 +114,6 @@ server {
fastcgi_pass unix:__PHP_SOCK__;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param HTTP_HOST $http_host;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
internal;
-1
View File
@@ -66,7 +66,6 @@ server {
fastcgi_pass unix:${PHP_SOCK};
fastcgi_param SCRIPT_FILENAME \$realpath_root\$fastcgi_script_name;
include fastcgi_params;
fastcgi_param HTTP_HOST \$http_host;
fastcgi_read_timeout 600;
}
}
+3
View File
@@ -25,6 +25,9 @@ $STD apt install -y \
msg_ok "Installed Dependencies"
fetch_and_deploy_gh_release "restic" "restic/restic" "singlefile" "latest" "/usr/local/bin" "restic_*_linux_$(arch_resolve).bz2"
mv /usr/local/bin/restic /usr/local/bin/restic.bz2
bzip2 -d /usr/local/bin/restic.bz2
chmod +x /usr/local/bin/restic
fetch_and_deploy_gh_release "rclone" "rclone/rclone" "prebuild" "latest" "/opt/rclone" "rclone-*-linux-$(arch_resolve).zip"
ln -sf /opt/rclone/rclone /usr/local/bin/rclone
-152
View File
@@ -1,152 +0,0 @@
#!/usr/bin/env bash
# Copyright (c) 2021-2026 community-scripts ORG
# Author: MickLesk (CanbiZ)
# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE
# Source: https://blissos.org/
COMMUNITY_SCRIPTS_URL="${COMMUNITY_SCRIPTS_URL:-https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main}"
source <(curl -fsSL "${COMMUNITY_SCRIPTS_CORE_URL:-https://raw.githubusercontent.com/community-scripts/core/main}/pve/vm-core.func")
load_functions
APP="BlissOS"
APP_TYPE="vm"
NSAPP="blissos-vm"
var_os="android"
var_version=" "
GEN_MAC=02:$(openssl rand -hex 5 | awk '{print toupper($0)}' | sed 's/\(..\)/\1:/g; s/.$//')
RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)"
METHOD=""
THIN="discard=on,ssd=1,"
header_info
echo -e "\n Loading..."
set -Eeo pipefail
shopt -s inherit_errexit
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
vm_require_arch amd64
TEMP_DIR=$(mktemp -d)
pushd "$TEMP_DIR" >/dev/null
vm_preflight
function default_settings() {
VMID=$(get_valid_nextid)
vm_apply_machine_type "q35"
DISK_SIZE="32G"
DISK_CACHE=""
HN="blissos"
CPU_TYPE=" -cpu host"
CORE_COUNT="4"
RAM_SIZE="4096"
BRG="vmbr0"
MAC="$GEN_MAC"
VLAN=""
MTU=""
START_VM="yes"
METHOD="default"
vm_echo_default_settings
}
function advanced_settings() {
METHOD="advanced"
vm_prompt_vmid "${VMID:-$(get_valid_nextid)}"
vm_prompt_machine_type "q35"
vm_prompt_disk_size "32G" "Set Disk Size in GiB"
vm_prompt_disk_cache "none"
vm_prompt_hostname "blissos"
vm_prompt_cpu_model "host"
vm_prompt_cpu_cores "4"
vm_prompt_ram "4096"
vm_prompt_bridge "vmbr0"
vm_prompt_mac "$GEN_MAC"
vm_prompt_vlan
vm_prompt_mtu
vm_prompt_keyboard
vm_prompt_verbose "no"
vm_prompt_start_vm "yes"
if vm_confirm_advanced_settings "Ready to create a BlissOS VM?"; then
echo -e "${CREATING}${BOLD}${DGN}Creating a BlissOS VM using the above advanced settings${CL}"
else
header_info
echo -e "${ADVANCED}${BOLD}${RD}Using Advanced Settings${CL}"
advanced_settings
fi
}
vm_start_script "Use Default Settings?\n\nDefaults:\n• 4 CPU Cores (Host model)\n• 4 GB RAM\n• 32 GB Disk\n• Q35 Machine Type" 14 58
post_to_api_vm
vm_select_storage "$HN"
msg_info "Retrieving the URL for the BlissOS installer"
# The official x86 builds live under blissos-x86. Both blissos and blissos-dev
# also exist and are years out of date, which is an easy way to end up shipping
# a script that points at a 2020 image. BlissOS17 has directories but no builds
# in them, so BlissOS16 really is the newest branch with files.
#
# FOSS rather than Gapps by default: the same build without the Google apps,
# which also avoids redistributing those. The Gapps tree sits beside it.
ISO_DIR="https://sourceforge.net/projects/blissos-x86/files/Official/BlissOS16/FOSS/Generic"
# The build date orders these, not the version -- 16.9.7 exists more than once
# with different dates.
if ! vm_latest_from_index "${ISO_DIR}/" 'Bliss-v[0-9.]+-x86_64-OFFICIAL-foss-[0-9]{8}\.iso' --sort-by 'foss-\K[0-9]{8}'; then
exit 1
fi
FILENAME="$VM_INDEX_LATEST"
BLISS_VERSION=$(echo "$FILENAME" | grep -oP 'Bliss-v\K[0-9.]+')
BLISS_BUILD=$(echo "$FILENAME" | grep -oP 'foss-\K[0-9]{8}')
var_version="${BLISS_VERSION}-${BLISS_BUILD}"
URL="${ISO_DIR}/${FILENAME}/download"
vm_select_iso_storage "$FILENAME" "$HN"
CACHE_FILE="$ISO_PATH"
msg_ok "BlissOS ${CL}${BL}${BLISS_VERSION}${CL} ${GN}(build ${BLISS_BUILD})"
# A bad SourceForge mirror serves an HTML notice with status 200, so the size
# decides whether this is an ISO, not curl's exit code. Learned from cachyos.
MIN_ISO_BYTES=$((1024 * 1024 * 1024))
[[ -s "$CACHE_FILE" ]] || msg_info "Downloading BlissOS (approximately 2 GB, this may take a while)"
vm_fetch_image "$URL" "$CACHE_FILE" --cache --min-bytes "$MIN_ISO_BYTES" || exit 115
msg_info "Creating a BlissOS VM"
# Android x86 carries virtio drivers -- the project targets QEMU as well as
# bare metal -- so unlike ChromeOS Flex this does not need SATA and e1000.
# UEFI with Secure Boot off; it will not boot with the Microsoft keys enrolled.
# virtio, not std: with stdvga Android gets as far as switch_root and then hangs
# on a black screen. vmwgfx does not help either; virtio-gpu is the DRM driver
# Android 13 actually carries. nomodeset also works but only until installation,
# since the installer writes its own bootloader config.
vm_claim_vmid
qm create $VMID -agent 1${MACHINE} -tablet 1 -bios ovmf${CPU_TYPE} -cores $CORE_COUNT -memory $RAM_SIZE \
-name $HN -tags community-script -net0 virtio,bridge=$BRG,macaddr=$MAC$VLAN$MTU -onboot 0 -ostype l26 -scsihw virtio-scsi-single \
-efidisk0 ${STORAGE}:1,efitype=4m,pre-enrolled-keys=0 -scsi0 ${STORAGE}:${DISK_SIZE%G},${DISK_CACHE}${THIN%,} \
-cdrom "$ISO_VOLUME" -boot order='scsi0;ide2' -vga virtio >/dev/null
vm_mark_created
set_description
msg_ok "Created a BlissOS VM ${CL}${BL}(${HN})"
vm_start_vm "BlissOS VM"
vm_print_summary "Version=${BLISS_VERSION} (build ${BLISS_BUILD})" "ISO=${FILENAME}"
vm_next_steps \
"Open the VM Console in Proxmox." \
"Pick Installation from the boot menu." \
"Create and format a partition on sda, then install there." \
"Say yes to GRUB and to a writable /system." \
"Reboot; the boot order prefers the disk. Detach the ISO afterwards to tidy up." \
"This is the FOSS build. Google apps are in the SourceForge BlissOS16/Gapps tree."
vm_finish "VM created; complete the BlissOS installation in the Proxmox console."