Docs: expand developer/debugging and tools references

Add extensive documentation and examples across contribution, guides, templates and tools references. Key changes: - Introduce a Developer Mode & Debugging section (dev_mode flags: trace, keep, pause, breakpoint, logs, dryrun, motd) in CONTRIBUTING.md with usage example. - Provide a standard update_script() pattern and BookStack example in GUIDE.md to clarify update flow (stop services, backup, deploy, restore, migrate, restart). - Add new helper entries (BookLore, KaraKeep) and advanced repository helpers (setup_deb822_repo, prepare_repository_setup, cleanup_tool_keyrings) plus utilities (setup_meilisearch, verify_tool_version) in HELPER_FUNCTIONS.md. - Update install template to suggest PNPM, Java 21 and Meilisearch; update example DB setup notes in AppName-install.sh. - Add var_diagnostics option and switch var_fuse to boolean/toggle wording in CONFIGURATION_REFERENCE.md; clarify privacy and defaults. - Adjust example container definitions in UNATTENDED_DEPLOYMENTS.md (container entries and resource values). - Change storage and flag variables and examples in BUILD_FUNC_USAGE_EXAMPLES.md (ssd-storage, var_fuse/var_tun, etc.). - Expand TOOLS_FUNC_FUNCTIONS_REFERENCE.md with many setup_* function signatures, environment vars, clarified fetch_and_deploy_gh_release modes/parameters, and additional tool docs (nodejs, php, mariadb_db, postgresql_db, java, uv, yq, meilisearch, composer, build tools). These updates improve onboarding, debugging guidance, and operational clarity for contributors and maintainers.
2026-02-06 05:13:27 +01:00 · 2026-02-05 15:16:35 +01:00
parent 30bc127011
commit 2c18646d82
8 changed files with 410 additions and 54 deletions
--- a/docs/contribution/CONTRIBUTING.md
+++ b/docs/contribution/CONTRIBUTING.md
@@ -147,6 +147,30 @@ Verify the PR shows ONLY these 3 files:

 ---

+# 🛠️ Developer Mode & Debugging
+
+When building or testing scripts, you can use the `dev_mode` variable to enable powerful debugging features. These flags can be combined (comma-separated).
+
+**Usage**:
+```bash
+# Example: Run with trace and keep the container even if it fails
+dev_mode="trace,keep" bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/myapp.sh)"
+```
+
+### Available Flags:
+
+| Flag | Description |
+| :--- | :--- |
+| `trace` | Enables `set -x` for maximum verbosity during execution. |
+| `keep` | Prevents the container from being deleted if the build fails. |
+| `pause` | Pauses execution at key points (e.g., before customization). |
+| `breakpoint` | Allows hardcoded `breakpoint` calls in scripts to drop to a shell. |
+| `logs` | Saves detailed build logs to `/var/log/community-scripts/`. |
+| `dryrun` | Bypasses actual container creation (limited support). |
+| `motd` | Forces an update of the Message of the Day (MOTD). |
+
+---
+
 ## 📚 Pages

 - [CT Template: AppName.sh](https://github.com/community-scripts/ProxmoxVE/blob/main/docs/contribution/templates_ct/AppName.sh)
--- a/docs/contribution/GUIDE.md
+++ b/docs/contribution/GUIDE.md
@@ -433,26 +433,44 @@ $STD apt-get install -y newdependency
 # 4. Test thoroughly before committing
 ```

-### Step 3: Update Update Function (if applicable)
+### Step 3: The Standard Update Pattern

+The `update_script()` function in `ct/appname.sh` should follow a robust pattern:
+
+1. **Check for updates**: Use `check_for_gh_release` to skip logic if no new version exists.
+2. **Stop services**: Stop all relevant services (`systemctl stop appname`).
+3. **Backup existing installation**: Move the old folder (e.g., `mv /opt/app /opt/app_bak`).
+4. **Deploy new version**: Use `CLEAN_INSTALL=1 fetch_and_deploy_gh_release`.
+5. **Restore configuration**: Copy `.env` or config files back from the backup.
+6. **Rebuild/Migrate**: Run `npm install`, `composer install`, or DB migrations.
+7. **Start services**: Restart services and cleanup the backup.
+
+**Example from `ct/bookstack.sh`**:
 ```bash
-# Edit: ct/existingapp.sh → update_script()
-
-# 1. Update GitHub API URL if repo changed
-RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | ...)
-
-# 2. Update backup/restore logic (if structure changed)
-# 3. Update cleanup paths
-
-# 4. Test update on existing installation
-```
-
-### Step 4: Document Your Changes
-
-```bash
-# Add comment at top of script
-# Co-Author: YourUsername
-# Updated: YYYY-MM-DD - Description of changes
+function update_script() {
+  if check_for_gh_release "bookstack" "BookStackApp/BookStack"; then
+    msg_info "Stopping Services"
+    systemctl stop apache2
+    
+    msg_info "Backing up data"
+    mv /opt/bookstack /opt/bookstack-backup
+    
+    fetch_and_deploy_gh_release "bookstack" "BookStackApp/BookStack" "tarball"
+    
+    msg_info "Restoring backup"
+    cp /opt/bookstack-backup/.env /opt/bookstack/.env
+    # ... restore uploads ...
+    
+    msg_info "Configuring"
+    cd /opt/bookstack
+    $STD composer install --no-dev
+    $STD php artisan migrate --force
+    
+    systemctl start apache2
+    rm -rf /opt/bookstack-backup
+    msg_ok "Updated successfully!"
+  fi
+}
 ```

 ---
--- a/docs/contribution/HELPER_FUNCTIONS.md
+++ b/docs/contribution/HELPER_FUNCTIONS.md
@@ -60,6 +60,22 @@ Install scripts are **not** run directly by users; they are invoked by the CT sc
 | CT (update logic) | [ct/endurain.sh](../../ct/endurain.sh) |
 | Install | [install/endurain-install.sh](../../install/endurain-install.sh) |

+### Java + Gradle
+
+**BookLore** - Book management with Java 21 + Gradle + MariaDB + Nginx
+| File | Link |
+| ----------------- | -------------------------------------------------------------- |
+| CT (update logic) | [ct/booklore.sh](../../ct/booklore.sh) |
+| Install | [install/booklore-install.sh](../../install/booklore-install.sh) |
+
+### Pnpm + Meilisearch
+
+**KaraKeep** - Bookmark manager with Pnpm + Meilisearch + Puppeteer
+| File | Link |
+| ----------------- | -------------------------------------------------------------- |
+| CT (update logic) | [ct/karakeep.sh](../../ct/karakeep.sh) |
+| Install | [install/karakeep-install.sh](../../install/karakeep-install.sh) |
+
 ### PHP + MariaDB/MySQL

 **Wallabag** - Read-it-later with PHP + MariaDB + Redis + Nginx
@@ -285,6 +301,43 @@ setup_clickhouse

 ---

+## Advanced Repository Management
+
+### `setup_deb822_repo`
+
+The modern standard (Debian 12+) for adding external repositories. Automatically handles GPG keys and sources.
+
+```bash
+setup_deb822_repo \
+  "nodejs" \
+  "https://deb.nodesource.com/gpgkey/nodesource.gpg.key" \
+  "https://deb.nodesource.com/node_22.x" \
+  "bookworm" \
+  "main"
+```
+
+### `prepare_repository_setup`
+
+A high-level helper that performs three critical tasks before adding a new repo:
+1. Cleans up old repo files matching the names provided.
+2. Removes old GPG keyrings from all standard locations.
+3. Ensures APT is in a working state (fixes locks, runs update).
+
+```bash
+# Clean up old mysql/mariadb artifacts before setup
+prepare_repository_setup "mariadb" "mysql"
+```
+
+### `cleanup_tool_keyrings`
+
+Force-removes GPG keys for specific tools from `/usr/share/keyrings/`, `/etc/apt/keyrings/`, and `/etc/apt/trusted.gpg.d/`.
+
+```bash
+cleanup_tool_keyrings "docker" "kubernetes"
+```
+
+---
+
 ## GitHub Release Helpers

 > **Note**: `fetch_and_deploy_gh_release` is the **preferred method** for downloading GitHub releases. It handles version tracking automatically. Only use `get_latest_github_release` if you need the version number separately.
@@ -345,26 +398,21 @@ RELEASE=$(get_latest_github_release "owner/repo")
 echo "Latest version: $RELEASE"
 ```

-# Examples
-
-fetch_and_deploy_gh_release "bookstack" "BookStackApp/BookStack"
-fetch_and_deploy_gh_release "appname" "owner/repo" "tarball" "latest" "/opt/myapp"
-
-````
-
-**Parameters:**
-| Parameter | Default       | Description                                  |
-| --------- | ------------- | -------------------------------------------- |
-| `name`    | required      | App name (for version tracking)              |
-| `repo`    | required      | GitHub repo (`owner/repo`)                   |
-| `type`    | `tarball`     | Release type: `tarball`, `zipball`, `binary` |
-| `version` | `latest`      | Version tag or `latest`                      |
-| `dest`    | `/opt/[name]` | Destination directory                        |
-
 ---

 ## Tools & Utilities

+### `setup_meilisearch`
+
+Install Meilisearch, a lightning-fast search engine.
+
+```bash
+setup_meilisearch
+
+# Use in script
+$STD php artisan scout:sync-index-settings
+```
+
 ### `setup_yq`

 Install yq YAML processor.
@@ -440,6 +488,15 @@ create_self_signed_cert

 ## Utility Functions

+### `verify_tool_version`
+
+Validate that the installed major version matches the expected version. Useful during upgrades or troubleshooting.
+
+```bash
+# Verify Node.js is version 22
+verify_tool_version "nodejs" "22" "$(node -v | grep -oP '^v\K[0-9]+')"
+```
+
 ### `get_lxc_ip`

 Set the `$LOCAL_IP` variable with the container's IP address.
--- a/docs/contribution/templates_install/AppName-install.sh
+++ b/docs/contribution/templates_install/AppName-install.sh
@@ -30,14 +30,16 @@ msg_ok "Installed Dependencies"
 # Examples (uncomment as needed):
 #
 #   NODE_VERSION="22" setup_nodejs
+#   NODE_VERSION="22" NODE_MODULE="pnpm" setup_nodejs  # Installs pnpm
 #   PYTHON_VERSION="3.13" setup_uv
-#   JAVA_VERSION="17" setup_java
+#   JAVA_VERSION="21" setup_java
 #   GO_VERSION="1.22" setup_go
 #   PHP_VERSION="8.4" PHP_FPM="YES" setup_php
 #   setup_postgresql           # Server only
 #   setup_mariadb              # Server only
+#   setup_meilisearch          # Search engine
 #
-#   Then set up DB and user:
+#   Then set up DB and user (sets $[DB]_DB_PASS):
 #   PG_DB_NAME="myapp" PG_DB_USER="myapp" setup_postgresql_db
 #   MARIADB_DB_NAME="myapp" MARIADB_DB_USER="myapp" setup_mariadb_db

--- a/docs/guides/CONFIGURATION_REFERENCE.md
+++ b/docs/guides/CONFIGURATION_REFERENCE.md
@@ -539,6 +539,25 @@ var_nesting=0    # Nested containers disabled

 ---

+### var_diagnostics
+
+**Type:** Boolean (yes or no)
+**Default:** `yes`
+**Description:** Determines if anonymous telemetry and diagnostic data is sent to Community-Scripts API.
+
+```bash
+var_diagnostics=yes      # Allow telemetry (helps us improve scripts)
+var_diagnostics=no       # Disable all telemetry
+```
+
+**Privacy & Usage:**
+- Data is strictly anonymous (random session ID)
+- Reports success/failure of installations
+- Maps error codes (e.g., APT lock, out of RAM)
+- No user-specific data, hostnames, or secret keys are ever sent
+
+---
+
 ### var_gpu

 **Type:** Boolean/Toggle
@@ -604,13 +623,14 @@ var_keyctl=0     # Keyctl disabled

 ### var_fuse

-**Type:** Boolean (0 or 1)
-**Default:** `0`
+**Type:** Boolean/Toggle
+**Options:** `yes` or `no`
+**Default:** `no`
 **Description:** Enable FUSE filesystem support.

 ```bash
-var_fuse=1       # FUSE enabled
-var_fuse=0       # FUSE disabled
+var_fuse=yes     # FUSE enabled
+var_fuse=no      # FUSE disabled
 ```

 **Required for:**
--- a/docs/guides/UNATTENDED_DEPLOYMENTS.md
+++ b/docs/guides/UNATTENDED_DEPLOYMENTS.md
@@ -220,10 +220,10 @@ done
 # batch-deploy-advanced.sh - Deploy multiple containers with individual configs

 declare -A CONTAINERS=(
-  ["pihole"]="2:1024:8:vmbr0:dns,network"
-  ["qui"]="4:4096:20:vmbr0:media,ui"
+  ["beszel"]="1:512:8:vmbr0:monitoring"
+  ["qui"]="2:1024:10:vmbr0:torrent,ui"
  ["thingsboard"]="6:8192:50:vmbr1:iot,industrial"
-  ["nginx"]="2:2048:10:vmbr0:webserver,proxy"
+  ["dockge"]="2:2048:10:vmbr0:docker,management"
 )

 for app in "${!CONTAINERS[@]}"; do
--- a/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md
+++ b/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md
@@ -60,10 +60,10 @@ export var_gateway="192.168.1.1"
 export var_ip="192.168.1.101"
 export var_vlan="100"
 export var_mtu="9000"
-export var_template_storage="nfs-storage"
+export var_template_storage="ssd-storage"
 export var_container_storage="ssd-storage"
-export ENABLE_FUSE="true"
-export ENABLE_TUN="true"
+export var_fuse="yes"
+export var_tun="yes"
 export SSH="true"

 # Execute build.func
--- a/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md
+++ b/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md
@@ -20,14 +20,19 @@ Complete alphabetical reference of all functions in tools.func with parameters,
 - `setup_nodejs(VERSION)` - Install Node.js and npm
 - `setup_php(VERSION)` - Install PHP-FPM and CLI
 - `setup_python(VERSION)` - Install Python 3 with pip
+- `setup_uv()` - Install Python uv (modern & fast)
 - `setup_ruby(VERSION)` - Install Ruby with gem
 - `setup_golang(VERSION)` - Install Go programming language
+- `setup_java(VERSION)` - Install OpenJDK (Adoptium)

 **Databases**:
- `setup_mariadb()` - Install MariaDB server (distro packages by default)
+- `setup_mariadb()` - Install MariaDB server
+- `setup_mariadb_db()` - Create user/db in MariaDB
 - `setup_postgresql(VERSION)` - Install PostgreSQL
+- `setup_postgresql_db()` - Create user/db in PostgreSQL
 - `setup_mongodb(VERSION)` - Install MongoDB
 - `setup_redis(VERSION)` - Install Redis cache
+- `setup_meilisearch()` - Install Meilisearch engine

 **Web Servers**:
 - `setup_nginx()` - Install Nginx
@@ -44,6 +49,7 @@ Complete alphabetical reference of all functions in tools.func with parameters,
 - `setup_docker_compose()` - Install Docker Compose
 - `setup_composer()` - Install PHP Composer
 - `setup_build_tools()` - Install build-essential
+- `setup_yq()` - Install mikefarah/yq processor

 **Monitoring**:
 - `setup_grafana()` - Install Grafana
@@ -112,15 +118,23 @@ The primary tool for downloading and installing software from GitHub Releases. S
 fetch_and_deploy_gh_release APPREPO TYPE [VERSION] [DEST] [ASSET_PATTERN]
 ```

-**Parameters**:
+**Environment Variables**:
 - `APPREPO`: GitHub repository (e.g., `owner/repo`)
- `TYPE`: Asset type (`binary`, `tarball`, `prebuild`, `singlefile`, `binary_tarball`)
+- `TYPE`: Asset type (`binary`, `tarball`, `prebuild`, `singlefile`)
 - `VERSION`: Specific tag or `latest` (Default: `latest`)
 - `DEST`: Target directory (Default: `/opt/$APP`)
- `ASSET_PATTERN`: Regex or string pattern to match the release asset
+- `ASSET_PATTERN`: Regex or string pattern to match the release asset (Required for `prebuild` and `singlefile`)
+
+**Supported Operation Modes**:
+- `tarball`: Downloads and extracts the source tarball.
+- `binary`: Detects host architecture and installs a `.deb` package using `apt` or `dpkg`.
+- `prebuild`: Downloads and extracts a pre-built binary archive (supports `.tar.gz`, `.zip`, `.tgz`, `.txz`).
+- `singlefile`: Downloads a single binary file to the destination.

 **Environment Variables**:
- `CLEAN_INSTALL=1`: Removes the destination directory before extracting.
+- `CLEAN_INSTALL=1`: Removes all contents of the destination directory before extraction.
+- `DPKG_FORCE_CONFOLD=1`: Forces `dpkg` to keep old config files during package updates.
+- `SYSTEMD_OFFLINE=1`: Used automatically for `.deb` installs to prevent systemd-tmpfiles failures in unprivileged containers.

 **Example**:
 ```bash
@@ -230,19 +244,240 @@ cleanup_repo_metadata
 **Example**:
 ```bash
 cleanup_repo_metadata
->>>>>>> origin/main
 ```

 ---

 ## Tool Installation Functions

-### setup_nodejs(VERSION)
+### setup_nodejs()

-Install Node.js and npm from official repositories.
+Install Node.js and npm from official repositories. Handles legacy version cleanup (nvm) automatically.

 **Signature**:
 ```bash
+setup_nodejs
+```
+
+**Environment Variables**:
+- `NODE_VERSION`: Major version to install (e.g. "20", "22", "24"). Default: "24".
+- `NODE_MODULE`: Optional npm package to install globally during setup (e.g. "pnpm", "yarn").
+
+**Example**:
+```bash
+NODE_VERSION="22" NODE_MODULE="pnpm" setup_nodejs
+```
+
+---
+
+### setup_php()
+
+Install PHP with configurable extensions and FPM/Apache integration.
+
+**Signature**:
+```bash
+setup_php
+```
+
+**Environment Variables**:
+- `PHP_VERSION`: Version to install (e.g. "8.3", "8.4"). Default: "8.4".
+- `PHP_MODULE`: Comma-separated list of additional extensions.
+- `PHP_FPM`: Set to "YES" to install php-fpm.
+- `PHP_APACHE`: Set to "YES" to install libapache2-mod-php.
+
+**Example**:
+```bash
+PHP_VERSION="8.3" PHP_FPM="YES" PHP_MODULE="mysql,xml,zip" setup_php
+```
+
+---
+
+### setup_mariadb_db()
+
+Creates a new MariaDB database and a dedicated user with all privileges. Automatically generates a password if not provided and saves it to a credentials file.
+
+**Environment Variables**:
+- `MARIADB_DB_NAME`: Name of the database (required)
+- `MARIADB_DB_USER`: Name of the database user (required)
+- `MARIADB_DB_PASS`: User password (optional, auto-generated if omitted)
+
+**Example**:
+```bash
+MARIADB_DB_NAME="myapp" MARIADB_DB_USER="myapp_user" setup_mariadb_db
+```
+
+---
+
+### setup_postgresql_db()
+
+Creates a new PostgreSQL database and a dedicated user/role with all privileges. Automatically generates a password if not provided and saves it to a credentials file.
+
+**Environment Variables**:
+- `PG_DB_NAME`: Name of the database (required)
+- `PG_DB_USER`: Name of the database user (required)
+- `PG_DB_PASS`: User password (optional, auto-generated if omitted)
+
+---
+
+### setup_java()
+
+Installs Temurin JDK.
+
+**Signature**:
+```bash
+JAVA_VERSION="21" setup_java
+```
+
+**Parameters**:
+- `JAVA_VERSION` - JDK version (e.g., "17", "21") (default: "21")
+
+**Example**:
+```bash
+JAVA_VERSION="17" setup_java
+```
+
+---
+
+### setup_uv()
+
+Installs `uv` (modern Python package manager).
+
+**Signature**:
+```bash
+PYTHON_VERSION="3.13" setup_uv
+```
+
+**Parameters**:
+- `PYTHON_VERSION` - Optional Python version to pre-install via uv (e.g., "3.12", "3.13")
+
+**Example**:
+```bash
+PYTHON_VERSION="3.13" setup_uv
+```
+
+---
+
+### setup_go()
+
+Installs Go programming language.
+
+**Signature**:
+```bash
+GO_VERSION="1.23" setup_go
+```
+
+**Parameters**:
+- `GO_VERSION` - Go version to install (default: "1.23")
+
+**Example**:
+```bash
+GO_VERSION="1.24" setup_go
+```
+
+---
+
+### setup_yq()
+
+Installs `yq` (YAML processor).
+
+**Signature**:
+```bash
+setup_yq
+```
+
+**Example**:
+```bash
+setup_yq
+```
+
+---
+
+### setup_composer()
+
+Installs PHP Composer.
+
+**Signature**:
+```bash
+setup_composer
+```
+
+**Example**:
+```bash
+setup_composer
+```
+
+---
+
+### setup_meilisearch()
+
+Install and configure Meilisearch search engine.
+
+**Environment Variables**:
+- `MEILISEARCH_BIND`: Address and port to bind to (Default: "127.0.0.1:7700")
+- `MEILISEARCH_ENV`: Environment mode (Default: "production")
+
+---
+
+### setup_yq()
+
+Install the `mikefarah/yq` YAML processor. Removes existing non-compliant versions.
+
+**Example**:
+```bash
+setup_yq
+yq eval '.app.version = "1.0.0"' -i config.yaml
+```
+
+---
+
+### setup_composer()
+
+Install or update the PHP Composer package manager. Handles `COMPOSER_ALLOW_SUPERUSER` automatically and performs self-updates if already installed.
+
+**Example**:
+```bash
+setup_php
+setup_composer
+$STD composer install --no-dev
+```
+
+---
+
+### setup_build_tools()
+
+Install the `build-essential` package suite for compiling software.
+
+---
+
+### setup_uv()
+
+Install the modern Python package manager `uv`. Extremely fast replacement for pip/venv.
+
+**Environment Variables**:
+- `PYTHON_VERSION`: Major.Minor version to ensure is installed.
+
+**Example**:
+```bash
+PYTHON_VERSION="3.12" setup_uv
+uv sync --locked
+```
+
+---
+
+### setup_java()
+
+Install OpenJDK via the Adoptium repository.
+
+**Environment Variables**:
+- `JAVA_VERSION`: Major version to install (e.g. "17", "21"). Default: "21".
+
+**Example**:
+```bash
+JAVA_VERSION="21" setup_java
+```
+
+---
+```bash
 setup_nodejs VERSION
 ```