diff --git a/docs/contribution/CONTRIBUTING.md b/docs/contribution/CONTRIBUTING.md index 5d7896fe2..619204378 100644 --- a/docs/contribution/CONTRIBUTING.md +++ b/docs/contribution/CONTRIBUTING.md @@ -147,6 +147,30 @@ Verify the PR shows ONLY these 3 files: --- +# 🛠️ Developer Mode & Debugging + +When building or testing scripts, you can use the `dev_mode` variable to enable powerful debugging features. These flags can be combined (comma-separated). + +**Usage**: +```bash +# Example: Run with trace and keep the container even if it fails +dev_mode="trace,keep" bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/myapp.sh)" +``` + +### Available Flags: + +| Flag | Description | +| :--- | :--- | +| `trace` | Enables `set -x` for maximum verbosity during execution. | +| `keep` | Prevents the container from being deleted if the build fails. | +| `pause` | Pauses execution at key points (e.g., before customization). | +| `breakpoint` | Allows hardcoded `breakpoint` calls in scripts to drop to a shell. | +| `logs` | Saves detailed build logs to `/var/log/community-scripts/`. | +| `dryrun` | Bypasses actual container creation (limited support). | +| `motd` | Forces an update of the Message of the Day (MOTD). | + +--- + ## 📚 Pages - [CT Template: AppName.sh](https://github.com/community-scripts/ProxmoxVE/blob/main/docs/contribution/templates_ct/AppName.sh) diff --git a/docs/contribution/GUIDE.md b/docs/contribution/GUIDE.md index e2d580976..a014b3cdb 100644 --- a/docs/contribution/GUIDE.md +++ b/docs/contribution/GUIDE.md @@ -433,26 +433,44 @@ $STD apt-get install -y newdependency # 4. Test thoroughly before committing ``` -### Step 3: Update Update Function (if applicable) +### Step 3: The Standard Update Pattern +The `update_script()` function in `ct/appname.sh` should follow a robust pattern: + +1. **Check for updates**: Use `check_for_gh_release` to skip logic if no new version exists. +2. **Stop services**: Stop all relevant services (`systemctl stop appname`). +3. **Backup existing installation**: Move the old folder (e.g., `mv /opt/app /opt/app_bak`). +4. **Deploy new version**: Use `CLEAN_INSTALL=1 fetch_and_deploy_gh_release`. +5. **Restore configuration**: Copy `.env` or config files back from the backup. +6. **Rebuild/Migrate**: Run `npm install`, `composer install`, or DB migrations. +7. **Start services**: Restart services and cleanup the backup. + +**Example from `ct/bookstack.sh`**: ```bash -# Edit: ct/existingapp.sh → update_script() - -# 1. Update GitHub API URL if repo changed -RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | ...) - -# 2. Update backup/restore logic (if structure changed) -# 3. Update cleanup paths - -# 4. Test update on existing installation -``` - -### Step 4: Document Your Changes - -```bash -# Add comment at top of script -# Co-Author: YourUsername -# Updated: YYYY-MM-DD - Description of changes +function update_script() { + if check_for_gh_release "bookstack" "BookStackApp/BookStack"; then + msg_info "Stopping Services" + systemctl stop apache2 + + msg_info "Backing up data" + mv /opt/bookstack /opt/bookstack-backup + + fetch_and_deploy_gh_release "bookstack" "BookStackApp/BookStack" "tarball" + + msg_info "Restoring backup" + cp /opt/bookstack-backup/.env /opt/bookstack/.env + # ... restore uploads ... + + msg_info "Configuring" + cd /opt/bookstack + $STD composer install --no-dev + $STD php artisan migrate --force + + systemctl start apache2 + rm -rf /opt/bookstack-backup + msg_ok "Updated successfully!" + fi +} ``` --- diff --git a/docs/contribution/HELPER_FUNCTIONS.md b/docs/contribution/HELPER_FUNCTIONS.md index e5794bec0..d32ab2474 100644 --- a/docs/contribution/HELPER_FUNCTIONS.md +++ b/docs/contribution/HELPER_FUNCTIONS.md @@ -60,6 +60,22 @@ Install scripts are **not** run directly by users; they are invoked by the CT sc | CT (update logic) | [ct/endurain.sh](../../ct/endurain.sh) | | Install | [install/endurain-install.sh](../../install/endurain-install.sh) | +### Java + Gradle + +**BookLore** - Book management with Java 21 + Gradle + MariaDB + Nginx +| File | Link | +| ----------------- | -------------------------------------------------------------- | +| CT (update logic) | [ct/booklore.sh](../../ct/booklore.sh) | +| Install | [install/booklore-install.sh](../../install/booklore-install.sh) | + +### Pnpm + Meilisearch + +**KaraKeep** - Bookmark manager with Pnpm + Meilisearch + Puppeteer +| File | Link | +| ----------------- | -------------------------------------------------------------- | +| CT (update logic) | [ct/karakeep.sh](../../ct/karakeep.sh) | +| Install | [install/karakeep-install.sh](../../install/karakeep-install.sh) | + ### PHP + MariaDB/MySQL **Wallabag** - Read-it-later with PHP + MariaDB + Redis + Nginx @@ -285,6 +301,43 @@ setup_clickhouse --- +## Advanced Repository Management + +### `setup_deb822_repo` + +The modern standard (Debian 12+) for adding external repositories. Automatically handles GPG keys and sources. + +```bash +setup_deb822_repo \ + "nodejs" \ + "https://deb.nodesource.com/gpgkey/nodesource.gpg.key" \ + "https://deb.nodesource.com/node_22.x" \ + "bookworm" \ + "main" +``` + +### `prepare_repository_setup` + +A high-level helper that performs three critical tasks before adding a new repo: +1. Cleans up old repo files matching the names provided. +2. Removes old GPG keyrings from all standard locations. +3. Ensures APT is in a working state (fixes locks, runs update). + +```bash +# Clean up old mysql/mariadb artifacts before setup +prepare_repository_setup "mariadb" "mysql" +``` + +### `cleanup_tool_keyrings` + +Force-removes GPG keys for specific tools from `/usr/share/keyrings/`, `/etc/apt/keyrings/`, and `/etc/apt/trusted.gpg.d/`. + +```bash +cleanup_tool_keyrings "docker" "kubernetes" +``` + +--- + ## GitHub Release Helpers > **Note**: `fetch_and_deploy_gh_release` is the **preferred method** for downloading GitHub releases. It handles version tracking automatically. Only use `get_latest_github_release` if you need the version number separately. @@ -345,26 +398,21 @@ RELEASE=$(get_latest_github_release "owner/repo") echo "Latest version: $RELEASE" ``` -# Examples - -fetch_and_deploy_gh_release "bookstack" "BookStackApp/BookStack" -fetch_and_deploy_gh_release "appname" "owner/repo" "tarball" "latest" "/opt/myapp" - -```` - -**Parameters:** -| Parameter | Default | Description | -| --------- | ------------- | -------------------------------------------- | -| `name` | required | App name (for version tracking) | -| `repo` | required | GitHub repo (`owner/repo`) | -| `type` | `tarball` | Release type: `tarball`, `zipball`, `binary` | -| `version` | `latest` | Version tag or `latest` | -| `dest` | `/opt/[name]` | Destination directory | - --- ## Tools & Utilities +### `setup_meilisearch` + +Install Meilisearch, a lightning-fast search engine. + +```bash +setup_meilisearch + +# Use in script +$STD php artisan scout:sync-index-settings +``` + ### `setup_yq` Install yq YAML processor. @@ -440,6 +488,15 @@ create_self_signed_cert ## Utility Functions +### `verify_tool_version` + +Validate that the installed major version matches the expected version. Useful during upgrades or troubleshooting. + +```bash +# Verify Node.js is version 22 +verify_tool_version "nodejs" "22" "$(node -v | grep -oP '^v\K[0-9]+')" +``` + ### `get_lxc_ip` Set the `$LOCAL_IP` variable with the container's IP address. diff --git a/docs/contribution/templates_install/AppName-install.sh b/docs/contribution/templates_install/AppName-install.sh index 0d1c2e8d4..58134a113 100644 --- a/docs/contribution/templates_install/AppName-install.sh +++ b/docs/contribution/templates_install/AppName-install.sh @@ -30,14 +30,16 @@ msg_ok "Installed Dependencies" # Examples (uncomment as needed): # # NODE_VERSION="22" setup_nodejs +# NODE_VERSION="22" NODE_MODULE="pnpm" setup_nodejs # Installs pnpm # PYTHON_VERSION="3.13" setup_uv -# JAVA_VERSION="17" setup_java +# JAVA_VERSION="21" setup_java # GO_VERSION="1.22" setup_go # PHP_VERSION="8.4" PHP_FPM="YES" setup_php # setup_postgresql # Server only # setup_mariadb # Server only +# setup_meilisearch # Search engine # -# Then set up DB and user: +# Then set up DB and user (sets $[DB]_DB_PASS): # PG_DB_NAME="myapp" PG_DB_USER="myapp" setup_postgresql_db # MARIADB_DB_NAME="myapp" MARIADB_DB_USER="myapp" setup_mariadb_db diff --git a/docs/guides/CONFIGURATION_REFERENCE.md b/docs/guides/CONFIGURATION_REFERENCE.md index 61aff509c..74728d3b2 100644 --- a/docs/guides/CONFIGURATION_REFERENCE.md +++ b/docs/guides/CONFIGURATION_REFERENCE.md @@ -539,6 +539,25 @@ var_nesting=0 # Nested containers disabled --- +### var_diagnostics + +**Type:** Boolean (yes or no) +**Default:** `yes` +**Description:** Determines if anonymous telemetry and diagnostic data is sent to Community-Scripts API. + +```bash +var_diagnostics=yes # Allow telemetry (helps us improve scripts) +var_diagnostics=no # Disable all telemetry +``` + +**Privacy & Usage:** +- Data is strictly anonymous (random session ID) +- Reports success/failure of installations +- Maps error codes (e.g., APT lock, out of RAM) +- No user-specific data, hostnames, or secret keys are ever sent + +--- + ### var_gpu **Type:** Boolean/Toggle @@ -604,13 +623,14 @@ var_keyctl=0 # Keyctl disabled ### var_fuse -**Type:** Boolean (0 or 1) -**Default:** `0` +**Type:** Boolean/Toggle +**Options:** `yes` or `no` +**Default:** `no` **Description:** Enable FUSE filesystem support. ```bash -var_fuse=1 # FUSE enabled -var_fuse=0 # FUSE disabled +var_fuse=yes # FUSE enabled +var_fuse=no # FUSE disabled ``` **Required for:** diff --git a/docs/guides/UNATTENDED_DEPLOYMENTS.md b/docs/guides/UNATTENDED_DEPLOYMENTS.md index 1df0cc8db..d1b7e7463 100644 --- a/docs/guides/UNATTENDED_DEPLOYMENTS.md +++ b/docs/guides/UNATTENDED_DEPLOYMENTS.md @@ -220,10 +220,10 @@ done # batch-deploy-advanced.sh - Deploy multiple containers with individual configs declare -A CONTAINERS=( - ["pihole"]="2:1024:8:vmbr0:dns,network" - ["qui"]="4:4096:20:vmbr0:media,ui" + ["beszel"]="1:512:8:vmbr0:monitoring" + ["qui"]="2:1024:10:vmbr0:torrent,ui" ["thingsboard"]="6:8192:50:vmbr1:iot,industrial" - ["nginx"]="2:2048:10:vmbr0:webserver,proxy" + ["dockge"]="2:2048:10:vmbr0:docker,management" ) for app in "${!CONTAINERS[@]}"; do diff --git a/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md b/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md index b5ad83d6e..299b2be71 100644 --- a/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md +++ b/docs/misc/build.func/BUILD_FUNC_USAGE_EXAMPLES.md @@ -60,10 +60,10 @@ export var_gateway="192.168.1.1" export var_ip="192.168.1.101" export var_vlan="100" export var_mtu="9000" -export var_template_storage="nfs-storage" +export var_template_storage="ssd-storage" export var_container_storage="ssd-storage" -export ENABLE_FUSE="true" -export ENABLE_TUN="true" +export var_fuse="yes" +export var_tun="yes" export SSH="true" # Execute build.func diff --git a/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md b/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md index 4c8bf762a..029095568 100644 --- a/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md +++ b/docs/misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md @@ -20,14 +20,19 @@ Complete alphabetical reference of all functions in tools.func with parameters, - `setup_nodejs(VERSION)` - Install Node.js and npm - `setup_php(VERSION)` - Install PHP-FPM and CLI - `setup_python(VERSION)` - Install Python 3 with pip +- `setup_uv()` - Install Python uv (modern & fast) - `setup_ruby(VERSION)` - Install Ruby with gem - `setup_golang(VERSION)` - Install Go programming language +- `setup_java(VERSION)` - Install OpenJDK (Adoptium) **Databases**: -- `setup_mariadb()` - Install MariaDB server (distro packages by default) +- `setup_mariadb()` - Install MariaDB server +- `setup_mariadb_db()` - Create user/db in MariaDB - `setup_postgresql(VERSION)` - Install PostgreSQL +- `setup_postgresql_db()` - Create user/db in PostgreSQL - `setup_mongodb(VERSION)` - Install MongoDB - `setup_redis(VERSION)` - Install Redis cache +- `setup_meilisearch()` - Install Meilisearch engine **Web Servers**: - `setup_nginx()` - Install Nginx @@ -44,6 +49,7 @@ Complete alphabetical reference of all functions in tools.func with parameters, - `setup_docker_compose()` - Install Docker Compose - `setup_composer()` - Install PHP Composer - `setup_build_tools()` - Install build-essential +- `setup_yq()` - Install mikefarah/yq processor **Monitoring**: - `setup_grafana()` - Install Grafana @@ -112,15 +118,23 @@ The primary tool for downloading and installing software from GitHub Releases. S fetch_and_deploy_gh_release APPREPO TYPE [VERSION] [DEST] [ASSET_PATTERN] ``` -**Parameters**: +**Environment Variables**: - `APPREPO`: GitHub repository (e.g., `owner/repo`) -- `TYPE`: Asset type (`binary`, `tarball`, `prebuild`, `singlefile`, `binary_tarball`) +- `TYPE`: Asset type (`binary`, `tarball`, `prebuild`, `singlefile`) - `VERSION`: Specific tag or `latest` (Default: `latest`) - `DEST`: Target directory (Default: `/opt/$APP`) -- `ASSET_PATTERN`: Regex or string pattern to match the release asset +- `ASSET_PATTERN`: Regex or string pattern to match the release asset (Required for `prebuild` and `singlefile`) + +**Supported Operation Modes**: +- `tarball`: Downloads and extracts the source tarball. +- `binary`: Detects host architecture and installs a `.deb` package using `apt` or `dpkg`. +- `prebuild`: Downloads and extracts a pre-built binary archive (supports `.tar.gz`, `.zip`, `.tgz`, `.txz`). +- `singlefile`: Downloads a single binary file to the destination. **Environment Variables**: -- `CLEAN_INSTALL=1`: Removes the destination directory before extracting. +- `CLEAN_INSTALL=1`: Removes all contents of the destination directory before extraction. +- `DPKG_FORCE_CONFOLD=1`: Forces `dpkg` to keep old config files during package updates. +- `SYSTEMD_OFFLINE=1`: Used automatically for `.deb` installs to prevent systemd-tmpfiles failures in unprivileged containers. **Example**: ```bash @@ -230,19 +244,240 @@ cleanup_repo_metadata **Example**: ```bash cleanup_repo_metadata ->>>>>>> origin/main ``` --- ## Tool Installation Functions -### setup_nodejs(VERSION) +### setup_nodejs() -Install Node.js and npm from official repositories. +Install Node.js and npm from official repositories. Handles legacy version cleanup (nvm) automatically. **Signature**: ```bash +setup_nodejs +``` + +**Environment Variables**: +- `NODE_VERSION`: Major version to install (e.g. "20", "22", "24"). Default: "24". +- `NODE_MODULE`: Optional npm package to install globally during setup (e.g. "pnpm", "yarn"). + +**Example**: +```bash +NODE_VERSION="22" NODE_MODULE="pnpm" setup_nodejs +``` + +--- + +### setup_php() + +Install PHP with configurable extensions and FPM/Apache integration. + +**Signature**: +```bash +setup_php +``` + +**Environment Variables**: +- `PHP_VERSION`: Version to install (e.g. "8.3", "8.4"). Default: "8.4". +- `PHP_MODULE`: Comma-separated list of additional extensions. +- `PHP_FPM`: Set to "YES" to install php-fpm. +- `PHP_APACHE`: Set to "YES" to install libapache2-mod-php. + +**Example**: +```bash +PHP_VERSION="8.3" PHP_FPM="YES" PHP_MODULE="mysql,xml,zip" setup_php +``` + +--- + +### setup_mariadb_db() + +Creates a new MariaDB database and a dedicated user with all privileges. Automatically generates a password if not provided and saves it to a credentials file. + +**Environment Variables**: +- `MARIADB_DB_NAME`: Name of the database (required) +- `MARIADB_DB_USER`: Name of the database user (required) +- `MARIADB_DB_PASS`: User password (optional, auto-generated if omitted) + +**Example**: +```bash +MARIADB_DB_NAME="myapp" MARIADB_DB_USER="myapp_user" setup_mariadb_db +``` + +--- + +### setup_postgresql_db() + +Creates a new PostgreSQL database and a dedicated user/role with all privileges. Automatically generates a password if not provided and saves it to a credentials file. + +**Environment Variables**: +- `PG_DB_NAME`: Name of the database (required) +- `PG_DB_USER`: Name of the database user (required) +- `PG_DB_PASS`: User password (optional, auto-generated if omitted) + +--- + +### setup_java() + +Installs Temurin JDK. + +**Signature**: +```bash +JAVA_VERSION="21" setup_java +``` + +**Parameters**: +- `JAVA_VERSION` - JDK version (e.g., "17", "21") (default: "21") + +**Example**: +```bash +JAVA_VERSION="17" setup_java +``` + +--- + +### setup_uv() + +Installs `uv` (modern Python package manager). + +**Signature**: +```bash +PYTHON_VERSION="3.13" setup_uv +``` + +**Parameters**: +- `PYTHON_VERSION` - Optional Python version to pre-install via uv (e.g., "3.12", "3.13") + +**Example**: +```bash +PYTHON_VERSION="3.13" setup_uv +``` + +--- + +### setup_go() + +Installs Go programming language. + +**Signature**: +```bash +GO_VERSION="1.23" setup_go +``` + +**Parameters**: +- `GO_VERSION` - Go version to install (default: "1.23") + +**Example**: +```bash +GO_VERSION="1.24" setup_go +``` + +--- + +### setup_yq() + +Installs `yq` (YAML processor). + +**Signature**: +```bash +setup_yq +``` + +**Example**: +```bash +setup_yq +``` + +--- + +### setup_composer() + +Installs PHP Composer. + +**Signature**: +```bash +setup_composer +``` + +**Example**: +```bash +setup_composer +``` + +--- + +### setup_meilisearch() + +Install and configure Meilisearch search engine. + +**Environment Variables**: +- `MEILISEARCH_BIND`: Address and port to bind to (Default: "127.0.0.1:7700") +- `MEILISEARCH_ENV`: Environment mode (Default: "production") + +--- + +### setup_yq() + +Install the `mikefarah/yq` YAML processor. Removes existing non-compliant versions. + +**Example**: +```bash +setup_yq +yq eval '.app.version = "1.0.0"' -i config.yaml +``` + +--- + +### setup_composer() + +Install or update the PHP Composer package manager. Handles `COMPOSER_ALLOW_SUPERUSER` automatically and performs self-updates if already installed. + +**Example**: +```bash +setup_php +setup_composer +$STD composer install --no-dev +``` + +--- + +### setup_build_tools() + +Install the `build-essential` package suite for compiling software. + +--- + +### setup_uv() + +Install the modern Python package manager `uv`. Extremely fast replacement for pip/venv. + +**Environment Variables**: +- `PYTHON_VERSION`: Major.Minor version to ensure is installed. + +**Example**: +```bash +PYTHON_VERSION="3.12" setup_uv +uv sync --locked +``` + +--- + +### setup_java() + +Install OpenJDK via the Adoptium repository. + +**Environment Variables**: +- `JAVA_VERSION`: Major version to install (e.g. "17", "21"). Default: "21". + +**Example**: +```bash +JAVA_VERSION="21" setup_java +``` + +--- +```bash setup_nodejs VERSION ```