Compare commits

..

8 Commits

Author SHA1 Message Date
CanbiZ (MickLesk)
e222a2718c Anonymize IP addresses in get_full_log
Mask IPv4 addresses in logs when collecting full log output: added a sed step that replaces the last two octets with "x.x" to avoid exposing full IPs (GDPR). Also updated the comment to reflect anonymization; existing steps that strip carriage returns and ANSI escape sequences remain in place before truncating with head -c.
2026-02-23 14:22:51 +01:00
CanbiZ (MickLesk)
dbff5db95a Include full log in error telemetry
Use get_full_log (up to 120KB) to populate the error telemetry field so the API receives the full installation trace; fall back to get_error_text (last ~20 lines) if the full log is empty. Removed collection and inclusion of a separate install_log field from the JSON payloads and simplified the retry payloads/comments accordingly. The change ensures error reports contain the complete trace while avoiding duplicate large log fields and keeps graceful failure handling (get_full_log || true).
2026-02-23 14:15:42 +01:00
CanbiZ (MickLesk)
272ce20a82 Enhance telemetry, signal handling, and logs
Improve failure telemetry and signal handling across the installer: add get_full_log() to collect/strip/truncate install logs and include them in API payloads with a truncated retry; add CONTAINER_INSTALLING flag around lxc-attach and stop containers on abort to avoid orphaned "installing/configuring" records; introduce _send_abort_telemetry() (curl fallback for container context) and _stop_container_if_installing() helpers; centralize and simplify EXIT/ERR/INT/TERM/HUP traps and handlers (including a new on_hangup handler) and update VM scripts to report numeric exit codes. Also ensure best-effort log collection is performed and tweak error categorization for certain signals.
2026-02-23 14:14:14 +01:00
CanbiZ (MickLesk)
a13caec262 allow Frigate's Intel media packages to overwrite files from system GPU driver packages 2026-02-23 13:25:20 +01:00
community-scripts-pr-app[bot]
df80c849f2 Update CHANGELOG.md (#12209)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-23 12:22:33 +00:00
CanbiZ (MickLesk)
69de9fa57e Improve error handling and logging for LXC builds (#12208)
Prevent host-side error_handler from being triggered during in-container install/recovery by delaying re-enabling set -Eeuo pipefail and the ERR trap in misc/build.func until after install/recovery completes; add explanatory comments. Update misc/error_handler.func to fall back to BUILD_LOG if container-internal log path is unavailable, show the last 20 log lines when present, refine container vs host detection (check INSTALL_LOG file and /root), copy INSTALL_LOG into /root and write a .failed flag with the exit code for host-side detection, and ensure full-log output and container removal prompt are shown appropriately in host context. Tweak misc/core.func silent() output to include a "Full log" path and adjust formatting.
2026-02-23 13:22:09 +01:00
community-scripts-pr-app[bot]
695a6ce29b chore: update github-versions.json (#12206)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-23 12:14:28 +00:00
CanbiZ (MickLesk)
1976a1715c bookworm arc support 2026-02-23 13:08:39 +01:00
23 changed files with 334 additions and 160 deletions

View File

@@ -413,6 +413,12 @@ Exercise vigilance regarding copycat or coat-tailing sites that seek to exploit
- Frigate v16.4 [@MickLesk](https://github.com/MickLesk) ([#11887](https://github.com/community-scripts/ProxmoxVE/pull/11887))
### 💾 Core
- #### 🔧 Refactor
- core: Improve error handling and logging for LXC builds [@MickLesk](https://github.com/MickLesk) ([#12208](https://github.com/community-scripts/ProxmoxVE/pull/12208))
## 2026-02-22
### 🆕 New Scripts

View File

@@ -1,5 +1,5 @@
{
"generated": "2026-02-23T06:26:10Z",
"generated": "2026-02-23T12:14:19Z",
"versions": [
{
"slug": "2fauth",
@@ -200,9 +200,9 @@
{
"slug": "cleanuparr",
"repo": "Cleanuparr/Cleanuparr",
"version": "v2.7.0",
"version": "v2.7.1",
"pinned": false,
"date": "2026-02-22T18:17:27Z"
"date": "2026-02-23T09:58:13Z"
},
{
"slug": "cloudreve",
@@ -421,6 +421,13 @@
"pinned": false,
"date": "2026-01-25T18:20:14Z"
},
{
"slug": "frigate",
"repo": "blakeblackshear/frigate",
"version": "v0.16.4",
"pinned": true,
"date": "2026-01-29T00:42:14Z"
},
{
"slug": "gatus",
"repo": "TwiN/gatus",
@@ -578,9 +585,9 @@
{
"slug": "huntarr",
"repo": "plexguide/Huntarr.io",
"version": "9.3.7",
"version": "9.4.1",
"pinned": false,
"date": "2026-02-19T01:03:53Z"
"date": "2026-02-23T08:46:37Z"
},
{
"slug": "immich-public-proxy",
@@ -1663,9 +1670,9 @@
{
"slug": "wanderer",
"repo": "meilisearch/meilisearch",
"version": "v1.35.1",
"version": "v1.36.0",
"pinned": false,
"date": "2026-02-16T17:01:17Z"
"date": "2026-02-23T08:13:32Z"
},
{
"slug": "warracker",

View File

@@ -175,7 +175,10 @@ cp -a /opt/frigate/docker/main/rootfs/. /
sed -i '/^.*unset DEBIAN_FRONTEND.*$/d' /opt/frigate/docker/main/install_deps.sh
echo "libedgetpu1-max libedgetpu/accepted-eula boolean true" | debconf-set-selections
echo "libedgetpu1-max libedgetpu/install-confirm-max boolean true" | debconf-set-selections
# Allow Frigate's Intel media packages to overwrite files from system GPU driver packages
echo 'force-overwrite' >/etc/dpkg/dpkg.cfg.d/force-overwrite
$STD bash /opt/frigate/docker/main/install_deps.sh
rm -f /etc/dpkg/dpkg.cfg.d/force-overwrite
$STD pip3 install -U /wheels/*.whl
ldconfig
msg_ok "Installed HailoRT Runtime"

View File

@@ -350,6 +350,55 @@ get_error_text() {
fi
}
# ------------------------------------------------------------------------------
# get_full_log()
#
# - Returns the FULL installation log (build + install combined)
# - Calls ensure_log_on_host() to pull container log if needed
# - Strips ANSI escape codes and carriage returns
# - Truncates to max_bytes (default: 120KB) to stay within API limits
# - Used for the error telemetry field (full trace instead of 20 lines)
# ------------------------------------------------------------------------------
get_full_log() {
local max_bytes="${1:-122880}" # 120KB default
local logfile=""
# Ensure logs are available on host (pulls from container if needed)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
ensure_log_on_host
fi
# Try combined log first (most complete)
if [[ -n "${CTID:-}" && -n "${SESSION_ID:-}" ]]; then
local combined_log="/tmp/${NSAPP:-lxc}-${CTID}-${SESSION_ID}.log"
if [[ -s "$combined_log" ]]; then
logfile="$combined_log"
fi
fi
# Fall back to INSTALL_LOG
if [[ -z "$logfile" || ! -s "$logfile" ]]; then
if [[ -n "${INSTALL_LOG:-}" && -s "${INSTALL_LOG}" ]]; then
logfile="$INSTALL_LOG"
fi
fi
# Fall back to BUILD_LOG
if [[ -z "$logfile" || ! -s "$logfile" ]]; then
if [[ -n "${BUILD_LOG:-}" && -s "${BUILD_LOG}" ]]; then
logfile="$BUILD_LOG"
fi
fi
if [[ -n "$logfile" && -s "$logfile" ]]; then
# Strip ANSI codes, carriage returns, and anonymize IP addresses (GDPR)
sed 's/\r$//' "$logfile" 2>/dev/null |
sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' |
sed -E 's/([0-9]{1,3}\.)[0-9]{1,3}\.[0-9]{1,3}/\1x.x/g' |
head -c "$max_bytes"
fi
}
# ------------------------------------------------------------------------------
# build_error_string()
#
@@ -782,11 +831,15 @@ post_update_to_api() {
else
exit_code=1
fi
# Get log lines and build structured error string
local error_text=""
error_text=$(get_error_text)
# Get full installation log for error field
local log_text=""
log_text=$(get_full_log 122880) || true # 120KB max
if [[ -z "$log_text" ]]; then
# Fallback to last 20 lines
log_text=$(get_error_text)
fi
local full_error
full_error=$(build_error_string "$exit_code" "$error_text")
full_error=$(build_error_string "$exit_code" "$log_text")
error=$(json_escape "$full_error")
short_error=$(json_escape "$(explain_exit_code "$exit_code")")
error_category=$(categorize_error "$exit_code")
@@ -807,7 +860,7 @@ post_update_to_api() {
local http_code=""
# ── Attempt 1: Full payload with complete error text ──
# ── Attempt 1: Full payload with complete error text (includes full log) ──
local JSON_PAYLOAD
JSON_PAYLOAD=$(
cat <<EOF
@@ -969,16 +1022,16 @@ categorize_error() {
# Python environment errors
# (already covered: 160-162 under dependency)
# Aborted by user
130) echo "aborted" ;;
# Aborted by user (SIGHUP=terminal closed, SIGINT=Ctrl+C, SIGTERM=killed)
129 | 130 | 143) echo "user_aborted" ;;
# Resource errors (OOM, SIGKILL, SIGABRT)
134 | 137) echo "resource" ;;
# Signal/Process errors (SIGTERM, SIGPIPE, SIGSEGV)
139 | 141 | 143) echo "signal" ;;
# Signal/Process errors (SIGPIPE, SIGSEGV)
139 | 141) echo "signal" ;;
# Shell errors (general error, syntax error)
# Shell errors (general error, syntax error)
1 | 2) echo "shell" ;;
# Default - truly unknown

View File

@@ -4052,11 +4052,19 @@ EOF'
set +Eeuo pipefail # Disable ALL error handling temporarily
trap - ERR # Remove ERR trap completely
# Signal handlers use this flag to stop the container on abort (SIGHUP/SIGINT/SIGTERM)
# Without this, SSH disconnects leave the container running as an orphan process
# that sends "configuring" status AFTER the host already reported "failed"
export CONTAINER_INSTALLING=true
lxc-attach -n "$CTID" -- bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/install/${var_install}.sh)"
local lxc_exit=$?
set -Eeuo pipefail # Re-enable error handling
trap 'error_handler' ERR # Restore ERR trap
unset CONTAINER_INSTALLING
# Keep error handling DISABLED during failure detection and recovery
# Re-enabling it here would cause any pct exec/pull failure to trigger
# error_handler() on the host, bypassing the recovery menu entirely
# Check for error flag file in container (more reliable than lxc-attach exit code)
local install_exit_code=0
@@ -4484,6 +4492,10 @@ EOF'
exit $install_exit_code
fi
# Re-enable error handling after successful install or recovery menu completion
set -Eeuo pipefail
trap 'error_handler' ERR
}
destroy_lxc() {
@@ -5561,9 +5573,13 @@ api_exit_script() {
# ALWAYS send telemetry FIRST - ensure status is reported even if
# ensure_log_on_host hangs (e.g. pct pull on dead container)
post_update_to_api "failed" "$exit_code" 2>/dev/null || true
# Best-effort log collection with timeout (non-critical after telemetry is sent)
# Best-effort log collection (non-critical after telemetry is sent)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
timeout 10 bash -c 'ensure_log_on_host' 2>/dev/null || true
ensure_log_on_host 2>/dev/null || true
fi
# Stop orphaned container if we're in the install phase
if [[ "${CONTAINER_INSTALLING:-}" == "true" && -n "${CTID:-}" ]] && command -v pct &>/dev/null; then
pct stop "$CTID" 2>/dev/null || true
fi
elif [[ "${POST_TO_API_DONE:-}" == "true" && "${POST_UPDATE_DONE:-}" != "true" ]]; then
# Script exited with 0 but never sent a completion status
@@ -5575,7 +5591,7 @@ api_exit_script() {
if command -v pveversion >/dev/null 2>&1; then
trap 'api_exit_script' EXIT
fi
trap 'local _ec=$?; if [[ $_ec -ne 0 ]]; then post_update_to_api "failed" "$_ec" 2>/dev/null || true; timeout 10 bash -c "ensure_log_on_host" 2>/dev/null || true; fi' ERR
trap 'post_update_to_api "failed" "129" 2>/dev/null || true; timeout 10 bash -c "ensure_log_on_host" 2>/dev/null || true; exit 129' SIGHUP
trap 'post_update_to_api "failed" "130" 2>/dev/null || true; timeout 10 bash -c "ensure_log_on_host" 2>/dev/null || true; exit 130' SIGINT
trap 'post_update_to_api "failed" "143" 2>/dev/null || true; timeout 10 bash -c "ensure_log_on_host" 2>/dev/null || true; exit 143' SIGTERM
trap 'local _ec=$?; if [[ $_ec -ne 0 ]]; then post_update_to_api "failed" "$_ec" 2>/dev/null || true; if declare -f ensure_log_on_host &>/dev/null; then ensure_log_on_host 2>/dev/null || true; fi; fi' ERR
trap 'post_update_to_api "failed" "129" 2>/dev/null || true; if [[ -n "${CTID:-}" ]] && command -v pct &>/dev/null; then pct stop "$CTID" 2>/dev/null || true; fi; exit 129' SIGHUP
trap 'post_update_to_api "failed" "130" 2>/dev/null || true; if [[ -n "${CTID:-}" ]] && command -v pct &>/dev/null; then pct stop "$CTID" 2>/dev/null || true; fi; exit 130' SIGINT
trap 'post_update_to_api "failed" "143" 2>/dev/null || true; if [[ -n "${CTID:-}" ]] && command -v pct &>/dev/null; then pct stop "$CTID" 2>/dev/null || true; fi; exit 143' SIGTERM

View File

@@ -524,7 +524,8 @@ silent() {
if [[ -s "$logfile" ]]; then
echo -e "\n${TAB}--- Last 10 lines of log ---"
tail -n 10 "$logfile"
echo -e "${TAB}-----------------------------------\n"
echo -e "${TAB}-----------------------------------"
echo -e "${TAB}📋 Full log: ${logfile}\n"
fi
exit "$rc"

View File

@@ -208,6 +208,10 @@ error_handler() {
# This ensures we capture failures that occur before/after container exists
if declare -f post_update_to_api &>/dev/null; then
post_update_to_api "failed" "$exit_code" 2>/dev/null || true
else
# Container context: post_update_to_api not available (api.func not sourced)
# Send status directly via curl so container failures are never lost
_send_abort_telemetry "$exit_code" 2>/dev/null || true
fi
# Use msg_error if available, fallback to echo
@@ -236,67 +240,54 @@ error_handler() {
active_log="$SILENT_LOGFILE"
fi
# If active_log points to a container-internal path that doesn't exist on host,
# fall back to BUILD_LOG (host-side log)
if [[ -n "$active_log" && ! -s "$active_log" && -n "${BUILD_LOG:-}" && -s "${BUILD_LOG}" ]]; then
active_log="$BUILD_LOG"
fi
# Show last log lines if available
if [[ -n "$active_log" && -s "$active_log" ]]; then
echo -e "\n${TAB}--- Last 20 lines of log ---"
tail -n 20 "$active_log"
echo -e "${TAB}-----------------------------------\n"
fi
# Detect context: Container (INSTALL_LOG set + /root exists) vs Host (BUILD_LOG)
if [[ -n "${INSTALL_LOG:-}" && -d /root ]]; then
# CONTAINER CONTEXT: Copy log and create flag file for host
local container_log="/root/.install-${SESSION_ID:-error}.log"
cp "$active_log" "$container_log" 2>/dev/null || true
# Detect context: Container (INSTALL_LOG set + inside container /root) vs Host
if [[ -n "${INSTALL_LOG:-}" && -f "${INSTALL_LOG:-}" && -d /root ]]; then
# CONTAINER CONTEXT: Copy log and create flag file for host
local container_log="/root/.install-${SESSION_ID:-error}.log"
cp "${INSTALL_LOG}" "$container_log" 2>/dev/null || true
# Create error flag file with exit code for host detection
echo "$exit_code" >"/root/.install-${SESSION_ID:-error}.failed" 2>/dev/null || true
# Log path is shown by host as combined log - no need to show container path
else
# HOST CONTEXT: Show local log path and offer container cleanup
# Create error flag file with exit code for host detection
echo "$exit_code" >"/root/.install-${SESSION_ID:-error}.failed" 2>/dev/null || true
# Log path is shown by host as combined log - no need to show container path
else
# HOST CONTEXT: Show local log path and offer container cleanup
if [[ -n "$active_log" && -s "$active_log" ]]; then
if declare -f msg_custom >/dev/null 2>&1; then
msg_custom "📋" "${YW}" "Full log: ${active_log}"
else
echo -e "${YW}Full log:${CL} ${BL}${active_log}${CL}"
fi
fi
# Offer to remove container if it exists (build errors after container creation)
if [[ -n "${CTID:-}" ]] && command -v pct &>/dev/null && pct status "$CTID" &>/dev/null; then
echo ""
if declare -f msg_custom >/dev/null 2>&1; then
echo -en "${TAB}${TAB}${YW}Remove broken container ${CTID}? (Y/n) [auto-remove in 60s]: ${CL}"
else
echo -en "${YW}Remove broken container ${CTID}? (Y/n) [auto-remove in 60s]: ${CL}"
fi
# Offer to remove container if it exists (build errors after container creation)
if [[ -n "${CTID:-}" ]] && command -v pct &>/dev/null && pct status "$CTID" &>/dev/null; then
echo ""
if declare -f msg_custom >/dev/null 2>&1; then
echo -en "${TAB}${TAB}${YW}Remove broken container ${CTID}? (Y/n) [auto-remove in 60s]: ${CL}"
else
echo -en "${YW}Remove broken container ${CTID}? (Y/n) [auto-remove in 60s]: ${CL}"
fi
if read -t 60 -r response; then
if [[ -z "$response" || "$response" =~ ^[Yy]$ ]]; then
echo ""
if declare -f msg_info >/dev/null 2>&1; then
msg_info "Removing container ${CTID}"
else
echo -e "${YW}Removing container ${CTID}${CL}"
fi
pct stop "$CTID" &>/dev/null || true
pct destroy "$CTID" &>/dev/null || true
if declare -f msg_ok >/dev/null 2>&1; then
msg_ok "Container ${CTID} removed"
else
echo -e "${GN}${CL} Container ${CTID} removed"
fi
elif [[ "$response" =~ ^[Nn]$ ]]; then
echo ""
if declare -f msg_warn >/dev/null 2>&1; then
msg_warn "Container ${CTID} kept for debugging"
else
echo -e "${YW}Container ${CTID} kept for debugging${CL}"
fi
fi
else
# Timeout - auto-remove
if read -t 60 -r response; then
if [[ -z "$response" || "$response" =~ ^[Yy]$ ]]; then
echo ""
if declare -f msg_info >/dev/null 2>&1; then
msg_info "No response - removing container ${CTID}"
msg_info "Removing container ${CTID}"
else
echo -e "${YW}No response - removing container ${CTID}${CL}"
echo -e "${YW}Removing container ${CTID}${CL}"
fi
pct stop "$CTID" &>/dev/null || true
pct destroy "$CTID" &>/dev/null || true
@@ -305,13 +296,35 @@ error_handler() {
else
echo -e "${GN}${CL} Container ${CTID} removed"
fi
elif [[ "$response" =~ ^[Nn]$ ]]; then
echo ""
if declare -f msg_warn >/dev/null 2>&1; then
msg_warn "Container ${CTID} kept for debugging"
else
echo -e "${YW}Container ${CTID} kept for debugging${CL}"
fi
fi
else
# Timeout - auto-remove
echo ""
if declare -f msg_info >/dev/null 2>&1; then
msg_info "No response - removing container ${CTID}"
else
echo -e "${YW}No response - removing container ${CTID}${CL}"
fi
pct stop "$CTID" &>/dev/null || true
pct destroy "$CTID" &>/dev/null || true
if declare -f msg_ok >/dev/null 2>&1; then
msg_ok "Container ${CTID} removed"
else
echo -e "${GN}${CL} Container ${CTID} removed"
fi
fi
# Force one final status update attempt after cleanup
# This ensures status is updated even if the first attempt failed (e.g., HTTP 400)
if declare -f post_update_to_api &>/dev/null; then
post_update_to_api "failed" "$exit_code" "force"
fi
# Force one final status update attempt after cleanup
# This ensures status is updated even if the first attempt failed (e.g., HTTP 400)
if declare -f post_update_to_api &>/dev/null; then
post_update_to_api "failed" "$exit_code" "force"
fi
fi
fi
@@ -320,40 +333,97 @@ error_handler() {
}
# ==============================================================================
# SECTION 3: SIGNAL HANDLERS
# SECTION 3: TELEMETRY & CLEANUP HELPERS FOR SIGNAL HANDLERS
# ==============================================================================
# ------------------------------------------------------------------------------
# _send_abort_telemetry()
#
# - Sends failure/abort status to telemetry API
# - Works in BOTH host context (post_update_to_api available) and
# container context (only curl available, api.func not sourced)
# - Container context is critical: without this, container-side failures
# and signal exits are never reported, leaving records stuck in
# "installing" or "configuring" forever
# - Arguments: $1 = exit_code
# ------------------------------------------------------------------------------
_send_abort_telemetry() {
local exit_code="${1:-1}"
# Try full API function first (host context - api.func sourced)
if declare -f post_update_to_api &>/dev/null; then
post_update_to_api "failed" "$exit_code" 2>/dev/null || true
return
fi
# Fallback: direct curl (container context - api.func NOT sourced)
# This is the ONLY way containers can report failures to telemetry
command -v curl &>/dev/null || return 0
[[ "${DIAGNOSTICS:-no}" == "no" ]] && return 0
[[ -z "${RANDOM_UUID:-}" ]] && return 0
curl -fsS -m 5 -X POST "${TELEMETRY_URL:-https://telemetry.community-scripts.org/telemetry}" \
-H "Content-Type: application/json" \
-d "{\"random_id\":\"${RANDOM_UUID}\",\"execution_id\":\"${EXECUTION_ID:-${RANDOM_UUID}}\",\"type\":\"${TELEMETRY_TYPE:-lxc}\",\"nsapp\":\"${NSAPP:-${app:-unknown}}\",\"status\":\"failed\",\"exit_code\":${exit_code}}" &>/dev/null || true
}
# ------------------------------------------------------------------------------
# _stop_container_if_installing()
#
# - Stops the LXC container if we're in the install phase
# - Prevents orphaned container processes when the host exits due to a signal
# (SSH disconnect, Ctrl+C, SIGTERM) — without this, the container keeps
# running and may send "configuring" status AFTER the host already sent
# "failed", leaving records permanently stuck in "configuring"
# - Only acts when:
# * CONTAINER_INSTALLING flag is set (during lxc-attach in build_container)
# * CTID is set (container was created)
# * pct command is available (we're on the Proxmox host, not inside a container)
# - Does NOT destroy the container — just stops it for potential debugging
# ------------------------------------------------------------------------------
_stop_container_if_installing() {
[[ "${CONTAINER_INSTALLING:-}" == "true" ]] || return 0
[[ -n "${CTID:-}" ]] || return 0
command -v pct &>/dev/null || return 0
pct stop "$CTID" 2>/dev/null || true
}
# ==============================================================================
# SECTION 4: SIGNAL HANDLERS
# ==============================================================================
# ------------------------------------------------------------------------------
# on_exit()
#
# - EXIT trap handler
# - Cleans up lock files if lockfile variable is set
# - Exits with captured exit code
# - Always runs on script termination (success or failure)
# - For signal exits (>128): sends telemetry FIRST before log collection
# to prevent pct pull hangs from blocking status updates
# - EXIT trap handler — runs on EVERY script termination
# - Catches orphaned "installing"/"configuring" records:
# * If post_to_api sent "installing" but post_update_to_api never ran
# * Reports final status to prevent records stuck forever
# - Best-effort log collection for failed installs
# - Stops orphaned container processes on failure
# - Cleans up lock files
# ------------------------------------------------------------------------------
on_exit() {
local exit_code=$?
# Report orphaned "installing" records to telemetry API
# Catches ALL exit paths: errors (non-zero), signals, AND clean exits where
# post_to_api was called ("installing" sent) but post_update_to_api was never called
# Catches ALL exit paths: errors, signals, AND clean exits where
# post_to_api was called but post_update_to_api was never called
if [[ "${POST_TO_API_DONE:-}" == "true" && "${POST_UPDATE_DONE:-}" != "true" ]]; then
if declare -f post_update_to_api >/dev/null 2>&1; then
# ALWAYS send telemetry FIRST - ensure status is reported even if
# ensure_log_on_host hangs (e.g. pct pull on dead/unresponsive container)
if [[ $exit_code -ne 0 ]]; then
post_update_to_api "failed" "$exit_code" 2>/dev/null || true
else
# exit_code=0 is never an error — report as success
post_update_to_api "done" "0" 2>/dev/null || true
fi
# Best-effort log collection with timeout (non-critical after telemetry is sent)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
timeout 10 bash -c 'ensure_log_on_host' 2>/dev/null || true
fi
if [[ $exit_code -ne 0 ]]; then
_send_abort_telemetry "$exit_code"
elif declare -f post_update_to_api >/dev/null 2>&1; then
post_update_to_api "done" "0" 2>/dev/null || true
fi
fi
# Best-effort log collection on failure (non-critical, telemetry already sent)
if [[ $exit_code -ne 0 ]] && declare -f ensure_log_on_host >/dev/null 2>&1; then
ensure_log_on_host 2>/dev/null || true
fi
# Stop orphaned container if we're in the install phase and exiting with error
if [[ $exit_code -ne 0 ]]; then
_stop_container_if_installing
fi
[[ -n "${lockfile:-}" && -e "$lockfile" ]] && rm -f "$lockfile"
exit "$exit_code"
}
@@ -362,22 +432,13 @@ on_exit() {
# on_interrupt()
#
# - SIGINT (Ctrl+C) trap handler
# - Reports to telemetry FIRST (time-critical: container may be dying)
# - Displays "Interrupted by user" message
# - Reports status FIRST (time-critical: container may be dying)
# - Stops orphaned container to prevent "configuring" ghost records
# - Exits with code 130 (128 + SIGINT=2)
# - Output redirected to /dev/null fallback to prevent SIGPIPE on closed terminals
# ------------------------------------------------------------------------------
on_interrupt() {
# CRITICAL: Send telemetry FIRST before any cleanup or output
# If ensure_log_on_host hangs (e.g. pct pull on dying container),
# the status update would never be sent, leaving records stuck in "installing"
if declare -f post_update_to_api >/dev/null 2>&1; then
post_update_to_api "failed" "130" 2>/dev/null || true
fi
# Best-effort log collection with timeout (non-critical after telemetry is sent)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
timeout 10 bash -c 'ensure_log_on_host' 2>/dev/null || true
fi
_send_abort_telemetry "130"
_stop_container_if_installing
if declare -f msg_error >/dev/null 2>&1; then
msg_error "Interrupted by user (SIGINT)" 2>/dev/null || true
else
@@ -390,23 +451,13 @@ on_interrupt() {
# on_terminate()
#
# - SIGTERM trap handler
# - Reports to telemetry FIRST (time-critical: process being killed)
# - Displays "Terminated by signal" message
# - Reports status FIRST (time-critical: process being killed)
# - Stops orphaned container to prevent "configuring" ghost records
# - Exits with code 143 (128 + SIGTERM=15)
# - Triggered by external process termination
# - Output redirected to /dev/null fallback to prevent SIGPIPE on closed terminals
# ------------------------------------------------------------------------------
on_terminate() {
# CRITICAL: Send telemetry FIRST before any cleanup or output
# Same rationale as on_interrupt: ensure status gets reported even if
# ensure_log_on_host hangs or terminal is already closed
if declare -f post_update_to_api >/dev/null 2>&1; then
post_update_to_api "failed" "143" 2>/dev/null || true
fi
# Best-effort log collection with timeout (non-critical after telemetry is sent)
if declare -f ensure_log_on_host >/dev/null 2>&1; then
timeout 10 bash -c 'ensure_log_on_host' 2>/dev/null || true
fi
_send_abort_telemetry "143"
_stop_container_if_installing
if declare -f msg_error >/dev/null 2>&1; then
msg_error "Terminated by signal (SIGTERM)" 2>/dev/null || true
else
@@ -415,8 +466,25 @@ on_terminate() {
exit 143
}
# ------------------------------------------------------------------------------
# on_hangup()
#
# - SIGHUP trap handler (SSH disconnect, terminal closed)
# - CRITICAL: This was previously MISSING from catch_errors(), causing
# container processes to become orphans on SSH disconnect — the #1 cause
# of records stuck in "installing" and "configuring" states
# - Reports status via direct curl (terminal is already closed, no output)
# - Stops orphaned container to prevent ghost records
# - Exits with code 129 (128 + SIGHUP=1)
# ------------------------------------------------------------------------------
on_hangup() {
_send_abort_telemetry "129"
_stop_container_if_installing
exit 129
}
# ==============================================================================
# SECTION 4: INITIALIZATION
# SECTION 5: INITIALIZATION
# ==============================================================================
# ------------------------------------------------------------------------------
@@ -428,10 +496,11 @@ on_terminate() {
# * set -o pipefail: Pipeline fails if any command fails
# * set -u: (optional) Exit on undefined variable (if STRICT_UNSET=1)
# - Sets up traps:
# * ERR → error_handler
# * EXIT → on_exit
# * INT → on_interrupt
# * TERM → on_terminate
# * ERR → error_handler (script errors)
# * EXIT → on_exit (any termination — cleanup + orphan detection)
# * INT → on_interrupt (Ctrl+C)
# * TERM → on_terminate (kill / systemd stop)
# * HUP → on_hangup (SSH disconnect / terminal closed)
# - Call this function early in every script
# ------------------------------------------------------------------------------
catch_errors() {
@@ -444,4 +513,5 @@ catch_errors() {
trap on_exit EXIT
trap on_interrupt INT
trap on_terminate TERM
trap on_hangup HUP
}

View File

@@ -63,8 +63,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -63,8 +63,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -63,8 +63,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -37,8 +37,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"

View File

@@ -67,8 +67,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -64,8 +64,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -63,8 +63,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -68,8 +68,9 @@ CLOUD="${TAB}☁️${TAB}${CL}"
set -Eeo pipefail
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -44,6 +44,9 @@ CROSS="${RD}✗${CL}"
set -Eeo pipefail
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -64,8 +64,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -72,8 +72,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -62,8 +62,9 @@ CLOUD="${TAB}☁️${TAB}${CL}"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -60,8 +60,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -63,8 +63,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -62,8 +62,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"

View File

@@ -62,8 +62,9 @@ THIN="discard=on,ssd=1,"
set -e
trap 'error_handler $LINENO "$BASH_COMMAND"' ERR
trap cleanup EXIT
trap 'post_update_to_api "failed" "INTERRUPTED"' SIGINT
trap 'post_update_to_api "failed" "TERMINATED"' SIGTERM
trap 'post_update_to_api "failed" "130"' SIGINT
trap 'post_update_to_api "failed" "143"' SIGTERM
trap 'post_update_to_api "failed" "129"; exit 129' SIGHUP
function error_handler() {
local exit_code="$?"
local line_number="$1"