Files
ProxmoxVE/.github/workflows/bak/crawl-versions.yaml
CanbiZ (MickLesk) 21d09cfb17 github: add GitHub-based versions.json updater (#10021)
* feat(workflow): add GitHub-based versions.json updater

Replaces newreleases.io with direct GitHub API queries.
Extracts repos from fetch_and_deploy_gh_release calls in install scripts.
Runs 2x daily (06:00 and 18:00 UTC).

* feat(workflow): extend version crawler with multiple sources

- Method 1: fetch_and_deploy_gh_release calls (direct)
- Method 2: GitHub URLs extracted from all scripts
- Method 3: VM image sources (HAOS)
- Method 4: Docker Hub / GHCR versions
- Method 5: npm Registry versions

Also tries tags fallback when no releases exist.

* feat(workflow): rewrite with version-sources.json config

- Generates version-sources.json with structured metadata
- Each entry has: slug, type, source, script, version, date
- Extracts from: fetch_and_deploy_gh_release, GitHub URLs, npm, Docker
- Generates versions.json for backward compatibility
- Fully automatic, no manual mapping needed

* feat(workflow): add manual GitHub mappings and pveam support

- Method 5: Manual GitHub mappings for 36 apt-based apps
  (grafana, redis, postgresql, mariadb, influxdb, etc.)
- Method 6: Proxmox LXC templates (debian, ubuntu, alpine)
  via download.proxmox.com index
- Method 7: Special sources (HAOS VM)

Total coverage: ~310+ apps

* feat(workflow): expand manual GitHub mappings to 75 apps

Added mappings for:
- Apache projects (cassandra, couchdb, guacamole, tomcat)
- Media apps (tdarr, unmanic, shinobi)
- DevOps (coolify, dokploy, runtipi, sonarqube)
- Databases (mongodb, mysql, neo4j, rabbitmq)
- And 30+ more apps

Total manual mappings: 75

* feat: add manual placeholders for 34 unknown-source apps

- Added 34 apps with 'manual:-' type for apps without known sources
- Added manual type handler in version-fetch (returns '-' placeholder)
- Added manual counter to summary output
- Coverage now 100% (all 405 scripts included)

Manual entries can be updated later when sources are discovered.

* Refactor and update GitHub workflow files

Moved several workflow files to a 'bak' backup directory and renamed 'close-ttek-issues.yaml' to 'close-tteck-issues.yaml'. Refactored 'update-versions-github.yml' to focus on extracting and updating GitHub versions, simplified the extraction logic, and updated the workflow schedule to run four times daily. Minor variable and logic improvements were made in 'close-discussion.yml'.

* clean file

* chore: empty versions.json for workflow test
2026-01-28 13:50:18 +01:00

127 lines
4.4 KiB
YAML
Generated

name: Crawl Versions from newreleases.io
on:
workflow_dispatch:
schedule:
# Runs at 12:00 AM and 12:00 PM UTC
- cron: "0 0,12 * * *"
permissions:
contents: write
pull-requests: write
jobs:
crawl-versions:
if: github.repository == 'community-scripts/ProxmoxVE'
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v2
with:
repository: community-scripts/ProxmoxVE
ref: main
- name: Generate a token
id: generate-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ vars.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
- name: Crawl from newreleases.io
env:
token: ${{ secrets.NEWRELEASES_TOKEN }}
run: |
page=1
projects_file="project_json"
output_file="frontend/public/json/versions.json"
echo "[]" > $output_file
while true; do
echo "Start loop on page: $page"
projects=$(curl -s -H "X-Key: $token" "https://api.newreleases.io/v1/projects?page=$page")
total_pages=$(echo "$projects" | jq -r '.total_pages')
if [ -z "$total_pages" ] || [ "$total_pages" -eq 0 ]; then
echo "No pages available. Exiting."
exit 1
fi
if [ $page == $total_pages ]; then
break
fi
if [ -z "$projects" ] || ! echo "$projects" | jq -e '.projects' > /dev/null; then
echo "No more projects or invalid response. Exiting."
break
fi
echo "$projects" > "$projects_file"
jq -r '.projects[] | "\(.id) \(.name)"' "$projects_file" | while read -r id name; do
version=$(curl -s -H "X-Key: $token" "https://api.newreleases.io/v1/projects/$id/latest-release")
version_data=$(echo "$version" | jq -r '.version // empty')
date=$(echo "$version" | jq -r '.date // empty')
if [ -n "$version_data" ]; then
jq --arg name "$name" --arg version "$version_data" --arg date "$date" \
'. += [{"name": $name, "version": $version, "date": $date}]' "$output_file" > "$output_file.tmp" && mv "$output_file.tmp" "$output_file"
fi
done
((page++))
done
- name: Commit JSON
env:
GH_TOKEN: ${{ steps.generate-token.outputs.token }}
run: |
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "GitHub Actions[bot]"
git checkout -b update_versions || git checkout update_versions
git add frontend/public/json/versions.json
if git diff --cached --quiet; then
echo "No changes detected."
echo "changed=false" >> "$GITHUB_ENV"
exit 0
else
echo "Changes detected:"
git diff --stat --cached
echo "changed=true" >> "$GITHUB_ENV"
fi
git commit -m "Update versions.json"
git push origin update_versions --force
gh pr create --title "[Github Action] Update versions.json" --body "Update versions.json, crawled from newreleases.io" --base main --head update_versions --label "automated pr"
- name: Approve pull request
if: env.changed == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER=$(gh pr list --head "update_versions" --json number --jq '.[].number')
if [ -n "$PR_NUMBER" ]; then
gh pr review $PR_NUMBER --approve
fi
- name: Approve pull request and merge
if: env.changed == 'true'
env:
GH_TOKEN: ${{ secrets.PAT_AUTOMERGE }}
run: |
PR_NUMBER=$(gh pr list --head "update_versions" --json number --jq '.[].number')
if [ -n "$PR_NUMBER" ]; then
gh pr review $PR_NUMBER --approve
gh pr merge $PR_NUMBER --squash --admin
fi
- name: Re-approve pull request after update
if: env.changed == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
PR_NUMBER=$(gh pr list --head "update_versions" --json number --jq '.[].number')
if [ -n "$PR_NUMBER" ]; then
gh pr review $PR_NUMBER --approve
fi