software package profiles implemeted, job deletion optimized

This commit is contained in:
2026-09-26 13:07:23 +02:00
parent 87a5041162
commit 7d314057e4
54 changed files with 3492 additions and 342 deletions
+1 -1
View File
@@ -34,7 +34,7 @@ Image removal and replacement only change the database reference. The physical u
Version 0.5.5.60 adds a reusable standard image library for assets and categories.
- Persistent Docker host directory: `./data/uploads/library`
- Container directory: `/app/app/static/uploads/library`
- Container directory: `/assetmanager-data/uploads/library`
- Public application path: `/static/uploads/library/`
- Supported formats: PNG, JPG/JPEG, WEBP and GIF
- Existing library images can be selected directly in the asset and category forms.
+36 -34
View File
@@ -4,14 +4,15 @@ This guide describes a Docker image based installation and the first-start behav
## 1. Prepare the installation directory
Create a dedicated directory and the persistent data directories:
Create only a dedicated installation directory. AssetManager creates its required application subdirectories below `./data` automatically on container start:
```bash
mkdir -p /srv/docker/assetmanager
cd /srv/docker/assetmanager
mkdir -p data/config data/uploads data/logs data/backups data/scripts data/software-packages data/postgres
```
No manual creation of `data/config`, `data/uploads`, `data/logs`, `data/backups`, `data/scripts`, `data/software-packages` or `data/analyzer-profiles` is required.
Create a `.env` file. Use strong, unique values for all secrets:
```env
@@ -67,14 +68,21 @@ services:
environment:
DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-assetmanager}:${POSTGRES_PASSWORD:-change-me}@db:5432/${POSTGRES_DB:-assetmanager}
APP_TITLE: AssetManager
APP_CONFIG: /app/config/config.json
APPINFO_PATH: /app/config/APPINFO.json
BACKUP_DIR: /data/backups
ASSETMANAGER_DATA_ROOT: /assetmanager-data
APP_CONFIG: /assetmanager-data/config/config.json
APPINFO_PATH: /assetmanager-data/config/APPINFO.json
UPLOAD_DIR: /assetmanager-data/uploads
STANDARD_IMAGE_DIR: /assetmanager-data/uploads/library
BACKUP_DIR: /assetmanager-data/backups
BACKUP_INTERVAL_HOURS: ${BACKUP_INTERVAL_HOURS:-8}
BACKUP_RETENTION_DAYS: ${BACKUP_RETENTION_DAYS:-3}
MESHCENTRAL_PASSWORD: ${MESHCENTRAL_PASSWORD:-}
SYNC_LOG_DIR: /app/data/logs/sync
DIAGNOSTIC_DIR: /app/data/logs/diagnostics
APP_LOG_DIR: /assetmanager-data/logs
SYNC_LOG_DIR: /assetmanager-data/logs/sync
DIAGNOSTIC_DIR: /assetmanager-data/logs/diagnostics
SOFTWARE_PACKAGE_DIR: /assetmanager-data/software-packages
ANALYZER_PROFILE_DIR: /assetmanager-data/analyzer-profiles
SCRIPT_DIR: /assetmanager-data/scripts
LDAP_BIND_PASSWORD: ${LDAP_BIND_PASSWORD:-}
SESSION_SECRET: ${SESSION_SECRET:-}
LOCAL_ADMIN_USERNAME: ${LOCAL_ADMIN_USERNAME:-}
@@ -84,12 +92,7 @@ services:
ports:
- "${APP_PORT:-8088}:8000"
volumes:
- ./data/config:/app/config
- ./data/uploads:/app/app/static/uploads
- ./data/logs:/app/data/logs
- ./data/backups:/data/backups
- ./data/scripts:/scripts
- ./data/software-packages:/app/data/software-packages
- ./data:/assetmanager-data
callback:
image: git.jusaro.de/roland/assetmanager:${ASSETMANAGER_VERSION:-0.5.5.68}
@@ -101,8 +104,18 @@ services:
environment:
DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-assetmanager}:${POSTGRES_PASSWORD:-change-me}@db:5432/${POSTGRES_DB:-assetmanager}
APP_TITLE: AssetManager
APP_CONFIG: /app/config/config.json
APPINFO_PATH: /app/config/APPINFO.json
ASSETMANAGER_DATA_ROOT: /assetmanager-data
APP_CONFIG: /assetmanager-data/config/config.json
APPINFO_PATH: /assetmanager-data/config/APPINFO.json
UPLOAD_DIR: /assetmanager-data/uploads
STANDARD_IMAGE_DIR: /assetmanager-data/uploads/library
BACKUP_DIR: /assetmanager-data/backups
APP_LOG_DIR: /assetmanager-data/logs
SYNC_LOG_DIR: /assetmanager-data/logs/sync
DIAGNOSTIC_DIR: /assetmanager-data/logs/diagnostics
SOFTWARE_PACKAGE_DIR: /assetmanager-data/software-packages
ANALYZER_PROFILE_DIR: /assetmanager-data/analyzer-profiles
SCRIPT_DIR: /assetmanager-data/scripts
MESHCENTRAL_PASSWORD: ${MESHCENTRAL_PASSWORD:-}
LDAP_BIND_PASSWORD: ${LDAP_BIND_PASSWORD:-}
SESSION_SECRET: ${SESSION_SECRET:-}
@@ -110,8 +123,7 @@ services:
ports:
- "${CALLBACK_BIND_IP:-127.0.0.1}:${CALLBACK_PORT:-8090}:8001"
volumes:
- ./data/config:/app/config
- ./data/logs:/app/data/logs
- ./data:/assetmanager-data
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8001/api/software-callback/health', timeout=3).read()"]
interval: 30s
@@ -136,14 +148,14 @@ docker compose up -d
docker compose ps
```
On the first start, the container creates the following files only when they do not already exist:
On every start, the container verifies and creates the required application directories below `./data` when they do not already exist. On the first start it also creates:
```text
data/config/config.json
data/config/APPINFO.json
```
Existing files are preserved during subsequent container starts and updates.
Existing directories and files are preserved during subsequent container starts and updates.
## 4. Important: authentication is initially disabled
@@ -268,17 +280,7 @@ If `config.json` or `APPINFO.json` is absent, restart the application container.
### Persistent standard image library
Add the following persistent volume to the application service:
```yaml
- ./data/uploads:/app/app/static/uploads
```
Create the host directory before the first start if desired:
```bash
mkdir -p data/uploads/library
```
No additional volume or manual directory creation is required. The application container mounts the common `./data` root and creates `data/uploads/library` automatically on start.
Images uploaded through the AssetManager standard image library are stored there. You can also copy PNG, JPG/JPEG, WEBP or GIF files directly into this directory. They are then offered for selection in asset and category forms after the page is reloaded.
@@ -291,20 +293,20 @@ Reusable asset/category images are stored persistently on the Docker host in:
./data/uploads/library
```
The library is a subdirectory of the already existing uploads volume. No additional Docker mount is required. The existing Compose mapping:
The library is a subdirectory of the common persistent data root. The Compose mapping:
```text
./data/uploads -> /app/app/static/uploads
./data -> /assetmanager-data
```
therefore exposes the library as:
```text
Host: ./data/uploads/library
Container: /app/app/static/uploads/library
Container: /assetmanager-data/uploads/library
```
Images may be uploaded from the AssetManager forms or copied directly into `./data/uploads/library` on the Docker host. They remain persistent because `data/uploads` is already part of the standard AssetManager volume layout.
Images may be uploaded from the AssetManager forms or copied directly into `./data/uploads/library` on the Docker host. They remain persistent because `data/uploads` is below the standard AssetManager data root.
For image-based installations use:
+39 -2
View File
@@ -13,7 +13,8 @@ The Setup Analyzer prepares Windows installation files for silent deployment wit
- InstallShield
- Advanced Installer
- Squirrel
- common 7-Zip and WinRAR SFX wrappers
- ZIP-compatible, 7-Zip and WinRAR/RAR SFX wrappers
- vendor profiles for Total Commander SFX and the PDF24 Creator online bootstrapper
- unknown EXE fallback
## Analysis output
@@ -50,6 +51,18 @@ Generated installation scripts no longer use `Start-Process -Wait` for the insta
The optional **Suppress post-install browser launch** setting terminates only browser processes that were newly created inside the installer process tree. Existing browser sessions are not touched. The option is preselected for Greenshot detections because Greenshot installers can open a completion web page after setup.
## SFX and embedded installers
Version 0.5.5.89 adds recursive static analysis of supported self-extracting installer containers. AssetManager never executes the uploaded SFX on the server. It first identifies the outer wrapper and then attempts to list and extract the payload with safe path, file-count, expanded-size and recursion limits.
Supported extraction paths include ZIP-compatible self-extracting files, 7-Zip SFX containers through the 7-Zip command-line tool, and WinRAR/RAR SFX containers through `unar` / `lsar` with 7-Zip as an additional fallback where supported by the installed build.
After extraction, embedded MSI/MSP/MSIX/AppX/MSU and EXE installer candidates are analyzed with the same static technology detector. Nested SFX candidates can be opened recursively up to the configured depth. Candidate selection is heuristic: known installer technologies, setup-like filenames and usable product metadata increase the score, while uninstallers and common prerequisite redistributables are strongly penalized. The selected candidate and alternatives are shown in the UI.
When an embedded installer is selected, package creation preserves the complete extracted payload rather than copying only the selected installer. AssetManager stores that payload in an internal ZIP, transfers it as one package file, expands it on the Windows target, and runs the selected embedded installer from its original relative directory. This preserves adjacent CAB files and subdirectories needed by many vendor packages.
The SFX feature still cannot guarantee that the heuristically selected inner installer is the vendor-supported deployment entry point. Always test the generated package on a designated test asset.
## Security
The uploaded installer is stored in a temporary directory and is not executed. Access is restricted to administrators. Old temporary analyses are removed after the configured retention period.
@@ -60,7 +73,10 @@ Environment variables:
SETUP_ANALYZER_TMP_DIR=/tmp/assetmanager-setup-analyzer
SETUP_ANALYZER_MAX_UPLOAD_MB=4096
SETUP_ANALYZER_RETENTION_HOURS=24
SOFTWARE_PACKAGE_DIR=/app/data/software-packages
SETUP_ANALYZER_MAX_EXTRACTED_MB=8192
SETUP_ANALYZER_MAX_EXTRACTED_FILES=20000
SETUP_ANALYZER_MAX_SFX_DEPTH=2
SOFTWARE_PACKAGE_DIR=/assetmanager-data/software-packages
```
## MSI metadata
@@ -70,3 +86,24 @@ Standard MSI silent-command generation works without additional system packages.
## Important
Installer technology detection and a suggested command do not guarantee vendor-specific compatibility. Test generated commands on a designated test asset before broad deployment.
## Vendor bootstrapper notes
Total Commander SFX installers are detected through their embedded `INSTALL.INF`. AssetManager uses `/AH1` for unattended hidden installation and shows `/A1` as the visible automatic alternative.
The small `pdf24-creator-installer.exe` is treated as an online bootstrapper rather than as the full PDF24 Creator Inno Setup package. It selects an architecture-specific current installer at runtime, so its package version is intentionally left unpinned. The analyzer surfaces `/SILENT` with medium confidence and recommends using the offline EXE or MSI when deterministic deployment is required.
## Analyzer profiles (0.5.5.90)
Vendor- and installer-specific detection knowledge is no longer added to the Python analyzer as product-specific branches. The generic analyzer engine loads declarative profiles from:
- `app/analyzer_profiles/system/` for profiles shipped with AssetManager,
- `/assetmanager-data/analyzer-profiles/community/` for imported/community profiles,
- `/assetmanager-data/analyzer-profiles/local/` for locally maintained overrides.
Profiles can contribute static markers, match rules, installer metadata extraction, architecture rules, silent parameters, process-control defaults and post-install defaults. Community and local profiles can override a system profile by using the same stable profile ID. The profile manager is available under **Software -> Setup Analyzer -> Analyzer profiles**.
Profiles are exchanged as `.amprofile` bundles and contain declarative JSON only. They cannot execute Python code. A separately hosted HTTPS repository can provide a catalog of `.amprofile` bundles through `ANALYZER_PROFILE_REPOSITORY_URL`; administrators explicitly choose profiles to install.
Software packages can be exported as `.ampkg` and re-imported. If the package was created from an analyzer profile, the export can embed that `.amprofile`; the importing administrator may explicitly choose whether to install the embedded profile as a Community profile.
+64
View File
@@ -0,0 +1,64 @@
# AssetManager Analyzer Profiles
AssetManager 0.5.5.90 separates installer-specific knowledge from the analyzer engine.
The Python engine performs generic operations such as PE inspection, MSI metadata parsing,
marker scanning, safe SFX extraction, embedded-installer selection and profile evaluation.
Product/vendor knowledge is stored in declarative JSON profiles.
## Profile types
Profiles use schema `assetmanager-analyzer-profile-v1` and profile API `1`.
They can be installed from three sources:
- `system`: shipped with AssetManager under `app/analyzer_profiles/system/`.
- `community`: imported manually or installed from a configured community repository.
- `local`: locally maintained profiles. Local profiles override profiles with the same ID.
Imported profiles are stored in `/assetmanager-data/analyzer-profiles` and are included in AssetManager backups.
Profiles are declarative data only and cannot contain executable Python code.
## Exchange format
Profiles are exported as `.amprofile` files. The file is a ZIP container with:
- `manifest.json`: bundle schema, profile ID/version/API and SHA-256 of `profile.json`.
- `profile.json`: the validated declarative profile definition.
The import validates paths, size, profile schema/API and SHA-256 before installing the profile.
## Community repository index
A repository is an HTTPS-hosted JSON index. Configure it with:
`ANALYZER_PROFILE_REPOSITORY_URL=https://example.org/assetmanager-profiles/index.json`
Index format:
```json
{
"schema": "assetmanager-analyzer-profile-repository-v1",
"name": "AssetManager Community Profiles",
"profiles": [
{
"id": "vendor.example-app",
"name": "Example App",
"version": "1.0.0",
"url": "https://example.org/profiles/vendor.example-app.amprofile",
"sha256": "<sha256 of the amprofile file>"
}
]
}
```
The AssetManager administrator explicitly loads the catalog and chooses which profile to install.
The bundle SHA-256 is verified when the repository provides one.
## Contributing profiles
A public profile repository can be maintained independently from the AssetManager application
repository. Contributors only need to submit declarative `.amprofile` bundles and index metadata;
no AssetManager source-code change is required for ordinary vendor/installer rules.
Use stable profile IDs. Increase the profile version when rules change. Avoid filename-only matching
when stronger static evidence is available. Silent parameters should only be marked high-confidence
when they are documented or clearly proven by installer metadata/static analysis.
+13
View File
@@ -0,0 +1,13 @@
{
"schema": "assetmanager-analyzer-profile-repository-v1",
"name": "AssetManager Community Profiles",
"profiles": [
{
"id": "vendor.example-app",
"name": "Example App",
"version": "1.0.0",
"url": "https://example.org/profiles/vendor.example-app.amprofile",
"sha256": "replace-with-bundle-sha256"
}
]
}
+3 -1
View File
@@ -1,3 +1,5 @@
- [0.5.5.90](UPDATE-0.5.5.90.md) - modular analyzer profiles, community repository foundation and portable package import/export.
- [0.5.5.89](UPDATE-0.5.5.89.md) - recursively analyze supported SFX payloads and package selected embedded installers.
- [0.5.5.88](UPDATE-0.5.5.88.md) - keep title/filter rows sticky while preserving unified column resizing.
- [0.5.5.87](UPDATE-0.5.5.87.md) - unify interactive table behavior and restore column resizing with sticky two-row headers.
- [0.5.5.86](UPDATE-0.5.5.86.md) - fix status filtering so assets without a status remain visible in global software lists.
@@ -24,7 +26,7 @@
Release notes are stored outside the project root to keep the repository overview compact.
The current release is **0.5.5.88**.
The current release is **0.5.5.90**.
Older notes are concise English summaries migrated from the original release documents. Git history remains authoritative for exact implementation details.
+62
View File
@@ -0,0 +1,62 @@
# Update 0.5.5.89
## Setup Analyzer: SFX and embedded installers
- Detect supported 7-Zip and WinRAR/RAR SFX wrappers as outer containers.
- Statically extract supported SFX payloads without executing uploaded installers.
- ZIP-compatible SFX files are handled by Python directly.
- Container image now includes 7-Zip and `unar` / `lsar` for additional SFX formats.
- Recursively analyze embedded installer candidates up to a configurable depth.
- Rank embedded MSI and known EXE installer technologies while penalizing uninstallers and common prerequisite packages.
- Display the selected embedded installer and alternate candidates in Setup Analyzer.
- Preserve the complete selected payload tree in an internal deployment ZIP so CAB files and subdirectories remain available.
- Software-package install scripts expand the payload on the target and run the selected installer from its original relative directory.
- Add extraction safety limits for total expanded size, file count, path traversal, symbolic links and recursion depth.
## New environment settings
```text
SETUP_ANALYZER_MAX_EXTRACTED_MB=8192
SETUP_ANALYZER_MAX_EXTRACTED_FILES=20000
SETUP_ANALYZER_MAX_SFX_DEPTH=2
```
Embedded-installer selection remains heuristic and should be verified on a test asset before broad deployment.
## Software package cleanup
- Added a Delete button directly to every row of the software-package overview.
- Broken packages that show a storage error can be deleted without opening their detail page.
- If associated software jobs exist, the overview confirmation explicitly states that those jobs will be deleted too.
- Package summary handling now keeps the overview usable when either the manifest or package-size scan fails.
## Architecture detection fix
- Distinguishes installer-launcher PE architecture from the target software architecture.
- Known setup wrappers such as NSIS/Inno no longer classify a package as x86 merely because their launcher stub is PE32.
- Explicit x64/x86/ARM64 package filename hints are used as target-architecture evidence.
- The analyzer shows the launcher architecture and architecture source separately in technical details.
- Verified with `npp.8.9.8.Installer.x64.exe`: NSIS launcher x86, target package x64.
## Dispatcher ACL retry fix
- Upload retry no longer reapplies directory ACLs recursively to files already uploaded into the job directory.
- Windows job-directory ACLs are now applied only to the directory itself; uploaded files inherit the directory permissions normally.
- Package-file retry removes only the failed target file and leaves `run.ps1` and other package files untouched.
- A remote script preflight now logs existence, size, readability and Windows ACLs before execution.
- If the uploaded `run.ps1` is not readable, the dispatcher stops with a dedicated diagnostic error before trying to launch PowerShell.
## Total Commander SFX profile
- Recognize Total Commander's self-extracting ZIP installer through its embedded `INSTALL.INF`.
- Read product name, version, publisher, target architecture and running-process hint from embedded installer metadata.
- Use `/AH1` as the unattended default (automatic + hidden installation) and expose `/A1` as the visible automatic alternative.
- Keep the outer SFX executable as the deployment installer; its embedded CAB/INF files are installation data, not a replacement setup executable.
- Recognize architecture suffixes attached directly to version numbers such as `tcmd1156x64.exe`.
## PDF24 online bootstrapper profile
- Detect the small `pdf24-creator-installer.exe` bootstrapper through multiple vendor-specific static markers instead of reporting an unknown EXE at 25%.
- Distinguish the bootstrapper from the full PDF24 Creator Inno Setup package.
- Report `PDF24 Creator`, `geek software GmbH` and the bootstrapper's architecture-selecting behavior (`x86+x64+arm64`).
- Do not misreport the bootstrapper's own `1.0.0` file version as the PDF24 Creator version, because the bootstrapper downloads the current Creator release at deployment time.
- Surface `/SILENT` with medium command confidence and warn that this online bootstrapper is network-dependent and version-unpinned; prefer the offline EXE/MSI for reproducible managed deployment.
+58
View File
@@ -0,0 +1,58 @@
# Update 0.5.5.90
## Modular Setup Analyzer profiles
- Installer/vendor-specific Setup Analyzer knowledge is moved into declarative analyzer profiles.
- System, Community and Local profile sources are supported.
- `.amprofile` import/export validates schema, profile API and SHA-256 and contains no executable plugin code.
- Profiles can be enabled/disabled; imported Community/Local profiles can be deleted without modifying application source.
- Existing Total Commander, PDF24 and Greenshot-specific analyzer behavior is supplied as system profiles instead of Python special cases.
- Generic technology signatures for Inno Setup, NSIS, WiX Burn, InstallShield, Advanced Installer, Squirrel, 7-Zip SFX and WinRAR SFX are also supplied as profiles.
## Community repository foundation
- Optional HTTPS profile repository support via `ANALYZER_PROFILE_REPOSITORY_URL`.
- Repository index schema `assetmanager-analyzer-profile-repository-v1`.
- Admins explicitly load the repository catalog and select profiles to install.
- Optional repository SHA-256 is checked before a bundle is imported.
- Repository format/contribution documentation is included under `docs/analyzer-profiles/`.
## Portable software packages
- Stored software packages can now be exported as `.ampkg` bundles.
- Package overview can import `.ampkg` bundles and compatible Setup Analyzer ZIP exports.
- Imports validate ZIP paths, symlinks, required package files and available SHA-256 metadata.
- Import UI requires acknowledgement that software packages may contain executable PowerShell scripts.
- Imported analyzer profiles are included in AssetManager backup/restore.
## Automatic persistent directory initialization
- Docker Compose now mounts one persistent application root (`./data` -> `/assetmanager-data`) instead of requiring a separate host bind directory for every feature.
- The container entrypoint verifies and creates all required application subdirectories on every start, including analyzer profiles and software packages.
- Existing host data remains in the same `./data/...` layout; no data migration is required.
- The established `/scripts` container path remains available for existing job definitions.
- Uploaded images remain available under their existing `/static/uploads/...` URLs while being stored below the persistent data root.
## Import form and generic installer metadata fixes
- Software-package import checkboxes are rendered left-aligned, vertically stacked, and no longer inherit full-width input sizing.
- EXE analyzer metadata now uses a conservative dotted-version fallback from the installer filename when MSI/MSIX/PE metadata has no product version.
- Signed PE installers can use the Authenticode code-signing certificate organization/common name as a manufacturer fallback.
- The Authenticode publisher is only a fallback; explicit MSI/MSIX/PE manufacturer metadata still has higher priority.
- The generic metadata enrichment contains no VLC-specific Python logic; product-specific naming is supplied by the declarative VLC analyzer profile.
## Follow-up: VLC metadata, localized file picker and client job retention
- Added declarative VLC analyzer profile so NSIS VLC packages receive the product name `VLC media player` without product-specific Python code.
- Replaced browser-native file selector text in Setup Analyzer, analyzer-profile import and software-package import with translated AssetManager controls.
- Added configurable stale client job-directory cleanup with a default retention of 24 hours. Cleanup is limited to AssetManager job directories and runs before a new file-based job is dispatched to that client.
## Follow-up: reliable client job retention cleanup
- Stale client cleanup now uses the job-directory creation time on Windows instead of the mutable last-write timestamp.
- Current/active job directories are explicitly excluded from cleanup.
- Legacy numeric job directories and current `JobID-Attempt` directories are both recognized below the dedicated AssetManager job root.
- If deletion of an expired directory fails because of legacy/broken ACLs, AssetManager repairs permissions only inside that already-expired job directory and retries deletion.
- Cleanup still runs before new file-based jobs and now also runs periodically for online managed clients with job history, so stale files do not depend on a later deployment to be removed.
- Cleanup diagnostics now report found, eligible, removed, failed, young, active and ignored directory counts.