diff --git a/README.md b/README.md
index 1d4d003..955c840 100644
--- a/README.md
+++ b/README.md
@@ -189,7 +189,7 @@ Void Linux is fast, simple and independent: runit instead of systemd, and its ow
- The GTK themes `DasOS` and `DasOS-Dark` build on adw-gtk3 and add the Das OS accent colours. They also style the Xfce panel, which adw-gtk3 leaves black.
- The xfwm4 window frames (rounded corners, round buttons, a red close button on hover) are generated as PNG files by `xfwm4gen` (Go) from a single palette.
- Light and dark wallpapers show the star over a rushnyk-style embroidered ribbon. The same star is the Das OS logo and the menu button.
-- Fonts are Inter for the interface and JetBrains Mono in the terminal. Icons are Papirus and the cursor theme is Adwaita.
+- Fonts are Inter for the interface and JetBrains Mono in the terminal. Icons are Papirus (the Papirus-Light variant, so the panel icons are dark) and the cursor theme is Adwaita.
#### Desktop defaults
**_The das-settings package configures Xfce without touching files owned by other packages._**
@@ -207,6 +207,7 @@ Void Linux is fast, simple and independent: runit instead of systemd, and its ow
- `srcpkgs/das-snapshots` contains the snapper configuration, `das-snapper-setup` and the dracut module for booting snapshots.
- `srcpkgs/das-installer` contains the Calamares settings and branding, `srcpkgs/das-server-installer` the console installer. The steps both share (`mount-source`, `postinstall`) are in `srcpkgs/das-install-common`. They exist only on the live ISOs.
- `srcpkgs/das-server` is the server metapackage with the nftables, fail2ban, sshd and Docker defaults. `srcpkgs/das-helpers` contains `dsv`, `duser`, `dfw`, `dsys`, `dprofile` and the profiles.
+- `srcpkgs/publisher` (`publish`) and `srcpkgs/deployer` (`deploy`) are the developer's TUI/CLI tools that publish builds and deploy projects to a server over ssh + rsync. Each one keeps its config in `~/.config//` (an example is copied there on the first run) and its history in `~/.local/state//`.
- `srcpkgs/calamares`, `srcpkgs/adw-gtk3` and `srcpkgs/font-jetbrains-mono` package upstream releases that Void does not have.
- `make packages` lays the templates over a shallow clone of void-packages and builds them with `xbps-src` into `out/hostdir/binpkgs`. The ISO build installs them from there.
- A package is rebuilt only when its directory in `srcpkgs/` changed, its binary package is missing, or Void no longer provides a shared library it links to (checked with a dry-run install). `DASOS_REBUILD=1 make packages` rebuilds everything.
diff --git a/README_ua.md b/README_ua.md
index 45b54d3..e37c585 100644
--- a/README_ua.md
+++ b/README_ua.md
@@ -189,7 +189,7 @@ Void Linux швидкий, простий і незалежний: runit зам
- GTK-теми `DasOS` і `DasOS-Dark` побудовані на adw-gtk3 і додають акцентні кольори Das OS. Вони також оформлюють панель Xfce, яку adw-gtk3 лишає чорною.
- Рамки вікон xfwm4 (заокруглені кути, круглі кнопки, червона кнопка закриття при наведенні) генерує у PNG утиліта `xfwm4gen` (Go) з однієї палітри.
- Світлі й темні шпалери зображують зірку над смугою орнаменту в стилі рушника. Та сама зірка є логотипом Das OS і кнопкою меню.
-- Шрифти: Inter для інтерфейсу і JetBrains Mono в терміналі. Іконки Papirus, тема курсора Adwaita.
+- Шрифти: Inter для інтерфейсу і JetBrains Mono в терміналі. Іконки Papirus (варіант Papirus-Light, тому значки панелі темні), тема курсора Adwaita.
#### Налаштування за замовчуванням
**_Пакет das-settings налаштовує Xfce, не чіпаючи файлів інших пакетів._**
@@ -207,6 +207,7 @@ Void Linux швидкий, простий і незалежний: runit зам
- `srcpkgs/das-snapshots` містить конфігурацію snapper, `das-snapper-setup` і dracut-модуль для завантаження знімків.
- `srcpkgs/das-installer` містить налаштування й брендинг Calamares, `srcpkgs/das-server-installer` — консольний інсталятор. Спільні для обох кроки (`mount-source`, `postinstall`) лежать у `srcpkgs/das-install-common`. Вони є лише на live-ISO.
- `srcpkgs/das-server` — серверний метапакет із налаштуваннями nftables, fail2ban, sshd і Docker. `srcpkgs/das-helpers` містить `dsv`, `duser`, `dfw`, `dsys`, `dprofile` і профілі.
+- `srcpkgs/publisher` (`publish`) і `srcpkgs/deployer` (`deploy`) — інструменти розробника з TUI і командним рядком: публікують збірки й розгортають проєкти на сервері через ssh + rsync. Конфіг кожного лежить у `~/.config/<назва>/` (приклад копіюється туди при першому запуску), історія — у `~/.local/state/<назва>/`.
- `srcpkgs/calamares`, `srcpkgs/adw-gtk3` і `srcpkgs/font-jetbrains-mono` пакують релізи, яких немає у Void.
- `make packages` накладає шаблони на неглибокий клон void-packages і збирає їх через `xbps-src` в `out/hostdir/binpkgs`. Збирання ISO встановлює їх звідти.
- Пакет перезбирається, лише коли змінився його каталог у `srcpkgs/`, зник бінарний пакет або Void більше не надає бібліотеку, з якою він зібраний (перевіряється пробним встановленням). `DASOS_REBUILD=1 make packages` перезбирає все.
diff --git a/TODO.md b/TODO.md
index 4287108..8f1b651 100644
--- a/TODO.md
+++ b/TODO.md
@@ -1,6 +1,6 @@
# Das OS — що зробити далі
-Стан на 2026-10-25: версія 0.9.4 (не закомічено), етапи 1–8 готові (live ISO, власний репозиторій, оформлення, інсталятор Calamares зі знімками Btrfs, Go-утиліти `das-update` і `das-welcome`, серверний образ з консольним інсталятором, менші ISO, консольні помічники, профіль розробника), CI (етап 9: пакети й обидва ISO збираються на власному runner, ~45 хв).
+Стан на 2026-10-25: версія 0.9.5 (не закомічено), етапи 1–8 готові (live ISO, власний репозиторій, оформлення, інсталятор Calamares зі знімками Btrfs, Go-утиліти `das-update` і `das-welcome`, серверний образ з консольним інсталятором, менші ISO, консольні помічники, профіль розробника), CI (етап 9: пакети й обидва ISO збираються на власному runner, ~45 хв).
## Етап 5. Go-утиліти (готово, лишилися перевірки)
- [x] `das-update`: вікно, значок у треї (лише коли є оновлення), консольний режим `--cli`, знімок snapper pre/post, встановлення через pkexec, підказка про відкат.
@@ -114,6 +114,10 @@ Live ISO без графіки з консольним інсталятором:
## Оформлення та дрібниці
- [x] Світла тема: значки панелі (трей, звук, розкладка) були блідо-сірими на світлій панелі; тепер текст і значки `#1f2430` (`das-theme` 0.2.0, 0.9.4; перевірити на ISO).
- [x] Значок кнопки меню на вибір: `das-panel-icon star|trident` (зірка `das-logo` за замовчуванням, тризуб `das-trident` жовтий із синім контуром); 0.9.4, перевірити на ISO.
+- [x] Значки Wi-Fi і Bluetooth у треї лишались блідими: панельні значки Papirus (`#dfdfdf`) зроблені для темних панелей, а CSS їх не перефарбовує. Світла тема тепер на `Papirus-Light` (`#444`): `xsettings.xml`, `index.theme`, екран входу; `das-theme` 0.3.0, `das-settings` 0.6.0, 0.9.5; перевірити на ISO.
+- [x] `das-panel-icon help` (`-h`, `--help`) показує підказку, а не ставить значок «?»; невідомі `-ключі` відхиляються.
+- [x] На десктопний образ додано `publisher` (`publish`) і `deployer` (`deploy`): копії проєктів з `~/Dev/Projects/Python`, конфіг у `~/.config/<назва>/` (створюється з прикладу при першому запуску), історія в `~/.local/state/<назва>/`; перевірити на ISO.
+- [ ] Перенести в оригінальні `publisher` і `deployer` зміну шляхів конфігу та `history.log` (зараз вони лише в копіях пакетів), щоб копії не розходились.
- [ ] Прибрати значки томів live-носія («Том 6,6 ГБ») з робочого столу live-сесії.
- [ ] Тема GRUB у встановленій системі (зараз стандартний чорний екран).
- [ ] Перевірити меню Whisker вручну.
diff --git a/VERSION b/VERSION
index a602fc9..b0bb878 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-0.9.4
+0.9.5
diff --git a/live/packages.x86_64 b/live/packages.x86_64
index e759dec..18a52c2 100644
--- a/live/packages.x86_64
+++ b/live/packages.x86_64
@@ -69,4 +69,6 @@ das-theme
das-tools
das-helpers
das-shell
+publisher
+deployer
xtools-minimal
diff --git a/srcpkgs/das-settings/files/bin/das-panel-icon b/srcpkgs/das-settings/files/bin/das-panel-icon
index 67d0fca..19f1fc2 100755
--- a/srcpkgs/das-settings/files/bin/das-panel-icon
+++ b/srcpkgs/das-settings/files/bin/das-panel-icon
@@ -2,12 +2,25 @@
# Picks the icon of the menu button on the panel: das-panel-icon star|trident|.
set -u
+usage() {
+ echo "usage: das-panel-icon star|trident|ICON-NAME" >&2
+ echo "now: $(xfconf-query -c xfce4-panel -p /plugins/plugin-1/button-icon 2>/dev/null || echo das-logo)" >&2
+}
+
case ${1:-} in
star) icon=das-logo ;;
trident) icon=das-trident ;;
+help | -h | --help)
+ usage
+ exit 0
+ ;;
"")
- echo "usage: das-panel-icon star|trident|ICON-NAME" >&2
- echo "now: $(xfconf-query -c xfce4-panel -p /plugins/plugin-1/button-icon 2>/dev/null || echo das-logo)" >&2
+ usage
+ exit 1
+ ;;
+-*)
+ echo "das-panel-icon: unknown option $1" >&2
+ usage
exit 1
;;
*) icon=$1 ;;
diff --git a/srcpkgs/das-settings/files/lightdm/50-das-greeter.conf b/srcpkgs/das-settings/files/lightdm/50-das-greeter.conf
index 0f0964f..8fd2bfd 100644
--- a/srcpkgs/das-settings/files/lightdm/50-das-greeter.conf
+++ b/srcpkgs/das-settings/files/lightdm/50-das-greeter.conf
@@ -2,7 +2,7 @@
[greeter]
background=/usr/share/backgrounds/das/das-light.svg
theme-name=DasOS
-icon-theme-name=Papirus
+icon-theme-name=Papirus-Light
font-name=Inter 11
cursor-theme-name=Adwaita
cursor-theme-size=24
diff --git a/srcpkgs/das-settings/files/xfconf/xsettings.xml b/srcpkgs/das-settings/files/xfconf/xsettings.xml
index befa45b..ab9b744 100644
--- a/srcpkgs/das-settings/files/xfconf/xsettings.xml
+++ b/srcpkgs/das-settings/files/xfconf/xsettings.xml
@@ -3,7 +3,7 @@
-
+
diff --git a/srcpkgs/das-settings/template b/srcpkgs/das-settings/template
index 0bc3b20..fbaaaea 100644
--- a/srcpkgs/das-settings/template
+++ b/srcpkgs/das-settings/template
@@ -4,7 +4,7 @@
# Defaults go to /etc/xdg/das, which /etc/profile.d/das-xdg.sh puts in front of
# /etc/xdg: stock Xfce files stay untouched and users' own changes still win.
pkgname=das-settings
-version=0.5.0
+version=0.6.0
revision=1
create_wrksrc=yes
depends="das-theme lightdm lightdm-gtk-greeter xfce4-notifyd xfce4-panel
diff --git a/srcpkgs/das-theme/files/gtk/DasOS/index.theme b/srcpkgs/das-theme/files/gtk/DasOS/index.theme
index 79bf54a..6e6ed83 100644
--- a/srcpkgs/das-theme/files/gtk/DasOS/index.theme
+++ b/srcpkgs/das-theme/files/gtk/DasOS/index.theme
@@ -7,6 +7,6 @@ Encoding=UTF-8
[X-GNOME-Metatheme]
GtkTheme=DasOS
MetacityTheme=DasOS
-IconTheme=Papirus
+IconTheme=Papirus-Light
CursorTheme=Adwaita
ButtonLayout=menu:minimize,maximize,close
diff --git a/srcpkgs/das-theme/template b/srcpkgs/das-theme/template
index ef0fe2d..fdf3e80 100644
--- a/srcpkgs/das-theme/template
+++ b/srcpkgs/das-theme/template
@@ -1,7 +1,7 @@
# Template file for 'das-theme'
# In-tree package: sources are in files/ (FILESDIR).
pkgname=das-theme
-version=0.2.0
+version=0.3.0
revision=1
create_wrksrc=yes
hostmakedepends="go"
diff --git a/srcpkgs/deployer/files/LICENSE b/srcpkgs/deployer/files/LICENSE
new file mode 100644
index 0000000..a8f7f83
--- /dev/null
+++ b/srcpkgs/deployer/files/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Das
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/srcpkgs/deployer/files/bin/deploy b/srcpkgs/deployer/files/bin/deploy
new file mode 100755
index 0000000..af18e6a
--- /dev/null
+++ b/srcpkgs/deployer/files/bin/deploy
@@ -0,0 +1,8 @@
+#!/bin/sh
+# deployer launcher: the config lives in ~/.config/deployer/deploy.toml (copied from the example on the first run).
+cfg=${XDG_CONFIG_HOME:-$HOME/.config}/deployer/deploy.toml
+if [ ! -f "$cfg" ]; then
+ mkdir -p "${cfg%/*}" && cp /usr/share/das/deployer/deploy.toml.example "$cfg" || exit 1
+ echo "deploy: created $cfg, edit [server] there" >&2
+fi
+PYTHONPATH=/usr/lib/das/deployer exec python3 -c 'import sys; from deployer.cli import main; sys.exit(main())' --config "$cfg" "$@"
diff --git a/srcpkgs/deployer/files/python/deployer/__init__.py b/srcpkgs/deployer/files/python/deployer/__init__.py
new file mode 100644
index 0000000..571c4f0
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/__init__.py
@@ -0,0 +1,2 @@
+"""Deploy local projects to the home server over ssh + rsync."""
+__version__ = "0.3.0"
diff --git a/srcpkgs/deployer/files/python/deployer/cli.py b/srcpkgs/deployer/files/python/deployer/cli.py
new file mode 100644
index 0000000..7c0ae0b
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/cli.py
@@ -0,0 +1,171 @@
+"""Command-line mode: the same pipeline as the TUI, for scripts and quick deploys."""
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from . import __version__, config, detect, pipeline, remote, snapshots
+from .kinds import Options, Plan
+
+C = {"new": "\033[32m+", "update": "\033[33m~", "delete": "\033[31m-"}
+RESET = "\033[0m"
+
+
+def color(on: bool):
+ if not on:
+ C.update(new="+", update="~", delete="-")
+ globals()["RESET"] = ""
+
+
+def print_plan(plan: Plan) -> None:
+ a = plan.app
+ print(f"\n{a.name} [{a.kind}] {plan.rsync_src} -> {plan.server.target}:{plan.dest}"
+ f" {('rev ' + plan.git_rev) if plan.git_rev else ''}")
+ for c in plan.changes:
+ print(f" {C[c.action]} {c.path}{RESET}")
+ if not plan.changes:
+ print(" (no file changes)")
+ for n in plan.notes:
+ print(f" ! {n}")
+
+
+def confirm(question: str, yes: bool) -> bool:
+ if yes:
+ return True
+ if not sys.stdin.isatty():
+ print("not a terminal: pass --yes to confirm", file=sys.stderr)
+ return False
+ return input(f"{question} [y/N] ").strip().lower() in ("y", "yes", "т", "так")
+
+
+def options(args) -> Options:
+ return Options(build=not args.no_build, install_deps=args.deps, restart=not args.no_restart,
+ copy_configs=not args.no_configs, delete=args.delete)
+
+
+def cmd_list(cfg, args):
+ st = {} if args.offline else pipeline.status_all(cfg)
+ for a in cfg.apps:
+ state, last, code = st.get(a.name, ("", "", ""))
+ print(f"{a.name:22} {a.kind:15} {state:14} {code:8} {cfg.dest_path(a):48} {last}")
+
+
+def cmd_deploy(cfg, args):
+ app = cfg.app(args.app)
+ plan = pipeline.preview(cfg, app, options(args), log=print)
+ print_plan(plan)
+ if args.dry_run:
+ return 0
+ if not plan.changes and not plan.first_install and not args.force:
+ print("nothing to sync: use --force to run install/restart anyway")
+ return 0
+ if not confirm("deploy?", args.yes):
+ return 1
+ res = pipeline.execute(plan, log=print)
+ if not res.ok and res.snapshot and confirm(f"roll back to {res.snapshot}?", False):
+ res = pipeline.rollback(cfg, app, res.snapshot, log=print)
+ return 0 if res.ok else 2
+
+
+def cmd_rollback(cfg, args):
+ app = cfg.app(args.app)
+ snaps = snapshots.listing(cfg.server, app.name)
+ if args.list or not snaps:
+ print("\n".join(snaps) or "no snapshots")
+ return 0
+ ts = args.ts or snaps[0]
+ if not confirm(f"restore {app.name} from {ts}?", args.yes):
+ return 1
+ res = pipeline.rollback(cfg, app, ts, log=print)
+ print(res.message)
+ return 0 if res.ok else 2
+
+
+def cmd_logs(cfg, args):
+ return remote.interactive(cfg.server, pipeline.logs_command(cfg, cfg.app(args.app)))
+
+
+def cmd_import(cfg, args):
+ found = detect.import_server(cfg, log=print)
+ if not found:
+ print("nothing new on the server")
+ return 0
+ ready = [i.app for i in found if i.matched and i.app.kind.startswith("python")]
+ drafts = [i.app for i in found if i.app not in ready]
+ print(f"\n{len(ready)} ready, {len(drafts)} as commented drafts")
+ if confirm(f"append to {cfg.path}?", args.yes):
+ config.append_apps(cfg.path, ready, drafts)
+ return 0
+
+
+def cmd_add(cfg, args):
+ app = detect.new_app(Path(args.path), args.name, args.kind)
+ print(config.app_to_toml(app))
+ if confirm(f"append to {cfg.path}?", args.yes):
+ config.append_apps(cfg.path, [app])
+ return 0
+
+
+def main(argv=None) -> int:
+ p = argparse.ArgumentParser(prog="deploy", description="Deploy local projects to the server. "
+ "No command: start the TUI.")
+ p.add_argument("--config", default=None, help="deploy.toml (default: next to deploy.py)")
+ p.add_argument("--no-color", action="store_true")
+ p.add_argument("-V", "--version", action="version", version=__version__)
+ sub = p.add_subparsers(dest="cmd")
+
+ s = sub.add_parser("list", help="apps and their state on the server")
+ s.add_argument("--offline", action="store_true", help="do not ask the server")
+ for name, hlp in (("deploy", "preview and deploy an app"), ("preview", "show what a deploy would change")):
+ s = sub.add_parser(name, help=hlp)
+ s.add_argument("app")
+ s.add_argument("-y", "--yes", action="store_true", help="do not ask for confirmation")
+ s.add_argument("-n", "--dry-run", action="store_true", default=name == "preview")
+ s.add_argument("--force", action="store_true", help="install/restart even without file changes")
+ s.add_argument("--no-build", action="store_true")
+ s.add_argument("--no-restart", action="store_true", help="no restart, no post command, no health check")
+ s.add_argument("--no-configs", action="store_true", help="do not create missing config links")
+ s.add_argument("--deps", action="store_true", help="pip install even if requirements.txt is unchanged")
+ s.add_argument("--delete", action=argparse.BooleanOptionalAction, default=None,
+ help="delete server files that are absent locally (default: app setting)")
+ s = sub.add_parser("rollback", help="restore the latest (or given) snapshot")
+ s.add_argument("app")
+ s.add_argument("ts", nargs="?")
+ s.add_argument("--list", action="store_true")
+ s.add_argument("-y", "--yes", action="store_true")
+ s = sub.add_parser("logs", help="follow the app log")
+ s.add_argument("app")
+ s = sub.add_parser("import", help="add apps that already run on the server to deploy.toml")
+ s.add_argument("-y", "--yes", action="store_true")
+ s = sub.add_parser("add", help="add a local project to deploy.toml")
+ s.add_argument("path")
+ s.add_argument("--name")
+ s.add_argument("--kind", choices=config.KINDS)
+ s.add_argument("-y", "--yes", action="store_true")
+ sub.add_parser("check", help="validate deploy.toml")
+ sub.add_parser("tui", help="start the TUI (default)")
+
+ args = p.parse_args(argv)
+ color(sys.stdout.isatty() and not args.no_color)
+ cfg_path = Path(args.config) if args.config else Path(__file__).resolve().parent.parent / "deploy.toml"
+ try:
+ cfg = config.load(cfg_path)
+ except (OSError, config.ConfigError) as e:
+ print(f"config: {e}", file=sys.stderr)
+ return 1
+ if args.cmd in (None, "tui"):
+ from .tui import run
+ return run(cfg)
+ if args.cmd == "check":
+ print(f"{cfg_path}: {len(cfg.apps)} apps, OK")
+ return 0
+ handlers = {"list": cmd_list, "deploy": cmd_deploy, "preview": cmd_deploy, "rollback": cmd_rollback,
+ "logs": cmd_logs, "import": cmd_import, "add": cmd_add}
+ try:
+ return handlers[args.cmd](cfg, args) or 0
+ except (config.ConfigError, pipeline.DeployError, remote.RemoteError) as e:
+ print(f"error: {e}", file=sys.stderr)
+ return 1
+ except KeyboardInterrupt:
+ return 130
diff --git a/srcpkgs/deployer/files/python/deployer/config.py b/srcpkgs/deployer/files/python/deployer/config.py
new file mode 100644
index 0000000..33386a9
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/config.py
@@ -0,0 +1,213 @@
+"""deploy.toml: server settings and the registry of deployable apps."""
+from __future__ import annotations
+
+import os
+import re
+import tomllib
+from dataclasses import dataclass, field, fields
+from pathlib import Path
+
+KINDS = ("python-service", "python-update", "docker", "files", "artifact")
+NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.-]{0,63}$")
+UNIT_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.@-]{0,99}$")
+
+
+class ConfigError(ValueError):
+ pass
+
+
+@dataclass
+class Server:
+ host: str = "192.168.88.60"
+ user: str = "das"
+ port: int = 22
+ app_root: str = "/home/das/app"
+ configs_root: str = "/home/das/app/configs"
+ state_dir: str = "/home/das/app_deploy"
+ keep: int = 5
+ local_roots: list[str] = field(default_factory=lambda: ["~/Dev/Projects"])
+ # Development-only files: not synced for python/docker/files apps (an app can sync them via its own rules)
+ dev_exclude: list[str] = field(default_factory=lambda: [
+ "/tests", "/test", "/CLAUDE.md", "/.claude", "/README*.md", "/requirements-dev.txt", "/.gitignore",
+ "/pytest.ini", "/.coverage", "/htmlcov"])
+
+ @property
+ def target(self) -> str:
+ return f"{self.user}@{self.host}"
+
+ @property
+ def snapshots(self) -> str:
+ return f"{self.state_dir}/snapshots"
+
+ @property
+ def home(self) -> str:
+ return "/root" if self.user == "root" else f"/home/{self.user}"
+
+
+@dataclass
+class App:
+ name: str
+ kind: str
+ src: str
+ dest: str
+ # python-*
+ unit: str = ""
+ entry: str = "main.py"
+ user: str = "das"
+ description: str = ""
+ env_file: str = ""
+ syslog_id: str = ""
+ # items kept in configs_root and linked into the app dir (python, docker)
+ configs: list[str] = field(default_factory=list)
+ # sync options
+ exclude: list[str] = field(default_factory=list)
+ use_gitignore: bool = True
+ delete: bool = False
+ # build / artifact
+ build: str = ""
+ build_env: dict[str, str] = field(default_factory=dict)
+ artifact: str = ""
+ # files / artifact
+ mode: str = ""
+ post: str = ""
+ # health
+ health_container: str = ""
+ health_url: str = ""
+ health_timeout: int = 120
+
+ @property
+ def src_path(self) -> Path:
+ return Path(os.path.expanduser(self.src))
+
+ @property
+ def is_python(self) -> bool:
+ return self.kind.startswith("python")
+
+
+@dataclass
+class Config:
+ path: Path
+ server: Server
+ apps: list[App]
+
+ def app(self, name: str) -> App:
+ for a in self.apps:
+ if a.name == name:
+ return a
+ raise ConfigError(f"unknown app: {name}")
+
+ def dest_path(self, app: App) -> str:
+ """Absolute remote path; relative dests are under app_root, '~' is the remote home."""
+ d = app.dest
+ if d.startswith("~"):
+ d = self.server.home + d[1:]
+ elif not d.startswith("/"):
+ d = f"{self.server.app_root}/{d}"
+ return d.rstrip("/") or "/"
+
+
+def _check_types(obj, data: dict, where: str):
+ known = {f.name: f for f in fields(obj)}
+ for k, v in data.items():
+ if k not in known:
+ raise ConfigError(f"{where}: unknown key '{k}'")
+ default = getattr(obj, k)
+ if isinstance(default, bool):
+ ok = isinstance(v, bool)
+ elif isinstance(default, int):
+ ok = isinstance(v, int) and not isinstance(v, bool)
+ elif isinstance(default, list):
+ ok = isinstance(v, list) and all(isinstance(x, str) for x in v)
+ elif isinstance(default, dict):
+ ok = isinstance(v, dict) and all(isinstance(x, str) for x in v.values())
+ else:
+ ok = isinstance(v, str)
+ if not ok:
+ raise ConfigError(f"{where}: '{k}' has wrong type")
+ setattr(obj, k, v)
+
+
+def validate_app(app: App) -> None:
+ w = f"app '{app.name}'"
+ if not NAME_RE.match(app.name):
+ raise ConfigError(f"{w}: bad name")
+ if app.kind not in KINDS:
+ raise ConfigError(f"{w}: kind must be one of {', '.join(KINDS)}")
+ if not app.src or not app.dest:
+ raise ConfigError(f"{w}: src and dest are required")
+ if ".." in Path(app.dest).parts or app.dest.strip("/~") == "":
+ raise ConfigError(f"{w}: unsafe dest '{app.dest}'")
+ if app.is_python:
+ if not UNIT_RE.match(app.unit):
+ raise ConfigError(f"{w}: python apps need a valid 'unit'")
+ if not app.entry.strip():
+ raise ConfigError(f"{w}: empty entry")
+ if app.kind == "artifact" and not app.artifact:
+ raise ConfigError(f"{w}: artifact apps need an 'artifact' glob")
+ for c in app.configs:
+ if "/" in c.strip("/") or c in ("", ".", ".."):
+ raise ConfigError(f"{w}: configs items must be top-level names, got '{c}'")
+ if app.mode and not re.fullmatch(r"[0-7]{3,4}", app.mode):
+ raise ConfigError(f"{w}: mode must be octal like 755")
+
+
+def load(path: str | Path) -> Config:
+ path = Path(path)
+ with open(path, "rb") as f:
+ try:
+ data = tomllib.load(f)
+ except tomllib.TOMLDecodeError as e:
+ raise ConfigError(f"{path}: {e}") from None
+ server = Server()
+ _check_types(server, data.get("server", {}), "[server]")
+ apps, seen = [], set()
+ for i, raw in enumerate(data.get("app", [])):
+ raw = dict(raw)
+ try:
+ app = App(name=raw.pop("name"), kind=raw.pop("kind"), src=raw.pop("src"), dest=raw.pop("dest"))
+ except KeyError as e:
+ raise ConfigError(f"app #{i + 1}: missing {e}") from None
+ _check_types(app, raw, f"app '{app.name}'")
+ validate_app(app)
+ if app.name in seen:
+ raise ConfigError(f"duplicate app '{app.name}'")
+ seen.add(app.name)
+ apps.append(app)
+ return Config(path, server, apps)
+
+
+def _toml_value(v) -> str:
+ if isinstance(v, bool):
+ return "true" if v else "false"
+ if isinstance(v, int):
+ return str(v)
+ if isinstance(v, list):
+ return "[" + ", ".join(_toml_value(x) for x in v) + "]"
+ if isinstance(v, dict):
+ return "{ " + ", ".join(f"{k} = {_toml_value(x)}" for k, x in v.items()) + " }"
+ s = str(v).replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")
+ return f'"{s}"'
+
+
+def app_to_toml(app: App) -> str:
+ """[[app]] block with the non-default keys only."""
+ blank = App(name="", kind="", src="", dest="")
+ lines = ["[[app]]"]
+ for f in fields(app):
+ v = getattr(app, f.name)
+ if f.name in ("name", "kind", "src", "dest") or v != getattr(blank, f.name):
+ lines.append(f"{f.name} = {_toml_value(v)}")
+ return "\n".join(lines) + "\n"
+
+
+def append_apps(path: Path, apps: list[App], drafts: list[App] = ()) -> None:
+ """Append blocks to the file: hand-written comments and order are kept.
+ drafts are written commented out, for review."""
+ text = path.read_text() if path.exists() else ""
+ for a in apps:
+ validate_app(a)
+ text = text.rstrip("\n") + "\n\n" + app_to_toml(a)
+ for a in drafts:
+ block = "".join(f"# {line}\n" for line in app_to_toml(a).splitlines())
+ text = text.rstrip("\n") + "\n\n# TODO: check src / build / dest, then uncomment\n" + block
+ path.write_text(text)
diff --git a/srcpkgs/deployer/files/python/deployer/detect.py b/srcpkgs/deployer/files/python/deployer/detect.py
new file mode 100644
index 0000000..301dfa1
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/detect.py
@@ -0,0 +1,176 @@
+"""Guess how a local project is deployed, and import the apps that already run on the server."""
+from __future__ import annotations
+
+import os
+import re
+from dataclasses import dataclass
+from pathlib import Path
+
+from . import remote
+from .config import App, Config
+
+ENTRIES = ("main.py", "app.py", "app/main.py", "run.py", "server.py")
+CONFIG_NAMES = ("configs", "config", ".env", "secrets.yaml", "secrets.yml")
+CONFIG_RE = re.compile(r"(config|secret)[^/]*\.(ya?ml|json|toml|ini)$", re.I)
+COMPOSE = ("docker-compose.yml", "docker-compose.yaml", "compose.yml", "compose.yaml")
+
+
+def norm(name: str) -> str:
+ return re.sub(r"[^a-z0-9]", "", name.lower())
+
+
+def safe_name(name: str) -> str:
+ return re.sub(r"[^A-Za-z0-9_.-]", "-", name).strip("-.") or "app"
+
+
+def guess_kind(path: Path) -> str:
+ if path.is_file():
+ return "artifact" if path.suffix in (".jar", ".war", ".zip") else "files"
+ names = {p.name for p in path.iterdir()}
+ if names & {"pom.xml", "build.gradle", "build.gradle.kts"}:
+ return "artifact"
+ if names & {"requirements.txt", "pyproject.toml", "main.py", "app.py"}:
+ return "python-service"
+ if names & set(COMPOSE):
+ return "docker"
+ return "files"
+
+
+def guess_entry(path: Path) -> str:
+ return next((e for e in ENTRIES if (path / e).is_file()), "main.py")
+
+
+def guess_configs(path: Path) -> list[str]:
+ found = [n for n in CONFIG_NAMES if (path / n).exists()]
+ found += sorted(p.name for p in path.iterdir()
+ if p.is_file() and CONFIG_RE.search(p.name) and p.name not in found and p.suffix != ".py")
+ return found
+
+
+def java_build(path: Path) -> tuple[str, str]:
+ """(build command, artifact glob) for Maven / Gradle projects."""
+ if (path / "pom.xml").is_file():
+ pom = (path / "pom.xml").read_text(errors="replace")
+ ext = "war" if re.search(r"\s*war\s*", pom) else "jar"
+ mvn = "./mvnw" if (path / "mvnw").is_file() else "mvn"
+ return f"{mvn} -B -q clean package -DskipTests", f"target/*.{ext}"
+ gradle = "./gradlew" if (path / "gradlew").is_file() else "gradle"
+ return f"{gradle} -q build -x test", "build/libs/*[!n].jar" # skip *-plain.jar
+
+
+def new_app(path: Path, name: str | None = None, kind: str | None = None) -> App:
+ """Defaults for a project that is not on the server yet."""
+ path = path.expanduser().resolve()
+ name = safe_name(name or path.stem)
+ kind = kind or guess_kind(path)
+ src = str(path).replace(str(Path.home()), "~", 1)
+ app = App(name=name, kind=kind, src=src, dest=name)
+ if kind.startswith("python"):
+ app.dest = f"python/{name}"
+ app.unit = name
+ app.entry = guess_entry(path)
+ app.description = name
+ app.configs = guess_configs(path)
+ elif kind == "docker":
+ app.dest = f"docker/{name}"
+ app.configs = []
+ elif kind == "artifact":
+ if path.is_dir():
+ app.build, app.artifact = java_build(path)
+ else:
+ app.src, app.artifact = str(Path(src).parent), path.name
+ app.dest = f"docker/java_services/data/app/{name}/"
+ else:
+ app.dest = f"other/{name}/" if path.is_file() else f"other/{name}"
+ return app
+
+
+_IMPORT = r'''
+for f in "$ROOT"/python/*/*.service; do
+ [ -f "$f" ] || continue
+ d=$(dirname "$f"); g() { sed -nE "s/^\s*$1\s*=\s*(.*\S)\s*$/\1/p" "$f" | head -1; }
+ entry=$(sed -nE 's/^ENTRY="(.*)"$/\1/p' "$d/start.sh" 2>/dev/null)
+ links=$(find "$d" -mindepth 1 -maxdepth 1 -type l -lname "$CFG/*" -printf '%f,')
+ echo "P|$(basename "$d")|$(basename "$f" .service)|$(g Description)|$(g User)|$(g SyslogIdentifier)|$(g EnvironmentFile)|$entry|$links"
+done
+for d in "$ROOT"/docker/*/; do
+ ls "$d" | grep -qE '^(docker-)?compose\.ya?ml$' && echo "D|$(basename "$d")"
+done
+for j in "$ROOT"/docker/java_services/data/app/*/*.jar; do
+ [ -f "$j" ] && echo "J|$(basename "$(dirname "$j")")|$(basename "$j")"
+done
+true
+'''
+
+
+@dataclass
+class Imported:
+ app: App
+ matched: bool # a local source was found
+
+
+def local_projects(cfg: Config) -> list[Path]:
+ found = []
+ for root in cfg.server.local_roots:
+ base = Path(os.path.expanduser(root))
+ if not base.is_dir():
+ continue
+ for p in base.glob("*/*"):
+ if p.is_dir() and not p.name.startswith("."):
+ found.append(p)
+ found += [q for q in p.iterdir() if q.is_dir() and not q.name.startswith(".")
+ and ((q / "pom.xml").exists() or (q / "build.gradle").exists())]
+ return found
+
+
+def match_local(projects: list[Path], names: list[str], unit: str = "") -> Path | None:
+ keys = [norm(n) for n in names if n]
+ for p in projects: # exact name
+ if norm(p.name) in keys:
+ return p
+ if unit: # the project ships a unit file with the same name
+ for p in projects:
+ if (p / f"{unit}.service").is_file():
+ return p
+ return None
+
+
+def tilde(p: Path) -> str:
+ return str(p).replace(str(Path.home()), "~", 1)
+
+
+def import_server(cfg: Config, log=print) -> list[Imported]:
+ out = remote.run(cfg.server, _IMPORT, log=None, ROOT=cfg.server.app_root, CFG=cfg.server.configs_root)
+ projects = local_projects(cfg)
+ known = {a.name for a in cfg.apps} | {cfg.dest_path(a) for a in cfg.apps}
+ res = []
+ for line in out.splitlines():
+ f = line.split("|")
+ if f[0] == "P" and len(f) == 9:
+ _, d, unit, desc, user, sid, envf, entry, links = f
+ app = App(name=d, kind="python-update", src="", dest=f"python/{d}", unit=unit, entry=entry or "main.py",
+ user=user or "das", description=desc, syslog_id="" if sid == unit else sid, env_file=envf,
+ configs=[x for x in links.split(",") if x])
+ local = match_local(projects, [d, sid], unit)
+ elif f[0] == "D" and len(f) == 2:
+ app = App(name=f[1], kind="docker", src="", dest=f"docker/{f[1]}")
+ local = match_local(projects, [f[1]])
+ if not local:
+ continue # image-only projects: nothing to deploy from here
+ elif f[0] == "J" and len(f) == 3:
+ dname, jar = f[1], f[2]
+ stem = re.sub(r"[-_]?\d[\w.]*$", "", Path(jar).stem)
+ app = App(name=f"{dname}-jar", kind="artifact", src="", dest=f"docker/java_services/data/app/{dname}/{jar}",
+ post="docker restart JavaApplications", health_container="JavaApplications")
+ local = match_local(projects, [stem, dname])
+ if local:
+ app.build, app.artifact = java_build(local)
+ app.artifact = app.artifact.replace("*.", f"{stem}*.", 1)
+ else:
+ continue
+ if app.name in known or cfg.dest_path(app) in known:
+ continue
+ app.src = tilde(local) if local else ""
+ res.append(Imported(app, local is not None))
+ log(f"{'+' if local else '?'} {app.kind:15} {app.name:20} {app.src or '(no local source found)'}")
+ return res
diff --git a/srcpkgs/deployer/files/python/deployer/kinds/__init__.py b/srcpkgs/deployer/files/python/deployer/kinds/__init__.py
new file mode 100644
index 0000000..045f6e1
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/kinds/__init__.py
@@ -0,0 +1,6 @@
+from .base import KINDS_BY_NAME, Kind, Options, Plan, link_configs # noqa: F401
+from . import docker, files, python # noqa: F401 (register the kinds)
+
+
+def get(name: str) -> Kind:
+ return KINDS_BY_NAME[name]
diff --git a/srcpkgs/deployer/files/python/deployer/kinds/base.py b/srcpkgs/deployer/files/python/deployer/kinds/base.py
new file mode 100644
index 0000000..89d1373
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/kinds/base.py
@@ -0,0 +1,159 @@
+"""Shared pieces of the install kinds."""
+from __future__ import annotations
+
+import shlex
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from .. import remote
+from ..config import App, Config
+
+
+@dataclass
+class Options:
+ build: bool = True
+ install_deps: bool = False # force pip install even if requirements.txt did not change
+ restart: bool = True
+ copy_configs: bool = True # upload missing config items and link them into the app dir
+ delete: bool | None = None # None = app setting
+
+
+@dataclass
+class Plan:
+ cfg: Config
+ app: App
+ opts: Options
+ dest: str
+ first_install: bool
+ rsync_src: str
+ filters: list[str]
+ delete: bool
+ extra: list[str] = field(default_factory=list) # more rsync options
+ # top-level entries of dest on the server: name -> f (file) | d (dir) | l (link)
+ remote_entries: dict[str, str] = field(default_factory=dict)
+ changes: list[remote.Change] = field(default_factory=list)
+ notes: list[str] = field(default_factory=list)
+ artifact_file: Path | None = None
+ git_rev: str = ""
+
+ @property
+ def server(self):
+ return self.cfg.server
+
+
+class Kind:
+ name = ""
+ # Server-side items that are never synced, relative to the app dir
+ protected: tuple[str, ...] = ()
+ dest_is_file = False
+
+ def config_items(self, plan: Plan) -> list[tuple[str, str]]:
+ """(name in the app dir, absolute target under configs_root) of config links."""
+ return []
+
+ def local_config(self, plan: Plan, name: str) -> Path:
+ return plan.app.src_path / name
+
+ def install(self, plan: Plan, log) -> None:
+ pass
+
+ def health(self, plan: Plan, log) -> bool:
+ return health_extra(plan, log)
+
+ def after_restore(self, plan: Plan, log) -> None:
+ self.install(plan, log)
+
+ def status_script(self) -> str:
+ """Bash fragment printing a one-word state; gets $DEST, $UNIT, $C."""
+ return ('if [ -n "$C" ]; then docker inspect -f "{{.State.Status}}" "$C" 2>/dev/null || echo missing; '
+ 'elif [ -e "$DEST" ]; then echo present; else echo missing; fi')
+
+ def logs_command(self, cfg: Config, app: App, dest: str) -> str:
+ if app.health_container:
+ return f"docker logs -f --tail 100 {shlex.quote(app.health_container)}"
+ return f"tail -n 50 -f {shlex.quote(cfg.server.state_dir + '/deploy.log')}"
+
+
+_WAIT_CONTAINER = r'''
+start=$SECONDS; end=$((SECONDS + TIMEOUT))
+while :; do
+ st=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' "$C" 2>/dev/null || echo missing)
+ case $st in
+ # "running" must hold for a few seconds: a crash loop shows up as restarting/exited
+ healthy) echo "$C: $st"; exit 0 ;;
+ running) [ $SECONDS -ge $((start + 6)) ] && { echo "$C: $st"; exit 0; } ;;
+ unhealthy|exited|dead|missing) echo "$C: $st"; docker logs --tail 50 "$C" 2>&1; exit 1 ;;
+ esac
+ [ $SECONDS -lt $end ] || { echo "$C: still $st after ${TIMEOUT}s"; docker logs --tail 50 "$C" 2>&1; exit 1; }
+ sleep 2
+done
+'''
+
+_CHECK_URL = r'''
+end=$((SECONDS + TIMEOUT))
+while :; do
+ code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "$URL")
+ case $code in 2*|3*) echo "$URL: HTTP $code"; exit 0 ;; esac
+ [ $SECONDS -lt $end ] || { echo "$URL: HTTP ${code:-none} after ${TIMEOUT}s"; exit 1; }
+ sleep 2
+done
+'''
+
+
+def health_extra(plan: Plan, log) -> bool:
+ """Optional checks for any kind: a container and/or an URL (requested from the server itself)."""
+ app = plan.app
+ try:
+ if app.health_container:
+ remote.run(plan.server, _WAIT_CONTAINER, log, C=app.health_container, TIMEOUT=app.health_timeout)
+ if app.health_url:
+ remote.run(plan.server, _CHECK_URL, log, URL=app.health_url, TIMEOUT=app.health_timeout)
+ except remote.RemoteError:
+ return False
+ return True
+
+
+_LINK = r'''
+for i in "${!NAMES[@]}"; do
+ n=${NAMES[$i]}; t=${TARGETS[$i]}
+ [ -e "$t" ] || { echo "WARN: $t is missing, $n is not linked"; continue; }
+ ln -s "$t" "$DEST/$n" && echo "linked $n -> $t"
+done
+find "$CFGROOT" -mindepth 1 -type d -exec chmod 700 {} + ; find "$CFGROOT" -type f -exec chmod 600 {} +
+'''
+
+
+def link_configs(plan: Plan, log) -> None:
+ """Config items missing in the app dir: upload the local copy into the configs dir (never overwriting)
+ and link it into the app dir."""
+ kind = KINDS_BY_NAME[plan.app.kind]
+ names, targets = [], []
+ for name, target in kind.config_items(plan):
+ state = plan.remote_entries.get(name)
+ if state == "l":
+ continue
+ if state:
+ log(f"WARN: {name} is a real {'dir' if state == 'd' else 'file'} on the server, left as is")
+ continue
+ if not names:
+ log("== configs")
+ local = kind.local_config(plan, name)
+ if local.is_dir():
+ remote.rsync(plan.server, f"{local}/", f"{target}/", filters=["- .git"], extra=["--ignore-existing"],
+ log=log)
+ elif local.is_file():
+ remote.rsync(plan.server, str(local), target, extra=["--ignore-existing"], log=log)
+ # No local copy: link anyway if the target already exists on the server
+ names.append(name)
+ targets.append(target)
+ if names:
+ remote.run(plan.server, _LINK, log, NAMES=names, TARGETS=targets, DEST=plan.dest,
+ CFGROOT=plan.cfg.server.configs_root)
+
+
+KINDS_BY_NAME: dict[str, Kind] = {}
+
+
+def register(cls: type[Kind]) -> type[Kind]:
+ KINDS_BY_NAME[cls.name] = cls()
+ return cls
diff --git a/srcpkgs/deployer/files/python/deployer/kinds/docker.py b/srcpkgs/deployer/files/python/deployer/kinds/docker.py
new file mode 100644
index 0000000..0ddd4a3
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/kinds/docker.py
@@ -0,0 +1,68 @@
+"""docker: compose project in ~/app/docker/, .env linked from configs/docker/.env."""
+from __future__ import annotations
+
+import shlex
+from pathlib import Path, PurePosixPath
+
+from .. import remote
+from ..config import App, Config
+from .base import Kind, Plan, health_extra, register
+
+_UP = r'''
+cd "$DEST" || exit 1
+if [ ! -e Makefile ] && [ ! -L Makefile ] && [ -f "$APPROOT/docker/Makefile" ] && [ "$(dirname "$DEST")" = "$APPROOT/docker" ]; then
+ ln -s "$APPROOT/docker/Makefile" Makefile && echo "linked Makefile"
+fi
+docker compose config -q || exit 1
+if [ "$RESTART" = 1 ]; then docker compose up -d --build --remove-orphans; else docker compose build; fi
+'''
+
+_HEALTH = r'''
+cd "$DEST" || exit 1
+start=$SECONDS; end=$((SECONDS + TIMEOUT))
+while :; do
+ ps=$(docker compose ps -a --format '{{.Name}} {{.State}} {{.Health}}')
+ bad=$(echo "$ps" | awk '$2=="exited" || $2=="dead" || $3=="unhealthy"')
+ if [ -n "$bad" ]; then echo "$ps"; docker compose logs --tail 30; exit 1; fi
+ wait=$(echo "$ps" | awk 'NF && ($2!="running" || ($3!="" && $3!="healthy"))')
+ if [ -z "$wait" ] && [ $SECONDS -ge $((start + 6)) ]; then echo "$ps"; exit 0; fi
+ [ $SECONDS -lt $end ] || { echo "$ps"; echo "not ready after ${TIMEOUT}s"; exit 1; }
+ sleep 3
+done
+'''
+
+
+@register
+class Docker(Kind):
+ name = "docker"
+ protected = (".env", "data", "Makefile")
+
+ def config_items(self, plan: Plan) -> list[tuple[str, str]]:
+ app = PurePosixPath(plan.dest).name
+ root = f"{plan.cfg.server.configs_root}/docker"
+ return [(".env", f"{root}/{app}.env")] + [(n, f"{root}/{app}/{n}") for n in plan.app.configs]
+
+ def local_config(self, plan: Plan, name: str) -> Path:
+ p = plan.app.src_path / name
+ if name == ".env" and not p.exists():
+ return plan.app.src_path / ".env.example"
+ return p
+
+ def install(self, plan: Plan, log) -> None:
+ remote.run(plan.server, _UP, log, DEST=plan.dest, APPROOT=plan.cfg.server.app_root,
+ RESTART=int(plan.opts.restart))
+
+ def health(self, plan: Plan, log) -> bool:
+ try:
+ remote.run(plan.server, _HEALTH, log, DEST=plan.dest, TIMEOUT=plan.app.health_timeout)
+ except remote.RemoteError:
+ return False
+ return health_extra(plan, log)
+
+ def status_script(self) -> str:
+ return ('cd "$DEST" 2>/dev/null || { echo missing; exit; }; '
+ 't=$(docker compose ps -a -q 2>/dev/null | wc -l); r=$(docker compose ps -q --status running 2>/dev/null | wc -l); '
+ '[ "$t" -gt 0 ] && echo "running $r/$t" || echo stopped')
+
+ def logs_command(self, cfg: Config, app: App, dest: str) -> str:
+ return f"cd {shlex.quote(dest)} && docker compose logs -f --tail=100"
diff --git a/srcpkgs/deployer/files/python/deployer/kinds/files.py b/srcpkgs/deployer/files/python/deployer/kinds/files.py
new file mode 100644
index 0000000..252e849
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/kinds/files.py
@@ -0,0 +1,42 @@
+"""files: a dir or a file copied to any place; artifact: a built file (jar, war, ...) under a fixed name.
+Both run the optional `post` command on the server after the copy (skipped when restart is off)."""
+from __future__ import annotations
+
+from .. import remote
+from .base import Kind, Plan, register
+
+# mode is applied by rsync (--chmod), so unchanged files keep it too
+_POST = r'''
+if [ -n "$POST" ]; then
+ if [ "$IS_FILE" = 1 ]; then cd "$(dirname "$DEST")"; else cd "$DEST"; fi || exit 1
+ echo "+ $POST"; bash -c "$POST" || exit 1
+fi
+'''
+
+
+def run_post(plan: Plan, log, is_file: bool) -> None:
+ if plan.app.post:
+ remote.run(plan.server, _POST, log, DEST=plan.dest, POST=plan.app.post, IS_FILE=int(is_file))
+
+
+@register
+class Files(Kind):
+ name = "files"
+
+ def install(self, plan: Plan, log) -> None:
+ if plan.opts.restart:
+ run_post(plan, log, is_file=plan.app.src_path.is_file())
+ else:
+ log("restart is off: post command skipped")
+
+
+@register
+class Artifact(Kind):
+ name = "artifact"
+ dest_is_file = True
+
+ def install(self, plan: Plan, log) -> None:
+ if plan.opts.restart:
+ run_post(plan, log, is_file=True)
+ else:
+ log("restart is off: post command skipped")
diff --git a/srcpkgs/deployer/files/python/deployer/kinds/python.py b/srcpkgs/deployer/files/python/deployer/kinds/python.py
new file mode 100644
index 0000000..a56197e
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/kinds/python.py
@@ -0,0 +1,103 @@
+"""python-service: full install via ~/app/python/make_py_service.sh; python-update: code + changed deps."""
+from __future__ import annotations
+
+import shlex
+from pathlib import PurePosixPath
+
+from .. import remote
+from ..config import App, Config
+from .base import Kind, Plan, health_extra, register
+
+_DEPS = r'''
+cd "$DEST" || exit 1
+[ -x .venv/bin/python ] || { echo "no .venv in $DEST: deploy it as python-service first"; exit 1; }
+if [ -f requirements.txt ]; then
+ new=$(sha256sum requirements.txt | cut -d' ' -f1); old=$(cat .deploy_req.sha 2>/dev/null)
+ if [ "$FORCE" = 1 ] || [ "$new" != "$old" ]; then
+ echo "installing requirements"
+ .venv/bin/pip install --disable-pip-version-check -q -r requirements.txt || exit 1
+ echo "$new" > .deploy_req.sha
+ else
+ echo "requirements.txt unchanged"
+ fi
+fi
+'''
+
+_SERVICE = r'''
+cd "$DEST" || exit 1
+args=(--entry "$ENTRY" --user "$RUSER" --syslog-id "$SID")
+[ -n "$ENVF" ] && args+=(--env-file "$ENVF")
+"$MAKE" "$DEST" "$UNIT" "$DESC" "${args[@]}" || exit 1
+[ -f requirements.txt ] && sha256sum requirements.txt | cut -d' ' -f1 > .deploy_req.sha
+'''
+
+_RESTART = r'''sudo systemctl restart "$UNIT" && echo "restarted $UNIT"'''
+
+_HEALTH = r'''
+sleep 3; a=$(systemctl show -p NRestarts --value "$UNIT")
+sleep 8; b=$(systemctl show -p NRestarts --value "$UNIT"); st=$(systemctl is-active "$UNIT")
+if [ "$st" = active ] && [ "$a" = "$b" ]; then echo "$UNIT: active, NRestarts=$b"; exit 0; fi
+echo "$UNIT: $st, NRestarts $a -> $b"
+journalctl -u "$UNIT" -n 40 --no-pager
+exit 1
+'''
+
+
+class PythonBase(Kind):
+ protected = (".env", "start.sh", "*.service", "Makefile", ".deploy_req.sha")
+
+ def config_items(self, plan: Plan) -> list[tuple[str, str]]:
+ """A single dir item becomes configs/python/ itself, otherwise each item goes inside it
+ (.env as app.env): the same layout as the existing apps."""
+ app = PurePosixPath(plan.dest).name
+ root = f"{plan.cfg.server.configs_root}/python/{app}"
+ items = plan.app.configs
+ if len(items) == 1 and (plan.app.src_path / items[0]).is_dir():
+ return [(items[0], root)]
+ return [(n, f"{root}/{'app.env' if n == '.env' else n}") for n in items]
+
+ def restart(self, plan: Plan, log) -> None:
+ if plan.opts.restart:
+ remote.run(plan.server, _RESTART, log, UNIT=plan.app.unit)
+
+ def health(self, plan: Plan, log) -> bool:
+ try:
+ remote.run(plan.server, _HEALTH, log, UNIT=plan.app.unit)
+ except remote.RemoteError:
+ return False
+ return health_extra(plan, log)
+
+ def deps(self, plan: Plan, log) -> None:
+ remote.run(plan.server, _DEPS, log, DEST=plan.dest, FORCE=int(plan.opts.install_deps))
+
+ def after_restore(self, plan: Plan, log) -> None:
+ self.deps(plan, log)
+ self.restart(plan, log)
+
+ def status_script(self) -> str:
+ return 'systemctl is-active "$UNIT" 2>/dev/null || true'
+
+ def logs_command(self, cfg: Config, app: App, dest: str) -> str:
+ return f"journalctl -u {shlex.quote(app.unit)} -n 100 -f"
+
+
+@register
+class PythonService(PythonBase):
+ name = "python-service"
+
+ def install(self, plan: Plan, log) -> None:
+ a = plan.app
+ remote.run(plan.server, _SERVICE, log, DEST=plan.dest, UNIT=a.unit, DESC=a.description or a.name,
+ ENTRY=a.entry, RUSER=a.user, SID=a.syslog_id or a.unit, ENVF=a.env_file,
+ MAKE=f"{plan.cfg.server.app_root}/python/make_py_service.sh")
+ # make_py_service.sh installs requirements.txt itself
+ self.restart(plan, log)
+
+
+@register
+class PythonUpdate(PythonBase):
+ name = "python-update"
+
+ def install(self, plan: Plan, log) -> None:
+ self.deps(plan, log)
+ self.restart(plan, log)
diff --git a/srcpkgs/deployer/files/python/deployer/pipeline.py b/srcpkgs/deployer/files/python/deployer/pipeline.py
new file mode 100644
index 0000000..9386c5a
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/pipeline.py
@@ -0,0 +1,271 @@
+"""Deploy steps: preview() builds a Plan (nothing changes on the server), execute() applies it."""
+from __future__ import annotations
+
+import os
+import glob
+import shlex
+import subprocess
+from dataclasses import dataclass
+from datetime import datetime
+from pathlib import Path
+
+from . import kinds, remote, snapshots
+from .config import App, Config
+from .kinds import Options, Plan
+
+# Das OS package: /usr/lib is read-only, so the history goes to the user's state dir
+HISTORY = Path(os.environ.get("XDG_STATE_HOME") or Path.home() / ".local/state") / "deployer" / "history.log"
+
+
+class DeployError(RuntimeError):
+ pass
+
+
+@dataclass
+class Result:
+ ok: bool
+ snapshot: str | None
+ message: str
+
+
+_PROBE = r'''
+if [ -d "$DEST" ]; then
+ echo "EXISTS d"
+ find "$DEST" -mindepth 1 -maxdepth 1 -printf 'E %y %f\n'
+ find "$DEST" -mindepth 2 -maxdepth 2 -type l -printf 'L %P\n'
+elif [ -e "$DEST" ]; then echo "EXISTS f"
+else echo "EXISTS -"
+fi
+'''
+
+
+def git_rev(src: Path) -> str:
+ try:
+ rev = subprocess.run(["git", "-C", str(src), "rev-parse", "--short", "HEAD"], capture_output=True,
+ text=True, timeout=10).stdout.strip()
+ if not rev:
+ return ""
+ dirty = subprocess.run(["git", "-C", str(src), "status", "--porcelain"], capture_output=True, text=True,
+ timeout=10).stdout.strip()
+ return rev + ("-dirty" if dirty else "")
+ except (OSError, subprocess.SubprocessError):
+ return ""
+
+
+def find_artifact(app: App) -> Path:
+ files = [Path(p) for p in glob.glob(str(app.src_path / app.artifact)) if Path(p).is_file()]
+ if not files:
+ raise DeployError(f"no file matches {app.artifact} in {app.src_path}")
+ return max(files, key=lambda p: p.stat().st_mtime)
+
+
+def probe(cfg: Config, dest: str) -> tuple[str, dict[str, str], list[str]]:
+ """(d|f|-, top-level entries, symlinks two levels deep)."""
+ out = remote.run(cfg.server, _PROBE, DEST=dest)
+ exists, entries, links = "-", {}, []
+ for line in out.splitlines():
+ tag, _, rest = line.partition(" ")
+ if tag == "EXISTS":
+ exists = rest
+ elif tag == "E":
+ t, _, name = rest.partition(" ")
+ entries[name] = "l" if t == "l" else ("d" if t == "d" else "f")
+ elif tag == "L":
+ links.append(rest)
+ return exists, entries, links
+
+
+def preview(cfg: Config, app: App, opts: Options, log=print) -> Plan:
+ kind = kinds.get(app.kind)
+ src = app.src_path
+ if not src.exists():
+ raise DeployError(f"local source not found: {src}")
+ if opts.build and app.build:
+ log(f"== build: {app.build}")
+ remote.run_local(app.build, src if src.is_dir() else src.parent, log, app.build_env)
+
+ dest = cfg.dest_path(app)
+ artifact = None
+ if app.kind == "artifact":
+ artifact = find_artifact(app)
+ if app.dest.endswith("/"):
+ dest = f"{dest}/{artifact.name}"
+ rsync_src = str(artifact)
+ elif src.is_file():
+ if app.dest.endswith("/"):
+ dest = f"{dest}/{src.name}"
+ rsync_src = str(src)
+ else:
+ rsync_src = f"{src}/"
+ is_file = rsync_src != f"{src}/"
+
+ log(f"== probe {cfg.server.target}:{dest}")
+ exists, entries, links = probe(cfg, dest)
+ if exists == ("d" if is_file else "f"):
+ raise DeployError(f"{dest} is a {'directory' if exists == 'd' else 'file'} on the server")
+ first = exists == "-"
+ if first and app.kind == "python-update":
+ raise DeployError(f"{dest} does not exist: deploy it with kind python-service first")
+
+ protected = set() if is_file else {*kind.protected, *app.configs, *links,
+ *(n for n, t in entries.items() if t == "l")}
+ filters = [] if is_file else remote.build_filters(protected=protected,
+ exclude=[*cfg.server.dev_exclude, *app.exclude],
+ use_gitignore=app.use_gitignore)
+ delete = (opts.delete if opts.delete is not None else app.delete) and not is_file
+ extra = [f"--chmod=F{app.mode}"] if app.mode else []
+ plan = Plan(cfg=cfg, app=app, opts=opts, dest=dest, first_install=first, rsync_src=rsync_src,
+ filters=filters, delete=delete, extra=extra, remote_entries=entries, artifact_file=artifact,
+ git_rev=git_rev(src if src.is_dir() else src.parent))
+
+ rsync_dest = dest if is_file else f"{dest}/"
+ plan.changes = remote.rsync(cfg.server, rsync_src, rsync_dest, filters=filters, delete=delete,
+ extra=extra, dry_run=True)
+
+ if first:
+ plan.notes.append(f"first install: {dest} will be created")
+ if artifact:
+ plan.notes.append(f"artifact: {artifact.relative_to(src)} ({artifact.stat().st_size // 1024} KiB, "
+ f"{datetime.fromtimestamp(artifact.stat().st_mtime):%Y-%m-%d %H:%M})")
+ if not is_file and opts.copy_configs:
+ for name, target in kind.config_items(plan):
+ state = entries.get(name)
+ if state is None:
+ local = kind.local_config(plan, name)
+ plan.notes.append(f"config {name}: link -> {target}"
+ + (f" (upload {local.name} if missing)" if local.exists() else " (no local copy)"))
+ elif state != "l":
+ plan.notes.append(f"config {name}: real {'dir' if state == 'd' else 'file'} on the server, kept")
+ deleted = sum(c.action == "delete" for c in plan.changes)
+ if deleted:
+ plan.notes.append(f"{deleted} file(s) on the server will be DELETED")
+ if plan.git_rev.endswith("-dirty"):
+ plan.notes.append("working tree has uncommitted changes")
+ return plan
+
+
+def _record(plan: Plan, action: str, result: str, snap: str | None, log) -> None:
+ ts = datetime.now().isoformat(timespec="seconds")
+ line = f"{ts} {plan.app.name} {action} {plan.app.kind} {result} rev={plan.git_rev or '-'} snap={snap or '-'}"
+ try:
+ remote.run(plan.server, 'mkdir -p "$STATE" && printf "%s\\n" "$LINE" >> "$STATE/deploy.log"',
+ STATE=plan.server.state_dir, LINE=line)
+ except remote.RemoteError as e:
+ log(f"WARN: deploy.log not written: {e}")
+ HISTORY.parent.mkdir(parents=True, exist_ok=True)
+ with open(HISTORY, "a") as f:
+ f.write(line + "\n")
+
+
+def execute(plan: Plan, log=print) -> Result:
+ kind = kinds.get(plan.app.kind)
+ is_file = plan.rsync_src != f"{plan.app.src_path}/"
+ snap = None
+ try:
+ if not plan.first_install:
+ added = [c.path for c in plan.changes if c.action == "new" and not c.path.endswith("/")]
+ snap = snapshots.create(plan.server, plan.app.name, plan.dest, [] if is_file else added, log)
+ log(f"== snapshot {snap}")
+ log("== sync")
+ rsync_dest = plan.dest if is_file else f"{plan.dest}/"
+ remote.rsync(plan.server, plan.rsync_src, rsync_dest, filters=plan.filters, delete=plan.delete,
+ extra=plan.extra, log=log)
+ if not is_file and plan.opts.copy_configs:
+ kinds.link_configs(plan, log)
+ log("== install")
+ kind.install(plan, log)
+ ok = True
+ if plan.opts.restart:
+ log("== health")
+ ok = kind.health(plan, log)
+ except (remote.RemoteError, DeployError) as e:
+ log(f"ERROR: {e}")
+ _record(plan, "deploy", "FAILED", snap, log)
+ return Result(False, snap, str(e))
+ _record(plan, "deploy", "ok" if ok else "UNHEALTHY", snap, log)
+ msg = "deployed" if ok else "deployed, but the health check failed"
+ log(f"== {msg}" + (f" (rollback: snapshot {snap})" if snap and not ok else ""))
+ return Result(ok, snap, msg)
+
+
+def base_plan(cfg: Config, app: App, opts: Options | None = None) -> Plan:
+ """Plan without a preview: for rollback, status and logs."""
+ dest = cfg.dest_path(app)
+ if app.kind == "artifact" and app.dest.endswith("/"):
+ dest = f"{dest}/{find_artifact(app).name}"
+ elif app.kind == "files" and app.src_path.is_file() and app.dest.endswith("/"):
+ dest = f"{dest}/{app.src_path.name}"
+ return Plan(cfg=cfg, app=app, opts=opts or Options(), dest=dest, first_install=False, rsync_src="",
+ filters=[], delete=False, git_rev=git_rev(app.src_path if app.src_path.is_dir() else app.src_path.parent))
+
+
+def rollback(cfg: Config, app: App, ts: str | None = None, log=print) -> Result:
+ kind = kinds.get(app.kind)
+ plan = base_plan(cfg, app)
+ snaps = snapshots.listing(cfg.server, app.name)
+ if not snaps:
+ return Result(False, None, "no snapshots")
+ ts = ts or snaps[0]
+ if ts not in snaps:
+ return Result(False, None, f"no snapshot {ts}")
+ is_file = kind.dest_is_file or app.src_path.is_file()
+ try:
+ log(f"== restore {ts}")
+ snapshots.restore(cfg.server, app.name, plan.dest, ts, is_file, log)
+ kind.after_restore(plan, log)
+ ok = kind.health(plan, log) if plan.opts.restart else True
+ except (remote.RemoteError, DeployError) as e:
+ log(f"ERROR: {e}")
+ _record(plan, "rollback", "FAILED", ts, log)
+ return Result(False, ts, str(e))
+ _record(plan, "rollback", "ok" if ok else "UNHEALTHY", ts, log)
+ return Result(ok, ts, f"rolled back to {ts}" + ("" if ok else ", health check failed"))
+
+
+def code_state(local: str, deployed: str) -> str:
+ """Local git rev vs the deployed one: same | changed | dirty (same commit, uncommitted changes) | ? (unknown)."""
+ if not local or not deployed or deployed == "-":
+ return "?"
+ base = local.removesuffix("-dirty")
+ if base != deployed.removesuffix("-dirty"):
+ return "changed"
+ return "same" if local == deployed == base else "dirty"
+
+
+def deployed_rev(fields: list[str]) -> str:
+ """Rev from a deploy.log line; "" when the server code is not known to match it (failed deploy, rollback)."""
+ if len(fields) >= 6 and fields[2] == "deploy" and fields[4] in ("ok", "UNHEALTHY") \
+ and fields[5].startswith("rev="):
+ return fields[5][4:]
+ return ""
+
+
+def status_all(cfg: Config) -> dict[str, tuple[str, str, str]]:
+ """name -> (state, last deploy line, code state vs local) in one ssh call."""
+ parts = []
+ for a in cfg.apps:
+ k = kinds.get(a.kind)
+ dest = cfg.dest_path(a)
+ parts.append(f"DEST={shlex.quote(dest)} UNIT={shlex.quote(a.unit)} C={shlex.quote(a.health_container)}\n"
+ f"printf 'S %s ' {shlex.quote(a.name)}; ( {k.status_script()} ) | head -1")
+ parts.append(f'[ -f {shlex.quote(cfg.server.state_dir)}/deploy.log ] && '
+ f'tail -n 1000 {shlex.quote(cfg.server.state_dir)}/deploy.log | sed "s/^/D /"')
+ out = remote.run(cfg.server, "\n".join(parts) + "\ntrue\n", check=False)
+ states, last, deployed = {}, {}, {}
+ for line in out.splitlines():
+ if line.startswith("S "):
+ _, name, *rest = line.split(" ", 2)
+ states[name] = rest[0].strip() if rest else "?"
+ elif line.startswith("D "):
+ f = line[2:].split()
+ if len(f) >= 5:
+ last[f[1]] = f"{f[0][:16].replace('T', ' ')} {f[2]} {f[4]}"
+ deployed[f[1]] = deployed_rev(f)
+ return {a.name: (states.get(a.name, "?"), last.get(a.name, ""),
+ code_state(git_rev(a.src_path if a.src_path.is_dir() else a.src_path.parent),
+ deployed.get(a.name, "")))
+ for a in cfg.apps}
+
+
+def logs_command(cfg: Config, app: App) -> str:
+ return kinds.get(app.kind).logs_command(cfg, app, cfg.dest_path(app))
diff --git a/srcpkgs/deployer/files/python/deployer/remote.py b/srcpkgs/deployer/files/python/deployer/remote.py
new file mode 100644
index 0000000..030b5e5
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/remote.py
@@ -0,0 +1,136 @@
+"""ssh / rsync wrappers. One multiplexed ssh connection is reused by all calls of a run."""
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+from dataclasses import dataclass
+from typing import Callable, Iterable
+
+from .config import Server
+
+Log = Callable[[str], None]
+
+# Never synced (and so never deleted on the server)
+ALWAYS_EXCLUDE = [".git", ".idea", ".vscode", ".venv", "venv", "__pycache__", ".pytest_cache",
+ ".ruff_cache", ".mypy_cache", "*.pyc", ".DS_Store", "*.iml"]
+
+
+class RemoteError(RuntimeError):
+ pass
+
+
+def ssh_opts(server: Server) -> list[str]:
+ return ["-o", "BatchMode=yes", "-o", "ConnectTimeout=10", "-o", "ControlMaster=auto",
+ "-o", f"ControlPath={os.path.expanduser('~/.ssh')}/cm-%r@%h:%p", "-o", "ControlPersist=120",
+ "-p", str(server.port)]
+
+
+def _stream(cmd: list[str], log: Log | None, stdin: str | None = None, cwd=None, env=None) -> tuple[int, str]:
+ proc = subprocess.Popen(cmd, stdin=subprocess.PIPE if stdin is not None else subprocess.DEVNULL,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, cwd=cwd, env=env,
+ errors="replace")
+ if stdin is not None:
+ proc.stdin.write(stdin)
+ proc.stdin.close()
+ out = []
+ for line in proc.stdout:
+ line = line.rstrip("\n")
+ out.append(line)
+ if log:
+ log(line)
+ return proc.wait(), "\n".join(out)
+
+
+def shell_vars(**values) -> str:
+ """Bash assignments for passing values into a remote script."""
+ lines = []
+ for k, v in values.items():
+ if isinstance(v, (list, tuple)):
+ lines.append(f"{k}=({' '.join(shlex.quote(str(x)) for x in v)})")
+ else:
+ lines.append(f"{k}={shlex.quote(str(v))}")
+ return "\n".join(lines) + "\n"
+
+
+def run(server: Server, script: str, log: Log | None = None, check: bool = True, **values) -> str:
+ """Run a bash script on the server; values become shell variables in front of it."""
+ body = "set -uo pipefail\n" + shell_vars(**values) + script
+ rc, out = _stream(["ssh", *ssh_opts(server), server.target, "bash", "-s"], log, stdin=body)
+ if check and rc != 0:
+ raise RemoteError(f"remote command failed (exit {rc})" + (f": {out.splitlines()[-1]}" if out else ""))
+ return out
+
+
+def run_local(cmd: str, cwd, log: Log | None, env: dict[str, str] | None = None) -> None:
+ full_env = {**os.environ, **{k: os.path.expandvars(os.path.expanduser(v)) for k, v in (env or {}).items()}}
+ rc, _ = _stream(["bash", "-c", cmd], log, cwd=cwd, env=full_env)
+ if rc != 0:
+ raise RemoteError(f"local command failed (exit {rc}): {cmd}")
+
+
+def interactive(server: Server, command: str) -> int:
+ """ssh with a terminal (logs -f); the caller suspends the TUI."""
+ return subprocess.call(["ssh", "-t", *ssh_opts(server), server.target, command])
+
+
+@dataclass
+class Change:
+ action: str # new | update | delete
+ path: str
+
+
+def rsync_cmd(server: Server, src: str, dest: str, *, filters: Iterable[str] = (), delete: bool = False,
+ dry_run: bool = False, extra: Iterable[str] = ()) -> list[str]:
+ # -c: compare by content, so a touched but unchanged file is not reported (or sent) as changed
+ cmd = ["rsync", "-a", "-c", "-z", "--mkpath", "-i", "--out-format=%i %n%L",
+ "-e", shlex.join(["ssh", *ssh_opts(server)])]
+ if delete:
+ cmd.append("--delete")
+ if dry_run:
+ cmd.append("-n")
+ cmd += list(extra)
+ cmd += [f"--filter={f}" for f in filters]
+ return cmd + [src, f"{server.target}:{dest}"]
+
+
+def build_filters(*, protected: Iterable[str], exclude: Iterable[str], use_gitignore: bool) -> list[str]:
+ """rsync filter rules. Anchored excludes keep server-side items (config links, .venv, data) untouched;
+ excluded paths are also protected from --delete."""
+ rules = []
+ for p in sorted(set(protected)):
+ rules.append(f"- /{p.strip('/')}")
+ for p in [*ALWAYS_EXCLUDE, *exclude]:
+ rules.append(f"- {p}")
+ if use_gitignore:
+ rules.append(":- .gitignore")
+ return rules
+
+
+def parse_itemize(output: str) -> list[Change]:
+ changes = []
+ for line in output.splitlines():
+ if line.startswith("*deleting"):
+ changes.append(Change("delete", line.split(None, 1)[1]))
+ continue
+ if len(line) < 13 or line[11] != " ":
+ continue
+ flags, path = line[:11], line[12:]
+ if path in ("./", "."):
+ continue
+ y, x = flags[0], flags[1]
+ if y not in "<>ch":
+ continue # "." = attributes only
+ if x == "d" and y == "c" and "+" not in flags:
+ continue
+ changes.append(Change("new" if flags[2:].strip("+") == "" else "update", path))
+ return changes
+
+
+def rsync(server: Server, src: str, dest: str, *, filters: Iterable[str] = (), delete: bool = False,
+ dry_run: bool = False, extra: Iterable[str] = (), log: Log | None = None) -> list[Change]:
+ cmd = rsync_cmd(server, src, dest, filters=filters, delete=delete, dry_run=dry_run, extra=extra)
+ rc, out = _stream(cmd, None if dry_run else log)
+ if rc != 0:
+ raise RemoteError(f"rsync failed (exit {rc})" + (f": {out.splitlines()[-1]}" if out else ""))
+ return parse_itemize(out)
diff --git a/srcpkgs/deployer/files/python/deployer/snapshots.py b/srcpkgs/deployer/files/python/deployer/snapshots.py
new file mode 100644
index 0000000..3ded863
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/snapshots.py
@@ -0,0 +1,70 @@
+"""Remote snapshots: /snapshots//.tgz (+ .added: files the deploy created)."""
+from __future__ import annotations
+
+import re
+
+from . import remote
+from .config import Server
+
+TS_RE = re.compile(r"^\d{8}-\d{6}$")
+# Heavy or runtime-owned items: not part of the code, never snapshotted or restored
+# .deploy_req.sha stays out so that a rollback sees changed requirements and reinstalls them
+SKIP_DIRS = [".venv", "data", "logs", "node_modules", ".deploy_req.sha"]
+
+_CREATE = r'''
+d="$SNAP/$APP"; mkdir -p "$d"; chmod 700 "$STATE" "$STATE/snapshots" "$d"
+ts=$(date +%Y%m%d-%H%M%S)
+if [ -f "$DEST" ]; then
+ tar czf "$d/$ts.tgz" -C "$(dirname "$DEST")" "$(basename "$DEST")"; rc=$?
+elif [ -d "$DEST" ]; then
+ ex=(); for s in "${SKIP[@]}"; do ex+=(--exclude="./$s"); done
+ tar czf "$d/$ts.tgz" --ignore-failed-read "${ex[@]}" -C "$DEST" .; rc=$?
+else
+ echo "nothing to snapshot"; exit 0
+fi
+# tar: 1 = some files changed while reading, still a usable archive
+[ $rc -le 1 ] || { rm -f "$d/$ts.tgz"; echo "snapshot failed"; exit 1; }
+chmod 600 "$d/$ts.tgz"
+printf '%s\n' "${ADDED[@]}" | sed '/^$/d' > "$d/$ts.added"
+ls -1t "$d"/*.tgz | tail -n +$((KEEP + 1)) | while read -r f; do rm -f "$f" "${f%.tgz}.added"; done
+echo "SNAPSHOT $ts"
+'''
+
+_LIST = r'''ls -1t "$SNAP/$APP" 2>/dev/null | sed -n 's/\.tgz$//p' '''
+
+_RESTORE = r'''
+f="$SNAP/$APP/$TS.tgz"; [ -f "$f" ] || { echo "no snapshot $TS"; exit 1; }
+if [ "$IS_FILE" = 1 ]; then
+ mkdir -p "$(dirname "$DEST")"; tar xzf "$f" -C "$(dirname "$DEST")"
+else
+ mkdir -p "$DEST"
+ # Files that this deploy created did not exist before it
+ if [ -f "$SNAP/$APP/$TS.added" ]; then
+ while IFS= read -r p; do
+ case "$p" in /*|*..*) continue ;; esac
+ [ -f "$DEST/$p" ] || [ -L "$DEST/$p" ] && rm -f "$DEST/$p" && echo "removed $p"
+ done < "$SNAP/$APP/$TS.added"
+ fi
+ tar xzf "$f" -C "$DEST" && echo "restored $TS into $DEST"
+fi
+'''
+
+
+def create(server: Server, app: str, dest: str, added: list[str], log=None) -> str | None:
+ out = remote.run(server, _CREATE, log, SNAP=server.snapshots, STATE=server.state_dir, APP=app, DEST=dest,
+ KEEP=server.keep, SKIP=SKIP_DIRS, ADDED=added)
+ for line in out.splitlines():
+ if line.startswith("SNAPSHOT "):
+ return line.split()[1]
+ return None
+
+
+def listing(server: Server, app: str) -> list[str]:
+ out = remote.run(server, _LIST, check=False, SNAP=server.snapshots, APP=app)
+ return [x for x in out.split() if TS_RE.match(x)]
+
+
+def restore(server: Server, app: str, dest: str, ts: str, is_file: bool, log=None) -> None:
+ if not TS_RE.match(ts):
+ raise ValueError(f"bad snapshot id: {ts}")
+ remote.run(server, _RESTORE, log, SNAP=server.snapshots, APP=app, DEST=dest, TS=ts, IS_FILE=int(is_file))
diff --git a/srcpkgs/deployer/files/python/deployer/tui.py b/srcpkgs/deployer/files/python/deployer/tui.py
new file mode 100644
index 0000000..e25ea73
--- /dev/null
+++ b/srcpkgs/deployer/files/python/deployer/tui.py
@@ -0,0 +1,528 @@
+"""Textual UI: app list, per-run options, preview → confirm → deploy, rollback, logs, add/import."""
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+from pathlib import Path
+
+from rich.markup import escape
+from rich.text import Text
+from textual import on, work
+from textual.app import App, ComposeResult
+from textual.binding import Binding
+from textual.containers import Horizontal, Vertical, VerticalScroll
+from textual.coordinate import Coordinate
+from textual.screen import ModalScreen
+from textual.theme import Theme
+from textual.widgets import Button, Checkbox, DataTable, Footer, Input, Label, OptionList, RichLog, Select, Static
+
+from . import __version__, config, detect, pipeline, remote, snapshots
+from .config import KINDS, Config
+from .kinds import Options, Plan
+
+# Classic CGA palette of Norton / Midnight Commander (same as ServiceManagerTUI)
+BLACK, BLUE, CYAN, GRAY, WHITE, YELLOW = "#000000", "#0000AA", "#00AAAA", "#AAAAAA", "#FFFFFF", "#FFFF55"
+
+MIDNIGHT = Theme(
+ name="midnight", primary=CYAN, secondary=BLUE, accent=YELLOW, warning=YELLOW, error="#FF5555",
+ success="#55FF55", foreground=GRAY, background=BLUE, surface=BLUE, panel=CYAN, dark=True,
+ variables={
+ "border": GRAY, "border-blurred": GRAY,
+ "block-cursor-foreground": BLACK, "block-cursor-background": CYAN, "block-cursor-text-style": "none",
+ "block-cursor-blurred-foreground": BLACK, "block-cursor-blurred-background": CYAN,
+ "block-cursor-blurred-text-style": "none", "block-hover-background": BLUE,
+ "footer-background": BLACK, "footer-foreground": WHITE, "footer-key-foreground": WHITE,
+ "footer-key-background": BLACK, "footer-description-foreground": BLACK,
+ "footer-description-background": CYAN, "footer-item-background": BLACK,
+ "input-cursor-background": BLACK, "input-cursor-foreground": CYAN,
+ "scrollbar": CYAN, "scrollbar-background": BLUE, "scrollbar-corner-color": BLUE,
+ },
+)
+
+STATE_STYLE = {"active": "bold #55FF55", "running": "bold #55FF55", "present": "#55FF55",
+ "failed": "bold #FF5555", "inactive": "#FFFF55", "missing": "#FF5555", "stopped": "#FFFF55",
+ "exited": "#FF5555"}
+CODE_STYLE = {"same": "#55FF55", "changed": "bold #FFFF55", "dirty": "#FFFF55"}
+CHANGE_STYLE = {"new": ("+", "#55FF55"), "update": ("~", "#FFFF55"), "delete": ("-", "bold #FF5555")}
+
+
+def line_text(line: str) -> Text:
+ style = ""
+ if line.startswith("== "):
+ style = "bold #55FFFF"
+ elif line.startswith(("ERROR", "error:")) or "failed" in line.lower():
+ style = "bold #FF5555"
+ elif line.startswith("WARN"):
+ style = "#FFFF55"
+ return Text(line, style=style)
+
+
+class ConfirmPlan(ModalScreen[bool]):
+ BINDINGS = [Binding("escape,n", "dismiss(False)", "Cancel"), Binding("y", "dismiss(True)", "Deploy")]
+
+ def __init__(self, plan: Plan):
+ super().__init__()
+ self.plan = plan
+
+ def compose(self) -> ComposeResult:
+ p = self.plan
+ body = Text()
+ body.append(f"{p.rsync_src}\n→ {p.server.target}:{p.dest}\n", style="bold")
+ if p.git_rev:
+ body.append(f"git {p.git_rev}\n")
+ body.append("\n")
+ for c in p.changes:
+ sign, style = CHANGE_STYLE[c.action]
+ body.append(f"{sign} {c.path}\n", style=style)
+ if not p.changes:
+ body.append("(no file changes: install/restart only)\n", style="italic")
+ for n in p.notes:
+ body.append(f"! {n}\n", style="bold #FFFF55" if "DELETED" not in n else "bold #FF5555")
+ with Vertical(id="dialog"):
+ yield Label(Text(f" Deploy {p.app.name} [{p.app.kind}] "), id="dialog-title")
+ with VerticalScroll():
+ yield Static(body)
+ with Horizontal(id="buttons"):
+ yield Button("Deploy (y)", id="yes", variant="success")
+ yield Button("Cancel (n)", id="no")
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ self.dismiss(e.button.id == "yes")
+
+
+class Ask(ModalScreen[bool]):
+ BINDINGS = [Binding("escape,n", "dismiss(False)", "No"), Binding("y", "dismiss(True)", "Yes")]
+
+ def __init__(self, title: str, text: str):
+ super().__init__()
+ self.title_, self.text = title, text
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog", classes="small"):
+ yield Label(Text(f" {self.title_} "), id="dialog-title")
+ with VerticalScroll():
+ yield Static(self.text)
+ with Horizontal(id="buttons"):
+ yield Button("Yes (y)", id="yes", variant="warning")
+ yield Button("No (n)", id="no")
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ self.dismiss(e.button.id == "yes")
+
+
+class PickSnapshot(ModalScreen[str | None]):
+ BINDINGS = [Binding("escape", "dismiss(None)", "Cancel")]
+
+ def __init__(self, app_name: str, snaps: list[str]):
+ super().__init__()
+ self.app_name, self.snaps = app_name, snaps
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog", classes="small"):
+ yield Label(Text(f" Rollback {self.app_name}: pick a snapshot "), id="dialog-title")
+ yield OptionList(*self.snaps)
+
+ @on(OptionList.OptionSelected)
+ def picked(self, e: OptionList.OptionSelected) -> None:
+ self.dismiss(self.snaps[e.option_index])
+
+
+class AddApp(ModalScreen[config.App | None]):
+ BINDINGS = [Binding("escape", "dismiss(None)", "Cancel")]
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog"):
+ yield Label(" New app ", id="dialog-title")
+ yield Label("Local project dir or file:")
+ yield Input(placeholder="~/Dev/Projects/Python/MyApp", id="path")
+ yield Label("Name (empty = folder name):")
+ yield Input(id="name")
+ yield Label("Kind (empty = detect):")
+ yield Select([(k, k) for k in KINDS], id="kind", prompt="detect")
+ yield Static("", id="toml")
+ with Horizontal(id="buttons"):
+ yield Button("Detect", id="detect")
+ yield Button("Add", id="add", variant="success")
+ yield Button("Cancel", id="cancel")
+
+ def build(self) -> config.App | None:
+ raw = self.query_one("#path", Input).value.strip()
+ path = Path(os.path.expanduser(raw)) if raw else None
+ if not path or not path.exists():
+ self.query_one("#toml", Static).update(Text("path not found", style="bold #FF5555"))
+ return None
+ kind = self.query_one("#kind", Select).value
+ app = detect.new_app(path, self.query_one("#name", Input).value.strip() or None,
+ None if kind is Select.BLANK else kind)
+ self.query_one("#toml", Static).update(Text(config.app_to_toml(app)))
+ return app
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ if e.button.id == "cancel":
+ self.dismiss(None)
+ elif e.button.id == "detect":
+ self.build()
+ else:
+ app = self.build()
+ if app:
+ try:
+ config.validate_app(app)
+ except config.ConfigError as err:
+ self.query_one("#toml", Static).update(Text(str(err), style="bold #FF5555"))
+ return
+ self.dismiss(app)
+
+
+class DeployerApp(App):
+ TITLE = f"Deployer {__version__}"
+ CSS = """
+ Screen { background: $background; }
+ #top { height: 1fr; }
+ #apps { width: 3fr; border: double $border; border-title-color: $accent; }
+ #apps > .datatable--header { background: $panel; color: #000000; text-style: bold; }
+ #apps > .datatable--header-hover { background: $panel; color: #000000; }
+ #apps > .datatable--header-cursor { background: $panel; color: #000000; }
+ #side { width: 2fr; border: double $border; border-title-color: $accent; padding: 0 1; }
+ #info { height: auto; margin-bottom: 1; }
+ #side Checkbox { background: $background; border: none; padding: 0; height: 1; }
+ #side Select { margin-top: 1; }
+ #log { height: 14; border: double $border; border-title-color: $accent; background: $background; }
+ ModalScreen { align: center middle; background: $background 60%; }
+ #dialog { width: 90; max-width: 95%; height: 80%; background: $panel; color: #000000;
+ border: double #FFFFFF; padding: 0 1; }
+ #dialog.small { height: auto; max-height: 80%; }
+ #dialog VerticalScroll { height: 1fr; background: $background; color: $foreground; }
+ #dialog.small VerticalScroll { height: auto; max-height: 20; }
+ #dialog-title { width: 100%; content-align: center middle; color: #0000AA; text-style: bold; }
+ #dialog Input, #dialog Select { margin-bottom: 1; }
+ #dialog OptionList { height: auto; max-height: 20; }
+ #toml { height: auto; background: $background; color: $foreground; }
+ #buttons { height: 3; align: center middle; }
+ #buttons Button { margin: 0 2; }
+ Toast { background: #000000; color: #FFFFFF; }
+ """
+ BINDINGS = [
+ Binding("f2", "edit", "Edit"),
+ Binding("f3", "preview", "Preview"),
+ Binding("f4", "logs", "Logs"),
+ Binding("f5", "deploy", "Deploy"),
+ Binding("f6", "refresh", "Refresh"),
+ Binding("f7", "add", "Add"),
+ Binding("f8", "rollback", "Rollback"),
+ Binding("f9", "import_apps", "Import"),
+ Binding("f10,q", "quit", "Quit"),
+ ]
+
+ def __init__(self, cfg: Config):
+ super().__init__()
+ self.cfg = cfg
+ self.status: dict[str, tuple[str, str, str]] = {}
+ self.busy = False
+
+ # ---------- layout ----------
+ def compose(self) -> ComposeResult:
+ with Horizontal(id="top"):
+ yield DataTable(id="apps", cursor_type="row", zebra_stripes=False)
+ with VerticalScroll(id="side"):
+ yield Static(id="info")
+ yield Select([(k, k) for k in KINDS], id="kind", allow_blank=False, value=KINDS[0])
+ yield Checkbox("Build locally", True, id="o_build")
+ yield Checkbox("Restart + health check", True, id="o_restart")
+ yield Checkbox("Force pip install", False, id="o_deps")
+ yield Checkbox("Create missing config links", True, id="o_configs")
+ yield Checkbox("Delete files absent locally", False, id="o_delete")
+ yield RichLog(id="log", wrap=True, markup=False, max_lines=5000)
+ yield Footer()
+
+ def on_mount(self) -> None:
+ self.register_theme(MIDNIGHT)
+ self.theme = "midnight"
+ t = self.query_one("#apps", DataTable)
+ t.border_title = f"Apps → {self.cfg.server.target}"
+ t.add_columns("Name", "Kind", "State", "Code", "Last deploy")
+ self.query_one("#side").border_title = "Details / options"
+ self.query_one("#log").border_title = "Output"
+ self.fill_table()
+ self.action_refresh()
+
+ def fill_table(self) -> None:
+ t = self.query_one("#apps", DataTable)
+ row = t.cursor_row
+ t.clear()
+ for a in self.cfg.apps:
+ state, last, code = self.status.get(a.name, ("…", "", ""))
+ t.add_row(a.name, a.kind, Text(state, style=STATE_STYLE.get(state.split()[0] if state else "", "")),
+ Text(code, style=CODE_STYLE.get(code, "")), last, key=a.name)
+ if self.cfg.apps:
+ t.move_cursor(row=min(row, len(self.cfg.apps) - 1))
+ self.show_details()
+
+ def current(self) -> config.App | None:
+ t = self.query_one("#apps", DataTable)
+ if not self.cfg.apps or t.row_count == 0:
+ return None
+ key = t.coordinate_to_cell_key(Coordinate(t.cursor_row, 0)).row_key.value
+ return self.cfg.app(key)
+
+ @on(DataTable.RowHighlighted)
+ def show_details(self) -> None:
+ a = self.current()
+ info = self.query_one("#info", Static)
+ if not a:
+ info.update("No apps: F7 add, F9 import from the server")
+ return
+ lines = [f"[b]{escape(a.name)}[/b]", f"src: {escape(a.src)}", f"dest: {escape(self.cfg.dest_path(a))}"]
+ if a.is_python:
+ lines.append(f"unit: {escape(a.unit)} entry: {escape(a.entry)} user: {escape(a.user)}")
+ if a.configs:
+ lines.append(f"configs: {escape(', '.join(a.configs))}")
+ if a.build:
+ lines.append(f"build: {escape(a.build)}")
+ if a.artifact:
+ lines.append(f"artifact: {escape(a.artifact)}")
+ if a.post:
+ lines.append(f"post: {escape(a.post)}")
+ hc = ", ".join(x for x in (a.health_container, a.health_url) if x)
+ if hc:
+ lines.append(f"health: {escape(hc)}")
+ info.update("\n".join(lines))
+ kind = self.query_one("#kind", Select)
+ kind.value = a.kind
+ self.query_one("#o_delete", Checkbox).value = a.delete
+ self.query_one("#o_build", Checkbox).disabled = not a.build
+
+ def options(self) -> Options:
+ v = lambda i: self.query_one(f"#{i}", Checkbox).value # noqa: E731
+ return Options(build=v("o_build"), restart=v("o_restart"), install_deps=v("o_deps"),
+ copy_configs=v("o_configs"), delete=v("o_delete"))
+
+ def run_app(self) -> config.App | None:
+ """The selected app with the kind chosen for this run."""
+ a = self.current()
+ if not a:
+ return None
+ kind = self.query_one("#kind", Select).value
+ if kind != a.kind:
+ from dataclasses import replace
+ a = replace(a, kind=kind)
+ try:
+ config.validate_app(a)
+ except config.ConfigError as e:
+ self.log_line(f"ERROR: {e}")
+ return None
+ return a
+
+ # ---------- output ----------
+ def log_line(self, line: str) -> None:
+ self.query_one("#log", RichLog).write(line_text(line))
+
+ def tlog(self, line: str) -> None:
+ """Log from a worker thread."""
+ self.call_from_thread(self.log_line, line)
+
+ def start(self) -> bool:
+ if self.busy:
+ self.notify("Another operation is running", severity="warning")
+ return False
+ self.busy = True
+ return True
+
+ def done(self) -> None:
+ self.busy = False
+
+ # ---------- actions ----------
+ def action_refresh(self) -> None:
+ self.refresh_status()
+
+ @work(thread=True, group="status", exclusive=True)
+ def refresh_status(self) -> None:
+ try:
+ st = pipeline.status_all(self.cfg)
+ except remote.RemoteError as e:
+ self.tlog(f"ERROR: status: {e}")
+ return
+ self.call_from_thread(self._set_status, st)
+
+ def _set_status(self, st) -> None:
+ self.status = st
+ self.fill_table()
+
+ def action_preview(self) -> None:
+ a = self.run_app()
+ if a and self.start():
+ self.do_preview(a, self.options(), then_deploy=False)
+
+ def action_deploy(self) -> None:
+ a = self.run_app()
+ if a and self.start():
+ self.do_preview(a, self.options(), then_deploy=True)
+
+ @work(thread=True, group="op")
+ def do_preview(self, a: config.App, opts: Options, then_deploy: bool) -> None:
+ try:
+ self.tlog(f"== preview {a.name} [{a.kind}]")
+ plan = pipeline.preview(self.cfg, a, opts, log=self.tlog)
+ except (pipeline.DeployError, remote.RemoteError, config.ConfigError) as e:
+ self.tlog(f"ERROR: {e}")
+ self.call_from_thread(self.done)
+ return
+ for c in plan.changes:
+ sign, _ = CHANGE_STYLE[c.action]
+ self.tlog(f" {sign} {c.path}")
+ for n in plan.notes:
+ self.tlog(f"WARN: {n}")
+ if not plan.changes:
+ self.tlog(" (no file changes)")
+ if not then_deploy:
+ self.call_from_thread(self.done)
+ return
+
+ def answered(ok: bool | None) -> None:
+ if ok:
+ self.do_execute(plan)
+ else:
+ self.log_line("cancelled")
+ self.done()
+
+ self.call_from_thread(self.push_screen, ConfirmPlan(plan), answered)
+
+ @work(thread=True, group="op")
+ def do_execute(self, plan: Plan) -> None:
+ res = pipeline.execute(plan, log=self.tlog)
+ self.call_from_thread(self.after_execute, plan, res)
+
+ def after_execute(self, plan: Plan, res: pipeline.Result) -> None:
+ self.done()
+ self.notify(f"{plan.app.name}: {res.message}", severity="information" if res.ok else "error",
+ timeout=8)
+ self.action_refresh()
+ if not res.ok and res.snapshot:
+ def answered(ok: bool | None) -> None:
+ if ok and self.start():
+ self.do_rollback(plan.app, res.snapshot)
+ self.push_screen(Ask("Deploy failed", f"Roll {plan.app.name} back to snapshot {res.snapshot}?"),
+ answered)
+
+ def action_rollback(self) -> None:
+ a = self.current()
+ if a and self.start():
+ self.load_snapshots(a)
+
+ @work(thread=True, group="op")
+ def load_snapshots(self, a: config.App) -> None:
+ try:
+ snaps = snapshots.listing(self.cfg.server, a.name)
+ except remote.RemoteError as e:
+ snaps = []
+ self.tlog(f"ERROR: {e}")
+ self.call_from_thread(self._pick_snapshot, a, snaps)
+
+ def _pick_snapshot(self, a: config.App, snaps: list[str]) -> None:
+ if not snaps:
+ self.done()
+ self.notify(f"{a.name}: no snapshots", severity="warning")
+ return
+
+ def picked(ts: str | None) -> None:
+ if not ts:
+ self.done()
+ return
+
+ def confirmed(ok: bool | None) -> None:
+ if ok:
+ self.do_rollback(a, ts)
+ else:
+ self.done()
+ self.push_screen(Ask("Rollback", f"Restore {a.name} from {ts} and restart it?"), confirmed)
+
+ self.push_screen(PickSnapshot(a.name, snaps), picked)
+
+ @work(thread=True, group="op")
+ def do_rollback(self, a: config.App, ts: str) -> None:
+ res = pipeline.rollback(self.cfg, a, ts, log=self.tlog)
+ self.tlog(f"== {res.message}")
+ self.call_from_thread(self._rolled_back, a, res)
+
+ def _rolled_back(self, a: config.App, res: pipeline.Result) -> None:
+ self.done()
+ self.notify(f"{a.name}: {res.message}", severity="information" if res.ok else "error", timeout=8)
+ self.action_refresh()
+
+ def action_logs(self) -> None:
+ a = self.current()
+ if not a or self.busy:
+ return
+ with self.suspend():
+ print(f"{a.name}: Ctrl+C to return\n")
+ try:
+ remote.interactive(self.cfg.server, pipeline.logs_command(self.cfg, a))
+ except KeyboardInterrupt:
+ pass
+
+ def action_edit(self) -> None:
+ if self.busy:
+ return
+ editor = os.environ.get("VISUAL") or os.environ.get("EDITOR") or "nano"
+ with self.suspend():
+ subprocess.call([*shlex.split(editor), str(self.cfg.path)])
+ self.reload()
+
+ def reload(self) -> None:
+ try:
+ self.cfg = config.load(self.cfg.path)
+ except (OSError, config.ConfigError) as e:
+ self.log_line(f"ERROR: config: {e} (old config kept)")
+ return
+ self.log_line(f"== config reloaded: {len(self.cfg.apps)} apps")
+ self.fill_table()
+ self.action_refresh()
+
+ def action_add(self) -> None:
+ def added(app: config.App | None) -> None:
+ if app:
+ if any(x.name == app.name for x in self.cfg.apps):
+ self.notify(f"{app.name} already exists", severity="error")
+ return
+ config.append_apps(self.cfg.path, [app])
+ self.log_line(f"== added {app.name}: review it with F2 before the first deploy")
+ self.reload()
+ self.push_screen(AddApp(), added)
+
+ def action_import_apps(self) -> None:
+ if self.start():
+ self.do_import()
+
+ @work(thread=True, group="op")
+ def do_import(self) -> None:
+ try:
+ found = detect.import_server(self.cfg, log=self.tlog)
+ except remote.RemoteError as e:
+ self.tlog(f"ERROR: {e}")
+ found = []
+ self.call_from_thread(self._imported, found)
+
+ def _imported(self, found) -> None:
+ self.done()
+ if not found:
+ self.notify("Nothing new on the server")
+ return
+ ready = [i.app for i in found if i.matched and i.app.kind.startswith("python")]
+ drafts = [i.app for i in found if i.app not in ready]
+ text = "\n".join(f"{'+' if i.app in ready else '?'} {i.app.name} {i.app.src or '(no local source)'}"
+ for i in found)
+
+ def answered(ok: bool | None) -> None:
+ if ok:
+ config.append_apps(self.cfg.path, ready, drafts)
+ self.reload()
+ self.push_screen(Ask(f"Import: {len(ready)} ready, {len(drafts)} commented drafts",
+ text + "\n\nAppend to deploy.toml?"), answered)
+
+
+def run(cfg: Config) -> int:
+ DeployerApp(cfg).run()
+ return 0
diff --git a/srcpkgs/deployer/files/share/deploy.toml.example b/srcpkgs/deployer/files/share/deploy.toml.example
new file mode 100644
index 0000000..afc5d8c
--- /dev/null
+++ b/srcpkgs/deployer/files/share/deploy.toml.example
@@ -0,0 +1,19 @@
+# Deployer registry. Edit by hand or append with `deploy add ` / `deploy import`.
+#
+# kind: python-service full install: .venv + unit via ~/app/python/make_py_service.sh + restart
+# python-update sync code, pip install only when requirements.txt changed, restart
+# docker sync a compose project, `docker compose up -d --build`
+# files sync a dir or a file anywhere (dest relative to app_root, or ~/..., /...)
+# artifact local build, copy the newest `artifact` match to dest (dest/ = keep its name)
+# configs: top-level items kept in configs_root and linked into the app dir; never synced
+# Common keys: exclude, use_gitignore (true), delete (false), build, build_env, post, mode,
+# health_container, health_url (requested on the server), health_timeout
+
+[server]
+host = "server.example.com"
+user = "user"
+app_root = "/home/user/app"
+configs_root = "/home/user/app/configs"
+state_dir = "/home/user/app_deploy"
+keep = 5
+local_roots = ["~/Projects"]
diff --git a/srcpkgs/deployer/files/tests/test_core.py b/srcpkgs/deployer/files/tests/test_core.py
new file mode 100644
index 0000000..fa37b0a
--- /dev/null
+++ b/srcpkgs/deployer/files/tests/test_core.py
@@ -0,0 +1,177 @@
+from pathlib import Path
+
+import pytest
+
+from deployer import config, detect, pipeline, remote
+from deployer.kinds import get
+
+SERVER = """
+[server]
+host = "h"
+app_root = "/srv/app"
+"""
+
+
+def write(tmp_path: Path, body: str) -> Path:
+ p = tmp_path / "deploy.toml"
+ p.write_text(SERVER + body)
+ return p
+
+
+def test_load_and_dest_paths(tmp_path):
+ cfg = config.load(write(tmp_path, """
+[[app]]
+name = "a"
+kind = "python-update"
+src = "~/x"
+dest = "python/a"
+unit = "a"
+configs = ["configs"]
+
+[[app]]
+name = "b"
+kind = "files"
+src = "/tmp"
+dest = "~/scripts"
+"""))
+ assert [a.name for a in cfg.apps] == ["a", "b"]
+ assert cfg.dest_path(cfg.app("a")) == "/srv/app/python/a"
+ assert cfg.dest_path(cfg.app("b")) == "/home/das/scripts"
+ assert cfg.app("a").src_path == Path.home() / "x"
+
+
+@pytest.mark.parametrize("body, msg", [
+ ('name = "a"\nkind = "nope"\nsrc = "x"\ndest = "y"', "kind must be"),
+ ('name = "a"\nkind = "python-update"\nsrc = "x"\ndest = "y"', "unit"),
+ ('name = "a"\nkind = "files"\nsrc = "x"\ndest = "../etc"', "unsafe dest"),
+ ('name = "a"\nkind = "files"\nsrc = "x"\ndest = "y"\ndelete = "yes"', "wrong type"),
+ ('name = "a"\nkind = "files"\nsrc = "x"\ndest = "y"\ntypo = 1', "unknown key"),
+ ('name = "a"\nkind = "artifact"\nsrc = "x"\ndest = "y"', "artifact"),
+ ('name = "a"\nkind = "files"\nsrc = "x"\ndest = "y"\nconfigs = ["a/b"]', "top-level"),
+])
+def test_validation(tmp_path, body, msg):
+ with pytest.raises(config.ConfigError, match=msg):
+ config.load(write(tmp_path, "[[app]]\n" + body))
+
+
+def test_append_round_trip(tmp_path):
+ p = write(tmp_path, "# keep me\n")
+ app = config.App(name="n", kind="docker", src="~/p", dest="docker/n", build_env={"JAVA_HOME": "~/jdk"},
+ post='echo "hi"')
+ draft = config.App(name="d", kind="python-update", src="", dest="python/d", unit="d")
+ config.append_apps(p, [app], [draft])
+ text = p.read_text()
+ assert "# keep me" in text and '# name = "d"' in text
+ cfg = config.load(p)
+ assert [a.name for a in cfg.apps] == ["n"]
+ assert cfg.app("n") == app
+
+
+def test_filters_protect_server_items():
+ rules = remote.build_filters(protected=["configs", "Makefile"], exclude=["/tests"], use_gitignore=True)
+ assert rules[:2] == ["- /Makefile", "- /configs"]
+ assert "- .venv" in rules and "- /tests" in rules
+ assert rules[-1] == ":- .gitignore"
+ assert ":- .gitignore" not in remote.build_filters(protected=[], exclude=[], use_gitignore=False)
+
+
+def test_rsync_cmd_options():
+ s = config.Server()
+ cmd = remote.rsync_cmd(s, "src/", "/d/", filters=["- x"], delete=True, dry_run=True, extra=["--chmod=F640"])
+ assert cmd[0] == "rsync" and "-n" in cmd and "--delete" in cmd and "-c" in cmd
+ assert cmd.index("--chmod=F640") < cmd.index("--filter=- x")
+ assert cmd[-2:] == ["src/", "das@192.168.88.60:/d/"]
+
+
+def test_parse_itemize():
+ out = "\n".join([
+ ".d..t...... ./",
+ " target",
+ "created 2 directories for /x",
+ ])
+ got = [(c.action, c.path) for c in remote.parse_itemize(out)]
+ assert got == [("new", "new.py"), ("update", "changed.py"), ("new", "pkg/"), ("delete", "old.py"),
+ ("new", "link -> target")]
+
+
+@pytest.mark.parametrize("local, deployed, state", [
+ ("abc1234", "abc1234", "same"),
+ ("def5678", "abc1234", "changed"),
+ ("def5678-dirty", "abc1234", "changed"),
+ ("abc1234-dirty", "abc1234", "dirty"),
+ ("abc1234", "abc1234-dirty", "dirty"),
+ ("", "abc1234", "?"),
+ ("abc1234", "", "?"),
+ ("abc1234", "-", "?"),
+])
+def test_code_state(local, deployed, state):
+ assert pipeline.code_state(local, deployed) == state
+
+
+def test_deployed_rev():
+ line = "2026-10-17T10:57:30 EO_Service {} python-update {} rev=22fc8b3 snap=20261017-105714"
+ assert pipeline.deployed_rev(line.format("deploy", "ok").split()) == "22fc8b3"
+ assert pipeline.deployed_rev(line.format("deploy", "UNHEALTHY").split()) == "22fc8b3"
+ assert pipeline.deployed_rev(line.format("deploy", "FAILED").split()) == ""
+ assert pipeline.deployed_rev(line.format("rollback", "ok").split()) == ""
+
+
+def test_shell_vars_quote():
+ v = remote.shell_vars(A="it's", L=["x y", "$z"])
+ assert v == "A='it'\"'\"'s'\nL=('x y' '$z')\n"
+
+
+def test_detect_kinds(tmp_path):
+ py = tmp_path / "MyApp"
+ (py / "configs").mkdir(parents=True)
+ (py / "app.py").write_text("")
+ (py / "secrets.yaml").write_text("")
+ (py / "docker-compose.yml").write_text("")
+ app = detect.new_app(py)
+ assert (app.kind, app.dest, app.unit, app.entry) == ("python-service", "python/MyApp", "MyApp", "app.py")
+ assert app.configs == ["configs", "secrets.yaml"]
+
+ dk = tmp_path / "stack"
+ dk.mkdir()
+ (dk / "compose.yaml").write_text("")
+ assert detect.new_app(dk).kind == "docker"
+
+ jv = tmp_path / "Svc"
+ jv.mkdir()
+ (jv / "pom.xml").write_text("war")
+ (jv / "mvnw").write_text("")
+ a = detect.new_app(jv)
+ assert a.kind == "artifact" and a.artifact == "target/*.war" and a.build.startswith("./mvnw")
+
+
+def test_match_local(tmp_path):
+ a = tmp_path / "DasSimpleExporter"
+ a.mkdir()
+ (a / "dasExporter.service").write_text("")
+ b = tmp_path / "weather-proxy"
+ b.mkdir()
+ assert detect.match_local([a, b], ["WeatherProxy"]) == b
+ assert detect.match_local([a, b], ["DasExporter"], unit="dasExporter") == a
+ assert detect.match_local([a, b], ["nothing"]) is None
+
+
+def test_python_config_layout(tmp_path):
+ (tmp_path / "configs").mkdir()
+ cfg = config.Config(tmp_path / "x.toml", config.Server(), [])
+ plan = type("P", (), {})()
+ plan.cfg, plan.dest = cfg, "/home/das/app/python/App"
+ plan.app = config.App(name="App", kind="python-update", src=str(tmp_path), dest="python/App", unit="u",
+ configs=["configs"])
+ k = get("python-update")
+ assert k.config_items(plan) == [("configs", "/home/das/app/configs/python/App")]
+ plan.app.configs = [".env", "secrets.yaml"]
+ assert k.config_items(plan) == [(".env", "/home/das/app/configs/python/App/app.env"),
+ ("secrets.yaml", "/home/das/app/configs/python/App/secrets.yaml")]
+ plan.dest = "/home/das/app/docker/stack"
+ plan.app = config.App(name="stack", kind="docker", src=str(tmp_path), dest="docker/stack")
+ assert get("docker").config_items(plan) == [(".env", "/home/das/app/configs/docker/stack.env")]
diff --git a/srcpkgs/deployer/template b/srcpkgs/deployer/template
new file mode 100644
index 0000000..ad64e6b
--- /dev/null
+++ b/srcpkgs/deployer/template
@@ -0,0 +1,27 @@
+# Template file for 'deployer'
+# In-tree package: sources are in files/ (FILESDIR), a copy of ~/Dev/Projects/Python/deployer.
+# Differences from the upstream: the config is in ~/.config/deployer (see bin/deploy) and
+# history.log is in ~/.local/state/deployer.
+pkgname=deployer
+version=0.3.0
+revision=1
+create_wrksrc=yes
+hostmakedepends="python3"
+depends="python3 python3-textual openssh rsync"
+short_desc="TUI/CLI deploy of local projects to a server over ssh + rsync"
+maintainer="Das "
+license="MIT"
+homepage="https://git.das.org.ua/das/Das-OS"
+pycompile_dirs="usr/lib/das/deployer"
+python_version=3
+
+do_install() {
+ local f dir
+ for f in "${FILESDIR}"/python/deployer/*.py "${FILESDIR}"/python/deployer/kinds/*.py; do
+ dir=${f#"${FILESDIR}"/python/}
+ vinstall "${f}" 644 "usr/lib/das/deployer/${dir%/*}"
+ done
+ vbin "${FILESDIR}/bin/deploy"
+ vinstall "${FILESDIR}/share/deploy.toml.example" 644 usr/share/das/deployer
+ vlicense "${FILESDIR}/LICENSE"
+}
diff --git a/srcpkgs/publisher/files/LICENSE b/srcpkgs/publisher/files/LICENSE
new file mode 100644
index 0000000..a8f7f83
--- /dev/null
+++ b/srcpkgs/publisher/files/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Das
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/srcpkgs/publisher/files/bin/publish b/srcpkgs/publisher/files/bin/publish
new file mode 100755
index 0000000..dd303fa
--- /dev/null
+++ b/srcpkgs/publisher/files/bin/publish
@@ -0,0 +1,8 @@
+#!/bin/sh
+# publisher launcher: the config lives in ~/.config/publisher/publish.toml (copied from the example on the first run).
+cfg=${XDG_CONFIG_HOME:-$HOME/.config}/publisher/publish.toml
+if [ ! -f "$cfg" ]; then
+ mkdir -p "${cfg%/*}" && cp /usr/share/das/publisher/publish.toml.example "$cfg" || exit 1
+ echo "publish: created $cfg, edit [server] there" >&2
+fi
+PYTHONPATH=/usr/lib/das/publisher exec python3 -c 'import sys; from publisher.cli import main; sys.exit(main())' --config "$cfg" "$@"
diff --git a/srcpkgs/publisher/files/python/publisher/__init__.py b/srcpkgs/publisher/files/python/publisher/__init__.py
new file mode 100644
index 0000000..5b10847
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/__init__.py
@@ -0,0 +1,2 @@
+"""Publish apk, AppImage and binaries to the apps page of the home Tomcat over ssh + rsync."""
+__version__ = "0.3.0"
diff --git a/srcpkgs/publisher/files/python/publisher/cli.py b/srcpkgs/publisher/files/python/publisher/cli.py
new file mode 100644
index 0000000..f1d8def
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/cli.py
@@ -0,0 +1,173 @@
+"""Command-line mode: the same pipeline as the TUI, for scripts and quick publishing."""
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from . import __version__, config, detect, packages, pipeline, remote
+from .pipeline import Options, Plan
+
+SIGN = {"new": ("+", "32"), "replace": ("~", "33"), "meta": ("~", "36"), "notes": ("~", "36"),
+ "delete": ("-", "31"), "same": ("=", "2"), "skip": ("!", "33")}
+COLOR = True
+
+
+def paint(text: str, code: str) -> str:
+ return f"\033[{code}m{text}\033[0m" if COLOR else text
+
+
+def print_plan(plan: Plan) -> None:
+ a = plan.app
+ print(f"\n{a.name} {a.src} -> {plan.server.target}:{plan.server.apps_dir}"
+ f" {('rev ' + plan.git_rev) if plan.git_rev else ''}")
+ for i in plan.items:
+ sign, code = SIGN[i.action]
+ print(paint(f" {sign} {i.describe(a)}", code))
+ for n in plan.notes:
+ print(f" ! {n}")
+
+
+def confirm(question: str, yes: bool) -> bool:
+ if yes:
+ return True
+ if not sys.stdin.isatty():
+ print("not a terminal: pass --yes to confirm", file=sys.stderr)
+ return False
+ return input(f"{question} [y/N] ").strip().lower() in ("y", "yes", "т", "так")
+
+
+def notes_text(args) -> str:
+ if args.notes_file:
+ return Path(args.notes_file).expanduser().read_text()
+ return (args.notes or "").replace("\\n", "\n")
+
+
+def options(args) -> Options:
+ return Options(build=not args.no_build, force=args.force, prune=not args.no_prune, meta=not args.no_meta,
+ notes=notes_text(args))
+
+
+def cmd_list(cfg, args):
+ st, others = pipeline.status_all(cfg)
+ style = {"newer": "1;32", "unpublished": "33", "older": "33", "error": "31", "no build": "2"}
+ for a in cfg.apps:
+ s = st[a.name]
+ print(f"{a.name:22} " + paint(f"{s.state:12}", style.get(s.state, "0"))
+ + f" local: {s.local or '-':32} site: {s.remote or '-'}")
+ if others:
+ print(f"\nnot in {cfg.path.name}: {', '.join(others)}")
+
+
+def cmd_publish(cfg, args):
+ names = [a.name for a in cfg.apps] if args.all else args.apps
+ if not names:
+ print("name apps or pass --all", file=sys.stderr)
+ return 1
+ if args.all:
+ st, _ = pipeline.status_all(cfg)
+ names = [n for n in names if st[n].state in ("newer", "unpublished")]
+ print(f"newer locally: {', '.join(names) or 'none'}")
+ rc = 0
+ for name in names:
+ plan = pipeline.preview(cfg, cfg.app(name), options(args), log=print)
+ print_plan(plan)
+ if args.dry_run or not plan.changes:
+ continue
+ if not confirm(f"publish {name}?", args.yes):
+ rc = rc or 1
+ continue
+ res = pipeline.execute(plan, log=print)
+ rc = rc if res.ok else 2
+ return rc
+
+
+def cmd_versions(cfg, args):
+ for p in pipeline.versions(cfg, cfg.app(args.app)):
+ print(f"{p.version:14} {p.kind.label:9} {p.size / 1048576:8.1f} MB {p.file}")
+ return 0
+
+
+def cmd_remove(cfg, args):
+ app = cfg.app(args.app)
+ if not confirm(f"delete {app.name} v{args.version} from the site?", args.yes):
+ return 1
+ res = pipeline.remove(cfg, app, args.version, log=print)
+ print(res.message)
+ return 0 if res.ok else 2
+
+
+def cmd_add(cfg, args):
+ app = detect.new_app(Path(args.path), args.name)
+ print(config.app_to_toml(app))
+ try:
+ builds, notes = packages.resolve(app)
+ for b in builds:
+ print(f"# found {b.path.name} -> {b.site_name(app.name)}")
+ for n in notes:
+ print(f"# {n}")
+ except packages.PackageError as e:
+ print(f"# {e}")
+ if confirm(f"append to {cfg.path}?", args.yes):
+ config.append_apps(cfg.path, [app])
+ return 0
+
+
+def main(argv=None) -> int:
+ p = argparse.ArgumentParser(prog="publish", description="Publish apk, AppImage and binaries to the apps page "
+ "of the home Tomcat. No command: start the TUI.")
+ p.add_argument("--config", default=None, help="publish.toml (default: next to publish.py)")
+ p.add_argument("--no-color", action="store_true")
+ p.add_argument("-V", "--version", action="version", version=__version__)
+ sub = p.add_subparsers(dest="cmd")
+
+ sub.add_parser("list", help="local and published versions of the apps")
+ for name, hlp in (("publish", "preview and publish apps"), ("preview", "show what a publish would change")):
+ s = sub.add_parser(name, help=hlp)
+ s.add_argument("apps", nargs="*")
+ s.add_argument("-a", "--all", action="store_true", help="every app with a newer local build")
+ s.add_argument("-y", "--yes", action="store_true", help="do not ask for confirmation")
+ s.add_argument("-n", "--dry-run", action="store_true", default=name == "preview")
+ s.add_argument("-m", "--notes", help="what's new in this version (\\n for new lines)")
+ s.add_argument("--notes-file", help="what's new, from a file")
+ s.add_argument("--force", action="store_true", help="publish an older build, overwrite changed files")
+ s.add_argument("--no-build", action="store_true")
+ s.add_argument("--no-prune", action="store_true", help="keep versions beyond `keep`")
+ s.add_argument("--no-meta", action="store_true", help="do not upload the description and the icon")
+ s = sub.add_parser("versions", help="published packages of an app")
+ s.add_argument("app")
+ s = sub.add_parser("remove", help="unpublish a version (all its packages and notes)")
+ s.add_argument("app")
+ s.add_argument("version")
+ s.add_argument("-y", "--yes", action="store_true")
+ s = sub.add_parser("add", help="add a local project to publish.toml")
+ s.add_argument("path")
+ s.add_argument("--name")
+ s.add_argument("-y", "--yes", action="store_true")
+ sub.add_parser("check", help="validate publish.toml")
+ sub.add_parser("tui", help="start the TUI (default)")
+
+ args = p.parse_args(argv)
+ global COLOR
+ COLOR = sys.stdout.isatty() and not args.no_color
+ cfg_path = Path(args.config) if args.config else Path(__file__).resolve().parent.parent / "publish.toml"
+ try:
+ cfg = config.load(cfg_path)
+ except (OSError, config.ConfigError) as e:
+ print(f"config: {e}", file=sys.stderr)
+ return 1
+ if args.cmd in (None, "tui"):
+ from .tui import run
+ return run(cfg)
+ if args.cmd == "check":
+ print(f"{cfg_path}: {len(cfg.apps)} apps, OK")
+ return 0
+ handlers = {"list": cmd_list, "publish": cmd_publish, "preview": cmd_publish, "versions": cmd_versions,
+ "remove": cmd_remove, "add": cmd_add}
+ try:
+ return handlers[args.cmd](cfg, args) or 0
+ except (config.ConfigError, pipeline.PublishError, remote.RemoteError, OSError) as e:
+ print(f"error: {e}", file=sys.stderr)
+ return 1
+ except KeyboardInterrupt:
+ return 130
diff --git a/srcpkgs/publisher/files/python/publisher/config.py b/srcpkgs/publisher/files/python/publisher/config.py
new file mode 100644
index 0000000..b521dbb
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/config.py
@@ -0,0 +1,187 @@
+"""publish.toml: server settings and the registry of published apps."""
+from __future__ import annotations
+
+import os
+import re
+import tomllib
+from dataclasses import dataclass, field, fields
+from pathlib import Path
+
+# as apps.jsp groups it: no "_", "-v", ... inside (that starts the version)
+NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.\-]{0,63}$")
+# Version lines in build files: versionName = "1.2.0" (gradle), Version = "0.10.0" (Go), version = "0.1.0"
+DEFAULT_VERSION_RE = r"""(?:versionName|Version|version)\s*[=:]?\s*["']([0-9][^"']*)["']"""
+
+
+class ConfigError(ValueError):
+ pass
+
+
+@dataclass
+class Server:
+ host: str = "192.168.88.60"
+ user: str = "das"
+ port: int = 22
+ # Package storage of apps.jsp: only download.jsp serves it
+ apps_dir: str = "/home/das/app/docker/tomcat/data/webapps/ttt/WEB-INF/apps"
+ # Requested on the server after an upload; the file name is appended (URL-encoded)
+ check_url: str = "http://localhost:8010/download.jsp?f="
+ state_dir: str = "/home/das/app_deploy"
+ keep: int = 3 # versions of each package type per app; 0 = keep all
+ local_roots: list[str] = field(default_factory=lambda: ["~/Dev/Projects"])
+
+ @property
+ def target(self) -> str:
+ return f"{self.user}@{self.host}"
+
+
+@dataclass
+class App:
+ name: str
+ src: str
+ # globs relative to src; each one gives one package (the newest match)
+ packages: list[str] = field(default_factory=list)
+ # version for files without _v in their name, and a staleness check for the others
+ version_file: str = ""
+ version_regex: str = ""
+ build: str = ""
+ build_env: dict[str, str] = field(default_factory=dict)
+ # local files uploaded as .txt (title + description) and .png|.webp (icon)
+ info: str = ""
+ icon: str = ""
+ keep: int = -1 # -1 = server keep
+
+ @property
+ def src_path(self) -> Path:
+ return Path(os.path.expanduser(self.src))
+
+ def local(self, rel: str) -> Path:
+ p = Path(os.path.expanduser(rel))
+ return p if p.is_absolute() else self.src_path / p
+
+
+@dataclass
+class Config:
+ path: Path
+ server: Server
+ apps: list[App]
+
+ def app(self, name: str) -> App:
+ for a in self.apps:
+ if a.name.lower() == name.lower():
+ return a
+ raise ConfigError(f"unknown app: {name}")
+
+ def keep(self, app: App) -> int:
+ return self.server.keep if app.keep < 0 else app.keep
+
+
+def _check_types(obj, data: dict, where: str):
+ known = {f.name: f for f in fields(obj)}
+ for k, v in data.items():
+ if k not in known:
+ raise ConfigError(f"{where}: unknown key '{k}'")
+ default = getattr(obj, k)
+ if isinstance(default, bool):
+ ok = isinstance(v, bool)
+ elif isinstance(default, int):
+ ok = isinstance(v, int) and not isinstance(v, bool)
+ elif isinstance(default, list):
+ ok = isinstance(v, list) and all(isinstance(x, str) for x in v)
+ elif isinstance(default, dict):
+ ok = isinstance(v, dict) and all(isinstance(x, str) for x in v.values())
+ else:
+ ok = isinstance(v, str)
+ if not ok:
+ raise ConfigError(f"{where}: '{k}' has wrong type")
+ setattr(obj, k, v)
+
+
+def validate_server(s: Server) -> None:
+ d = s.apps_dir.rstrip("/")
+ if not d.startswith("/") or d.count("/") < 3 or ".." in Path(d).parts:
+ raise ConfigError(f"[server]: unsafe apps_dir '{s.apps_dir}'")
+ if s.keep < 0:
+ raise ConfigError("[server]: keep must be >= 0")
+
+
+def validate_app(app: App) -> None:
+ w = f"app '{app.name}'"
+ if not NAME_RE.match(app.name) or re.search(r"[_-]v?\d", app.name):
+ raise ConfigError(f"{w}: bad name (letters, digits, '_.-'; no '_' or '_v')")
+ if not app.src:
+ raise ConfigError(f"{w}: src is required")
+ if not app.packages:
+ raise ConfigError(f"{w}: packages (globs relative to src) are required")
+ for g in app.packages:
+ if not g.strip() or ".." in Path(g).parts:
+ raise ConfigError(f"{w}: bad package glob '{g}'")
+ if app.version_regex:
+ try:
+ if re.compile(app.version_regex).groups < 1:
+ raise ConfigError(f"{w}: version_regex needs a (group)")
+ except re.error as e:
+ raise ConfigError(f"{w}: version_regex: {e}") from None
+ if app.icon and not app.icon.lower().endswith((".png", ".webp")):
+ raise ConfigError(f"{w}: icon must be .png or .webp")
+ if app.keep < -1:
+ raise ConfigError(f"{w}: keep must be >= 0")
+
+
+def load(path: str | Path) -> Config:
+ path = Path(path)
+ with open(path, "rb") as f:
+ try:
+ data = tomllib.load(f)
+ except tomllib.TOMLDecodeError as e:
+ raise ConfigError(f"{path}: {e}") from None
+ server = Server()
+ _check_types(server, data.get("server", {}), "[server]")
+ validate_server(server)
+ apps, seen = [], set()
+ for i, raw in enumerate(data.get("app", [])):
+ raw = dict(raw)
+ try:
+ app = App(name=raw.pop("name"), src=raw.pop("src"))
+ except KeyError as e:
+ raise ConfigError(f"app #{i + 1}: missing {e}") from None
+ _check_types(app, raw, f"app '{app.name}'")
+ validate_app(app)
+ if app.name.lower() in seen:
+ raise ConfigError(f"duplicate app '{app.name}'")
+ seen.add(app.name.lower())
+ apps.append(app)
+ return Config(path, server, apps)
+
+
+def _toml_value(v) -> str:
+ if isinstance(v, bool):
+ return "true" if v else "false"
+ if isinstance(v, int):
+ return str(v)
+ if isinstance(v, list):
+ return "[" + ", ".join(_toml_value(x) for x in v) + "]"
+ if isinstance(v, dict):
+ return "{ " + ", ".join(f"{k} = {_toml_value(x)}" for k, x in v.items()) + " }"
+ s = str(v).replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")
+ return f'"{s}"'
+
+
+def app_to_toml(app: App) -> str:
+ """[[app]] block with the non-default keys only."""
+ blank = App(name="", src="")
+ lines = ["[[app]]"]
+ for f in fields(app):
+ v = getattr(app, f.name)
+ if f.name in ("name", "src") or v != getattr(blank, f.name):
+ lines.append(f"{f.name} = {_toml_value(v)}")
+ return "\n".join(lines) + "\n"
+
+
+def append_apps(path: Path, apps: list[App]) -> None:
+ """Append blocks to the file: hand-written comments and order are kept."""
+ text = path.read_text() if path.exists() else ""
+ for a in apps:
+ validate_app(a)
+ text = text.rstrip("\n") + "\n\n" + app_to_toml(a)
+ path.write_text(text)
diff --git a/srcpkgs/publisher/files/python/publisher/detect.py b/srcpkgs/publisher/files/python/publisher/detect.py
new file mode 100644
index 0000000..7705342
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/detect.py
@@ -0,0 +1,81 @@
+"""Guess an [[app]] block for a local project: Android (+ electron desktop), Go, or any dir with builds."""
+from __future__ import annotations
+
+import os
+import re
+from pathlib import Path
+
+from . import packages
+from .config import App
+
+_ROOT_NAME = re.compile(r"""rootProject\.name\s*=\s*["']([^"']+)["']""")
+_GO_VERSION = re.compile(r'Version\s*=\s*"')
+
+
+def _rel(p: Path, base: Path) -> str:
+ return str(p.relative_to(base))
+
+
+def _home(p: Path) -> str:
+ home = Path.home()
+ return "~/" + str(p.relative_to(home)) if p.is_relative_to(home) else str(p)
+
+
+def _site_name(raw: str) -> str:
+ """A name apps.jsp groups correctly: no spaces, no _."""
+ name = re.sub(r"[^A-Za-z0-9_.\-]+", "", raw)
+ return re.sub(r"[_-]v?(?=\d)", "", name) or "App"
+
+
+def _name_from_builds(files: list[Path]) -> str:
+ for f in files:
+ m = packages.LOCAL_VERSION.search(f.name)
+ if m and m.start() > 0:
+ return f.name[:m.start()]
+ return ""
+
+
+def _first(base: Path, *names: str) -> Path | None:
+ for n in names:
+ if (base / n).is_file():
+ return base / n
+ return None
+
+
+def new_app(path: Path, name: str | None = None) -> App:
+ path = path.expanduser().resolve()
+ # no build command is guessed: Android Studio signs into app/release, gradle into app/build/outputs
+ globs, builds, version_file = [], [], ""
+
+ gradle = _first(path, "app/build.gradle.kts", "app/build.gradle")
+ if gradle:
+ version_file = _rel(gradle, path)
+ # both places of a signed release APK: the best build of the two is published
+ for d in ("app/release", "app/build/outputs/apk/release"):
+ globs.append(f"{d}/*.apk")
+ builds += sorted((path / d).glob("*.apk"))
+ dist = path / "desktop/dist"
+ if dist.is_dir():
+ for ext in (".AppImage", ".deb", ".rpm", ".exe", ".msi", ".dmg"):
+ if any(dist.glob(f"*{ext}")):
+ globs.append(f"desktop/dist/*{ext}")
+ else:
+ go_version = next((p for p in path.glob("internal/version/*.go")
+ if _GO_VERSION.search(p.read_text(errors="replace"))), None)
+ if go_version:
+ version_file = _rel(go_version, path)
+ elif (path / "pyproject.toml").is_file():
+ version_file = "pyproject.toml"
+ for d in ("build", "dist", "bin", "release"):
+ for f in sorted((path / d).glob("*")) if (path / d).is_dir() else []:
+ if f.is_file() and packages.kind_of(f.name, os.access(f, os.X_OK)):
+ builds.append(f)
+ k = packages.kind_of(f.name)
+ globs.append(f"{d}/*{k.ext}" if k and k.ext else f"{d}/{f.name}")
+ globs = list(dict.fromkeys(globs))
+
+ if not name:
+ settings = _first(path, "settings.gradle.kts", "settings.gradle")
+ m = _ROOT_NAME.search(settings.read_text(errors="replace")) if settings else None
+ name = _name_from_builds(builds) or (m.group(1) if m else "") or path.name
+ return App(name=_site_name(name), src=_home(path), packages=globs, version_file=version_file)
diff --git a/srcpkgs/publisher/files/python/publisher/packages.py b/srcpkgs/publisher/files/python/publisher/packages.py
new file mode 100644
index 0000000..4c32b88
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/packages.py
@@ -0,0 +1,152 @@
+"""Package types and names, the same rules as WEB-INF/incl/packages.jspf and apps.jsp on the server."""
+from __future__ import annotations
+
+import glob
+import os
+import re
+from dataclasses import dataclass
+from pathlib import Path
+
+from .config import DEFAULT_VERSION_RE, App
+
+
+@dataclass(frozen=True)
+class Kind:
+ ext: str # as written in published names; "" = executable binary
+ label: str
+
+ @property
+ def executable(self) -> bool:
+ return self.ext in ("", ".AppImage")
+
+
+# Order of apps.jsp download buttons. ".tar.gz" is a compound extension, so it must stay last:
+# kind_of() returns the first endswith match and nothing else ends in ".tar.gz".
+KINDS = (Kind(".apk", "APK"), Kind(".AppImage", "AppImage"), Kind(".deb", "DEB"), Kind(".rpm", "RPM"),
+ Kind(".exe", "EXE"), Kind(".msi", "MSI"), Kind(".dmg", "DMG"), Kind(".tar.gz", "tar.gz"))
+BINARY = Kind("", "binary")
+NOT_PACKAGE = re.compile(r"(?i).*\.(txt|md|png|webp|jpe?g|svg|sh|jsp|html?|json|ya?ml|zip|log)$")
+
+# Server side: apps.jsp PKG_NAME
+SITE_NAME = re.compile(r"(?i)^(.+?)[_-]v?(\d+(?:\.\d+)*[\w.\-]*)$")
+# Local build names: Foo_v1.2.0.apk, Foo_v1.2.0-release.apk, Foo-1.5-release.apk, Foo_v0.7.0_linux_x86_64.AppImage
+LOCAL_VERSION = re.compile(r"[_-]v(\d+(?:\.\d+)*)|[_-](\d+(?:\.\d+)+)")
+
+
+def kind_of(name: str, executable: bool = True) -> Kind | None:
+ if name.startswith("."): # also rsync temp files of an upload in progress
+ return None
+ lower = name.lower()
+ for k in KINDS:
+ if lower.endswith(k.ext.lower()):
+ return k
+ return BINARY if executable and not NOT_PACKAGE.match(name) else None
+
+
+def version_key(v: str) -> tuple[int, ...]:
+ """Numeric-aware, like compareVersions() in apps.jsp: 1.10.0 > 1.9.3, 1.0 == 1.0.0."""
+ parts = [int(x) for x in re.split(r"\D+", v) if x]
+ while parts and parts[-1] == 0:
+ parts.pop()
+ return tuple(parts)
+
+
+def site_name(app: str, version: str, kind: Kind) -> str:
+ return f"{app}_v{version}{kind.ext}"
+
+
+@dataclass
+class Published:
+ """A package file in apps_dir."""
+ file: str
+ app: str
+ version: str
+ kind: Kind
+ size: int = 0
+ mtime: float = 0.0
+
+
+def parse_published(file: str, executable: bool, size: int = 0, mtime: float = 0.0) -> Published | None:
+ kind = kind_of(file, executable)
+ if kind is None:
+ return None
+ base = file[:len(file) - len(kind.ext)] if kind.ext else file
+ m = SITE_NAME.match(base)
+ return Published(file, m.group(1) if m else base, m.group(2) if m else "", kind, size, mtime)
+
+
+@dataclass
+class Local:
+ """A local build chosen for publishing."""
+ path: Path
+ kind: Kind
+ version: str
+ glob: str
+
+ def site_name(self, app: str) -> str:
+ return site_name(app, self.version, self.kind)
+
+
+class PackageError(ValueError):
+ pass
+
+
+def file_version(path: Path) -> str:
+ stem = path.name
+ kind = kind_of(path.name)
+ if kind and kind.ext:
+ stem = stem[:len(stem) - len(kind.ext)]
+ m = LOCAL_VERSION.search(stem)
+ return (m.group(1) or m.group(2)) if m else ""
+
+
+def source_version(app: App) -> str:
+ """Version from app.version_file, '' when not configured."""
+ if not app.version_file:
+ return ""
+ path = app.local(app.version_file)
+ try:
+ text = path.read_text(errors="replace")
+ except OSError as e:
+ raise PackageError(f"version_file: {e}") from None
+ m = re.search(app.version_regex or DEFAULT_VERSION_RE, text)
+ if not m:
+ raise PackageError(f"no version found in {path}")
+ return m.group(1).strip()
+
+
+def resolve(app: App) -> tuple[list[Local], list[str]]:
+ """The best build of each package type over all globs (highest version, then the newest file),
+ and notes about missing or suspicious ones. Several globs of one type are alternative places
+ of the same build, e.g. app/release (Android Studio) and app/build/outputs/apk/release (gradle)."""
+ notes = []
+ src_version = source_version(app)
+ by_kind: dict[Kind, list[Local]] = {}
+ for pattern in app.packages:
+ for p in sorted(glob.glob(str(app.local(pattern)))):
+ path = Path(p)
+ if not path.is_file():
+ continue
+ kind = kind_of(path.name, os.access(path, os.X_OK))
+ if kind is not None:
+ by_kind.setdefault(kind, []).append(Local(path, kind, file_version(path) or src_version, pattern))
+ for pattern in app.packages:
+ expected = kind_of(Path(pattern).name)
+ if expected not in by_kind and not any(b.glob == pattern for bs in by_kind.values() for b in bs):
+ notes.append(f"no build matches {pattern}")
+
+ found = []
+ for kind in [k for k in (*KINDS, BINARY) if k in by_kind]:
+ builds = {b.path.resolve(): b for b in by_kind[kind]}.values() # globs may overlap
+ best = max(builds, key=lambda b: (version_key(b.version), b.path.stat().st_mtime))
+ if not best.version:
+ raise PackageError(f"{best.path.name}: no version in the name and no version_file")
+ twins = [b for b in builds if b is not best and version_key(b.version) == version_key(best.version)
+ and b.path.stat().st_size != best.path.stat().st_size]
+ if twins:
+ notes.append(f"{len(twins) + 1} different builds of {kind.label} v{best.version}: the newest is used, "
+ f"{best.path.relative_to(app.src_path) if best.path.is_relative_to(app.src_path) else best.path}")
+ if src_version and version_key(best.version) < version_key(src_version):
+ notes.append(f"{best.path.name} is v{best.version}, the source is v{src_version}: rebuild?")
+ found.append(best)
+ return found, notes
diff --git a/srcpkgs/publisher/files/python/publisher/pipeline.py b/srcpkgs/publisher/files/python/publisher/pipeline.py
new file mode 100644
index 0000000..9a14afc
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/pipeline.py
@@ -0,0 +1,407 @@
+"""Publish steps: preview() builds a Plan (nothing changes on the server), execute() applies it."""
+from __future__ import annotations
+
+import os
+import hashlib
+import subprocess
+from dataclasses import dataclass, field
+from datetime import datetime
+from pathlib import Path
+from urllib.parse import quote
+
+from . import packages, remote
+from .config import App, Config
+from .packages import Kind, Local, Published, version_key
+
+# Das OS package: /usr/lib is read-only, so the history goes to the user's state dir
+HISTORY = Path(os.environ.get("XDG_STATE_HOME") or Path.home() / ".local/state") / "publisher" / "history.log"
+
+# Plan item actions; the first five change the server
+NEW, REPLACE, META, NOTES, DELETE, SAME, SKIP = "new", "replace", "meta", "notes", "delete", "same", "skip"
+CHANGING = (NEW, REPLACE, META, NOTES, DELETE)
+
+
+class PublishError(RuntimeError):
+ pass
+
+
+@dataclass
+class Options:
+ build: bool = True
+ force: bool = False # publish an older version, overwrite a published file with other content
+ prune: bool = True # delete versions beyond keep
+ meta: bool = True # upload changed .txt / icon
+ notes: str = "" # "what's new" of the published version
+
+
+@dataclass
+class Item:
+ action: str
+ target: str # file name in apps_dir
+ local: Path | None = None
+ kind: Kind | None = None
+ version: str = ""
+ reason: str = ""
+
+ @property
+ def is_package(self) -> bool:
+ return self.kind is not None
+
+ def describe(self, app: App) -> str:
+ """' -> ' and size/reason, for plan listings."""
+ src = ""
+ if self.local:
+ rel = self.local.relative_to(app.src_path) if self.local.is_relative_to(app.src_path) else self.local
+ src = f"{rel} -> " if str(rel) != self.target else ""
+ size = f" ({self.local.stat().st_size / 1048576:.1f} MB)" if self.local and self.is_package else ""
+ return f"{src}{self.target}{size}" + (f" [{self.reason}]" if self.reason else "")
+
+
+@dataclass
+class Plan:
+ cfg: Config
+ app: App
+ opts: Options
+ items: list[Item] = field(default_factory=list)
+ notes: list[str] = field(default_factory=list)
+ git_rev: str = ""
+
+ @property
+ def server(self):
+ return self.cfg.server
+
+ @property
+ def changes(self) -> list[Item]:
+ return [i for i in self.items if i.action in CHANGING]
+
+
+@dataclass
+class Result:
+ ok: bool
+ message: str
+
+
+@dataclass
+class RemoteState:
+ # name -> (size, mtime, executable)
+ files: dict[str, tuple[int, float, bool]] = field(default_factory=dict)
+ hashes: dict[str, str] = field(default_factory=dict)
+
+ def published(self, app: str | None = None) -> list[Published]:
+ """Packages of an app (all packages for None), grouped case-insensitively like apps.jsp."""
+ out = []
+ for name, (size, mtime, ex) in self.files.items():
+ p = packages.parse_published(name, ex, size, mtime)
+ if p and (app is None or p.app.lower() == app.lower()):
+ out.append(p)
+ return out
+
+
+_STATE = r'''
+[ -d "$DIR" ] || { echo "no such dir: $DIR" >&2; exit 3; }
+find "$DIR" -maxdepth 1 -type f -printf 'F %s %T@ %m %f\n'
+cd "$DIR" && for n in "${HASH[@]}"; do [ -f "$n" ] && sha256sum -- "$n" | sed 's/^/H /'; done
+true
+'''
+
+
+def remote_state(cfg: Config, hash_names: list[str] = ()) -> RemoteState:
+ out = remote.run(cfg.server, _STATE, DIR=cfg.server.apps_dir, HASH=list(hash_names))
+ st = RemoteState()
+ for line in out.splitlines():
+ if line.startswith("F "):
+ _, size, mtime, mode, name = line.split(" ", 4)
+ st.files[name] = (int(size), float(mtime), int(mode, 8) & 0o111 != 0)
+ elif line.startswith("H "):
+ _, sha, name = line.split(None, 2)
+ st.hashes[name] = sha
+ return st
+
+
+def sha256(path: Path) -> str:
+ h = hashlib.sha256()
+ with open(path, "rb") as f:
+ for chunk in iter(lambda: f.read(1 << 20), b""):
+ h.update(chunk)
+ return h.hexdigest()
+
+
+def git_rev(src: Path) -> str:
+ try:
+ rev = subprocess.run(["git", "-C", str(src), "rev-parse", "--short", "HEAD"], capture_output=True,
+ text=True, timeout=10).stdout.strip()
+ if not rev:
+ return ""
+ dirty = subprocess.run(["git", "-C", str(src), "status", "--porcelain"], capture_output=True, text=True,
+ timeout=10).stdout.strip()
+ return rev + ("-dirty" if dirty else "")
+ except (OSError, subprocess.SubprocessError):
+ return ""
+
+
+def meta_files(app: App) -> list[tuple[Path, str]]:
+ """(local file, name in apps_dir) of the description and the icon."""
+ out = []
+ if app.info:
+ out.append((app.local(app.info), f"{app.name}.txt"))
+ if app.icon:
+ icon = app.local(app.icon)
+ out.append((icon, f"{app.name}{icon.suffix.lower()}"))
+ return out
+
+
+def notes_name(app: str, version: str) -> str:
+ return f"{app}_v{version}.txt"
+
+
+def prune(published: list[Published], keep: int) -> list[Published]:
+ """Packages beyond the newest `keep` versions of each type."""
+ if keep <= 0:
+ return []
+ out = []
+ for kind in {p.kind for p in published}:
+ same = sorted((p for p in published if p.kind == kind), key=lambda p: (version_key(p.version), p.mtime),
+ reverse=True)
+ out += same[keep:]
+ return out
+
+
+def preview(cfg: Config, app: App, opts: Options, log=print) -> Plan:
+ src = app.src_path
+ if not src.exists():
+ raise PublishError(f"local source not found: {src}")
+ if opts.build and app.build:
+ log(f"== build: {app.build}")
+ remote.run_local(app.build, src if src.is_dir() else src.parent, log, app.build_env)
+ try:
+ builds, notes = packages.resolve(app)
+ except packages.PackageError as e:
+ raise PublishError(str(e)) from None
+ if not builds:
+ raise PublishError("nothing to publish: " + "; ".join(notes))
+ meta = [(p, n) for p, n in meta_files(app) if opts.meta]
+ for p, _ in meta:
+ if not p.is_file():
+ notes.append(f"no local {p}")
+ meta = [(p, n) for p, n in meta if p.is_file()]
+
+ log(f"== probe {cfg.server.target}:{cfg.server.apps_dir}")
+ st = remote_state(cfg, [b.site_name(app.name) for b in builds] + [n for _, n in meta])
+ published = st.published(app.name)
+ plan = Plan(cfg, app, opts, notes=notes, git_rev=git_rev(src if src.is_dir() else src.parent))
+
+ for b in builds:
+ plan.items.append(_package_item(app, b, st, published, opts))
+ added = [i for i in plan.items if i.action in (NEW, REPLACE)]
+
+ for path, name in meta:
+ if st.hashes.get(name) != sha256(path):
+ plan.items.append(Item(META, name, path, reason="update" if name in st.files else "new"))
+
+ if opts.notes.strip():
+ for v in sorted({b.version for b in builds}, key=version_key):
+ name = notes_name(app.name, v)
+ plan.items.append(Item(NOTES, name, version=v, reason="update" if name in st.files else "new"))
+
+ if opts.prune:
+ after = [p for p in published if p.file not in {i.target for i in added}]
+ after += [Published(i.target, app.name, i.version, i.kind, mtime=float("inf")) for i in added]
+ gone = prune(after, cfg.keep(app))
+ for p in gone:
+ plan.items.append(Item(DELETE, p.file, kind=p.kind, version=p.version, reason=f"keep {cfg.keep(app)}"))
+ left = {version_key(p.version) for p in after if p not in gone}
+ for v in sorted({p.version for p in gone}, key=version_key):
+ n = notes_name(app.name, v)
+ if version_key(v) not in left and n in st.files:
+ plan.items.append(Item(DELETE, n, version=v, reason="notes of a deleted version"))
+
+ if not added:
+ plan.notes.append("no new packages" + ("" if opts.force else " (--force publishes older or changed builds)"))
+ if plan.git_rev.endswith("-dirty"):
+ plan.notes.append("working tree has uncommitted changes")
+ return plan
+
+
+def _package_item(app: App, b: Local, st: RemoteState, published: list[Published], opts: Options) -> Item:
+ target = b.site_name(app.name)
+ item = Item(NEW, target, b.path, b.kind, b.version)
+ if target in st.files:
+ if st.hashes.get(target) == sha256(b.path):
+ item.action, item.reason = SAME, "already published"
+ elif opts.force:
+ item.action, item.reason = REPLACE, "same name, other content"
+ else:
+ item.action, item.reason = SKIP, "published with other content: --force overwrites"
+ return item
+ newest = max((p for p in published if p.kind == b.kind), key=lambda p: version_key(p.version), default=None)
+ if newest and version_key(b.version) <= version_key(newest.version):
+ what = "older than" if version_key(b.version) < version_key(newest.version) else "same version as"
+ if opts.force:
+ item.reason = f"{what} {newest.file}"
+ else:
+ item.action, item.reason = SKIP, f"{what} {newest.file}"
+ return item
+
+
+_WRITE = r'''
+umask 022
+printf '%s\n' "$TEXT" > "$DIR/.$NAME.tmp" && mv -f "$DIR/.$NAME.tmp" "$DIR/$NAME" && echo "wrote $NAME"
+'''
+
+_DELETE = r'''
+cd "$DIR" || exit 1
+for n in "${NAMES[@]}"; do rm -f -- "$n" && echo "deleted $n"; done
+'''
+
+_CHECK = r'''
+rc=0
+for i in "${!URLS[@]}"; do
+ h=$(curl -sI --max-time 15 "${URLS[$i]}" | tr -d '\r')
+ code=$(printf '%s\n' "$h" | head -1 | cut -d' ' -f2)
+ len=$(printf '%s\n' "$h" | grep -i '^content-length:' | tail -1 | awk '{print $2}')
+ if [ "$code" = 200 ] && [ "$len" = "${SIZES[$i]}" ]; then
+ echo "${NAMES[$i]}: HTTP 200, $len bytes"
+ else
+ echo "ERROR: ${NAMES[$i]}: HTTP ${code:-none}, ${len:-?} bytes (expected ${SIZES[$i]})"; rc=1
+ fi
+done
+exit $rc
+'''
+
+
+def _delete(cfg: Config, names: list[str], log) -> None:
+ if any("/" in n or n.startswith(".") or not n for n in names):
+ raise PublishError(f"refusing to delete {names}")
+ remote.run(cfg.server, _DELETE, log, DIR=cfg.server.apps_dir, NAMES=names)
+
+
+def check(cfg: Config, files: list[tuple[str, int]], log) -> bool:
+ """The site serves each file with the right size (asked on the server)."""
+ if not files or not cfg.server.check_url:
+ return True
+ try:
+ remote.run(cfg.server, _CHECK, log, URLS=[cfg.server.check_url + quote(n) for n, _ in files],
+ NAMES=[n for n, _ in files], SIZES=[s for _, s in files])
+ except remote.RemoteError:
+ return False
+ return True
+
+
+def _record(cfg: Config, app: App, action: str, result: str, files: list[str], rev: str, log) -> None:
+ ts = datetime.now().isoformat(timespec="seconds")
+ line = f"{ts} {app.name} {action} {result} rev={rev or '-'} {','.join(files) or '-'}"
+ try:
+ remote.run(cfg.server, 'mkdir -p "$STATE" && printf "%s\\n" "$LINE" >> "$STATE/publish.log"',
+ STATE=cfg.server.state_dir, LINE=line)
+ except remote.RemoteError as e:
+ log(f"WARN: publish.log not written: {e}")
+ HISTORY.parent.mkdir(parents=True, exist_ok=True)
+ with open(HISTORY, "a") as f:
+ f.write(line + "\n")
+
+
+def execute(plan: Plan, log=print) -> Result:
+ cfg, app, d = plan.cfg, plan.app, plan.server.apps_dir
+ done: list[str] = []
+ try:
+ # notes first: the page shows them together with the new package
+ for i in plan.items:
+ if i.action == NOTES:
+ remote.run(cfg.server, _WRITE, log, DIR=d, NAME=i.target, TEXT=plan.opts.notes.strip())
+ done.append(i.target)
+ for i in plan.items:
+ if i.action in (NEW, REPLACE, META):
+ size = i.local.stat().st_size
+ log(f"== upload {i.local.name} -> {i.target} ({size / 1048576:.1f} MB)")
+ mode = "755" if i.kind and i.kind.executable else "644"
+ remote.upload(cfg.server, str(i.local), f"{d}/{i.target}", mode, log)
+ done.append(i.target)
+ gone = [i.target for i in plan.items if i.action == DELETE]
+ if gone:
+ log("== prune")
+ _delete(cfg, gone, log)
+ done += [f"-{n}" for n in gone]
+ uploaded = [(i.target, i.local.stat().st_size) for i in plan.items if i.action in (NEW, REPLACE)]
+ ok = True
+ if uploaded and cfg.server.check_url:
+ log("== check")
+ ok = check(cfg, uploaded, log)
+ except (remote.RemoteError, PublishError, OSError) as e:
+ log(f"ERROR: {e}")
+ _record(cfg, app, "publish", "FAILED", done, plan.git_rev, log)
+ return Result(False, str(e))
+ _record(cfg, app, "publish", "ok" if ok else "UNCHECKED", done, plan.git_rev, log)
+ msg = "published" if ok else "published, but the site does not serve it correctly"
+ log(f"== {msg}")
+ return Result(ok, msg)
+
+
+def versions(cfg: Config, app: App) -> list[Published]:
+ """Published packages of the app, newest first."""
+ pub = remote_state(cfg).published(app.name)
+ return sorted(pub, key=lambda p: (version_key(p.version), -packages.KINDS.index(p.kind)
+ if p.kind in packages.KINDS else 0), reverse=True)
+
+
+def remove(cfg: Config, app: App, version: str, log=print) -> Result:
+ """Unpublish one version: all its packages and its notes."""
+ st = remote_state(cfg)
+ names = [p.file for p in st.published(app.name) if version_key(p.version) == version_key(version)]
+ if not names:
+ return Result(False, f"{app.name} v{version} is not published")
+ n = notes_name(app.name, version)
+ if n in st.files:
+ names.append(n)
+ try:
+ _delete(cfg, names, log)
+ except (remote.RemoteError, PublishError) as e:
+ log(f"ERROR: {e}")
+ return Result(False, str(e))
+ _record(cfg, app, "remove", "ok", [f"-{x}" for x in names], "", log)
+ return Result(True, f"removed {', '.join(names)}")
+
+
+@dataclass
+class Status:
+ state: str # current | newer | unpublished | older | no build | error
+ local: str = "" # "apk 1.2.0, AppImage 1.2.0"
+ remote: str = ""
+
+
+def _summary(pairs: list[tuple[Kind, str]]) -> str:
+ return ", ".join(f"{k.label} {v}" for k, v in sorted(pairs, key=lambda x: version_key(x[1]), reverse=True))
+
+
+def status_all(cfg: Config) -> tuple[dict[str, Status], list[str]]:
+ """name -> Status (versions only, contents are compared by preview), and published apps
+ that are not in publish.toml."""
+ st = remote_state(cfg)
+ pub = st.published()
+ out = {}
+ for a in cfg.apps:
+ mine = [p for p in pub if p.app.lower() == a.name.lower()]
+ latest: dict[Kind, Published] = {}
+ for p in mine:
+ if p.kind not in latest or version_key(p.version) > version_key(latest[p.kind].version):
+ latest[p.kind] = p
+ rem = _summary([(k, p.version) for k, p in latest.items()])
+ try:
+ builds, _ = packages.resolve(a)
+ except packages.PackageError as e:
+ out[a.name] = Status("error", str(e), rem)
+ continue
+ loc = _summary([(b.kind, b.version) for b in builds])
+ if not builds:
+ state = "no build"
+ elif not latest:
+ state = "unpublished"
+ else:
+ cmp = [version_key(b.version) > version_key(latest[b.kind].version) if b.kind in latest else True
+ for b in builds]
+ older = [b.kind in latest and version_key(b.version) < version_key(latest[b.kind].version)
+ for b in builds]
+ state = "newer" if any(cmp) else ("older" if any(older) else "current")
+ out[a.name] = Status(state, loc, rem)
+ known = {a.name.lower() for a in cfg.apps}
+ others = sorted({p.app for p in pub if p.app.lower() not in known}, key=str.lower)
+ return out, others
diff --git a/srcpkgs/publisher/files/python/publisher/remote.py b/srcpkgs/publisher/files/python/publisher/remote.py
new file mode 100644
index 0000000..d1cde1f
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/remote.py
@@ -0,0 +1,74 @@
+"""ssh / rsync wrappers. One multiplexed ssh connection is reused by all calls of a run."""
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+from typing import Callable
+
+from .config import Server
+
+Log = Callable[[str], None]
+
+
+class RemoteError(RuntimeError):
+ pass
+
+
+def ssh_opts(server: Server) -> list[str]:
+ return ["-o", "BatchMode=yes", "-o", "ConnectTimeout=10", "-o", "ControlMaster=auto",
+ "-o", f"ControlPath={os.path.expanduser('~/.ssh')}/cm-%r@%h:%p", "-o", "ControlPersist=120",
+ "-p", str(server.port)]
+
+
+def _stream(cmd: list[str], log: Log | None, stdin: str | None = None, cwd=None, env=None) -> tuple[int, str]:
+ proc = subprocess.Popen(cmd, stdin=subprocess.PIPE if stdin is not None else subprocess.DEVNULL,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, cwd=cwd, env=env,
+ errors="replace")
+ if stdin is not None:
+ proc.stdin.write(stdin)
+ proc.stdin.close()
+ out = []
+ for line in proc.stdout:
+ line = line.rstrip("\n")
+ out.append(line)
+ if log:
+ log(line)
+ return proc.wait(), "\n".join(out)
+
+
+def shell_vars(**values) -> str:
+ """Bash assignments for passing values into a remote script."""
+ lines = []
+ for k, v in values.items():
+ if isinstance(v, (list, tuple)):
+ lines.append(f"{k}=({' '.join(shlex.quote(str(x)) for x in v)})")
+ else:
+ lines.append(f"{k}={shlex.quote(str(v))}")
+ return "\n".join(lines) + "\n"
+
+
+def run(server: Server, script: str, log: Log | None = None, check: bool = True, **values) -> str:
+ """Run a bash script on the server; values become shell variables in front of it."""
+ body = "set -uo pipefail\n" + shell_vars(**values) + script
+ rc, out = _stream(["ssh", *ssh_opts(server), server.target, "bash", "-s"], log, stdin=body)
+ if check and rc != 0:
+ raise RemoteError(f"remote command failed (exit {rc})" + (f": {out.splitlines()[-1]}" if out else ""))
+ return out
+
+
+def run_local(cmd: str, cwd, log: Log | None, env: dict[str, str] | None = None) -> None:
+ full_env = {**os.environ, **{k: os.path.expandvars(os.path.expanduser(v)) for k, v in (env or {}).items()}}
+ rc, _ = _stream(["bash", "-c", cmd], log, cwd=cwd, env=full_env)
+ if rc != 0:
+ raise RemoteError(f"local command failed (exit {rc}): {cmd}")
+
+
+def upload(server: Server, src: str, dest: str, mode: str, log: Log | None = None) -> None:
+ """Copy one file. rsync writes a hidden temp file and renames it, so the site never lists a partial
+ package; the server time becomes the publish time (the "new" badge of apps.jsp)."""
+ cmd = ["rsync", "--checksum", "--perms", f"--chmod=F{mode}", "--mkpath", "--out-format=sent %n %b bytes",
+ "-e", shlex.join(["ssh", *ssh_opts(server)]), src, f"{server.target}:{dest}"]
+ rc, out = _stream(cmd, log)
+ if rc != 0:
+ raise RemoteError(f"rsync failed (exit {rc})" + (f": {out.splitlines()[-1]}" if out else ""))
diff --git a/srcpkgs/publisher/files/python/publisher/tui.py b/srcpkgs/publisher/files/python/publisher/tui.py
new file mode 100644
index 0000000..269f79e
--- /dev/null
+++ b/srcpkgs/publisher/files/python/publisher/tui.py
@@ -0,0 +1,466 @@
+"""Textual UI: app list with local and published versions, preview → confirm → publish, remove versions, add."""
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+from pathlib import Path
+
+from rich.markup import escape
+from rich.text import Text
+from textual import on, work
+from textual.app import App, ComposeResult
+from textual.binding import Binding
+from textual.containers import Horizontal, Vertical, VerticalScroll
+from textual.coordinate import Coordinate
+from textual.screen import ModalScreen
+from textual.theme import Theme
+from textual.widgets import Button, Checkbox, DataTable, Footer, Input, Label, OptionList, RichLog, Static, TextArea
+
+from . import __version__, config, detect, pipeline, remote
+from .config import Config
+from .packages import Published
+from .pipeline import Options, Plan
+
+# Classic CGA palette of Norton / Midnight Commander (same as Deployer)
+BLACK, BLUE, CYAN, GRAY, WHITE, YELLOW = "#000000", "#0000AA", "#00AAAA", "#AAAAAA", "#FFFFFF", "#FFFF55"
+
+MIDNIGHT = Theme(
+ name="midnight", primary=CYAN, secondary=BLUE, accent=YELLOW, warning=YELLOW, error="#FF5555",
+ success="#55FF55", foreground=GRAY, background=BLUE, surface=BLUE, panel=CYAN, dark=True,
+ variables={
+ "border": GRAY, "border-blurred": GRAY,
+ "block-cursor-foreground": BLACK, "block-cursor-background": CYAN, "block-cursor-text-style": "none",
+ "block-cursor-blurred-foreground": BLACK, "block-cursor-blurred-background": CYAN,
+ "block-cursor-blurred-text-style": "none", "block-hover-background": BLUE,
+ "footer-background": BLACK, "footer-foreground": WHITE, "footer-key-foreground": WHITE,
+ "footer-key-background": BLACK, "footer-description-foreground": BLACK,
+ "footer-description-background": CYAN, "footer-item-background": BLACK,
+ "input-cursor-background": BLACK, "input-cursor-foreground": CYAN,
+ "scrollbar": CYAN, "scrollbar-background": BLUE, "scrollbar-corner-color": BLUE,
+ },
+)
+
+STATE_STYLE = {"newer": "bold #55FF55", "current": "#55FF55", "unpublished": "#FFFF55", "older": "#FFFF55",
+ "error": "bold #FF5555", "no build": GRAY}
+ITEM_STYLE = {"new": ("+", "#55FF55"), "replace": ("~", "#FFFF55"), "meta": ("~", "#55FFFF"),
+ "notes": ("~", "#55FFFF"), "delete": ("-", "bold #FF5555"), "same": ("=", GRAY),
+ "skip": ("!", "#FFFF55")}
+
+
+def line_text(line: str) -> Text:
+ style = ""
+ if line.startswith("== "):
+ style = "bold #55FFFF"
+ elif line.startswith(("ERROR", "error:")) or "failed" in line.lower():
+ style = "bold #FF5555"
+ elif line.startswith("WARN"):
+ style = "#FFFF55"
+ return Text(line, style=style)
+
+
+def item_line(i: pipeline.Item, app: config.App) -> tuple[str, str]:
+ sign, style = ITEM_STYLE[i.action]
+ return f"{sign} {i.describe(app).replace(' -> ', ' → ', 1)}", style
+
+
+class ConfirmPlan(ModalScreen[bool]):
+ BINDINGS = [Binding("escape,n", "dismiss(False)", "Cancel"), Binding("y", "dismiss(True)", "Publish")]
+
+ def __init__(self, plan: Plan):
+ super().__init__()
+ self.plan = plan
+
+ def compose(self) -> ComposeResult:
+ p = self.plan
+ body = Text()
+ body.append(f"{p.app.src}\n→ {p.server.target}:{p.server.apps_dir}\n", style="bold")
+ if p.git_rev:
+ body.append(f"git {p.git_rev}\n")
+ body.append("\n")
+ for i in p.items:
+ text, style = item_line(i, p.app)
+ body.append(text + "\n", style=style)
+ if p.opts.notes.strip():
+ body.append("\nWhat's new:\n", style="bold")
+ body.append(p.opts.notes.strip() + "\n")
+ for n in p.notes:
+ body.append(f"! {n}\n", style="bold #FFFF55")
+ with Vertical(id="dialog"):
+ yield Label(Text(f" Publish {p.app.name} "), id="dialog-title")
+ with VerticalScroll():
+ yield Static(body)
+ with Horizontal(id="buttons"):
+ yield Button("Publish (y)", id="yes", variant="success")
+ yield Button("Cancel (n)", id="no")
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ self.dismiss(e.button.id == "yes")
+
+
+class Ask(ModalScreen[bool]):
+ BINDINGS = [Binding("escape,n", "dismiss(False)", "No"), Binding("y", "dismiss(True)", "Yes")]
+
+ def __init__(self, title: str, text: str):
+ super().__init__()
+ self.title_, self.text = title, text
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog", classes="small"):
+ yield Label(Text(f" {self.title_} "), id="dialog-title")
+ with VerticalScroll():
+ yield Static(self.text)
+ with Horizontal(id="buttons"):
+ yield Button("Yes (y)", id="yes", variant="warning")
+ yield Button("No (n)", id="no")
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ self.dismiss(e.button.id == "yes")
+
+
+class PickVersion(ModalScreen[str | None]):
+ BINDINGS = [Binding("escape", "dismiss(None)", "Cancel")]
+
+ def __init__(self, app_name: str, pub: list[Published]):
+ super().__init__()
+ self.app_name = app_name
+ # one line per version: all its packages are removed together
+ self.versions = list(dict.fromkeys(p.version for p in pub))
+ self.lines = [f"v{v:12} " + ", ".join(f"{p.kind.label} {p.size / 1048576:.1f} MB"
+ for p in pub if p.version == v) for v in self.versions]
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog", classes="small"):
+ yield Label(Text(f" {self.app_name}: published versions (Enter removes) "), id="dialog-title")
+ yield OptionList(*self.lines)
+
+ @on(OptionList.OptionSelected)
+ def picked(self, e: OptionList.OptionSelected) -> None:
+ self.dismiss(self.versions[e.option_index])
+
+
+class AddApp(ModalScreen[config.App | None]):
+ BINDINGS = [Binding("escape", "dismiss(None)", "Cancel")]
+
+ def compose(self) -> ComposeResult:
+ with Vertical(id="dialog"):
+ yield Label(" New app ", id="dialog-title")
+ yield Label("Local project dir:")
+ yield Input(placeholder="~/Dev/Projects/android/MyApp", id="path")
+ yield Label("Name on the site (empty = detect):")
+ yield Input(id="name")
+ yield Static("", id="toml")
+ with Horizontal(id="buttons"):
+ yield Button("Detect", id="detect")
+ yield Button("Add", id="add", variant="success")
+ yield Button("Cancel", id="cancel")
+
+ def build(self) -> config.App | None:
+ raw = self.query_one("#path", Input).value.strip()
+ path = Path(os.path.expanduser(raw)) if raw else None
+ if not path or not path.is_dir():
+ self.query_one("#toml", Static).update(Text("dir not found", style="bold #FF5555"))
+ return None
+ app = detect.new_app(path, self.query_one("#name", Input).value.strip() or None)
+ self.query_one("#toml", Static).update(Text(config.app_to_toml(app)))
+ return app
+
+ @on(Button.Pressed)
+ def pressed(self, e: Button.Pressed) -> None:
+ if e.button.id == "cancel":
+ self.dismiss(None)
+ elif e.button.id == "detect":
+ self.build()
+ else:
+ app = self.build()
+ if app:
+ try:
+ config.validate_app(app)
+ except config.ConfigError as err:
+ self.query_one("#toml", Static).update(Text(str(err), style="bold #FF5555"))
+ return
+ self.dismiss(app)
+
+
+class PublisherApp(App):
+ TITLE = f"Publisher {__version__}"
+ CSS = """
+ Screen { background: $background; }
+ #top { height: 1fr; }
+ #apps { width: 3fr; border: double $border; border-title-color: $accent; }
+ #apps > .datatable--header { background: $panel; color: #000000; text-style: bold; }
+ #apps > .datatable--header-hover { background: $panel; color: #000000; }
+ #apps > .datatable--header-cursor { background: $panel; color: #000000; }
+ #side { width: 2fr; border: double $border; border-title-color: $accent; padding: 0 1; }
+ #info { height: auto; margin-bottom: 1; }
+ #side Checkbox { background: $background; border: none; padding: 0; height: 1; }
+ #side Label { margin-top: 1; }
+ #notes { height: 6; background: $background; border: round $border; }
+ #log { height: 14; border: double $border; border-title-color: $accent; background: $background; }
+ ModalScreen { align: center middle; background: $background 60%; }
+ #dialog { width: 100; max-width: 95%; height: 80%; background: $panel; color: #000000;
+ border: double #FFFFFF; padding: 0 1; }
+ #dialog.small { height: auto; max-height: 80%; }
+ #dialog VerticalScroll { height: 1fr; background: $background; color: $foreground; }
+ #dialog.small VerticalScroll { height: auto; max-height: 20; }
+ #dialog-title { width: 100%; content-align: center middle; color: #0000AA; text-style: bold; }
+ #dialog Input { margin-bottom: 1; }
+ #dialog OptionList { height: auto; max-height: 20; }
+ #toml { height: auto; background: $background; color: $foreground; }
+ #buttons { height: 3; align: center middle; }
+ #buttons Button { margin: 0 2; }
+ Toast { background: #000000; color: #FFFFFF; }
+ """
+ BINDINGS = [
+ Binding("f2", "edit", "Edit"),
+ Binding("f3", "preview", "Preview"),
+ Binding("f5", "publish", "Publish"),
+ Binding("f6", "refresh", "Refresh"),
+ Binding("f7", "add", "Add"),
+ Binding("f8", "versions", "Versions"),
+ Binding("f10", "quit", "Quit"),
+ ]
+
+ def __init__(self, cfg: Config):
+ super().__init__()
+ self.cfg = cfg
+ self.status: dict[str, pipeline.Status] = {}
+ self.busy = False
+
+ # ---------- layout ----------
+ def compose(self) -> ComposeResult:
+ with Horizontal(id="top"):
+ yield DataTable(id="apps", cursor_type="row", zebra_stripes=False)
+ with VerticalScroll(id="side"):
+ yield Static(id="info")
+ yield Checkbox("Build locally", True, id="o_build")
+ yield Checkbox("Upload description / icon", True, id="o_meta")
+ yield Checkbox("Delete versions beyond keep", True, id="o_prune")
+ yield Checkbox("Force: older build, overwrite", False, id="o_force")
+ yield Label("What's new in this version:")
+ yield TextArea(id="notes", soft_wrap=True)
+ yield RichLog(id="log", wrap=True, markup=False, max_lines=5000)
+ yield Footer()
+
+ def on_mount(self) -> None:
+ self.register_theme(MIDNIGHT)
+ self.theme = "midnight"
+ t = self.query_one("#apps", DataTable)
+ t.border_title = f"Apps → {self.cfg.server.target}"
+ t.add_columns("Name", "State", "Local", "Site")
+ self.query_one("#side").border_title = "Details / options"
+ self.query_one("#log").border_title = "Output"
+ self.fill_table()
+ self.action_refresh()
+
+ def fill_table(self) -> None:
+ t = self.query_one("#apps", DataTable)
+ row = t.cursor_row
+ t.clear()
+ for a in self.cfg.apps:
+ s = self.status.get(a.name, pipeline.Status("…"))
+ t.add_row(a.name, Text(s.state, style=STATE_STYLE.get(s.state, "")), s.local, s.remote, key=a.name)
+ if self.cfg.apps:
+ t.move_cursor(row=min(row, len(self.cfg.apps) - 1))
+ self.show_details()
+
+ def current(self) -> config.App | None:
+ t = self.query_one("#apps", DataTable)
+ if not self.cfg.apps or t.row_count == 0:
+ return None
+ key = t.coordinate_to_cell_key(Coordinate(t.cursor_row, 0)).row_key.value
+ return self.cfg.app(key)
+
+ @on(DataTable.RowHighlighted)
+ def show_details(self) -> None:
+ a = self.current()
+ info = self.query_one("#info", Static)
+ if not a:
+ info.update("No apps: F7 add one")
+ return
+ lines = [f"[b]{escape(a.name)}[/b]", f"src: {escape(a.src)}"]
+ lines += [f" {escape(g)}" for g in a.packages]
+ if a.version_file:
+ lines.append(f"version: {escape(a.version_file)}")
+ if a.build:
+ lines.append(f"build: {escape(a.build)}")
+ meta = ", ".join(x for x in (a.info, a.icon) if x)
+ if meta:
+ lines.append(f"meta: {escape(meta)}")
+ lines.append(f"keep: {self.cfg.keep(a) or 'all'}")
+ info.update("\n".join(lines))
+ self.query_one("#o_build", Checkbox).disabled = not a.build
+
+ def options(self) -> Options:
+ v = lambda i: self.query_one(f"#{i}", Checkbox).value # noqa: E731
+ return Options(build=v("o_build"), meta=v("o_meta"), prune=v("o_prune"), force=v("o_force"),
+ notes=self.query_one("#notes", TextArea).text)
+
+ # ---------- output ----------
+ def log_line(self, line: str) -> None:
+ self.query_one("#log", RichLog).write(line_text(line))
+
+ def tlog(self, line: str) -> None:
+ """Log from a worker thread."""
+ self.call_from_thread(self.log_line, line)
+
+ def start(self) -> bool:
+ if self.busy:
+ self.notify("Another operation is running", severity="warning")
+ return False
+ self.busy = True
+ return True
+
+ def done(self) -> None:
+ self.busy = False
+
+ # ---------- actions ----------
+ def action_refresh(self) -> None:
+ self.refresh_status()
+
+ @work(thread=True, group="status", exclusive=True)
+ def refresh_status(self) -> None:
+ try:
+ st, others = pipeline.status_all(self.cfg)
+ except remote.RemoteError as e:
+ self.tlog(f"ERROR: status: {e}")
+ return
+ if others:
+ self.tlog(f"published, not in {self.cfg.path.name}: {', '.join(others)}")
+ self.call_from_thread(self._set_status, st)
+
+ def _set_status(self, st) -> None:
+ self.status = st
+ self.fill_table()
+
+ def action_preview(self) -> None:
+ a = self.current()
+ if a and self.start():
+ self.do_preview(a, self.options(), then_publish=False)
+
+ def action_publish(self) -> None:
+ a = self.current()
+ if a and self.start():
+ self.do_preview(a, self.options(), then_publish=True)
+
+ @work(thread=True, group="op")
+ def do_preview(self, a: config.App, opts: Options, then_publish: bool) -> None:
+ try:
+ self.tlog(f"== preview {a.name}")
+ plan = pipeline.preview(self.cfg, a, opts, log=self.tlog)
+ except (pipeline.PublishError, remote.RemoteError, OSError) as e:
+ self.tlog(f"ERROR: {e}")
+ self.call_from_thread(self.done)
+ return
+ for i in plan.items:
+ self.tlog(" " + item_line(i, a)[0])
+ for n in plan.notes:
+ self.tlog(f"WARN: {n}")
+ if not then_publish or not plan.changes:
+ if then_publish:
+ self.tlog("nothing to publish")
+ self.call_from_thread(self.done)
+ return
+
+ def answered(ok: bool | None) -> None:
+ if ok:
+ self.do_execute(plan)
+ else:
+ self.log_line("cancelled")
+ self.done()
+
+ self.call_from_thread(self.push_screen, ConfirmPlan(plan), answered)
+
+ @work(thread=True, group="op")
+ def do_execute(self, plan: Plan) -> None:
+ res = pipeline.execute(plan, log=self.tlog)
+ self.call_from_thread(self.after_execute, plan, res)
+
+ def after_execute(self, plan: Plan, res: pipeline.Result) -> None:
+ self.done()
+ self.notify(f"{plan.app.name}: {res.message}", severity="information" if res.ok else "error", timeout=8)
+ if res.ok:
+ self.query_one("#notes", TextArea).text = ""
+ self.action_refresh()
+
+ def action_versions(self) -> None:
+ a = self.current()
+ if a and self.start():
+ self.load_versions(a)
+
+ @work(thread=True, group="op")
+ def load_versions(self, a: config.App) -> None:
+ try:
+ pub = pipeline.versions(self.cfg, a)
+ except remote.RemoteError as e:
+ pub = []
+ self.tlog(f"ERROR: {e}")
+ self.call_from_thread(self._pick_version, a, pub)
+
+ def _pick_version(self, a: config.App, pub: list[Published]) -> None:
+ if not pub:
+ self.done()
+ self.notify(f"{a.name}: nothing published", severity="warning")
+ return
+
+ def picked(version: str | None) -> None:
+ if not version:
+ self.done()
+ return
+
+ def confirmed(ok: bool | None) -> None:
+ if ok:
+ self.do_remove(a, version)
+ else:
+ self.done()
+ self.push_screen(Ask("Remove", f"Delete {a.name} v{version} (all its packages and notes) "
+ "from the site?"), confirmed)
+
+ self.push_screen(PickVersion(a.name, pub), picked)
+
+ @work(thread=True, group="op")
+ def do_remove(self, a: config.App, version: str) -> None:
+ res = pipeline.remove(self.cfg, a, version, log=self.tlog)
+ self.tlog(f"== {res.message}")
+ self.call_from_thread(self._removed, a, res)
+
+ def _removed(self, a: config.App, res: pipeline.Result) -> None:
+ self.done()
+ self.notify(f"{a.name}: {res.message}", severity="information" if res.ok else "error", timeout=8)
+ self.action_refresh()
+
+ def action_edit(self) -> None:
+ if self.busy:
+ return
+ editor = os.environ.get("VISUAL") or os.environ.get("EDITOR") or "nano"
+ with self.suspend():
+ subprocess.call([*shlex.split(editor), str(self.cfg.path)])
+ self.reload()
+
+ def reload(self) -> None:
+ try:
+ self.cfg = config.load(self.cfg.path)
+ except (OSError, config.ConfigError) as e:
+ self.log_line(f"ERROR: config: {e} (old config kept)")
+ return
+ self.log_line(f"== config reloaded: {len(self.cfg.apps)} apps")
+ self.fill_table()
+ self.action_refresh()
+
+ def action_add(self) -> None:
+ def added(app: config.App | None) -> None:
+ if app:
+ if any(x.name.lower() == app.name.lower() for x in self.cfg.apps):
+ self.notify(f"{app.name} already exists", severity="error")
+ return
+ config.append_apps(self.cfg.path, [app])
+ self.log_line(f"== added {app.name}: review it with F2, then F3 preview")
+ self.reload()
+ self.push_screen(AddApp(), added)
+
+
+def run(cfg: Config) -> int:
+ PublisherApp(cfg).run()
+ return 0
diff --git a/srcpkgs/publisher/files/share/publish.toml.example b/srcpkgs/publisher/files/share/publish.toml.example
new file mode 100644
index 0000000..c0070e6
--- /dev/null
+++ b/srcpkgs/publisher/files/share/publish.toml.example
@@ -0,0 +1,20 @@
+# Publisher registry. Edit by hand or append with `publish add `.
+#
+# Packages land in apps_dir as _v (the format of apps.jsp):
+# .apk, .AppImage, .deb, .rpm, .exe, .msi, .dmg, .tar.gz, or no extension for an executable Linux binary.
+# packages: globs relative to src; of all matches the best build of each package type is published
+# (the highest version, then the newest file), so one type may come from several places
+# version_file: version for builds without _v in the file name (and a "rebuild?" check)
+# version_regex: its (group); default finds versionName = "..", Version = "..", version = ".."
+# info: local file uploaded as .txt (first line: title, the rest: description)
+# icon: local .png / .webp uploaded as .png / .webp
+# Common keys: build, build_env, keep (versions of each package type; server default)
+
+[server]
+host = "server.example.com"
+user = "user"
+apps_dir = "/home/user/app/docker/tomcat/data/webapps/ttt/WEB-INF/apps"
+check_url = "http://localhost:8010/download.jsp?f="
+state_dir = "/home/user/app_deploy"
+keep = 3
+local_roots = ["~/Projects"]
diff --git a/srcpkgs/publisher/files/tests/test_core.py b/srcpkgs/publisher/files/tests/test_core.py
new file mode 100644
index 0000000..c5d8b7f
--- /dev/null
+++ b/srcpkgs/publisher/files/tests/test_core.py
@@ -0,0 +1,217 @@
+import os
+from pathlib import Path
+
+import pytest
+
+from publisher import config, detect, packages, pipeline
+from publisher.packages import BINARY, KINDS, Published
+
+APK, APPIMAGE, TARGZ = KINDS[0], KINDS[1], KINDS[-1]
+
+SERVER = """
+[server]
+host = "h"
+apps_dir = "/srv/www/apps"
+"""
+
+
+def write(tmp_path: Path, body: str) -> Path:
+ p = tmp_path / "publish.toml"
+ p.write_text(SERVER + body)
+ return p
+
+
+def touch(path: Path, data: bytes = b"x", exe: bool = False) -> Path:
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_bytes(data)
+ if exe:
+ path.chmod(0o755)
+ return path
+
+
+def test_load(tmp_path):
+ cfg = config.load(write(tmp_path, """
+[[app]]
+name = "Foo"
+src = "~/x"
+packages = ["app/release/*.apk"]
+keep = 1
+"""))
+ a = cfg.app("foo")
+ assert a.src_path == Path.home() / "x"
+ assert cfg.keep(a) == 1
+ assert a.local("/abs/f") == Path("/abs/f")
+
+
+@pytest.mark.parametrize("body, msg", [
+ ('name = "Foo_v1"\nsrc = "x"\npackages = ["a"]', "bad name"),
+ ('name = "Foo 2"\nsrc = "x"\npackages = ["a"]', "bad name"),
+ ('name = "Foo"\nsrc = "x"', "packages"),
+ ('name = "Foo"\nsrc = "x"\npackages = ["../a"]', "glob"),
+ ('name = "Foo"\nsrc = "x"\npackages = ["a"]\nicon = "i.jpg"', "icon"),
+ ('name = "Foo"\nsrc = "x"\npackages = ["a"]\nversion_regex = "v.*"', "group"),
+ ('name = "Foo"\nsrc = "x"\npackages = ["a"]\ntypo = 1', "unknown key"),
+ ('name = "Foo"\nsrc = "x"\npackages = "a"', "wrong type"),
+])
+def test_validation(tmp_path, body, msg):
+ with pytest.raises(config.ConfigError, match=msg):
+ config.load(write(tmp_path, "[[app]]\n" + body))
+
+
+def test_unsafe_apps_dir(tmp_path):
+ p = tmp_path / "publish.toml"
+ p.write_text('[server]\napps_dir = "/srv"\n')
+ with pytest.raises(config.ConfigError, match="apps_dir"):
+ config.load(p)
+
+
+def test_append_round_trip(tmp_path):
+ p = write(tmp_path, "# keep me\n")
+ app = config.App(name="n", src="~/p", packages=["a/*.apk"], build_env={"JAVA_HOME": "~/jdk"}, build='echo "hi"')
+ config.append_apps(p, [app])
+ assert "# keep me" in p.read_text()
+ loaded = config.load(p).app("n")
+ assert loaded.build == 'echo "hi"' and loaded.build_env == {"JAVA_HOME": "~/jdk"}
+
+
+@pytest.mark.parametrize("name, version", [
+ ("Foo_v1.2.0.apk", "1.2.0"), ("Foo_v1.2.0-release.apk", "1.2.0"), ("Foo-1.5-release.apk", "1.5"),
+ ("ExtMess_v0.7.0_linux_x86_64.AppImage", "0.7.0"), ("ExtMess_v0.7.0_linux_amd64.deb", "0.7.0"),
+ ("Stylometer_v0.9.0_linux.tar.gz", "0.9.0"),
+ ("app-release.apk", ""), ("multitext", ""),
+])
+def test_file_version(name, version):
+ assert packages.file_version(Path(name)) == version
+
+
+@pytest.mark.parametrize("file, exe, app, version, kind", [
+ ("DasWeather_v1.4.0.apk", False, "DasWeather", "1.4.0", APK),
+ ("WhereAreYou_v1.1.0.AppImage", False, "WhereAreYou", "1.1.0", APPIMAGE),
+ ("multitext_v0.6", True, "multitext", "0.6", BINARY),
+ ("Foo_v1.0-release.apk", False, "Foo", "1.0-release", APK),
+ ("Stilometer_Linux_v0.9.0.tar.gz", False, "Stilometer_Linux", "0.9.0", TARGZ),
+])
+def test_parse_published(file, exe, app, version, kind):
+ p = packages.parse_published(file, exe)
+ assert (p.app, p.version, p.kind) == (app, version, kind)
+
+
+@pytest.mark.parametrize("file, exe", [("README.txt", True), ("Foo.png", False), ("multitext_v0.6", False),
+ (".Foo_v1.apk.Xy12", False), ("run.sh", True)])
+def test_not_packages(file, exe):
+ assert packages.parse_published(file, exe) is None
+
+
+def test_version_key():
+ assert packages.version_key("1.10.0") > packages.version_key("1.9.3")
+ assert packages.version_key("1.0") == packages.version_key("1.0.0")
+ assert packages.version_key("1.2.0-release") == packages.version_key("1.2.0")
+
+
+def test_resolve(tmp_path):
+ touch(tmp_path / "app/release/Foo_v1.9.0.apk")
+ touch(tmp_path / "app/release/Foo_v1.10.0-release.apk")
+ touch(tmp_path / "build/foo", exe=True)
+ touch(tmp_path / "app/build.gradle.kts", b'android {\n versionName = "1.11.0"\n}\n')
+ app = config.App(name="Foo", src=str(tmp_path), packages=["app/release/*.apk", "build/foo", "dist/*.deb"],
+ version_file="app/build.gradle.kts")
+ builds, notes = packages.resolve(app)
+ assert [(b.site_name("Foo"), b.version) for b in builds] == [("Foo_v1.10.0.apk", "1.10.0"),
+ ("Foo_v1.11.0", "1.11.0")]
+ assert any("rebuild?" in n for n in notes) and any("dist/*.deb" in n for n in notes)
+
+
+def test_resolve_best_of_several_places(tmp_path):
+ touch(tmp_path / "app/release/Foo_v1.0.apk", b"old")
+ touch(tmp_path / "app/build/outputs/apk/release/Foo_v1.1.apk", b"new")
+ app = config.App(name="Foo", src=str(tmp_path),
+ packages=["app/release/*.apk", "app/build/outputs/apk/release/*.apk"])
+ builds, notes = packages.resolve(app)
+ assert [b.path.name for b in builds] == ["Foo_v1.1.apk"] and notes == []
+
+
+def test_resolve_same_version_newest_wins(tmp_path):
+ a = touch(tmp_path / "app/release/Foo_v1.0.apk", b"studio")
+ b = touch(tmp_path / "app/build/outputs/apk/release/Foo_v1.0.apk", b"gradle!")
+ os.utime(a, (1000, 1000))
+ app = config.App(name="Foo", src=str(tmp_path), packages=["app/release/*.apk", "app/build/outputs/apk/release/*.apk"])
+ builds, notes = packages.resolve(app)
+ assert builds[0].path == b
+ assert any("2 different builds" in n and "app/build/outputs" in n for n in notes)
+
+
+def test_resolve_needs_a_version(tmp_path):
+ touch(tmp_path / "build/foo", exe=True)
+ with pytest.raises(packages.PackageError, match="no version"):
+ packages.resolve(config.App(name="Foo", src=str(tmp_path), packages=["build/foo"]))
+
+
+def test_prune():
+ pub = [Published(f"F_v{v}.apk", "F", v, APK) for v in ("1.0", "1.10", "1.9", "1.2")]
+ pub.append(Published("F_v1.0.AppImage", "F", "1.0", APPIMAGE))
+ assert sorted(p.file for p in pipeline.prune(pub, 2)) == ["F_v1.0.apk", "F_v1.2.apk"]
+ assert pipeline.prune(pub, 0) == []
+
+
+def fake_state(monkeypatch, files: dict[str, tuple[int, bool]], hashes: dict[str, str]):
+ def state(cfg, hash_names=()):
+ st = pipeline.RemoteState({n: (s, 1.0, ex) for n, (s, ex) in files.items()})
+ st.hashes = {n: h for n, h in hashes.items() if n in hash_names}
+ return st
+ monkeypatch.setattr(pipeline, "remote_state", state)
+ monkeypatch.setattr(pipeline, "git_rev", lambda src: "")
+
+
+def test_preview(tmp_path, monkeypatch):
+ apk = touch(tmp_path / "app/release/Foo_v1.3.0.apk", b"new")
+ img = touch(tmp_path / "dist/Foo_v1.3.0_linux_x86_64.AppImage", b"img")
+ info = touch(tmp_path / "site.txt", b"Foo\nabout")
+ fake_state(monkeypatch, {
+ "Foo_v1.0.0.apk": (1, False), "Foo_v1.1.0.apk": (1, False), "Foo_v1.2.0.apk": (1, False),
+ "Foo_v1.0.0.txt": (1, False), "Foo_v1.3.0.AppImage": (1, True), "Foo.txt": (1, False),
+ "Other_v9.apk": (1, False),
+ }, {"Foo_v1.3.0.AppImage": pipeline.sha256(img), "Foo.txt": pipeline.sha256(info)})
+ cfg = config.load(write(tmp_path, f'[[app]]\nname = "Foo"\nsrc = "{tmp_path}"\n'
+ 'packages = ["app/release/*.apk", "dist/*.AppImage"]\ninfo = "site.txt"\n'))
+ plan = pipeline.preview(cfg, cfg.app("Foo"), pipeline.Options(notes="fixes"), log=lambda s: None)
+ got = {(i.action, i.target) for i in plan.items}
+ assert got == {("new", "Foo_v1.3.0.apk"), ("same", "Foo_v1.3.0.AppImage"), ("notes", "Foo_v1.3.0.txt"),
+ ("delete", "Foo_v1.0.0.apk"), ("delete", "Foo_v1.0.0.txt")}
+ assert next(i for i in plan.items if i.target == "Foo_v1.3.0.apk").local == apk
+
+
+def test_preview_older_and_changed(tmp_path, monkeypatch):
+ touch(tmp_path / "Foo_v1.0.apk", b"old")
+ touch(tmp_path / "Foo_v2.0.deb", b"changed")
+ fake_state(monkeypatch, {"Foo_v1.1.apk": (1, False), "Foo_v2.0.deb": (1, False)}, {"Foo_v2.0.deb": "0" * 64})
+ cfg = config.load(write(tmp_path, f'[[app]]\nname = "Foo"\nsrc = "{tmp_path}"\npackages = ["*.apk", "*.deb"]\n'))
+ plan = pipeline.preview(cfg, cfg.app("Foo"), pipeline.Options(), log=lambda s: None)
+ assert [i.action for i in plan.items] == ["skip", "skip"] and not plan.changes
+ plan = pipeline.preview(cfg, cfg.app("Foo"), pipeline.Options(force=True, prune=False), log=lambda s: None)
+ assert [i.action for i in plan.items] == ["new", "replace"]
+
+
+def test_detect_android(tmp_path):
+ touch(tmp_path / "settings.gradle.kts", b'rootProject.name = "DasPressureAndMedicines"\n')
+ touch(tmp_path / "app/build.gradle.kts", b'versionName = "3.13.0"\n')
+ touch(tmp_path / "app/release/DasPressureMedicines_v3.13.0.apk")
+ touch(tmp_path / "desktop/dist/DasPressureMedicines_v3.13.0_linux_x86_64.AppImage", exe=True)
+ app = detect.new_app(tmp_path)
+ assert app.name == "DasPressureMedicines"
+ assert app.packages == ["app/release/*.apk", "app/build/outputs/apk/release/*.apk", "desktop/dist/*.AppImage"]
+ assert app.version_file == "app/build.gradle.kts"
+ config.validate_app(app)
+
+
+def test_detect_go(tmp_path):
+ touch(tmp_path / "internal/version/version.go", b'package version\n\nconst (\n\tVersion = "0.10.0"\n)\n')
+ touch(tmp_path / "build/multitext", exe=True)
+ app = detect.new_app(tmp_path, "multitext")
+ assert (app.packages, app.version_file) == (["build/multitext"], "internal/version/version.go")
+ builds, _ = packages.resolve(app)
+ assert builds[0].site_name("multitext") == "multitext_v0.10.0"
+
+
+def test_site_name_cleanup():
+ assert detect._site_name("Das Pressure 2") == "DasPressure2"
+ assert os.path.basename(detect._home(Path.home() / "a")) == "a"
diff --git a/srcpkgs/publisher/template b/srcpkgs/publisher/template
new file mode 100644
index 0000000..38a5bf3
--- /dev/null
+++ b/srcpkgs/publisher/template
@@ -0,0 +1,26 @@
+# Template file for 'publisher'
+# In-tree package: sources are in files/ (FILESDIR), a copy of ~/Dev/Projects/Python/publisher.
+# Differences from the upstream: the config is in ~/.config/publisher (see bin/publish) and
+# history.log is in ~/.local/state/publisher.
+pkgname=publisher
+version=0.3.0
+revision=1
+create_wrksrc=yes
+hostmakedepends="python3"
+depends="python3 python3-textual openssh rsync"
+short_desc="TUI/CLI publishing of apk, AppImage, archives and binaries over ssh + rsync"
+maintainer="Das "
+license="MIT"
+homepage="https://git.das.org.ua/das/Das-OS"
+pycompile_dirs="usr/lib/das/publisher"
+python_version=3
+
+do_install() {
+ local f
+ for f in "${FILESDIR}"/python/publisher/*.py; do
+ vinstall "${f}" 644 usr/lib/das/publisher/publisher
+ done
+ vbin "${FILESDIR}/bin/publish"
+ vinstall "${FILESDIR}/share/publish.toml.example" 644 usr/share/das/publisher
+ vlicense "${FILESDIR}/LICENSE"
+}