Soar: A fast, modern package manager for Static Binaries, Portable Formats (AppImage) & More
Code: https://github.com/pkgforge/soar
Soar is like linuxbrew (homebrew) but whose packages are 100% static & relocatable on any Linux Distro.
I have been using your stuff since they were called toolpacks.
https://moonpiedumplings.github.io/playground/ape-experiments/
Welcome to Lemmy, Azathothas. It's nice to see more and more usernames I recognize show up here.
Wow! Thank you for that blog post, it was a great read.
I went through similar experience when experimenting with these tools, wish I can write a similar blog one day. Meanwhile, here's my takeaways:
The project still has a long way to go especially for serving as drop-in-replacement for CC/compilers.
And most programs I tried compiling required too much work (custom patches all over the place), to ever be able to scale up to thousands of packages.
This actually works & we used it quite a bit before running into the limitations. Ours is/was based on ralismark/nix-appimage, used bubblewrap & bundled to AppImages : https://github.com/pkgforge/nix-appimage
I was able to make guix pack relocatable, i.e work outside the store on another system with no guix, but the size was huge (even when compared to Nix bundle). And also guix is quite slow & expensive (even with substitutes), so we had to disable it.
You can still find a few packages from when we used to use guix
bash
$ soar search guix [-] bash#gnuguix.bash:bincache | 5.1.8 | static - GNU Bourne-Again Shell, the de facto standard shell on Linux (1.79 MiB) [-] proot#gnuguix.proot-static:bincache | 5.3.0 | static - User-space implementation of chroot, bind mount, and binfmt_misc (1.15 MiB)
Them requiring fuse2 is quite an old issue, and no longer true for appimages made post 2022.
We have a lot of these properly made AppImages (even steam) that work on both glibc/musl and require no dependency.
Check out: https://github.com/pkgforge-dev/Anylinux-AppImages
There's only one project that provides truly static/relocatable python that work on both glibc/musl: https://github.com/leleliu008/python-distribution
All others are only partially static or meant only for either glibc or musl, not both.
Here, I would also like to introduce Sharun. This is the modern successor of staticx with many features, one of those include ability to pack python/pip packages very easily.
Here are some packages we have made using Sharun
bash
$ soar search pypi [-] anipy-cli#sharun.wrappe.pypi.anipy-cli:pkgcache | 3.5.8 | archive - Watch and Download Anime from the terminal (34.13 MiB) [-] apprise#sharun.wrappe.pypi.apprise:pkgcache | 1.9.3 | archive - Send notifications from the command-line to popular notification services (23.83 MiB) [-] archey#sharun.wrappe.pypi.archey:pkgcache | 4.15.0.0 | archive - A simple system information tool written in Python (24.88 MiB) ...and many more...
There are many more packages now (especially if you configure soar with soar defconfig --external): https://docs.pkgforge.dev/soar/readme/packages#total
But this came at the cost of discontinuing support for Android & Windows
We also had to discontinue upxed versions because we used up too much storage (yes, ironic)
The installation instruction & script have also been improved since, it now shows you what to do & where packages are installed.
We also added this as a source: https://github.com/pkgforge/soarpkgs/issues/191 And have a few packages already:
bash
$ soar search stalix [-] arti#stalix.bin_arti.stable:bincache | 1.4.1 | static - Tor reimplementation in Rust (22.61 MiB) [-] fish#stalix.bin_fish.stable:bincache | 3.7.1 | static - Smart and User-friendly Command Line Shell (3.86 MiB) [-] wget#stalix.bin_wget.stable:bincache | 1.25.0 | static - Tool for retrieving files using HTTP, HTTPS, and FTP (8.94 MiB) [-] wget2#stalix.bin_wget_2.stable:bincache | 2.2.0 | static - The successor of GNU Wget, a file and recursive website downloader (10.50 MiB)
I also worked closely with the author to introduce binary caches & docker image.
Another project you might be interested in is ppkg: https://github.com/leleliu008/ppkg
This one can compile static binaries, even for BSDs. We have this in soarpkgs as well.
bash
$ soar search ppkg [-] alacritty#ppkg.alacritty:bincache | 0.15.1 | static - GPU-accelerated terminal emulator (14.75 MiB) [-] dash#ppkg.dash:bincache | 0.5.12 | static - POSIX compliant shell that aims to be as small as possible (754.03 KiB) ... Total 334...
Similar to your experience, dockerC indeed turned out to be quite underwhelming & we are slowly phasing these out from soarpkgs.
There’s only one project that provides truly static/relocatable python that work on both glibc/musl: https://github.com/leleliu008/python-distribution
There is the python provided by APE/cosmo. They also have two other distributions containing various goodies, pypack1, and pypack2. https://cosmo.zip/pub/cosmos/bin/
But this came at the cost of discontinuing support for Android & Windows
I don't care about android support, but for the competition, and I don't really know about Windows support. Right now, RDP is used to authenticate and managed the machines, but maybe a portable VNC we can quickly spin up, so more than one person can be on the same machine, would be useful.
My original thought was to replace in place, insecure services with secure one's via something like docker containers or nix. But I think many of the machines have too little ram bundled libraries for the services to be viable. I actually tested replacing apache, but it simply wouldn't launch (I think the machine only had 2 GB of ram?).
Does it store a complete dependency graph for each of your statically built (or containerized) applications?
For example, if there's an exploit for libwebp and you need to update all the binaries that link it, can it find which binaries need updating from that information?
Does it store a complete dependency graph for each of your statically built (or containerized) applications?
No. Though some of it can be inferred from the overtly verbose logs
For example, if there’s an exploit for libwebp and you need to update all the binaries that link it, can it find which binaries need updating from that information?
No again.
Currently, most packages are built from git HEAD on alpine:edge or debian-unstable build containers.
So if the fix for this affected libwebp is shipped to the images that the build containers are based on (likely because we use edge/unstable images), then any affected packages would also automatically receive this fix.
To store a complete dependency graph, we will most likely need some custom tooling because our build recipes differ wildly for each package. If you have any ideas, please open a discussion on the repo. Thanks!
Thanks for the reply.
Currently, most packages are built from git HEAD on alpine:edge or debian-unstable build containers. So if the fix for this affected libwebp is shipped to the images that the build containers are based on (likely because we use edge/unstable images), then any affected packages would also automatically receive this fix.
How often do packages get rebuilt? Is it only when there's a new version? The problem in that case would be that a package that is no longer developed (or has very long release cycles) would not receive the fix.
I count 341 dependencies. This massive attack surface might be a problem for people who care about security.
To be clear, this is not unusual in the Rust ecosystem. It's a bit of an awkward situation: Rust's big value proposition is security through memory safety, but that is undermined by its not-especially-rich standard library and Cargo's encouragement of pulling in many dependencies. I hope this will improve in the years to come.