Solo – a .so loader for static Linux binaries

This is a big forwards-compatibility risk. Suppose glibc adds a new symbol, and then a GPU driver adds a dependency on that symbol. The user wants to run an old executable with the updated GPU driver (maybe the old GPU driver doesn’t support their GPU). Normally, this would work fine: the user has to use a new copy of glibc, which will be compatible with both the new GPU driver and the old executable. But with your approach, the GPU driver is forced to use the glibc reimplementation which has been statically linked into the executable. Which, since the executable is old, can’t possibly implement the new symbol.

The same issue would occur if glibc adds a new version of an existing symbol and then the GPU driver is recompiled. (Or, for that matter, if a GPU driver adds a dependency on a symbol which glibc has always supported but which isn’t in the subset that you reimplemented, though in theory that could be solved if you reimplemented 100% of the symbols.)

So we are re-inventing patched a.out files, back when UNIX systems started to introduce dynamic loading, before ELF was invented?

Advocates of static linking keep forgetting once upon a time UNIX only had static linking, then we had overlays, and eventually dynamic linking came to be.

I don't know much about musl.

> GPU: Vulkan and OpenGL drivers are supplied by the host as shared objects, usually built against glibc, and a fully static musl binary cannot normally dlopen() them.

Why? Have people managed to break the ancient concept of shared libraries, and this is a fix for that?

Every couple of years, I revisit my PL dev hobby and this time I decided to create a language/runtime with pre-emptive scheduling using instruction fuel. While I always do freestanding builds, this time I decided that I also wanted to support native FFI.

That is when I realized the true horror of (g)libc. It wants to inject itself at the root of the library/program and everything from threading to dlopen/dlsym is impacted. I tried a lot of workarounds including trying to implement a loader myself, but the complexity (and fragility) grew so much that I felt it was not worth it.

Finally, I retreated into the safe world of a freestanding runtime + syscalls. FFI, if it has to happen, will occur via IPC of some kind. A second process linked against glibc that will manage calls on behalf of the clean first one.

If you can figure out your own ELF loader, you can figure out how to build a partially static executable that doesn't need this. You can mix static and dynamic linking. Build tooling around that is just shit.

How this differs (is better!) from prior art - https://github.com/pg83/solo#how-this-differs-from-prior-wor...

I've implemented the same thing for micron (more or less). One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE). That being said the way you're doing is also tricky(ish) because if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you. Doing this is safer if you control the entire runtime.

I have faced a similar issue in the past, and I don't understand how static binaries from the host are supposed to solve this.

From what I remember, GPU access on Linux 'works' by accessing specific FDs under /dev, which are vendor specific - this is what these libs do under the hood.

The libraries don't have any magic powers - if the FD is inaccessible, you won't be able to do anything.

So there's some vendor specific access needed in containers anyway (or a blanket allow, which is a BAD idea).

Also not sure why dynamic linking isn't good enough for this - the issue lies with the permissions, not how you load/link libraries.

How are we supposed to take this stuff seriously if the author (sic) isn't even willing to write the readme? Claude exists! If I want some slop I can push the button myself.

If you dynamically sold an SO are you still static even if you did it "custom" ? At that point it's a dynamic loader in another name?

Do people say "so", "ess-oh" or "dot-ess-oh"? The title "a .so" is clunky to the "ess-oh" gang.

i'm not sure how much people realize that the modern "graphics driver" is actually just "the kernel multiplexes userspace messages to/from the GPU and we've taught mesa to understand each family of GPU you'd ever care to support."

i helped somebody get Doom running on an old embedded system running some 4.x kernel. we just built everything, including mesa, statically and deployed that. imagine building everything static but loading the system's libgl.so instead (which was probably just a symlink or abstraction over mesa's own implementation). what's the benefit: we'd get older, less optimized graphics routines from 5 years ago?

if you're statically linking, then just bring your own graphics "driver". the kernel interfaces are stable enough. it's not conceptually different than embedding `syscall`s directly into your application the same way you do when statically linking libc.

if you really want a single elf that could link to system libraries with a foreign libc (you shouldn't), the somehow correct-ish way to do it is map your preferred ldso manually , setup stack, call DT_ENTRY etc etc like how kernel does it, and just yank the libdl symbols.

see cosmo_dlopen for an example implementation.

You normally do this sort of thing with plan old `dlopen` and `dlsym` and make your own "plugin" system.

It is a testament to the complete failure of the GNU/Linux userland that something like this seems at all attractive to spend time on (or, it seems, LLM tokens).

Actually, scratch that, because Windows and macOS have historically struggled with ABI compatibility as well (macOS less so, due to not caring about backward compatibility in the first place).

How did we get to the point where people feel they need to go to the length of embedding an ELF loader in their binary (!!) rather than just linking with glibc?