The GPUs in one home, working for the apps you run in the cloud.
An app in the cloud sends SpareMesh a job. SpareMesh finds the home machine with the job installed and the memory free, runs it there, and hands back the result. Files are read where they already are, and no home machine listens to the internet.
Running on its first machine. It reserves GPU memory next to a model server it watches but never touches; apps and the other machines are next.
One job, followed
An app sends a batch job that needs about 12 GB of GPU memory. The biggest card is busy, so it is skipped, and the desktop is chosen. Only the job and its one input file cross the internet.
In the cloud
An app sends one job: an input file, and its own id for the job.
At home · the only part the internet can reach
The coordinator stores the job before it answers, then picks a machine.
The 32 GB card
A language model holds 30 GB all day, serving other apps.
Skipped: 2 GB free, 12 needed. The model is marked never-stop, so SpareMesh will not free its memory.
The 16 GB desktop
Often switched off; on today, with 14 GB free.
Chosen: 12 GB reserved in one step, so two jobs cannot both take it.
The Mac
Apple silicon; its GPU shares memory with everything else.
Skipped: this job has not yet been checked on Apple GPUs.
- memory held by something else
- memory reserved for this job
- free · every bar is drawn to the same scale (32 GB wide)
- dashed: memory shared with the rest of the machine, not counted
- The app uploads the input file and asks for the job with its own id. If the answer is lost and it asks again, it gets the same job back, not a second one.
- The coordinator looks for a machine that has the job installed, checked on its kind of GPU, and 12 GB free.
- It reserves 12 GB on the desktop, and hands the job over the next time that machine's agent asks.
- The agent checks the memory again for itself, fetches the input file over the home network, and runs the job. It renews the reservation every 15 seconds.
- It uploads the result. The coordinator frees the 12 GB, and the app collects the result.
If the desktop is off too, the job waits and says why ("no machine has 12 GB free"). It starts when one does.
If the desktop is shut down mid-job, its reservation runs out, the job runs again elsewhere, and a late answer from the lost run is refused.
Files stay where they are
A job can name a file by its app's id and version, instead of uploading it. SpareMesh reads the file where it already is. A file that only a user's laptop has goes once, straight to the home machine, and never through the cloud.
| Where it is | How the job reads it | Bytes moved |
|---|---|---|
| On the machine that runs the job | In place, from a folder its owner listed | none |
| On a shared disk at home | Over the home network | home network only |
| Only on a user's laptop | The laptop uploads it to the home machine with a ticket that works once | one upload, not via the cloud |
Each file is checked by size and fingerprint before the job runs. A file that changed fails the job instead of being used.
SpareMesh keeps no copy afterwards, so running the job again on a laptop's file uploads it again.
One door, and every request carries a key
Only the coordinator faces the internet, and it refuses any request that is not signed. The machines that do the work stay on the home network.
- Every app and every machine signs each request with its own key. The coordinator keeps only the public halves, so a copy of its database opens nothing.
- A job names work that the machine already has installed. No request can make a machine run a command.
- A file is named by an id and a version, never a path or a link, and a machine opens files only inside folders its owner listed.
- An upload ticket is good for one file, one run of the job, and about ten minutes.
- People reach the dashboard through a sign-in with a second factor.
A new machine joins only after its owner compares a fingerprint and approves it at the coordinator.
What it is for, and how far along it is
SpareMesh is for a handful of machines that one person owns. It runs on its first machine, beside a language model server it watches but never touches; the other machines and the first app are next.
It is
- Reservations of GPU memory, and jobs your machines already have installed, for 3 to 20 machines
- A keeper of model servers: started, stopped, and later put to sleep to lend their memory
- NVIDIA cards on Linux, and Apple silicon
It is not
- A cluster shared by many people or companies
- A broker for renting cloud GPUs
- A way to split one model across machines
- A place to store files
Ray, Nomad, SkyPilot and others were weighed first. None schedules on Apple GPUs, none sees the memory a model server outside it holds, and each brings a control plane bigger than a few machines need.
- Done The coordinator and one agent per machine; reservations of GPU memory
- Done Signed requests, and machines that join only when approved
- Done Jobs that survive restarts and lost answers, and can be cancelled
- Done Files read in place, and one-time upload tickets
- Done Running on the first real machine, beside a model server it never touches
- Done A dashboard behind single sign-on: each GPU's memory over time, and why each job waits
- Next Apps reserving memory instead of stopping a model, then the other machines
- Planned At night, putting the language model to sleep to lend its memory to a waiting job, then waking it
- Planned Jobs on the Mac, once their results match the NVIDIA card's