Using Filespooler Without Queues to Pass More Metadata

One frustration people sometimes have with ssh or NNCP is that they’d like to pass along a lot of metadata to the receiving end. Both ssh and nncp-exec allow you to pass along command-line parameters, but neither of them permit passing along more than that. What if you have a whole host of data to pass? Maybe a dozen things, some of them optional? It would be very nice if you could pass along the environment.

Filespooler lets you do just that. As mentioned in the Filespooler Reference, any environment variable beginning with FSPL_SET_ that’s set when fspl prepare is called is written into the header and will also be set for the command run by fspl queue-process or fspl stdin-process.

The support added in Filespooler 1.0.8 for not using a sequence file on the sender makes this easier.

Consider this example:

FSPL_SET_FOO=123 fspl prepare -s - | \
  fspl stdin-process bash -- -c set | grep FSPL_SET_FOO

Of course, this can be combined with ssh:

FSPL_SET_FOO=123 fspl prepare -s - | \
  ssh otherhost fspl stdin-process bash -- -c 'set' | \

Of course, you can both set the environment and use stdin. Let’s say you wrote this script and saved it in ~/

#!/usr/bin/env bash

set -euo pipefail

echo "Hex dump of $FSPL_SET_FILE:"

Now you can run this service remotely:

echo hi | \
  FSPL_SET_FILE=hi.txt fspl prepare -i - -s - | \
  ssh otherhost fspl stdin-process ~/
Hex dump of hi.txt:
00000000  68 69 0a                                          |hi.|

This technique also works with nncp-exec; see also Using Filespooler over NNCP.

NNCP is a powerful tool for building Asynchronous Communication networks. It features end-to-end Encryption as well as all sorts of other features; see my NNCP Concepts page for some more ideas.

Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication.