Processing Multiple Commands in a Single Filespooler Queue

You’ll notice that Filespooler’s fspl queue-process command takes a single command. What if you want to permit the sender to select any of several commands to run?

That’s actually quite easy, because Filespooler has three ways of passing data from a sending system to the receiver:

  1. Command-line parameters
  2. Environment variables
  3. Piped input

The two easiest ways to select a command are options 1 and 2. The Filespooler Reference lays out how to pass parameters and environment variables; here I will provide an example of parameters.

Example of parameters

Let’s say you want a Filespooler service that lets the caller pass data to one of two programs: hd or echo. On the receiver, we may have a script like this:

#!/usr/bin/env bash

set -euo pipefail

COMMAND="$1"

shift

case "$COMMAND" in
  "echo")
     echo "$@"
     ;;
  "sha256sum")
     sha256sum -
     ;;
  *)
     echo "Unsupported command $COMMAND"
     exit 5
     ;;
esac

Save that as proc.sh and mark it executable.

Now on the sender, you could request commands like this:

sender$ fspl prepare -s /tmp/seq -- echo hi there | fspl queue-write -q /tmp/q
sender$ echo Testing SHA | fspl prepare -s /tmp/seq -i - -- sha256sum | fspl queue-write -q /tmp/q

Now, to process:

receiver$ fspl queue-process --allow-job-params -q /tmp/q /tmp/proc.sh
hi there
33a1365a4cb5b26c7fd2636dfa26c471b350ce5d9ec0172441499062517ae252  -

And that was exactly the right answer.

Note that this allowed us to pass the parameters to echo but ignore any additional parameters that may have been passed to sha256sum.

Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing, NNCP, ssh, UUCP, USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler. Filespooler is particularly suited to distributed and Asynchronous Communication.