NOTE: this is the old version of plumb provided for running old scripts.

Plumb2 is a program for starting and managing multiple processes and various pipes between them. Plumb2 itself can be controlled from programs running inside it, through pipes; so a program running under Plumb2 can add or kill processes or pipes during runtime.

For futher documentation, see the man page.

Plumb2 comes with an additional tool called plumbnet, for managing multiple TCP connections from a single program without having to do networking, threads, or select(2). See man page for more information.

Plumb2 and plumbnet are written in C99 (they currently depend on GCC to compile). The only other external dependency is libevent.

Get Plumb2

You can get Plumb2 from Subversion:

svn co svn:// plumb2

Debian packages are available from the Debian repository.

Other useful programs

Plumb2 may be useful together with other programs that communicate over their STDIN/STDOUT:


Copyright (C) 2009-2010 Máté Nagy <mnagy AT>

Plumb2 is licensed under the GNU General Public License Version 3 (available in the file COPYING) or any later version.

IRC: IRCNet #awk

Man page for Plumb2

       plumb2 - connect multiple programs with lots of pipes

       plumb2  [-v  key  value]+  [-V key values.. .]+ [-f filename] [-d] "com-
       69 "command1" "command2"

       Plumb2 is a program for starting and  managing  multiple  processes  and
       various  pipes  between  them. Plumb2 itself can be controlled from pro-
       grams running inside it, through pipes;  so  a  program  running  under
       Plumb2 can add or kill processes or pipes during runtime.

       Multiple  pipes may go from a single source to various targets, or from
       various sources to a single target. Line buffering can be used to avoid
       mixing  up data; adding and filtering by line prefixes serve to distin-
       guish sources and targets.

       In the end a single program can communicate with and control  a  multi-
       tude  of  other programs through just reading from STDIN and writing to
       its STDOUT.

       The Plumb2 command line is a sequence of four different kinds of parame-
       ters,  in  any order. Some of the parameters serve to set the values of
       variables.  These variables are used to pass in  data  (such  as  file-
       names) to the Plumb2 control language.

       -69 command1 command2
              The  two  commands  are  started  behind a shell (like with sys-
              tem(3)) and connected together with two pipes (STDOUT to STDIN).
              Invoking Plumb2 as "69 command1 command2" is equivalent to "plumb2
              -69 command1 command2".

       -v key value
              Assigns a single value to a variable.

       -V key value1 value2... .
              Assigns a list of values to a variable (terminate  list  with  a
              single period).

       -f filename
              Read Plumb2 control commands from the specified file.

       -d     Increases  the  (debug) log verbosity. May be specified multiple
              times.  Log messages are written to Plumb2's STDERR. The  initial
              level is ERROR; further levels are WARNING, DEBUG, JUNK.

              Execute the given commands as Plumb2 control commands.

       Processes provide sources (such as their STDOUT and STDERR) and targets
       (their STDIN). There are special sources  and  targets  not  associated
       with processes (for example, Plumb2's own STDOUT is a target).

       Pipes  go  between sources and targets. Each source has an ordered list
       of pipes going from it. When data arrives at a source, the data will be
       written  into the pipes, in sequence. In certain conditions, a pipe can
       decide to "break the chain" and stop data from  getting  to  subsequent

       There  is a special target (called :ctrl ) from which Plumb2 parses con-
       trol instructions, and a source (called :ev )  to  which  Plumb2  writes
       events  (e.g.  when a process dies). Not connecting them to anywhere is
       not an error.

       Process sources have a file  descriptor  and  a  buffering  scheme.  By
       default,  processes  are  started  with  two  sources: STDOUT for FD 1,
       STDERR for FD 2, both line-buffered.  Fixed  record  buffering  (always
       waiting  until  N bytes are accumulated) and no buffering (sending any-
       thing received right away) are the possible alternatives. Processes can
       be started with more pre-opened sources on user-specified FDs.

       Process  targets  have  a  file  descriptor.  By default, processes are
       started with one target: STDIN for FD 0. Processes can be started  with
       more pre-opened targets on user-specified FDs.

       Processes  have  an  associated  termination  action: either "event" or
       "kill". When a process terminates, either of two things can happen.  If
       "kill"  is  set,  then Plumb2 exits completely (killing all started pro-
       cesses). If "event" is set, only an event is generated, and  everything
       continues.  The  global default is "kill", adjustable per process or by
       the "default proc" command.

       Each source has an associated termination action: also  either  "event"
       or  "kill".  This  action  is  taken when the source gets an EOF (or an
       error).  The global default is "kill", adjustable per source or by  the
       "default eof" command.

       Each  target  has an associated termination action: also either "event"
       or "kill". This action is taken when the target gets a write error (not
       when  otherwise  closed,  by  e.g.  the  termination  of the associated
       process). The global default is "kill", adjustable per target or by the
       "default write" command.

       Each  source  and target has an associated "cascade" flag. When this is
       true, the source or target gets closed when all pipes going from/to  it
       are closed. The global default is true, adjustable per source/target or
       by the "default source|target keep|cascade" commands.

       The Plumb2 control language is line-based. Each line is fed to the token
       parser,  which splits it into a sequence of strings called tokens; this
       usually just means splitting the line into whitespace-separated  words.
       The generated lists of tokens are then interpreted as Plumb2 commands.

       Lines  are split into tokens along the whitespace between them.  A hash
       mark (#) begins a comment, the rest of the line will be ignored.

       Multiple Plumb2 commands can be given in a  single  line,  separated  by
       semicolons (;). No ending semicolon is necessary.

       Any string of non-whitespace, non-special characters will be considered
       a token in itself.

       Tokens may be string literals, enclosed in  quotation  marks  (").  All
       characters until the closing quotation marks are considered part of the
       same token, including  spaces.  Some  special  characters  have  to  be
       escaped using a backslash (\). Supported escape notation: \\ for a sin-
       gle backslash, \n for a newline, \t for a tab, \b for a backspace, \xAB
       for an arbitrary character given in hex.

       Words beginning with the dollar sign ($) are considered variable names.
       The variable name is the part after the $ (until the next whitespace or
       special  character).  In place of the word, the tokens set as the value
       of the variable will be substituted. Undefined variables will result in
       no  inserted  tokens  (and  no error). A variable may have an arbitrary
       number of tokens associated with it (including zero).

       Some commands have IDs as parameters. IDs are ordinary tokens, but  the
       usable character set is restricted: [a-zA-Z_][a-zA-Z0-9_]*

       set ID [values...]
               Set  the variable named by the ID to the given list of (zero or
              more) tokens.

       proc ID [kill|event] [-pause] [inputfd]* [outputfd]* =  program  [argu-
               This command creates a new process.

               The ID identifies the process.

               [kill|event] specifies the action to take when the process ter-
              minates, overriding the default given with  the  "default  proc"

               -pause  disables  reading  from all sources (until an "unpause"
              command is issued for the process). This may be useful  to  stop
              data  being  lost  if  a  process  writes to a source before the
              appropriate pipes going from the source are defined. It's  guar-
              anteed that no reading will take place during the interpretation
              of commands given on the plumb2 command line, or between  execut-
              ing  multiple  commands  given on a single line (using semicolon

               Zero or more target (input FD) definitions may be given, in the
              following form:
                -i FD [kill|event] [keep|cascade]

              [kill|event]  specifies  the  EOF action, overriding the default
              given with the "default eof" command.

              [keep|cascade] overrides the cascade behavior,  given  with  the
              "default target keep|cascade" commands.

              If -i 0 (STDIN) is not specified, it is added implicitly.

               Zero  or  more  source (output FD) definitions may be given, in
              the following form:
                -o FD [recordsize] [kill|event] [keep|cascade]

              [recordsize] overrides the buffering scheme  (by  default  line-
              based). 0 means anything read is transmitted immediately; a pos-
              itive value specifies a record size.

              [kill|event] specifies the write error  action,  overriding  the
              default given with the "default write" command.

              [keep|cascade]  overrides  the  cascade behavior, given with the
              "default source keep|cascade" commands.

              If -o 1 (STDOUT) or -o 2 (STDERR)  are  missing,  they're  added
              implicitly.   If  STDERR  is  implicit, an implicit pipe is also
              generated, going to Plumb2's STDERR. You can inhibit this by giv-
              ing -o 2 explicitly.

               The  part  after  the  = is the program ARGV, as interpreted by
              execvp(3). At least one token  has  to  be  given  (the  program

       pipe [as ID] [-front] SOURCE TARGET [-(filter|cut)[:hex] prefix] [-pre-
       fix[:hex] prefix] [-endchain]
               This command creates a new pipe, from the given source  to  the
              given target.

               Usually  unique IDs are automatically generated for pipes. Fol-
              lowing the "as" keyword, the automatic ID can be overridden.

               The -front option inserts the pipe  at  the  beginning  of  the
              source's pipe sequence. Otherwise it's appended to the end.

               A source definition looks like this: processid[/FD] - if the FD
              is not given, /1 is used (STDOUT).  Instead  of  the  processid,
              :plumb2  may  be used to get plumb2's STDIN as a source, or :ev to
              get events.

               A target definition also looks like this: processid[/FD]  -  if
              the  FD is not given, /0 is used (STDIN). Instead of the proces-
              sid, :plumb2 may be used  to  get  plumb2's  STDOUT  as  a  target
              (:plumb2/2  can be used to get plumb2's STDERR), or :ctrl to write
              to the control parser.

               An optional filter definition follows the target, with  one  of
              four options: -filter -filter:hex -cut -cut:hex, followed by the
              filter prefix.  If such an option is given,  only  lines/records
              with  the  given  prefix  are  let through the pipe, the rest is
              ignored. -cut will remove the  prefix  before  transmitting  the
              data,  -filter will leave it. Using :hex in the option indicates
              that the prefix is given  in  hexadecimal  (two  characters  per
              byte),  so  any binary data can be specified (useful for record-
              based binary transmission).

               An optional prefix definition follows, with one of two options:
              -prefix  or  -prefix:hex,  followed  by  the  prefix string. The
              string will be prepended to each transmitted line/record.

               The -endchain option indicates  that  processing  the  source's
              pipe  sequence  should  stop  after  this  pipe - tipically used
              together with a filter option, in which case the  pipe  sequence
              continues if the filter doesn't match.

       unpause PROCESSID
              Start  reading  from the sources of the given process (used with
              the proc -pause option).

       endproc PROCESSID
              Kill the given process.

       endpipe PIPEID
              Shut down the given pipe.

       default proc kill|event
              Set default process termination action. Only  affects  processes
              started after using the command. The initial default is "kill".

       default eof kill|event
              Set  default source EOF termination action. Only affects sources
              started after using the command. The initial default is "kill".

       default write kill|event
              Set default target write error termination action. Only  affects
              targets  started after using the command. The initial default is

       default source|target keep|cascade
              Set default cascade flag for sources or  targets.  Only  affects
              sources  and  targets started after using the command. Both ini-
              tial defaults are "cascade".

       timer [as ID] [repeat] TIMEOUT OUTPUT
              Create a timer. OUTPUT will be written to :ev after TIMEOUT sec-
              onds  (a  floating point value). If "repeat" is given, the timer
              will repeat until plumb2 shutdown, or until it's deleted with the
              endtimer command.

       endtimer ID
              Delete the given timer. The timer will not trigger after this.

       When  reading  from  the  :ev source, a process can get notification of
       events happening in Plumb2. The following events are generated:

       ENDPROC processid KILL|EXIT exitvalue
              A process has ended (either KILLed or EXITed normally).

              A pipe has shut down for some reason.

       To simulate cat: plumb2 "pipe :plumb2 :plumb2"

       For a simple example, see the test/count10 script.

       For more complicated programs, you can set "default eof  event",  leave
       "default  proc kill" for the controlling process set, then set "default
       proc event" for the temporary processes started after.

       Need more examples...

       Lots. EOF termination action cannot be overridden for :plumb2/0, so it's
       always  KILL.  Need better shutdown procedure logic so outputs from all
       dying processes can be gathered.


       Mate Nagy (mnagy (at)

Home Page

Man page for Plumbnet

       plumbnet - line-based TCP relay server

       plumbnet [-i interface] -p port [-n] [-c hostname] [-s]

       Plumbnet waits for TCP connections on a specified port. It assigns each
       new connection an unique ID, and prints this ID in the  new  connection
       event to STDOUT. It will prints each subsequent line received from this
       connection to STDOUT, prefixed with the unique ID.

       Plumbnet also reads lines from its STDIN. It looks for a connection  ID
       prefix;  the  rest of the line will then be sent to the referenced con-

       In short: a single controlling program can manage a  multitude  of  TCP
       connections  by  communicating  with  only  plumbnet  (possibly through

       -i interface
              The interface address to bind to (by default

       -p port
              This mandatory argument specifies the TCP port to listen on.

       -n     Enable TCP_NODELAY.

       -c hostname
              Netcat mode: instead of usual plumbnet operation, connect to the
              hostname  (to  the  port given with -p). Data read from STDIN is
              sent to the host; data read from the host is written to STDOUT.

       -s     In netcat mode, shut down only after EOF both on STDIN  and  the
              socket.  If  -s is not given, plumbnet will shut down after get-
              ting EOF from the socket.

       -s     In plumbnet mode, report the EOF event instead of the END  event
              on  EOF  from a socket. The user is responsible to shut down the
              socket explicitly using the "end" (or "kill") command.

       Plumbnet reports events on its STDOUT. The following events are  possi-

       NEW id addr
       A  new  connection  is  opened,  with  the given ID, from the given TCP

       END id
       A connection is closed.

       EOF id
       A connection received EOF. (Only reported when the -s  flag  is  speci-

       IDs are integers, unique for each new connection.

       Furthermore,  all  lines  received  from connections will be printed to
       plumbnet's STDOUT, prefixed by their ID, followed by a single space.

       Plumbnet reads line by line from its STDIN. Each line has  to  be  com-
       posed of two parts, separated by a single space.

       In  the  common case, the first part before the first space is the con-
       nection ID; the rest of the line will be sent to the given connection.

       Instead of the connection ID, the controlling program may give commands
       (the parameter of the command follows the first space):

       end ID
       Close  the connection referenced by the ID, but wait for the write buf-
       fer to empty first.

       kill ID
       Close the connection referenced by the ID (the contents  of  the  write
       buffer are lost).

       It's also possible to use an asterisk (*) instead of the connection ID.
       In this case, the rest of the line will be sent  to  all  open  connec-


       Mate Nagy (mnagy (at)

Home Page