Redirection in bash

<< Here document

<< is known as here-document structure. You let the program know what will be the ending text, and whenever that delimiter is seen, the program will read all the stuff you’ve given to the program as input and perform a task upon it. An example:

$ wc << fin
> apple banana orange
> eggs bacon
> toast
> fin
 3  6 37

In this example we tell wc program to wait for fin string, then type in six words, and then type in fin to signal that we’re done giving input. In effect, it’s similar to running wc by itself, typing in words, then pressing Ctrl+D

In bash these are implemented via temp files, usually in the form /tmp/sh-thd.<random string>, while in dash they are implemented as anonymous pipes. This can be observed via tracing system calls with strace command. Replace bash with sh to see how /bin/sh performs this redirection

$ strace -e open,dup2,pipe,write -f bash -c 'cat <<fin
> test
> fin'

<<< Here string

<<< is known as here-string . Instead of typing in text, you give a pre-made string of text to a program. For example, with such program as bc we can do bc <<< 5*4 to just get output for that specific case, no need to run bc interactively.

Here-strings in bash are implemented via temporary files, usually in the format /tmp/sh-thd.<random string>, which are later unlinked, thus making them occupy some memory space temporarily but not show up in the list of /tmp directory entries, and effectively exist as anonymous files, which may still be referenced via file descriptor by the shell itself, and that file descriptor being inherited by the command and later duplicated onto file descriptor 0 (stdin) via dup2() function. This can be observed in this example:

$ ls -l /proc/self/fd/ <<< "TEST"
total 0
lr-x------ 1 user1 user1 64 Aug 20 13:43 0 -> /tmp/sh-thd.761Lj9 (deleted)
lrwx------ 1 user1 user1 64 Aug 20 13:43 1 -> /dev/pts/4
lrwx------ 1 user1 user1 64 Aug 20 13:43 2 -> /dev/pts/4
lr-x------ 1 user1 user1 64 Aug 20 13:43 3 -> /proc/10068/fd

And via tracing syscalls (output shortened for readability; notice how temp file is opened as fd 3, data written to it, then it is re-opened with O_RDONLY flag as fd 4 and later unlinked, then dup2() onto fd 0, which is inherited by cat later ):

$ strace -f -e open,read,write,dup2,unlink,execve bash -c 'cat <<< "TEST"'
execve("/bin/bash", ["bash", "-c", "cat <<< \"TEST\""], [/* 47 vars */]) = 0
...
strace: Process 10229 attached
[pid 10229] open("/tmp/sh-thd.uhpSrD", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
[pid 10229] write(3, "TEST", 4)         = 4
[pid 10229] write(3, "\n", 1)           = 1
[pid 10229] open("/tmp/sh-thd.uhpSrD", O_RDONLY) = 4
[pid 10229] unlink("/tmp/sh-thd.uhpSrD") = 0
[pid 10229] dup2(4, 0)                  = 0
[pid 10229] execve("/bin/cat", ["cat"], [/* 47 vars */]) = 0
...
[pid 10229] read(0, "TEST\n", 131072)   = 5
[pid 10229] write(1, "TEST\n", 5TEST
)       = 5
[pid 10229] read(0, "", 131072)         = 0
[pid 10229] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=10229, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
+++ exited with 0 +++

Opinion: potentially because here strings make use of temporary text files, it is the possible reason why here-strings always insert a trailing newline, since text file by POSIX definition has to have lines that end in newline character.

<(...) Process Substitution

As tldp.org explains,

Process substitution feeds the output of a process (or processes) into the stdin of another process.

So in effect this is similar to piping stdout of one command to the other , e.g. echo foobar barfoo | wc. But notice: in the bash manpage you will see that it is denoted as <(list). So basically you can redirect output of multiple (!) commands.

Note: technically when you say < <(...) you aren’t referring to one thing, but two redirection with single < and process redirection of output from <( . . .).

Now what happens if we do just process substitution?

$ echo <(echo bar)
/dev/fd/63

As you can see, the shell creates temporary file descriptor /dev/fd/63 where the output goes. That means < redirects that file descriptor as input into a command. So very simple example would be to make process substitution of output from two echo commands into wc:

$ wc < <(echo bar;echo foo)
      2       2       8

So here we make shell create a file descriptor for all the output that happens in the parenthesis and redirect that as input to wc .As expected, wc receives that stream from two echo commands, which by itself would output two lines, each having a word, and appropriately we have 2 words, 2 lines, and 6 characters plus two newlines counted. Side Note: Process substitution may be referred to as a bashism (a command or structure usable in advanced shells like bash, but not specified by POSIX), but it was implemented in ksh before bash’s existence as ksh man page and this answer suggest. Shells like tcsh and mksh however do not have process substitution. So how could we go around redirecting output of multiple commands into another command without process substitution? Grouping plus piping!

$ (echo foo;echo bar) | wc
      2       2       8

Effectively this is the same as above example, However, this is different under the hood from process substitution, since we make stdout of the whole subshell and stdin of wc linked with the pipe. On the other hand, process substitution makes a command read a temporary file descriptor. So if we can do grouping with piping, why do we need process substitution? Because sometimes we cannot use piping. Consider the example below - comparing outputs of two commands with diff (which needs two files, and in this case we are giving it two file descriptors)

diff <(ls /bin) <(ls /usr/bin)

Sources:

Ask Ubuntu
Ubuntu BASH manpage

Terminal: Ctrl+Alt+T

$ man bash