Pipes I

Introduction

Pipes are cool, and as with everything that is cool, I want to know how it works. This set of tutorials will be around this cool stuff called pipe. They won’t be extensive, the idea is to keep them simple. Given that I don’t have a background on Computer Science, I will take The Linux Programming Interface by Michael Timothy Kerrisk as my guide. So all that is on this tutorial, can also be found in there, with a very detailed explanation. By the way, the book has about 1556 pages, so be prepared. Of course is not 1556 pages about pipes but still is intimidating. Also you have the option to stay with me and try to figure out how the magic works.

Prerequisites

  1. Been on a Linux OS.
  2. A C/C++ compiler to compile a code of less than 10 lines, nothing fancy.

Pipes

Well pipes are used to pass data between process, here is a common example of a pipe been used

cat file.txt | less

In this case, we take the output of the command cat file.txt as the input of the command less . The | represents the pipe, pretty cool, right?

Note

In case you are a newbie to Linux, you can find the description of these commands running man cat for example. In that way you also learn about manpages, a cool way to document your code on UNIX based systems.

Back to pipes, you can imagine that it works like a piece of plumbing that allows the flow from one part to the other.

cat file.txt ====> [PIPE] ====> less
                   we   re
  • we stands for write end of pipe.
  • re stands for read end of pipe.

Cool features of a pipe

  1. It is unidirectional, meaning that the flow runs only from one process to the other and not otherwise. In our example from the process associated with cat to the process associated with less.

  2. Is a byte stream, meaning that the reading process(less) can read blocks of data of any size, regardless of the size of the data written by the writer process, which in our case is the process corresponding to cat.

  3. The data is read in the same order is written.

Points to clarify

  1. Related to the point 1, according to The Linux Programming Interface(TLPI) page 890:

On some other UNIX implementations—notably those derived from System V Release 4—pipes are bidirectional (so-called stream pipes). Bidirectional pipes are not specified by any UNIX standards, so that, even on implementations where they are provided, it is best to avoid reliance on their semantics.

  1. Another point to clarify is point 2. Given my poor English, it may seems that there’s no restriction on this writing and reading, even if you are writing from multiple process to one single magic pipe. Well my friend as you may imagine there’s a limitation on this specific case. The limitation is the following, again from TLPI:

If multiple processes are writing to a single pipe, then it is guaranteed that their data won’t be intermingled if they write no more than PIPE_BUF bytes at a time.

NOTE: Intermingled, really tricky word for a Cuban, it means mix or mingled together, according to Google, thanks again Google.

Now, when I read this, I was curios about the value of PIPE_BUF in my OS, but had no clue of where to look for this. I kept reading TLPI and the book gave me the clue that I needed, well more than a clue really :smile: , he said to me:

TLPI: Look man, take a look at , it got to be there.

Me: But I don’t know where is :sweat_smile:

In times of despair we need to use our head :thinking: , Eureka!!!:bath: just print it my friend.

   #include <stdio.h>
   #include <limits.h>
   
   int main(int argc, char *argv[])
   {
       printf("%d", PIPE_BUF);
       return 0;
   }

😎 Nice!!

This printed 4096, which by the way, can be found in /usr/include/linux/limits.h, at least on Ubuntu 19.10, don’t know if this depends on the OS.

So on my OS, as long as each process write 4096 bytes at a time, the data won’t be mixed.

  1. Last but not least, pipes have a limited capacity, so they seems to be some kind of special buffer. This imply that further writes to this pipe need to wait until the reading process remove some data from the pipe.

Conclusion

In this article I hope you get a basic idea of what is a pipe or maybe you already knew, but I hope you learn something new. On the next tutorial we will see how to create one using C to start tinkering with it, break some stuffs, because in the breaking we also learn.

Bibliography

  1. The Linux Programming Interface. A Linux and UNIX System Programming Handbook, by Michael Kerrisk.