Signals and sockets for querying a process

Hello again, dear reader!

Another part of the research I'm doing entails capturing, processing, and storing network packet attributes. This is done in a nifty application that invol... oh, but that is a post on its own! What I'd like to share today is an interesting little way of sharing data between the packet capture process and another running process.

So here's the skinny: my application uses libpcap to do packet capture. Pcap has a couple ways to process the packets it grabs off the wire, both of which are blocking. My code also has to answer queries from a single other process on the same machine. But, even if my while loop (if using pcap_next()) or callback (if using pcap_loop() or pcap_dispatch()) checks somehow for pending queries, the querying process has to wait until the pcap process gets another packet for that check to occur. The question arises: how can this application respond immediately to a query, regardless if packets are currently being captured?

Shared memory and multithreading is an option, as is pushing data to a separate database. But we want simple (my entire application is under 300 lines of code, counting the solution I describe here), and the machines I want to run this code on may not be able to support a database server. Besides, what's the fun in doing this if there isn't an opportunity for a bit of hackery?

It turns out that a combination of sockets and signals does just the trick. We're going to give the pcap process a listening Unix socket and and a function to handle signals, and let the OS do the rest of the work for us.

Before we jump into the code, let's making life simpler and take all this packet capture business out of the picture - that's complicated enough on its own, and may be the subject of another post in the future. Instead, let's say we have a table (2-D array) of students and the classes they must take. Each spot in the table is a struct with the quarter in which they took the class and the grade they received. That way we get a struct for the query (student and class) and another for the response (quarter and grade). And to keep things easy on ourselves, we'll make everything a number except for the grade, which will be a single character ('A', 'B', 'C', and so on).

Let's look at the code that processes a query (all error-checking has been removed for simplicity):

  void handle_query(int sig) {

    char buffer[BUF_SIZE];

   
    int sd = accept(sock, NULL, NULL);
    int len = recv(sd, buffer, BUF_SIZE, 0);
    struct query *q = (struct query *)buffer;
   
    struct record *r = &records[q->student][q->class];
   
    send(sd, (char *)r, sizeof(struct record), 0);
    close(sd);
  }

Wow, that was easy! Looks a lot like the standard TCP server from a network programming 101 class, doesn't it? Accept a connection from a listening socket, receive a query, typecast it into a struct, do a lookup, send the result typecast as a byte array, and close the connection. If you haven't seen something similar before, check this out or do a quick Google search for "Linux TCP server in C". I'll provide the definitions of struct query and struct record at the end; for now, just know that sock and records are global variables.

So what's with this funky-looking function declaration? It's a void; that's okay, but what's this int sig that never gets used in the function body? Well, this function isn't actually called by any code in the program per se; it's a signal handler. "A signal what?" you ask...

Smoke, hand, or turn _ _ _ _ _ _

When you type Ctrl+C or use the kill command, you're sending a signal to a process. The operating system interrupts whatever the process was doing and performs some action. Often this action is to terminate the process, but signals can be used for all sorts of things. In fact, there are two particular signals that Linux gives us to do with whatever we want. The header file <asm/signal.h> gives them the friendly names SIGUSR1 and SIGUSR2. Any time our process is sent one of those signals, some custom code of our creation (a.k.a. the function above) is invoked... if we tell it to do so. That happens with this system function:

  signal(SIGUSR1, handle_query);

This function, declared in <sys/signal.h>, registers a function (essentially a callback) to be invoked by the OS within the running context of the process whenever that signal is raised. That means that the function has access to all global data and process state; conveniently, this includes records and sock, which is what we need to answer a query. Great, so we have a way to interrupt the process, give a function the right of way, provide that function access to global data and open sockets, and a clean return back to whatever the process was doing before the interruption. Now, how do we trigger this behavior?

Linux gives us a programmatic version of the kill command:

  kill(pid, SIGUSR1);

Despite its suggestive name, this is simply the way to raise a signal at a process. The only trick is knowing the Process ID (PID) of that process. So we have a bit more work to do:

  int fd = creat(PID_FILE, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
  
  char *pid;
  int pidlen = asprintf(&pid, "%u", getpid());
  
  write(fd, pid, pidlen);
  close(fd);
  
  free(pid);

This code creates a file (defined by PID_FILE) and writes the PID as an ASCII-encoded number there. All those odd-looking arguments OR'ed together in the creat() define read/write permission for the owner of the process and read-only permission for everyone else. The PID of the process is provided by getpid(), courtesy of <unistd.h>. (Note that other header files include this by default.) Now a client application can open this file (assuming it knows which file to open) and read in the PID:

  int fd = open(PID_FILE, O_RDONLY);
  

  char pidch[PID_SZ];
  read(fd, pidch, PID_SZ);

  int pid = strtol(pidch, NULL, 10);
  

  close(fd);

We assume PID_SZ is large enough for the buffer to hold the PID as an ASCII string; 10 bytes should be plenty for any 32-bit integer, but a few extra wouldn't hurt. That function strtol() (string-to-long) is pretty neat; it makes parsing the ASCII PID back into an integer a snap (and lets you select what base you want it converted from!). An alternative is atoi().

A socket by any other family...

Okay. Method to interrupt a process and service a query: check. Now, let's take a look at how we define that socket we saw earlier:

  sock = socket(AF_UNIX, SOCK_STREAM, 0);
 
  struct sockaddr_un svraddr;
  memset(&svraddr, 0, sizeof(svraddr));
  svraddr.sun_family = AF_UNIX;
  strcpy(svraddr.sun_path, SOCK_FILE);

 
  bind(sock, (struct sockaddr *)&svraddr, SUN_LEN(&svraddr));
  listen(sock, 5);

Hmm... looks familiar. We create a socket, fill in a sockaddr struct, bind the socket to a port, and tell it to start listening for incoming requests. But wait! What's this sockaddr_un struct? We saw sockaddr_ll last time when using raw packet sockets; this time we introduce Unix sockets, which provide all the benefits of a connection-oriented stream but are completely internal to the machine. Fortunately, this one is very simple to use. Rather than specifying port numbers and other shenanigans, we simply declare the socket to be of family AF_UNIX (which is a hand-me-down from Sun Unix, hence the "sun" in the field names) and give a path to a file (SOCK_FILE) that will serve as a connection point for the socket. This file is of a special type in the Unix filesystem, and is created by the call to bind(). A client process wanting to connect will use analogous code to "connect" to that file:

  int sock = socket(AF_UNIX, SOCK_STREAM, 0);
 
  struct sockaddr_un svraddr;
  memset(&svraddr, 0, sizeof(svraddr));
  svraddr.sun_family = AF_UNIX;
  strcpy(svraddr.sun_path, SOCK_FILE);
 
  connect(sock, (struct sockaddr *)&svraddr, SUN_LEN(&svraddr));

Putting it together

Beyond that, it's all sends, receives, and bit-flipping. Since I've omitted the discussion of pcap, let's just create a never-ending loop that notionally does some processing but never checks the socket for incoming requests:

  int main(int argc, char **argv) {
    setup_ipc();
   
    running = 1;
    while (running) {
     
      /*
         Read in report cards and update table,
         never looking to see if a query has arrived
      */
     
    }
   
    return 0;
  }

Let's also try to be kind and clean up after ourselves:

  void destroy_ipc(int sig) {
    unlink(SOCK_FILE);
    close(sock);
    unlink(PID_FILE);
    running = 0;
  }

This is another signal handler, in this case for the signal raised by a Ctrl+C (SIGINT). It unlinks (deletes) the two files we created earlier and closes the Unix socket. creat() will fail and there will be issues with your IPC if those files weren't deleted on the last execution; just delete them by hand if your code crashes, or check for them and delete if necessary on code startup. The variable running is another global which for our server breaks the loop and allows the process to end cleanly. In the original pcap code, the assignment to zero would be replaced with a call to pcap_breakloop().

As is becoming customary (if I can keep it up), I've included the full code. Creating some code inside the while loop to update grades is left as an exercise for the reader. Good luck and happy bit-twiddling!

The server (download sock_sig_server.c)

  #include <string.h>
  #include <stdlib.h>
  #include <signal.h>
  #include <fcntl.h>
  #include <sys/socket.h>
  #include <sys/un.h>
 
  #define PID_FILE  "/tmp/query.pid"
  #define SOCK_FILE "/tmp/query.sock"
  #define BUF_SIZE  8
 
  int sock;
 
  int running;
 
  struct query {
    unsigned short student;
    unsigned short class;
  };
 
  struct record {
    unsigned short quarter;
    char grade;
  };
 
  #define STUDENTS 50
  #define CLASSES 16
 
  struct record records[STUDENTS][CLASSES];
 
  void handle_query(int sig) {
    int sd = accept(sock, NULL, NULL);
    if (sd < 0) return;
   
    char buffer[BUF_SIZE];
    int len = recv(sd, buffer, BUF_SIZE, 0);
   
    struct query *q = (struct query *)buffer;
   
    struct record *r = &records[q->student][q->class];
   
    send(sd, (char *)r, sizeof(struct record), 0);
   
    close(sd);
  }
 
  void destroy_ipc(int sig) {
    unlink(SOCK_FILE);
    close(sock);
    unlink(PID_FILE);
    running = 0;
  }
 
  void setup_ipc() {
    signal(SIGUSR1, handle_query);
    signal(SIGINT, destroy_ipc);
   
    sock = socket(AF_UNIX, SOCK_STREAM, 0);
   
    struct sockaddr_un svraddr;
    memset(&svraddr, 0, sizeof(svraddr));
    svraddr.sun_family = AF_UNIX;
    strcpy(svraddr.sun_path, SOCK_FILE);
    bind(sock, (struct sockaddr *)&svraddr, SUN_LEN(&svraddr));
    listen(sock, 5);
   
    int fd = creat(PID_FILE, S_IRUSR | S_IWUSR 

                           | S_IRGRP | S_IROTH);
    char *pid;
    int pidlen;
    pidlen = asprintf(&pid, "%u", getpid());
    write(fd, pid, pidlen);
    close(fd);
    free(pid);
  }
 
  int main(int argc, char **argv) {
    setup_ipc();
   
    running = 1;
    while (running) {
     
      /*
         Read in report cards and update table,
         never looking to see if a query has arrived
      */
     
    }
   
    return 0;
  }

The client (download sock_sig_client.c)

  #include <stdio.h>
  #include <fcntl.h>
  #include <signal.h>
  #include <sys/socket.h>
  #include <sys/un.h>
 
  #define PID_FILE "/tmp/query.pid"
  #define SOCK_FILE "/tmp/query.sock"
  #define PID_SZ 16
  #define BUF_SZ 8
 
  struct query {
    unsigned short student;
    unsigned short class;
  };
 
  struct record {
    unsigned short quarter;
    char grade;
  };
 
  int main(int argc, char **argv) {
    if (argc != 3) {
      fprintf(stderr, "Usage: ./dbcli <student> <class>\n");
      return -1;
    }
   
    char qbuf[sizeof(struct query)];
    struct query *q = (struct query *)qbuf;
    q->student = atoi(argv[1]);
    q->class = atoi(argv[2]);
   
    int fd = open(PID_FILE, O_RDONLY);
    char pidch[PID_SZ];
    read(fd, pidch, PID_SZ);
    close(fd);
    int pid = strtol(pidch, NULL, 10);
   
    int sock = socket(AF_UNIX, SOCK_STREAM, 0);
   
    struct sockaddr_un svraddr;
    memset(&svraddr, 0, sizeof(svraddr));
    svraddr.sun_family = AF_UNIX;
    strcpy(svraddr.sun_path, SOCK_FILE);
   
    connect(sock, (struct sockaddr *)&svraddr, SUN_LEN(&svraddr));
   
    send(sock, qbuf, sizeof(qbuf), 0);
   
    kill(pid, SIGUSR1);
   
    char rbuf[sizeof(struct record)];
   
    recv(sock, rbuf, sizeof(rbuf), 0);
    struct record *r = (struct record *)rbuf;
   
    printf("Student %u class %u -> quarter %u grade %c\n",
           q->student, q->class, r->quarter, r->grade);
   
    close(sock);
   
    return 0;
  }