Concurrency problems

Many network services are designed to serve multiple clients in a single process. When doing this, special care needs to be taken to make sure the server won't get stuck serving a single client. Being able to stop a network service from serving other, legitimate clients, is also a denial of service attack.

Probably the worst offenders are RPC based services; because of the way RPC over TCP is implemented, an RPC server that has received an incomplete request over TCP will wait for 30 seconds for more input before giving up. During that time, the server is not able to service any other requests. That means you can render an RPC server pretty much unusable by sending it a request packet, one byte at a time, 29 seconds apart.

Below is some typical code for this kind of problem.^9.7 Note that this type of problem applies only to TCP-based services. For UDP-based services, you do not have this problem because if you receive a UDP datagram, it contains the entire request already; there is no need to wait for additional data.

XXX: This may not be immediately obvious to people who gave never written a network daemon. Explain in more detail.

    while (1) {
        fd = accept(sock, &clientaddr, &addrlen);
        process_and_read_data_as_you_go(fd);
        close(fd);
    }

The solution for problems like this to either fork a new process for every connection accepted (which works great for most services), or to receive requests in the main dispatch loop, and not start processing until a complete request has arrived.

The pseudo code for the fork approach looks as shown below. Note that without the call to signal, the exit status of the child processes would never be collected, leaving them in ``zombie'' state. Alternatively, you can install a real signal handler for SIGCHLD and collect the status of child processes whenever one of them exits.

XXX: These solutions need to be described in more detail.

     signal(SIGCHLD, SIG_IGN);
     while (1) {
          fd = accept(sock, &clientaddr, &addrlen);
          if (fork() == 0) {
               process_and_read_data_as_you_go(fd);
               exit(0);
          }
          close(fd);
     }

The deferred processing approach can be implemented like this:

    FD_SET(sock, &filemap);
    while (1) {
        fd_set  rfds = filemap;

        select(getdtablesize(), &rfds, NULL, NULL, timeout);
        if (FD_ISSET(sock, &rfds)) {
            fd = accept(sock, &clientaddr, &addrlen);
            FD_SET(fd, &filemap);
        }
        for (all open connections) {
            if (!FD_ISSET(fd, &rfds))
                continue;
            read_some_more_data(fd);
            if (request complete) {
                process(request);
                FD_CLR(fd, &filemap);
                close(fd);
            }
        }
    }

Here, read_some_more_data reads newly arrived data from the socket connection and puts it into a per-connection buffer.

Note that this implementation is still a bit oversimplified. When large amounts of data are transferred, a network client can also make a network server block in the write() call. This can happen if the client does not read the server responses, so data starts to accumulate at the client, which soon refuses to queue any more packets received from the server for this connection. So the backlog spills over to the server, and eventually the server's kernel stops the server process from transmitting any more data. Thus, if your server sends replies of more than a few hundred bytes, you should also take into account that write() calls may block.

Unfortunately, the RPC code in most Unix implementations doesn't attempt to address any of these concurrency problems. They can be worked around, but only at a considerable cost in terms of code complexity (you basically need to re-implement much of the network handling). So if you need a network server that cannot be incapacitated that easily, don't use RPC.