People usually expect a paper or tutorial about secure programming to cover buffer overflows. However buffer overflows are terminally boring (especially once you've had to fix a couple of them), so let's start with them so we have them out of our way.
Buffer overflows occur when your application uses a chunk of memory of a certain size, and writes past the end of this memory chunk when copying data to it. This will destroy any data stored in memory locations following it (also referred to as memory corruption).
This type of bug has been the root of the problem (excuse this bad pun) of probably four out of five major vulnerabilities in Unix based network servers, including the Washington University IMAP server and various FTP daemons. However, buffer overflows are also dangerous in setuid commands such as su, and applications such as mail readers (Eudora) and web browser (netscape).
Memory corrption bugs (weŽll see shortly that buffer overflows are but one species of a broader family of problems) allow an attacker to completely change what a process is doing. Consider a network service like syslogd, the Unix system log daemon. All it does is accept data from local or possibly remote processes, and write it to a file. Most security problems (if syslogd were affected by them) would do something as an by-product of normal operation: writing to other files, writing random stuff to other people's terminals, that sort of stuff. The system log daemon was never intended to invoke external programs, hence these ``normal'' security bugs will never result in the execution of external programs. By corrupting its memory however, an attacker can do anything, such as executing /bin/sh or replacing the syslogd process with a web server!
TODO: Memory bugs on OSes that don't have a per process address space