Usually, this type of bug can only occur in compiled languages that do no bounds checking when accessing arrays, such as C. In many other languages, string objects (and arrays in general) simply refuse to let you write past their end. In contrast, strings in C are simply character arrays of unspecified size, and programmers are left to check memory accesses themselves.
Some people have drawn the conclusion that we should therefore switch to a real programming language and good riddance to memory corruption.2.1 This argument is bogus, in my opinion. Many useful tools have a potential for maiming the user (and bystanders) if pointed in the wrong direction; programming languages are no different from power drills or chainsaws in this respect. A sloppily written network server implemented in a language such as Pascal may not be susceptible to buffer overflows, but it may be an easy target for other types of memory attacks.
Writing past the end of some buffer is merely one particularly foolish thing you can do to a chunk of memory. There are other memory corruption bugs as well, for instance those related to dynamic memory allocation, that can have security implications. You learn about these almost the same day you learn about pointers: freeing memory twice, accessing memory that has been freed, etc.
For a very long time, common wisdom had it that these bugs were not exploitable. However, it has been proven only recently that bugs like passing a bad pointer to free() can be exploited to attack e.g. a setuid application.
If we follow the argument of ``don't use a programming language that lets programmers make mistake X,'' we would thus also have to chuck out PASCAL and C++ because they allow dynamic allocation and destruction of objects as well. Which pretty much narrows the field down to one mainstream language, Java, because it doesn't have a pointer type. Needless to say, Java is not really one of the hottest contenders when it comes e.g. to system level programming, where, coincidentally, most of the security critical mistakes are still being made.
In my opinion, a good design and a programmer's awareness of common security problems make much more of a difference than the choice of programming language. In fact, programmers may make more mistakes in a programming language they're less familiar with than one they've been using for years.