Today I was going through the Planet SysAdmin rss feed and came across a posting by therek about the roll or /var (actually it’s a response to  blog posting by Chris Siebenmann about /var).   Somehow this got me thinking about something that had happened at work.
On one of our development servers Apache had been configured to core dump into /tmp.  Not a big deal…usually.  In this case it was.  When this server had been setup /tmp wasn’t put onto it’s own partition.  So, as Apache went happily along creating core files, it also ended up filling  the partition, the root partition.   Linux doesn’t like it when the partition holding ‘/’ and most of it’s friends fills up.  Clearing out all the core files freed the space, but the box was still not happy.  Linux had decided that none of the file systems were mounted.  In the end, the server needed to be rebooted to set things right.
So, what’s the lession?  Make sure /tmp is either a separate parition or a tmpfs parition and make sure it gets cleaned up regularly.