Copied from the now diseased Dinix web forum:
How can I tell what is using up all of my server’s memory?
Linux actually uses two different forms of disk caching, referred to as ‘cached’ and ‘buff’ in top. The amount of free memory displayed by the program top is misleading, since the Linux operating system will use most of the available memory for disk caching.
When you launch new processes, Linux will automatically shrink the cache to free up needed memory. If more memory is needed
after Linux has shrunk the disk cache, Linux will swap out currently unused portions of memory to disk.
Swapping out parts of a process’ memory is the reason for the difference between the SIZE and RSS columns for some
processes listed in top. The SIZE column is the amount of memory allocated to the process that can reside in physical memory
or on the swap partition. The RSS column shows how much of a process’ memory resides in physical memory. (The SIZE column
may actually show less memory than the process has requested, since Linux will wait to assign a unit of memory to the process
until the process has accessed memory inside that unit by reading or writing to it.)
Linux will move data from the swap partition back into physical memory once a process requires that piece of data. This is why
the data may remain on the swap partition even though enough physical memory has been freed to contain all the processes.
If you want to know how much physical memory your programs our actually using, you can subtract the ‘cached’ and ‘buff’
fields in top from the used memory field. Conversely, you can add the ‘cached’ and ‘buff’ fields to the free memo