Linux: find out what process is using all the RAM?

On Linux in the top process you can press < key to shift the output display sort left. By default it is sorted by the %CPU so if you press the key 4 times you will sort it by VIRT which is virtual memory size giving you your answer.

Another way to do this is:

ps -e -o pid,vsz,comm= | sort -n -k 2

should give you and output sorted by processes virtual size.

Here's the long version:

ps --everyone --format=pid,vsz,comm= | sort --numeric-sort --key=2

Show the processes memory in megabytes and the process path.

ps aux  | awk '{print $6/1024 " MB\t\t" $11}'  | sort -n

Just a side note on a server showing the same symptoms but still showing memory exhaustion. What ended up finding was a sysctl.conf from a box with 32 GB of RAM and setup for a DB with huge pages configured to 12000. This box only has 2 GB of RAM so it was assigning all free RAM to the huge pages (only 960 of them). Setting huge pages to 10, as none were used anyway, freed up all of the memory.

A quick check of /proc/meminfo to look for the HugePages_ settings can be a good start to troubleshooting at least one unexpected memory hog.

Make a script called show-memory-usage.sh with content:

#!/bin/sh
ps -eo rss,pid,user,command | sort -rn | head -$1 | awk '{ hr[1024**2]="GB"; hr[1024]="MB";
 for (x=1024**3; x>=1024; x/=1024) {
 if ($1>=x) { printf ("%-6.2f %s ", $1/x, hr[x]); break }
 } } { printf ("%-6s %-10s ", $2, $3) }
 { for ( x=4 ; x<=NF ; x++ ) { printf ("%s ",$x) } print ("\n") }
 '

Make it executable with chmod +x show-memory-usage.sh and call it like this ./show-memory-usage.sh 10 (10 => show max 10 lines)

More "human-readable" version:

#!/bin/sh

# This script displays the top memory-consuming processes in a human-readable format.
# It accepts one argument: the number of top processes to display.

# List processes with their memory usage, PID, user, and command line,
# then sort them by memory usage in descending order and pick the top ones as specified by the script's argument.
ps -eo rss,pid,user,command | sort -rn | head -$1 | awk '
  BEGIN {
    # Define human-readable memory size units.
    hr[1024**2]="GB"; hr[1024]="MB";
  }
  {
    # Convert the memory usage to a human-readable format.
    for (x=1024**3; x>=1024; x/=1024) {
      if ($1>=x) {
        printf ("%-6.2f %s ", $1/x, hr[x]); 
        break;
      }
    }
  }
  {
    # Print the process ID and user.
    printf ("%-6s %-10s ", $2, $3);
  }
  {
    # Print the command line, handling commands with spaces.
    for (x=4; x<=NF; x++) {
      printf ("%s ", $x);
    }
    print ("\n"); # Ensure each process info is on a new line.
  }
'

Output Example:

5.54   GB 12783  mysql      /usr/sbin/mysqld

1.02   GB 27582  root       /usr/local/cpanel/3rdparty/bin/clamd

131.82 MB 1128   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start inventory.mass.update --single-thread --max-messages=10000

131.21 MB 1095   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start product_action_attribute.update --single-thread --max-messages=10000

131.19 MB 1102   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start product_action_attribute.website.update --single-thread --max-messages=10000

130.80 MB 1115   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start exportProcessor --single-thread --max-messages=10000

130.69 MB 1134   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start inventory.reservations.update --single-thread --max-messages=10000

130.69 MB 1131   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start inventory.reservations.cleanup --single-thread --max-messages=10000

130.69 MB 1107   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start codegeneratorProcessor --single-thread --max-messages=10000

130.58 MB 1120   company+   /opt/cpanel/ea-php73/root/usr/bin/php /home/companyde/redesign.company.de/bin/magento queue:consumers:start inventory.source.items.cleanup --single-thread --max-messages=10000

Script explanation

1. Shebang (#!/bin/sh): Indicates that the script should be executed with the Bourne shell, a common command interpreter.

2. Process Listing and Sorting (ps -eo rss,pid,user,command | sort -rn | head -$1):

   • The ps command lists processes with specific details: rss (memory usage), pid (process ID), user (the process owner), and command (the command that initiated the process).
   • The output is sorted in reverse numeric order (sort -rn) based on memory usage, ensuring the most resource-intensive processes are listed first.
   • head -$1 limits the output to the top $1 processes, where $1 is the script's input argument.

3. AWK Script:

   • The AWK script is structured for readability and is divided into distinct parts, each performing a specific function, with comments explaining each part.
   • BEGIN Block: Sets up an associative array hr that maps memory sizes to their units (GB and MB), preparing for memory size conversion.
   • Memory Conversion Loop: Iterates through possible memory sizes (GB, MB), converting the RSS value from kilobytes (the default unit in ps) to a more readable format. It prints the converted value along with its unit.
   • Printing Process Details: Outputs the process ID and the user in a formatted manner, ensuring alignment and readability.
   • Command Line Printing: Handles commands with spaces correctly by iterating from the fourth field to the last ($4 to $NF). This ensures the entire command line that started the process is printed accurately. It concludes with printing a newline character to separate each process's details.