Linux – Use perl and snmp to monitor actual memory usage in Linux

I need to get real free memory in a stripped Linux device. I want to get real free memory.
Getting free memory doesn’t take buffers/caches into account, so I don’t seem to have memory, but I still have enough space for the app to use.

I’m using snmp to get this value from a Perl script, and I have these oids available to me :

> HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: Physical memory 
> HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: Virtual memory
> HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: Memory buffers
> HOST-RESOURCES-MIB::hrStorageDescr.7 = STRING: Cached memory
> HOST-RESOURCES-MIB::hrStorageDescr.10 = STRING: Swap space
> HOST-RESOURCES-MIB::hrStorageDescr.31 = STRING: /
> HOST-RESOURCES-MIB::hrStorageDescr.32 = STRING: /boot
> HOST-RESOURCES-MIB::hrStorageDescr.33 = STRING: /var/log
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.1 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.3 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.6 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.7 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.10 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.31 = INTEGER: 4096 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.32 = INTEGER: 1024 Bytes
> HOST-RESOURCES-MIB::hrStorageAllocationUnits.33 = INTEGER: 4096 Bytes
> HOST-RESOURCES-MIB::hrStorageSize.1 = INTEGER: 8240104
> HOST-RESOURCES-MIB::hrStorageSize.3 = INTEGER: 16626024
> HOST-RESOURCES-MIB::hrStorageSize.6 = INTEGER: 8240104
> HOST-RESOURCES-MIB::hrStorageSize.7 = INTEGER: 4697308
> HOST-RESOURCES-MIB::hrStorageSize.10 = INTEGER: 8385920
> HOST-RESOURCES-MIB::hrStorageSize.31 = INTEGER: 4062954
> HOST-RESOURCES-MIB::hrStorageSize.32 = INTEGER: 295561
> HOST-RESOURCES-MIB::hrStorageSize.33 = INTEGER: 33011530
> HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 5610512
> HOST-RESOURCES-MIB::hrStorageUsed.3 = INTEGER: 5610512
> HOST-RESOURCES-MIB::hrStorageUsed.6 = INTEGER: 326360
> HOST-RESOURCES-MIB::hrStorageUsed.7 = INTEGER: 4697308
> HOST-RESOURCES-MIB::hrStorageUsed.10 = INTEGER: 0
> HOST-RESOURCES-MIB::hrStorageUsed.31 = INTEGER: 1673253
> HOST-RESOURCES-MIB::hrStorageUsed.32 = INTEGER: 24061
> HOST-RESOURCES-MIB::hrStorageUsed.33 = INTEGER: 19467049

I

can’t get my exact RAM usage in Linux I have to add physical free memory, what is stored in the cache and buffer?
What is the best approach in Perl? Is hrStorage the best or best to use de UCD-SNMP_MIB (.1.3.6.1.4.1.2021.4)?

EDIT: I created the following scriptlet, which calculates the percentage of memory actually used based on the comments here. Do you think this is the best I can do, or is it something else I should do?

my $memRealTotalOID = '.1.3.6.1.4.1.2021.4.5.0';
my $memRealFreeOID = '.1.3.6.1.4.1.2021.4.6.0';
my $memRealCachedOID = '.1.3.6.1.4.1.2021.4.15.0';
my $memRealBuffersOID = '.1.3.6.1.4.1.2021.4.14.0';

my ($session, $error) = Net::SNMP->session(
     -hostname  => $np->opts->host,
     -community => $np->opts->community,
   );

if (!defined $session) {
      $np->nagios_exit (WARNING, $error)
      #printf "ERROR: %s.\n", $error;
   }

my $memRealFree = $session->get_request(-varbindlist => [ $memRealFreeOID],);
   my $memRealTotal = $session->get_request(-varbindlist => [ $memRealTotalOID],);
   my $memRealCached = $session->get_request(-varbindlist => [ $memRealCachedOID],);
   my $memRealBuffers = $session->get_request(-varbindlist => [ $memRealBuffersOID],);

$session->close();

my $buffers = $memRealBuffers->{$memRealBuffersOID}; 
   my $cache = $memRealCached->{$memRealCachedOID};
   my $total=  $memRealTotal->{$memRealTotalOID};
   my $free = $memRealFree->{$memRealFreeOID};

my $memRealUsed = $total - $free;
   my $memRealUsedMB = round ($memRealUsed / 1024);
   my $totalMB = round($total / 1024);

my $realTPercent = (($memRealUsed - $buffers - $cache)/ $total) * 100;
  my $realPercent = sprintf "%.2f", $realTPercent;

hrStorageUsed includes buffers and caches, so you can estimate the total available RAM: like this

totFree = total - used + buffers + cache

For example, have the following value:

$ snmpwalk -Os -v1 -cpublic localhost hrStorage | grep '\.[ 167] ='
hrStorageIndex.1 = INTEGER: 1
hrStorageIndex.6 = INTEGER: 6
hrStorageIndex.7 = INTEGER: 7
hrStorageType.1 = OID: hrStorageRam  
hrStorageType.6 = OID: hrStorageOther
hrStorageType.7 = OID: hrStorageOther
hrStorageDescr.1 = STRING: Physical memory
hrStorageDescr.6 = STRING: Memory buffers
hrStorageDescr.7 = STRING: Cached memory
hrStorageAllocationUnits.1 = INTEGER: 1024 Bytes
hrStorageAllocationUnits.6 = INTEGER: 1024 Bytes
hrStorageAllocationUnits.7 = INTEGER: 1024 Bytes
hrStorageSize.1 = INTEGER: 1016436
hrStorageSize.6 = INTEGER: 1016436
hrStorageSize.7 = INTEGER: 436156
hrStorageUsed.1 = INTEGER: 882112
hrStorageUsed.6 = INTEGER: 103056
hrStorageUsed.7 = INTEGER: 436156

We have:

totFree = hrStorageSize.1 - hrStorageUsed.1 + hrStorageUsed.6 + hrStorageUsed.7
        = 1016436         - 882112          + 103056          + 436156
        = 673536 KB

But wait, free(1) doesn’t say that:

$ free -k
              total        used        free      shared  buff/cache   available
Mem:        1016436      235480      135680       51272      645276      555492
Swap:       1048572      113076      935496

According to this, we actually have:

totFree = free   + buff/cache
        = 135680 + 645276
        = 780956 KB

Why is there a discrepancy? IT TURNS OUT THAT BOTH FREE AND HOST-RESOURCES-MIB GET DATA FROM /PROC/MEMINFO, BUT USE SLIGHTLY DIFFERENT METRICS. From man free:

buffers

Memory used by kernel buffers (Buffers in /proc/meminfo)

cache

Memory used by the page cache and slabs (Cached and Slab in /proc/meminfo)

SO FREE INCLUDES SLAB ALLOCATIONS IN THE CACHE, WHILE HOST-RESOURCES-MIB DOES NOT. If we get the slab allocation:

$ grep Slab /proc/meminfo
Slab:             106064 kB

And add it to what we get from SNMP and we have:

totFree = 673536 + 106064
        = 779600 KB

This is closer to what we get from free (780956 KB).

However, there is actually a better indicator. Again from man free:

available

Estimation of how much memory is available for starting new applications, without swapping. Unlike the data provided by the cache or free fields, this field takes into account page cache and also that not all reclaimable memory slabs will be reclaimed due to items being in use (MemAvailable in /proc/meminfo, available on kernels 3.14, emulated on kernels 2.6.27+, otherwise the same as free)

Note that in my example, the available memory (555492 KB) is significantly lower than the sum of available memory + buffer + cache (780956 KB).

Unfortunately, I can’t find the MIB that reports this value (neither does ucd-snmp-mib), so you might stick to the rough estimate I showed at the beginning.