Linux monitoring, log and traffic analysis

First of all, here is a good link with a comprehensive list of monitoring tools for Linux:
20 Linux System Monitoring Tools Every SysAdmin Should Know

Monitoring performance

• top - standard utility for monitoring CPU, processes, memory and load average.
• htop - enhanced top program, it's more comfortable, I think.
• ps - command can be used to monitor memory and CPU too, for example: ps ux
• free - shows information about RAM. Usually, the most interesting is amount of free RAM in megabytes without buffers:

server:/home/user#free -m
total used free shared buffers cached
Mem: 2003 1330 673 0 83 865
-/+ buffers/cache: 381 1621
Swap: 3514 0 3514

 
In our example amount of free RAM is 1621 megabytes.

• Load average - consist of 3 numbers: load average for last 1,5 and 15 minutes. The load average tries to measure the number of active processes at any time. As a measure of CPU utilization, the load average is simplistic, poorly defined, but far from useless. A system with one program running CPU-intensive tasks has a load average of 1. Higher load averages reflect programs competing for available CPU time. You can also find the current load average via the uptime command, which displays the load average along with information on how long the computer has been running. The load average can be useful in detecting runaway processes. For instance, if a system normally has a load average of 0.5 but it suddenly gets stuck at a load average of 2.5, a couple of CPU-hogging processes may have hung—that is, become unresponsive. Hung processes sometimes needlessly consume a lot of CPU time. You can use top to locate these processes and, if necessary, kill them.


• uptime


• sar - very powerful tool, it can monitor almost all parameters, including disk, network, ram, cpu….. It launches periodically and stores statistics in /var/log/atsar.


• iostat - monitors nfs, cpu and disks.


• vmstat - monitors virtual memory.


• hdparm - shows information about hard disk, tests HDD speed


• hddtemp - shows HDD temprature


• smartd - SMART Disk Monitoring Daemon


• Monitorix

logs analysis

• logcheck


• logwatch

users monitoring

• w


• whoami


• who


• last


• lastlog


• finger

traffic analysis

• iptraf - very cool programm. It's interactive, it monitors in real time various network parameters.


• iftop - monitors bandwidth by hosts


• bwm-ng - very simple command-line interactive bandwidth monitor


• tc - monitors and shape traffic


• dsniff - package of various tools, including urlsnarf - for monitoring HTTP requests in real time


• ntop


• darkstat - very nice traffic graphing tool with embedded web server.


• Bandwidth Monitoring Tools For Linux

Security

• OSSEC -  Open Source Host-based Intrusion Detection System. It performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response.


• OSSIM - Open Source Security Information Management.


• Samhain