How to grep a log file within a specific time period

egrep '^[^ ]+ (0[89]|1[0-9]|2[012]):'

Detailed explanation can be found in various regex (regular expression) tutorials; egrep uses "POSIX extended" syntax (man 7 regex).

• The first ^ means "start of the line".

• [^ ]+ just matches the date field, regardless of the actual date.

  • [...] means "any character between the brackets", so [89] will match either 8 or 9; [0-9] is any number, and [^ ] is anything except a space (because of the ^ inside brackets).

  • + means "one or more of the previous" (for example, a+ would match a, aaa, and aaaaaaaa).

  • So ^[^ ]+ will start with the beginning of line, and match as many non-space characters as it can.

• (...|...|...) means "either of the given patterns", so (0[89]|1[0-9]|2[012]) means "either 0[89] or 1[0-9] or 2[012]". It will match all numbers from 08 to 22.

A somewhat better option is:

awk -F'[: ]' '$2 >= 8 && $2 <= 22 { print }'

The -F option splits every line into separate fields according to the [: ] regex (matching either : or a space), and the awk script checks the 2nd column (the hour).

Why bother using grep? You can simply use sed.

example:

sed -n '/Jun 17 13:39:54/ , /Jun 18 10:50:28/p' kern.log

This will print all the logs between June 17 13:39:54 and June 18 10:50:28

There's actually a much easier way to do this.

Download/Documentation: autodrgrep.kl.sh

Command:

./autodrgrep.kl.sh   notchef   /tmp/client.log   '2016-05-08_08:00:00,2016-05-08_23:00:00'   'INFO'   'a2ensite'   5  10  -show

Explanation:

• autodrgrep.kl.sh is the tool name.

• notchef is an option that is passed to the tool to tell it what to do. In this particular case, it is telling the tool what type of log file /tmp/client.log is.

• /tmp/client.log is of course the log file.

• 2016-05-08_19:12:00,2016-05-08_21:13:00 is the range of date from within the log that you wish to scan

• "INFO" is one of the strings that is in the lines of logs that you're interested in.

• "a2ensite" is another string on the same line that you expect to find the "INFO" string on. Specifying these two strings (INFO and a2ensite) isolates and processes the lines you want a lot quicker, particularly if you're dealing with a huge log file.

• 5 specifies Warning. By specifying 5, you're telling the program to alert as WARNING if there are at least 5 occurrences of the search strings you specified

• 10 specifies Critical. By specifying 10, you're telling the program to alert as CRITICAL if there are at least 10 occurrences of the search strings you specified.

• -show specifies what type of response you'll get. By specifying -shown, you're saying if anything is found that matches the specified patterns, output to screen.

Sample run:

# ./autodrgrep.kl.sh notchef  /tmp/client.log   '2016-05-08_19:12:00,2016-05-08_21:13:00' 'INFO' 'a2ensite'  5  10  -show

[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully
23
2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

What if the user specified date range or time frame is not in the log?

Each run of the above command will always have a line (last line of the output) that either says "ATWFILF" or "ETWNFILF".

• ATWFILF means that the actual date range or time frame you requested searched was found in the log. So this is very good.

• ETWNFILF means the actual date range or time frame you requested searched was NOT found in the log. In this case, the closest time to the time you specified will be detected and used instead.