[PHP] log tailing

Hi all,

I have a /var/log/messages and /var/log/syslog file to parse to extract
information from.

It's mainly to insert the data to several SQL tables.

I have the to extract the date, and some information in the line.

Doing it with preg_match() and extracting the data is the first solution
comming to my mind.

Typically, a log line like:
Jun 29 11:24:10 dev5 sshd[12775]: Accepted password \
for dev5 from 192.168.0.12 port 50544 ssh2

should drive me to extract to $out[] like:
- $out[0] is "Jun 29 11:24:10"
- $out[1] is "sshd"
- $out[3] is "192.168.0.12"

So that I can:

INSERT INTO ssh_activity \
VALUES ('2012-06-29 11:24:10', '192.168.0.12')

I just need help on the right regexp function to use.
Would you know some PHP/regexp tutorials for that?

--
RMA.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

On Fri, Jun 29, 2012 at 3:49 AM, Mihamina Rakotomandimby
<[email protected]> wrote:
> I have a /var/log/messages and /var/log/syslog file to parse to extract
> information from.
>
> I have the to extract the date, and some information in the line.
[snip]
> I just need help on the right regexp function to use.
> Would you know some PHP/regexp tutorials for that?

The best documentation I've ever found on Regexes is O'Reilly's
"Mastering Regular Expressions" (3rd ed is 2006, but REs haven't
changed since then).

http://shop.oreilly.com/product/9780596528126.do

Only problem is it is hella expensive. (Well another problem is it is
hella big: >500 pages.)

Apart from that, there are *tons* of tutorials on the net. Just google
up "regular expression tutorials" and you should see pages and pages
of them.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

On Fri, Jun 29, 2012 at 4:49 AM, Mihamina Rakotomandimby
<[email protected]> wrote:
[snip!]
>
> Typically, a log line like:
> Jun 29 11:24:10 dev5 sshd[12775]: Accepted password \
>            for dev5 from 192.168.0.12 port 50544 ssh2
>
[snip!]
> So that I can:
>
> INSERT INTO ssh_activity \
>   VALUES ('2012-06-29 11:24:10', '192.168.0.12')
>
> I just need help on the right regexp function to use.
> Would you know some PHP/regexp tutorials for that?

You could take the pattern-matching load off of PHP entirely if
you used something along these lines. Just remember to adjust and
clean up as necessary.

<?php

$ssh_entries = explode(PHP_EOL,trim(`tail /var/log/syslog | awk
{'print $1,$2,$3 "|" $5 "|" $11'}`));

foreach ($ssh_entries as $s) {
$l = explode('|',$s);

// Remember to do whatever sanity necessary!
$sql = "INSERT INTO ssh_activity
VALUES('".$l[0]."','".$l[1]."','".$l[2]."')";
}
?>

--
</Daniel P. Brown>
Network Infrastructure Manager
http://www.php.net/

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

On Sat, Jun 30, 2012 at 2:30 PM, Daniel Brown <[email protected]> wrote:
>
> <?php
>
> $ssh_entries = explode(PHP_EOL,trim(`tail /var/log/syslog | awk
> {'print $1,$2,$3 "|" $5 "|" $11'}`));

Actually, the above was intended to grab just sshd entries, so
instead of 'tail' you should use 'grep sshd' in the line above.


> foreach ($ssh_entries as $s) {
>        $l = explode('|',$s);
>
>        // Remember to do whatever sanity necessary!
>        $sql = "INSERT INTO ssh_activity
> VALUES('".$l[0]."','".$l[1]."','".$l[2]."')";
> }
> ?>
>
> --
> </Daniel P. Brown>
> Network Infrastructure Manager
> http://www.php.net/



--
</Daniel P. Brown>
Network Infrastructure Manager
http://www.php.net/

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

---------- Forwarded message ----------
From: "tamouse mailing lists" <[email protected]>
Date: Jun 30, 2012 4:35 PM
Subject: Re: [PHP] log tailing
To: "Daniel Brown" <[email protected]>

On Jun 30, 2012 1:34 PM, "Daniel Brown" <[email protected]> wrote:
>
> On Sat, Jun 30, 2012 at 2:30 PM, Daniel Brown <[email protected]> wrote:
> >
> > <?php
> >
> > $ssh_entries = explode(PHP_EOL,trim(`tail /var/log/syslog | awk
> > {'print $1,$2,$3 "|" $5 "|" $11'}`));
>
> Actually, the above was intended to grab just sshd entries, so
> instead of 'tail' you should use 'grep sshd' in the line above.
>
>
> > foreach ($ssh_entries as $s) {
> > $l = explode('|',$s);
> >
> > // Remember to do whatever sanity necessary!
> > $sql = "INSERT INTO ssh_activity
> > VALUES('".$l[0]."','".$l[1]."','".$l[2]."')";
> > }
> > ?>
> >
> > --
> > </Daniel P. Brown>
> > Network Infrastructure Manager
> > http://www.php.net/
>
>
>
> --
> </Daniel P. Brown>
> Network Infrastructure Manager
> http://www.php.net/
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php

Crud-- sent to Dan instead of list:

Or just let awk do it:

tail /var/log/syslog | awk '/sshd/{print($1,$2,$3 "|" $5 "|" $11)}'

On 06/30/2012 09:32 PM, Daniel Brown wrote:
>> <?php
>> $ssh_entries = explode(PHP_EOL,trim(`tail /var/log/syslog | awk
>> {'print $1,$2,$3 "|" $5 "|" $11'}`));

This will tail a default number of lines.

I'm looking for a way to identify the last line, and when launching the
PHP script I get the added line between now and that last one.

There is a "logtail" utility in the "logtool" package, but I want a full
PHP equivalent.

The "logtail" utility inserts a "marker" in the logfile, which I find
intrusive and requiring root privilege.

My guess is identifying lines with hash or storing the last line in e
tmp file, or...

I'm looking for the least worst solution.

--
RMA.



--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

On Mon, Jul 2, 2012 at 3:23 PM, Mihamina Rakotomandimby
<[email protected]> wrote:
> On 06/30/2012 09:32 PM, Daniel Brown wrote:
>>>
>>> <?php
>>> $ssh_entries = explode(PHP_EOL,trim(`tail /var/log/syslog | awk
>>> {'print $1,$2,$3 "|" $5 "|" $11'}`));
>
>
> This will tail a default number of lines.
>
> I'm looking for a way to identify the last line, and when launching the PHP
> script I get the added line between now and that last one.
>
> There is a "logtail" utility in the "logtool" package, but I want a full PHP
> equivalent.
>
> The "logtail" utility inserts a "marker" in the logfile, which I find
> intrusive and requiring root privilege.
>
> My guess is identifying lines with hash or storing the last line in e tmp
> file, or...
>
> I'm looking for the least worst solution.
>
> --
> RMA.

You could also remember the number of bytes read, store that (in a tmp
file or database), and use fseek() to skip to that exact position.

- Matijn

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php