Re: Re: Tail -f in lua, truncate aware?
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Re: Re: Tail -f in lua, truncate aware?
- From: "ms2008vip" <ms2008vip@...>
- Date: 2013年6月29日 13:56:49 +0800
First please excuse my English, It's not my natural language, Sorry. Let's on-topic,
I just want to emulate "tailf" or "tail -F" in lua, and I need that to monitor log files in
order to see new messages as soon they are enqueued(follow the growth of a log file).
Then I need to handle the output by line, format that or do something other. And I hope
it will recognize when file that I tailing is been truncated, it also isn't need too much
overhead(like tailf that does not access the file when it is not growing). I scoured the web,
but wasn't able to find a precise answer.
Does the above sounds familiar to someone? Any hints, tips or comments would be most
gratefully received. Awaiting your reply.
-------------------------------------------------------------
> >>In Linux you can use poll() or rpoll() from the luaposix library
> >>(https://github.com/luaposix/luaposix)
> >poll doesn't work on regular files. Or rather, it's defined to immediately
> >signal readiness, which makes it useless for this case.
> >
> > "Regular files shall always poll TRUE for reading and writing."
> >
> > -- http://www.opengroup.org/susv3xsh/poll.html
> >
> yes, by Bernd mentions output of process 'tail -F'. If he want output
> from an other program into Lua, it will most likely involve a pipe or a
> fifo, and pipes and fifos can be poll()'ed or select()'ed. Use a pipe if
> you want to fork and exec the 'tail -F' directly from Lua, or just make
> a named /path/to/fifo and do 'tail -F > /path/to/fifo'
This still doesn't handle the log rotation part of the question [1]. But
this seems like a "please help me solve my solution to this unstated
problem" instead of "here's what I want, how so I do it?" If the problem is
"I want to write a program that scans the log files generated by syslog"
then I do have a solution that will work (because I'm doing it). If it's
"I want to write a program that scans a file some arbitrary program is
logging to (not syslog)" then the problem becomes a bit harder.
So, assuming it's the "watch the output of syslog" problem, here's how I
solved that issue. I caused syslog to forward the relevent logs (for me,
all of them) to a local IP address on the box [5], and then wrote a program
[6] to receive this stream [7] and display the results in real time [8].
Because the data is being sent to a socket, if there's no program listening,
the packets are dropped. There's no issue with log rotation, since the data
isn't going to a file---it's being continuously streamed. And this is
on-topic, since the program in question [7][8] use Lua.
-spc
[1] Most log rotation programs will rename the current file to a new
name, then cause the program that created the file to re-create a
new one [2]. Any program doing a "tail -f"-like operation will
never see this [3].
The program *might* be able to detect a log truncation, but I would
have to check the Unix semantics to see what happens. [4]
[2] Except, it seems, for Solaris. There, the log rotation program will
rename the original file, create a 0-byte version of the new file,
then cause the program that wants to log to that file to reopen the
newly created 0-byte file. I learned this the hard way.
[3] Except for programs that use inotify() on the directory (I
think---I've never worked with inotify()).
[4] For instance, program A can open a file, then program B can delete
the file, but A can still read the file, since Unix doesn't actually
remove the file until all open instances of said file are closed.
[5] Technically, it's a multicast address, which only works for the UDP
based syslog protocol (the universal default). Before forwarding to
a multicast address, be aware that any workstation on the local
segment can tap into this stream of data. In my case, that's
exactly what I wanted.
[6] https://github.com/spc476/syslogintr
[7] https://github.com/spc476/syslogintr/blob/master/realtime.lua
[8] Color coded too, based on the error level.