homepage

I'm using SyslogHandler from logging.handlers to send syslog messages to a Fluentd input (https://github.com/fluent/fluentd/blob/master/lib/fluent/plugin/in_syslog.rb), both in TCP and UDP. UDP works fine, but TCP does not work. 
The "problem" is that the handler is not ending messages with a new line '\n' character (I realized that using tcpdump). I've temporarily added this to line 855 of handlers.py: 
 msg = prio + msg + '\n' 
And now is working. 
Now I'm confused because maybe this is not an issue but a problem of Fluentd. For the time, I will create a new class extending SyslogHandler and override the emit function.
Thank you for your time.

After reading the RFC5424 it seems that there is no such "new line message delimiter":
--------------------------------
4.3.1. Message Length
 The message length is the octet count of the SYSLOG-MSG in the
 SYSLOG-FRAME. A transport receiver MUST use the message length to delimit a syslog message
--------------------------------
So I think it must be a Fluentd error. This is what caused my confusion:
From in_syslog.rb (https://github.com/athenahealth/fluent-plugin-newsyslog/blob/master/lib/fluent/plugin/in_newsyslog.rb):
--------------------------------
# syslog family add "\n" to each message and this seems only way to split messages in tcp stream
Coolio::TCPServer.new(@bind, @port, SocketUtil::TcpHandler, log, "\n", callback)
--------------------------------

> So I think it must be a Fluentd error.
So I'll close this.

Sorry to bother you again, but I've tested this not only with Fluentd, but with a RSYSLOG server and it does not work with TCP except if you manually add the trailer LF character. Other than that, UDP default transport protocol has no issues and works fine with both systems. Here's my simple code:
-------
sHandler = logging.handlers.SysLogHandler(address=(address[0], address[1]), socktype = socket.SOCK_STREAM)
sHandler.setFormatter(logging.Formatter(fmt=MSG_SYSLOG_FORMAT, datefmt=DATE_FMT))
self.addHandler(sHandler)
-------
After reading RFC 6587 I think the SyslogHandler class should implement at least one of the framing mechanisms proposed by this RFC, meant for TCP transmission:
- Octet counting
- Trailer character (e.g. LF)
Besides, I've being checking out the library "pyloggr" (https://github.com/stephane-martin/pyloggr) and they are implementing both mechanisms. As for SyslogHandler, it will be as simple as adding another field to the class constructor (use_delimiter?) and to add these lines to the emit code (it works):
-------
if (self.use_delimiter):
 msg = msg + '\n'
else:
 msg = str(len(msg)) + ' ' + msg # default behavior
-------
Thank you again