I am using this tool to get metrics on the appliance.
The script is using sockets to send syslog message, and I'm attempting to get it to send the message using native syslog functions.
I added the below code along but I cannot seem to get this working.
def sendSyslog2(jsonObj):
if (logHostAvailable["syslog"]==True):
logging.info("SYSLOG REQUEST: "+json.dumps(jsonObj))
try:
syslog.openlog(facility=syslog.LOG_LOCAL5)
syslog.syslog(syslog.LOG_INFO, json.dumps(jsonObj))
except:
logging.error("syslog failed")
Even using test scripts is failing. I'm not well versed in python or programming but I can get by with some reference, any pointers in right direction appreciated.
From: https://stackoverflow.com/a/38929289/1033927
You can use a remote syslog server: rsyslog or the python package loggerglue implements the syslog protocol as decribed in rfc5424 and rfc5425. . When you use a port above 1024 you can run it as a non-root user.
Within the python logging module you have a SyslogHandler which also supports the syslog remote logging.
import logging
import logging.handlers
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.DEBUG)
handler = logging.handlers.SysLogHandler(address = ('127.0.0.1',514))
my_logger.addHandler(handler)
my_logger.debug('this is debug')
my_logger.critical('this is critical')
Related
I try to connect to Pusher Websocket API using the following code :
https://github.com/nlsdfnbch/Pysher/
import pysher
# Add a logging handler so we can see the raw communication data
import logging
root = logging.getLogger()
root.setLevel(logging.INFO)
ch = logging.StreamHandler(sys.stdout)
root.addHandler(ch)
pusher = pysher.Pusher('de504dc5763aeef9ff52')
# We can't subscribe until we've connected, so we use a callback handler
# to subscribe when able
def connect_handler(data):
channel = pusher.subscribe('live_trades')
channel.bind('trade', callback)
pusher.connection.bind('pusher:connection_established', connect_handler)
pusher.connect()
while True:
# Do other things in the meantime here...
time.sleep(1)
instead of some valid response, i get this every few seconds :
Connection: Error - [WinError 10042] An unknown, invalid, or
unsupported option or level was specified in a getsockopt or
setsockopt call Connection: Connection closed Attempting to connect
again in 10 seconds.
what is the problem ?
I saw the same error using a different library that uses websockets. I can see from your description (and link) that Pysher uses websockets.
I found (yet another) websocket client for Python that reported an issue with websockets, specifically with Python 3.6.4: [https://github.com/websocket-client/websocket-client/issues/370]
It references the bug in Python tracker as well [https://bugs.python.org/issue32394]
Upgrading to Python 3.6.5 worked for me. Alternatively, they suggest that upgrading to Windows 10 1703+ should work too (just for completeness; I have not verified this).
I am using wsgi server to spawn the servers for my web application. I am having problem with information logging.
This is how I am running the app
from gevent import monkey; monkey.patch_all()
from logging.handlers import RotatingFileHandler
import logging
from app import app # this imports app
# create a file to store weblogs
log = open(ERROR_LOG_FILE, 'w'); log.seek(0); log.truncate();
log.write("Web Application Log\n"); log.close();
log_handler = RotatingFileHandler(ERROR_LOG_FILE, maxBytes =1000000, backupCount=1)
formatter = logging.Formatter(
"[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s"
)
log_handler.setFormatter(formatter)
app.logger.setLevel(logging.DEBUG)
app.logger.addHandler(log_handler)
# run the application
server= wsgi.WSGIServer(('0.0.0.0', 8080), app)
server.serve_forever()
However, on running the application it is not logging anything. I guess it must be because of WSGI server because app.logger works in the absence of WSGI. How can I log information when using WSGI?
According to the gevent uwsgi documentation you need to pass your log handler object to the WSGIServer object at creation:
log – If given, an object with a write method to which request (access) logs will be written. If not given, defaults to sys.stderr. You may pass None to disable request logging. You may use a wrapper, around e.g., logging, to support objects that don’t implement a write method. (If you pass a Logger instance, or in general something that provides a log method but not a write method, such a wrapper will automatically be created and it will be logged to at the INFO level.)
error_log – If given, a file-like object with write, writelines and flush methods to which error logs will be written. If not given, defaults to sys.stderr. You may pass None to disable error logging (not recommended). You may use a wrapper, around e.g., logging, to support objects that don’t implement the proper methods. This parameter will become the value for wsgi.errors in the WSGI environment (if not already set). (As with log, wrappers for Logger instances and the like will be created automatically and logged to at the ERROR level.)
so you should be able to do wsgi.WSGIServer(('0.0.0.0', 8080), app, log=app.logger)
You can log like this -
import logging
import logging.handlers as handlers
.
.
.
logger = logging.getLogger('MainProgram')
logger.setLevel(10)
logHandler = handlers.RotatingFileHandler(filename.log,maxBytes =1000000, backupCount=1)
logger.addHandler(logHandler)
logger.info("Logging configuration done")
.
.
.
# run the application
server= wsgi.WSGIServer(('0.0.0.0', 8080), app, log=logger)
server.serve_forever()
I have minimalistic Tornado application:
import tornado.ioloop
import tornado.web
class PingHandler(tornado.web.RequestHandler):
def get(self):
self.write("pong\n")
if __name__ == "__main__":
application = tornado.web.Application([ ("/ping", PingHandler), ])
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
Tornado keeps reporting error requests to stderr:
WARNING:tornado.access:404 GET / (127.0.0.1) 0.79ms
Question: It want to prevent it from logging error messages. How?
Tornado version 3.1; Python 2.6
Its clear that "someone" is initializing logging subsystem when we start Tornado. Here is the code from ioloop.py that reveals the mystery:
def start(self):
if not logging.getLogger().handlers:
# The IOLoop catches and logs exceptions, so it's
# important that log output be visible. However, python's
# default behavior for non-root loggers (prior to python
# 3.2) is to print an unhelpful "no handlers could be
# found" message rather than the actual log entry, so we
# must explicitly configure logging if we've made it this
# far without anything.
logging.basicConfig()
basicConfig is called and configures default stderr handler.
So to setup proper logging for tonado access, you need to:
Add a handler to tornado.access logger: logging.getLogger("tornado.access").addHandler(...)
Disable propagation for the above logger: logging.getLogger("tornado.access").propagate = False. Otherwise messages will arrive BOTH to your handler AND to stderr
The previous answer was correct, but a little incomplete. This will send everything to the NullHandler:
hn = logging.NullHandler()
hn.setLevel(logging.DEBUG)
logging.getLogger("tornado.access").addHandler(hn)
logging.getLogger("tornado.access").propagate = False
You could also quite simply (in one line) do:
logging.getLogger('tornado.access').disabled = True
I use logging to one file with different scripts of one program - such as cron tasks, twisted daemons (HttpServers with some data) etc.
If I use default Python logging in base class such as
import logging
import logging.handlers
....
self.__fname = open(logname, 'a')
logging.basicConfig(format=FORMAT, filename=logname, handler=logging.handlers.RotatingFileHandler)
self._log = logging.getLogger(self.__pname)
self._log.setLevel(loglevel)
logging.warn('%s %s \033[0m' % (self.__colors[colortype], msg))
then it's work normally, sending output of all scripts in one file, but some important part of default twisted log missing - such as info about http request/headers etc
else I use twisted logging such as
from twisted.python.logfile import DailyLogFile
from twisted.python import log
from twisted.application.service import Application
....
application = Application("foo")
log.startLogging(DailyLogFile.fromFullPath(logname))
print '%s %s \033[0m' % (self.__colors[colortype], msg)
then works with additional data, but some trouble with logging from different scripts exists - looks like cron tasks trouble appears. Looks like these cron tasks switch context of output and some part of logging output is missing and not restored
Of, course - cron tasks working without Twisted reactor, but using twisted logging.
What I should do with logging for log all data printed both Twisted/cron parts of app?
Thanks for any help!
I think the point is that you should not use DailyLogFile but use PythonLOggingObserver to redirect the log to standard lib log
from twisted.python import log
observer = log.PythonLoggingObserver()
observer.start()
log.msg('%s %s \033[0m' % (self.__colors[colortype], msg))
Also you might want to see the example in docs: http://twistedmatrix.com/documents/current/core/howto/logging.html#auto3
I have logging configured using logging.fileConfig(). I have a the root logger going to a handler that uses SysLogHandler('/dev/log', handlers.SysLogHandler.LOG_USER)
This all works perfectly well, and I see my log entries in /var/log/user.log
The question is how can I set the syslog ident string to something other than python? It appears the syslog module in the standard lib allows setting this when opening a log, but the logging handler doesn't offer this feature.
Would the solution be to subclass SysLogHandler and use the syslog library inside it's emit method? This is a unix only program, so using syslog directly doesn't pose a portability problem.
This is a bit old but new information should be recorded here so people don't feel the need to write their own syslog handler.
Since Python 3.3, the SysLogHandler has a class attribute of .ident precisely for this purpose; the default for it is ''.
Example:
import logging
from logging.handlers import SysLogHandler
h = SysLogHandler(address=('some.destination.com',514), facility=SysLogHandler.LOG_LOCAL6)
h.setFormatter(
logging.Formatter('%(name)s %(levelname)s %(message)s')
)
h.ident = 'conmon'
syslog = logging.getLogger('syslog')
syslog.setLevel(logging.DEBUG)
syslog.addHandler(h)
syslog.debug('foo syslog message')
Syslog implementations accepting RFC3164 messages should recognize first part of the message ("foo:" in the example) as TAG.
The MSG part has two fields known as the TAG field and the CONTENT
field. The value in the TAG field will be the name of the program or
process that generated the message.
Python code..
import logging
from logging.handlers import SysLogHandler
h = SysLogHandler(address='/dev/log')
h.setFormatter(logging.Formatter('foo: %(message)s'))
logging.getLogger().addHandler(h)
logging.error('bar')
..will send this into syslog socket
connect(3, {sa_family=AF_UNIX, sun_path="/dev/log"}, 10) = 0
sendto(3, "<11>foo: bar\0", 13, 0, NULL, 0) = 13
close(3)
Which in turn, produces this in systemd's journal.
Dec 13 14:48:20 laptop foo[1928]: bar
Journal message details:
{
..
"PRIORITY" : "3",
"SYSLOG_FACILITY" : "1",
"SYSLOG_IDENTIFIER" : "foo",
"MESSAGE" : "bar",
"_PID" : "1928",
}
It works with Py2.6, 2.7, 3.4, 3.5 and Systemd's syslog server. It may work with other syslog implementations as well (if they accept RFC3164) messages. This solution will probably break when python's SysLogHandler will default to newer RFC5424.
AFAIK, the ident string is an artifact of the syslog API, see this page. It's just using the C argv[0] which would of course be "python".
I'm surprised that you're getting this using SysLogHandler with a domain socket, as the message sent to syslog daemons across domain or TCP sockets is just a string with the priority in <angle brackets> followed by the formatted message and a NUL byte. There's no ident string specified by SysLogHandler, as it doesn't use the syslog API (which has some thread-safety issues in some versions, IIRC).
For Python 2.7, you could do something like this:
class MySysLogHandler(logging.handlers.SysLogHandler):
def __init__(self):
super(MySysLogHandler, self).__init__(address='/dev/log')
def emit(self, record):
priority = self.encodePriority(self.facility, self.mapPriority(record.levelname))
record.ident = "My[" + str(priority) + "]:"
super(MySysLogHandler, self).emit(record)
handler = MySysLogHandler()
handler.formatter = logging.Formatter(fmt="%(ident)s %(levelname)s: %(message)s")
logging.root.addHandler(handler)
logging.info("hello world")
This will produce in the syslog:
Sep 3 16:28:53 hostname My[14]: INFO: hello world