Python logging handlers with GELF (Graylog Extended Log Format) support
|travis| |coveralls| |pypi| |downloads|
.. |travis| image:: https://travis-ci.org/keeprocking/pygelf.svg?branch=master :target: https://travis-ci.org/keeprocking/pygelf .. |pypi| image:: https://badge.fury.io/py/pygelf.svg :target: https://pypi.python.org/pypi/pygelf .. |coveralls| image:: https://coveralls.io/repos/github/keeprocking/pygelf/badge.svg?branch=master :target: https://coveralls.io/github/keeprocking/pygelf?branch=master .. |downloads| image:: https://pepy.tech/badge/pygelf/month :target: https://pypi.python.org/pypi/pygelf
Python logging handlers with GELF (Graylog Extended Log Format) support.
Currently TCP, UDP, TLS (encrypted TCP) and HTTP logging handlers are supported.
.. code:: python
pip install pygelf
.. code:: python
from pygelf import GelfTcpHandler, GelfUdpHandler, GelfTlsHandler, GelfHttpHandler, GelfHttpsHandler
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
logger.addHandler(GelfTcpHandler(host='127.0.0.1', port=9401))
logger.addHandler(GelfUdpHandler(host='127.0.0.1', port=9402))
logger.addHandler(GelfTlsHandler(host='127.0.0.1', port=9403))
logger.addHandler(GelfHttpHandler(host='127.0.0.1', port=9404))
logger.addHandler(GelfHttpsHandler(host='127.0.0.1', port=9405))
logger.info('hello gelf')
According to the GELF spec, each message has the following mandatory fields:
.. code:: python
try:
1/0
except ZeroDivisionError as e:
logger.exception(e)
.. _syslog-compliant: https://en.wikipedia.org/wiki/Syslog#Severity_level
In debug mode (when handler was created with debug=True option) each message contains some extra fields (which are pretty self-explanatory):
Each handler has the following parameters:
str
with exception for several :code:datetime
objects): function that is called for objects that cannot be serialized to JSON natively by python. Default implementation is custom function that returns result of :code:isoformat()
method for :code:datetime.datetime
, :code:datetime.time
, :code:datetime.date
objects and result of :code:str(obj)
call for other objects (which is string representation of an object with fallback to :code:repr
)Also, there are some handler-specific parameters.
UDP:
.. _MTU: https://en.wikipedia.org/wiki/Maximum_transmission_unit
TLS:
HTTP:
HTTPS:
If you need to include some static fields into your logs, simply pass them to the handler constructor. Each additional field should start with underscore. You can't add field '_id'.
Example:
.. code:: python
handler = GelfUdpHandler(host='127.0.0.1', port=9402, _app_name='pygelf', _something=11)
logger.addHandler(handler)
If you need to include some dynamic fields into your logs, add them to record by using LoggingAdapter or logging.Filter and create handler with include_extra_fields set to True. All the non-trivial fields of the record will be sent to graylog2 with '_' added before the name
Example:
.. code:: python
class ContextFilter(logging.Filter):
def filter(self, record):
record.job_id = threading.local().process_id
return True
logger.addFilter(ContextFilter())
handler = GelfUdpHandler(host='127.0.0.1', port=9402, include_extra_fields=True)
logger.addHandler(handler)
If you need to include some fields from the environment into your logs, add them to record by using additional_env_fields
.
The following example will add an env
field to the logs, taking its value from the environment variable FLASK_ENV
.
.. code:: python
handler = GelfTcpHandler(host='127.0.0.1', port=9402, include_extra_fields=True, additional_env_fields={'env': 'FLASK_ENV'})
logger.addHandler(handler)
The following can also be used in defining logging from configuration files (yaml/ini):
.. code:: ini
[formatters]
keys=standard
[formatter_standard]
class=logging.Formatter
format=%(message)s
[handlers]
keys=graylog
[handler_graylog]
class=pygelf.GelfTcpHandler
formatter=standard
args=('127.0.0.1', '12201')
kwargs={'include_extra_fields': True, 'debug': True, 'additional_env_fields': {'env': 'FLASK_ENV'}}
[loggers]
keys=root
[logger_root]
level=WARN
handlers=graylog
To run tests, you'll need tox_. After installing, simply run it:
.. code::
tox
.. _tox: https://pypi.python.org/pypi/tox