Handlers

This module contains handlers that can be used to forward logs from python logging module to a Redis database.

class rlh.handlers.RedisLogHandler(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, **redis_args)

Default class for Redis log handlers.

Attributes

redisredis.Redis

The Redis client.

batch_sizeint

The batch size, if this value is > 1, logs will be processed by batches.

log_bufferlist

The list containing the batched logs.

Methods

emit(record: logging.LogRecord)

This method is intended to be implemented by subclasses and so raises a NotImplementedError.

__init__(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, **redis_args) → None

Init RedisLogHandler

Parameters

redis_clientredis.Redis, optional

The Redis client to forward logs to, by default None.

batch_sizeint, optional

The batch size, if > 1 logs will be processed by batches, by default 1.

check_connbool, optional

Wether to check of not if the Redis is available with a ping, by default True.

Raises

TypeError

Raised if one of the aditional argument passed to Redis is invalid.

ConnectionError

Raised if the Redis DB is unavailable.

close()

Make sure to add all remaining logs in buffer to Redis before object is destroyed.

emit(record: logging.LogRecord) → None

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

class rlh.handlers.RedisPubSubLogHandler(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, channel_name: str = 'logs', fields: list = None, as_pkl: bool = False, **redis_args)

Handler used to publish logs to a Redis pub/sub channel.

Notes

Redis pub/sub: https://redis.io/docs/manual/pubsub/

Attributes

redisredis.Redis

The Redis client.

batch_sizeint

The batch size, if this value is > 1, logs will be processed by batches.

log_bufferlist

The list containing the batched logs.

channel_namestr

The name of the Redis pub/sub channel.

fieldslist(str)

The list of logs fields to forward.

as_pklbool

If true, the logs are written as pickle format in the message.

Methods

emit(record: logging.LogRecord)

Publish log to the Redis pub/sub channel.

__init__(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, channel_name: str = 'logs', fields: list = None, as_pkl: bool = False, **redis_args) → None

Init RedisPubSubLogHandler

Parameters

redis_clientredis.Redis, optional

The Redis client to forward logs to, by default None.

batch_sizeint, optional

The batch size, if > 1 logs will be processed by batches, by default 1.

check_connbool, optional

Wether to check of not if the Redis is available with a ping, by default True.

channel_namestr, optional

The name of the Redis pub/sub channel where the logs are pushed, by default “logs”.

fieldslist, optional

The list of logs fields to save, by default None.

as_pklbool, optional

Wether to save the log as its pickle format or not, by default False.

emit(record: logging.LogRecord)

Publish the log record in the Redis pub/sub channel.

Every time a log is emitted, an entry is published on the channel. This entry is encoded as JSON whose format depends on the handler attributes. If as_pkl is set to true, the records are saved as their pickle format with the key “pkl”. Otherwise we use the different fields as keys and their associated value in the record as the value (default fields are used if not specified).

Parameters

recordlogging.LogRecord

The log record to emit.

class rlh.handlers.RedisStreamLogHandler(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, stream_name: str = 'logs', maxlen: int = None, approximate: bool = True, fields: list = None, as_pkl: bool = False, as_json: bool = False, **redis_args)

Handler used to forward logs to a Redis stream.

Notes

Redis streams: https://redis.io/docs/data-types/streams/

Attributes

redisredis.Redis

The Redis client.

batch_sizeint

The batch size, if this value is > 1, logs will be processed by batches.

log_bufferlist

The list containing the batched logs.

stream_namestr

The name of the Redis stream.

fieldslist(str)

The list of logs fields to forward.

as_pklbool

If true, the logs are written as pickle format in the stream.

as_jsonbool

If true, the logs are written as JSON in the stream.

Methods

emit(record: logging.LogRecord)

Forward log to the Redis stream.

__init__(redis_client: redis.client.Redis = None, batch_size: int = 1, check_conn: bool = True, stream_name: str = 'logs', maxlen: int = None, approximate: bool = True, fields: list = None, as_pkl: bool = False, as_json: bool = False, **redis_args) → None

Init RedisStreamLogHandler

Parameters

redis_clientredis.Redis, optional

The Redis client to forward logs to, by default None.

batch_sizeint, optional

The batch size, if > 1 logs will be processed by batches, by default 1.

check_connbool, optional

Wether to check of not if the Redis is available with a ping, by default True.

stream_namestr, optional

The name of the Redis stream where the logs are stored, by default “logs”.

maxlenint, optional

The maximum lenght of the Redis stream, if 0 no limit applied, by default 0.

approximatebool, optional

If True, the Redis size won’t be exactly equals to maxlen, but will be at least maxlen, by default True.

fieldslist, optional

The list of logs fields to save, by default None.

as_pklbool, optional

Wether to save the log as its pickle format or not, by default False.

as_jsonbool, optional

Wether to save the log as JSON format or not, by default False.

Notes

More info about Redis caped stream: https://redis.io/docs/data-types/streams-tutorial/#capped-streams

emit(record: logging.LogRecord)

Write the log record in the Redis stream.

Every time a log is emitted, an entry is inserted in the stream. This entry is a dict whose format depends on the handler attributes.

If as_pkl is set to true, the records are saved as their pickle format with the key “pkl”. If as_json is set to true, the records are saved as their JSON representation with the key “json”. Otherwise we use the different fields as keys and their associated value in the record as the value.

If batch_size=n, the logs are emited by batches of size n.

Parameters

recordlogging.LogRecord

The log record to emit.