messages#

Functions#

setup_queue_handler(queue[, logger_name])

Helper function to register a queue handler with a multiprocessing.Queue to collect all messages during a multi-process execution environment.

collect_and_clean_queue_handler(queue[, handler, ...])

Helper function to clean up registered queue handler and optionally collect all messages collected in the queue.

handle_records(records, logger)

Helper function to deal with collected logging records.

Module Contents#

setup_queue_handler(queue, logger_name='multiprocessing_logger')#

Helper function to register a queue handler with a multiprocessing.Queue to collect all messages during a multi-process execution environment.

Returns the queue (which is created if not provided), the QueueHandler and the Logger object specified. Can be used for cleanup in collect_and_clean_queue_handler().

Parameters:

  • queue (multiprocessing.Queue | None) – Optionally the queue to use for the message collection. Will be created if not provided.

  • logger_name (str, optional) – The name of the logger to use for the collection. Defaults to ‘multiprocessing_logger’.

Returns:

The tuple of Queue, QueueHandler and Logger that has been registered with each other. Finally the original set of handlers registered with the logger that should be restored afterwards

Return type:

tuple[multiprocessing.Queue, logging.handlers.QueueHandler, logging.Logger, list[logging.Handler]]

collect_and_clean_queue_handler(queue, handler=None, logger=None, original_handlers=None, doCollect=False)#

Helper function to clean up registered queue handler and optionally collect all messages collected in the queue.

Parameters:

  • queue (multiprocessing.Queue) – The queue to retrieve log records from.

  • handler (logging.handlers.QueueHandler | None, optional) – The queue handler previously registered that should now be removed. Defaults to None.

  • logger (logging.Logger | None, optional) – The logger that the queue handler has been registered to and should be removed from now. Defaults to None.

  • original_handlers (list[logging.Handler] | None, optional) – List of original handlers to restore to the logger. If not provided or empty, none will be applied. Defaults to None.

  • doCollect (bool, optional) – Flag to make the function collect all messages in the queue and return them as the result list. Defaults to False.

Returns:

Either the list of collected LogRecords if doCollect=True or None otherwise.

Return type:

list[logging.LogRecord] | None

handle_records(records, logger)#

Helper function to deal with collected logging records.

If the debug level of the provided logger (which defaults to the root logger) is debug, all messages will be passed on. Otherwise, all records of level warning or below will be compressed, i.e. each message string of each log level only presented once.

Parameters:

  • records (list[logging.LogRecord]) – The list of collected log records to be handled and compressed.

  • logger (logging.Logger | None) – The logger to handle the collected records with. Defaults to the root logger if not provided.