详解Python logging调用Logger.info方法的处理过程


Posted in Python onFebruary 12, 2019

本次分析一下Logger.info的流程

1. Logger.info源码:

def info(self, msg, *args, **kwargs):
  """
  Log 'msg % args' with severity 'INFO'.

  To pass exception information, use the keyword argument exc_info with
  a true value, e.g.

  logger.info("Houston, we have a %s", "interesting problem", exc_info=1)
  """
  if self.isEnabledFor(INFO):
   self._log(INFO, msg, args, **kwargs)

注释中反应了可以通过 msg和不定参数args来进行日志的格式化。
真实的调用为:_log方法:

2. Logger._log方法:

def _log(self, level, msg, args, exc_info=None, extra=None, stack_info=False):
  """
  Low-level logging routine which creates a LogRecord and then calls
  all the handlers of this logger to handle the record.
  """
  sinfo = None
  if _srcfile:
   #IronPython doesn't track Python frames, so findCaller raises an
   #exception on some versions of IronPython. We trap it here so that
   #IronPython can use logging.
   try:
    fn, lno, func, sinfo = self.findCaller(stack_info)
   except ValueError: # pragma: no cover
    fn, lno, func = "(unknown file)", 0, "(unknown function)"
  else: # pragma: no cover
   fn, lno, func = "(unknown file)", 0, "(unknown function)"
  if exc_info:
   if isinstance(exc_info, BaseException):
    exc_info = (type(exc_info), exc_info, exc_info.__traceback__)
   elif not isinstance(exc_info, tuple):
    exc_info = sys.exc_info()
  record = self.makeRecord(self.name, level, fn, lno, msg, args,
         exc_info, func, extra, sinfo)
  self.handle(record)

最后两行:

生成日志记录:

record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info, func, extra, sinfo)

处理日志记录

self.handle(record)

2 生成日志记录:

def makeRecord(self, name, level, fn, lno, msg, args, exc_info,
     func=None, extra=None, sinfo=None):
  """
  A factory method which can be overridden in subclasses to create
  specialized LogRecords.
  """
  rv = _logRecordFactory(name, level, fn, lno, msg, args, exc_info, func,
        sinfo)
  if extra is not None:
   for key in extra:
    if (key in ["message", "asctime"]) or (key in rv.__dict__):
     raise KeyError("Attempt to overwrite %r in LogRecord" % key)
    rv.__dict__[key] = extra[key]
  return rv

调用_logRecordFactory初始化一个日志记录实例,_logRecordFactory 其实就是LogRecord类,初始化时,可能包含logger的name, level、调用的函数、行号、日志字符串、模板参数、堆栈信息等。

再看extra信息,extra到底有何用?现在从代码中可以看到,只是更新到生成的日志记录实例的__dict__中去.猜测:肯定会在生成最终的日志字符串的时候会用到。继续往下看。

3 处理日志记录self.handle(record):

Logger继承自Filterer,

def handle(self, record):
  """
  Call the handlers for the specified record.

  This method is used for unpickled records received from a socket, as
  well as those created locally. Logger-level filtering is applied.
  """
  if (not self.disabled) and self.filter(record):
   self.callHandlers(record)

3.1 if语句中有一self.filter(record)的判断,看函数名,是来筛选是否要继续处理消息的,其核心源码如下:

def filter(self, record):
  """
  Determine if a record is loggable by consulting all the filters.

  The default is to allow the record to be logged; any filter can veto
  this and the record is then dropped. Returns a zero value if a record
  is to be dropped, else non-zero.

  .. versionchanged:: 3.2

   Allow filters to be just callables.
  """
  rv = True
  for f in self.filters:
   if hasattr(f, 'filter'):
    result = f.filter(record)
   else:
    result = f(record) # assume callable - will raise if not
   if not result:
    rv = False
    break
  return rv

可以看到, 如果在handler中的filter中如果有返回为False或空,则会屏蔽对应的record,返回True或部位空的值,则会将record放行。那么我们就可以自定义自己的filter。

3.2 让Logger中所有的handles去处理record:

def callHandlers(self, record):
  """
  Pass a record to all relevant handlers.

  Loop through all handlers for this logger and its parents in the
  logger hierarchy. If no handler was found, output a one-off error
  message to sys.stderr. Stop searching up the hierarchy whenever a
  logger with the "propagate" attribute set to zero is found - that
  will be the last logger whose handlers are called.
  """
  c = self
  found = 0
  while c:
   for hdlr in c.handlers:
    found = found + 1
    if record.levelno >= hdlr.level:
     hdlr.handle(record)
   if not c.propagate:
    c = None #break out
   else:
    c = c.parent
  if (found == 0):
   if lastResort:
    if record.levelno >= lastResort.level:
     lastResort.handle(record)
   elif raiseExceptions and not self.manager.emittedNoHandlerWarning:
    sys.stderr.write("No handlers could be found for logger"
         " \"%s\"\n" % self.name)
    self.manager.emittedNoHandlerWarning = True

代码中会去循环调用当前logger的所有handlers去处理record,for循环部分,之后,如果当前的logger的propagate的值为False或空,则不向logger的父logger传递,即向上传递。

4. Handler 中的 handler(record) 部分:

def handle(self, record):
  """
  Conditionally emit the specified logging record.

  Emission depends on filters which may have been added to the handler.
  Wrap the actual emission of the record with acquisition/release of
  the I/O thread lock. Returns whether the filter passed the record for
  emission.
  """
  rv = self.filter(record)
  if rv:
   self.acquire()
   try:
    self.emit(record)
   finally:
    self.release()
  return rv

可以看到, Handler在处理record时, 会去加锁,然后调用self.emit(record)方法去处理。

4.1 emit(record)

def emit(self, record):
  """
  Do whatever it takes to actually log the specified logging record.

  This version is intended to be implemented by subclasses and so
  raises a NotImplementedError.
  """
  raise NotImplementedError('emit must be implemented '
         'by Handler subclasses')

看到需要由子类去实现,以StreamHandler为例子:

def emit(self, record):
  """
  Emit a record.

  If a formatter is specified, it is used to format the record.
  The record is then written to the stream with a trailing newline. If
  exception information is present, it is formatted using
  traceback.print_exception and appended to the stream. If the stream
  has an 'encoding' attribute, it is used to determine how to do the
  output to the stream.
  """
  try:
   msg = self.format(record)
   stream = self.stream
   stream.write(msg)
   stream.write(self.terminator)
   self.flush()
  except Exception:
   self.handleError(record)

4.2 Handler.format(record):

def format(self, record):
  """
  Format the specified record.

  If a formatter is set, use it. Otherwise, use the default formatter
  for the module.
  """
  if self.formatter:
   fmt = self.formatter
  else:
   fmt = _defaultFormatter
  return fmt.format(record)

如果handler有自定义的formatter就用自定义的,如果没有则用默认的Formatter的实例, 初始化元源码为:

def __init__(self, fmt=None, datefmt=None, style='%'):
  """
  Initialize the formatter with specified format strings.

  Initialize the formatter either with the specified format string, or a
  default as described above. Allow for specialized date formatting with
  the optional datefmt argument (if omitted, you get the ISO8601 format).

  Use a style parameter of '%', '{' or '$' to specify that you want to
  use one of %-formatting, :meth:`str.format` (``{}``) formatting or
  :class:`string.Template` formatting in your format string.

  .. versionchanged:: 3.2
   Added the ``style`` parameter.
  """
  if style not in _STYLES:
   raise ValueError('Style must be one of: %s' % ','.join(
        _STYLES.keys()))
  self._style = _STYLES[style][0](fmt)
  self._fmt = self._style._fmt
  self.datefmt = datefmt

有三个参数:

  • fmt: 格式化模板
  • datefmt: 时间格式化参数
  • style: 日志格式化的样式。

style有三种:

_STYLES = {
 '%': (PercentStyle, BASIC_FORMAT),
 '{': (StrFormatStyle, '{levelname}:{name}:{message}'),
 '$': (StringTemplateStyle, '${levelname}:${name}:${message}'),

可以看出对应到:% 操作符的格式化, format方法的格式化以及Template的格式化。

Formatter的format方法源码为:

def format(self, record):
  """
  Format the specified record as text.
  The record's attribute dictionary is used as the operand to a
  string formatting operation which yields the returned string.
  Before formatting the dictionary, a couple of preparatory steps
  are carried out. The message attribute of the record is computed
  using LogRecord.getMessage(). If the formatting string uses the
  time (as determined by a call to usesTime(), formatTime() is
  called to format the event time. If there is exception information,
  it is formatted using formatException() and appended to the message.
  """
  record.message = record.getMessage()
  if self.usesTime():
   record.asctime = self.formatTime(record, self.datefmt)
  s = self.formatMessage(record)
  if record.exc_info:
   # Cache the traceback text to avoid converting it multiple times
   # (it's constant anyway)
   if not record.exc_text:
    record.exc_text = self.formatException(record.exc_info)
  if record.exc_text:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + record.exc_text
  if record.stack_info:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + self.formatStack(record.stack_info)

看到会调用record.getMessage(),这里仅仅是获取我们需要的日志信息。

之后会调用s = self.formatMessage(record):

def formatMessage(self, record):
  return self._style.format(record)

其实是调用了当前style的format方法,以%这一类型为例PercentStyle:

class PercentStyle(object):

 default_format = '%(message)s'
 asctime_format = '%(asctime)s'
 asctime_search = '%(asctime)'

 def __init__(self, fmt):
  self._fmt = fmt or self.default_format

 def usesTime(self):
  return self._fmt.find(self.asctime_search) >= 0

 def format(self, record):
  return self._fmt % record.__dict__

从其中的format方法可以看出,是针对record的__dict__属性中的所有参数进行格式化,这下,就清楚了之前的extra参数是干嘛用的了:可以在formatter中加入自己自定义的一些参数,如固定的用户信息等等。

之后,将最终的message flush到对应的Stream里面去就行了,就是整个流程:

详解Python logging调用Logger.info方法的处理过程

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Django视图之ORM数据库查询操作API的实例
Oct 27 Python
浅谈用Python实现一个大数据搜索引擎
Nov 28 Python
django用户注册、登录、注销和用户扩展的示例
Mar 19 Python
Python多继承原理与用法示例
Aug 23 Python
10招!看骨灰级Pythoner玩转Python的方法
Apr 15 Python
pyqt 实现为长内容添加滑轮 scrollArea
Jun 19 Python
python 实现一个反向单位矩阵示例
Nov 29 Python
Django对接支付宝实现支付宝充值金币功能示例
Dec 17 Python
关于python中的xpath解析定位
Mar 06 Python
python 三种方法实现对Excel表格的读写
Nov 19 Python
PyTorch中的拷贝与就地操作详解
Dec 09 Python
python中pivot()函数基础知识点
Jan 03 Python
Python numpy中矩阵的基本用法汇总
Feb 12 #Python
python读取csv和txt数据转换成向量的实例
Feb 12 #Python
python 读取文件并把矩阵转成numpy的两种方法
Feb 12 #Python
Python把对应格式的csv文件转换成字典类型存储脚本的方法
Feb 12 #Python
python 实现读取一个excel多个sheet表并合并的方法
Feb 12 #Python
python 利用pandas将arff文件转csv文件的方法
Feb 12 #Python
python将pandas datarame保存为txt文件的实例
Feb 12 #Python
You might like
PHP+ACCESS 文章管理程序代码
2010/06/21 PHP
PHP IF ELSE简化/三元一次式的使用
2011/08/22 PHP
php实现的日历程序
2015/06/18 PHP
php求一个网段开始与结束IP地址的方法
2015/07/09 PHP
php实现的简单美国商品税计算函数
2015/07/13 PHP
一些javascript一些题目的解析
2010/12/25 Javascript
解析John Resig Simple JavaScript Inheritance代码
2012/12/03 Javascript
Web跨浏览器进程通信(Web跨域)
2013/04/17 Javascript
js调用打印机打印网页字体总是缩小一号的解决方法
2014/01/24 Javascript
jQuery中extend函数详解
2015/02/13 Javascript
Javascript中的作用域和上下文深入理解
2015/07/03 Javascript
原生javascript实现自动更新的时间日期
2016/02/12 Javascript
JavaScript String 对象常用方法总结
2016/04/28 Javascript
浅谈JS中逗号运算符的用法
2016/06/12 Javascript
浅谈angularJS中的事件
2016/07/12 Javascript
浅析BootStrap Treeview的简单使用
2016/10/12 Javascript
Extjs让combobox写起来简洁又漂亮
2017/01/05 Javascript
canvas实现探照灯效果
2017/02/07 Javascript
浅谈JavaScript作用域和闭包
2017/09/18 Javascript
细说webpack源码之compile流程-rules参数处理技巧(1)
2017/12/26 Javascript
浅谈Angular 的变化检测的方法
2018/03/01 Javascript
使用Angular CLI进行单元测试和E2E测试的方法
2018/03/24 Javascript
微信小程序实现两边小中间大的轮播效果的示例代码
2018/12/07 Javascript
使用VueCli3+TypeScript+Vuex一步步构建todoList的方法
2019/07/25 Javascript
no-vnc和node.js实现web远程桌面的完整步骤
2019/08/11 Javascript
深入分析JavaScript 事件循环(Event Loop)
2020/06/19 Javascript
Vue父组件监听子组件生命周期
2020/09/03 Javascript
numpy.delete删除一列或多列的方法
2018/04/03 Python
Win下PyInstaller 安装和使用教程
2019/12/25 Python
如何对python的字典进行排序
2020/06/19 Python
德国团购网站:Groupon德国
2018/03/13 全球购物
Java中的基本数据类型所占存储空间大小固定的吗
2012/02/15 面试题
档案室主任岗位职责
2014/02/12 职场文书
学生会主席竞聘书
2014/03/31 职场文书
乡镇一岗双责责任书
2015/01/29 职场文书
使用CSS实现小三角边框原理解析
2021/11/07 HTML / CSS