详解Python logging调用Logger.info方法的处理过程


Posted in Python onFebruary 12, 2019

本次分析一下Logger.info的流程

1. Logger.info源码:

def info(self, msg, *args, **kwargs):
  """
  Log 'msg % args' with severity 'INFO'.

  To pass exception information, use the keyword argument exc_info with
  a true value, e.g.

  logger.info("Houston, we have a %s", "interesting problem", exc_info=1)
  """
  if self.isEnabledFor(INFO):
   self._log(INFO, msg, args, **kwargs)

注释中反应了可以通过 msg和不定参数args来进行日志的格式化。
真实的调用为:_log方法:

2. Logger._log方法:

def _log(self, level, msg, args, exc_info=None, extra=None, stack_info=False):
  """
  Low-level logging routine which creates a LogRecord and then calls
  all the handlers of this logger to handle the record.
  """
  sinfo = None
  if _srcfile:
   #IronPython doesn't track Python frames, so findCaller raises an
   #exception on some versions of IronPython. We trap it here so that
   #IronPython can use logging.
   try:
    fn, lno, func, sinfo = self.findCaller(stack_info)
   except ValueError: # pragma: no cover
    fn, lno, func = "(unknown file)", 0, "(unknown function)"
  else: # pragma: no cover
   fn, lno, func = "(unknown file)", 0, "(unknown function)"
  if exc_info:
   if isinstance(exc_info, BaseException):
    exc_info = (type(exc_info), exc_info, exc_info.__traceback__)
   elif not isinstance(exc_info, tuple):
    exc_info = sys.exc_info()
  record = self.makeRecord(self.name, level, fn, lno, msg, args,
         exc_info, func, extra, sinfo)
  self.handle(record)

最后两行:

生成日志记录:

record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info, func, extra, sinfo)

处理日志记录

self.handle(record)

2 生成日志记录:

def makeRecord(self, name, level, fn, lno, msg, args, exc_info,
     func=None, extra=None, sinfo=None):
  """
  A factory method which can be overridden in subclasses to create
  specialized LogRecords.
  """
  rv = _logRecordFactory(name, level, fn, lno, msg, args, exc_info, func,
        sinfo)
  if extra is not None:
   for key in extra:
    if (key in ["message", "asctime"]) or (key in rv.__dict__):
     raise KeyError("Attempt to overwrite %r in LogRecord" % key)
    rv.__dict__[key] = extra[key]
  return rv

调用_logRecordFactory初始化一个日志记录实例,_logRecordFactory 其实就是LogRecord类,初始化时,可能包含logger的name, level、调用的函数、行号、日志字符串、模板参数、堆栈信息等。

再看extra信息,extra到底有何用?现在从代码中可以看到,只是更新到生成的日志记录实例的__dict__中去.猜测:肯定会在生成最终的日志字符串的时候会用到。继续往下看。

3 处理日志记录self.handle(record):

Logger继承自Filterer,

def handle(self, record):
  """
  Call the handlers for the specified record.

  This method is used for unpickled records received from a socket, as
  well as those created locally. Logger-level filtering is applied.
  """
  if (not self.disabled) and self.filter(record):
   self.callHandlers(record)

3.1 if语句中有一self.filter(record)的判断,看函数名,是来筛选是否要继续处理消息的,其核心源码如下:

def filter(self, record):
  """
  Determine if a record is loggable by consulting all the filters.

  The default is to allow the record to be logged; any filter can veto
  this and the record is then dropped. Returns a zero value if a record
  is to be dropped, else non-zero.

  .. versionchanged:: 3.2

   Allow filters to be just callables.
  """
  rv = True
  for f in self.filters:
   if hasattr(f, 'filter'):
    result = f.filter(record)
   else:
    result = f(record) # assume callable - will raise if not
   if not result:
    rv = False
    break
  return rv

可以看到, 如果在handler中的filter中如果有返回为False或空,则会屏蔽对应的record,返回True或部位空的值,则会将record放行。那么我们就可以自定义自己的filter。

3.2 让Logger中所有的handles去处理record:

def callHandlers(self, record):
  """
  Pass a record to all relevant handlers.

  Loop through all handlers for this logger and its parents in the
  logger hierarchy. If no handler was found, output a one-off error
  message to sys.stderr. Stop searching up the hierarchy whenever a
  logger with the "propagate" attribute set to zero is found - that
  will be the last logger whose handlers are called.
  """
  c = self
  found = 0
  while c:
   for hdlr in c.handlers:
    found = found + 1
    if record.levelno >= hdlr.level:
     hdlr.handle(record)
   if not c.propagate:
    c = None #break out
   else:
    c = c.parent
  if (found == 0):
   if lastResort:
    if record.levelno >= lastResort.level:
     lastResort.handle(record)
   elif raiseExceptions and not self.manager.emittedNoHandlerWarning:
    sys.stderr.write("No handlers could be found for logger"
         " \"%s\"\n" % self.name)
    self.manager.emittedNoHandlerWarning = True

代码中会去循环调用当前logger的所有handlers去处理record,for循环部分,之后,如果当前的logger的propagate的值为False或空,则不向logger的父logger传递,即向上传递。

4. Handler 中的 handler(record) 部分:

def handle(self, record):
  """
  Conditionally emit the specified logging record.

  Emission depends on filters which may have been added to the handler.
  Wrap the actual emission of the record with acquisition/release of
  the I/O thread lock. Returns whether the filter passed the record for
  emission.
  """
  rv = self.filter(record)
  if rv:
   self.acquire()
   try:
    self.emit(record)
   finally:
    self.release()
  return rv

可以看到, Handler在处理record时, 会去加锁,然后调用self.emit(record)方法去处理。

4.1 emit(record)

def emit(self, record):
  """
  Do whatever it takes to actually log the specified logging record.

  This version is intended to be implemented by subclasses and so
  raises a NotImplementedError.
  """
  raise NotImplementedError('emit must be implemented '
         'by Handler subclasses')

看到需要由子类去实现,以StreamHandler为例子:

def emit(self, record):
  """
  Emit a record.

  If a formatter is specified, it is used to format the record.
  The record is then written to the stream with a trailing newline. If
  exception information is present, it is formatted using
  traceback.print_exception and appended to the stream. If the stream
  has an 'encoding' attribute, it is used to determine how to do the
  output to the stream.
  """
  try:
   msg = self.format(record)
   stream = self.stream
   stream.write(msg)
   stream.write(self.terminator)
   self.flush()
  except Exception:
   self.handleError(record)

4.2 Handler.format(record):

def format(self, record):
  """
  Format the specified record.

  If a formatter is set, use it. Otherwise, use the default formatter
  for the module.
  """
  if self.formatter:
   fmt = self.formatter
  else:
   fmt = _defaultFormatter
  return fmt.format(record)

如果handler有自定义的formatter就用自定义的,如果没有则用默认的Formatter的实例, 初始化元源码为:

def __init__(self, fmt=None, datefmt=None, style='%'):
  """
  Initialize the formatter with specified format strings.

  Initialize the formatter either with the specified format string, or a
  default as described above. Allow for specialized date formatting with
  the optional datefmt argument (if omitted, you get the ISO8601 format).

  Use a style parameter of '%', '{' or '$' to specify that you want to
  use one of %-formatting, :meth:`str.format` (``{}``) formatting or
  :class:`string.Template` formatting in your format string.

  .. versionchanged:: 3.2
   Added the ``style`` parameter.
  """
  if style not in _STYLES:
   raise ValueError('Style must be one of: %s' % ','.join(
        _STYLES.keys()))
  self._style = _STYLES[style][0](fmt)
  self._fmt = self._style._fmt
  self.datefmt = datefmt

有三个参数:

  • fmt: 格式化模板
  • datefmt: 时间格式化参数
  • style: 日志格式化的样式。

style有三种:

_STYLES = {
 '%': (PercentStyle, BASIC_FORMAT),
 '{': (StrFormatStyle, '{levelname}:{name}:{message}'),
 '$': (StringTemplateStyle, '${levelname}:${name}:${message}'),

可以看出对应到:% 操作符的格式化, format方法的格式化以及Template的格式化。

Formatter的format方法源码为:

def format(self, record):
  """
  Format the specified record as text.
  The record's attribute dictionary is used as the operand to a
  string formatting operation which yields the returned string.
  Before formatting the dictionary, a couple of preparatory steps
  are carried out. The message attribute of the record is computed
  using LogRecord.getMessage(). If the formatting string uses the
  time (as determined by a call to usesTime(), formatTime() is
  called to format the event time. If there is exception information,
  it is formatted using formatException() and appended to the message.
  """
  record.message = record.getMessage()
  if self.usesTime():
   record.asctime = self.formatTime(record, self.datefmt)
  s = self.formatMessage(record)
  if record.exc_info:
   # Cache the traceback text to avoid converting it multiple times
   # (it's constant anyway)
   if not record.exc_text:
    record.exc_text = self.formatException(record.exc_info)
  if record.exc_text:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + record.exc_text
  if record.stack_info:
   if s[-1:] != "\n":
    s = s + "\n"
   s = s + self.formatStack(record.stack_info)

看到会调用record.getMessage(),这里仅仅是获取我们需要的日志信息。

之后会调用s = self.formatMessage(record):

def formatMessage(self, record):
  return self._style.format(record)

其实是调用了当前style的format方法,以%这一类型为例PercentStyle:

class PercentStyle(object):

 default_format = '%(message)s'
 asctime_format = '%(asctime)s'
 asctime_search = '%(asctime)'

 def __init__(self, fmt):
  self._fmt = fmt or self.default_format

 def usesTime(self):
  return self._fmt.find(self.asctime_search) >= 0

 def format(self, record):
  return self._fmt % record.__dict__

从其中的format方法可以看出,是针对record的__dict__属性中的所有参数进行格式化,这下,就清楚了之前的extra参数是干嘛用的了:可以在formatter中加入自己自定义的一些参数,如固定的用户信息等等。

之后,将最终的message flush到对应的Stream里面去就行了,就是整个流程:

详解Python logging调用Logger.info方法的处理过程

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python正则表达式教程之二:捕获篇
Mar 02 Python
python 接口_从协议到抽象基类详解
Aug 24 Python
简单实现python画圆功能
Jan 25 Python
python 显示数组全部元素的方法
Apr 19 Python
对Python3 pyc 文件的使用详解
Feb 16 Python
如何利用Python模拟GitHub登录详解
Jul 15 Python
Python爬虫 scrapy框架爬取某招聘网存入mongodb解析
Jul 31 Python
django 快速启动数据库客户端程序的方法示例
Aug 16 Python
在notepad++中实现直接运行python代码
Dec 18 Python
浅谈keras的深度模型训练过程及结果记录方式
Jan 24 Python
python 通过exifread读取照片信息
Dec 24 Python
python 遍历磁盘目录的三种方法
Apr 02 Python
Python numpy中矩阵的基本用法汇总
Feb 12 #Python
python读取csv和txt数据转换成向量的实例
Feb 12 #Python
python 读取文件并把矩阵转成numpy的两种方法
Feb 12 #Python
Python把对应格式的csv文件转换成字典类型存储脚本的方法
Feb 12 #Python
python 实现读取一个excel多个sheet表并合并的方法
Feb 12 #Python
python 利用pandas将arff文件转csv文件的方法
Feb 12 #Python
python将pandas datarame保存为txt文件的实例
Feb 12 #Python
You might like
把从SQL中取出的数据转化成XMl格式
2006/10/09 PHP
PHP5.3.1 不再支持ISAPI
2010/01/08 PHP
php selectradio和checkbox默认选择的实现方法详解
2013/06/29 PHP
Win下如何安装PHP的APC拓展
2013/08/07 PHP
thinkphp实现多语言功能(语言包)
2014/03/04 PHP
PHP防止刷新重复提交页面的示例代码
2015/11/11 PHP
基础的WordPress插件制作教程
2015/11/24 PHP
php实现session共享的实例方法
2019/09/19 PHP
toString()一个会自动调用的方法
2010/02/08 Javascript
jquery dialog键盘事件代码
2010/08/01 Javascript
关于firefox的ElementTraversal 接口 使用说明
2010/11/11 Javascript
Javascript事件热键兼容ie|firefox
2010/12/30 Javascript
Javascript this 的一些学习总结
2012/08/02 Javascript
jQuery渐变发光导航菜单的实例代码
2013/03/27 Javascript
一个支持任意尺寸的图片上下左右滑动效果
2014/08/24 Javascript
Javascript中的call()方法介绍
2015/03/15 Javascript
jQuery的Scrollify插件实现滑动到页面下一节点
2015/07/05 Javascript
JS版元素周期表实现方法
2015/08/05 Javascript
javascript图片切换综合实例(循环切换、顺序切换)
2016/01/13 Javascript
JS中的hasOwnProperty()、propertyIsEnumerable()和isPrototypeOf()
2016/08/11 Javascript
Vue.js组件使用开发实例教程
2016/11/01 Javascript
jQuery选取所有复选框被选中的值并用Ajax异步提交数据的实例
2017/08/04 jQuery
JS数组操作之增删改查的简单实现
2017/08/21 Javascript
jquery 获取索引值在一定范围的列表方法
2018/01/25 jQuery
jQuery实现左右滑动的toggle方法
2018/03/03 jQuery
Python每天必学之bytes字节
2016/01/28 Python
使用PyInstaller将Python程序文件转换为可执行程序文件
2016/07/08 Python
浅谈Python实现Apriori算法介绍
2017/12/20 Python
python替换字符串中的子串图文步骤
2019/06/19 Python
CSS3实现淘宝留白的方法
2020/06/05 HTML / CSS
HTML5混合开发二维码扫描以及调用本地摄像头
2017/12/27 HTML / CSS
解决HTML5手机端页面缩放的问题
2017/10/27 HTML / CSS
《沉香救母》教学反思
2014/04/19 职场文书
商场促销活动总结
2014/07/10 职场文书
2014年政工师工作总结
2014/12/18 职场文书
离婚承诺书格式范文
2015/05/04 职场文书