python的Tqdm模块的使用


Posted in Python onJanuary 10, 2018

Tqdm 是一个快速,可扩展的Python进度条,可以在 Python 长循环中添加一个进度提示信息,用户只需要封装任意的迭代器 tqdm(iterator)。

我的系统是window环境,首先安装python,接下来就是pip。

pip安装:

在python根目录下创建一个get-pip.py的文件,内容:

https://bootstrap.pypa.io/get-pip.py 

然后在CMD窗口进入python下面:

输出:

python -m pip install -U pip

由于Tqdm要求的pip版本是9.0所以需要手动安装pip9.0
http://pypi.python.org/pypi/pip

下载安装包9.0

然后解压进入,CMD窗口输入:python setup.py install

然后就可以安装Tqdm了,

pip install tqdm

安装最新的开发版的话

pip install -e git+https://github.com/tqdm/tqdm.git@master#egg=tqdm

最后看看怎么用呢?https://pypi.python.org/pypi/tqdm

基本用法:

from tqdm import tqdm 
for i in tqdm(range(10000)): 
   sleep(0.01)

当然除了tqdm,还有trange,使用方式完全相同

for i in trange(100): 
    sleep(0.1)

只要传入list都可以:

pbar = tqdm(["a", "b", "c", "d"]) 
for char in pbar: 
  pbar.set_description("Processing %s" % char)

也可以手动控制更新

with tqdm(total=100) as pbar: 
  for i in range(10): 
    pbar.update(10)

也可以这样:

pbar = tqdm(total=100) 
for i in range(10): 
  pbar.update(10) 
pbar.close()

在Shell的tqdm用法

统计所有python脚本的行数:

$ time find . -name '*.py' -exec cat \{} \; | wc -l 
857365 
 
real  0m3.458s 
user  0m0.274s 
sys   0m3.325s 
 
$ time find . -name '*.py' -exec cat \{} \; | tqdm | wc -l 
857366it [00:03, 246471.31it/s] 
857365 
 
real  0m3.585s 
user  0m0.862s 
sys   0m3.358s

使用参数:

$ find . -name '*.py' -exec cat \{} \; | 
  tqdm --unit loc --unit_scale --total 857366 >> /dev/null 
100%|???????????????????????????????????| 857K/857K [00:04<00:00, 246Kloc/s]

备份一个目录:

$ 7z a -bd -r backup.7z docs/ | grep Compressing | 
  tqdm --total $(find docs/ -type f | wc -l) --unit files >> backup.log 
100%|????????????????????????????????| 8014/8014 [01:37<00:00, 82.29files/s]

通过看示范的代码,我们能发现使用的核心是tqdm和trange这两个函数,从代码层面分析tqdm的功能,那首先是init.py

__all__ = ['tqdm', 'tqdm_gui', 'trange', 'tgrange', 'tqdm_pandas', 
      'tqdm_notebook', 'tnrange', 'main', 'TqdmKeyError', 'TqdmTypeError', 
      '__version__']

跟踪到_tqdm.py,能看到tqdm类的声明,首先是初始化

def __init__(self, iterable=None, desc=None, total=None, leave=True, 
         file=sys.stderr, ncols=None, mininterval=0.1, 
         maxinterval=10.0, miniters=None, ascii=None, disable=False, 
         unit='it', unit_scale=False, dynamic_ncols=False, 
         smoothing=0.3, bar_format=None, initial=0, position=None, 
         gui=False, **kwargs):
Parameters 
 
iterable : iterable, optional 
Iterable to decorate with a progressbar. 
可迭代的进度条。 
Leave blank to manually manage the updates. 
留空手动管理更新?? 
desc : str, optional 
Prefix for the progressbar. 
进度条的描述 
total : int, optional 
The number of expected iterations. If unspecified, 
len(iterable) is used if possible. As a last resort, only basic 
progress statistics are displayed (no ETA, no progressbar). 
If gui is True and this parameter needs subsequent updating, 
specify an initial arbitrary large positive integer, 
e.g. int(9e9). 
预期的迭代数目,默认为None,则尽可能的迭代下去,如果gui设置为True,这里则需要后续的更新,将需要指定为一个初始随意值较大的正整数,例如int(9e9) 
leave : bool, optional 
If [default: True], keeps all traces of the progressbar 
upon termination of iteration. 
保留进度条存在的痕迹,简单来说就是会把进度条的最终形态保留下来,默认为True 
file : io.TextIOWrapper or io.StringIO, optional 
Specifies where to output the progress messages 
[default: sys.stderr]. Uses file.write(str) and file.flush() 
methods. 
指定消息的输出 
ncols : int, optional 
The width of the entire output message. If specified, 
dynamically resizes the progressbar to stay within this bound. 
If unspecified, attempts to use environment width. The 
fallback is a meter width of 10 and no limit for the counter and 
statistics. If 0, will not print any meter (only stats). 
整个输出消息的宽度。如果指定,动态调整的进度停留在这个边界。如果未指定,尝试使用环境的宽度。如果为0,将不打印任何东西(只统计)。 
mininterval : float, optional 
Minimum progress update interval, in seconds [default: 0.1]. 
最小进度更新间隔,以秒为单位(默认值:0.1)。 
maxinterval : float, optional 
Maximum progress update interval, in seconds [default: 10.0]. 
最大进度更新间隔,以秒为单位(默认值:10)。 
miniters : int, optional 
Minimum progress update interval, in iterations. 
If specified, will set mininterval to 0. 
最小进度更新周期 
ascii : bool, optional 
If unspecified or False, use unicode (smooth blocks) to fill 
the meter. The fallback is to use ASCII characters 1-9 #. 
如果不设置,默认为unicode编码 
disable : bool, optional 
Whether to disable the entire progressbar wrapper 
[default: False]. 
是否禁用整个进度条包装(如果为True,进度条不显示) 
unit : str, optional 
String that will be used to define the unit of each iteration 
[default: it]. 
将被用来定义每个单元的字符串??? 
unit_scale : bool, optional 
If set, the number of iterations will be reduced/scaled 
automatically and a metric prefix following the 
International System of Units standard will be added 
(kilo, mega, etc.) [default: False]. 
如果设置,迭代的次数会自动按照十、百、千来添加前缀,默认为false 
dynamic_ncols : bool, optional 
If set, constantly alters ncols to the environment (allowing 
for window resizes) [default: False]. 
不断改变ncols环境,允许调整窗口大小 
smoothing : float, optional 
Exponential moving average smoothing factor for speed estimates 
(ignored in GUI mode). Ranges from 0 (average speed) to 1 
(current/instantaneous speed) [default: 0.3]. 
 
bar_format : str, optional 
Specify a custom bar string formatting. May impact performance. 
If unspecified, will use ‘{l_bar}{bar}{r_bar}', where l_bar is 
‘{desc}{percentage:3.0f}%|' and r_bar is 
‘| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str}, {rate_fmt}]' 
Possible vars: bar, n, n_fmt, total, total_fmt, percentage, 
rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc. 
自定义栏字符串格式化…默认会使用{l_bar}{bar}{r_bar}的格式,格式同上 
 
initial : int, optional 
The initial counter value. Useful when restarting a progress 
bar [default: 0]. 
初始计数器值,默认为0 
position : int, optional 
Specify the line offset to print this bar (starting from 0) 
Automatic if unspecified. 
Useful to manage multiple bars at once (eg, from threads). 
指定偏移,这个功能在多个条中有用 
gui : bool, optional 
WARNING: internal parameter - do not use. 
Use tqdm_gui(…) instead. If set, will attempt to use 
matplotlib animations for a graphical output [default: False]. 
内部参数… 
Returns 
out : decorated iterator. 
返回为一个迭代器

其实不用分析更多代码,多看看几个例子:(官网的例子)

7zx.py压缩进度条

# -*- coding: utf-8 -*- 
"""Usage: 
 7zx.py [--help | options] <zipfiles>... 
Options: 
 -h, --help   Print this help and exit 
 -v, --version Print version and exit 
 -c, --compressed    Use compressed (instead of uncompressed) file sizes 
 -s, --silent  Do not print one row per zip file 
 -y, --yes   Assume yes to all queries (for extraction) 
 -D=<level>, --debug=<level> 
         Print various types of debugging information. Choices: 
             CRITICAL|FATAL 
             ERROR 
             WARN(ING) 
             [default: INFO] 
             DEBUG 
             NOTSET 
 -d, --debug-trace   Print lots of debugging information (-D NOTSET) 
""" 
from __future__ import print_function 
from docopt import docopt 
import logging as log 
import subprocess 
import re 
from tqdm import tqdm 
import pty 
import os 
import io 
__author__ = "Casper da Costa-Luis <casper.dcl@physics.org>" 
__licence__ = "MPLv2.0" 
__version__ = "0.2.0" 
__license__ = __licence__ 
 
 
RE_SCN = re.compile("([0-9]+)\s+([0-9]+)\s+(.*)$", flags=re.M) 
 
 
def main(): 
  args = docopt(__doc__, version=__version__) 
  if args.pop('--debug-trace', False): 
    args['--debug'] = "NOTSET" 
  log.basicConfig(level=getattr(log, args['--debug'], log.INFO), 
          format='%(levelname)s: %(message)s') 
  log.debug(args) 
 
  # Get compressed sizes 
  zips = {} 
  for fn in args['<zipfiles>']: 
    info = subprocess.check_output(["7z", "l", fn]).strip() 
    finfo = RE_SCN.findall(info) 
 
    # builtin test: last line should be total sizes 
    log.debug(finfo) 
    totals = map(int, finfo[-1][:2]) 
    # log.debug(totals) 
    for s in range(2): 
      assert(sum(map(int, (inf[s] for inf in finfo[:-1]))) == totals[s]) 
    fcomp = dict((n, int(c if args['--compressed'] else u)) 
           for (u, c, n) in finfo[:-1]) 
    # log.debug(fcomp) 
    # zips : {'zipname' : {'filename' : int(size)}} 
    zips[fn] = fcomp 
 
  # Extract 
  cmd7zx = ["7z", "x", "-bd"] 
  if args['--yes']: 
    cmd7zx += ["-y"] 
  log.info("Extracting from {:d} file(s)".format(len(zips))) 
  with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()), 
       unit="B", unit_scale=True) as tall: 
    for fn, fcomp in zips.items(): 
      md, sd = pty.openpty() 
      ex = subprocess.Popen(cmd7zx + [fn], 
                 bufsize=1, 
                 stdout=md, # subprocess.PIPE, 
                 stderr=subprocess.STDOUT) 
      os.close(sd) 
      with io.open(md, mode="rU", buffering=1) as m: 
        with tqdm(total=sum(fcomp.values()), disable=len(zips) < 2, 
             leave=False, unit="B", unit_scale=True) as t: 
          while True: 
            try: 
              l_raw = m.readline() 
            except IOError: 
              break 
            l = l_raw.strip() 
            if l.startswith("Extracting"): 
              exname = l.lstrip("Extracting").lstrip() 
              s = fcomp.get(exname, 0) # 0 is likely folders 
              t.update(s) 
              tall.update(s) 
            elif l: 
              if not any(l.startswith(i) for i in 
                    ("7-Zip ", 
                    "p7zip Version ", 
                    "Everything is Ok", 
                    "Folders: ", 
                    "Files: ", 
                    "Size: ", 
                    "Compressed: ")): 
                if l.startswith("Processing archive: "): 
                  if not args['--silent']: 
                    t.write(t.format_interval( 
                      t.start_t - tall.start_t) + ' ' + 
                      l.lstrip("Processing archive: ")) 
                else: 
                  t.write(l) 
      ex.wait() 
 
 
main.__doc__ = __doc__ 
 
 
if __name__ == "__main__": 
  main()

tqdm_wget.py

"""An example of wrapping manual tqdm updates for urllib reporthook. 
# urllib.urlretrieve documentation 
> If present, the hook function will be called once 
> on establishment of the network connection and once after each block read 
> thereafter. The hook will be passed three arguments; a count of blocks 
> transferred so far, a block size in bytes, and the total size of the file. 
Usage: 
  tqdm_wget.py [options] 
Options: 
-h, --help 
  Print this help message and exit 
-u URL, --url URL : string, optional 
  The url to fetch. 
  [default: http://www.doc.ic.ac.uk/~cod11/matryoshka.zip] 
-o FILE, --output FILE : string, optional 
  The local file path in which to save the url [default: /dev/null]. 
""" 
 
import urllib 
from tqdm import tqdm 
from docopt import docopt 
 
 
def my_hook(t): 
  """ 
  Wraps tqdm instance. Don't forget to close() or __exit__() 
  the tqdm instance once you're done with it (easiest using `with` syntax). 
  Example 
  ------- 
  >>> with tqdm(...) as t: 
  ...   reporthook = my_hook(t) 
  ...   urllib.urlretrieve(..., reporthook=reporthook) 
  """ 
  last_b = [0] 
 
  def inner(b=1, bsize=1, tsize=None): 
    """ 
    b : int, optional 
      Number of blocks just transferred [default: 1]. 
    bsize : int, optional 
      Size of each block (in tqdm units) [default: 1]. 
    tsize : int, optional 
      Total size (in tqdm units). If [default: None] remains unchanged. 
    """ 
    if tsize is not None: 
      t.total = tsize 
    t.update((b - last_b[0]) * bsize) 
    last_b[0] = b 
  return inner 
 
 
opts = docopt(__doc__) 
 
eg_link = opts['--url'] 
eg_file = eg_link.replace('/', ' ').split()[-1] 
with tqdm(unit='B', unit_scale=True, leave=True, miniters=1, 
     desc=eg_file) as t: # all optional kwargs 
  urllib.urlretrieve(eg_link, filename=opts['--output'], 
            reporthook=my_hook(t), data=None)

examples.py

""" 
# Simple tqdm examples and profiling 
# Benchmark 
for i in _range(int(1e8)): 
  pass 
# Basic demo 
import tqdm 
for i in tqdm.trange(int(1e8)): 
  pass 
# Some decorations 
import tqdm 
for i in tqdm.trange(int(1e8), miniters=int(1e6), ascii=True, 
           desc="cool", dynamic_ncols=True): 
  pass 
# Nested bars 
from tqdm import trange 
for i in trange(10): 
  for j in trange(int(1e7), leave=False, unit_scale=True): 
    pass 
# Experimental GUI demo 
import tqdm 
for i in tqdm.tgrange(int(1e8)): 
  pass 
# Comparison to https://code.google.com/p/python-progressbar/ 
try: 
  from progressbar.progressbar import ProgressBar 
except ImportError: 
  pass 
else: 
  for i in ProgressBar()(_range(int(1e8))): 
    pass 
# Dynamic miniters benchmark 
from tqdm import trange 
for i in trange(int(1e8), miniters=None, mininterval=0.1, smoothing=0): 
  pass 
# Fixed miniters benchmark 
from tqdm import trange 
for i in trange(int(1e8), miniters=4500000, mininterval=0.1, smoothing=0): 
  pass 
""" 
 
from time import sleep 
from timeit import timeit 
import re 
 
# Simple demo 
from tqdm import trange 
for i in trange(16, leave=True): 
  sleep(0.1) 
 
# Profiling/overhead tests 
stmts = filter(None, re.split(r'\n\s*#.*?\n', __doc__)) 
for s in stmts: 
  print(s.replace('import tqdm\n', '')) 
  print(timeit(stmt='try:\n\t_range = xrange' 
           '\nexcept:\n\t_range = range\n' + s, number=1), 
     'seconds')

pandas_progress_apply.py

import pandas as pd 
import numpy as np 
from tqdm import tqdm 
 
df = pd.DataFrame(np.random.randint(0, 100, (100000, 6))) 
 
# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm` 
# (can use `tqdm_gui`, `tqdm_notebook`, optional kwargs, etc.) 
tqdm.pandas(desc="my bar!") 
 
# Now you can use `progress_apply` instead of `apply` 
# and `progress_map` instead of `map` 
df.progress_apply(lambda x: x**2) 
# can also groupby: 
# df.groupby(0).progress_apply(lambda x: x**2) 
 
 
# -- Source code for `tqdm_pandas` (really simple!) 
# def tqdm_pandas(t): 
#  from pandas.core.frame import DataFrame 
#  def inner(df, func, *args, **kwargs): 
#    t.total = groups.size // len(groups) 
#    def wrapper(*args, **kwargs): 
#      t.update(1) 
#      return func(*args, **kwargs) 
#    result = df.apply(wrapper, *args, **kwargs) 
#    t.close() 
#    return result 
#  DataFrame.progress_apply = inner

引用tqdm并非强制作为依赖:

include_no_requirements.py

# How to import tqdm without enforcing it as a dependency 
try: 
  from tqdm import tqdm 
except ImportError: 
  def tqdm(*args, **kwargs): 
    if args: 
      return args[0] 
    return kwargs.get('iterable', None)

参考:
https://github.com/tqdm/tqdm/tree/master/examples
https://pypi.python.org/pypi/tqdm
https://github.com/tqdm/tqdm

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
详解Python中find()方法的使用
May 18 Python
Python使用os模块和fileinput模块来操作文件目录
Jan 19 Python
Python实现调用另一个路径下py文件中的函数方法总结
Jun 07 Python
win8下python3.4安装和环境配置图文教程
Jul 31 Python
在python里协程使用同步锁Lock的实例
Feb 19 Python
pandas数据筛选和csv操作的实现方法
Jul 02 Python
PIL对上传到Django的图片进行处理并保存的实例
Aug 07 Python
Python如何获取Win7,Win10系统缩放大小
Jan 10 Python
python GUI库图形界面开发之PyQt5多行文本框控件QTextEdit详细使用方法实例
Feb 28 Python
在Keras中利用np.random.shuffle()打乱数据集实例
Jun 15 Python
Python机器学习工具scikit-learn的使用笔记
Jan 28 Python
Python基础数据类型tuple元组的概念与用法
Aug 02 Python
python3.6 实现AES加密的示例(pyCryptodome)
Jan 10 #Python
Python设计模式之MVC模式简单示例
Jan 10 #Python
Python设计模式之命令模式简单示例
Jan 10 #Python
Python爬虫实例_利用百度地图API批量获取城市所有的POI点
Jan 10 #Python
Python之多线程爬虫抓取网页图片的示例代码
Jan 10 #Python
Python设计模式之观察者模式简单示例
Jan 10 #Python
Python爬虫实例_城市公交网络站点数据的爬取方法
Jan 10 #Python
You might like
深入理解php printf() 输出格式化的字符串
2016/05/23 PHP
PHP页面输出搜索后跳转下一页的处理方法
2016/09/30 PHP
php json中文编码为null的解决办法
2016/12/14 PHP
PHP使用数组实现矩阵数学运算的方法示例
2017/05/29 PHP
JavaScript.The.Good.Parts阅读笔记(一)假值与===运算符
2010/11/16 Javascript
基于JQuery实现CheckBox全选全不选
2011/06/27 Javascript
JavaScript面向对象设计二 构造函数模式
2011/12/20 Javascript
浅析js封装和作用域
2013/07/09 Javascript
关于js中for in的缺陷浅析
2013/12/02 Javascript
Javascript学习笔记之 对象篇(一) : 对象的使用和属性
2014/06/24 Javascript
jquery实现鼠标悬浮停止轮播特效
2020/08/20 Javascript
微信小程序 教程之列表渲染
2016/10/18 Javascript
AngularJS用户选择器指令实例分析
2016/11/04 Javascript
利用Chrome DevTools直接调试Node.js和JavaScript的方法详解(并行)
2017/02/16 Javascript
利用Angular+Angular-Ui实现分页(代码加简单)
2017/03/10 Javascript
JavaScript实现分页效果
2017/03/28 Javascript
js,jq,css多方面实现简易下拉菜单功能
2017/05/13 Javascript
jquery实现简单实用的轮播器
2017/05/23 jQuery
微信小程序实现点击返回顶层的方法
2017/07/12 Javascript
详解使用nvm管理多版本node的方法
2017/08/30 Javascript
详解js中的几种常用设计模式
2020/07/16 Javascript
Vue仿百度搜索功能
2020/12/28 Vue.js
Python中的pass语句使用方法讲解
2015/05/14 Python
Python抽象类的新写法
2015/06/18 Python
Python实现对象转换为xml的方法示例
2017/06/08 Python
python连接mongodb密码认证实例
2018/10/16 Python
python打包exe开机自动启动的实例(windows)
2019/06/28 Python
深入了解Python iter() 方法的用法
2019/07/11 Python
Python学习笔记之Django创建第一个数据库模型的方法
2019/08/07 Python
numpy ndarray 取出满足特定条件的某些行实例
2019/12/05 Python
利用Python的folium包绘制城市道路图的实现示例
2020/08/24 Python
Python读取ini配置文件传参的简单示例
2021/01/05 Python
美国玛丽莎收藏奢华时尚商店:Marissa Collections
2016/11/21 全球购物
数据库测试通常都包括哪些方面
2015/11/30 面试题
Python数据可视化之绘制柱状图和条形图
2021/05/25 Python
V Rising 服务器搭建图文教程
2022/06/16 Servers