flask开启多线程的具体方法


Posted in Python onAugust 02, 2020

在我之前解释了flask如何支持多线程主要通过两个类来实现,LocalStack和Local,在Local中有两个属性,__storage__和__ident_func__,后者用来获取线程id,从而区分不同线程发来的请求

这次要说的是flask如何开启多线程

先从app.run()这个方法看起

def run(self, host=None, port=None, debug=None, **options):
 from werkzeug.serving import run_simple
 if host is None:
  host = '127.0.0.1'
 if port is None:
  server_name = self.config['SERVER_NAME']
  if server_name and ':' in server_name:
   port = int(server_name.rsplit(':', 1)[1])
  else:
   port = 5000
 if debug is not None:
  self.debug = bool(debug)
 options.setdefault('use_reloader', self.debug)
 options.setdefault('use_debugger', self.debug)
 try:
  run_simple(host, port, self, **options) #会进入这个函数
 finally:
  # reset the first request information if the development server
  # reset normally. This makes it possible to restart the server
  # without reloader and that stuff from an interactive shell.
  self._got_first_request = False

经过判断和设置后进入run_simple()这个函数,看下源码

def run_simple(hostname, port, application, use_reloader=False,

use_debugger=False, use_evalex=True,
   extra_files=None, reloader_interval=1,
   reloader_type='auto', threaded=False,
   processes=1, request_handler=None, static_files=None,
   passthrough_errors=False, ssl_context=None):
"""Start a WSGI application. Optional features include a reloader,
multithreading and fork support.
 
This function has a command-line interface too::
 
 python -m werkzeug.serving --help
 
.. versionadded:: 0.5
 `static_files` was added to simplify serving of static files as well
 as `passthrough_errors`.
 
.. versionadded:: 0.6
 support for SSL was added.
 
.. versionadded:: 0.8
 Added support for automatically loading a SSL context from certificate
 file and private key.
 
.. versionadded:: 0.9
 Added command-line interface.
 
.. versionadded:: 0.10
 Improved the reloader and added support for changing the backend
 through the `reloader_type` parameter. See :ref:`reloader`
 for more information.
 
:param hostname: The host for the application. eg: ``'localhost'``
:param port: The port for the server. eg: ``8080``
:param application: the WSGI application to execute
:param use_reloader: should the server automatically restart the python
      process if modules were changed?
:param use_debugger: should the werkzeug debugging system be used?
:param use_evalex: should the exception evaluation feature be enabled?
:param extra_files: a list of files the reloader should watch
     additionally to the modules. For example configuration
     files.
:param reloader_interval: the interval for the reloader in seconds.
:param reloader_type: the type of reloader to use. The default is
      auto detection. Valid values are ``'stat'`` and
      ``'watchdog'``. See :ref:`reloader` for more
      information.
:param threaded: should the process handle each request in a separate
     thread?
:param processes: if greater than 1 then handle each request in a new process
     up to this maximum number of concurrent processes.
:param request_handler: optional parameter that can be used to replace
      the default one. You can use this to replace it
      with a different
      :class:`~BaseHTTPServer.BaseHTTPRequestHandler`
      subclass.
:param static_files: a list or dict of paths for static files. This works
      exactly like :class:`SharedDataMiddleware`, it's actually
      just wrapping the application in that middleware before
      serving.
:param passthrough_errors: set this to `True` to disable the error catching.
       This means that the server will die on errors but
       it can be useful to hook debuggers in (pdb etc.)
:param ssl_context: an SSL context for the connection. Either an
     :class:`ssl.SSLContext`, a tuple in the form
     ``(cert_file, pkey_file)``, the string ``'adhoc'`` if
     the server should automatically create one, or ``None``
     to disable SSL (which is the default).
"""
if not isinstance(port, int):
 raise TypeError('port must be an integer')
if use_debugger:
 from werkzeug.debug import DebuggedApplication
 application = DebuggedApplication(application, use_evalex)
if static_files:
 from werkzeug.wsgi import SharedDataMiddleware
 application = SharedDataMiddleware(application, static_files)
def log_startup(sock):
 display_hostname = hostname not in ('', '*') and hostname or 'localhost'
 if ':' in display_hostname:
  display_hostname = '[%s]' % display_hostname
 quit_msg = '(Press CTRL+C to quit)'
 port = sock.getsockname()[1]
 _log('info', ' * Running on %s://%s:%d/ %s',
   ssl_context is None and 'http' or 'https',
   display_hostname, port, quit_msg)
def inner():
 try:
  fd = int(os.environ['WERKZEUG_SERVER_FD'])
 except (LookupError, ValueError):
  fd = None
 srv = make_server(hostname, port, application, threaded,
      processes, request_handler,
      passthrough_errors, ssl_context,
      fd=fd)
 if fd is None:
  log_startup(srv.socket)
 srv.serve_forever()
if use_reloader:
 # If we're not running already in the subprocess that is the
 # reloader we want to open up a socket early to make sure the
 # port is actually available.
 if os.environ.get('WERKZEUG_RUN_MAIN') != 'true':
  if port == 0 and not can_open_by_fd:
   raise ValueError('Cannot bind to a random port with enabled '
        'reloader if the Python interpreter does '
        'not support socket opening by fd.')
 
  # Create and destroy a socket so that any exceptions are
  # raised before we spawn a separate Python interpreter and
  # lose this ability.
  address_family = select_ip_version(hostname, port)
  s = socket.socket(address_family, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  s.bind(get_sockaddr(hostname, port, address_family))
  if hasattr(s, 'set_inheritable'):
   s.set_inheritable(True)
 
  # If we can open the socket by file descriptor, then we can just
  # reuse this one and our socket will survive the restarts.
  if can_open_by_fd:
   os.environ['WERKZEUG_SERVER_FD'] = str(s.fileno())
   s.listen(LISTEN_QUEUE)
   log_startup(s)
  else:
   s.close()
 
 # Do not use relative imports, otherwise "python -m werkzeug.serving"
 # breaks.
 from werkzeug._reloader import run_with_reloader
 run_with_reloader(inner, extra_files, reloader_interval,
      reloader_type)
else:
 inner() #默认会执行

经过判断和设置后进入run_simple()这个函数,看下源码

def run_simple(hostname, port, application, use_reloader=False,

use_debugger=False, use_evalex=True,
   extra_files=None, reloader_interval=1,
   reloader_type='auto', threaded=False,
   processes=1, request_handler=None, static_files=None,
   passthrough_errors=False, ssl_context=None):
"""Start a WSGI application. Optional features include a reloader,
multithreading and fork support.
 
This function has a command-line interface too::
 
 python -m werkzeug.serving --help
 
.. versionadded:: 0.5
 `static_files` was added to simplify serving of static files as well
 as `passthrough_errors`.
 
.. versionadded:: 0.6
 support for SSL was added.
 
.. versionadded:: 0.8
 Added support for automatically loading a SSL context from certificate
 file and private key.
 
.. versionadded:: 0.9
 Added command-line interface.
 
.. versionadded:: 0.10
 Improved the reloader and added support for changing the backend
 through the `reloader_type` parameter. See :ref:`reloader`
 for more information.
 
:param hostname: The host for the application. eg: ``'localhost'``
:param port: The port for the server. eg: ``8080``
:param application: the WSGI application to execute
:param use_reloader: should the server automatically restart the python
      process if modules were changed?
:param use_debugger: should the werkzeug debugging system be used?
:param use_evalex: should the exception evaluation feature be enabled?
:param extra_files: a list of files the reloader should watch
     additionally to the modules. For example configuration
     files.
:param reloader_interval: the interval for the reloader in seconds.
:param reloader_type: the type of reloader to use. The default is
      auto detection. Valid values are ``'stat'`` and
      ``'watchdog'``. See :ref:`reloader` for more
      information.
:param threaded: should the process handle each request in a separate
     thread?
:param processes: if greater than 1 then handle each request in a new process
     up to this maximum number of concurrent processes.
:param request_handler: optional parameter that can be used to replace
      the default one. You can use this to replace it
      with a different
      :class:`~BaseHTTPServer.BaseHTTPRequestHandler`
      subclass.
:param static_files: a list or dict of paths for static files. This works
      exactly like :class:`SharedDataMiddleware`, it's actually
      just wrapping the application in that middleware before
      serving.
:param passthrough_errors: set this to `True` to disable the error catching.
       This means that the server will die on errors but
       it can be useful to hook debuggers in (pdb etc.)
:param ssl_context: an SSL context for the connection. Either an
     :class:`ssl.SSLContext`, a tuple in the form
     ``(cert_file, pkey_file)``, the string ``'adhoc'`` if
     the server should automatically create one, or ``None``
     to disable SSL (which is the default).
"""
if not isinstance(port, int):
 raise TypeError('port must be an integer')
if use_debugger:
 from werkzeug.debug import DebuggedApplication
 application = DebuggedApplication(application, use_evalex)
if static_files:
 from werkzeug.wsgi import SharedDataMiddleware
 application = SharedDataMiddleware(application, static_files)
def log_startup(sock):
 display_hostname = hostname not in ('', '*') and hostname or 'localhost'
 if ':' in display_hostname:
  display_hostname = '[%s]' % display_hostname
 quit_msg = '(Press CTRL+C to quit)'
 port = sock.getsockname()[1]
 _log('info', ' * Running on %s://%s:%d/ %s',
   ssl_context is None and 'http' or 'https',
   display_hostname, port, quit_msg)
def inner():
 try:
  fd = int(os.environ['WERKZEUG_SERVER_FD'])
 except (LookupError, ValueError):
  fd = None
 srv = make_server(hostname, port, application, threaded,
      processes, request_handler,
      passthrough_errors, ssl_context,
      fd=fd)
 if fd is None:
  log_startup(srv.socket)
 srv.serve_forever()
if use_reloader:
 # If we're not running already in the subprocess that is the
 # reloader we want to open up a socket early to make sure the
 # port is actually available.
 if os.environ.get('WERKZEUG_RUN_MAIN') != 'true':
  if port == 0 and not can_open_by_fd:
   raise ValueError('Cannot bind to a random port with enabled '
        'reloader if the Python interpreter does '
        'not support socket opening by fd.')
 
  # Create and destroy a socket so that any exceptions are
  # raised before we spawn a separate Python interpreter and
  # lose this ability.
  address_family = select_ip_version(hostname, port)
  s = socket.socket(address_family, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  s.bind(get_sockaddr(hostname, port, address_family))
  if hasattr(s, 'set_inheritable'):
   s.set_inheritable(True)
 
  # If we can open the socket by file descriptor, then we can just
  # reuse this one and our socket will survive the restarts.
  if can_open_by_fd:
   os.environ['WERKZEUG_SERVER_FD'] = str(s.fileno())
   s.listen(LISTEN_QUEUE)
   log_startup(s)
  else:
   s.close()
 
 # Do not use relative imports, otherwise "python -m werkzeug.serving"
 # breaks.
 from werkzeug._reloader import run_with_reloader
 run_with_reloader(inner, extra_files, reloader_interval,
      reloader_type)
else:
 inner() #默认会执行

还是经过一系列判断后默认会进入inner()函数,这个函数定义在run_simple()内,属于闭包,inner()中会执行make_server()这个函数,看下源码:

def make_server(host=None, port=None, app=None, threaded=False, processes=1,

request_handler=None, passthrough_errors=False,
   ssl_context=None, fd=None):
"""Create a new server instance that is either threaded, or forks
or just processes one request after another.
"""
if threaded and processes > 1:
 raise ValueError("cannot have a multithreaded and "
      "multi process server.")
elif threaded:
 return ThreadedWSGIServer(host, port, app, request_handler,
        passthrough_errors, ssl_context, fd=fd)
elif processes > 1:
 return ForkingWSGIServer(host, port, app, processes, request_handler,
        passthrough_errors, ssl_context, fd=fd)
else:
 return BaseWSGIServer(host, port, app, request_handler,
       passthrough_errors, ssl_context, fd=fd)

看到这也很明白了,想要配置多线程或者多进程,则需要设置threaded或processes这两个参数,而这两个参数是从app.run()中传递过来的:

app.run(**options) ---> run_simple(threaded,processes) ---> make_server(threaded,processes)

默认情况下flask是单线程,单进程的,想要开启只需要在run中传入对应的参数:app.run(threaded=True)即可.

从make_server中可知,flask提供了三种server:ThreadedWSGIServer,ForkingWSGIServer,BaseWSGIServer,默认情况下是BaseWSGIServer
以线程为例,看下ThreadedWSGIServer这个类:

class ThreadedWSGIServer(ThreadingMixIn, BaseWSGIServer): #继承自ThreadingMixIn, BaseWSGIServer

"""A WSGI server that does threading."""
multithread = True
daemon_threads = True

ThreadingMixIn = socketserver.ThreadingMixIn

class ThreadingMixIn:

"""Mix-in class to handle each request in a new thread."""
 
# Decides how threads will act upon termination of the
# main process
daemon_threads = False
 
def process_request_thread(self, request, client_address):
 """Same as in BaseServer but as a thread.
 In addition, exception handling is done here.
 """
 try:
  self.finish_request(request, client_address)
  self.shutdown_request(request)
 except:
  self.handle_error(request, client_address)
  self.shutdown_request(request)
 
def process_request(self, request, client_address):
 """Start a new thread to process the request."""
 t = threading.Thread(target = self.process_request_thread,
       args = (request, client_address))
 t.daemon = self.daemon_threads
 t.start()

process_request就是对每个请求产生一个新的线程来处理
最后写一个非常简单的应用来验证以上说法:

from flask import Flask
from flask import _request_ctx_stack

app = Flask(__name__)

@app.route('/')
def index():

print(_request_ctx_stack._local.__ident_func__())
while True:
 pass
return '<h1>hello</h1>'

app.run() #如果需要开启多线程则app.run(threaded=True)

_request_ctx_stack._local.__ident_func__()对应这get_ident()这个函数,返回当前线程id,为什么要在后面加上while True这句呢,我们看下get_ident()这个函数的说明:

Return a non-zero integer that uniquely identifies the current thread amongst other threads that exist simultaneously. This may be used to identify per-thread resources. Even though on some platforms threads identities may appear to be allocated consecutive numbers starting at 1, this behavior should not be relied upon, and the number should be seen purely as a magic cookie. A thread's identity may be reused for another thread after it exits.

关键字我已经加粗了,线程id会在线程结束后重复利用,所以我在路由函数中加了这个死循环来阻塞请求以便于观察到不同的id,这就会产生两种情况:

1.没开启多线程的情况下,一次请求过来,服务器直接阻塞,并且之后的其他请求也都阻塞

2.开启多线程情况下,每次都会打印出不同的线程id

结果:

第一种情况

Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
139623180527360

第二种情况

Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
140315469436672
140315477829376
140315486222080
140315316901632
140315105163008
140315096770304
140315088377600

结果显而易见

综上所述:flask支持多线程,但默认没开启,其次app.run()只适用于开发环境,生产环境下可以使用uWSGI,Gunicorn等web服务器

内容扩展:

flask开启多线程还是多进程

Flask 默认是单进程,单线程阻塞的任务模式,在项目上线的时候可以通过nginx+gunicorn 的方式部署flask任务。

但是在开发的过程中如果想通过延迟的方式测试高并发怎么实现呢,其实非常简单,

app.run()中可以接受两个参数,分别是threaded和processes,用于开启线程支持和进程支持。

1.threaded : 多线程支持,默认为False,即不开启多线程;

2.processes:进程数量,默认为1.

开启方式:

if __name__ == '__main__':
   app.run(threaded=True)
 
   # app.run(processes=4)

注意:多进程或多线程只能选择一个,不能同时开启。

以上就是flask开启多线程的具体方法的详细内容,更多关于flask如何开启多线程详解的资料请关注三水点靠木其它相关文章!

Python 相关文章推荐
python3模拟百度登录并实现百度贴吧签到示例分享(百度贴吧自动签到)
Feb 24 Python
python中的字典详细介绍
Sep 18 Python
python使用wxpython开发简单记事本的方法
May 20 Python
对numpy中数组元素的统一赋值实例
Apr 04 Python
Flask框架WTForm表单用法示例
Jul 20 Python
python实现傅里叶级数展开的实现
Jul 21 Python
用Python将mysql数据导出成json的方法
Aug 21 Python
python程序封装为win32服务的方法
Mar 07 Python
Python FtpLib模块应用操作详解
Dec 12 Python
Python爬虫爬取微信朋友圈
Aug 06 Python
Python Pygame实现俄罗斯方块
Feb 19 Python
python3 实现mysql数据库连接池的示例代码
Apr 17 Python
基于opencv实现简单画板功能
Aug 02 #Python
django下创建多个app并设置urls方法
Aug 02 #Python
Django如何在不停机的情况下创建索引
Aug 02 #Python
如何用Anaconda搭建虚拟环境并创建Django项目
Aug 02 #Python
如何解决flask修改静态资源后缓存文件不能及时更改问题
Aug 02 #Python
Flask缓存静态文件的具体方法
Aug 02 #Python
Flask中sqlalchemy模块的实例用法
Aug 02 #Python
You might like
php 定界符格式引起的错误
2011/05/24 PHP
php中计算中文字符串长度、截取中文字符串的函数代码
2011/08/09 PHP
Symfony2在Nginx下的配置方法图文教程
2016/02/04 PHP
php实现贪吃蛇小游戏
2016/07/26 PHP
js 事件小结 表格区别
2007/08/13 Javascript
js树形控件脚本代码
2008/07/24 Javascript
jQuery 加上最后自己的验证
2009/11/04 Javascript
google 搜索框添加关键字实现代码
2010/04/24 Javascript
.net,js捕捉文本框回车键事件的小例子(兼容多浏览器)
2013/03/11 Javascript
使用JavaScript 实现对象 匀速/变速运动的方法
2013/05/08 Javascript
js实现鼠标滑过文字链接色彩变化的效果
2015/05/06 Javascript
js实现横向伸展开的二级导航菜单代码
2015/08/28 Javascript
Flow之一个新的Javascript静态类型检查器
2015/12/21 Javascript
浅谈JS中的!=、== 、!==、===的用法和区别
2016/09/24 Javascript
Javascript之深入浅出prototype
2017/02/06 Javascript
vue项目配置使用flow类型检查的步骤
2020/03/18 Javascript
python编码最佳实践之总结
2016/02/14 Python
利用Python命令行传递实例化对象的方法
2016/11/02 Python
使用 Python 实现简单的 switch/case 语句的方法
2018/09/17 Python
Python实现按逗号分隔列表的方法
2018/10/23 Python
Python检测数据类型的方法总结
2019/05/20 Python
Python时间差中seconds和total_seconds的区别详解
2019/12/26 Python
如何利用Python matplotlib绘制雷达图
2020/12/21 Python
英国办公家具网站:Furniture At Work
2019/10/07 全球购物
请解释在new与override的区别
2012/10/29 面试题
我们在web应用开发过程中经常遇到输出某种编码的字符,如iso8859-1等,如何输出一个某种编码的字符串?
2014/03/30 面试题
房地产销售大学生自我评价分享
2013/11/11 职场文书
班长演讲稿范文
2014/04/24 职场文书
计算机网络及管理学专业求职信
2014/06/05 职场文书
安全目标管理责任书
2014/07/25 职场文书
大明湖导游词
2015/02/03 职场文书
中国梦党课学习心得体会
2016/01/05 职场文书
2019学子的答谢词范本!
2019/07/05 职场文书
只需要这一行代码就能让python计算速度提高十倍
2021/05/24 Python
详细聊聊关于Mysql联合查询的那些事儿
2021/10/24 MySQL
pytest实现多进程与多线程运行超好用的插件
2022/07/15 Python