低版本Druid连接池+MySQL驱动8.0导致线程阻塞、性能受限


Posted in MySQL onJuly 01, 2021

现象

应用升级MySQL驱动8.0后,在并发量较高时,查看监控打点,Druid连接池拿到连接并执行SQL的时间大部分都超过200ms

对系统进行压测,发现出现大量线程阻塞的情况,线程dump信息如下:

"http-nio-5366-exec-48" #210 daemon prio=5 os_prio=0 tid=0x00000000023d0800 nid=0x3be9 waiting for monitor entry [0x00007fa4c1400000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader.loadClass(TomcatEmbeddedWebappClassLoader.java:66)
        - waiting to lock <0x0000000775af0960> (a java.lang.Object)
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1186)
        at com.alibaba.druid.util.Utils.loadClass(Utils.java:220)
        at com.alibaba.druid.util.MySqlUtils.getLastPacketReceivedTimeMs(MySqlUtils.java:372)

根因分析

public class MySqlUtils {

    public static long getLastPacketReceivedTimeMs(Connection conn) throws SQLException {
        if (class_connectionImpl == null && !class_connectionImpl_Error) {
            try {
                class_connectionImpl = Utils.loadClass("com.mysql.jdbc.MySQLConnection");
            } catch (Throwable error){
                class_connectionImpl_Error = true;
            }
        }

        if (class_connectionImpl == null) {
            return -1;
        }

        if (method_getIO == null && !method_getIO_error) {
            try {
                method_getIO = class_connectionImpl.getMethod("getIO");
            } catch (Throwable error){
                method_getIO_error = true;
            }
        }

        if (method_getIO == null) {
            return -1;
        }

        if (class_MysqlIO == null && !class_MysqlIO_Error) {
            try {
                class_MysqlIO = Utils.loadClass("com.mysql.jdbc.MysqlIO");
            } catch (Throwable error){
                class_MysqlIO_Error = true;
            }
        }

        if (class_MysqlIO == null) {
            return -1;
        }

        if (method_getLastPacketReceivedTimeMs == null && !method_getLastPacketReceivedTimeMs_error) {
            try {
                Method method = class_MysqlIO.getDeclaredMethod("getLastPacketReceivedTimeMs");
                method.setAccessible(true);
                method_getLastPacketReceivedTimeMs = method;
            } catch (Throwable error){
                method_getLastPacketReceivedTimeMs_error = true;
            }
        }

        if (method_getLastPacketReceivedTimeMs == null) {
            return -1;
        }

        try {
            Object connImpl = conn.unwrap(class_connectionImpl);
            if (connImpl == null) {
                return -1;
            }

            Object mysqlio = method_getIO.invoke(connImpl);
            Long ms = (Long) method_getLastPacketReceivedTimeMs.invoke(mysqlio);
            return ms.longValue();
        } catch (IllegalArgumentException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        } catch (IllegalAccessException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        } catch (InvocationTargetException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        }
    }

MySqlUtils中的getLastPacketReceivedTimeMs()方法会加载com.mysql.jdbc.MySQLConnection这个类,但在MySQL驱动8.0中类名改为com.mysql.cj.jdbc.ConnectionImpl,所以MySQL驱动8.0中加载不到com.mysql.jdbc.MySQLConnection

getLastPacketReceivedTimeMs()方法实现中,如果Utils.loadClass("com.mysql.jdbc.MySQLConnection")加载不到类并抛出异常,会修改变量class_connectionImpl_Error,下次调用不会再进行加载

public class Utils {

    public static Class<?> loadClass(String className) {
        Class<?> clazz = null;

        if (className == null) {
            return null;
        }

        try {
            return Class.forName(className);
        } catch (ClassNotFoundException e) {
            // skip
        }

        ClassLoader ctxClassLoader = Thread.currentThread().getContextClassLoader();
        if (ctxClassLoader != null) {
            try {
                clazz = ctxClassLoader.loadClass(className);
            } catch (ClassNotFoundException e) {
                // skip
            }
        }

        return clazz;
    }

但是,在Utils的loadClass()方法中同样catch了ClassNotFoundException,这就导致loadClass()在加载不到类的时候,并不会抛出异常,从而会导致每调用一次getLastPacketReceivedTimeMs()方法,就会加载一次MySQLConnection这个类

线程dump信息中可以看到是在调用TomcatEmbeddedWebappClassLoader的loadClass()方法时,导致线程阻塞的

public class TomcatEmbeddedWebappClassLoader extends ParallelWebappClassLoader {

public Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
  synchronized (JreCompat.isGraalAvailable() ? this : getClassLoadingLock(name)) {
   Class<?> result = findExistingLoadedClass(name);
   result = (result != null) ? result : doLoadClass(name);
   if (result == null) {
    throw new ClassNotFoundException(name);
   }
   return resolveIfNecessary(result, resolve);
  }
 }

这是因为TomcatEmbeddedWebappClassLoader在加载类的时候,会加synchronized锁,这就导致每调用一次getLastPacketReceivedTimeMs()方法,就会加载一次com.mysql.jdbc.MySQLConnection,而又始终加载不到,在加载类的时候会加synchronized锁,所以会出现线程阻塞,性能下降的现象

getLastPacketReceivedTimeMs()方法调用时机

public abstract class DruidAbstractDataSource extends WrapperAdapter implements DruidAbstractDataSourceMBean, DataSource, DataSourceProxy, Serializable {

    protected boolean testConnectionInternal(DruidConnectionHolder holder, Connection conn) {
        String sqlFile = JdbcSqlStat.getContextSqlFile();
        String sqlName = JdbcSqlStat.getContextSqlName();

        if (sqlFile != null) {
            JdbcSqlStat.setContextSqlFile(null);
        }
        if (sqlName != null) {
            JdbcSqlStat.setContextSqlName(null);
        }
        try {
            if (validConnectionChecker != null) {
                boolean valid = validConnectionChecker.isValidConnection(conn, validationQuery, validationQueryTimeout);
                long currentTimeMillis = System.currentTimeMillis();
                if (holder != null) {
                    holder.lastValidTimeMillis = currentTimeMillis;
                    holder.lastExecTimeMillis = currentTimeMillis;
                }

                if (valid && isMySql) { // unexcepted branch
                    long lastPacketReceivedTimeMs = MySqlUtils.getLastPacketReceivedTimeMs(conn);
                    if (lastPacketReceivedTimeMs > 0) {
                        long mysqlIdleMillis = currentTimeMillis - lastPacketReceivedTimeMs;
                        if (lastPacketReceivedTimeMs > 0 //
                                && mysqlIdleMillis >= timeBetweenEvictionRunsMillis) {
                            discardConnection(holder);
                            String errorMsg = "discard long time none received connection. "
                                    + ", jdbcUrl : " + jdbcUrl
                                    + ", jdbcUrl : " + jdbcUrl
                                    + ", lastPacketReceivedIdleMillis : " + mysqlIdleMillis;
                            LOG.error(errorMsg);
                            return false;
                        }
                    }
                }

                if (valid && onFatalError) {
                    lock.lock();
                    try {
                        if (onFatalError) {
                            onFatalError = false;
                        }
                    } finally {
                        lock.unlock();
                    }
                }

                return valid;
            }

            if (conn.isClosed()) {
                return false;
            }

            if (null == validationQuery) {
                return true;
            }

            Statement stmt = null;
            ResultSet rset = null;
            try {
                stmt = conn.createStatement();
                if (getValidationQueryTimeout() > 0) {
                    stmt.setQueryTimeout(validationQueryTimeout);
                }
                rset = stmt.executeQuery(validationQuery);
                if (!rset.next()) {
                    return false;
                }
            } finally {
                JdbcUtils.close(rset);
                JdbcUtils.close(stmt);
            }

            if (onFatalError) {
                lock.lock();
                try {
                    if (onFatalError) {
                        onFatalError = false;
                    }
                } finally {
                    lock.unlock();
                }
            }

            return true;
        } catch (Throwable ex) {
            // skip
            return false;
        } finally {
            if (sqlFile != null) {
                JdbcSqlStat.setContextSqlFile(sqlFile);
            }
            if (sqlName != null) {
                JdbcSqlStat.setContextSqlName(sqlName);
            }
        }
    }

只有DruidAbstractDataSource的testConnectionInternal()方法中会调用getLastPacketReceivedTimeMs()方法

testConnectionInternal()是用来检测连接是否有效的,在获取连接和归还连接时都有可能会调用该方法,这取决于Druid检测连接是否有效的参数

Druid检测连接是否有效的参数:

  • testOnBorrow:每次获取连接时执行validationQuery检测连接是否有效(会影响性能)
  • testOnReturn:每次归还连接时执行validationQuery检测连接是否有效(会影响性能)
  • testWhileIdle:申请连接的时候检测,如果空闲时间大于timeBetweenEvictionRunsMillis,执行validationQuery检测连接是否有效
  • 应用中设置了testOnBorrow=true,每次获取连接时,都会去抢占synchronized锁,所以性能下降的很明显

解决方案

经验证,使用Druid 1.x版本<=1.1.22会出现该bug,解决方案就是升级至Druid 1.x版本>=1.1.23或者Druid 1.2.x版本

GitHub issue:https://github.com/alibaba/druid/issues/3808

到此这篇关于低版本Druid连接池+MySQL驱动8.0导致线程阻塞、性能受限的文章就介绍到这了,更多相关MySQL驱动8.0低版本Druid连接池内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

MySQL 相关文章推荐
MySQL索引篇之千万级数据实战测试
Apr 05 MySQL
MySQL 表空间碎片的概念及相关问题解决
May 07 MySQL
MySQL 自定义变量的概念及特点
May 13 MySQL
Mysql官方性能测试工具mysqlslap的使用简介
May 21 MySQL
MySql 缓存查询原理与缓存监控和索引监控介绍
Jul 02 MySQL
MySQL系列之一 MariaDB-server安装
Jul 02 MySQL
mysql配置SSL证书登录的实现
Sep 04 MySQL
MySQL中几种插入和批量语句实例详解
Sep 14 MySQL
MySQL窗口函数的具体使用
Nov 17 MySQL
一文搞懂MySQL索引页结构
Feb 28 MySQL
mysql 生成连续日期及变量赋值
Mar 20 MySQL
WINDOWS下安装mysql 8.x 的方法图文教程
Apr 19 MySQL
MySQL 8.0 驱动与阿里druid版本兼容问题解决
MySQL query_cache_type 参数与使用详解
Jul 01 #MySQL
mysql 数据插入优化方法之concurrent_insert
Jul 01 #MySQL
MySQL的Query Cache图文详解
MySQL高速缓存启动方法及参数详解(query_cache_size)
Jul 01 #MySQL
mysql优化之query_cache_limit参数说明
Jul 01 #MySQL
MySQL中存储时间的最佳实践指南
Jul 01 #MySQL
You might like
第六节 访问属性和方法 [6]
2006/10/09 PHP
Fatal error: Call to undefined function curl_init()解决方法
2010/04/09 PHP
Windows下部署Apache+PHP+MySQL运行环境实战
2012/08/31 PHP
php编写的简单页面跳转功能实现代码
2013/11/27 PHP
ThinkPHP实现一键清除缓存方法
2014/06/26 PHP
PHP与Ajax相结合实现登录验证小Demo
2016/03/16 PHP
php使用curl并发减少后端访问时间的方法分析
2016/05/12 PHP
JS短路原理的应用示例 精简代码的途径
2013/12/13 Javascript
Nodejs学习笔记之Global Objects全局对象
2015/01/13 NodeJs
JavaScript实现页面5秒后自动跳转的方法
2015/04/16 Javascript
网页从弹窗页面单选框传值至父页面代码分享
2015/09/29 Javascript
WEB前端开发都应知道的jquery小技巧及jquery三个简写
2015/11/15 Javascript
JS获取随机数和时间转换的简单实例
2016/07/10 Javascript
js中判断变量类型函数typeof的用法总结
2016/08/09 Javascript
jQuery选择器实例应用
2017/01/05 Javascript
vue.js加载新的内容(实例代码)
2017/06/01 Javascript
基于 Vue 实现一个酷炫的 menu插件
2017/11/14 Javascript
一个简单的node.js界面实现方法
2018/06/01 Javascript
基于React Native 0.52实现轮播图效果
2020/08/25 Javascript
详解JavaScript栈内存与堆内存
2019/04/04 Javascript
小程序:授权、登录、session_key、unionId的详解
2019/05/15 Javascript
layui table 表格模板按钮的实例代码
2019/09/21 Javascript
[50:02]完美世界DOTA2联赛PWL S2 Magma vs FTD 第三场 11.29
2020/12/03 DOTA
Python读写ini文件的方法
2015/05/28 Python
Python tkinter的grid布局及Text动态显示方法
2018/10/11 Python
python实现多层感知器MLP(基于双月数据集)
2019/01/18 Python
python基础教程之while循环
2019/08/14 Python
python函数装饰器之带参数的函数和带参数的装饰器用法示例
2019/11/06 Python
python GUI库图形界面开发之pyinstaller打包python程序为exe安装文件
2020/02/26 Python
如何设置PyCharm中的Python代码模版(推荐)
2020/11/20 Python
化工专业应届生求职信
2013/11/08 职场文书
英文自荐信
2013/12/15 职场文书
平面设计求职信
2014/03/10 职场文书
党员三严三实心得体会
2014/10/13 职场文书
求职自荐信该如何书写?
2019/06/24 职场文书
科学家测试在太空中培育人造肉,用于未来太空旅行
2022/04/29 数码科技