低版本Druid连接池+MySQL驱动8.0导致线程阻塞、性能受限


Posted in MySQL onJuly 01, 2021

现象

应用升级MySQL驱动8.0后,在并发量较高时,查看监控打点,Druid连接池拿到连接并执行SQL的时间大部分都超过200ms

对系统进行压测,发现出现大量线程阻塞的情况,线程dump信息如下:

"http-nio-5366-exec-48" #210 daemon prio=5 os_prio=0 tid=0x00000000023d0800 nid=0x3be9 waiting for monitor entry [0x00007fa4c1400000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader.loadClass(TomcatEmbeddedWebappClassLoader.java:66)
        - waiting to lock <0x0000000775af0960> (a java.lang.Object)
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1186)
        at com.alibaba.druid.util.Utils.loadClass(Utils.java:220)
        at com.alibaba.druid.util.MySqlUtils.getLastPacketReceivedTimeMs(MySqlUtils.java:372)

根因分析

public class MySqlUtils {

    public static long getLastPacketReceivedTimeMs(Connection conn) throws SQLException {
        if (class_connectionImpl == null && !class_connectionImpl_Error) {
            try {
                class_connectionImpl = Utils.loadClass("com.mysql.jdbc.MySQLConnection");
            } catch (Throwable error){
                class_connectionImpl_Error = true;
            }
        }

        if (class_connectionImpl == null) {
            return -1;
        }

        if (method_getIO == null && !method_getIO_error) {
            try {
                method_getIO = class_connectionImpl.getMethod("getIO");
            } catch (Throwable error){
                method_getIO_error = true;
            }
        }

        if (method_getIO == null) {
            return -1;
        }

        if (class_MysqlIO == null && !class_MysqlIO_Error) {
            try {
                class_MysqlIO = Utils.loadClass("com.mysql.jdbc.MysqlIO");
            } catch (Throwable error){
                class_MysqlIO_Error = true;
            }
        }

        if (class_MysqlIO == null) {
            return -1;
        }

        if (method_getLastPacketReceivedTimeMs == null && !method_getLastPacketReceivedTimeMs_error) {
            try {
                Method method = class_MysqlIO.getDeclaredMethod("getLastPacketReceivedTimeMs");
                method.setAccessible(true);
                method_getLastPacketReceivedTimeMs = method;
            } catch (Throwable error){
                method_getLastPacketReceivedTimeMs_error = true;
            }
        }

        if (method_getLastPacketReceivedTimeMs == null) {
            return -1;
        }

        try {
            Object connImpl = conn.unwrap(class_connectionImpl);
            if (connImpl == null) {
                return -1;
            }

            Object mysqlio = method_getIO.invoke(connImpl);
            Long ms = (Long) method_getLastPacketReceivedTimeMs.invoke(mysqlio);
            return ms.longValue();
        } catch (IllegalArgumentException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        } catch (IllegalAccessException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        } catch (InvocationTargetException e) {
            throw new SQLException("getLastPacketReceivedTimeMs error", e);
        }
    }

MySqlUtils中的getLastPacketReceivedTimeMs()方法会加载com.mysql.jdbc.MySQLConnection这个类,但在MySQL驱动8.0中类名改为com.mysql.cj.jdbc.ConnectionImpl,所以MySQL驱动8.0中加载不到com.mysql.jdbc.MySQLConnection

getLastPacketReceivedTimeMs()方法实现中,如果Utils.loadClass("com.mysql.jdbc.MySQLConnection")加载不到类并抛出异常,会修改变量class_connectionImpl_Error,下次调用不会再进行加载

public class Utils {

    public static Class<?> loadClass(String className) {
        Class<?> clazz = null;

        if (className == null) {
            return null;
        }

        try {
            return Class.forName(className);
        } catch (ClassNotFoundException e) {
            // skip
        }

        ClassLoader ctxClassLoader = Thread.currentThread().getContextClassLoader();
        if (ctxClassLoader != null) {
            try {
                clazz = ctxClassLoader.loadClass(className);
            } catch (ClassNotFoundException e) {
                // skip
            }
        }

        return clazz;
    }

但是,在Utils的loadClass()方法中同样catch了ClassNotFoundException,这就导致loadClass()在加载不到类的时候,并不会抛出异常,从而会导致每调用一次getLastPacketReceivedTimeMs()方法,就会加载一次MySQLConnection这个类

线程dump信息中可以看到是在调用TomcatEmbeddedWebappClassLoader的loadClass()方法时,导致线程阻塞的

public class TomcatEmbeddedWebappClassLoader extends ParallelWebappClassLoader {

public Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
  synchronized (JreCompat.isGraalAvailable() ? this : getClassLoadingLock(name)) {
   Class<?> result = findExistingLoadedClass(name);
   result = (result != null) ? result : doLoadClass(name);
   if (result == null) {
    throw new ClassNotFoundException(name);
   }
   return resolveIfNecessary(result, resolve);
  }
 }

这是因为TomcatEmbeddedWebappClassLoader在加载类的时候,会加synchronized锁,这就导致每调用一次getLastPacketReceivedTimeMs()方法,就会加载一次com.mysql.jdbc.MySQLConnection,而又始终加载不到,在加载类的时候会加synchronized锁,所以会出现线程阻塞,性能下降的现象

getLastPacketReceivedTimeMs()方法调用时机

public abstract class DruidAbstractDataSource extends WrapperAdapter implements DruidAbstractDataSourceMBean, DataSource, DataSourceProxy, Serializable {

    protected boolean testConnectionInternal(DruidConnectionHolder holder, Connection conn) {
        String sqlFile = JdbcSqlStat.getContextSqlFile();
        String sqlName = JdbcSqlStat.getContextSqlName();

        if (sqlFile != null) {
            JdbcSqlStat.setContextSqlFile(null);
        }
        if (sqlName != null) {
            JdbcSqlStat.setContextSqlName(null);
        }
        try {
            if (validConnectionChecker != null) {
                boolean valid = validConnectionChecker.isValidConnection(conn, validationQuery, validationQueryTimeout);
                long currentTimeMillis = System.currentTimeMillis();
                if (holder != null) {
                    holder.lastValidTimeMillis = currentTimeMillis;
                    holder.lastExecTimeMillis = currentTimeMillis;
                }

                if (valid && isMySql) { // unexcepted branch
                    long lastPacketReceivedTimeMs = MySqlUtils.getLastPacketReceivedTimeMs(conn);
                    if (lastPacketReceivedTimeMs > 0) {
                        long mysqlIdleMillis = currentTimeMillis - lastPacketReceivedTimeMs;
                        if (lastPacketReceivedTimeMs > 0 //
                                && mysqlIdleMillis >= timeBetweenEvictionRunsMillis) {
                            discardConnection(holder);
                            String errorMsg = "discard long time none received connection. "
                                    + ", jdbcUrl : " + jdbcUrl
                                    + ", jdbcUrl : " + jdbcUrl
                                    + ", lastPacketReceivedIdleMillis : " + mysqlIdleMillis;
                            LOG.error(errorMsg);
                            return false;
                        }
                    }
                }

                if (valid && onFatalError) {
                    lock.lock();
                    try {
                        if (onFatalError) {
                            onFatalError = false;
                        }
                    } finally {
                        lock.unlock();
                    }
                }

                return valid;
            }

            if (conn.isClosed()) {
                return false;
            }

            if (null == validationQuery) {
                return true;
            }

            Statement stmt = null;
            ResultSet rset = null;
            try {
                stmt = conn.createStatement();
                if (getValidationQueryTimeout() > 0) {
                    stmt.setQueryTimeout(validationQueryTimeout);
                }
                rset = stmt.executeQuery(validationQuery);
                if (!rset.next()) {
                    return false;
                }
            } finally {
                JdbcUtils.close(rset);
                JdbcUtils.close(stmt);
            }

            if (onFatalError) {
                lock.lock();
                try {
                    if (onFatalError) {
                        onFatalError = false;
                    }
                } finally {
                    lock.unlock();
                }
            }

            return true;
        } catch (Throwable ex) {
            // skip
            return false;
        } finally {
            if (sqlFile != null) {
                JdbcSqlStat.setContextSqlFile(sqlFile);
            }
            if (sqlName != null) {
                JdbcSqlStat.setContextSqlName(sqlName);
            }
        }
    }

只有DruidAbstractDataSource的testConnectionInternal()方法中会调用getLastPacketReceivedTimeMs()方法

testConnectionInternal()是用来检测连接是否有效的,在获取连接和归还连接时都有可能会调用该方法,这取决于Druid检测连接是否有效的参数

Druid检测连接是否有效的参数:

  • testOnBorrow:每次获取连接时执行validationQuery检测连接是否有效(会影响性能)
  • testOnReturn:每次归还连接时执行validationQuery检测连接是否有效(会影响性能)
  • testWhileIdle:申请连接的时候检测,如果空闲时间大于timeBetweenEvictionRunsMillis,执行validationQuery检测连接是否有效
  • 应用中设置了testOnBorrow=true,每次获取连接时,都会去抢占synchronized锁,所以性能下降的很明显

解决方案

经验证,使用Druid 1.x版本<=1.1.22会出现该bug,解决方案就是升级至Druid 1.x版本>=1.1.23或者Druid 1.2.x版本

GitHub issue:https://github.com/alibaba/druid/issues/3808

到此这篇关于低版本Druid连接池+MySQL驱动8.0导致线程阻塞、性能受限的文章就介绍到这了,更多相关MySQL驱动8.0低版本Druid连接池内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

MySQL 相关文章推荐
详解MySQL InnoDB存储引擎的内存管理
Apr 08 MySQL
将图片保存到mysql数据库并展示在前端页面的实现代码
May 02 MySQL
JDBC连接的六步实例代码(与mysql连接)
May 12 MySQL
正确使用MySQL update语句
May 26 MySQL
MySQL系列之八 MySQL服务器变量
Jul 02 MySQL
MySQL图形化管理工具Navicat安装步骤
Dec 04 MySQL
一文搞懂MySQL索引页结构
Feb 28 MySQL
mysql中数据库覆盖导入的几种方式总结
Mar 25 MySQL
MySQL数据库如何使用Shell进行连接
Apr 12 MySQL
Mysql InnoDB 的内存逻辑架构
May 06 MySQL
MySQL普通表如何转换成分区表
May 30 MySQL
MySQL添加索引特点及优化问题
Jul 23 MySQL
MySQL 8.0 驱动与阿里druid版本兼容问题解决
MySQL query_cache_type 参数与使用详解
Jul 01 #MySQL
mysql 数据插入优化方法之concurrent_insert
Jul 01 #MySQL
MySQL的Query Cache图文详解
MySQL高速缓存启动方法及参数详解(query_cache_size)
Jul 01 #MySQL
mysql优化之query_cache_limit参数说明
Jul 01 #MySQL
MySQL中存储时间的最佳实践指南
Jul 01 #MySQL
You might like
PHP 源代码压缩小工具
2009/12/22 PHP
php实现执行某一操作时弹出确认、取消对话框
2013/12/30 PHP
Linux中用PHP判断程序运行状态的2个方法
2014/05/04 PHP
Codeigniter实现处理用户登录验证后的URL跳转
2014/06/12 PHP
推荐一本PHP程序猿都应该拜读的书
2014/12/31 PHP
脚本之家贴图转换+转贴工具用到的js代码超级推荐
2007/04/05 Javascript
JavaScript脚本语言在网页中的简单应用
2007/05/13 Javascript
js设置组合快捷键/tabindex功能的方法
2013/11/21 Javascript
js的回调函数详解
2015/01/05 Javascript
JavaScript中模拟实现jsonp
2015/06/19 Javascript
javascript判断复选框是否选中的方法
2015/10/16 Javascript
对js中回调函数的一些看法
2016/08/29 Javascript
详解webpack 如何集成第三方js库
2017/06/29 Javascript
eslint 的三大通用规则详解
2019/05/16 Javascript
微信小程序 高德地图路线规划实现过程详解
2019/08/05 Javascript
JS异步处理的进化史深入讲解
2019/08/25 Javascript
python实现用户管理系统
2018/01/10 Python
python中使用iterrows()对dataframe进行遍历的实例
2018/06/09 Python
基于python绘制科赫雪花
2018/06/22 Python
python 动态生成变量名以及动态获取变量的变量名方法
2019/01/20 Python
python实现在一个画布上画多个子图
2020/01/19 Python
Django ORM判断查询结果是否为空,判断django中的orm为空实例
2020/07/09 Python
openCV提取图像中的矩形区域
2020/07/21 Python
python中xlutils库用法浅析
2020/12/29 Python
希尔顿酒店中国网站:Hilton中国
2017/03/11 全球购物
意大利和国际最佳时尚品牌:Drestige
2019/12/28 全球购物
请写出 BOOL flag 与"零值"比较的 if 语句
2016/02/29 面试题
顺丰快递Java软件工程师面试题
2015/07/31 面试题
采购部部门职责
2013/12/15 职场文书
餐厅执行经理岗位职责范本
2014/02/26 职场文书
媒体宣传策划方案
2014/05/25 职场文书
中华在我心中演讲稿
2014/09/13 职场文书
毕业实习自我鉴定范文2014
2014/09/26 职场文书
告诉你创业计划书的8个实用技巧
2019/07/12 职场文书
Go语言 详解net的tcp服务
2022/04/14 Golang
python数字图像处理:图像简单滤波
2022/06/28 Python