多线程判断redis key导致Java hep space内存溢出

 

线上经常发现报如下错误，后来发现root cause是selectConfigitemforinstanceMap这个方法会查出几十万的结果集然后json压缩解压，写map等等操作，但是这个selectConfigitemforinstanceMap方法被很多地方调用到了，导致极有可能多线程同时都在查出几十万的结果集然后json压缩解压，写map等等操作（判断条件是某个redis的key如果被del或者expire就查库等等复杂操作），然后我改成了线程安全的就Ok了。后面观察日志ConfigModelForInstanceKey不存在的时候只能有一个thread进行查出几十万的结果集然后json压缩解压，写map等等操作

 

 

@Override
public synchronized void selectConfigitemforinstanceMapNew() {

    boolean configItemKeyExist = redisClusterUtil.hasKey(ConfigModelForInstanceKey);

    logger.info("configItemKeyExist======{}",configItemKeyExist);

    if(level5Map!=null)
    {
        logger.info("level5Map.size()======{}",level5Map.size());
    }

    if(level4Map!=null)
    {
        logger.info("level4Map.size()======{}",level4Map.size());
    }

    if(level3Map!=null)
    {
        logger.info("level3Map.size()======{}",level3Map.size());
    }

    if(level2Map!=null)
    {
        logger.info("level2Map.size()======{}",level2Map.size());
    }


    if (!configItemKeyExist) {
        List<ConfigModelForInstance> result = configitemdao.selectConfigitemforinstance();
        if (result != null && !result.isEmpty()) {
            populateMaps(result); // Helper method to populate level maps
            cacheDataToRedis(result);
        }
    } else {
        if(level2Map!=null&&level2Map.size()==0) {
            //when restart novatask service and redis key is not expired,this code will exec
            loadFromRedis();
        }
    }

    logger.info("end_selectConfigitemforinstanceMapNew");
}

 

 

 

 

Caused by: java.lang.OutOfMemoryError: Java heap space

2024-11-21 06:20:51.500 [] [Thread-3126] ERROR c.h.n.service.impl.DataFromRedis.DataFromRedisImpl.run [958] : error:org.springframework.jdbc.UncategorizedSQLException: Error attempting to get column 'PN' from result set. Cause: java.sql.SQLException: Error
; uncategorized SQLException; SQL state [null]; error code [0]; Error
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:93)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:439)
at jdk.proxy2/jdk.proxy2.$Proxy120.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:224)
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:147)
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:80)
at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:141)
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86)
at jdk.proxy2/jdk.proxy2.$Proxy161.selectConfigitemforinstance(Unknown Source)
at com.hp.nova.service.impl.configitem.ConfigItemServiceImpl.selectConfigitemforinstanceMap(ConfigItemServiceImpl.java:304)
at com.hp.nova.service.impl.configitem.ConfigItemServiceImpl.getConfigModelForInstanceBydb(ConfigItemServiceImpl.java:180)
at com.hp.nova.service.impl.configitem.ConfigItemServiceImpl.lambda$getCOnfigitemviewByconfigids$4(ConfigItemServiceImpl.java:551)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at com.hp.nova.service.impl.configitem.ConfigItemServiceImpl.getCOnfigitemviewByconfigids(ConfigItemServiceImpl.java:536)
at com.hp.nova.service.impl.DataFromRedis.DataFromRedisImpl.GenerateColumns(DataFromRedisImpl.java:1301)
at com.hp.nova.service.impl.DataFromRedis.DataFromRedisImpl.geteinstancefromredis(DataFromRedisImpl.java:642)
at com.hp.nova.service.impl.DataFromRedis.DataFromRedisImpl.gettaskassimentview(DataFromRedisImpl.java:773)
at com.hp.nova.service.impl.DataFromRedis.DataFromRedisImpl$Threadtaskassimentview.run(DataFromRedisImpl.java:955)
Caused by: java.sql.SQLException: Error
at com.alibaba.druid.pool.DruidDataSource.handleConnectionException(DruidDataSource.java:1910)
at com.alibaba.druid.pool.DruidPooledConnection.handleException(DruidPooledConnection.java:124)
at com.alibaba.druid.pool.DruidPooledStatement.checkException(DruidPooledStatement.java:87)
at com.alibaba.druid.pool.DruidPooledResultSet.checkException(DruidPooledResultSet.java:42)