1、第一步:sql优化
正对于海量数据的查询优化,且外键关联比较多的情况,通常情况是下sql层面的优化,有些时候是由于sql不合理的编写导致,如尽量少使用sql内查询等
如:避免使用 left join (select * form table) a 等查询,一般这种都可以在sql 层面进行优化操作
2、第二步:创建索引
创建索引首当注意的是要匹配最佳左前缀法则,避免创建的索引失效,其实使用EXPLAIN关键字最索引进行分析,查看命中情况,然后适当调整索引的命中策略
3、第三步:通过前面两种还是未能显著提高效率,可分两次查询,减少索引的回表次数
如:第一次通过复杂的操作只差数据的主键ID,第二次直接通过主键ID的集合去查找完整的数据记录,即可获得不错的效果提升
实操案例:
第一次查询:
SELECTa.id,
FROMLEFT JOIN device_info b on a.terminal_id = b.idLEFT JOIN car_info c ON c.id = b.caridLEFT JOIN device_alarm_settings d on d.alarmnumber = a.alarm_numberLEFT JOIN sys_auth_dept f on f.deptid = b.deptid and f.is_delete = 0
WHEREa.start_time >= '2023-11-23 00:00:00' AND a.start_time <= '2023-11-25 10:55:38' AND f.deptidnew LIKE CONCAT( '001', '%' )
GROUP BYa.id
ORDER BYa.start_time DESC LIMIT 50,10
第二次查询只需要通过第一次的ID集合查询,即可获得数据,且全部命中索引:
SELECTa.id,a.is_handle AS isHandle,m.driver_code AS driverCode,m.drivername,c.carnumber,d.minclass_name AS minclassName,d.confirm,a.start_time AS startTime,a.end_time AS endTime,a.alarm_info AS alarmInfo,a.start_location AS startLocation,b.carid,a.end_location AS endLocation,a.handle_content AS handleContent,e.loginname,a.handle_time AS handleTime,a.end_lon AS endLon,a.end_lat AS endLat,a.start_lon AS startLon,a.start_lat AS startLat,b.terminal,a.alarm_number AS alarmNumber,f.deptname AS deptname,b.channeltotals,a.alarm_duration alarmDuration,mrm.rule_name AS ruleName,v.logic_channel AS videoChannel,v.url AS videoUrl,p.channelid AS photoChannel,p.multimediapath AS photoUrl,att.file_path,att.file_typeFROMdevice_alarm aLEFT JOIN device_info b ON a.terminal_id = b.idLEFT JOIN car_info c ON c.id = b.caridLEFT JOIN device_alarm_settings d ON d.alarmnumber = a.alarm_numberLEFT JOIN sys_auth_user e ON a.handle_by = e.useridLEFT JOIN car_driver_link cdl ON c.id = cdl.caridLEFT JOIN car_driver_info m ON m.id = cdl.driveridLEFT JOIN device_video_file_record v ON v.alarm_id = a.idLEFT JOIN device_media_info p ON a.id = p.alarmidLEFT JOIN monitor_rule_maintain mrm ON mrm.id = a.rule_idLEFT JOIN device_safety_alarm sda ON sda.device_alarm_id = a.idLEFT JOIN device_safety_attachment att ON att.safety_alarm_id = sda.idJOIN sys_auth_dept f ON f.deptid = b.deptidAND f.is_delete = 0WHEREa.id in <foreach collection="idList" open="(" close=")" separator="," item="id">#{id}</foreach>
先后两次关联表相同,只有where条件不同,却能得出意想不到的效果,其中奥妙一言以概之:“通过减少回表次数,增加查找次数来提高整体数据的相应速度”