使用 pg_profile 在 Postgres 中生成性能分析报告-编程知识

前言：

postgres数据库中拥有大量的辅助插件用于帮助DBA更好的分析数据库性能或整个集群，包括索引、I/O、CPU和内存等，pg_profile是基于PostgreSQL标准统计信息视图的诊断工具，它类似于Oracle AWR架构，和Oracle一样，它在指定时间生成快照，并且提供html格式来解释快照之间的统计数据；今天我们就来聊聊pg_profile这款性能分析工具有哪些功能和优点。

pg_profile工作原理：

pg_profile通过捕获数据库中被任意服务器进程执行的命令、收集关于表和索引访问的统计信息、监控对块读写次数、对用户定义函数使用的跟踪等四个方面来进行性能分析汇总展示，另外它还需追踪pg_stat_statements所有服务数据库执行sql语句的执行统计信息。

1.安装部署与配置：

提示：安装pg_profile扩展需要使用dblink扩展和pg_stat_statements扩展，因此在安装pg_profile扩展前需要先安装dblink扩展和pg_stat_statements扩展这两个扩展。安装：pg_stat_statements(pg_stat_statements是数据库自带一种扩展)1.修改配置文件vi postgresql.conf修改：shared_preload_libraries = 'pg_stat_statements,pg_cron'track_activities = on   //默认开启track_counts = on       //默认开启track_io_timing = on    //默认为off,需要修改成ontrack_functions = all   //默认为none，需要修改成all2.保存配置文件，并重启数据库生效pg_ctl -D $PGDTA restart3.使用管理员用户创建pg_stat_statements扩展antdb=# create extension pg_stat_statements;    CREATE EXTENSION
注：开启track_io_timing和track_functions这两个参数是为了pg_profile扩展更好的分析数据库使用安装：dblink1.使用管理员用户创建dblink扩展antdb=# create extension dblink ;    CREATE EXTENSION安装：pg_profile    1.tar xzf pg_profile--4.3.tar.gz --directory $(pg_config --sharedir)/extension2.antdb=# create schema profile;3.antdb=# create extension pg_profile schema profile;    CREATE EXTENSION
注：1.单独创建一个profile模式，是为将pg_profile扩展的表、视图、序列和函数单独创建到自定义模式中，与其他用户数据对象进行有效隔离。2.如果没有安装dblink和pg_stat_statements会报ERROR:  required extension "dblink" is not installed错。查询扩展是否部署成功：antdb=# \dxList of installed extensionsName        | Version |   Schema   |                              Description                               
--------------------+---------+------------+------------------------------------------------------------------------dblink             | 1.2     | public     | connect to other PostgreSQL databases from within a databasepg_cron            | 1.3     | public     | Job scheduler for PostgreSQLpg_profile         | 4.1     | public     | PostgreSQL load profile repository and report builderpg_repack          | 1.5.0   | public     | Reorganize tables in PostgreSQL databases with minimal lockspg_stat_statements | 1.8     | public     | track planning and execution statistics of all SQL statements executedplorasql           | 1.0     | pg_catalog | PL/oraSQL procedural languageplpgsql            | 1.0     | pg_catalog | PL/pgSQL procedural language
(7 rows)

2.查询pg_profile扩展信息


antdb=# \dx+ profile.pg_profile Objects in extension "pg_profile"Object description                               
--------------------------------------------------------------------------------function check_stmt_cnt_all_htbl(jsonb,integer)function check_stmt_cnt_first_htbl(jsonb,integer)function check_stmt_cnt(integer,integer,integer)function check_stmt_cnt_second_htbl(jsonb,integer)function cleanup_report_temp_tables(jsonb,integer)function cluster_stats_diff_htbl(jsonb,integer)function cluster_stats_htbl(jsonb,integer)function cluster_stats(integer,integer,integer)function cluster_stats_reset_diff_htbl(jsonb,integer)function cluster_stats_reset_htbl(jsonb,integer)function cluster_stats_reset(integer,integer,integer)function collect_obj_stats(jsonb,integer,integer,boolean)function collect_pg_stat_statements_stats(jsonb,integer,integer,integer)function collect_pg_wait_sampling_stats_11(jsonb,integer,integer,integer)function collect_pg_wait_sampling_stats(jsonb,integer,integer,integer)function collect_queries(oid,oid,bigint)function create_baseline(character varying,integer,integer,integer)function create_baseline(character varying,tstzrange,integer)function create_baseline(name,character varying,integer,integer,integer)function create_baseline(name,character varying,tstzrange,integer)function create_server(name,text,boolean,integer,text)function create_server_partitions(integer)function dbagg_jit_stats_diff_htbl(jsonb,integer)function dbagg_jit_stats_htbl(jsonb,integer)function dbstats_diff_htbl(jsonb,integer)function dbstats_htbl(jsonb,integer)function dbstats(integer,integer,integer)function dbstats_reset_diff_htbl(jsonb,integer)function dbstats_reset_htbl(jsonb,integer)function dbstats_reset(integer,integer,integer)function dbstats_sessions_diff_htbl(jsonb,integer)function dbstats_sessions_htbl(jsonb,integer)function dbstats_sessions(integer,integer,integer,integer)function delete_samples(integer,integer)function delete_samples(integer,integer,integer)function delete_samples(name,integer,integer)function delete_samples(name,tstzrange)function delete_samples(tstzrange)function disable_server(name)function drop_baseline(character varying)function drop_baseline(name,character varying)function drop_server(name)function enable_server(name)function export_data(name,integer,integer,boolean)function func_top_calls_diff_htbl(jsonb,integer)function func_top_calls_htbl(jsonb,integer)function func_top_time_diff_htbl(jsonb,integer)function func_top_time_htbl(jsonb,integer)function func_top_trg_diff_htbl(jsonb,integer)function func_top_trg_htbl(jsonb,integer)function get_baseline_samples(integer,character varying)function get_connstr(integer,jsonb)function get_diffreport(character varying,character varying,text,boolean)function get_diffreport(character varying,integer,integer,text,boolean)function get_diffreport(integer,integer,character varying,text,boolean)function get_diffreport(integer,integer,integer,integer,integer,text,boolean)function get_diffreport(integer,integer,integer,integer,text,boolean)function get_diffreport(name,character varying,character varying,text,boolean)function get_diffreport(name,character varying,integer,integer,text,boolean)function get_diffreport(name,character varying,tstzrange,text,boolean)function get_diffreport(name,integer,integer,character varying,text,boolean)function get_diffreport(name,integer,integer,integer,integer,text,boolean)function get_diffreport(name,tstzrange,character varying,text,boolean)function get_diffreport(name,tstzrange,tstzrange,text,boolean)function get_report(character varying,text,boolean)function get_report_context(integer,integer,integer,text,integer,integer)function get_report(integer,integer,integer,text,boolean)function get_report(integer,integer,text,boolean)function get_report(integer,tstzrange,text,boolean)function get_report_latest(name)function get_report(name,character varying,text,boolean)function get_report(name,integer,integer,text,boolean)function get_report(name,tstzrange,text,boolean)function get_report_template(jsonb,integer)function get_report(tstzrange,text,boolean)function get_sampleids_by_timerange(integer,tstzrange)function get_server_by_name(name)function get_sized_bounds(integer,integer,integer)function import_data(regclass,text)function init_report_temp_tables(jsonb,integer)function ix_top_fetch_diff_htbl(jsonb,integer)function ix_top_fetch_htbl(jsonb,integer)function ix_top_io_diff_htbl(jsonb,integer)function ix_top_io_htbl(jsonb,integer)function ix_unused_htbl(jsonb,integer)function jsonb_replace(jsonb,jsonb)function keep_baseline(character varying,integer)function keep_baseline(name,character varying,integer)function mark_pg_stat_statements(integer,integer,integer)function profile_checkavail_functions(integer,integer,integer)function profile_checkavail_io_times(integer,integer,integer)function profile_checkavail_planning_times(integer,integer,integer)function profile_checkavail_rusage(integer,integer,integer)function profile_checkavail_rusage_planstats(integer,integer,integer)function profile_checkavail_sessionstats(integer,integer,integer)function profile_checkavail_statements_jit_stats(integer,integer,integer)function profile_checkavail_statstatements(integer,integer,integer)function profile_checkavail_stmt_wal_bytes(integer,integer,integer)function profile_checkavail_trg_functions(integer,integer,integer)function profile_checkavail_wait_sampling_total(integer,integer,integer)function profile_checkavail_walstats(integer,integer,integer)function rename_server(name,name)function report_queries(jsonb,integer)function sample_dbobj_delta(jsonb,integer,integer,integer,boolean)function save_pg_stat_statements(integer,integer)function set_server_connstr(name,text)function set_server_db_exclude(name,name[])function set_server_description(name,text)function set_server_max_sample_age(name,integer)function set_server_size_sampling(name,time with time zone,interval,interval)function settings_and_changes_diff_htbl(jsonb,integer)function settings_and_changes_htbl(jsonb,integer)function settings_and_changes(integer,integer,integer)function show_baselines(name)function show_samples(integer)function show_samples(name,integer)function show_servers()function show_servers_size_sampling()function snapshot()function snapshot(name)function statements_stats_diff_htbl(jsonb,integer)function statements_stats_htbl(jsonb,integer)function statements_stats(integer,integer,integer,integer)function tablespaces_stats_diff_htbl(jsonb,integer)function tablespaces_stats_htbl(jsonb,integer)function tablespace_stats(integer,integer,integer)function take_sample()function take_sample(integer,boolean)function take_sample(name,boolean)function take_sample_subset(integer,integer)function tbl_top_dead_htbl(jsonb,integer)function tbl_top_fetch_diff_htbl(jsonb,integer)function tbl_top_fetch_htbl(jsonb,integer)function tbl_top_io_diff_htbl(jsonb,integer)function tbl_top_io_htbl(jsonb,integer)function tbl_top_mods_htbl(jsonb,integer)function template_populate_sections(jsonb,integer,text,integer)function top_analyzed_tables_diff_htbl(jsonb,integer)function top_analyzed_tables_htbl(jsonb,integer)function top_cpu_time_diff_htbl(jsonb,integer)function top_cpu_time_htbl(jsonb,integer)function top_dml_tables_diff_htbl(jsonb,integer)function top_dml_tables_htbl(jsonb,integer)function top_elapsed_diff_htbl(jsonb,integer)function top_elapsed_htbl(jsonb,integer)function top_exec_diff_htbl(jsonb,integer)function top_exec_htbl(jsonb,integer)function top_exec_time_diff_htbl(jsonb,integer)function top_exec_time_htbl(jsonb,integer)function top_functions(integer,integer,integer,boolean)function top_growth_indexes_diff_htbl(jsonb,integer)function top_growth_indexes_htbl(jsonb,integer)function top_growth_tables_diff_htbl(jsonb,integer)function top_growth_tables_htbl(jsonb,integer)function top_indexes(integer,integer,integer)function top_io_filesystem_diff_htbl(jsonb,integer)function top_io_filesystem_htbl(jsonb,integer)function top_io_indexes(integer,integer,integer)function top_io_tables(integer,integer,integer)function top_iowait_diff_htbl(jsonb,integer)function top_iowait_htbl(jsonb,integer)function top_jit_diff_htbl(jsonb,integer)function top_jit_htbl(jsonb,integer)function top_kcache_statements(integer,integer,integer)function top_plan_time_diff_htbl(jsonb,integer)function top_plan_time_htbl(jsonb,integer)function top_scan_tables_diff_htbl(jsonb,integer)function top_scan_tables_htbl(jsonb,integer)function top_shared_blks_fetched_diff_htbl(jsonb,integer)function top_shared_blks_fetched_htbl(jsonb,integer)function top_shared_dirtied_diff_htbl(jsonb,integer)function top_shared_dirtied_htbl(jsonb,integer)function top_shared_reads_diff_htbl(jsonb,integer)function top_shared_reads_htbl(jsonb,integer)function top_shared_written_diff_htbl(jsonb,integer)function top_shared_written_htbl(jsonb,integer)function top_statements(integer,integer,integer)function top_tables(integer,integer,integer)function top_temp_diff_htbl(jsonb,integer)function top_temp_htbl(jsonb,integer)function top_upd_vac_tables_diff_htbl(jsonb,integer)function top_upd_vac_tables_htbl(jsonb,integer)function top_vacuumed_indexes_diff_htbl(jsonb,integer)function top_vacuumed_indexes_htbl(jsonb,integer)function top_vacuumed_tables_diff_htbl(jsonb,integer)function top_vacuumed_tables_htbl(jsonb,integer)function top_wait_sampling_events_diff_htbl(jsonb,integer)function top_wait_sampling_events_htbl(jsonb,integer)function top_wal_size_diff_htbl(jsonb,integer)function top_wal_size_htbl(jsonb,integer)function wait_sampling_totals_diff_htbl(jsonb,integer)function wait_sampling_totals_htbl(jsonb,integer)function wait_sampling_total_stats(integer,integer,integer)function wal_stats_diff_htbl(jsonb,integer)function wal_stats_htbl(jsonb,integer)function wal_stats(integer,integer,integer)function wal_stats_reset_diff_htbl(jsonb,integer)function wal_stats_reset_htbl(jsonb,integer)function wal_stats_reset(integer,integer,integer)sequence baselines_bl_id_seqsequence servers_server_id_seqtable baselinestable bl_samplestable funcs_listtable import_queriestable import_queries_version_ordertable indexes_listtable last_stat_archivertable last_stat_clustertable last_stat_databasetable last_stat_database_srv1table last_stat_indexestable last_stat_indexes_srv1table last_stat_kcachetable last_stat_kcache_srv1table last_stat_statementstable last_stat_statements_srv1table last_stat_tablestable last_stat_tablespacestable last_stat_tablespaces_srv1table last_stat_tables_srv1table last_stat_user_functionstable last_stat_user_functions_srv1table last_stat_waltable reporttable report_statictable report_structtable roles_listtable sample_kcachetable sample_kcache_totaltable samplestable sample_settingstable sample_stat_archivertable sample_stat_clustertable sample_stat_databasetable sample_statementstable sample_statements_totaltable sample_stat_indexestable sample_stat_indexes_totaltable sample_stat_tablestable sample_stat_tablespacestable sample_stat_tables_totaltable sample_stat_user_functionstable sample_stat_user_func_totaltable sample_stat_waltable sample_timingstable serverstable stmt_listtable tables_listtable tablespaces_listtable wait_sampling_totalview v_sample_settingsview v_sample_stat_indexesview v_sample_stat_tablesview v_sample_stat_tablespacesview v_sample_stat_user_functionsview v_sample_timings
(257 rows)

3.pg_profile使用

1.打开一个psql命令行窗口，并使用数据库管理账户登录数据库，执行数据库的快照
antdb=# select profile.snapshot();    snapshot        
------------------------(local,OK,00:00:01.45)
(1 row)提示：为了使生成的数据库性能报告更加准确，可以在执行PL/PGSQL程序的过程中多生成几次快照信息。2.查询生成的快照信息
antdb=# select profile.show_samples();      show_samples            
-----------------------------------(1,"2024-03-13 17:00:48+08",t,,,)(2,"2024-03-13 17:02:23+08",t,,,)
(2 rows)3.生成数据库性能报告
psql -d antdb -qtc "select profile.get_report(1,2)" --output ~/awr_report_antdb_1_2.html提示：pg_profile可以和pg_cron扩展一起使用，使用pg_cron扩展定义数据库定时任务去每半分钟生成一次快照信息。后期方便排查数据库问题。

4.报告分析

Server statistics：服务器的统计信息，包含整个数据库在此快照运行期间的相关统计信息，如事务数、内存命中率、元组的操作统计数据、数据库调用次数、数据库集群的统计信息和表空间信息。
SQL query statistics：SQL查询的统计信息，主要包含Top SQL的相关信息，如执行时长、执行次数、执行消耗的I/O，以及逻辑读信息和完整的SQL语句。更具Query ID可以查看具体的SQL语句。
Schema object statistics：模式对象的统计信息，这里主要包含访问频率最高的对象的信息，根据这部分信息可以定位到DML操作最频繁的表和索引、以及无效的索引等。
Vacuum-related statistics：VACUUM相关的统计信息。
Cluster settings during the report interval：报告快照期间的参数设置。