轻量级锁实现1——结构体解析、初始化

瀚高数据库
目录
环境
文档用途
详细信息

环境
系统平台:Linux x86-64 Red Hat Enterprise Linux 7
版本:14
文档用途
从底层理解轻量级锁的实现,从保护共享内存的角度理解轻量级锁的使用场景,包括上锁、等待、释放,理解轻量级锁的互斥(execlusive)和共享(shared)2种状态。

详细信息
1.轻量级锁空间分配
锁作为并发场景下使用的一种进程间的同步机制,因此有必要将其放在共享内存中。

/** Set up shared memory and semaphores.** Note: if using SysV shmem and/or semas, each postmaster startup will* normally choose the same IPC keys.  This helps ensure that we will* clean up dead IPC objects if the postmaster crashes and is restarted.*/CreateSharedMemoryAndSemaphores();

image.png

以下代码是轻量级锁的初始化代码:LWLock系统通常由一个主要的LWLock数组组成,每个数组元素对应一个具体的锁资源。而"tranche" 是指将LWLock数组进一步分割成更小的单元,以提供更细粒度的并发控制。

/** Allocate shmem space for the main LWLock array and all tranches and* initialize it.  We also register extension LWLock tranches here.*/void CreateLWLocks(void){if (!IsUnderPostmaster) // 主进程或者是standlone进程{Size		spaceLocks = LWLockShmemSize(); int		   *LWLockCounter;char	   *ptr;/* Allocate space */ptr = (char *) ShmemAlloc(spaceLocks);/* Leave room for dynamic allocation of tranches */ptr += sizeof(int);/* Ensure desired alignment of LWLock array */ptr += LWLOCK_PADDED_SIZE - ((uintptr_t) ptr) % LWLOCK_PADDED_SIZE;MainLWLockArray = (LWLockPadded *) ptr;/** Initialize the dynamic-allocation counter for tranches, which is* stored just before the first LWLock.*/LWLockCounter = (int *) ((char *) MainLWLockArray - sizeof(int));*LWLockCounter = LWTRANCHE_FIRST_USER_DEFINED;/* Initialize all LWLocks */InitializeLWLocks();}/* Register named extension LWLock tranches in the current process. */for (int i = 0; i < NamedLWLockTrancheRequests; i++)LWLockRegisterTranche(NamedLWLockTrancheArray[i].trancheId,NamedLWLockTrancheArray[i].trancheName);}

1.1计算轻量级锁数组占用的共享内存的存储空间的大小:

/** Compute shmem space needed for LWLocks and named tranches.*/Size LWLockShmemSize(void){Size		size;int			i;//NUM_FIXED_LWLLOCKS是pg内核使用的锁的数量,写死的数量int			numLocks = NUM_FIXED_LWLOCKS;//加上其它模块比如动态库请求的锁的数量,例如pg_stat_statements在请求共享内存的钩子函数的代码/*static voidpgss_shmem_request(void){if (prev_shmem_request_hook)prev_shmem_request_hook();RequestAddinShmemSpace(pgss_memsize());RequestNamedLWLockTranche("pg_stat_statements", 1);}*//* Calculate total number of locks needed in the main array. */numLocks += NumLWLocksForNamedTranches();/* Space for the LWLock array. */size = mul_size(numLocks, sizeof(LWLockPadded)); // 1.2/* Space for dynamic allocation counter, plus room for alignment. */size = add_size(size, sizeof(int) + LWLOCK_PADDED_SIZE);/* space for named tranches. */size = add_size(size, mul_size(NamedLWLockTrancheRequests, sizeof(NamedLWLockTranche)));/* space for name of each tranche. */for (i = 0; i < NamedLWLockTrancheRequests; i++)size = add_size(size, strlen(NamedLWLockTrancheRequestArray[i].tranche_name) + 1);return size;}

1.2 LWLockPadded结构体解析

/** In most cases, it's desirable to force each tranche of LWLocks to be aligned* on a cache line boundary and make the array stride a power of 2.  This saves* a few cycles in indexing, but more importantly ensures that individual* LWLocks don't cross cache line boundaries.  This reduces cache contention* problems, especially on AMD Opterons.  In some cases, it's useful to add* even more padding so that each LWLock takes up an entire cache line; this is* useful, for example, in the main LWLock array, where the overall number of* locks is small but some are heavily contended.*/通俗一点说,为什么要设计一个LWLockPadded的union?因为union成员共享存储空间,所以整个union占用的空间大小将为LWLOCK_PADDED_SIZE的大小(这个已经断言过了)。LWLOCK_PADDED_SIZE的大小是cpu一级缓存的大小,所以结果将是LWLock独占一个缓存行,也就是CPU一次读取的单位大小。这么做的好处是提高性能:CPU使用分布式一致性缓存协议MESI(当然实际情况不止这四种状态),当CPU1要改动LWLock,那么对其它的CPU会发送read invalidate消息,其它CPU如果有LWLock在其缓存行,将会清空。如果CPU0下次要修改LWLock,同样也会对其它CPU发送read invalidate消息。但是如果缓存行保存了非LWLock内容,那么对非LWLock内容的修改操作,将会产生invalidate影响(false sharing).因此将其扩充到1个缓存行大小,这样避免其它变量对该LWLock的影响。#define LWLOCK_PADDED_SIZE	PG_CACHE_LINE_SIZEStaticAssertDecl(sizeof(LWLock) <= LWLOCK_PADDED_SIZE,"Miscalculated LWLock padding");/* LWLock, padded to a full cache line size */typedef union LWLockPadded{LWLock		lock;char		pad[LWLOCK_PADDED_SIZE];} LWLockPadded;

举例说明:读取物理核心数,将每个线程绑定一个物理cpu,本机只有cpu0~cpu3,然后对atomics进行单字节的修改,随着线程增多,每个线程的执行时间会随时间增长,因为cpu重复清空(invalidate)和加载(read -> modified)。如果是根据业务将数据分在不同的cache line,那么效率会有提升。

#define _GNU_SOURCE#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <pthread.h>#include <stdatomic.h>#include <time.h>#include <assert.h>#include <sched.h>#define CL_SIZE 64 pthread_barrier_t barrier;pthread_mutex_t sync_mutex;atomic_long nsSum;int sync_count;struct CacheLine {atomic_char atomics[CL_SIZE];};// this function used to get cache line size from runtime/** unsigned int get_cache_line_size_x86() {unsigned int cache_line_size = 0;// Execute CPUID instruction with EAX=0x80000006 to get cache information__asm__ volatile ("mov $0x80000006, %%eax\n\t""cpuid\n\t""mov %%ecx, %0\n\t": "=r" (cache_line_size)  // Output: store cache line size in variable:                         // No input: "%eax", "%ebx", "%ecx", "%edx"  // Clobbered registers);// Extract cache line size from bit 0 to 7 of ECX registercache_line_size = cache_line_size & 0xFF;return cache_line_size;}*/// thread function void* threadFunc(void* arg) {struct CacheLine* cacheLine = (struct CacheLine*)arg;pthread_barrier_wait(&barrier);pthread_mutex_lock(&sync_mutex);// .....pthread_mutex_unlock(&sync_mutex);struct timespec start, end;clock_gettime(CLOCK_MONOTONIC, &start);for (size_t r = 10000000LL; r > 0; --r) {atomic_fetch_add_explicit(&cacheLine->atomics[r % CL_SIZE], 1, memory_order_relaxed);}clock_gettime(CLOCK_MONOTONIC, &end);int64_t diff = (end.tv_sec - start.tv_sec) * 1000000000LL + (end.tv_nsec - start.tv_nsec);atomic_fetch_add_explicit(&nsSum, diff, memory_order_relaxed);return NULL;}int main() {int hc = sysconf(_SC_NPROCESSORS_ONLN);cpu_set_t cs;// loops just for counterfor (int nThreads = 1; nThreads <= hc; ++nThreads){pthread_t threads[nThreads];struct CacheLine cacheLine;nsSum = 0;pthread_barrier_init(&barrier, NULL, nThreads);pthread_mutex_init(&sync_mutex, NULL);sync_count = nThreads;// loops for threads numberfor (int t = 0; t < nThreads; ++t) {pthread_create(&threads[t], NULL, threadFunc, &cacheLine);CPU_ZERO(&cs);CPU_SET(t, &cs);assert(pthread_setaffinity_np(threads[t], sizeof(cs), &cs) == 0); }for (int t = 0; t < nThreads; ++t) {pthread_join(threads[t], NULL);}pthread_barrier_destroy(&barrier);pthread_mutex_destroy(&sync_mutex);printf("%d: %ld\n", nThreads, (long)(nsSum/(1.0e7 * nThreads) + 0.5));}return 0;}

结果:

image.png
在这里插入图片描述

1.3 轻量级锁在共享内存中的布局
lwlock.jpg

2.轻量级锁的初始化
对于上图的空间顺序进行初始化,主要看下LWLockInitialize的实现:

/** Initialize LWLocks that are fixed and those belonging to named tranches.*/static voidInitializeLWLocks(void){int			numNamedLocks = NumLWLocksForNamedTranches();int			id;int			i;int			j;LWLockPadded *lock;/* Initialize all individual LWLocks in main array */for (id = 0, lock = MainLWLockArray; id < NUM_INDIVIDUAL_LWLOCKS; id++, lock++)LWLockInitialize(&lock->lock, id);/* Initialize buffer mapping LWLocks in main array */lock = MainLWLockArray + BUFFER_MAPPING_LWLOCK_OFFSET;for (id = 0; id < NUM_BUFFER_PARTITIONS; id++, lock++)LWLockInitialize(&lock->lock, LWTRANCHE_BUFFER_MAPPING);/* Initialize lmgrs' LWLocks in main array */lock = MainLWLockArray + LOCK_MANAGER_LWLOCK_OFFSET;for (id = 0; id < NUM_LOCK_PARTITIONS; id++, lock++)LWLockInitialize(&lock->lock, LWTRANCHE_LOCK_MANAGER);/* Initialize predicate lmgrs' LWLocks in main array */lock = MainLWLockArray + PREDICATELOCK_MANAGER_LWLOCK_OFFSET;for (id = 0; id < NUM_PREDICATELOCK_PARTITIONS; id++, lock++)LWLockInitialize(&lock->lock, LWTRANCHE_PREDICATE_LOCK_MANAGER);/** Copy the info about any named tranches into shared memory (so that* other processes can see it), and initialize the requested LWLocks.*/if (NamedLWLockTrancheRequests > 0){char	   *trancheNames;NamedLWLockTrancheArray = (NamedLWLockTranche *)&MainLWLockArray[NUM_FIXED_LWLOCKS + numNamedLocks];trancheNames = (char *) NamedLWLockTrancheArray +(NamedLWLockTrancheRequests * sizeof(NamedLWLockTranche));lock = &MainLWLockArray[NUM_FIXED_LWLOCKS];for (i = 0; i < NamedLWLockTrancheRequests; i++){NamedLWLockTrancheRequest *request;NamedLWLockTranche *tranche;char	   *name;request = &NamedLWLockTrancheRequestArray[i];tranche = &NamedLWLockTrancheArray[i];name = trancheNames;trancheNames += strlen(request->tranche_name) + 1;strcpy(name, request->tranche_name);tranche->trancheId = LWLockNewTrancheId();tranche->trancheName = name;for (j = 0; j < request->num_lwlocks; j++, lock++)LWLockInitialize(&lock->lock, tranche->trancheId);}}}

2.1 LWLock赋初值
分别对LWLock中的成员变量state(包括exclusive、shared等状态)、tranche_id(锁的粒度划分,一个tranche_name下可以有多个lock,每个lock有唯一的tranche_id)、waiters(等待队列,是双向链表实现)进行初始化,当中state成员比较特殊。

/** LWLockInitialize - initialize a new lwlock; it's initially unlocked*/voidLWLockInitialize(LWLock *lock, int tranche_id){pg_atomic_init_u32(&lock->state, LW_FLAG_RELEASE_OK);#ifdef LOCK_DEBUGpg_atomic_init_u32(&lock->nwaiters, 0);#endiflock->tranche = tranche_id;proclist_init(&lock->waiters);}
/** Code outside of lwlock.c should not manipulate the contents of this* structure directly, but we have to declare it here to allow LWLocks to be* incorporated into other data structures.*/对于LWLock不应该外部操作,但是有些结构体需要包含这一LWLock类型,所以把LWLock结构声明放在/src/include/storage/lwlock.htypedef struct LWLock{uint16		tranche;		/* tranche ID */pg_atomic_uint32 state;		     /* state of exclusive/nonexclusive lockers */proclist_head waiters;		     /* list of waiting PGPROCs */#ifdef LOCK_DEBUGpg_atomic_uint32 nwaiters;	     /* number of waiters */struct PGPROC *owner;		     /* last exclusive owner of the lock */#endif} LWLock;

使用lwlock时,会有多个进程进行争用,因此需要保证对该变量的修改原子化,然而在初始化时还未有并发的使用,所以只是简单的赋值,但是该变量要保证修改完成时对其它进程的可见性,所以必须加上volatile关键字,后续每个进程的访问和修改都要从内存读取。

typedef struct pg_atomic_uint32{volatile uint32 value;} pg_atomic_uint32;
/** pg_atomic_init_u32 - initialize atomic variable** Has to be done before any concurrent usage..** No barrier semantics.*/static inline voidpg_atomic_init_u32(volatile pg_atomic_uint32 *ptr, uint32 val){AssertPointerAlignment(ptr, 4);pg_atomic_init_u32_impl(ptr, val);}

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.hqwc.cn/news/60163.html

如若内容造成侵权/违法违规/事实不符,请联系编程知识网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

MongoDB SQL

Microsoft Windows [版本 6.1.7601] 版权所有 (c) 2009 Microsoft Corporation。保留所有权利。C:\Users\Administrator>cd C:\MongoDB\Server\3.4\binC:\MongoDB\Server\3.4\bin> C:\MongoDB\Server\3.4\bin> C:\MongoDB\Server\3.4\bin>net start MongoDB 请求的…

vue中使用this.$refs获取不到子组件的方法,属性方法都为undefined的解决方法

问题描述 vue2中refs获取不到子组件中的方法&#xff1f;&#xff0c;而获取到的是undefined 原因及解决方案&#xff1a; 第一种、在循环中注册了很多个ref 因为注册了多个ref&#xff0c;获取是不能单单知识refs.xxx&#xff0c;需要使用数组和索引来获取具体一个组件refs[…

游戏类APP的如何设置广告场景最大化用户价值?提升变现收益?

最近几年&#xff0c;游戏类市场的不断增长激发了广告主预算的不断投入&#xff0c;越来越多的游戏类APP通过广告变现的方式增收&#xff0c;并且获得持续上涨的eCPM。 具有潜力的游戏品类参考 根据游戏品类的增长数据和广告收益规模&#xff0c;休闲/模拟/街机均为收益正向的…

linuxARM裸机学习笔记(5)----定时器按键消抖和高精度延时实验

定时器按键消抖 之前的延时消抖&#xff0c;是直接借助delay函数进行的&#xff0c;但是这样会浪费CPU的性能。我们采用延时函数的方式实现&#xff0c;可以实现快进快出。 定时器消抖&#xff0c;必须是在t3的时间点才可以&#xff0c;当在t1,t2的时间点每次进入中断函数都要…

模仿火星科技 基于cesium+ 贴地测量+可编辑

当您进入Cesium的编辑贴地测量世界&#xff0c;下面是一个详细的操作过程&#xff0c;帮助您顺利使用这些功能&#xff1a; 1. 创建提示窗&#xff1a; 启动Cesium应用&#xff0c;地图场景将打开&#xff0c;欢迎您进入编辑模式。在屏幕的一角&#xff0c;一个友好的提示窗将…

【云原生】Kubernetes节点亲和性分配 Pod

目录 1 给节点添加标签 2 根据选择节点标签指派 pod 到指定节点[nodeSelector] 3 根据节点名称指派 pod 到指定节点[nodeName] 4 根据 亲和性和反亲和性 指派 pod 到指定节点 5 节点亲和性权重 6 pod 间亲和性和反亲和性及权重 7 污点和容忍度 8 Pod 拓扑分布约束 官方…

【基础IO】动静态库 {动静态库的创建和使用;动态库的加载;默认优先使用动态链接;为什么要有库;动态链接的优缺点;静态链接的优缺点;一些有趣的库}

动静态库 一、静态库(.a) 1.1 如何创建静态库&#xff1f; 编写源文件与头文件。注意&#xff1a;库的源文件没有main函数&#xff01; 将所有的源文件编译生成目标文件。(如果只提供.h和.o给用户&#xff0c;用户也可以成功编译运行。但这样的做法太过麻烦&#xff0c;且容易…

Maven出现报错 ; Unable to import maven project: See logs for details错误的多种解决方法

问题现象; IDEA版本&#xff1a; Maven 版本 &#xff1a; 3.3.9 0.检查 maven 的设置 &#xff1a;F:\softeware\maven\apache-maven-3.9.3\conf 检查setting.xml 配置 本地仓库<localRepository>F:\softeware\maven\local\repository</localRepository>镜像…

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

Improved Deep Metric Learning with Multi-class N-pair Loss Objective 来源&#xff1a; NIPS’2016NEC Laboratories America 文章目录 Improved Deep Metric Learning with Multi-class N-pair Loss ObjectiveDistance Metric LearningDeep Metric Learning with Multip…

redis的持久化

第一章、redis的持久化 1.1&#xff09;持久化概述 ①持久化可以理解为将数据存储到一个不会丢失的地方&#xff0c;Redis 的数据存储在内存中&#xff0c;电脑关闭数据就会丢失&#xff0c;所以放在内存中的数据不是持久化的&#xff0c;而放在磁盘就算是一种持久化。 ②为…

python实现简单的爬虫功能

前言 Python是一种广泛应用于爬虫的高级编程语言&#xff0c;它提供了许多强大的库和框架&#xff0c;可以轻松地创建自己的爬虫程序。在本文中&#xff0c;我们将介绍如何使用Python实现简单的爬虫功能&#xff0c;并提供相关的代码实例。 如何实现简单的爬虫 1. 导入必要的…

Python中的dataclass:简化数据类的创建

Python中的dataclass是一个装饰器&#xff0c;用于自动添加一些常见的方法&#xff0c;如构造函数、__repr__、__eq__等。它简化了创建数据类的过程&#xff0c;减少了样板代码&#xff0c;提高了代码的可读性和可维护性。有点类似java里面的Java Bean。 让我们看一个简单的例子…