问题描述
之前在ubuntu16上对Clickhouse21.12.1.1的基础上改了一些代码，近期把操作系统升级到unbutu 22.04后，发现编译不通过，具体报错为：
[ 22%] Linking CXX shared library libclickhouse_common_iod.so
/home/herry/dev/c++/ClickHouse/src/Common/ThreadFuzzer.cpp:293: error: undefined reference to '__pthread_mutex_lock'
/home/herry/dev/c++/ClickHouse/src/Common/ThreadFuzzer.cpp:293: error: undefined reference to '__pthread_mutex_unlock'
分析问题
没有链接pthread？
因为连接libckickhouse_common_iod.so时找不到__pthread_mutex_lock，首先考虑的是没有连接pthread，在src/CMakeLists.txt中添加
target_link_libraries(clickhouse_common_io PUBLIC pthread)
编译仍然提示同样的错误
用make VERBOSE=1可以看到libclickhouse_common_iod.so的链接过程中，已经有-lpthread了，看起来不是这个问题了。
查看pthread的符号表
在ubuntu16中
herry@herrypc:/lib/x86_64-linux-gnu$ nm -D libpthread.so.0 |grep pthread_mutex_lock
000000000000b650 T __pthread_mutex_lock@@GLIBC_2.2.5
000000000000b650 W pthread_mutex_lock@@GLIBC_2.2.5
可以看到pthread_mutex_lock和__pthread_mutex_lock都在libpthread.so.0中
在ubuntu22中
herry@inc:/lib/x86_64-linux-gnu$ nm -D libpthread.so.0 |grep pthread_mutex_lock
herry@inc:/lib/x86_64-linux-gnu$
herry@inc:/lib/x86_64-linux-gnu$ readelf -s libpthread.so.0 |grep pthread_mutex_
herry@inc:/lib/x86_64-linux-gnu$
发现libpthread.so.0中已经没有pthread_mutex_lock和__pthread_mutex_lock了。
glibc2.34 removed libpthread
参考文章[glibc2.34 removed libpthread
(https://developers.redhat.com/articles/2021/12/17/why-glibc-234-removed-libpthread#)。
从glibc2.34开始，glibc已经把pthread的功能基本都去掉了，剩下的libpthread.so基本上是个空的，只是为了让编译适配早期版本。
所以虽然我们有-lpthread，但是在其中仍然找不到__pthread_mutex_lock/unlock符号。
查看glibc版本中的内容
glibc版本号查看
方法1：
herry@inc:/lib/x86_64-linux-gnu$ ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3) 2.35
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
方法2：
root@inc:/usr/lib/x86_64-linux-gnu# chmod +x libc.so.6
root@inc:/usr/lib/x86_64-linux-gnu# /lib/x86_64-linux-gnu/libc.so.6
GNU C Library (Ubuntu GLIBC 2.35-0ubuntu3) stable release version 2.35.
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 11.2.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
For bug reporting instructions, please see:
<https://bugs.launchpad.net/ubuntu/+source/glibc/+bugs>.
两个方法都看出在ubuntu22.04上的glibc版本已经升级到2.35。
查看ubuntu22.04上的libc的符号列表
root@inc:/usr/lib/x86_64-linux-gnu# nm -D /lib/x86_64-linux-gnu/libc.so.6 |grep pthread_mutex_lock
0000000000097f70 T __pthread_mutex_lock@GLIBC_2.2.5
0000000000097f70 T pthread_mutex_lock@@GLIBC_2.2.5
果然发现了__pthread_mutex_lock符号。
解决问题
在clickhouse的问题列表中，找到了Fix undefined __pthread_mutex_lock/unlock for glibc，果然在新版本中，已经对该问题就是修复。
增加了针对glibc不同版本的适配代码。
/// Starting from glibc 2.34 there are no internal symbols without version,
/// so not __pthread_mutex_lock but __pthread_mutex_lock@2.2.5
#if defined(OS_LINUX) and !defined(USE_MUSL)
/// You can get version from glibc/sysdeps/unix/sysv/linux/$ARCH/$BITS_OR_BYTE_ORDER/libc.abilist
#if defined(__amd64__)
# define GLIBC_SYMVER "GLIBC_2.2.5"
#elif defined(__aarch64__)
# define GLIBC_SYMVER "GLIBC_2.17"
#elif defined(__riscv) && (__riscv_xlen == 64)
# define GLIBC_SYMVER "GLIBC_2.27"
#elif (defined(__PPC64__) || defined(__powerpc64__)) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
# define GLIBC_SYMVER "GLIBC_2.17"
#else
# error Your platform is not supported.
#endif
#define GLIBC_COMPAT_SYMBOL(func) __asm__(".symver " #func "," #func "@" GLIBC_SYMVER);
GLIBC_COMPAT_SYMBOL(__pthread_mutex_unlock)
GLIBC_COMPAT_SYMBOL(__pthread_mutex_lock)
#endif
把这段代码拷贝到我的工作代码中，编译，果然能够能够成功了。
