linux 上一些软件包的安装,往往需要很多手工介入的安装步骤,不利于进行大规模集群部署,而且免不了对一些软件需要定制化改造的需求,需要源码安装。如果需要一台台地去编译安装,那么运维就不用出机房了
文章以mysql打包构建为例,实现 基于源码构建定制化rpm包 和基于 二进制安装包构建rpm包 两种方式的的打包
打包好了,将该rpm包发布到构建的本地yum源,集群中所有机器一个yum install 命令就可以搞定了。如何在本地搭建yum源,可以见上一篇文章:自动化部署 - 构建YUM本地源
准备 安装需求包
1 yum -y install rpm-build redhat-rpm-config gcc gcc-c++ cmake make zlib-devel openssl-devel
创建rpmbuild构建目录:
1 2 mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS} echo "%_topdir $HOME /rpmbuild" > ~/.rpmmacros
SPEC 文件分析 (1)文件头
一般的spec文件头包含以下几个域:
Summary:用一句话概括该软件包尽量多的信息。
Name:软件包的名字,最终RPM软件包是用该名字与版本号,释出号及体系号来命名软件包的。
Version:软件版本号。仅当软件包比以前有较大改变时才增加版本号。
Release: 软件包释出号。一般我们对该软件包做了一些小的补丁的时候就应该把释出号加1。
Vendor: 软件开发者的名字。
Copyright: 软件包所采用的版权规则。具体有:GPL(自由软件),BSD,MIT,Public Domain(公共域),Distributable(贡献),commercial(商业),Share(共享)等,一般的开发都写GPL。
Group: 软件包所属类别,具体类别有:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Amusements/Games (娱乐/游戏) Amusements/Graphics(娱乐/图形) Applications/Archiving (应用/文档) Applications/Communications(应用/通讯) Applications/Databases (应用/数据库) Applications/Editors (应用/编辑器) Applications/Emulators (应用/仿真器) Applications/Engineering (应用/工程) Applications/File (应用/文件) Applications/Internet (应用/因特网) Applications/Multimedia(应用/多媒体) Applications/Productivity (应用/产品) Applications/Publishing(应用/印刷) Applications/System(应用/系统) Applications/Text (应用/文本) Development/Debuggers (开发/调试器) Development/Languages (开发/语言) Development/Libraries (开发/函数库) Development/System (开发/系统) Development/Tools (开发/工具) Documentation (文档) System Environment/Base(系统环境/基础) System Environment/Daemons (系统环境/守护) System Environment/Kernel (系统环境/内核) System Environment/Libraries (系统环境/函数库) System Environment/Shells (系统环境/接口) User Interface/Desktops(用户界面/桌面) User Interface/X (用户界面/X窗口) User Interface/X Hardware Support (用户界面/X硬件支持)
Source:源程序软件包的名字。如 stardict-2.0.tar.gz。
%description:软件包详细说明,可写在多个行上。
2)%prep段
这个段是预处理段,通常用来执行一些解开源程序包的命令,为下一步的编译安装作准备。%prep和下面的%build,%install段一样,除了可以执行RPM所定义的宏命令(以%开头)以外,还可以执行SHELL命令,命令可以有很多行,如我们常写的tar解包命令。
(3)build段
本段是建立段,所要执行的命令为生成软件包服务,如make 命令。
(4)%install段
本段是安装段,其中的命令在安装软件包时将执行,如make install命令。
(5)%files段
本段是文件段,用于定义软件包所包含的文件,分为三类–说明文档(doc),配置文件(config)及执行程序,还可定义文件存取权限,拥有者及组别。
(6)%changelog段
本段是修改日志段。你可以将软件的每次修改记录到这里,保存到发布的软件包中,以便查询之用。每一个修改日志都有这样一种格式:第一行是:* 星期 月 日 年 修改人 电子信箱。其中:星期、月份均用英文形式的前3个字母,用中文会报错。接下来的行写的是修改了什么地方,可写多行。一般以减号开始,便于后续的查阅。
示例一:MYSQL 源码编译打包rpm 源码编译过程
源码–>预处理–>编译–>汇编–>链接–>执行
1 2 3 ./configure make make install
源码编译打包rpm,所以在rpm 的 build
段,需要完成生成mysql软件包服务,包含cmake和make操作。在install
阶段,完成 make install操作。具体spec文件如下
cat mysql.spec
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 Name: ifengkou-mysql Version:5.7.20 Release: 18 License: GPL Group: applications/database URL: https://www.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.20-18/source /tarball/percona-server-5.7.20-18.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root BuildRequires: cmake Requires: coreutils,shadow-utils Packager: [email protected] Autoreq: no Source: percona-server-5.7.20-18.tar.gz prefix: /opt/soft/ifengkou_mysql-%{version}-%{release} Summary: percona server 5.7.20-18 %description Percona Server is an enhanced, drop-in MySQL® replacement which has been downloaded more than 1,000,000 times . A free open source solution, Percona Server is a MySQL alternative which offers breakthrough performance, scalability, features, and instrumentation. Self-tuning algorithms and support for extremely high-performance hardware make it the clear choice for organizations that demand excellent performance and reliability from their MySQL database server. %define MYSQL_USER mysql %define MYSQL_GROUP mysql %define __os_install_post %{nil} %prep %setup -n %{name}-%{version}-%{release} %build cmake . \ -DSYSCONFDIR:PATH=%{prefix} \ -DCMAKE_INSTALL_PREFIX:PATH=%{prefix} \ -DCMAKE_BUILD_TYPE:STRING=Release \ -DENABLE_PROFILING:BOOL=ON \ -DWITH_DEBUG:BOOL=OFF \ -DWITH_VALGRIND:BOOL=OFF \ -DENABLE_DEBUG_SYNC:BOOL=OFF \ -DWITH_EXTRA_CHARSETS:STRING=all \ -DWITH_SSL:STRING=bundled \ -DWITH_UNIT_TESTS:BOOL=OFF \ -DWITH_ZLIB:STRING=bundled \ -DWITH_PARTITION_STORAGE_ENGINE:BOOL=ON \ -DWITH_INNOBASE_STORAGE_ENGINE:BOOL=ON \ -DWITH_ARCHIVE_STORAGE_ENGINE:BOOL=ON \ -DWITH_BLACKHOLE_STORAGE_ENGINE:BOOL=ON \ -DWITH_PERFSCHEMA_STORAGE_ENGINE:BOOL=ON \ -DDEFAULT_CHARSET=utf8 \ -DDEFAULT_COLLATION=utf8_general_ci \ -DENABLED_LOCAL_INFILE:BOOL=ON \ -DWITH_EMBEDDED_SERVER=0 \ -DINSTALL_LAYOUT:STRING=STANDALONE \ -DCOMMUNITY_BUILD:BOOL=ON \ -DWITHOUT_EXAMPLE_STORAGE_ENGINE=1 \ -DWITHOUT_NDBCLUSTER_STORAGE_ENGINE=1 \ -DENABLED_PROFILING=1 \ -DINNODB_PAGE_ATOMIC_REF_COUNT=1 \ -DWITH_INNODB_MEMCACHED=1 \ -DDOWNLOAD_BOOST=1 \ -DWITH_BOOST=/usr/local /boost_1_59_0; make -j `cat /proc/cpuinfo | grep processor| wc -l` %install echo "-----------------" $RPM_BUILD_ROOT make DESTDIR=$RPM_BUILD_ROOT install cp %{_sourcedir}/my.cnf $RPM_BUILD_ROOT %{prefix}/ %clean rm -rf $RPM_BUILD_DIR /* %files %defattr(-, %{MYSQL_USER}, %{MYSQL_GROUP}) %attr(755, %{MYSQL_USER}, %{MYSQL_GROUP}) %{prefix}/* %pre if ! id %{MYSQL_USER} > /dev/null 2>&1;then useradd -M -s /sbin/nologin %{MYSQL_USER} fi %post if [ -f %{prefix}/support-files/mysql.server > /dev/null 2>&1 ] && [ ! -f %{_initddir}/mysql > /dev/null 2>&1 ];then cp -f %{prefix}/support-files/mysql.server %{_initddir}/mysql chmod +x %{_initddir}/mysql chkconfig --level 2345 %{_initddir}/mysql on fi if [ ! -f %{_sysconfdir}/my.cnf ];then cp %{prefix}/my.cnf %{_sysconfdir}/my.cnf else cp %{prefix}/my.cnf %{_sysconfdir}/my.cnf.rpmnew fi %preun if [ -f %{_sysconfdir}/my.cnf ];then mv %{_sysconfdir}/my.cnf %{_sysconfdir}/my.cnf.rpmold fi if [ -f %{_initddir}/mysql ];then mv %{_initddir}/mysql %{_initddir}/mysql.rpmold fi %postun rm -rf %{prefix} userdel -r %{MYSQL_USER} >/dev/null 2>&1 %changelog
mysql源码包下载(下载地址 ),然后拷贝到~/rpmbuild/SOURCES
目录,然后使用rpmbuild进行构建
1 2 3 cp percona-server-5.6.19-67.0.tar.gz ~/rpmbuild/SOURCES/ cp my.cnf ~/rpmbuild/SOURCES/ rpmbuild -bb mysql.spec
make 编译有点慢,需要几十分钟,还好这种软件只需要搞定一次,后续rpm 包安装的时候只需要make DESTDIR=$RPM_BUILD_ROOT install
,拷贝文件到目标文件夹,这个还是非常快的。
rpmbuild -bb xxx.spec 只生成二进制格式的rpm包
rpmbuild -bs xxx.spec 只生成src格式的rpm包
rpmbuild -bp xxx.spec 只需要生成完整的源文件
rpmbuild -ba xxx.spec 完全打包,产生以上3个过程分别生成的包
安装过程中可能出现的一些异常
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 CMake Error at cmake/readline.cmake:66 (MESSAGE): Curses library not found. Please install appropriate package, remove CMakeCache.txt and rerun cmake.On Debian/Ubuntu, package name is libncurses5-dev, on Redhat and derivates it is ncurses-devel. Call Stack (most recent call first): cmake/readline.cmake:190 (FIND_CURSES) cmake/readline.cmake:264 (FIND_SYSTEM_READLINE) CMakeLists.txt:538 (MYSQL_CHECK_EDITLINE) 解决:yum install ncurses-devel CMake Error at cmake/readline.cmake:266 (MESSAGE): Cannot find system readline libraries. Call Stack (most recent call first): CMakeLists.txt:538 (MYSQL_CHECK_EDITLINE) 解决:yum install -y readline-devel CMake Error at plugin/keyring_vault/CMakeLists.txt:18 (message): Not building keyring_vault, could not find library: CURL Call Stack (most recent call first): plugin/keyring_vault/CMakeLists.txt:23 (CHECK_IF_LIB_FOUND) 解决:yum install libcurl-devel
如果make过程中,出现编译失败,很可能是内存不足导致,释放些内存就好了
最后生成的rpm包:
1 ~/rpmbuild/RPMS/x86_64/ifengkou_mysql-5.7.20-18.x86_64.rpm
示例二:MYSQL 二进制包打包成rpm(推荐) 示例一因为有 make步骤,所以打包过程十分慢。定制化mysql,诉求一般是安装在指定位置,使用制定配置,安装卸载方便。所以并不需要从源码开始构建,直接从官方下载最新的二进制安装包
https://dev.mysql.com/downloads/mysql/5.7.html#downloads
下载 mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz,拷贝到 ~/rpmbuild/SOURCES目录,我在这做了次重命名,所以需要先解压,重命名文件夹,再打包。
1 2 3 4 5 6 7 8 9 wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz tar -xvzf mysql-5.7.22-linux-glibc2.12-x86_64.tar.gz mv mysql-5.7.22-linux-glibc2.12-x86_64/* ifengkou-mysql-5.7.22/ tar czf ifengkou-mysql-5.7.22.tar.gz ifengkou-mysql-5.7.22 mv ifengkou-mysql-5.7.22.tar.gz ~/rpmbuild/SOURCES/
修改下spec 文件,cat mysql-5.7.22.spec
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Summary : ifengkou mysql Name : ifengkou-mysql Version : 5.7.22 Release : 1 License : GPL Source : ifengkou-mysql-5.7.22.tar.gz BuildArch : x86_64 BuildRoot : root AutoReqProv: no AutoReq: no Packager : [email protected] %define debug_package %{nil} %description This package is just a mysql RPM. %prep rm -rf /opt/soft/ifengkou-mysql %setup -q %build %install echo $RPM_BUILD_ROOT mkdir -p $RPM_BUILD_ROOT /opt/soft/ifengkou-mysql install -d -m 755 $RPM_BUILD_ROOT /opt/soft/ifengkou-mysql cp -rf * $RPM_BUILD_ROOT /opt/soft/ifengkou-mysql %post rm -rf /etc/init.d/ifengkou-mysqld /etc/my.cnf cp /opt/soft/ifengkou-mysql/support-files/mysql.server /etc/init.d/ifengkou-mysqld cp /opt/soft/ifengkou-mysql/support-files/my.cnf /etc/my.cnf rm -rf /etc/profile.d/ifengkou-mysql.sh echo "export IFENGKOU_MYSQL_HOME=/opt/soft/ifengkou-mysql" >> /etc/profile.d/ifengkou-mysql.shecho "export PATH=\$IFENGKOU_MYSQL_HOME/bin:\$PATH" >> /etc/profile.d/ifengkou-mysql.shchmod +x /etc/init.d/ifengkou-mysqld useradd mysql chown -R mysql:mysql /opt/soft/ifengkou-mysql/data mkdir /var/log /ifengkou-mysql touch /var/log /ifengkou-mysql/mysql.log chown -R mysql.mysql /var/log /ifengkou-mysql mkdir /var/run/ifengkou-mysql chown -R mysql.mysql /var/run/ifengkou-mysql %check %files /opt/soft/ifengkou-mysql %postun rm -rf /opt/soft/ifengkou-mysql rm -rf /etc/my.cnf rm -rf /etc/profile.d/ifengkou-mysql.sh rm -rf /var/log /ifengkou-mysql rm -rf /var/run/ifengkou-mysql
最后打成了一个 rpmbuild/RPMS/x86_64/ifengkou-mysql-5.7.20-1.x86_64.rpm ,将rpm倒入到yum本地源后,直接 yum安装即可,这样在mysql 安装的时候就省去了很多手动配置参数的步骤,后续安装后启动 下面还包含了mysql-connector-java.jar 包的处理
1 2 3 4 5 6 7 8 9 10 11 12 13 echo "安装新的mysql..." echo "yum -y install libaio libnuma* ifengkou-mysql -d 0 -e 0" yum -y install libaio libnuma* ifengkou-mysql -d 0 -e 0 rsync -avvP ./tool/mysql-connector-java-5.1.37.jar /usr/share/java/ if [ -h "/usr/share/java/mysql-connector-java.jar" ];then rm -f /usr/share/java/mysql-connector-java.jar fi ln -s /usr/share/java/mysql-connector-java-5.1.37.jar /usr/share/java/mysql-connector-java.jar if [ $? = 0 ];then echo "install mysql sucess" else echo "install mysql failed" fi
安装后启动
1 2 3 4 5 6 7 8 9 10 11 12 13 14 sed -i '/innodb_buffer_pool_size/c\innodb_buffer_pool_size=32G' /etc/my.cnf sed -i '/max_allowed_packet/c\max_allowed_packet=25M' /etc/my.cnf /sbin/chkconfig --add ifengkou-mysqld systemctl enable ifengkou-mysqld.service /etc/init.d/ifengkou-mysqld start if [ $? = 0 ];then echo "start mysql sucess" else echo "start mysql failed" fi . /etc/profile.d/ifengkou-mysql.sh
参考 Linux 源码编译安装MySQL 5.7https://itbilu.com/database/mysql/VJVOut01M.html
打包mysql成rpm包https://my.oschina.net/anthonyyau/blog/287063
RPM 打包技术与典型 SPEC 文件分析https://www.ibm.com/developerworks/cn/linux/l-rpm/index.html