持续集成CI
[Chapter 零] * Ref.
基于内部研发框架pactera jeesit版本 1.2.11-SNAPSHOT 发行之际，引入CI服务Jenkins，并通过In-house Conf.内部大会分享，特此整理成文以资备案和参考。 By Jeffen Apr. 15 , 2016
[Chapter Ⅰ] * Continuous Integration
- 最核心的敏捷（Agile）实践，--持续集成（Continuous Integration），
- 可以快速、高频率、自动、构建、所有源码，并反馈。
- 在不断变化的需求中快速适应和保证软件的质量。
- 倡导团队开发成员必须经常集成他们的工作，甚至每天都可能发生多次集成。
- 而每次的集成都是通过自动化的构建来验证，包括自动编译、发布和测试，从而尽快地发现集成错误，
- 让团队能够更快的开发内聚的软件。
持续集成的核心价值在于：
- 持续集成中的任何一个环节都是自动完成的，无需太多的人工干预，有利于减少重复过程以节省时间、费用和工作量；
- 持续集成保障了每个时间点上团队成员提交的代码是能成功集成的。换言之，任何时间点都能第一时间发现软件的集成问题，使任意时间发布可部署的软件成为了可能；
- 持续集成还能利于软件本身的发展趋势，这点在需求不明确或是频繁性变更的情景中尤其重要，持续集成的质量能帮助团队进行有效决策，同时建立团队对开发产品的信心。
持续集成的原则：
- 需要版本控制软件保障团队成员提交的代码不会导致集成失败。常用的版本控制软件有 IBM Rational ClearCase、CVS、Subversion 等；
- 开发人员必须及时向版本控制库中提交代码，也必须经常性地从版本控制库中更新代码到本地；
- 需要有专门的集成服务器来执行集成构建。根据项目的具体实际，集成构建可以被软件的修改来直接触发，也可以定时启动，如每半个小时构建一次；
7） 必须保证构建的成功。如果构建失败，修复构建过程中的错误是优先级最高的工作。一旦修复，需要手动启动一次构建。
引入Hudson3.3.3持续集成发布定制的框架版本，过程：
- 持续编译，SVN
- 持续数据库脚本过程集成
- 持续测试，Junit
- 持续审查，引入Checkstyle 、PMD持续审查代码BadSmell；引入FindBug持续进行代码auto review；，产出各类check report；
- 持续部署，容器环境相关，Cargo
- 持续反馈，将集成失败报告发给本次集成的相关代码提交者，CC项目经理。
因svn plugin地址中文寻址问题及Ugly UI，放弃hudson，
引入 Jenkins1.656 持续集成，前身是hudson，做了扩展和包装，UI/UX交互友好，apache foundation开源项目。
[Chapter Ⅱ] * Install
2.1 (×)Hudson
- Hudson downloads
- Installing Hudson
- Hudson 3.3.2 (ChangeLog)
- Installing Hudson as a Windows service
2.2 (√)Jenkins
- https://jenkins.io/index.html
- √ recommend, run as windows server by background
- https://jenkins.io/content/thank-you-downloading-windows-installer#stable
- run as terminal
- http://www.ibm.com/developerworks/cn/java/j-lo-jenkins/index.html
2.3 Plugins
2.3.1 FindBugs
2.3.1.1 Description
- 寻找代码缺陷：
- 未关闭的数据库连接；
- 缺少必要的null check;
- 多余的null check;
- 多余的if后置条件；
- 相同的条件分支；
- 重复的代码块；
- 错误的使用了“＝＝”；
- 建议使用stringbuffer代替字符串连加；
2.3.1.2 URL
- http://wiki.hudson-ci.org/display/HUDSON/FindBugs+Plugin
- https://wiki.jenkins-ci.org/display/JENKINS/FindBugs+Plugin
2.3.1.3 Commands
- mvn findbugs:help 查看findbugs插件的帮助
- mvn findbugs:check 检查代码是否通过findbugs检查，如果没有通过检查，检查会失败，但检查不会生成结果报表
- mvn findbugs:findbugs 检查代码是否通过findbugs检查，如果没有通过检查，检查不会失败，会生成结果报表保存在target/findbugsXml.xml文件中
- mvn findbugs:gui 检查代码并启动gui界面来查看结果
2.3.2 PMD
2.3.3 CheckStyle
[Chapter Ⅲ] * Run
3.1 Enviroments
JDK
- %JAVA_HOME% : c:\Program Files\Java\jdk.1.7.0_67
Maven
- %M3_HOME%: e:maven/maven3
- E:\maven\maven3\conf\settings.xml
- 建议repository设定到nexus私服下
- 设定nexus的服务认证以便deploy
<localRepository>D:/sonatype-work/nexus/storage/central</localRepository>
<!-- Nexus servers-->
<server>
<id>releases</id>
<username>admin</username>
<password>admin123</password>
</server>
<server>
<id>snapshots</id>
<username>admin</username>
<password>admin123</password>
</server>
- 如maven无法自动下载，可手动安装ojdbc14
cd E:\maven\repository
e:
mvn install:install-file -Dpackaging=jar -DgroupId=com.oracle -Dversion=10.2.0.4.0 -DartifactId=ojdbc14 -Dfile=ojdbc14-10.2.0.4.0.jar
3.2 (×)Hudson
Path: 192.168.5.5$d:/Hudson/
$ java –jar Hudson-3.3.2.war --httpPort=8082
3.3 (√)Jenkins
Path: 192.168.5.5$d:/jenkins/
$ java -Dhudson.util.ProcessTree.disable=true -jar jenkins.war --httpPort=8082
ProcessTree:
Jenkins ProcessTreeKiller. A more detailed explanation here. It's a design decision to kill any processes that are spawned by the build to maintain a clean environment. Unfortunately that means you can't leave a process (such as Tomcat) running after the build.
You can disable this functionality globally (not recommended) by launching Jenkins
Show:
3.4 URL
Jenkins
http://localhost:8082/
Install plugins on homepage by guidlines.
pactera jeesite
http://192.168.5.5:8080/pactera-jeesite
[Chapter Ⅳ] * Enjoy
4.1 Setting
http://localhost:8082/configure
System Configration>>
- JDK : jdk-1.7 (%JAVA_HOME%)
- Maven : maven-3.1.1 (%M3_HOME%)
- MAVEN_OPTS: -Xms128m –Xmx512m
4.2 Create
4.2.1 Job Configurations
- Build: a free-style software project
- Name: pactera-jeesite-aggregator
- Description：pactera jeesite 1.2.10+ web show
- Max # of builds to keep: 20 (only up to this number of build records are kept)
BUILDS:
- pactera-jeesite-common
- pactera-jeesite-cms
- pactera-jeesite-gen
- pactera-jeesite-oa
- pactera-jeesite-oa-core
- pactera-jeesite-web
- pactera-jeesite-web-dist
4.2.2 Source Code Management
Subversion svn url :
http://192.168.5.5:8088/svn/产品/jeesite/trunk/aggregator
Repository URL:
http://192.168.5.5:8088/svn/产品/jeesite/trunk/aggregator
Click “enter credential / update credential” to subversion authentication.
URLS:
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-common
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-cms
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-gen
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-oa
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-oa-core
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-soa
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-web
- http://192.168.5.5:8088/svn/产品/jeesite/trunk/jeesite-web-dist
4.2.3 Build Triggers
√ Poll SCM
- 第一个参数代表的是分钟 minute，取值 0~59；
- 第二个参数代表的是小时 hour，取值 0~23；
- 第三个参数代表的是天 day，取值 1~31；
- 第四个参数代表的是月 month，取值 1~12；
- 最后一个参数代表的是星期 week，取值 0~7，0 和 7 都是表示星期天。
说明：构建前提是，配置的代码库有更新，再触发。
Schedule :*/59 * * * *(每一个小时轮寻源码仓库，发现更新时构建本项目)
Schedule :00 01 * * 1-5(每周一到周五凌晨一点)
4.2.4 Build
4.2.4.1 Invoke top-level maven targets
- Maven Version: maven-3.1.1
-
Golals(-aggrerator/-web-dist):
clean deploy -Pdev_pdt -Dmaven.test.skip=true -
Golals:
clean deploy -Dmaven.test.skip=true
(×)Hudson
(√)Jenkins
4.2.4.2 (×)Execute Windows batch command type1
For project: pactera-jeesite-web-dist
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
restart.bat
Restart tomcat by %CATALINA_HOME%/bin/restart.bat for Windows:
@echo restart tomcat6 service
@echo Created by Jeffen@pactera
@echo Date 2016/4/7
@echo Version 0.1
@echo 1. stop tomcat6 service
@echo 2. sleep 10 seconds
@echo 3. start tomcat6 service
echo --------------------------------------------------------
echo [%date%%time%] prepare to restart tomcat
echo [%date%%time%] stop tomcat
call "shutdown.bat"
@echo waiting 10 seconds
ping 127.0.0.1 -n 10
echo [%date%%time%] restart tomcat
call "startup.bat"
echo --------------------------------------------------------
echo
4.2.4.3 (√)Execute Windows batch command type2
http://stackoverflow.com/questions/6204003/kill-a-process-by-looking-up-the-port-being-used-by-it-from-a-bat
a. 先杀进程，以防deploy时无法删除target folder：
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
call "kill_process_by_port.bat"
b. 再maven deploy发布项目：
clean deploy -Pdev_pdt -Dmaven.test.skip=true
c. 最后重启tomcat container：
cd D:\apache-tomcat-6.0.20-jeecg\bin
d:
@echo [%date%%time%] restart tomcat
call "startup.bat"
@echo waiting 30 seconds
ping 127.0.0.1 -n 30
Kill tomcat by %CATALINA_HOME%/bin/kill_process_by_port.bat for Windows:
@echo kill tomcat service.
@echo if run by terminal then change %%P to %P
@echo Created by Jeffen@pactera
@echo Date 2016/4/14
@echo Version 0.1
@echo [%date%%time%] kill tomcat container process by port
FOR /F "tokens=5 delims= " %%P IN ('netstat -a -n -o ^| findstr :8080.*LISTENING') DO TaskKill.exe /F /PID %%P
@echo waiting 5 seconds
ping 127.0.0.1 -n 5
HardCopy:
4.2.4.4 Publish FindBugs Analysis results
4.2.4.5 Publish PMD Analysis results
4.2.5 Reports
4.2.5.1 FindBugs Warinings
Solution：
There are different ways of encoding a String as bytes -- the charset determines that encoding. If you don't specify a charset, as in your call to str.getBytes(), it uses the system default.
FindBugs is warning you about this because you should think about what encoding you want to use for your output. If you're writing to a file, what are the readers of that file expecting? It is safest if you can specify an explicit encoding for the file so you don't write it one way and read it another way.
To specify an explicit charset, use str.getBytes(Charset.forName("UTF-8")), for example. UTF-8 is a good choice because it is always supported and can encode any character.
For example, .properties files are always ISO 8859-1 (i.e. Latin-1). That's documented so there is no ambiguity around what encoding to use.
[Chapter Ⅴ] * FAQ&ISSUE
5.1 Hudson Svn E2049000 could not open the requested SVN file system
ISSUE:
FAILED: svn: E204900: Could not open the requested SVN filesystem
org.tmatesoft.svn.core.SVNException: svn: E204900: Could not open the requested SVN filesystem
svn: E175002: OPTIONS of '/svn/%C3%AF%C2%BF%C2%BD%C3%AF%C2%BF%C2%BD%C3%86%C2%B7/jeesite/trunk/aggregator': 500 Internal Server Error (http://192.168.5.5:8088)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
Solution：
hudson subversion plugin 不支持中文路径，更换jenkins。
因后者源于hudson ，之前的配置大部分可自动复用！
jenkins对hudson某些插件做了调整，对界面UX设计做了极大的改进。
5.2 Build Failure ：download ojdbc14
ISSUE:
[ERROR] Failed to execute goal on project jeesite-common: Could not resolve dependencies for project com.pactera.jeesite:jeesite-common:jar:1.2.10: Could not transfer artifact com.oracle:ojdbc14:jar:10.2.0.4.0 from/to oschina-repos (http://maven.oschina.net/content/groups/public): Connection to http://maven.oschina.net refused: Connection refused: connect
Solution：手动安装ojdbc14，参看3.1章节
5.3 Batch NG：此时不应有…
ISSUE:
FOR /F "tokens=5 delims= " %P IN ('netstat -a -n -o ^| findstr :8080.*LISTENING') DO TaskKill.exe /F /PID %P
Run by batch file, loging:此时不应有8080*LISTENING
Solution：
批处理文件变量用 %%P 两个百分号；DOS终端执行时用 %P 一个百分号。
