Technical three-way with Jenkins, Git and Tomcat
The goal was clear: Our Java code shall be regularly compiled by a Jenkins build server, then deployed to our Tomcat server. So far, so good. But implementing this automatic workflow saw quite some obstacles to jump over.
When it comes to code versioning my personal programming past mainly brought me close to CVS and Subversion, but both don't really hold up any more today, so I decided to use git for source code maintenance. And because so many famous projects cannot be wrong, I chose Github as our online hosting service for this.
For professional reasons I am used to my program code being built at least once a day, so the Jenkins build server was an obvious choice. And a Tomcat server for our Java web service should be an easy target to deploy to, or so I thought. Well, step 1: install git plugin for Jenkins.
No problems here.
Step 2: create a git job in Jenkins which checks out our code from the Github repository.
This step, unfortunately, was the first with problems. The thing was authentication with Github, because I thought that it supports "anonymous" check-out of the code when using the correct URL. Doesn't seem to be that way, at least not via http(s) and ssh, because the URL git://github.com/oregami/oregami.org.git (advertized as "read-only") didn't work. This may well have been a problem with the firewall on my own server, but I don't dare touching that stuff at all.
The solution for this problem was the Tomcat user, under which Tomcat is actually running, receiving his/her own couple of private/public key using ssh-keygen. The public key was then lodged at Github, the email address and Github user name configured via "git config" - and it should have worked. Fat chance, no chance! One last step to go before this stopover: Call "git clone" once within the shell and have the SSH server added to the "known hosts". And finally, the git checkout was working within my build job!
Step 3: let Jenkins compile our source code via Maven. Another one of those "simple" things. But the devil was in the details in here, too: the program code was located in a subdirectory within the repository, cause there's also other things in there beside the code. Obviously, that seemed to be a problem for Jenkins' git plugin. Browsing various Google searches didn't bring up a helpful result, so I finally went for a quite-less-than-ideal solution: I created a new pom.xml file within the root of the repository which pointed to a module named after the subdirectory. Little abuse of Maven's technology, but it worked. At long last, the build runs and displays, among other things, this here:
Step 4: deploy the WAR file from the build to our Tomcat server. Give the Jenkins deploy plugin to me! But this was another case where the most natural things are not so natural at all. Deploy failed, and after another round of Google research I concluded that with Tomcat 7 the URLs for the managing tool had changed. They said it was fixed, but the Tomcat version I had installed (the latest) didn't seem to care about this fix. Luckily I was also able to find an alternative solution for this problem even when it won't win a beauty contest: instead of using the deploy plugin I executed a shell command "curl" which simply sends the WAR file to the respective Tomcat URL. The last thing missing was authentication of the "curl" command which is what you need a role called "manager-script" within the file tomcat-users.xml for. Another new thing in version 7 of Tomcat, not a boring software to use, I guess.
And the most final thing to do was finding out how to hide the transmitted user/password combination when "curling": parse it from a text file using the option "-K". Brilliant! The result of all this is the following command:
curl -K /opt/tomcat/.tomcat_curl_user -T - 'http://demo.oregami.org/manager/text/deploy?update=true&path=/' < /opt/jenkins/jobs/oregami-git/workspace/java/target/web.war
And what's the end of this story?
Following the principle of continuous integration, programmers should regularly commit their changes, and there should be regular builds of the software. That's what was achieved for our brand new online games database Oregami. At least the regular builds are running, more hackers are needed for more commits. :-)
So join the club and write a mail to sebastian[at]oregami.org !
When it comes to code versioning my personal programming past mainly brought me close to CVS and Subversion, but both don't really hold up any more today, so I decided to use git for source code maintenance. And because so many famous projects cannot be wrong, I chose Github as our online hosting service for this.
For professional reasons I am used to my program code being built at least once a day, so the Jenkins build server was an obvious choice. And a Tomcat server for our Java web service should be an easy target to deploy to, or so I thought. Well, step 1: install git plugin for Jenkins.
No problems here.
Step 2: create a git job in Jenkins which checks out our code from the Github repository.
This step, unfortunately, was the first with problems. The thing was authentication with Github, because I thought that it supports "anonymous" check-out of the code when using the correct URL. Doesn't seem to be that way, at least not via http(s) and ssh, because the URL git://github.com/oregami/oregami.org.git (advertized as "read-only") didn't work. This may well have been a problem with the firewall on my own server, but I don't dare touching that stuff at all.
The solution for this problem was the Tomcat user, under which Tomcat is actually running, receiving his/her own couple of private/public key using ssh-keygen. The public key was then lodged at Github, the email address and Github user name configured via "git config" - and it should have worked. Fat chance, no chance! One last step to go before this stopover: Call "git clone" once within the shell and have the SSH server added to the "known hosts". And finally, the git checkout was working within my build job!
Step 3: let Jenkins compile our source code via Maven. Another one of those "simple" things. But the devil was in the details in here, too: the program code was located in a subdirectory within the repository, cause there's also other things in there beside the code. Obviously, that seemed to be a problem for Jenkins' git plugin. Browsing various Google searches didn't bring up a helpful result, so I finally went for a quite-less-than-ideal solution: I created a new pom.xml file within the root of the repository which pointed to a module named after the subdirectory. Little abuse of Maven's technology, but it worked. At long last, the build runs and displays, among other things, this here:
[INFO] Building Oregami Webapp
[INFO] task-segment: [clean, install]
Step 4: deploy the WAR file from the build to our Tomcat server. Give the Jenkins deploy plugin to me! But this was another case where the most natural things are not so natural at all. Deploy failed, and after another round of Google research I concluded that with Tomcat 7 the URLs for the managing tool had changed. They said it was fixed, but the Tomcat version I had installed (the latest) didn't seem to care about this fix. Luckily I was also able to find an alternative solution for this problem even when it won't win a beauty contest: instead of using the deploy plugin I executed a shell command "curl" which simply sends the WAR file to the respective Tomcat URL. The last thing missing was authentication of the "curl" command which is what you need a role called "manager-script" within the file tomcat-users.xml for. Another new thing in version 7 of Tomcat, not a boring software to use, I guess.
And the most final thing to do was finding out how to hide the transmitted user/password combination when "curling": parse it from a text file using the option "-K". Brilliant! The result of all this is the following command:
curl -K /opt/tomcat/.tomcat_curl_user -T - 'http://demo.oregami.org/manager/text/deploy?update=true&path=/' < /opt/jenkins/jobs/oregami-git/workspace/java/target/web.war
And what's the end of this story?
Following the principle of continuous integration, programmers should regularly commit their changes, and there should be regular builds of the software. That's what was achieved for our brand new online games database Oregami. At least the regular builds are running, more hackers are needed for more commits. :-)
So join the club and write a mail to sebastian[at]oregami.org !