Continuous Delivery Using Jenkins

Let me introduce our Jenkins jobs. If Jenkins was not exist, we would suffer from HARD HARD build and deploy jobs.

First of all, we made our server list file. Because this is very often used by Jenkins deploy and restart jobs.
Add your Jenkins user key to my_name/.ssh/authorized_key on each server.
-Jenkins Server List Configure Job-

Next step, we build our project and make backup too.

-Jenkins Build Job-
Build our project by running maven.
  clean release:clean release:prepare package -P prd

And make backup directory.


And run deploy job. Don't forget set post-build Actions. We set Build other projects(Jenkins Restart Job)
-Jenkins Deploy Job-


-Jenkins Restart Job-


Sometimes we need to revert our service. We use this job. Don't forget to set Build other projects(Jenkins Deploy Job) that it will execute Jenkins Restart Job.
-Jenkins Roll Back Job-

[memo] jmagick 6.4.0 configure problem

I had to install jmagick 6.4.0. And I tried to install as the following.



Jmagick dose not care "--with-java-home=/opt/java/". It just uses "/usr/bin/java javac javah jar".
So you need to link them to /usr/bin

#ln -s /opt/java/bin/java /usr/bin/java

[memo] svc: warning: unable to control file does not exist.

I solved this problem here.

The key is the following command.
# svscanboot &

[memo] How to add many nodes to Chef Server

I created my node.json file with jackson. And I need to add 100 over nodes to Chef server.
I uploaded to Chef server's tmp directory my nodes files and ran the following script.


It is simple!

[memo] How to delete old files in Linux

maven scala java mixed project

I develop my product with Java and have test code written by Scala(Specs2). I wanted run test cases on my PC and Jenkins.
I have got some problems and solved by the following pom.xml. It works well. I hope this helps you.

Don't forget add "@RunWith(classOf[JUnitRunner])" to your xxxSpec class.

error: src refspec master does not match any.

When I executed the following command, I got an error.

#] git push origin env-properites
  error: src refspec env-properites does not match any.
  error: failed to push some refs to 'http://localhost/mygit/firsttest.git'

I can solve this problem by doing as here is written. It is easy!
Just modify unnecessary file like .gitignore.
And add & commit.
Run the above command again.

You can see the following message.

  Compressing objects: 100% (8/8), done.
  Writing objects: 100% (12/12), 1.82 KiB, done.
  Total 12 (delta 2), reused 0 (delta 0)
  To http://localhost/mygit/firsttest.git
   * [new branch]      env-properties -> env-properties

com.aerospike.client.AerospikeException: Error Code 13: Record too big

Recently I am trying to verify Aerospike can satisfy our requirements -
1. Persistent Cache for images
2. Time To Live(TTL)
3. Eviction
4. Remove all of related data by a key
5. Scale Out easily

I am testing it with YCSB to check performance. When I tried 256KB data size test, YCSB just stopped. As you know, YCSB is not kind. It doesn't show me any error message. So I ran the following code.

You can find out how to use Aerospike client here.


The result was "Record too big".


What? I already ran same test case on AWS and had no problem. They say "The size of the object or record is exceeding the limit, currently at 1MB". I just ran 256KB and 456KB. Something is wrong. I tried applying several data size files to find out MAX_SIZE. MAX_SIZE was 128KB. Finally I got a hint to solve this problem.

citrusleaf.conf

I am using data file because server was already configured RAID 10 with 4 SSDs and I did not want to go far far our data center. :) Anyway the reason was [write-block-size]. It makes limitation on using data file. When I use raw device([device]), I had no problem. I removed it and could run my test case.

You know what?  The max size of our images is 10MB. orz... 

java: invalid flag: -target=1.6 intellij idea

When I ran my test cases, I got this message "error: java: invalid flag: -target=1.6". I search this -"target=1.6" phrase in my project. So I re installed Intellij idea and removed .idea directory and .iml file. But I got same messages.

Finally I solved this problem by removing Additional command line parameters of java compiler.

How to add new node to an existing Aerospike Cluster - Using multicast

Modify /etc/citrusleaf/citrusleaf.conf as my example.

After modifying just start new node, it will be added an existing cluster automatically. You can confirm it on /var/log/citrusleaf.log with the following message.


Tip(?).
I changed ip address to 172.28.206.73 - it's wrong address -,  after I got succeeded with above configuration. So I fixed it to 172.28.206.72 and restarted node. But I got some non normal message.
I issued the command and failed to get expected result - ClusterSize 2 -.

I fixed this event by restarting all nodes.

Install and Run Aerospike Community Edition on CentOS 6.2

This installation is just for CentOS 6.2. If you use other OS like Ubuntu, please see Aerospike Standard Installation.

Step 1. Download Aerospike Database Server rpm packages
You can download here.

Step 2. Untar file


Step 3. Install the Aerospike Server


Step 4. Run the Aerospike Server for Testing

Oops! I got a error message. It means "CLD is stopped!".  Let me see the log.

Aha! Aerospike can't get ip address by eth, bond, wlan. Wait! my network interface name is "bond0.2032". If they use regular expression, it would be solved. But I can't fix it, so I changed my configuration file - /etc/citrusleaf/citrusleaf.conf - as blow. I just added network-interface-name item.

......

network {
        service {
                address any
                port 3000
                reuse-address
                network-interface-name bond0.2032

......

I ran it again and got succeeded.


Step 5. Verify Installation
I used the Aerospike command line interface tool to verity installation. It is installed as /opt/citrusleaf/bin/cli and lined in /usr/bin/cli. So you can use it normally.


It's simple, right?

How to install Jenkins and plugins offline.

I tried to the following commands that are introduced on Installing Jenkins on Red Hat distributions.


But I got failed. This server is not connected to internet. So I downloaded rpm package from RedHat Linux RPM packages for Jenkins and git, git-client plugins from Jenkins plugins download site to my MBA. And I uploaded them to the server and executed as blow.

No runnable methods junit 4.11

I am implementing Swift(OpenStack Object Storage) Client to use Swift simply. I ran each of test cases successfully on IDE, but I ran test cases with Class. It throws the following exception messages.



Here is my code.


I googled and searched several pages.

http://stackoverflow.com/questions/15383387/no-runnable-methods-on-junit-with-custom-annotation-and-all-tests-filtered

It says

You get this error due to the validation performed in validateInstanceMethods since computeTestMethods() returns an empty List: 

He suggested override or remove List size validation. I couldn't agree with him. Cause I have run my test cases with no problem on other projects. I read my code and error message again.

at swift.client.SwiftConfigTest.suite(SwiftConfigTest.java:17)

I felt it weird.  Because I don't need to run suite method. I removed suite method and ran all tests successfully.
Here is my right source code.

[memo] Kill all process by uid in Linux

Sometimes flunetd porecess is not stopped by "/etc/init.d/td-agent stop".

So I executed the following command.

bash while loopの変数が初期化される問題

業務上処理件数を調べる必要があったため軽くスクリプトを書くつもりが変数が初期化される現象に遭遇してはまった。

最初書いたスクリプトはこちらです。

特に問題ないように見えますが、最後のecho文の結果が0(zero)になります。

Google先生に聞くと下記のサイトを教えて下れました。
http://www.edwardawebb.com/linux/scope-issue-bash-loops

それで修正したスクリプトがこちら。正しくカウントできました。


ではなぜこの現象が起きるかというとパイプ(|)からのstdinはいつもsubshellを使うそうです。なのでループの中のcountはループ内の変数になってしまいます。
while文前パイプを使った場合よく起きるそうです。w

上で修正したスクリプトは自分のスタイルではないので下記のように見やすく修正しました。

how to remove remote branch in git(github)

It's simple to delete your remote branch.

  git push origin :

Deleting your local branch is simple too.

  git branch -d

You need to be on another branch. If you execute the command above on , you will see error message.

error: Cannot delete the branch '' which you are currently on.

How to make IntelliJ IDEA dark(Darcula)

If you use previous version (under 12), please import setting of Ted Wise. You can download this theme from the following URL.

http://tedwise.com/2009/02/26/dark-pastels-theme-for-intellij-idea/

If you just started IntelliJ IDEA 12,
1. Choose IntelliJ IDEA -> Preferences on Menu
2. Choose Editor -> Colors & fonts -> Scheme name
3. Choose Darcula! It's done.

Look it's beautiful!

Configuration of maven files to deploy to Nexus(Sonatype Nexus)

When I make a new Jenkins server, I always get a trouble with deploying module to Nexus. This memo  is just for me from future. Because I know I will search it again.

You need to write your Nexus repository information as the following.


ID is defined at pom.xml as below.

error: Setup script exited with error: command 'gcc' failed with exit status 1

I get the following error message when I tried to install tomahawk as following process of "How to install tomahawk"

Python's version is 2.4.3. tomahawk requires upper 2.4. My environments meets requirements.

Suddenly I remember that gcc troubles was solved by installing xxx-devel package.

I ran the following command.

  #]yum install python-devel

I could install tomahawk!

failed to flush the buffer, retrying. error="no nodes are available" fluentd

I got other error message from fluentd.
  failed to flush the buffer, retrying. error="no nodes are available"

The detail message is the following.


The reason
flunetd uses UDP for health check. But my target servers are on staging environment and collecting server is on production environment. They are on different segment. UDP is prohibited by our network policy.

The solution
Configure your Fire Wall to permit UDP over different segment.