May 29, 2013

error: src refspec master does not match any.

When I executed the following command, I got an error.
#] git push origin env-properites
  error: src refspec env-properites does not match any.
  error: failed to push some refs to 'http://localhost/mygit/firsttest.git'

I can solve this problem by doing as here is written. It is easy!
Just modify unnecessary file like .gitignore.
And add & commit.
Run the above command again.

You can see the following message.
  Compressing objects: 100% (8/8), done.   Writing objects: 100% (12/12), 1.82 KiB, done.   Total 12 (delta 2), reused 0 (delta 0)   To http://localhost/mygit/firsttest.git    * [new branch]      env-properties -> env-properties

May 16, 2013

com.aerospike.client.AerospikeException: Error Code 13: Record too big

Recently I am trying to verify Aerospike can satisfy our requirements -
1. Persistent Cache for images
2. Time To Live(TTL)
3. Eviction
4. Remove all of related data by a key
5. Scale Out easily

I am testing it with YCSB to check performance. When I tried 256KB data size test, YCSB just stopped. As you know, YCSB is not kind. It doesn't show me any error message. So I ran the following code.

You can find out how to use Aerospike client here.
package com.bae;
import com.aerospike.client.*;
import com.aerospike.client.policy.WritePolicy;
import java.io.*;
import java.util.concurrent.atomic.AtomicInteger;
/**
* @author BJ2(Joongjin Bae)
* @version $Rev: $Rev$ By $Author: BJ2 A.K.A. bae_joongjin $
* $Date: 13/05/16 11:47 $
* @since 13/05/16
*/
public class Main {
private static String NAME_SPACE = "test";
private static String SET = "YCSB";
public static void main(String[] args) {
AtomicInteger counter = new AtomicInteger();
AerospikeClient client = null;
byte[] data = getOriginalData("486KB.jpg");
System.out.println(data.length / 1024);
try {
client = new AerospikeClient("localhost", 3000);
WritePolicy policy = new WritePolicy();
policy.timeout = 50;
Key key = new Key(NAME_SPACE, SET, "user" + counter.getAndIncrement());
Bin bin = new Bin("field0", data);
System.out.println(bin.value.estimateSize() / 1024);
client.put(policy, key, bin);
} catch (AerospikeException e) {
e.printStackTrace();
}
client.close();
}
public static byte[] getOriginalData(final String path) {
FileInputStream fis = null;
try {
fis = new FileInputStream(new File(path));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
}
} catch (IOException ex) {
}
return bos.toByteArray();
}
}
view raw aeroclient.java hosted with ❤ by GitHub


The result was "Record too big".
com.aerospike.client.AerospikeException: Error Code 13: Record too big
at com.aerospike.client.command.SingleCommand.parseResult(SingleCommand.java:262)
at com.aerospike.client.command.Command.execute(Command.java:181)
at com.aerospike.client.command.SingleCommand.write(SingleCommand.java:95)
at com.aerospike.client.AerospikeClient.put(AerospikeClient.java:226)
at com.bae.Main.main(Main.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
view raw gistfile1.java hosted with ❤ by GitHub


What? I already ran same test case on AWS and had no problem. They say "The size of the object or record is exceeding the limit, currently at 1MB". I just ran 256KB and 456KB. Something is wrong. I tried applying several data size files to find out MAX_SIZE. MAX_SIZE was 128KB. Finally I got a hint to solve this problem.

citrusleaf.conf
namespace test {
replication-factor 1
high-water-memory-pct 80
high-water-disk-pct 75
stop-writes-pct 85
memory-size 12884901888
default-ttl 864000
storage-engine device {
file /var/data/citrusleaf/test.data
filesize 858993459200
data-in-memory false
write-block-size 131072 # Here!!!
defrag-period 120
defrag-lwm-pct 50
defrag-max-blocks 4000
defrag-startup-minimum 10
}
}
view raw citrusleaf.conf hosted with ❤ by GitHub

I am using data file because server was already configured RAID 10 with 4 SSDs and I did not want to go far far our data center. :) Anyway the reason was [write-block-size]. It makes limitation on using data file. When I use raw device([device]), I had no problem. I removed it and could run my test case.

You know what?  The max size of our images is 10MB. orz...