Jun 6, 2012

How to scan/fetch all of keys in Voldemort

I'm supposed to provide a list that shows all of data in Voldemort. At first, I look around voldemort.client.StroreClient class. It has #getAll() method but this method has a parameter that is collections of key. Now I want to know all of keys!

Next step, I searched "getAll", "scan" and "fetch" that heats what I want. I found voldemort.client.protocol.admin.AdminClient#fetchKeys(). But it fetches all keys on a particular node. As you know I want all of keys in Voldemort cluster. Let me do coding little bit.

Here is my code. Forgive me for my dirty code. I just wrote this in an instant.
 


    private List getAllKeys() {
        String bootstrapUrl = "tcp://localhost:6666";
        StoreClientFactory factory = new SocketStoreClientFactory(new ClientConfig().setBootstrapUrls(bootstrapUrl));
        AdminClient adminClient = new AdminClient(bootstrapUrl, new AdminClientConfig());

        Collection nodes = adminClient.getAdminClientCluster().getNodes();
        List list = new ArrayList();
        int count = 0;
        for (Node node : nodes) {
            if (!factory.getFailureDetector().isAvailable(node))
            {
                System.out.println("Oops Node " + node.getId() + " is unavailable");
                continue;
            }
            List partitionIds = node.getPartitionIds();
            Iterator iter = adminClient.fetchKeys(node.getId(), "tabe_name", partitionIds, null, true);
            while (iter.hasNext()) {
                String key = new String(iter.next().get());
                list.add(key);
                count++;
                System.out.println("Node id:" + node.getId() + " key: " + key);
            }
        }

        System.out.println(count);
        return list;
    }



No comments: