Java – HBase Java client not working (MasterNotRunningException exception)

I’m trying to write a remote HBase client in Java. Here is the code for reference:

package ttumdt.app.connector;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.FilterList;
import org.apache.hadoop.hbase.filter.RowFilter;
import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
import org.apache.hadoop.hbase.util.Bytes;

import java.io.IOException;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;

public class HBaseClusterConnector {
    private final String MASTER_IP = "10.138.168.185";
    private final String ZOOKEEPER_PORT = "2181";

final String TRAFFIC_INFO_TABLE_NAME = "TrafficLog";
    final String TRAFFIC_INFO_COLUMN_FAMILY = "TimeStampIMSI";

final String KEY_TRAFFIC_INFO_TABLE_BTS_ID = "BTS_ID";
    final String KEY_TRAFFIC_INFO_TABLE_DATE = "DATE";
    final String COLUMN_IMSI = "IMSI";
    final String COLUMN_TIMESTAMP = "TIME_STAMP";

private final byte[] columnFamily = Bytes.toBytes(TRAFFIC_INFO_COLUMN_FAMILY);
    private final byte[] qualifier= Bytes.toBytes(COLUMN_IMSI);

private Configuration conf = null;

public HBaseClusterConnector () throws MasterNotRunningException, ZooKeeperConnectionException {
        conf = HBaseConfiguration.create();
        conf.set("hbase.zookeeper.quorum",MASTER_IP);
        conf.set("hbase.zookeeper.property.clientPort",ZOOKEEPER_PORT);
        HBaseAdmin.checkHBaseAvailable(conf);
    }

/**
     * This filter will return list of IMSIs for a given btsId and ime interval
     * @param btsId : btsId for which the query has to run
     * @param startTime : start time for which the query has to run
     * @param endTime : end time for which the query has to run
     * @return returns IMSIs as set of Strings
     * @throws IOException
     */
    public Set<String> getInfoPerBTSID(String btsId, String date,
                                       String startTime, String endTime)
            throws IOException {
        Set<String> imsis = new HashSet<String>();

ToDo : better exception handling
        HTable table = new HTable(conf, TRAFFIC_INFO_TABLE_NAME);
        Scan scan = new Scan();

scan.addColumn(columnFamily,qualifier);
        scan.setFilter(prepFilter(btsId, date, startTime, endTime));

 filter to build where timestamp

Result result = null;
        ResultScanner resultScanner = table.getScanner(scan);

while ((result = resultScanner.next())!= null) {
            byte[] obtainedColumn = result.getValue(columnFamily,qualifier);
            imsis.add(Bytes.toString(obtainedColumn));
        }

resultScanner.close();

return imsis;
    }

ToDo : Figure out how valid is this filter code?? How comparison happens
     with eqaul or grater than equal etc

private Filter prepFilter (String btsId, String date,
                               String startTime, String endTime)
    {
        byte[] tableKey = Bytes.toBytes(KEY_TRAFFIC_INFO_TABLE_BTS_ID);
        byte[] timeStamp = Bytes.toBytes(COLUMN_TIMESTAMP);

 filter to build -> where BTS_ID = <<btsId>> and Date = <<date>>
        RowFilter keyFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
                new BinaryComparator(Bytes.toBytes(btsId+date)));

 filter to build -> where timeStamp >= startTime
        SingleColumnValueFilter singleColumnValueFilterStartTime =
                new SingleColumnValueFilter(columnFamily, timeStamp,
                        CompareFilter.CompareOp.GREATER_OR_EQUAL,Bytes.toBytes(startTime));

 filter to build -> where timeStamp <= endTime
        SingleColumnValueFilter singleColumnValueFilterEndTime =
                new SingleColumnValueFilter(columnFamily, timeStamp,
                        CompareFilter.CompareOp.LESS_OR_EQUAL,Bytes.toBytes(endTime));

FilterList filterList = new FilterList(FilterList.Operator.MUST_PASS_ALL, Arrays
                .asList((Filter) keyFilter,
                        singleColumnValueFilterStartTime, singleColumnValueFilterEndTime));
        return filterList;
    }

public static void main(String[] args) throws IOException {
        HBaseClusterConnector flt = new HBaseClusterConnector();
        Set<String> imsis= flt.getInfoPerBTSID("AMCD000784", "26082013","104092","104095");
        System.out.println(imsis.toString());
    }
}

I’m currently testing this with Cloudera Quick Start VM.

The question is; If I run this code on a VM, it definitely works. But if you run it externally, it fails with the following error. And I suspect it has something to do with VM settings and nothing else. Note that I’ve checked if I can connect to the VM’s node manager/job tracker from the host and it works really well. When I run code from my host OS instead of running it on a VM; I get the following error:

2013-10-15 18:16:04.185 java[652:1903] Unable to load realm info from SCDynamicStore
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: Retried 1 times
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:138)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1774)
    at ttumdt.app.connector.HBaseClusterConnector.<init>(HBaseClusterConnector.java:47)
    at ttumdt.app.connector.HBaseClusterConnector.main(HBaseClusterConnector.java:117)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Process finished with exit code 1

Please note; The master node is actually running. The zookeper log shows that it has established a connection with the host operating system:

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

6:16:03.274 PM  INFO    org.apache.zookeeper.server.ZooKeeperServer     

Client attempting to establish new session at /10.138.169.81:50567

6:16:03.314 PM  INFO    org.apache.zookeeper.server.ZooKeeperServer     

Established session 0x141bc2487440004 with negotiated timeout 60000 for client /10.138.169.81:50567

6:16:03.964 PM  INFO    org.apache.zookeeper.server.PrepRequestProcessor    

Processed session termination for sessionid: 0x141bc2487440004

6:16:03.996 PM  INFO    org.apache.zookeeper.server.NIOServerCnxn   

Closed socket connection for client /10.138.169.81:50567 which had sessionid 0x141bc2487440004

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

But I don’t see any traces of activity in the Master or RegionServer logs.

Note that my host operating system is Mac OSX 10.7.5

According to available resources; This should work fine; Although some people think that a simple HBase java client will never work. I’m confused; And eagerly await the pointers! Please reply

Start your HiveServer2 on a different port and try to connect

Command to connect hiveserver2 on a different port (make sure the hive is in the path):

hive --service hiveserver2 --hiveconf hive.server2.thrift.port=13000