Showing posts with label HBase Hive Integration. Show all posts
Showing posts with label HBase Hive Integration. Show all posts

Thursday, 15 May 2014

Accesing HBase Data using Hive Query Language ( with Probable Exceptions)


1. create a table in HBase

hbase(main):001:0> create 'hbaseTable','cf1'
0 row(s) in 1.4830 seconds

2. insert data into table

hbase(main):002:0> put 'hbaseTable','row1','cf1:name','giri'
0 row(s) in 0.0800 seconds

hbase(main):003:0> put 'hbaseTable','row2','cf1:name','Anamika'

0 row(s) in 0.0070 seconds

3. scan the table data

hbase(main):004:0> scan 'hbaseTable'
ROW                   COLUMN+CELL                                               
 row1                 column=cf1:name, timestamp=1400133482419, value=giri      
 row2                 column=cf1:name, timestamp=1400133502249, value=Anamika   
2 row(s) in 0.0360 seconds

4. Now we need to add the below jar files to hive.

guava-11.0.2.jar,
hive-hbase-handler-0.10.0.24.jar,  
hbase-0.94.5.jar, 
zookeeper-3.4.5.23.jar

we have number of ways to do this,

one way is to add jar files to

export HIVE_AUX_JARS_PATH=/usr/lib/guava-11.0.2.jar:/usr/lib/hive-hbase-handler-0.10.0.24.jar/ ...... remaining jars

other way directly add jars in the hive console,

hive> add jar /usr/lib/hbase/lib/guava-11.0.2.jar;
Added /usr/lib/hbase/lib/guava-11.0.2.jar to class path
Added resource: /usr/lib/hbase/lib/guava-11.0.2.jar

..... add remaining jars also.

now create hive table using the below syntax:

hive> CREATE TABLE hiveTable(key int, name string) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name")
TBLPROPERTIES ("hbase.table.name" = "hbaseTable");

now you can use hiveql language to query HBase data.

Troubleshooting:

You may get the below exception :

java.lang.ClassNotFoundException: org.apache.hadoop.hbase.MasterNotRunningException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
… 21 more 

or you may get any zookeeper related issues.

you can resolve the above issues by setting the below 2 properties in hive prompt;


set hbase.zookeeper.quorum=your zookeeper nodes;

set zookeeper.znode.parent=hbase-unsecure;