Java – Hive reaches max worker and cannot connect to hiveserver2

Hive reaches max worker and cannot connect to hiveserver2… here is a solution to the problem.

Hive reaches max worker and cannot connect to hiveserver2

When I connect to hiveserver2 using a straight line, the error message goes something like this.
I previously connected to hiveserver2.
This error is displayed after I connect to hiveserver2 a few times.
I can connect using jdbc:hive2://

beeline> !connect jdbc:hive2://master:10000

SLF4J: The classpath contains multiple SLF4J bindings (binds).
SLF4J: Found binding (bind) in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings Ask for explanations.
SLF4J: The actual binding (bind) type is [org.apache.logging.slf4j.Log4jLoggerFactory]
Connect to jdbc:hive2://master:10000
Enter the username for jdbc: hive2://master:10000:
Enter the password for jdbc:hive2://master:10000:
11/17/14 22:09:36 [main]: warning jdbc. HiveConnection: Unable to connect to host: 10000
Unexpected end-of-file endings when reading from an HS2 server. The root cause may be too many concurrent connections. Ask your administrator to check the number of activity connections and adjust hive.server2.thrift.max.worker.threads, if applicable.
Error: Unable to open client transport with JDBC URI: jdbc:hive2://master:10000: null (state=08S01,code=0).

The hive-site.xml is set up like this

<property>
 39         <name>hive.server2.thrift.min.worker.threads</name>
 40         <value>5</value>
 41     </property>
 42     <property>
 43         <name>hive.server2.thrift.max.worker.threads</name>
 44         <value>500</value>
 45     </property>
 46     <property>
 47         <name>hive.server2.thrift.bind.host</name>
 48         <value>master</value>
 49     </property>
 50     <property>
 51         <name>hive.server2.thrift.port</name>
 52         <value>10000</value>
 53     </property>

I’ve checked the netstat that port 10000 is listening on.
Even if I restart the hadoop server and hiveserver2, the problem is not solved.
How do I know how many connections are in the active state or can I clear the thread pool?
When you close HiveServer, Hadoop, and Linux, will Active not shut down?

Solution

It works fine in my environment :

$ netstat -an | grep 10000
tcp4       0      0  *.10000                *.*                    LISTEN
  • Hive site .xml

Set nosasl not to use SASLTransport

<property>
    <name>hive.server2.authentication</name>
    <value>NOSASL</value> <!-- default NONE is for SASLTranspor
</property>

<property>
    <name>hive.server2.enable.doAs</name>
    <value>false</value> <!-- Execute query as hiveserver2 proc
</property>

and

!connect jdbc:hive2://localhost:10000/default; auth=noSasl hive password org.apache.hive.jdbc.HiveDriver
  • Citations

This is a blog in Japanese, but it will help: http://tagomoris.hatenablog.com/entry/20131226/1388044783

Related Problems and Solutions