Python Pyhive modules cannot import name hives
I want to use pyhive to connect Python to hive. I’m using the python script below to execute locally on my part.
#!/usr/bin/env python
# coding: utf-8
from pyhive import hive
from TCLIService.ttypes import TOperationState
def mysql_connect(host, port, username):
conn = hive. Connection(host=host, port=port, username=username)
return conn.cursor()
cursor = mysql_connect("localhost", 50070, "hduser")
cursor.execute("show databases")
print_log(cursor)
I
put pyhive in the /usr/local/lib/python2.7/dist-packages location, but I ended up with the following output
vaibhav@vaibhav-Lenovo-G570:~/Desktop/Python/Automation$ ./pyhive_test.py
Traceback (most recent call last):
File "./pyhive_test.py", line 9, in <module>
cursor = mysql_connect("localhost", 50070, "hduser")
File "./pyhive_test.py", line 6, in mysql_connect
conn = hive. Connection(host=host, port=port, username=username)
File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 131, in __init__
self._transport.open()
File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/__init__.py", line 80, in open
status, payload = self._recv_sasl_message()
File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/__init__.py", line 101, in _recv_sasl_message
payload = read_all_compat(self._trans, length)
File "/usr/local/lib/python2.7/dist-packages/thrift_sasl/six.py", line 31, in <lambda>
read_all_compat = lambda trans, sz: trans.readAll(sz)
File "/home/vaibhav/.local/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/home/vaibhav/.local/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 132, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
Edit
1. The file name is changed from Pyhive to pyhive_test
- pyhive.py has been removed from the catalog
Possible solutions tried :
1. Two versions of Python 2.7 and Python 3.4 are installed. I
Uninstalled Python 3.4 but the folder still seems to be there
/usr/local/library/. I RAN SOME OF THE COMMANDS BELOW TO CHECK WHERE MY PYTHON IS INSTALLED AND THE PACKAGES AVAILABLE IN PYTHONPATH
vaibhav@vaibhav-Lenovo-G570:~$ which -a python
/usr/bin/python
vaibhav@vaibhav-Lenovo-G570:~$ python -c "import sys, pprint; pprint.pprint(sys.path)"
['',
'/home/vaibhav',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/home/vaibhav/.local/lib/python2.7/site-packages',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PILcompat',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
2。 Get a quote from the link mentioned here They mention using it in a virtual environment or using clean python. None of them are used, nor is it known how it will affect the existing configuration.
3. I installed Pyhive using sudo, so I changed the permissions link But still experiencing the same problem.
Solution
The file you want to start is named pyhive.py
.
When you do
from pyhive import hive
In your pyhive.py
, then it will try to import Hive
from your module, not from the pyhive
library.
Name the file you are launching something else and avoid using the name of an existing module/library.
From docs :
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
- the directory containing the input script (or the current directory).
- PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
- the installation-dependent default.