Error installing scrapy from zsh… here is a solution to the problem.
Error installing scrapy from zsh
I
have Python Scrapy on my Mac and everything works fine until I updated Bash to Zsh and now I try to install it using pip install scrapy but come across
pip install Scrapy
Collecting Scrapy
Downloading Scrapy-1.3.2-py2.py3-none-any.whl (239kB)
100% |████████████████████████████████| 245kB 280kB/s
Requirement already satisfied: service-identity in /Library/Python/2.7/site-packages (from Scrapy)
Collecting parsel>=1.1 (from Scrapy)
Downloading parsel-1.1.0-py2.py3-none-any.whl
Collecting six>=1.5.2 (from Scrapy)
Downloading six-1.10.0-py2.py3-none-any.whl
Collecting lxml (from Scrapy)
Downloading lxml-3.7.3.tar.gz (3.8MB)
100% |████████████████████████████████| 3.8MB 41kB/s
Collecting Twisted>=13.1.0 (from Scrapy)
Downloading Twisted-17.1.0.tar.bz2 (3.0MB)
100% |████████████████████████████████| 3.0MB 134kB/s
Collecting cssselect>=0.9 (from Scrapy)
Downloading cssselect-1.0.1-py2.py3-none-any.whl
Collecting PyDispatcher>=2.0.5 (from Scrapy)
Downloading PyDispatcher-2.0.5.tar.gz
Collecting queuelib (from Scrapy)
Downloading queuelib-1.4.2-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
Downloading w3lib-1.17.0-py2.py3-none-any.whl
Requirement already satisfied: pyOpenSSL in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Scrapy)
Requirement already satisfied: pyasn1 in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: pyasn1-modules in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: attrs in /Library/Python/2.7/site-packages (from service-identity->Scrapy)
Requirement already satisfied: zope.interface>=3.6.0 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Twisted>=13.1.0->Scrapy)
Collecting constantly>=15.1 (from Twisted>=13.1.0->Scrapy)
Downloading constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=13.1.0->Scrapy)
Downloading incremental-16.10.1-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=13.1.0->Scrapy)
Downloading Automat-0.5.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from zope.interface>=3.6.0->Twisted>=13.1.0- >Scrapy)
Installing collected packages: six, w3lib, cssselect, lxml, parsel, constantly, incremental, Automat, Twisted, PyDispatcher, queuelib, Scrapy
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/rd/72lp2tzs02l049d0d33bzgxw0000gn/T/pip-DS44dD-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/ Extras/lib/python/six-1.4.1-py2.7.egg-info'
According to the logs I think it was because the six module tried to install
sudo pip install haxor-news --upgrade --ignore-installed 6
It is installed, but when trying to test if it installs correctly, I type scrapy and face
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 7, in <module>
from scrapy.cmdline import execute
File "/Library/Python/2.7/site-packages/scrapy/__init__.py", line 34, in <module>
from scrapy.spiders import Spider
File "/Library/Python/2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
from scrapy.http import Request
File "/Library/Python/2.7/site-packages/scrapy/http/__init__.py", line 12, in <module>
from scrapy.http.request.rpc import XmlRpcRequest
File "/Library/Python/2.7/site-packages/scrapy/http/request/rpc.py", line 7, in <module>
from six.moves import xmlrpc_client as xmlrpclib
ImportError: cannot import name xmlrpc_client
I found a problem about this issue and it was solved by
sudo rm -rf /Library/Python/2.7/site-packages/six*
sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
sudo pip install six
But for sudo rm -rf/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six
sudo rm -rf /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six*
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.py: Operation not permitted
rm: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six.pyc: Operation not permitted
Also tried updating 6 with pip install --upgrade six, but again it got an error
pip install --upgrade six
Collecting six
Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/rd/72lp2tzs02l049d0d33bzgxw0000gn/T/pip-_MQPsR-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/ Extras/lib/python/six-1.4.1-py2.7.egg-info'
What’s the problem and how do I fix it?
Solution
I solved the problem of upgrading 6 modules,
sudo easy_install --upgrade six
export PYTHONPATH=/Library/Python/2.7/site-packages:$PYTHONPATH
source ~/.zshrc
Resolved issues