安装pip与Scrapy(具体过程参考官网、与:
http://cuiqingcai.com/912.html
http://pip-cn.readthedocs.io/en/latest/installing.html
https://raw.github.com/pypa/pip/master/contrib/get-pip.py(GitHub官网脚本)
https://github.com/pypa/pip
)后,发现root权限执行pip、scrapy正常,而普通用户报错:
执行python -m pip -V提示/usr/bin/python: No module named pip
执行scrapy提示:
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 7, in
from scrapy.cmdline import execute
ImportError: No module named scrapy.cmdline
执行python -c "import pip; print(pip.__version__)"提示:
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named pip
执行pip提示:
Traceback (most recent call last):
File "/usr/local/bin/pip", line 7, in
from pip import main
ImportError: No module named pip
而/usr/local/bin/pip的第七行是“from pip import main”
在自己的py程序中手动import pip也不行,于是查看路径,print sys.path显示:
['/home/****/py', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/home/****/py/mymod']
看了下,主要是/usr/lib/和/usr/local/lib/,pip和scrapy装在/usr/local/lib/,而这个目录下的文件和文件夹权限有些异常:
alex@ubuntu:~$ ls -al /usr/local/lib/python2.7/dist-packages/
total 1176
drwxrwsr-x 56 root staff 4096 Dec 25 20:11 .
drwxrwsr-x 4 root staff 4096 Dec 4 22:04 ..
drwx--S--- 2 root staff 4096 Dec 4 22:17 attr
drwx--S--- 2 root staff 4096 Dec 4 22:17 attrs-16.3.0.dist-info
drwx--S--- 2 root staff 4096 Dec 4 22:09 certifi
drwx--S--- 2 root staff 4096 Dec 4 22:09 certifi-2016.9.26.dist-info
drwx--S--- 2 root staff 4096 Dec 4 21:58 cffi
drwx--S--- 2 root staff 4096 Dec 4 21:58 cffi-1.9.1.dist-info
-rwx--x--x 1 root staff 733736 Dec 4 21:58 _cffi_backend.so
drwx--S--- 2 root staff 4096 Dec 4 22:17 constantly
drwx--S--- 2 root staff 4096 Dec 4 22:17 constantly-15.1.0.dist-info
drwx--S--- 4 root staff 4096 Dec 4 22:06 cryptography
drwx--S--- 2 root staff 4096 Dec 4 22:06 cryptography-1.6-py2.7.egg-info
drwx--S--- 2 root staff 4096 Dec 4 22:17 cssselect
drwx--S--- 2 root staff 4096 Dec 4 22:17 cssselect-1.0.0.dist-info
-rw------- 1 root staff 126 Dec 4 20:57 easy_install.py
-rw------- 1 root staff 315 Dec 4 20:57 easy_install.pyc
drwx--S--- 2 root staff 4096 Dec 25 19:48 enum
drwx--S--- 2 root staff 4096 Dec 4 21:58 enum34-1.1.6.dist-info
drwx--S--- 2 root staff 4096 Dec 4 21:58 idna
drwx--S--- 2 root staff 4096 Dec 4 21:58 idna-2.1.dist-info
drwx--S--- 3 root staff 4096 Dec 4 22:17 incremental
drwx--S--- 2 root staff 4096 Dec 4 22:17 incremental-16.10.1.dist-info
drwx--S--- 2 root staff 4096 Dec 4 21:58 ipaddress-1.0.17.dist-info
-rw------- 1 root staff 80176 Dec 4 21:58 ipaddress.py
-rw------- 1 root staff 75654 Dec 4 21:58 ipaddress.pyc
drwx--S--- 2 root staff 4096 Dec 4 21:58 .libs_cffi_backend
drwx--S--- 6 root staff 4096 Dec 4 21:23 lxml
drwx--S--- 2 root staff 4096 Dec 4 21:23 lxml-3.6.4.dist-info
drwx--S--- 3 root staff 4096 Dec 4 21:31 ndg
drwx--S--- 2 root staff 4096 Dec 4 21:31 ndg_httpsclient-0.4.2-py2.7.egg-info
-rw------- 1 root staff 346 Dec 4 21:31 ndg_httpsclient-0.4.2-py2.7-nspkg.pth
drwx--S--- 2 root staff 4096 Dec 4 22:06 OpenSSL
drwx--S--- 2 root staff 4096 Dec 4 22:17 parsel
drwx--S--- 2 root staff 4096 Dec 4 22:17 parsel-1.1.0.dist-info
drwx--S--- 10 root staff 4096 Dec 25 19:49 pip
drwx--S--- 2 root staff 4096 Dec 4 20:57 pip-9.0.1.dist-info
drwx--S--- 4 root staff 4096 Dec 4 20:57 pkg_resources
drwx--S--- 5 root staff 4096 Dec 4 21:31 pyasn1
drwx--S--- 2 root staff 4096 Dec 4 21:31 pyasn1-0.1.9.dist-info
drwx--S--- 2 root staff 4096 Dec 4 22:17 pyasn1_modules
drwx--S--- 2 root staff 4096 Dec 4 22:17 pyasn1_modules-0.0.8.dist-info
drwx--S--- 3 root staff 4096 Dec 4 21:58 pycparser
drwx--S--- 2 root staff 4096 Dec 4 21:58 pycparser-2.17-py2.7.egg-info
drwx--S--- 2 root staff 4096 Dec 4 22:17 pydispatch
drwx--S--- 2 root staff 4096 Dec 4 22:17 PyDispatcher-2.0.5-py2.7.egg-info
drwx--S--- 2 root staff 4096 Dec 4 22:06 pyOpenSSL-16.2.0.dist-info
drwx--S--- 3 root staff 4096 Dec 4 22:17 queuelib
drwx--S--- 2 root staff 4096 Dec 4 22:17 queuelib-1.4.2.dist-info
drwx--S--- 20 root staff 4096 Dec 4 22:17 scrapy
drwx--S--- 2 root staff 4096 Dec 4 22:17 Scrapy-1.2.1.dist-info
drwx--S--- 2 root staff 4096 Dec 4 22:17 service_identity
drwx--S--- 2 root staff 4096 Dec 4 22:17 service_identity-16.0.0.dist-info
drwx--S--- 4 root staff 4096 Dec 4 20:57 setuptools
drwx--S--- 2 root staff 4096 Dec 4 20:57 setuptools-30.1.0.dist-info
drwx--S--- 2 root staff 4096 Dec 4 21:58 six-1.10.0.dist-info
-rw------- 1 root staff 30098 Dec 4 21:58 six.py
-rw------- 1 root staff 29545 Dec 4 21:58 six.pyc
drwx--S--- 26 root staff 4096 Dec 4 22:17 twisted
drwx--S--- 2 root staff 4096 Dec 4 22:17 Twisted-16.6.0-py2.7.egg-info
drwx--S--- 2 root staff 4096 Dec 4 22:17 w3lib
drwx--S--- 2 root staff 4096 Dec 4 22:17 w3lib-1.16.0.dist-info
drwx--S--- 5 root staff 4096 Dec 4 20:57 wheel
drwx--S--- 2 root staff 4096 Dec 4 20:57 wheel-0.29.0.dist-info
alex@ubuntu:~$ ls -al /usr/local/lib/python2.7/dist-packages/pip
ls: cannot open directory /usr/local/lib/python2.7/dist-packages/pip: Permission denied
alex@ubuntu:~$ sudo ls -al /usr/local/lib/python2.7/dist-packages/pip
[sudo] password for alex:
total 420
drwx--S--- 10 root staff 4096 Dec 25 19:49 .
drwxrwsr-x 56 root staff 4096 Dec 25 20:11 ..
-rw------- 1 root staff 11910 Dec 4 20:57 basecommand.py
-rw------- 1 root staff 8874 Dec 4 20:57 basecommand.pyc
-rw------- 1 root staff 10465 Dec 4 20:57 baseparser.py
-rw------- 1 root staff 10596 Dec 4 20:57 baseparser.pyc
-rw------- 1 root staff 16474 Dec 4 20:57 cmdoptions.py
-rw------- 1 root staff 15486 Dec 4 20:57 cmdoptions.pyc
drwx--S--- 2 root staff 4096 Dec 4 20:57 commands
drwx--S--- 2 root staff 4096 Dec 4 20:57 compat
-rw------- 1 root staff 32171 Dec 4 20:57 download.py
-rw------- 1 root staff 24968 Dec 4 20:57 download.pyc
-rw------- 1 root staff 8121 Dec 4 20:57 exceptions.py
-rw------- 1 root staff 12182 Dec 4 20:57 exceptions.pyc
-rw------- 1 root staff 39950 Dec 4 20:57 index.py
-rw------- 1 root staff 34576 Dec 4 20:57 index.pyc
-rw------- 1 root staff 11348 Dec 4 20:57 __init__.py
-rw------- 1 root staff 9163 Dec 4 20:57 __init__.pyc
-rw------- 1 root staff 5626 Dec 4 20:57 locations.py
-rw------- 1 root staff 4734 Dec 4 20:57 locations.pyc
-rw------- 1 root staff 584 Dec 4 20:57 __main__.py
-rw------- 1 root staff 477 Dec 4 20:57 __main__.pyc
drwx--S--- 2 root staff 4096 Dec 4 20:57 models
drwx--S--- 2 root staff 4096 Dec 4 20:57 operations
-rw------- 1 root staff 10980 Dec 4 20:57 pep425tags.py
-rw------- 1 root staff 9431 Dec 4 20:57 pep425tags.pyc
drwx--S--- 2 root staff 4096 Dec 4 20:57 req
-rw------- 1 root staff 156 Dec 4 20:57 status_codes.py
-rw------- 1 root staff 386 Dec 4 20:57 status_codes.pyc
drwx--S--- 2 root staff 4096 Dec 4 20:57 utils
drwx--S--- 2 root staff 4096 Dec 4 20:57 vcs
drwx--S--- 12 root staff 4096 Dec 4 20:57 _vendor
-rw------- 1 root staff 32010 Dec 4 20:57 wheel.py
-rw------- 1 root staff 25061 Dec 4 20:57 wheel.pyc
在自己的py程序中手动import enum(与pip都在/usr/local/lib/)也不行,于是怀疑整个/usr/local/lib/python2.7/dist-packages/目录都有问题,查了下,参考:
https://groups.google.com/forum/#!topic/comp.lang.python/CfBiVvvFOdY
执行“chmod -R a+rX 目录**”解决(-R递归,a表示u用户g组o其他 这些所有角色,X和可执行权限x相关但不是所有都加: execute/search only if the file is a directory or already has execute permission for some user (X)),执行后:
alex@ubuntu:~/py$ ls -al /usr/local/lib/python2.7/dist-packages
total 1176
drwxrwsr-x 56 root staff 4096 Dec 25 20:11 .
drwxrwsr-x 4 root staff 4096 Dec 4 22:04 ..
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 attr
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 attrs-16.3.0.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:09 certifi
drwxr-sr-x 2 root staff 4096 Dec 4 22:09 certifi-2016.9.26.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 cffi
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 cffi-1.9.1.dist-info
-rwxr-xr-x 1 root staff 733736 Dec 4 21:58 _cffi_backend.so
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 constantly
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 constantly-15.1.0.dist-info
drwxr-sr-x 4 root staff 4096 Dec 4 22:06 cryptography
drwxr-sr-x 2 root staff 4096 Dec 4 22:06 cryptography-1.6-py2.7.egg-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 cssselect
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 cssselect-1.0.0.dist-info
-rw-r--r-- 1 root staff 126 Dec 4 20:57 easy_install.py
-rw-r--r-- 1 root staff 315 Dec 4 20:57 easy_install.pyc
drwxr-sr-x 2 root staff 4096 Dec 25 19:48 enum
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 enum34-1.1.6.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 idna
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 idna-2.1.dist-info
drwxr-sr-x 3 root staff 4096 Dec 4 22:17 incremental
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 incremental-16.10.1.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 ipaddress-1.0.17.dist-info
-rw-r--r-- 1 root staff 80176 Dec 4 21:58 ipaddress.py
-rw-r--r-- 1 root staff 75654 Dec 4 21:58 ipaddress.pyc
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 .libs_cffi_backend
drwxr-sr-x 6 root staff 4096 Dec 4 21:23 lxml
drwxr-sr-x 2 root staff 4096 Dec 4 21:23 lxml-3.6.4.dist-info
drwxr-sr-x 3 root staff 4096 Dec 4 21:31 ndg
drwxr-sr-x 2 root staff 4096 Dec 4 21:31 ndg_httpsclient-0.4.2-py2.7.egg-info
-rw-r--r-- 1 root staff 346 Dec 4 21:31 ndg_httpsclient-0.4.2-py2.7-nspkg.pth
drwxr-sr-x 2 root staff 4096 Dec 4 22:06 OpenSSL
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 parsel
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 parsel-1.1.0.dist-info
drwxr-sr-x 10 root staff 4096 Dec 25 19:49 pip
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 pip-9.0.1.dist-info
drwxr-sr-x 4 root staff 4096 Dec 4 20:57 pkg_resources
drwxr-sr-x 5 root staff 4096 Dec 4 21:31 pyasn1
drwxr-sr-x 2 root staff 4096 Dec 4 21:31 pyasn1-0.1.9.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 pyasn1_modules
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 pyasn1_modules-0.0.8.dist-info
drwxr-sr-x 3 root staff 4096 Dec 4 21:58 pycparser
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 pycparser-2.17-py2.7.egg-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 pydispatch
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 PyDispatcher-2.0.5-py2.7.egg-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:06 pyOpenSSL-16.2.0.dist-info
drwxr-sr-x 3 root staff 4096 Dec 4 22:17 queuelib
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 queuelib-1.4.2.dist-info
drwxr-sr-x 20 root staff 4096 Dec 4 22:17 scrapy
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 Scrapy-1.2.1.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 service_identity
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 service_identity-16.0.0.dist-info
drwxr-sr-x 4 root staff 4096 Dec 4 20:57 setuptools
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 setuptools-30.1.0.dist-info
drwxr-sr-x 2 root staff 4096 Dec 4 21:58 six-1.10.0.dist-info
-rw-r--r-- 1 root staff 30098 Dec 4 21:58 six.py
-rw-r--r-- 1 root staff 29545 Dec 4 21:58 six.pyc
drwxr-sr-x 26 root staff 4096 Dec 4 22:17 twisted
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 Twisted-16.6.0-py2.7.egg-info
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 w3lib
drwxr-sr-x 2 root staff 4096 Dec 4 22:17 w3lib-1.16.0.dist-info
drwxr-sr-x 5 root staff 4096 Dec 4 20:57 wheel
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 wheel-0.29.0.dist-info
alex@ubuntu:~/py$ ls -al /usr/local/lib/python2.7/dist-packages/pip
total 420
drwxr-sr-x 10 root staff 4096 Dec 25 19:49 .
drwxrwsr-x 56 root staff 4096 Dec 25 20:11 ..
-rw-r--r-- 1 root staff 11910 Dec 4 20:57 basecommand.py
-rw-r--r-- 1 root staff 8874 Dec 4 20:57 basecommand.pyc
-rw-r--r-- 1 root staff 10465 Dec 4 20:57 baseparser.py
-rw-r--r-- 1 root staff 10596 Dec 4 20:57 baseparser.pyc
-rw-r--r-- 1 root staff 16474 Dec 4 20:57 cmdoptions.py
-rw-r--r-- 1 root staff 15486 Dec 4 20:57 cmdoptions.pyc
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 commands
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 compat
-rw-r--r-- 1 root staff 32171 Dec 4 20:57 download.py
-rw-r--r-- 1 root staff 24968 Dec 4 20:57 download.pyc
-rw-r--r-- 1 root staff 8121 Dec 4 20:57 exceptions.py
-rw-r--r-- 1 root staff 12182 Dec 4 20:57 exceptions.pyc
-rw-r--r-- 1 root staff 39950 Dec 4 20:57 index.py
-rw-r--r-- 1 root staff 34576 Dec 4 20:57 index.pyc
-rw-r--r-- 1 root staff 11348 Dec 4 20:57 __init__.py
-rw-r--r-- 1 root staff 9163 Dec 4 20:57 __init__.pyc
-rw-r--r-- 1 root staff 5626 Dec 4 20:57 locations.py
-rw-r--r-- 1 root staff 4734 Dec 4 20:57 locations.pyc
-rw-r--r-- 1 root staff 584 Dec 4 20:57 __main__.py
-rw-r--r-- 1 root staff 477 Dec 4 20:57 __main__.pyc
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 models
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 operations
-rw-r--r-- 1 root staff 10980 Dec 4 20:57 pep425tags.py
-rw-r--r-- 1 root staff 9431 Dec 4 20:57 pep425tags.pyc
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 req
-rw-r--r-- 1 root staff 156 Dec 4 20:57 status_codes.py
-rw-r--r-- 1 root staff 386 Dec 4 20:57 status_codes.pyc
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 utils
drwxr-sr-x 2 root staff 4096 Dec 4 20:57 vcs
drwxr-sr-x 12 root staff 4096 Dec 4 20:57 _vendor
-rw-r--r-- 1 root staff 32010 Dec 4 20:57 wheel.py
-rw-r--r-- 1 root staff 25061 Dec 4 20:57 wheel.pyc
普通用户可执行pip、scrapy,而且原有/usr/local/lib/python2.7/dist-packages 整个目录及子目录均可让普通用户访问了,解决了之前一直存在但一直没发现的问题
后记20170211:
1.虚拟机安装新系统Ubuntu 14.04.5,/usr/local/lib/python2.7/dist-packages 和 /usr/local/lib/python3.4/dist-packages 均为空,且属性为root:staff
2.安装pip与scrapy时会有一些小问题,安装好相关的依赖就可以了,例如sudo apt-get install gcc、g++、libssl-dev、libffi-dev、python-dev,以及pip install pyopenssl、pyasn1、ndg-httpsclient、lxml、twisted
3.此次安装又遇到了另一个问题,运行scrapy提示:
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 7, in
from scrapy.cmdline import execute
File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 9, in
from scrapy.crawler import CrawlerProcess
File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 7, in
from twisted.internet import reactor, defer
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/reactor.py", line 38, in
from twisted.internet import default
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/default.py", line 56, in
install = _getInstallFunction(platform)
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/default.py", line 44, in _getInstallFunction
from twisted.internet.epollreactor import install
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/epollreactor.py", line 24, in
from twisted.internet import posixbase
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 18, in
from twisted.internet import error, udp, tcp
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 28, in
from twisted.internet._newtls import (
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/_newtls.py", line 21, in
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
File "/usr/local/lib/python2.7/dist-packages/twisted/protocols/tls.py", line 63, in
from twisted.internet._sslverify import _setAcceptableProtocols
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/_sslverify.py", line 38, in
TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'
查了下很有可能是twisted版本与scrapy不匹配导致,原有Ubuntu上scrapy是1.2.1,twisted是16.6.0,而新Ubuntu上scrapy是1.3.1,twisted是17.1.0,于是
sudo pip uninstall twisted(卸载)
sudo pip install twisted==16.6.0(指定版本安装)
sudo pip show twisted(查询)
sudo chmod -R a+rX /usr/local/lib/python2.7/dist-packages/(更改相关文件夹权限)
后再次运行scrapy就OK
后记20170226:
昨天和今天继续整了下Python2、Python3以及pip、pip3
Python2和Python3可以看做两个独立的软件,pip作为Python的模块可以安装在Python2(/usr/local/lib/python2.7/dist-packages/)也可以安装在Python3(/usr/local/lib/python3.4/dist-packages/),执行pip和pip3分别为Python何Python3安装模块,也会默认安装到对应的这两个不同路径中,而查询的时候,例如pip/pip3 show coursera-dl,pip和pip3显示的结果是一样的,不管是2还是3版本的模块都能显示。默认安装模块后普通用户没有权限,主要是/usr/local/lib/python2.7/dist-packages/、/usr/local/lib/python3.4/dist-packages/,以及/usr/local/bin/这几个路径里的文件没权限,使用这些命令增加权限:sudo chmod -R a+rx /usr/local/bin/ 、 sudo chmod -R a+rX 目录 ,sudo chmod -R a+x *py 、sudo chmod -R a+x *pyc ,注意小写x是所有全加可执行权限,大写X是如果owner有x或者是目录,则加x;而 sudo chmod -R a+x *py 只能对当前目录的*py增加x,如果下一级子目录,需要 sudo chmod -R a+x */*py ,再下一级则需 sudo chmod -R a+x */*/*py ,增加权限后就可以不加sudo执行/查看了。另外需要注意的是apt-get安装的python3-pip版本太旧没法用,需要用官方的get-pip.py或者源码编译。而且Ubuntu14.04自带Python3.4但没见有pip3需手动装。整好后 python -m pip 和 python3 -m pip 都可以用了,python -m pip list 和 python3 -m pip list 可以分别列出2和3版本安装的模块。
理一下过程:
coursera-dl作为Python模块,建议装在Python3目录,而自己的系统默认Python2,直接把Python命令指向Python3就可以直接安装pip到Python3对应路径,但会影响全局,影响其他程序,更好的方法是下载get-pip.py(https://bootstrap.pypa.io/get-pip.py),修改第一行 #!/usr/bin/env python 为 #!/usr/bin/env python3 (加一个3),这样 sudo ./get-pip.py 即可作为Python3的模块安装到 /usr/local/lib/python3.4/dist-packages/ ,接下来使用pip3安装coursera-dl:sudo pip3 install coursera-dl (或者 sudo python3 -m pip install ** 指定Python3的模块pip去install),也是直接装到 /usr/local/lib/python3.4/dist-packages/ ,注意增加相关目录权限。接下来就可以使用Python3的模块coursera-dl了(命令在/usr/local/bin/coursera-dl,相关文件在/usr/local/lib/python3.4/dist-packages/)。
后记20170516:
这几天安装TensorFlow遇到一些问题,走了一大圈弯路,细节省略了,大致过程如下:
Ubuntu14.04装了Python2.7.6想安装最新的2.7.13,apt-get源里面没有,于是从源码安装,哪知道安装的路径不一致,原有在/usr/bin/、/usr/lib/ 这些目录里,新装的在/usr/local/bin/、/usr/local/lib/里面,导致一系列问题,源码命令没有卸载命令,只能手动一点点删,删完再重新试着安装2.7.6,重新pip:
起因是想装TensorFlow,因为主用Python2.7(2.7.6,虽然也装了Python3),所以想把2.7.6升到最新的2.7.13,以为这样可以避免Python的SNIMissingWarning和InsecurePlatformWarning告警(事实上怎么折腾还是没法避免)。但是apt源里面最新的就是2.7.6没法升,于是就下载了2.7.13源码编译安装,装好之后用pip安装TensorFlow,装好后import tensorflow告警:
No module named tensorflow /usr/local/lib/python2.7/dist-packages/tensorflow
查看/usr/local/lib/python2.7/dist-packages/里面明明是有这个tensorflow文件夹啊,于是 print sys.path :
['/home/****', '/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-linux2', '/usr/local/lib/python2.7/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages']
(或者
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
print '\n'.join(sys.path)
分行显示
而使用 #!/usr/bin/env python 与 #!/usr/local/bin/python 与 #!/usr/bin/python 与 #!/usr/bin/python3 的结果不尽相同,/usr/bin/ 里是原来装的python,/usr/local/bin/ 里是新的、源码安装的python。
)
sys.path和原来不一样,/usr/lib/ 变为了 /usr/local/lib/ 而且 dist-packages 变为了 site-packages ,这就是源码安装和apt-get安装造成的差异,怎么办?
有人建议装 virtualenv (参考:https://github.com/tensorflow/tensorflow/issues/647),这个 virtualenv 帮助管理路径,但我不想装,想自己掌握操作,于是试着添加路径到sys.path,不是在**.py里面临时sys.path.append,而是永久增加,参考:
https://askubuntu.com/questions/798382/cant-import-installed-packages-in-python
https://leemendelowitz.github.io/blog/how-does-python-find-packages.html
https://docs.python.org/2/tutorial/modules.html (里面搜索 sys.path)
https://docs.python.org/2/library/sys.html#sys.path
https://docs.python.org/2/library/site.html#module-site
http://www.tuicool.com/articles/EB3miqr
http://www.jb51.net/article/42183.htm
选择在 site-packages 目录下加 ****.pth 文件的方法,创建 mypath.pth 里面写上目录,路径可以用相对路径,如果同一目录,直接文件名也可,添加之后 sys.path 确实有增加****.pth里的内容,现在sys.path的问题看起来解决了,但是import tensorflow仍然有问题:
ImportError: Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. If you're working with a numpy git repo, try `git clean -xdf` (removes all files not under version control). Otherwise reinstall numpy.
参考:
https://github.com/tensorflow/tensorflow/issues/559
http://stackoverflow.com/questions/34051737/numpy-core-multiarray-failed-to-import
http://stackoverflow.com/questions/20518632/importerror-numpy-core-multiarray-failed-to-import
还是版本冲突的问题,一下也找不出哪儿装了两个版本的numpy了,于是考虑卸载源码安装的Python2.7.13,但是源码目录里面没有卸载选项,只能手动删除:
/usr/local/bin/python*
/usr/local/bin/pydoc2
/usr/local/lib/python2
/usr/local/include/python2
/usr/local/lib/pkgconfig/python2.pc
/usr/local/lib/libpython2
删除和以上类似的路径和文件后,/usr/local/bin/ 里面仍有一些文件,看了下有些是pip安装的Python包,例如pillow(PIL),产生的,于是直接删掉,找了会也没法100%确认干净,算了(期间误删了一些lib、pip文件,只好从另一个系统里复制过来一遍)
接下来,恢复安装Python2.7.6(其实原本就有,现在是重新安装、修复下,以免缺少文件之类的),这之前要添加第三方apt源(ppa),参考:
https://www.zhihu.com/question/37021474?sort=created
https://launchpad.net/ubuntu/+ppas?name_filter=python
https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes
https://askubuntu.com/questions/865554/how-do-i-install-python-3-6-using-apt-get
执行:
sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update
再执行:
sudo apt-get install --reinstall python2.7
sudo apt-get install --only-upgrade python2.7
sudo apt-get -f install python2.7
然后再apt-get装Python3.6,再从 https://pip.pypa.io/en/stable/installing/ 下载get-pip.py安装pip: python2 get-pip.py,下载 https://bootstrap.pypa.io/3.4****(try3.2、3.3/3.4、etc)/get-pip.py 来安装pip3: python3 get-pip.py(将get-pip.py首行改为:#!/usr/bin/env python3,注意写明python3而不是python),接下来又出现:
File "get-pip.py", line 20061, in lsb_release ****的问题,查了下是lsb_release的问题,看到某网友 which lsb_release (查看lsb_release程序所在位置),于是 vi lsb_release 进去看了看,第一行是:
#! /usr/bin/python3 -Es
修改Python3为python,再执行就OK了,看来还是安装Python3或者pip3时把 lsb_release 覆盖了,还是版本2和3的冲突。参考:
https://github.com/pypa/setuptools/issues/937
https://discourse.mailinabox.email/t/unable-to-run-mailinabox-command/1840/11
(居然有网友卸载了重装来解决。。。。)
这个问题解决后接下来把pip和python指向2版本的可执行文件(通过强制修改软链接 ln -sf),但是每次pip之后还是需要修改 /usr/local/lib/python2.7/dist-packages 目录的权限很麻烦,查了下解决方法:
http://stackoverflow.com/questions/23629898/problems-installing-matplotlib-in-linux
http://caligari.treboada.net/2014/09/23/python-pip-and-the-staff-group/
https://askubuntu.com/questions/736738/i-tried-to-install-python-package-then-some-error-is-showing-like-exeption
https://github.com/sindresorhus/weechat-notification-center/issues/1
https://superuser.com/questions/272061/reload-a-linux-users-group-assignments-without-logging-out
https://arkaitzj.wordpress.com/2010/03/08/linux-add-user-to-a-group-without-logout/
有把当前用户加入staff组的、有修改目录所有者、修改目录权限的,有用apt-get代替pip安装的,还有使用 virtualenv 的,试了下加入staff组然后 newgrp staff 不logout就即时生效,不好使,还是用 sudo pip **** 然后 sudo chmod -R a+rX /usr/local/lib/python2.7/dist-packages/ 。
最终的 sys.path 是(分行显示):
/home/****
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
而
from distutils.sysconfig import get_python_lib
print(get_python_lib())
是 /usr/lib/python2.7/dist-packages
最后参考 https://github.com/martin-gorner/tensorflow-mnist-tutorial/blob/master/INSTALL.txt 进行测试:
git clone https://github.com/martin-gorner/tensorflow-mnist-tutorial.git
cd tensorflow-mnist-tutorial
python mnist_1.0_softmax.py
提示 ImportError: No module named _tkinter, please install the python-tk package
缺少tkinter库于是执行sudo apt-get install python-tk,OK
终于基于python2.7.6成功安装TensorFlow,同时保持了python2和python3并存,pip2和pip3并存
中间的小问题省略了,最后总结的经验就是:
尽量别源码安装python,可以用第三方源加入PPA后(可用ppa:fkrull/deadsnakes),用apt-get等工具安装新版本,没有最新的就算了。装好python再用pip安装package或lib,装完 sudo chmod -R a+rX /****/lib/****/dist-packages/
tensorflow的安装可参考:
https://github.com/martin-gorner/tensorflow-mnist-tutorial/blob/master/INSTALL.txt
sudo -H apt-get install git
sudo -H pip install --upgrade matplotlib
sudo -H pip install --upgrade tensorflow
(缺少tkinter库需执行sudo apt-get install python-tk)
Labels: Coding, Linux, networks, Operation and Maintenance
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home