First, some context: I'm following a web scraping tutorial and am running into (what I believe are) setup-related issues. I keep running into errors with the last step of Part 0 (bottom of page). After creating my virtualenv (using mkvirtualenv), I receive the following error when running the following command (from inside my virtualenv): pip install -r requirements.txt:
[Errono 2] No such file or directory: 'requirements.txt'
Storing debug log for failure in /home/dchaudh/.pip/pip.log
I did some quick googling and came across this solution which seems promising: it suggests that the problem is due to Salt (bc of something called chowning) and that adding no_chown: True to my salt state should fix the issue.
How do I add to the salt state, i.e., what file do I modify? Any help would be appreciated
As you can tell from the tutorial, I'm a relative newbie (even moreso to virtual environments) and I feel like I quickly got in over my head in the process theof trying to find a solution.
Are you actually using Salt? I feel that you found that solution online and attributed your error to Salt. Nothing in the tutorial mentions Salt, so it seems unlikely that you would be using it.
Make sure you are actually in the new-coder/scrape directory before running pip install -r requirements.txt
Related
First, background: I'm an undergraduate student in physics and mathematics. I have experience programming in Python, and I've gained a lot of experience with using various shell programs while trying to solve this problem.
I'm needing to run some software on a cluster at my University. I'm trying to do this by running ssh from my mac to the machine. I've been having a lot of trouble installing packages because of permissions, so I thought I would try using virtualenv.
Despite the fact that other packages would not install, virtualenv seemed to install. However, when I try to run it, I get
virtualenv: Command not found.
In trying to solve my problem, I tried reinstalling it and got
Requirement already satisfied: virtualenv in ./.local/lib/python3.4/site-packages.
I though maybe the issue was that my path was not set right. The remote machine is using tcsh, and so I ran
setenv PATH ${PATH}/local/local/lib/python3.4/site-packages
and the path now is:
/usr/local/texlive/2017/bin/sparc-solaris-10:/local/bin:/usr/plocal/bin:/usr/local/bin/X11:
/usr/local/bin:/usr/local/share/bin:
/usr/openwin/bin:/usr/dt/bin:/usr/bin/X11:
/usr/X11/bin:/usr/local/pbmplus:/usr/local/X11R5/bin:/usr/local/plot79:/usr/java/bin:
/usr/local/java/bin:
/usr/local/hotjava/bin:/opt/SUNWspro/bin:
/usr/ccs/bin:/usr/ucb:/usr/sbin:/usr/bin:/bin:/sbin:/usr/5bin:
/usr/local/etc:/etc:/usr/plocal/sun/bin:./local/local/lib/python3.4/site-packages
Yet, it remains that if I try to run virtualenv, it claims that the command is not found!
FULL DISCLOSURE: This question is similar to the following question listed here. But, the solution there won't work for me because I can't use sudo:
sudo: unable to stat /etc/sudoers: Bad file number
sudo: no valid sudoers sources found, quitting
sudo: unable to initialize policy plugin
Segmentation fault
As another note, I have emailed the guy in charge of all the Unix stuff to help me with, as have my advising professors, and he has done nothing. If at all possible, it would be best if there were a way to do this that did not involve me needing to get permissions from him.
Having virtualenv in .local/lib/python3.4/site-packages means that you've installed it only for you using --user option; hence the scripts are in ~/.local/bin. So do
setenv PATH ${PATH}:$HOME/.local/bin
I'm having a problem with this package that I installed in Python 3.5. After installing it, I try to run requestProxy.py but it won't import any of its own packages. Here's what I did, and what's happening.
I cloned it and created a private repo using these instructions.
I installed in an activated virtualenv, created without using sudo, using:
pip3 install -e HTTP_Proxy_Randomizer
Terminal said it installed ok.
I can find the egg link in my virtualenv's site-packages folder, but when I try to run the main file, it says:
from project.http.requests.parsers.freeproxyParser import freeproxyParser
ImportError: No module named project.http.requests.parsers.freeproxyParser
I had to write a setup.py for the package, which didn't seem to come with its own. I came up with:
setup(name='HTTP_Request_Randomizer',
version='1.0',
description='HTTP Proxy Request Randomizer',
package_dir={'project': 'project','http':'project/http',\
'requests':'project/http/requests','errors':'project/http/requests/errors',\
'parsers':'project/http/requests/parsers','proxy':'project/http/requests/proxy'},
packages=['project','http','requests','errors','parsers','proxy']
Here's the package structure:
pip3 freeze
gives me:
Complete output from command git config --get-regexp remote\..*\.url:
fatal: bad config file line 4 in /home/danny/.gitconfig
----------------------------------------
Error when trying to get requirement for VCS system Command "git config --get-regexp remote\..*\.url" failed with error code 128 in /home/danny/Documents/HTTP_Request_Randomizer, falling back to uneditable format
Could not determine repository location of /home/danny/Documents/HTTP_Request_Randomizer
Django==1.9.7
## !! Could not determine repository location
HTTP-Request-Randomizer==1.0
mysqlclient==1.3.7
So I want to have requestProxy.py install the other necessary packages and not fail at line 1. I'm sure this is a problem with my implementation and not the original author's coding. I was experimenting with this package a couple of weeks ago before I was aware of virtualenvs or pip install -e, and just copied it manually to site-packages. It worked then. Now I understand the concepts to do it more cleanly, but I can't get those to work.
It feels as though I have done something wrong with my git config or with my package_dir structure in setup.py, perhaps?
I've been pythoning for maybe a month and have a lot to learn. I normally find what I need on Stack Overflow without having to bother anyone, but after trying everything with this, I really need some help. Any advice much appreciated.
I figured it out. I was using Ninja IDE, and even though I entered the virtualenv for the project and restarted, it still wasn't recognizing it. I was able to run it from the terminal, and also in Pycharm and Liclipse.
so I tried to set up an working environment to code some stuff with python. I used this really nice website newcoder.io to do it the right way.
Unfortunately I came across a big problem I cannot solve by myself even after several hours of trial and error (sponsored by Google).
I installed all required packages starting from python, virtualenv, virtualenvwrapper. I also changed the .bashprofile for using Terminal as stated. Then I tried to test the working environment like described in the aforementioned website newcoder.io Test.
As I was within the working environment named "TestEnv" I tried to install django via pip. Here is the result coming from Terminal:
(TestEnv)username:~ username$ pip install django
-bash: /Users/username/.virtualenvs/TestEnv/bin/pip: "/Users/brokenusername: bad interpreter: No such file or directory
As you can see there seems to be something wrong with the underlying links. I have to admit, that I recently changed my account name and the name of the home directory in OS X Mavericks (see "username" and "brokenusername" within the code). Everything went fine so far. The username is now without blank spaces in the name (a different, bigger problem, but solved). But pip still seems to keep the old "brokenusername".
My Question is, how do I change the "brokenusername" to "username" so that pip is able to its work.
Thanks for all advices.
Here is a sloth for all your efforts to help !
Please note: I am newbie when it comes to understand and change these kind of working environments. I tried my best to find a solution by myself. But it seems like I need some advice from the Internetz.
virtualenv creates symlinks, environment variables and other path links in places like .pth files that are invalidated when you change the base path of the env. But environments are cheap to create so (assuming you haven't placed other files in the virtualenv directory) just delete it and build it again.
i wanted to use the pyfire github library https://github.com/mariano/pyfire
this is what pip freeze produced for me:
-e git+ssh://git#github.com/mariano/pyfire.git#db856bb666c4b381c08f2f4bf7c9ac7aaa233221#egg=pyfire-dev
but when doing so the clone that is being done to install the dependencies fails with error code, i can't
heroku run console
to check out the full error log...
any experience with this or ideas?
thanks a lot in advance
pip freeze appears to produce the wrong results you should be able to modify your requirements.txt to:
git+https://github.com/mariano/pyfire.git#db856bb666c4b381c08f2f4bf7c9ac7aaa233221#egg=pyfire-dev
I realized this was already solved for you, but this didn't work for me or #amrox or #tomtaylor.
If you remove the commit part, it works for me. i.e. changing the line in requirements.txt to:
git+https://github.com/mariano/pyfire.git
When I install the git repo including the specific commit locally, git seems to realize the end part is a specific commit, but when I try this on heroku and track the progress, it is pretty clear it is treating the commit part as a tag. Since there is no tag with that name, that's why it fails. They may be using a different version of git.
I'm trying to install python to a 1and1.com shared linux hosting account.
There is a nice guide at this address:
http://www.jacksinner.com/wordpress/?p=3
However I get stuck at step 6 which is: "make install". The error I get is as follows:
(uiserver):u58399657:~/bin/python > make install
Creating directory /~/bin/python/bin
/usr/bin/install: cannot create directory `/~’: Permission denied
Creating directory /~/bin/python/lib
/usr/bin/install: cannot create directory `/~’: Permission denied
make: *** [altbininstall] Error 1
I look forward to some suggestions.
UPDATE:
Here is an alternative version of the configure step to fix the above error, however this time I'm getting a different error:
(uiserver):u58399657:~ > cd Python-2.6.3
(uiserver):u58399657:~/Python-2.6.3 > ./configure -prefix=~/bin/python
configure: error: expected an absolute directory name for --prefix: ~/bin/python
(uiserver):u58399657:~/Python-2.6.3 >
The short version is, it looks like you've set the prefix to /~/bin/python instead of simply ~/bin/python. This is typically done with a --prefix=path argument to configure or some other similar script. Try fixing this and it should then work. I'd suggest actual commands, but it's been a while (hence my request to see what you've been typing.)
Because of the above mistake, it is trying to install to a subdirectory called ~ of the root directory (/), instead of your home directory (~).
EDIT: Looking at the linked tutorial, this step is incorrect:
./configure --prefix=/~/bin/python
It should instead read:
./configure --prefix=~/bin/python
Note, this is addressed in the very first comment to that post.
EDIT 2: It seems that whatever shell you are using isn't expanding the path properly. Try this instead:
./configure --prefix=$HOME/bin/python
Failing even that, run echo $HOME and substitute that for $HOME above. It should look something like --prefix=/home/mscharley/bin/python
You really should consider using the AS binary package from Activestate for this kind of thing. Download the .tar.gz file, unpack it, change to the python directory and run the install shell script. This installs a completely standalone version of python without touching any of the system stuff. You don't need root permissions and you don't need to mess around with make.
Of course, maybe you are a C/C++ developer, make is a familiar tool and you are experienced at building packages from source. But if any of those is not true then it is worth your while to try out the Activestate AS binary package.
I was facing same issue with 1and1 shared hosting (Your provided linked tutorial is not available now). I followed Installing Python modules on Hostgator shared hosting using VirtualEnv tutorial with only one change for 1and1. That is:
Instead of:
> python virtualenv-1.11.6/virtualenv.py /home1/yourusername/public_html/yourdomain.com/env --no-site-package
I used:
> python virtualenv-1.11.6/virtualenv.py /kunden/homepages/29/yourusername/htdocs/env --no-site-package
Rest of the instructions worked and I successfully installed VirtualEnv.
Example: 1and1 does not provide Requests module and pip cannot be used in shared hosting. This screenshot demonstrates that after installing VirtualEnv, pip command can be used and at the end >>> import requests successfully worked.