I'm running ansible-playbook via something like
subprocess.run(['bash', '-c', "ansible-playbook some_playbook.yml"])
After uninstalling & reinstalling ansible-playbook my project somehow can't execute it anymore claims "bash: ansible-playbook: command not found".
Facts:
running on Ubuntu 20.04
ansible-playbook was installed via pip3
Tried the following:
ensured that the command is available and runnable via terminal & python3
deleted the venv folder and reinstalled everything (using PyCharm does it automatically)
deleted Bash cache via hash -r & hash -d ansible-playbook
rebooted
installed pyclean and ran it
deleted .idea folder (PyCharm)
recloned the repository into different directory and set up everything again. Same problem.
created a dummy test in the same project folder and found out that it runs fine:
> cat test.py
import subprocess
process = subprocess.run(['bash', '-c', "ansible-playbook some_playbook.yml"])
Everything worked fine before the reinstall. My program also runs various Bash commands before executing ansible-playbook and I swear on my left testicle that it's not my code. There must be some hidden mystery cache I'm missing.
Thanks for any suggestion!
To make my comment an answer:
If you've installed the package as a regular user with a recent pip, it will have installed the libraries and binaries into ~/.local/bin which isn't on PATH by default. (Pip will warn you about that.)
Once the binary is on PATH, you can skip the extra shell process (which is unnecessary and may cause Weird Things in certain situations) and not be vulnerable to shell injection attacks with
import subprocess, shutil
subprocess.run([shutil.which("ansible-playbook"), "some_playbook.yml"])
Related
I have just started working on my new pc and just to get a feel for it I wanted first to start working on python files, so I started first by just wanting to run WSL on windows and it installed correctly but when I want to run any python using the run python file on the top right on VS code, this is what gets executed $ C:/Users/jaffe/AppData/Local/Microsoft/WindowsApps/python3.10.exe f:/Projects/hello.py
And this is the error: -bash: C:/Users/jaffe/AppData/Local/Microsoft/WindowsApps/python3.10.exe: No such file or directory
I have no idea what's causing it but when I run the file using 'Shift + Enter' which is: Python: Run Selection/Line in Python Terminal it seems to run the single line correctly but it gives me this error instead:
print("Hello, world")
-bash: syntax error near unexpected token `"Hello, world"'
but when I run it using python3 hello.py, it works perfectly fine?! I'm so lost as to why this is happening and how could I fix it.
Might be relevant: I'm using windows 10, installed python 3.10.2 from windows store, all of that is in VS code and the python code is one line: print("Hello, world") and I changed the permissions of Local/Microsoft/WindowsApps so it's now accessible by all users to view/read/edit/run, made sure that python3.10.exe exists(on the WindowsApps and it works perfectly) and reinstalled it many times, tired python3.9, and tried to install python from the website instead of the windows store and still the same, manually added python to PATH and tried .venv and didn't work. when I launch python3.10.exe outside vs code it seems to run perfectly, I have worked with python before and it used to work fine now I don't know what's wrong.
I have seen other questions of the same problem I'm having here but none of them solve the problem.
No such file or directory C:/Users/...
For wsl, the Windows filesystem is accessible, but it has a different path. It is mounted under the /mnt folder. So you would find your python .exe under /mnt/c/Users/jaffe/AppData/Local/Microsoft/WindowsApps/python3.10.exe. This said, the executable file is meant to work on Windows, and it doesn't really makes sense to use it on Linux when you could run python within your wsl distro.
python3 works perfectly fine
This is because most Linux distributions come with python3 pre-installed, so you can use it already. To see where it is located, you can run the command which python3, or python3 --version to check its version.
If you want to change version, you may consider download it from you package manager, apt.
I also suggest to install python3-pip if you don't have it already to get the pip package manager for python.
In my case when I ran into this.. I discovered pyenv. This allows you to download more than one version of python. You can then go into a specific directory, such as your python project and issue a python local 3.10.0 (for example). Here's a link on how to install it as well as poetry which is a virtual environment manager that is become very popular. You can also create an alias for python that works off of this. I add this command to my alias file and source it from my .bashrc. alias python='pyenv exec python3'
I have recently started a new project in PyCharm, finally utilizing anaconda environments. However, after trying to make my first commit through PyCharm, it appears to be using my native python, not the environment set in PyCharm. I've tried to restart PyCharm several times, restarted my computer, and reinstalled the virtual environment.
Here is a copy of pre-commit hook:
set -e
# Run linters and tests
source scripts/lint.sh
The linter is the following: (which python has been added to highlight the issue)
set -e
set -v
which python
flake8 ~project name~
mypy ~project name~
pytest -x
black --check --fast --quiet ~project name~
set +v
I am running the commit through PyCharm -> VCS -> Commit. Inside PyCharm, the commit fails
(below this are a large amount of mypy errors, but note the environment)
However, if I run the commit from the terminal with $ git commit -m "testing commit" the commit works. It provides the following response:
This is the correct virtual environment inside of the project, seen here:
Am I setting something up incorrectly? I vastly prefer PyCharm's VCS and would prefer not to have to use git from the terminal.
PyCharm doesn't run git hooks under the virtual environment. The relevant ticket in the bug tracker: https://youtrack.jetbrains.com/issue/PY-12988
Here's what worked for me.
I was getting the following error:
18:37 Commit failed with error
0 file committed, 1 file failed to commit: Update pre-commit hooks
env: python3.7: No such file or directory
When I navigated to .git/hooks/pre-commit in my project repo, it turned out that the shebang line is #!/usr/bin/env python3.7.
This was an issue, since calling python3.7 on my MacOS would end up with the following:
zsh: command not found: python3.7
I could have either added a global python3.7 or, alternatively, updated the shebang. I went with the latter one and changed the shebang line to:
#!/usr/bin/env python3
This resolved the issue for me.
It seems that the aforementioned PyCharm ticket won't be fixed soon (it's there since 2014).
This hack below works for me; I added this to the PyCharm ticket:
This is a slightly annoying workaround that works for me:
Close PyCharm.
cd /your/project/dir
Open PyCharm from the command line: PYENV_VERSION="$(pyenv local | head -1)" open /Applications/PyCharm.app/. I'm using macOS, you
should adapt the open command to your OS.
I have to do it every time I switch projects, otherwise the pylint
pre-commit hook doesn't work. If you have a
similar config for your projects (Python version and not using
PyLint), just run PyCharm from the CLI once.
You can manually edit the auto-generated pre-commit file (located in your project dir at .git/hooks/pre-commit) to add the path to your virtual environment, replacing:
# start templated
INSTALL_PYTHON = 'PATH/TO/YOUR/ENV/EXECUTABLE'
with
# start templated
INSTALL_PYTHON = 'PATH/TO/YOUR/ENV/EXECUTABLE'
os.environ['PATH'] = f'{os.path.dirname(INSTALL_PYTHON)}{os.pathsep}{os.environ["PATH"]}'
None of the above solutions worked for me: PyCharm 2020.3 on Windows 10
What I did is to rename the .git\hooks\pre-commit -> .git\hooks\pre-commit.py
and created a new .git\hooks\pre-commit with next content:
#!/bin/bash
BASEDIR=$(dirname "$0")
/c/<PATH-to-YOUR-Python>/python.exe $BASEDIR/pre-commit.py $#
Worked as a charm!
I'm going through the Flask tutorial, and I encounter a problem where I can't use pip (and thus anything else really) a venv in Powershell. The same flask app runs correctly in an Ubuntu terminal with Python 3.6.6 in WSL.
The problem seems to depend on the directory, which makes me think it's somehow related to file path length; I enabled long file paths in the windows Group Editor but this hasn't fixed the problem. In the below steps my venv directory is c:\users\rwgpu\google-drive\code\flask-tutorial\winEnvflaskr\ and I experience the error, but everything works correctly in the directory C:\python\test\testVenv. (in each case I'm running the commands in the directory one up from the listed above, flask-tutorial and test respectively)
Minimal steps to reproduce are:
Completely fresh Python 3.7 installation.
Change directory to app folder.
py -m venv winEnvFlaskr
./winEnvFlaskr/Scripts/activate
pip list
The ultimate goal is to then
pip install FLask
and run the Flask tutorial app. Again, all steps work correctly in Bash on Ubuntu in WSL (running its own Python) and in a different windows directory with the same Python 3.7.
After pip list I get the error:
Fatal error in launcher: Unable to create process using '"c:\users\rwgpu\google-drive\code\flask-tutorial\winenvflaskr\scripts\python.exe" "C:\Users\rwgpu\Google-Drive\Code\flask-tutorial\winEnvFlaskr\Scripts\pip.exe" list'
and if I try
python -m pip list
I get nothing; the terminal hangs for a second and returns with no output. If I run
py -m pip install -U pip
in the bugged venv it will try to install and report success. It will do this again, and will never report "requirement already satisfied" which would be correct.
I had the same problem on Windows with running flask in command line from venv(for example, "(venv)...\flask run"). I resolved the problem with changing path in flask.exe code(open "your_venv\Scripts\flask.exe" with notepad or etc.): in my way I correct the 436-th line at the end; you need to put there "your_absolute_path_to_venv\Scripts\python.exe" instead of the path indicated there.
Good luck!
Well, I still don't know why this was happening -- it persisted through deleting and recreating the venv (obviously), but when I deleted the entire folder, that somehow cleared it up. I just copied the code into a new directory in the same parent and everything seems to be working ¯\_(ツ)_/¯
If anyone knows what would cause this in my case I still welcome input.
I'm trying to setup regular backups of rethinkdb, but keep running into issues. How do you setup rethinkdb-dump to run from cron?
Here is my script:
$ cat backup.sh
#!/bin/bash
NOW=$(date +"%Y-%m-%d-%H-%M")
/usr/bin/rethinkdb dump -e my_db -f /root/db_backup/$NOW.tar.gz
The script runs just fine when I run it manually. However, when try and run it from cron it doesn't work and I get the following at stderr:
Error when launching 'rethinkdb-dump': No such file or directory
The rethinkdb-dump command depends on the RethinkDB Python driver, which must be installed.
If the Python driver is already installed, make sure that the PATH environment variable
includes the location of the backup scripts, and that the current user has permission to
access and run the scripts.
Instructions for installing the RethinkDB Python driver are available here:
http://www.rethinkdb.com/docs/install-drivers/python/
It appears to be a Python environment issue, but I cannot figure out how to make it happy... thoughts? Help!
When you run it from that backup.sh script, it maybe run without correct PATH setup and cannot found the PATH of rethinkdb-dump.
First, let find out where is rethinkdb-dump
which rethinkdb-dump
(on my pc, I guess it's very different on your pc)
/usr/local/bin/rethinkdb-dump
Now, try to append the PATH to your script backup.sh
#!/bin/bash
export PATH="$PATH:/path/to/folder/contain-rethinkdb-dump"
# The rest of your script normally
So take my example, I will put it like this:
export PATH="$PATH:/usr/local/bin"
I think your rethinkdb-dump live outside normal bin folder (/usr/bin, /usr/local/bin etc)
The python installer for windows installs scripts and packages in subfolders here:
$env:APPDATA\Python\Python37 for powershell
%APPDATA%\Python\Python37 for cmd
cd this directory to see /Scripts and /site-packages (pip packages)
I've added deploy commands to my Elastic Beanstalk deployment which download the Anaconda installer, and install it into /anaconda. Everything goes well, but I cannot seem to correctly modify the PATH of my instance to include /anaconda/bin as suggested by the Anaconda installation page. If I SSH into an instance and manually add it, everything works fine. But this is obviously not the correct approach, as machines will be added automatically by EB.
So my question is: how can I use Anaconda in my script?
A couple more details:
I've tried adding /anaconda/bin to the system PATH all ways I can think of. Pre/post deploy scripts, custom environment variables, etc. It seems that no matter what I do, the modifications don't persist to when the application is run.
I've tried to include Anaconda via adding it to sys.path: sys.path.append('/anaconda/bin')
to no avail. Using the following: sys.path.append('/anaconda/lib/python2.7/site-packages') allows me to import some packages but fails on import pandas. Strangely enough, if I SSH into the instance and run the application with their python (/opt/python/run/venv/bin/python2.7) it runs fine. Am I going crazy? Why does it fail on a specific import statement when run via EB?
Found the answer: import pandas was failing because matplotlib was failing to initialize, because it was trying to get the current user's home directory. Since the application is run via WSGI, the HOME variable is set to /home/wsgi but this directory doesn't exist. So, creating this directory via deployment command fixed this issue.
My overall setup to use Anaconda on Elastic Beanstalk is as follows:
.ebextensions/options.config contains:
commands:
00_download_conda:
command: 'wget http://repo.continuum.io/archive/Anaconda-2.0.1-Linux-x86_64.sh'
test: test ! -d /anaconda
01_install_conda:
command: 'bash Anaconda-2.0.1-Linux-x86_64.sh -b -f -p /anaconda'
test: test ! -d /anaconda
02_create_home:
command: 'mkdir -p /home/wsgi'
00_download_conda simply downloads Anaconda. See here for latest Anaconda version download link. The test commands are EB's way of letting you only execute the command if the test fails...Just prevents double downloading when in development.
01_install_conda installs Anaconda with options -b -f -p /anaconda which allows it to be installed in the specified directory, without user input, and skips installation if it has already been installed.
02_create_home creates the missing directory.
And finally - to use Anaconda inside your python application: sys.path.append('/anaconda/lib/python2.7/site-packages')
Cheers!