I'm trying to setup regular backups of rethinkdb, but keep running into issues. How do you setup rethinkdb-dump to run from cron?
Here is my script:
$ cat backup.sh
#!/bin/bash
NOW=$(date +"%Y-%m-%d-%H-%M")
/usr/bin/rethinkdb dump -e my_db -f /root/db_backup/$NOW.tar.gz
The script runs just fine when I run it manually. However, when try and run it from cron it doesn't work and I get the following at stderr:
Error when launching 'rethinkdb-dump': No such file or directory
The rethinkdb-dump command depends on the RethinkDB Python driver, which must be installed.
If the Python driver is already installed, make sure that the PATH environment variable
includes the location of the backup scripts, and that the current user has permission to
access and run the scripts.
Instructions for installing the RethinkDB Python driver are available here:
http://www.rethinkdb.com/docs/install-drivers/python/
It appears to be a Python environment issue, but I cannot figure out how to make it happy... thoughts? Help!
When you run it from that backup.sh script, it maybe run without correct PATH setup and cannot found the PATH of rethinkdb-dump.
First, let find out where is rethinkdb-dump
which rethinkdb-dump
(on my pc, I guess it's very different on your pc)
/usr/local/bin/rethinkdb-dump
Now, try to append the PATH to your script backup.sh
#!/bin/bash
export PATH="$PATH:/path/to/folder/contain-rethinkdb-dump"
# The rest of your script normally
So take my example, I will put it like this:
export PATH="$PATH:/usr/local/bin"
I think your rethinkdb-dump live outside normal bin folder (/usr/bin, /usr/local/bin etc)
The python installer for windows installs scripts and packages in subfolders here:
$env:APPDATA\Python\Python37 for powershell
%APPDATA%\Python\Python37 for cmd
cd this directory to see /Scripts and /site-packages (pip packages)
Related
OS : Windows 10
It is the first time to install python.
I want to run python on git bash.
I haven't used python or git bash before.
I installed portable git.
C:\Users\ username \Desktop\PortableGit
I created a small file named ib in Portable git folder, whose content is
I installed python
C:\Users\ username \AppData\Local\Programs\Python\Python310
Run git-bash.exe in portable git folder,
and check the version of python, and run ib.
python --version
./ib
It says no such file or directory. The code ./ib didn't work. Can you teach me?
If you need more information, you can ask me!(tasks/screenshots)
You need to add the directory which contains the python executable to the PATH variable. One way to do this is to edit .bashrc in your home directory and add a line similar to
export PATH=$PATH:/path/to/python
replacing /path/to/python with the directory where python3 is located.
I used this tutorial (rec by GoDaddy because I have a GoDaddy VPS)
https://www.godaddy.com/garage/how-to-install-and-configure-python-on-a-hosted-server/
to install a secondary version of Python (3.6.6) and create a Virtual Environment to run modules with the secondary version of Python. I want to be able to run scripts in the virtual environment but do not want to always use the admin account to do this. I have one other account set up that does not have admin privileges. Is there a way to extend authority to more than the admin account that created it? Do I need to do a virtual environment on every user account that is going to have access to run python?
It says to add a line to the bash_profile script. The script already had this:
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
This makes sense in that it seems to be defining the location of underlying Python (i.e., 2.6.6). I have modules in a venv that require 3.4+ so I created the virtual environment. I want to be able to call the script from Chrome and run it for some scripts and run it from cronjob for other scripts and the scripts need to always point to the virtualenv to use that installed interpreter. I have no idea how to do this correctly. Here's my system information and the results of some SSH printouts:
$ cd $PATH
-bash: cd: /home/flohosti/.local/bin:/home/flohosti/perl5/bin:/usr/local/cpanel/3rdparty/lib/path-bin:/usr/local/cpanel/3rdparty/lib/path-bin:/usr/local/cpanel/3rdparty/lib/path-bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/cpanel/composer/bin: No such file or directory
Before going into the VirtualEnv I run which python and get /usr/bin/python which I expected.
There are multiple ways to do that.
You can simply give read access to the directory where virtual environment is installed and then export its path in /etc/profile so that its automatically added to every users path.
Or you can give access to python binary inside the virtual env folder.
That being said this isn't the right way to go about it, most of the times you would want to have virtual environment inside your projet folder so that it is automatically available the project and who so ever has access to that project.
Edit 1
So the answer to your comment
Yes absolutely you can use the same virtual env to execute script in cron but you have to give absolute path to virtaul environment and your script in cron, like /home/flohosti/projectOne/env/bin/python <script path> or you can just set the PATH in cron and then you can use relative path as well.
I currently have an executable file that is running Python code inside a zipfile following this: https://blogs.gnome.org/jamesh/2012/05/21/python-zip-files/
The nice thing about this is that I release a single file containing the app. The problems arise in the dependencies. I have attempted to install files using pip in custom locations and when I embed them in the zip I always have import issues or issues that end up depending on host packages.
I then started looking into virtual environments as a way to ensure package dependencies. However, it seems that the typical workflow on the target machine is to source the activation script and run the code within the virtualenv. What I would like to do is have a single file containing a Python script and all its dependencies and for the user to just execute the file. Is this possible given that the Python interpreter is actually packaged with the virtualenv? Is it possible to invoke the Python interpreter from within the zip file? What is the recommended approach for this from a Python point of view?
You can create a bash script that creates the virtual env and runs the python scripts aswell.
!#/bin/bash
virtualenv .venv
.venv/bin/pip install <python packages>
.venv/bin/python script
I've added deploy commands to my Elastic Beanstalk deployment which download the Anaconda installer, and install it into /anaconda. Everything goes well, but I cannot seem to correctly modify the PATH of my instance to include /anaconda/bin as suggested by the Anaconda installation page. If I SSH into an instance and manually add it, everything works fine. But this is obviously not the correct approach, as machines will be added automatically by EB.
So my question is: how can I use Anaconda in my script?
A couple more details:
I've tried adding /anaconda/bin to the system PATH all ways I can think of. Pre/post deploy scripts, custom environment variables, etc. It seems that no matter what I do, the modifications don't persist to when the application is run.
I've tried to include Anaconda via adding it to sys.path: sys.path.append('/anaconda/bin')
to no avail. Using the following: sys.path.append('/anaconda/lib/python2.7/site-packages') allows me to import some packages but fails on import pandas. Strangely enough, if I SSH into the instance and run the application with their python (/opt/python/run/venv/bin/python2.7) it runs fine. Am I going crazy? Why does it fail on a specific import statement when run via EB?
Found the answer: import pandas was failing because matplotlib was failing to initialize, because it was trying to get the current user's home directory. Since the application is run via WSGI, the HOME variable is set to /home/wsgi but this directory doesn't exist. So, creating this directory via deployment command fixed this issue.
My overall setup to use Anaconda on Elastic Beanstalk is as follows:
.ebextensions/options.config contains:
commands:
00_download_conda:
command: 'wget http://repo.continuum.io/archive/Anaconda-2.0.1-Linux-x86_64.sh'
test: test ! -d /anaconda
01_install_conda:
command: 'bash Anaconda-2.0.1-Linux-x86_64.sh -b -f -p /anaconda'
test: test ! -d /anaconda
02_create_home:
command: 'mkdir -p /home/wsgi'
00_download_conda simply downloads Anaconda. See here for latest Anaconda version download link. The test commands are EB's way of letting you only execute the command if the test fails...Just prevents double downloading when in development.
01_install_conda installs Anaconda with options -b -f -p /anaconda which allows it to be installed in the specified directory, without user input, and skips installation if it has already been installed.
02_create_home creates the missing directory.
And finally - to use Anaconda inside your python application: sys.path.append('/anaconda/lib/python2.7/site-packages')
Cheers!
I have installed the new python release and would like to run .py files from the terminal.
How is this done from the terminal? I dont want to include the path in each command to run a .py file.
If you want to override the python command, you can set your PATH variable correctly, e.g. in your ~/.bash_profile:
export PATH=/path/to/python/:$PATH
That said, for managing different versions of components that are also provided by Mac OS X, I suggest to use a package manager such as Homebrew.
if you add a shebang at the start of the python file then you can run a python file by just its name from terminal
add #!/usr/bin/python
for mac(others add your respective path for python)
at the top of your python program and from your terminal you can run it just by filename(if it has executable permissions).
Have a look at the Python package under Applications. There is a shell script there called Update Shell Profile.command
Run this and it should set your path up properly.
Unless you mark you script as executable with chmod +x, you'll need to run python over it first. e.g. `python myscript.py'
I installed all of my python through macports, which has pros and cons. One of the benefits is that you don't have to worry about stuff like this, it just works. You can install python 2.6 and python 2.7 (and others), and then use the python_select utility to set up which python is run when you call "python blah.py"
Since you have installed a working python, the easiest way to run python files from the terminal is to cd your terminal to the directory where the file is located and then just type python my_code.py in the terminal.