Develop Reference

Conda: Command not found

I'm a 17-year-old, and I am new to programming. I installed anaconda and python on my laptop however every time that I run code in Sublime Text it gives this error before printing whatever I want it to print:
/Users/eyan/.bash_profile: line 20: conda: command not found
It's really annoying so is there a way to get rid of it.

The difference between bash and zshell are minimal. They are both a "shell" that run in the terminal. A shell is what allows you to run commands in the terminal. It has its own language called bash or alternatively shell scripting. If you have ever typed ls or cd in a terminal, you are using bash commands to move around or look at things in the terminal. (The terminal itself is just the little oftentimes black square you type into.) It doesn't really matter which one you pick at this point, but you should just go with one.
If you pick bash, the files that interact with it will be under your home directory, the error tells you exactly where. /Users/eyan/.bash_profile the .bash_profile sets some configurations for bash.
If you pick zshell, the files will instead of starting with '.bash_ they'll start with .zsh the most common one will be .zshrc and it will be in the same home directory. So /Users/eyan/.zshrc will be the config file for zshell.
Pick one and just stay with it for now.
In either case, you want to initialize conda.
In your terminal, if you type:
which conda
It should tell you where your conda is installed. If you get a bunch of nonsense running that, try which python. One of them should give you something like this output:
/Users/eyan/opt/anaconda3/bin/python
Change the python in that line to activate and add the command source to the front. So type:
source /Users/eyan/opt/anaconda3/bin/activate
Once you've done that type:
conda init
All of this is done in the terminal. And that should add a bunch of stuff to your shell profiles. (That's those files in your home directory like .bash and .zshrc.)
If you close and reopen your terminal or type EXEC $SHELL in the same terminal window, you should be able to use conda.
You can also see what was added to your bash (or zsh) profiles by typing:
cat /Users/eyan/.bash_profile
(Or if you've chosen zshell, run cat /Users/eyan/.zshrc)
You'll see that conda added quite a few lines to it. Don't worry about what those lines do for now, but you can see them there.

That means that your shell doesn't know where to find conda binary. In order for him to know where to look for it you should run:
/absolute/path/to/anaconda3/folder/bin/conda init
source ~/.bash_profile
usually the default path should be ~/anaconda3 and therefore you should be good with
~/anaconda3/bin/conda init
source ~/.bash_profile

Loading a virtualenv in a job file to run on linux cluster

I'm trying to set up a job file to run on a linux cluster using
$ qsub network.job
I have loaded the module needed to execute the script, however a also need to activate a virtual environment that comes with the module.
#$ -S /bin/bash
#$ -cwd
#$ -j y
#$ -o output/
echo "Running job script"
module load python/python3
./network.py
echo "Finished job script"
Including
$source activate machinelearning
does not activate the virtualenv
What do I need to add to the file to activate the virtualenv machine learning that comes when loading the module python/python3?

source /PATH/WHERE/YOUR/VIRTUAL/ENVIRONMENT/EXISTS/bin/activate
for example in the subdirectory venv of your homedir
source ~/venv/machinelearning/bin/activate

Judging by your comment response, the source command may not be the problem. The typical approach in python virtual environments is to call source on the activate script inside the virtual environment. For more information on source, see this superuser post.
In this case, since the command source activate machinelearning works for you in an interactive session, it's likely that there's a script called activate in your home directory (or whichever directory you were working from) that takes machinelearning as an argument, and activates the appropriate virtual environment.
Here's my best stab at solving your immediate problem:
Since you are able to activate the virtual environment in an interactive session, log in to an interactive session and activate the virtual environment. Once you have done so, type which python3 in your terminal, and it should give you the full path to the virtual environment you're working in. Take that full path and replace the last python3 with activate, and that is the script you should source. For example:
$>which python3
/home/cmf05/environments/machinelearning/bin/python3
Then place this in your script between module load python/python3 and ./network.py:
source /home/cmf05/environments/machinelearning/bin/activate
As an aside: module load isn't something that's built in to any systems I've ever worked on, but it looks like you're submitting a job to a cluster that's managed by slurm or similar. It seems like this is loading all of the specific files you need for your compute job. I would recommend tagging the question with your specific cluster computing environment and putting that in the title for more focused help, in case this doesn't solve your issue.

Activating conda environment in bash script that runs on startup

So I have a python script that generates an animation - and it requires libraries that I have in a conda environment. I need to run this script as soon as my computer turns on, so I've written a short bash script that I added to "startup applications". This bash script runs on startup, and reads like this:
#!/bin/bash
conda activate myenv
cd ~/scripts
python generate.py
When I run this in terminal myself, it's fine, but whenever I turn on the computer, the python part of the script doesn't execute, and when I check errors i find:
conda: command not found
and then i also see the python script failed to run because it's missing libraries (from the conda environment not activating)
I have tried adding lines to the bash script replacing "conda activate" with "source activate", I have tried adding echo ". /home/<user>/anaconda3/etc/profile.d/conda.sh" >> ~/.bashrc to the bash script, replacing "conda" with /home/barrat/anaconda3/bin/conda, and even adding whoami to the bash script that runs at startup to make sure that i haven't magically become root by chance... none of this has worked. I would really appreciate any help. it's 3 AM and i'm a bit desperate.

You might have solved the issue yet, but for future viewers, this worked for me:
if [ -f "/path/to/anaconda3/etc/profile.d/conda.sh" ]; then
. "/path/to/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate myenv
fi
Add this instead of conda activate myenv.

Well as you are trying to activate an environment to start your scripts, it may also be possible for you to make a startup script itself to do the desired task by using subprocess module from python.
Try making a demo.py script like :
import os
import system
import subprocess
import x
subprocess.run(["command name", "value"]) #for all scripts you want to execute
and then you can put this python script to run at startup.
You can start quite a few amount of operations without noticable speed changes to your system and always can monitor it easily by starting the processes one after the other using time.sleep() in between two calls.

bash: 'command not found' for python, r and nano

On git bash, I receive a command not found message for python, r and nano.
I am on Windows 10. I do not know what I am doing wrong.
I type
r --version
python --version
nano --version
and they all return command not found.
I read something that had me try
alias python='winpty python.exe'
and it renamed python, but when I tried
touch .bashrc
it returned
touch: command not found
I followed the instructions in a Github course, PS239T. I have been fortunate, except for getting these three programs to work.
I tried
build/console.exe c:/Python36/python.exe
and it returned
build/console.exe: No such file or directory
I tried
PATH=$PATH:/c/Python36/
$ export PATH="$PATH:/c/Python36"
echo 'export PATH="$PATH:/c/Python36"' > .profile
and nothing.
I got Python 64-bit via Anaconda.
I tried setting up the Python 3.4 environment
conda create –n py34 python=3.4 anaconda
./python name_of_your_python_script.py
nothing.
which python
got me a huge directory path:
which: no python in (/c/Users/Edward/bin:/mingw64/bin:/usr/local/bin:usr
/bin:/bin:/mingw64/bin:/usr/bin:c/user/Edward/bin:/c/Program Files
(X86)/Intel/ 1CLS client:/c/Program Files/Intel/1CLS Client:/c/Windows
/System32:/C/Windows:/C/Windows/Sytem32/Wbem:/c/Windows/System32
WindowsPowerShell/V1.0:/c/Program Files/Intel/Intel(R) Management Engine
Components/Dal:/c/Program Files/Intel/Intel(R) Management Engine
Components/IPT:/c/Program Files (x86)/Intel/Intel(R) magament Engine
Components/IPT:/c/Program Files (X86)/Intel/Intel(R) Management Engine
Components/IPT:/c/WINDOWS/Syste,32/Wbem:/c/WINDOWS/System32
/WindowsPowerShell/v1.0:/c/Program Files (x86)/Rand McNally/RNDDock
/GtkSharp/2.12/bin/:/c/Program Files d/AppData/Local/Mircosoft
/WindowsApps:/c/Users/Edward/AppData/Local/Pandoc:/usr/bin/vendor_perl:
/usr/bincore_perl:/c/python36
I would have just put up the screenshot or copied and pasted it, but I do not know how.
I am sure I made an error somewhere.
I get the same type of results when I try to figure out what is going on with r and nano.
Update 1
Download and install 'Anaconda'
'https://store.continuum.io/cshop/anaconda/'.
Download the default Python 3 installer 'do not follow the link to version 2'.
Use all of the defaults for installation except make sure to check 'Make Anaconda the default Python.'
Install R by downloading and running
'this .exe file from CRAN'
'http://cran.r-project.org/bin/windows/base/release.htm'.
Also, please install the 'RStudio IDE'
'http://www.rstudio.com/ide/download/desktop'.
All you need is RStudio Desktop.
nano is a basic editor and the default that we use in this class. To install
it, download the Software Carpentry Windows installer
http://files.software-carpentry.org/SWCarpentryInstaller.exe'
and double click on the file to run it. 'This installer requires an active internet connection.'
Sublime Text is a more advanced editor. Download Sublime Text 3 'here'
'https://www.sublimetext.com/3'.
Update 2
I began trying janos advice and ran
/c/Python36/python.exe --version
and it returned
bash: '/c/python36/python.exe: No such file or directory.
I tried to run
PATH="/c/Python36:$PATH" python.exe --version
but it returned
bash: python.exe.: command not found
I tried to run
PATH="/c/Python36:$PATH" python --version
and it returned
bash: python: command not found
Update 3
janos told me I went too far without figuring out the problem, then told me to run
ls -l /c/Python36/python.exe
and it returned
ls: cannot access '/c/Python36/python.exe': no such file or directory.
janos told me to focus on the first error
bash: '/c/python36/python.exe: No such file or directory.
after entering
/c/Python36/python.exe --version
and so there I will focus.
janos asserted
Can you open a file explorer and navigate to C:\Python36, and see python.exe there? Probably not.
and is correct.
I ran
C/Users/Edward/Anaconda3/pkgs/python-3.6.3-h9e2ca53_1/python.exe
with no success.
I ran that in explorer and found that the publisher could not be verified.
Update 4
I went to
'python.org/downloads/release/python-363'
and downloaded 'Windows x86 embeddable zip file' and Unzipped it in
C/Users/Edward/Anaconda3/pkgs
it worked.
Thank you.
I bet I am having the same problems with 'r' and 'nano'.
I am going to run down the list again and perform all the check. I will do the same thing with 'r' and 'nano'.
Everything worked out for Python. I will do 'nano' and 'r' in due course.
I could not find
/c/python36/
so I used
/c/Users/Edward/Anaconda3/pkgs/python-3.6.3-embed-win32/
Update 5
janos is right again. I would need to create the python36 folder as it was not built automatically during the installation. janos' method worked on 'r' and 'subl' or sublime text 3. I gave up on 'nano'.
I found that some of my '.exe' files were in listing with spaces in the names. I moved to the user folder and eliminated any spaces.
I also downloaded version from the official website, and that too helped.
I could not figure out how to get 'nano' to work. I could not even find it on my pc. I used 'subl' instead.

Start from the basics. Try the simplest thing that can possibly work,
and make progress in baby steps,
from one sane state to the next.
The first step is to run a by entering its absolute path directly.
If the Python executable is at /c/Python36/python.exe, then run this:
/c/Python36/python.exe --version
This is very simple and it should just work.
If it doesn't, the command will give you an error message with clues.
A next step could be to simulate adding to PATH. Try this:
PATH="/c/Python36:$PATH" python.exe --version
This is one line. It sets a value to PATH in a way such that it's only active during the execution of the command. After the command completes, the value of PATH will be back to what it was before. This is a good way to test things. Also notice that I prepended to PATH the directory that contains python.exe, not the full path to python.exe.
That's an important point, this is how the PATH variable works.
It's a list of directories, and all executable files in those directories become easily executable by simply typing their name, without having to type their absolute paths.
Next, I would try this:
PATH="/c/Python36:$PATH" python --version
That is, see if you can drop the .exe from the name of the command.
I don't have Windows so I cannot test if this works.
And maybe it doesn't. (But I think it does.)
If everything worked so far, then the next step is to make the PATH setting permanent.
The way to do that is to put the command PATH="/c/Python36:$PATH" into a file that is always executed when you start a new Git Bash session.
If I remember correctly on Windows you can put it in ~/.profile
(a file named .profile in your home directory).
Where is ~? Here's one way to find it:
cd
explorer .
The above opens a file manager inside that directory.
You can use a plain-text editor like Notepad or Wordpad to edit it.
You can also use this shell command to append the line that updates PATH:
echo 'PATH="/c/Python36:$PATH"' >> ~/.profile
This line will get executed in all new Git Bash session.
Not in the current session,
because this file is only executed once.
If everything above worked, then in a new Git Bash session you should be able to run python --version.
If not everything worked, then you need to read the error message you get carefully, and not advance to the next step until the problem is resolved.
It's useless to advance to a next step when you are already not in a sane state.
You can follow the exact same logical process for all the other programs too.

Problems running rethinkdb-dump from cron

I'm trying to setup regular backups of rethinkdb, but keep running into issues. How do you setup rethinkdb-dump to run from cron?
Here is my script:
$ cat backup.sh
#!/bin/bash
NOW=$(date +"%Y-%m-%d-%H-%M")
/usr/bin/rethinkdb dump -e my_db -f /root/db_backup/$NOW.tar.gz
The script runs just fine when I run it manually. However, when try and run it from cron it doesn't work and I get the following at stderr:
Error when launching 'rethinkdb-dump': No such file or directory
The rethinkdb-dump command depends on the RethinkDB Python driver, which must be installed.
If the Python driver is already installed, make sure that the PATH environment variable
includes the location of the backup scripts, and that the current user has permission to
access and run the scripts.
Instructions for installing the RethinkDB Python driver are available here:
http://www.rethinkdb.com/docs/install-drivers/python/
It appears to be a Python environment issue, but I cannot figure out how to make it happy... thoughts? Help!

When you run it from that backup.sh script, it maybe run without correct PATH setup and cannot found the PATH of rethinkdb-dump.
First, let find out where is rethinkdb-dump
which rethinkdb-dump
(on my pc, I guess it's very different on your pc)
/usr/local/bin/rethinkdb-dump
Now, try to append the PATH to your script backup.sh
#!/bin/bash
export PATH="$PATH:/path/to/folder/contain-rethinkdb-dump"
# The rest of your script normally
So take my example, I will put it like this:
export PATH="$PATH:/usr/local/bin"
I think your rethinkdb-dump live outside normal bin folder (/usr/bin, /usr/local/bin etc)

The python installer for windows installs scripts and packages in subfolders here:
$env:APPDATA\Python\Python37 for powershell
%APPDATA%\Python\Python37 for cmd
cd this directory to see /Scripts and /site-packages (pip packages)