Develop Reference

env command not working with find command

Im trying to write a script:
env PYTHONPATH=$PYTHONPATH: $Dir/scripts find * -name ‘*.py' -exec pylint (} \\; | grep . && exit 1
The code is finding all scripts in the root directory instead of using the environmental variables I set. Any help on writing this code to only look for files in the directory I set as a value in PYTHONPATH.

env PYTHONPATH=$PYTHONPATH: $Dir/scripts isn't doing what you think it's doing. Including $PYTHONPATH includes the former value of PYTHONPATH, meaning whatever you have it already set to or a blank default. The space in your variable also makes it invalid, and instead interprets the $Dir/scripts as a new command. It looks like what you want would be env PYTHONPATH=$Dir/scripts — but there's actually an easier way.
If you have __init__.py files in your directory, you can just do pylint ./some-directory. If you don't, you can use xargs: find . -type f -name "*.py" | xargs pylint. If you wanted to pass the directory instead of have it coded to . (your current calling directory) you could do that too:
# set directory to first argument
dir="$1"
# check if "dir" was actually provided, if not, set to `.`
if [ -z "$dir" ]; then dir=.; fi
find "$dir" -type f -name "*.py" | xargs pylint
You could save that in a script or function and then call it either with a directory (like run-pylint-on-everything.sh ~/foo/bar, or not, in which case it would run starting from your current shell location.

There’s no space between the PYTHONPATH value, it was a typo mistake, I want to run the command on a CLI instead of a script.

Shell/Terminal: Execute a command for all files in a directory using their absolute path

I'm trying to execute a command for each file in a directory but while using their absolute path (such as /home/richi/mydir/myfile.py) instead of their relative path (such as myfile.py).
In other words, I want to execute a command on files in a directory based on their absolute path - similar to for file in *.py; do thecommand -a "$file"; done but not quite.
I'm asking this because I'm trying to implement a Travis CI script running in an Ubuntu 14.04 environment which will install and use pyminifier to recursively minify all the Python code files in a directory.
Please note that I'm asking may be similar to this post but it's not.

Since you're on a standard Linux distro with a full userland, you can just use the realpath command:
Print the resolved absolute file name…
For example:
$ pwd
/home/abarnert/src/test
$ touch 1
$ realpath 1
/home/abarnert/src/test/1
That's it.
If you don't know how to use that from within bash, you can call a subcommand using $(…) syntax:
$ echo $(realpath 1)
/home/abarnert/src/test/1
Of course you want to pass it the value of the variable file, but that's just as easy:
$ file=1
$ echo $(realpath "$file")
/home/abarnert/src/test/1
I'm assuming you're using bash here. With a different sh-style shell, things will be different; with tcsh or zsh or fish or something, it may be even more different.
A really old userland, or a really stripped down one (e.g., for an embedded system) might not include realpath. In that case, you can use readlink, since the GNU version, as usually, adds everything including a couple kitchen sinks, and can be used as a realpath substitute.
Or, if worst comes to worst, Python has come with a realpath function since 2.2:
$(python -c 'import os,sys; print(os.path.realpath(sys.argv[1]))' "$file")

Bash script which sets pythonpath

The mssql-cli uses the following bash script to execute the actual python script. As I understand the code, the while loop determines the current directory of the script executed, this path gets then added to PYTHONPATH.
There are no .py files in the current directory so why is the path added to PYTHONPATH? Could someone please explain to me what the first part of the script is doing. Thank you for helping me out here.
#!/bin/bash
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != \/* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
# Set the python io encoding to UTF-8 by default if not set.
if [ -z ${PYTHONIOENCODING+x} ]; then export PYTHONIOENCODING=utf8; fi
export PYTHONPATH="${DIR}:${PYTHONPATH}"
python -m mssqlcli.main "$#"

Wonder if this is still relevant for you, but I've marked it as fun puzzle to revisit later... Long story short: It adds directory of location where this script file is with all symbolic links resolved (neither the acquired filename nor a directory leading up to is is a symbolic link) to PYTHONPATH.
It's basically the same thing as doing so using readlink (or realpath):
export PYTHONPATH="$(dirname $(readlink -f ${BASH_SOURCE})):${PYTHONOATH}"
Line by line dissection:
SOURCE="${BASH_SOURCE[0]}"
This sets SOURCE to be path with which this script was called or sourced.
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
We enter the wile loop if SOURCE path refers to a symbolic link. I.e. in the first iteration of this file was a symbolic link. Subsequently if this was a link pointing to another link.
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
This (a bit simplified explanation of -P) changes into directory where SOURCE is resolving symbolic links along the way (i.e. lands in the directory the link(s) was/were pointing to) and prints working directory after that change (absolute path). All that happens in a subshell and result is assign to variable DIR.
SOURCE="$(readlink "$SOURCE")"
SOURCE is assigned a new value of path resulting from symlink resolution. Literally a target the link points to (as seen by for instance ls -l) relative or absolute.
[[ $SOURCE != \/* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
If the SOURCE value we have obtained by symbolic link resolution does not begin with / (i.e. is an absolute path), DIR (directory where the SOURCE with which we have entered the loop resides) and resolved symbolic link SOURCE are concatenated over / to form a new SOURCE (we make it into an absolute path) and we go back to the top of this loop. NOTE: escaping of / by \ seems in this case unnecessary and arbitrary.
done
When done. SOURCE points to a file that is not a symblic link. It's path may still contain symbolic links at this point which is taken care of in the next step.
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
Once more, like in the loop. DIR should now be pointing to a directory where resolved (not a symlink) SOURCE file (target of what was originally called/sourced) resides.
# Set the python io encoding to UTF-8 by default if not set.
if [ -z ${PYTHONIOENCODING+x} ]; then export PYTHONIOENCODING=utf8; fi
Exports an environmental variable if a shell variable was not set or equals to an empty string. NOTE: ${PYTHONIOENCODING+x} seems to be an alternative form of ${PYTHONIOENCODING:+x} and its use seems absolutely arbitrary. There is also a test to check if variable was set (regardless of its value).
export PYTHONPATH="${DIR}:${PYTHONPATH}"
PYTHONPATH is now set to start with an absolute resolved path (no symbolic links should be anywhere along the path) of where does this very script (or file this link points to) reside.
python -m mssqlcli.main "$#"
Calls python...

How do I install a script to run anywhere from the command line?

If I have a basic Python script, with it's hashbang and what-not in place, so that from the terminal on Linux I can run
/path/to/file/MyScript [args]
without executing through the interpreter or any file extensions, and it will execute the program.
So would I install this script so that I can type simply
MyScript [args]
anywhere in the system and it will run? Can this be implemented for all users on the system, or must it be redone for each one? Do I simply place the script in a specific directory, or are other things necessary?

The best place to put things like this is /usr/local/bin.
This is the normal place to put custom installed binaries, and should be early in your PATH.
Simply copy the script there (probably using sudo), and it should work for any user.

Walkthrough of making a python script available anywhere:
Make a python script:
cd /home/el/bin
touch stuff.py
chmod +x stuff.py
Find out where your python is:
which python
/usr/bin/python
Put this code in there:
#!/usr/bin/python
print "hi"
Run in it the same directory:
python stuff.py
Go up a directory and it's not available:
cd ..
stuff.py
-bash: stuff.py: command not found
Not found! It's as we expect, add the file path of the python file to the $PATH
vi ~/.bashrc
Add the file:
export PATH=$PATH:/home/el/bin
Save it out, re apply the .bashrc, and retry
source ~/.bashrc
Try again:
cd /home/el
stuff.py
Prints:
hi
The trick is that the bash shell knows the language of the file via the shebang.

you can also use setuptools (https://pypi.org/project/setuptools/)
your script will be:
def hi():
print("hi")
(suppose the file name is hello.py)
also add __init__.py file next to your script (with nothing in it).
add setup.py script, with the content:
#!/usr/bin/env python3
import setuptools
install_requires = [
'WHATEVER PACKAGES YOU NEED GOES HERE'
]
setuptools.setup(
name="some_utils",
version="1.1",
packages=setuptools.find_packages(),
install_requires=install_requires,
entry_points={
'console_scripts': [
'cool_script = hello:hi',
],
},
include_package_data=True,
)
you can now run python setup.py develop in this folder
then from anywhere, run cool_script and your script will run.

Just create ~/bin and put export PATH=$PATH:$HOME/bin in your bashrc/profile. Don't mess with the system, it will bite you back, trust me.
Few more things (relevant to the question but not part of the answer):
The other way export PATH=$HOME/bin:$PATH is NOT safe, for bash will will look into your ~/bin folder for executables, and if their name matches with other executables in your original $PATH you will be surprised by unexpected/non working command execution.
Don't forget to chmod+x when you save your script in ~/bin.
Be aware of what you are putting in your ~/bin folder, if you are just testing something or working on unfinished script, its always better to use ./$SCRIPT_NAME from your CWD to execute the script than putting it under ~/bin.

The quick answer is to symlink your script to any directory included in your system $PATH.
The long answer is described below with a walk through example, (this is what I normally do):
a) Create the script e.g. $HOME/Desktop/myscript.py:
#!/usr/bin/python
print("Hello Pythonista!")
b) Change the permission of the script file to make it executable:
$ chmod +x myscript.py
c) Add a customized directory to the $PATH (see why in the notes below) to use it for the user's scripts:
$ export PATH="$PATH:$HOME/bin"
d) Create a symbolic link to the script as follows:
$ ln -s $HOME/Desktop/myscript.py $HOME/bin/hello
Notice that hello (can be anything) is the name of the command that you will use to invoke your script.
Note:
i) The reason to use $HOME/bin instead of the /usr/local/bin is to separate the local scripts from those of other users (if you wish to) and other installed stuff.
ii) To create a symlink you should use the complete correct path, i.e.
$HOME/bin GOOD ~/bin NO GOOD!
Here is a complete example:
$ pwd
~/Desktop
$ cat > myscript.py << EOF
> #!/usr/bin/python
> print("Hello Pythonista!")
> EOF
$ export PATH="$PATH:$HOME/bin"
$ ln -s $HOME/Desktop/myscript.py $HOME/bin/hello
$ chmod +x myscript.py
$ hello
Hello Pythonista!

Just create symbolic link to your script in /usr/local/bin/:
sudo ln -s /path/to/your/script.py /usr/local/bin/script

Putting the script somewhere in the PATH (like /usr/local/bin) is a good solution, but this forces all the users of your system to use/see your script.
Adding an alias in /etc/profile could be a way to do what you want allowing the users of your system to undo this using the unalias command. The line to be added would be:
alias MyScript=/path/to/file/MyScript

i find a simple alias in my ~/.bash_profile or ~/.zshrc is the easiest:
alias myscript="python path/to/my/script.py"

Type echo $PATH in a shell. Those are the directories searched when you type command, so put it in one of those.
Edit: Apparently don't use /usr/bin, use /usr/local/bin

Acording to FHS, the /usr/local/bin/ is the good place for custom scripts.
I prefer to make them 755 root:root, after copying them there.

Turn an application or script into a shell command

When I want to run my python applications from commandline (under ubuntu) I have to be in the directory where is the source code app.py and run the application with command
python app.py
How can I make it (how is it conventionally done) to run the application from arbitrary directory with the command: app ? Similarly as you type ls, mkdir and other commands?
thank you

Add a shebang line at the beginning of your file:
#!/usr/bin/env python
Make your file executable by calling
chmod +x app.py
in the shell.
Move it to some location included in the PATH environment variable and rename it to app. Alternatively, add the path of the directory containing app to the PATH environment variable by adding the line
export PATH=$PATH:/path/to/app
to your .bash_profile.

Add the directory that the script is in to your path, make it executable, and add a proper shebang line.
In your .bashrc:
PATH=$PATH:/dir/to/the/script
Executable:
chmod +x myscript.py
At the top of the script, add the shebang line:
#!/usr/bin/env python
Then, from anywhere, you can just do:
myscript.py
(Note that you don't need a .py suffix, it could be called anything, e.g. app if you have a proper shebang line).

Add a shebang: as the top line of the file: #!/usr/bin/python or #!/usr/bin/python3 (you can use the python -B to prevent generation of .pyc files, which is why I don't use /usr/bin/env)
Make it executable: You will need to do chmod +x app.py
(optional) Add directory to path, so can call it anywhere: Add a directory with your executable to your $PATH environment variable. How you do so depends on your shell, but is either export PATH=$PATH:/home/you/some/path/to/myscripts (e.g. Linux distros which use bash) or setenv PATH $PATH:/home/you/some/path/to/myscripts (e.g. tcsh like in Mac OS X). You will want to put this, for example, in your .bashrc or whatever startup script you have, or else you will have to repeat this step every time you log in.
app.py will need to be in the myscripts (or whatever you name it) folder. You don't even need to call it app.py, but you can just rename it app.
If you wish to skip step #3, you can still do ./app to run it if you are in the same directory.

Probably you want to symlink to your file location instead of adding another location to the path
chmod +x app.py
ln ~app.py /opt/local/bin/app
...assuming that /opt/local/bin is already in your path,.
Also, do not forget to add the shebang line to the first line of your script:
#!/usr/bin/env python

A solution some what different from the ones mentioned here: Use an alias.
alias app='python /path/to/app.py'
Add the above line to your ~/.bashrc or ~/.bash_login file (or preferably to ~/.bash_aliases if you are on Ubuntu). Then you can simply use your script as a command line tool with app.
No need to add a shebang (thereby modifying your existing Python script), no need to make the script executable and no need to change your PATH.

I'm pretty sure you have to make the script executable via chmod +x and put it in the PATH variable of your system.