Develop Reference

Python3: command not found, when running from cli

I am trying to run my python module as a command, however I am always getting the error: command not found.
#!/usr/bin/env python
import sys
import re
from sys import stdin
from sys import stdout
class Grepper(object):
def __init__(self, pattern):
self.pattern = pattern
def pgreper(self):
y = (str(self.pattern))
for line in sys.stdin:
regex = re.compile(y)
x = re.search(regex, line)
if x:
sys.stdout.write(line)
if __name__ == "__main__":
print("hello")
pattern = str(sys.argv[1])
Grepper(pattern).pgreper()
else:
print("nope")
I am sure whether it has something to do with the line:
if __name__ == "__main__":
However I just can't figure it out, this is a new area for me, and it's a bit stressful.

Your script name should have a .py extension, so it should be named something like pgreper.py.
To run it, you need to do either python pgreper.py pattern_string or if it has executable permission, as explained by Gabriel, you can do ./pgreper.py pattern_string. Note that you must give the script path (unless the current directory is in your command PATH); pgreper.py pattern_string will cause bash to print the "command not found" error message.
You can't pass the pattern data to it by piping, IOW, cat input.txt | ./pgreper.py "pattern_string" won't work: the pattern has to be passed as an argument on the command line. I guess you could do ./pgreper.py "$(cat input.txt)" but it'd be better to modify the script to read from stdin if you need that functionality.
Sorry, I didn't read the body of your script properly. :embarrassed:
I now see that your pgreper() method reads data from stdin. Sorry if the paragraph above caused any confusion.
By way of apology for my previous gaffe, here's a slightly cleaner version of your script.
#! /usr/bin/env python
import sys
import re
class Grepper(object):
def __init__(self, pattern):
self.pattern = pattern
def pgreper(self):
regex = re.compile(self.pattern)
for line in sys.stdin:
if regex.search(line):
sys.stdout.write(line)
def main():
print("hello")
pattern = sys.argv[1]
Grepper(pattern).pgreper()
if __name__ == "__main__":
main()
else:
print("nope")

Make sure you have something executable here : /usr/bin/env.
When you try to run your python module as a command, it will call this as an interpreter. You may need to replace it with /usr/bin/python or /usr/bin/python3 if you don't have an env command.
Also, make sure your file is executable : chmod +x my_module.py and try to run it with ./my_module.py.

python 3 open terminal and run program

I made a small script in sublime that will extract commands from a json file that is on the user's computer and then it will open the terminal and run the settings/command. This works, except that it doesn't really open up the terminal. It only runs the command (and it works, as in my case it will run gcc to compile a simple C file), and pipes to STDOUT without opening up the terminal.
import json
import subprocess
import sublime_plugin
class CompilerCommand(sublime_plugin.TextCommand):
def get_dir(self, fullpath):
path = fullpath.split("\\")
path.pop()
path = "\\".join(path)
return path
def get_settings(self, path):
_settings_path = path + "\\compiler_settings.json"
return json.loads(open(_settings_path).read())
def run(self, edit):
_path = self.get_dir(self.view.file_name())
_settings = self.get_settings(_path)
_driver = _path.split("\\")[0]
_command = _driver + " && cd " + _path + " && " + _settings["compile"] + " && " + _settings["exec"]
proc = subprocess.Popen(_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
I'm not sure if using subprocess.Popen is the right way to go about it as I'm new to Python.
So to re-iterate; I want it to open up the terminal, run the command, and have the terminal stay open until the user presses ENTER or something. I'm running Windows 7 and Python 3, if that matters.

subprocess.Popen simply creates a subprocess with the given command. It is in no way related to opening a terminal window or any other windows for that matter.
You'll have to look into your platform specific UI automation solutions in order to achieve what you want. Or see if maybe the Sublime plugins mechanism can already do that.
NOTES:
Also, you should be using os.path.join/os.path.split/os.path.sep etc for your path operations—Sublime also runs on OS X for example, and OS X does not use backslashes. Also, file handles need to be closed, so use:
with open(...) as f:
return json.load(f) # also not that there is no nead to f.read()+json.loads()
# if you can just json.load() on the file handle
Furthermore, strings should usually be built using string interpolation:
_command = "{} && cd {} && {} && {}".format(_driver, _path, _settings["compile"], _settings["exec"])
...and, you should not be prefixing your local variables with _—it doesn't look nice and serves no purpose in Python either; and while we're at it, I might as well use the chance to recommend you to read PEP8: http://www.python.org/dev/peps/pep-0008/.

Why I got "sh: 1: Syntax error: Unterminated quoted string" when I run my Python program?

I want to count how many lines of code I have written.
Here is the Python code:
import os
import sys
EXT = ['.c','.cpp','.java','.py']
def main():
l = []
if os.path.isdir(sys.argv[1]):
for root, dirs, files in os.walk(sys.argv[1]):
l.extend([os.path.join(root, name) for name in files])
else:
l.append(sys.argv[1])
params = ["'"+p+"'" for p in l if os.path.splitext(p)[1] in EXT]
result = os.popen("wc -l %s "%" ".join(params)).read()
print result
if __name__ == '__main__':
main()
Before this, it was running as expected. But today, it give me this error:
sh: 1: Syntax error: Unterminated quoted string
I don't know what happened.

Your Python script is missing a shebang line. Add the following to the top of your file:
#!/usr/bin/env python
Then you should be able to run the following, assuming your script is at /path/to/your_script.py and it has the executable bit set:
/path/to/your_script.py arg1 arg2 [...]
Alternatively:
python /path/to/your_script.py arg1 arg2 [...]
Update following comments
I suspect what has changed is that a source file containing a ' in its name has been added to the directory you are checking and the shell is choking on this.
You could add the following function to your program:
def shellquote(s):
return "'" + s.replace("'", "'\\''") + "'"
[Lifted from Greg Hewgill's answer to How to escape os.system() calls in Python? .]
And call it like this:
params = [shellquote(p) for p in l if os.path.splitext(p)[1] in EXT]

#Johnsyweb's updated answer seems to have the correct diagnostic, but the correct fix is to not use a shell to invoke wc. Try something like this instead:
cmd = ['/bin/wc', '-l'] # Need full path!
[cmd.extend(p) for p in l if os.path.splitext(p)[1] in EXT]
result = os.popen2(cmd).read()
Note that the subprocess module is the recommended solution now. Switching to that requires a less intrusive change to your current code, though; see http://docs.python.org/2/library/subprocess.html#replacing-os-popen-os-popen2-os-popen3

Looks like your Python program was parsed like a shell script. Add something like this at the header to indicate where your Python is:
#!/usr/bin/python
or you just run python a.py.

how can i get the executable's current directory in py2exe?

I use this bit of code in my script to pinpoint, in a cross-platform way, where exactly it's being run from:
SCRIPT_ROOT = os.path.dirname(os.path.realpath(__file__))
Pretty simple. I then go on to use SCRIPT_ROOT in other areas of my script to make sure everything is properly relative. My problem occurs when I run it through py2exe, because the generated executable doesn't set __file__, therefore my script breaks. Does anyone know how to fix or work around this?

Here is the py2exe documentation reference and here are the relevant items:
sys.executable is set to the full pathname of the exe-file.
The first item in sys.argv is the full pathname of the executable, the rest are the command line arguments.
sys.frozen only exists in the executable. It is set to "console_exe" for a console executable, to "windows_exe" for a console-less gui executable, and to "dll" for a inprocess dll server.
__file__ is not defined (you might want to use sys.argv[0] instead)
It is not apparent from those docs whether "the exe-file" and "the executable" are the same thing, and thus whether sys.executable and sys.argv[0] are the same thing. Looking at code that worked for both script.py and py2exe_executable.exe last time I had to do this, I find something like:
if hasattr(sys, 'frozen'):
basis = sys.executable
else:
basis = sys.argv[0]
required_folder = os.path.split(basis)[0]
As I say that worked, but I don't recall why I thought that was necessary instead of just using sys.argv[0].
Using only basis was adequate for the job in hand (read files in that directory). For a more permanent record, split something like os.path.realpath(basis).
Update Actually did a test; beats guesswork and armchair pontification :-)
Summary: Ignore sys.frozen, ignore sys.executable, go with sys.argv[0] unconditionally.
Evidence:
=== foo.py ===
# coding: ascii
import sys, os.path
print 'sys has frozen:', hasattr(sys, 'frozen')
print 'using sys.executable:', repr(os.path.dirname(os.path.realpath(sys.executable)))
print 'using sys.argv[0]:', repr(os.path.dirname(os.path.realpath(sys.argv[0] )))
=== setup.py ===
from distutils.core import setup
import py2exe
setup(console=['foo.py'])
=== results ===
C:\junk\so\py2exe>\python26\python foo.py
sys has frozen: False
using sys.executable: 'C:\\python26'
using sys.argv[0]: 'C:\\junk\\so\\py2exe' # where foo.py lives
C:\junk\so\py2exe>dist\foo
sys has frozen: True
using sys.executable: 'C:\\junk\\so\\py2exe\\dist'
using sys.argv[0]: 'C:\\junk\\so\\py2exe\\dist' # where foo.exe lives

Py2exe does not define __file__: http://www.py2exe.org/index.cgi/Py2exeEnvironment
The OP requested a py2exe friendly version of:
SCRIPT_ROOT = os.path.dirname(os.path.realpath(__file__))
The best answer is to determine if python is frozen in an exe, py2exe has documentation on this:
http://www.py2exe.org/index.cgi/HowToDetermineIfRunningFromExe
import imp, os, sys
def main_is_frozen():
return (hasattr(sys, "frozen") or # new py2exe
hasattr(sys, "importers") # old py2exe
or imp.is_frozen("__main__")) # tools/freeze
def get_main_dir():
if main_is_frozen():
return os.path.dirname(sys.executable)
return os.path.dirname(os.path.realpath(__file__))
SCRIPT_ROOT = get_main_dir()
Since, the python is EAFP, here's an EAFP version ...
try:
if sys.frozen or sys.importers:
SCRIPT_ROOT = os.path.dirname(sys.executable)
except AttributeError:
SCRIPT_ROOT = os.path.dirname(os.path.realpath(__file__))
Cheers!

sys.argv[0] is a reliable way to get the path, as it will give the same result irrespective of being run as a script or exe
. To get the directory os.path.dirname(sys.argv[0])

Try this:
import os
import sys
os.path.realpath(os.path.dirname(sys.argv[0]))

Equivalent of shell 'cd' command to change the working directory?

cd is the shell command to change the working directory.
How do I change the current working directory in Python?

You can change the working directory with:
import os
os.chdir(path)
There are two best practices to follow when using this method:
Catch the exception (WindowsError, OSError) on invalid path. If the exception is thrown, do not perform any recursive operations, especially destructive ones. They will operate on the old path and not the new one.
Return to your old directory when you're done. This can be done in an exception-safe manner by wrapping your chdir call in a context manager, like Brian M. Hunt did in his answer.
Changing the current working directory in a subprocess does not change the current working directory in the parent process. This is true of the Python interpreter as well. You cannot use os.chdir() to change the CWD of the calling process.

Here's an example of a context manager to change the working directory. It is simpler than an ActiveState version referred to elsewhere, but this gets the job done.
Context Manager: cd
import os
class cd:
"""Context manager for changing the current working directory"""
def __init__(self, newPath):
self.newPath = os.path.expanduser(newPath)
def __enter__(self):
self.savedPath = os.getcwd()
os.chdir(self.newPath)
def __exit__(self, etype, value, traceback):
os.chdir(self.savedPath)
Or try the more concise equivalent(below), using ContextManager.
Example
import subprocess # just to call an arbitrary command e.g. 'ls'
# enter the directory like this:
with cd("~/Library"):
# we are in ~/Library
subprocess.call("ls")
# outside the context manager we are back wherever we started.

cd() is easy to write using a generator and a decorator.
from contextlib import contextmanager
import os
#contextmanager
def cd(newdir):
prevdir = os.getcwd()
os.chdir(os.path.expanduser(newdir))
try:
yield
finally:
os.chdir(prevdir)
Then, the directory is reverted even after an exception is thrown:
os.chdir('/home')
with cd('/tmp'):
# ...
raise Exception("There's no place like /home.")
# Directory is now back to '/home'.

I would use os.chdir like this:
os.chdir("/path/to/change/to")
By the way, if you need to figure out your current path, use os.getcwd().
More here

If you're using a relatively new version of Python, you can also use a context manager, such as this one:
from __future__ import with_statement
from grizzled.os import working_directory
with working_directory(path_to_directory):
# code in here occurs within the directory
# code here is in the original directory
UPDATE
If you prefer to roll your own:
import os
from contextlib import contextmanager
#contextmanager
def working_directory(directory):
owd = os.getcwd()
try:
os.chdir(directory)
yield directory
finally:
os.chdir(owd)

As already pointed out by others, all the solutions above only change the working directory of the current process. This is lost when you exit back to the Unix shell. If desperate you can change the parent shell directory on Unix with this horrible hack:
def quote_against_shell_expansion(s):
import pipes
return pipes.quote(s)
def put_text_back_into_terminal_input_buffer(text):
# use of this means that it only works in an interactive session
# (and if the user types while it runs they could insert characters between the characters in 'text'!)
import fcntl, termios
for c in text:
fcntl.ioctl(1, termios.TIOCSTI, c)
def change_parent_process_directory(dest):
# the horror
put_text_back_into_terminal_input_buffer("cd "+quote_against_shell_expansion(dest)+"\n")

os.chdir() is the right way.

os.chdir() is the Pythonic version of cd.

import os
abs_path = 'C://a/b/c'
rel_path = './folder'
os.chdir(abs_path)
os.chdir(rel_path)
You can use both with os.chdir(abs_path) or os.chdir(rel_path), there's no need to call os.getcwd() to use a relative path.

Further into direction pointed out by Brian and based on sh (1.0.8+)
from sh import cd, ls
cd('/tmp')
print ls()

If You would like to perform something like "cd.." option, just type:
os.chdir("..")
it is the same as in Windows cmd: cd..
Of course import os is neccessary (e.g type it as 1st line of your code)

The Path objects in path (a third-party package available on PyPI, different from pathlib) offer both a context manager and a chdir method for this purpose:
from path import Path # pip install path
with Path("somewhere"):
...
Path("somewhere").chdir()

If you use spyder and love GUI, you can simply click on the folder button on the upper right corner of your screen and navigate through folders/directories you want as current directory.
After doing so you can go to the file explorer tab of the window in spyder IDE and you can see all the files/folders present there.
to check your current working directory
go to the console of spyder IDE and simply type
pwd
it will print the same path as you have selected before.

Changing the current directory of the script process is trivial. I think the question is actually how to change the current directory of the command window from which a python script is invoked, which is very difficult. A Bat script in Windows or a Bash script in a Bash shell can do this with an ordinary cd command because the shell itself is the interpreter. In both Windows and Linux Python is a program and no program can directly change its parent's environment. However the combination of a simple shell script with a Python script doing most of the hard stuff can achieve the desired result. For example, to make an extended cd command with traversal history for backward/forward/select revisit, I wrote a relatively complex Python script invoked by a simple bat script. The traversal list is stored in a file, with the target directory on the first line. When the python script returns, the bat script reads the first line of the file and makes it the argument to cd. The complete bat script (minus comments for brevity) is:
if _%1 == _. goto cdDone
if _%1 == _? goto help
if /i _%1 NEQ _-H goto doCd
:help
echo d.bat and dSup.py 2016.03.05. Extended chdir.
echo -C = clear traversal list.
echo -B or nothing = backward (to previous dir).
echo -F or - = forward (to next dir).
echo -R = remove current from list and return to previous.
echo -S = select from list.
echo -H, -h, ? = help.
echo . = make window title current directory.
echo Anything else = target directory.
goto done
:doCd
%~dp0dSup.py %1
for /F %%d in ( %~dp0dSupList ) do (
cd %%d
if errorlevel 1 ( %~dp0dSup.py -R )
goto cdDone
)
:cdDone
title %CD%
:done
The python script, dSup.py is:
import sys, os, msvcrt
def indexNoCase ( slist, s ) :
for idx in range( len( slist )) :
if slist[idx].upper() == s.upper() :
return idx
raise ValueError
# .........main process ...................
if len( sys.argv ) < 2 :
cmd = 1 # No argument defaults to -B, the most common operation
elif sys.argv[1][0] == '-':
if len(sys.argv[1]) == 1 :
cmd = 2 # '-' alone defaults to -F, second most common operation.
else :
cmd = 'CBFRS'.find( sys.argv[1][1:2].upper())
else :
cmd = -1
dir = os.path.abspath( sys.argv[1] ) + '\n'
# cmd is -1 = path, 0 = C, 1 = B, 2 = F, 3 = R, 4 = S
fo = open( os.path.dirname( sys.argv[0] ) + '\\dSupList', mode = 'a+t' )
fo.seek( 0 )
dlist = fo.readlines( -1 )
if len( dlist ) == 0 :
dlist.append( os.getcwd() + '\n' ) # Prime new directory list with current.
if cmd == 1 : # B: move backward, i.e. to previous
target = dlist.pop(0)
dlist.append( target )
elif cmd == 2 : # F: move forward, i.e. to next
target = dlist.pop( len( dlist ) - 1 )
dlist.insert( 0, target )
elif cmd == 3 : # R: remove current from list. This forces cd to previous, a
# desireable side-effect
dlist.pop( 0 )
elif cmd == 4 : # S: select from list
# The current directory (dlist[0]) is included essentially as ESC.
for idx in range( len( dlist )) :
print( '(' + str( idx ) + ')', dlist[ idx ][:-1])
while True :
inp = msvcrt.getche()
if inp.isdigit() :
inp = int( inp )
if inp < len( dlist ) :
print( '' ) # Print the newline we didn't get from getche.
break
print( ' is out of range' )
# Select 0 means the current directory and the list is not changed. Otherwise
# the selected directory is moved to the top of the list. This can be done by
# either rotating the whole list until the selection is at the head or pop it
# and insert it to 0. It isn't obvious which would be better for the user but
# since pop-insert is simpler, it is used.
if inp > 0 :
dlist.insert( 0, dlist.pop( inp ))
elif cmd == -1 : # -1: dir is the requested new directory.
# If it is already in the list then remove it before inserting it at the head.
# This takes care of both the common case of it having been recently visited
# and the less common case of user mistakenly requesting current, in which
# case it is already at the head. Deleting and putting it back is a trivial
# inefficiency.
try:
dlist.pop( indexNoCase( dlist, dir ))
except ValueError :
pass
dlist = dlist[:9] # Control list length by removing older dirs (should be
# no more than one).
dlist.insert( 0, dir )
fo.truncate( 0 )
if cmd != 0 : # C: clear the list
fo.writelines( dlist )
fo.close()
exit(0)