Python - When parsing the following command-line arguments...
sys.argv = ('%s %s build_ext --inplace' % ('python', sys.argv[0]))
...for Cython inside my buildscript "myscript_pyd_setup.py" for "myscript.pyd" that uses the "myscript.py" file I get the following error (e1):
python error 1: invalid command 'y'
The python command-line option python -- help revealed no corresponding option 'y'. This makes sense otherwise the error would not exist. Searching SO, Python docs and the web resulted in unrelated articles about Tkinter. As I do absolutely nothing with Tkinter, as far as I know it, I'm wondering if this error is from the python command-line interpreter at all?
My effort:
What I tried to accomplish is parsing arguments sys.argv.append('build_ext --inplace') to python command-line interpreter when running the "buildscript" from within the editor (Komodo edit 11.x) but it returns also with an error (e2) similar to the version from command-line or another editor.
python error 2: invalid command name 'build_ext --inplace'
Running "myscript_pyd_setup.py" with sys.argv.append('build_ext') builds the required *.pyd file just fine. What I don't understand is why its choking on --infile. This works for py2exe and pyinstaller.
The objective:
To run the setup-script of "myscript.py" from within the editor and not having to flip back-and-forth to the command-line editor for compiling *.py > *.pyd when I changed code inside "myscript.py" and want to see the result quickly.
Note: parsing command-line python myscript_pyd_setup.py build_ext --inplace works fine when sys.argv... is commented-out in the "buildscript"!
A third option was to use cythonize in combo with the "myscript.py" and "myscript.pyd" files but that showed a copyfile error for "myscript.pyd". Relevant but not for the above asked "error = y" question.
Any thoughts and help on how to automate this part to prevent RSI are more than welcome! Thx.
My "myscript.py" example code:
import sys, time, os
#...snippet...
def print_me():
text = "bar(man), yes, Hello, how do you do Mr. foo?"
return text
if __name__ == '__main__':
#...snippet...
print_me()
The myscript_pyd_setup.py:
# myscript.py
try:
from setuptools import setup
from setuptools import Extension
except ImportError:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
#from Cython.Build import cythonize
import sys
print 'sys.argv[0] : %s' % sys.argv[0]
#sys.argv = ('%s %s build_ext --inplace' % ('python', sys.argv[0]))
sys.argv.append('build_ext --inplace')
ext_modules = [Extension("myscript",['myscript.py'])]
#ext_modules = cythonize("myscript.py")
#setup: "name" and "cmdclass" are commented-out when using cythonize.
setup(
name= 'XYZ model class',
cmdclass = {'build_ext': build_ext},
include_dirs = [],
ext_modules = ext_modules)
sys.argv is a list of arguments. When you append 'build_ext --inplace', you literally append that as a single argument (as if you've passed it enclosed in quotes from the shell command line). That is what happened in the second case.
In the first one, you've assigned string back to sys.argv. But string is also a sequence, so your commands ran as if called with (argv[1:]): ['y', 't', `h`, ...]
In either case you ended up with an option/sub-command unknown the the argument parser.
Related
I, like many python programmers find the command line arguments for python described here very useful. specifically "-i" which keeps the python interpreter running after a program finishes for a stack trace. How can I use these with an exe file compiled with py2exe? Not to be confused with regular argument parsing in an exe file. If you came looking for that, find it here.
My first thought was to try:
pyprogram.exe -i -foo -bar
but that didn't work.
it should be noted that
pyprogram.exe -foo -bar
does in fact work for me.
what I am looking for is the .exe equivalent of
python -i pyprogram.py foo bar
Failing to find an implementation that works for all of the python command line options, what could I do just to make the "-i" argument work? it is the most important to have as an option in my executable.
I did not find anything on the py2exe wiki about passing arguments like -i (to enter interactive mode after execution).
One might be able to discover information about the argument handling in the py2exe source files.
Update: It indeed looks like py2exe does not handle any command line options like the normal interpreter does, instead it just passes them to the script. But it does handle the respective environment variable, which can be used as shown below.
However, as a workaround, you could try to set the PYTHONINSPECT Environment variable:
If this is set to a non-empty string it is equivalent to specifying the -i option.
E.g. run set PYTHONINSPECT=TRUE before running the program.
But, probably even better, this can be done from within the Python script:
This variable can also be modified by Python code using os.environ to force inspect mode on program termination.
Here's a little test script for os.environ (also os.putenv):
import os
one = os.environ
os.putenv("PYTHONINSPECT", "TRUE")
two = os.environ
os.environ["PYTHONINSPECT"] = "TRUE"
three = os.environ
print(one)
print(two)
print(three)
print( set(one.items()) ^ set(two.items()) )
print( set(one.items()) ^ set(three.items()) )
The behaviour is a little weird: there does not seem to be a difference, and it seems to only last until you exit the interactive mode:
G:\>py test.py > test.txt
>>> exit()
G:\>set PYTHONINSPECT
Environment variable PYTHONINSPECT not defined
The contents of test.txt are:
environ({'ALLUSERSPROFILE': 'C:\\ProgramData', ... 'PYTHONINSPECT': 'TRUE'})
environ({'ALLUSERSPROFILE': 'C:\\ProgramData', ... 'PYTHONINSPECT': 'TRUE'})
environ({'ALLUSERSPROFILE': 'C:\\ProgramData', ... 'PYTHONINSPECT': 'TRUE'})
set()
set()
But it seems to work either way (double check the documentation for yourself to ensure you are not corrupting your environment variables), so you could even implement an -i argument for yourself like:
import sys, os
if len(sys.argv) > 1 and sys.argv[1] == '-i':
os.putenv("PYTHONINSPECT", "TRUE")
#os.environ["PYTHONINSPECT"] = "TRUE"
print("interactive")
else:
print("normal")
which runs as follows
G:\>py test.py
normal
G:\>py test.py -i
interactive
>>> quit()
G:\>set PYTHONINSPECT
Environment variable PYTHONINSPECT not defined
Trying with py2exe and Python 3.4.3 (newer versions are apparently not supported and you get an IndexError):
setup.py:
from distutils.core import setup
import py2exe
setup(console=['test.py'])
Get py2exe
G:\>c:\Python34\Scripts\pip.exe install py2exe
You are using pip version 6.0.8, however version 10.0.0b2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting py2exe
Using cached py2exe-0.9.2.2-py33.py34-none-any.whl
Installing collected packages: py2exe
Successfully installed py2exe-0.9.2.2
Run py2exe
G:\>c:\Python34\python.exe setup.py py2exe
running py2exe
3 missing Modules
------------------
? readline imported from cmd, code, pdb
? win32api imported from platform
? win32con imported from platform
Building 'dist\test.exe'.
Building shared code archive 'dist\library.zip'.
Copy c:\windows\system32\python34.dll to dist
Copy c:\Python34\DLLs\_hashlib.pyd to dist\_hashlib.pyd
Copy c:\Python34\DLLs\pyexpat.pyd to dist\pyexpat.pyd
Copy c:\Python34\DLLs\select.pyd to dist\select.pyd
Copy c:\Python34\DLLs\unicodedata.pyd to dist\unicodedata.pyd
Copy c:\Python34\DLLs\_ctypes.pyd to dist\_ctypes.pyd
Copy c:\Python34\DLLs\_socket.pyd to dist\_socket.pyd
Copy c:\Python34\DLLs\_lzma.pyd to dist\_lzma.pyd
Copy c:\Python34\DLLs\_ssl.pyd to dist\_ssl.pyd
Copy c:\Python34\DLLs\_bz2.pyd to dist\_bz2.pyd
Test
G:\>dist\test.exe
normal
G:\>dist\test.exe -i
interactive
>>> sys.exit()
Does not seem to have changed the environment variables permanently:
G:\>set PYTHONINSPECT
Environment variable PYTHONINSPECT not defined
Also works with single exe:
from distutils.core import setup
import py2exe
setup(
options = {'py2exe': {'bundle_files': 1, 'compressed': True}},
console = [{'script': "test.py"}],
zipfile = None,
)
Referring to A Simple Example from python.org:
This is setup.py:
from distutils.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'],
)
And then it says to run this command:
python setup.py sdist
How/what/where is the parameter sdist being parsed?
The setup function parses the command line arguments: it reads sys.argv. See source code on GitHub.
You're certainly confusing between function arguments (the ones defined in the setup.py script) and the command line arguments: (which are setup.py sdist).
A piece of information is given in the run_setup function comment:
'sys.argv[0]' will be replaced with 'script' for the duration of the call. 'script_args' is a list of strings; if supplied,
'sys.argv[1:]' will be replaced by 'script_args' for the duration of the call.
I have a Python code that uses two command line arguments. I am using linux terminal for all command line tasks. Now I am trying to use Cython to speed up my Python code.
For that I have compiled the Python code to C using build_ext module by creating this setup.py file:
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup (
cmdclass = {'build_ext': build_ext } ,
ext_modules = [
Extension ("myCode", ["myCode.py"]) ,
])
And then compiling my Python code into C using:
python setup.py build_ext -i
The following were generated:
[file]myCode.c
[file]myCode.so
[folder]build
--[folder]temp.linux-x86_64-2.7
----[file]myCode.o
I want to run the generated file with command line arguments.
Till now in Python I was using the usual command
>> python myCode.py arg1 arg2
I am very new to Cython, infact I started using it to address the inherent speed issue of Python after code level algorithm optimization. I need inputs on which files to run, and how to run the converted C code and with command line arguments. Thanks in advance.
As mentioned you compiled a Python module. So to call from Linux you have to write a .py script that imports your compiled module and does any calculations needed. Then you can run it with your typical Linux command.
I'm trying to cythonize some python modules (on windows) and want the resulting .c files stored in a different directory.
I tried something like this:
from Cython.Build import cythonize
module_list = cythonize(r'C:\myproject\module.py',
force=True,
build_dir=r'C:\myproject\compiled_modules')
The compilation is done but the file module.c is created in C:\myproject and not in C:\myproject\compiled_modules as expected. What am I doing wrong?
I'm not sure the cythonize function can be coerced into doing that. I would just use the command line:
/path/to/cython yourmod.py -o compiled_modules/yourmod.c
Edit:
This command can be called from a python script using the subprocess module:
import subprocess
subprocess.check_call(["/path/to/cython", "yourmod.py", "-o", "compiled_modules/yourmod.c"])
Edit 2:
According to this Sage developer, relocating the c/cpp files is not supported by cython, so it looks like either compiling from the command line or using a normal configuration and then moving the c/cpp files are your only options.
Following file as helloworld.pyx:
print("Hello World")
Following file as setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("helloworld",["helloworld.pyx"]
setup(
name = 'HW',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
After i use python setup.py build_ext --inplace i got my *.so file
So, i rename the *.so to hw.so, for getting a shorter import name.
But if i sart python and type in: import hw i got this error:
ImportError: dynamic module does not define init function (PyInit_hw)
I was doing the exact thing bout 3 hours ago and all was ok. But i tryed something out from this side: http://sourceforge.net/p/ubertooth/mailman/message/31699880/
I tryed the following:
cmake -DPYTHON_EXECUTABLE=$(which python2) \
-DPYTHON_INCLUDE_DIR=$(echo /usr/include/python2*) \
-DPYTHON_LIBRARY=$(echo /usr/lib/libpython2.*.so) \
because i wanted to fix something. I replaced all "2" with "3" cause i am working with python3.4
After i made this i always got the error above. Did i destroy any path? How can i undo it? Thank u for your help
When looking at the Python 3 documentation on "Extending Python with C or C++" we see that
The initialization function must be named PyInit_name(), where name is the name of the module, and should be the only non-static item defined in the module file.
(https://docs.python.org/3/extending/extending.html#the-module-s-method-table-and-initialization-function)
This means that we cannot just change the file name of the module without changing the init function. We have to compile the module with the final name in the first place. Renaming the .so file after compiling it will not work.