Develop Reference

Python 2.7 to 3.4 Subprocess Issue

I'm calling another program with a piece of code that looks like this:
import subprocess
lc="/package/bin/program --do stuff"
command_list = lc.split()
ljs=subprocess.Popen(command_list,stdout=subprocess.PIPE)
ljs.communicate()[0]
The string works fine at the UNIX command line and the code works in Python 2.7. But, in Python 3.4, I get an error like this:
File "/package/bin/program", line 2, in <module>
from package import module
ImportError: No module named package
"/package/bin/program" is calling a dependency from another file in the package here, which I think is the core issue. I have calls to other programs that are working fine in 3.4.
What's changed in 3.4 that might be causing this?
(Sorry in advance for the cryptic code - I'm calling company internal tools that I can't expose here).

The problem is that the working directory of the subproccess instance is default the directory of bash shell. To set a new working directory, set the cwd argument in your Popen to your working directory.
Here's an example:
subprocess.Popen(['random' '--command'], stdout = subprocess.PIPE, cwd='C:/Path/To/Working/Directory/')

Comments above have been helpful in exploring the issue, but at the end of the day this seems to be some permissions conflict - adding a sudo -u <user> before the command fixes the issue. Still not clear why Py3 requires this and Py2 doesn't, but perhaps I need to explore the issue more closely with other internal users.
Thanks!

Problems when showing Interpreter (IPython) after running Program in PyCharm

I have PyCharm Professional Edition 3.5 5.0 configured to
use IPython when possible
and in my Run/Debug Configurations I set
show interpreter afterwards
I use the interactive Interpreter a lot and I really like IPython, but there are some things that I don't like about the way this is handled in PyCharm:
any input() in my programs return empty strings.
Additionally, when an error occurs I can't interact with the Program anymore. (you can when you run a Python program with the -i flag)
There is a lot of space between the last line in the Console and the current line
In IPython the ...: prompt in a code block isn't indented 2 spaces and therefore not aligned to the In [?]: prompt.
When an error occurs I get something like this:
Traceback (most recent call last):
File "C:\Program Files (x86)\PyCharm\helpers\pydev\pydev_run_in_console.py", line 69, in <module>
globals = run_file(file, None, None)
File "C:\Program Files (x86)\PyCharm\helpers\pydev\pydev_run_in_console.py", line 29, in run_file
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files (x86)\PyCharm\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/ca/Python/Bundeswettbewerb Informatik/Aufgabe2/Ameisen.py", line 133, in <module>
function_that_caused_error()
I don't need/want to see the traceback from the internals.
When running a file IPython needs to be started which takes some seconds even if I'm not going use the interpreted afterwards. I would like PyCharm to start IPython after the program has ended or when I start debugging (you can start IPython in an interactive console by doing import IPython; IPython.start_ipython()
There are some other minor things that I don't like:
When IPython is started it prints a lot of text to the console. I don't want to see any of it except maybe the version number (you can usually do this with the --no-banner option, but adding it to the interpreter options doesn't work)
when you type something and press Up it replaces what I have written with the last item of my history instead of replacing it with the last item of my history that start with what I have typed. Plain IPython does this.
I would like to have automatic code completion without having to press Ctrl + Space in the console
The "problems" are ordered by importance. Does anybody know how to change some of them? I could stop using IPython which would solve the second , the third and the fourth problem, but the other ones would still persist. All of this behavior (excluding the IPython stuff) is implemented very well in PyScripter.
EDIT:
I have found solutions to the first two problems and the problem with the IPython banner. The source for the PyDev interactive interpreter (which is used by PyCharm) is located, on Windows, in C:\Program Files (x86)\PyCharm\helpers\PyDev (path my vary of course).
So the first problem can be solved by editing the file _pydev_imps/_pydev_execfile.py. Wrap line 18 (exec(compile(contents+"\n", file, 'exec'), glob, loc)) in a try ... except block with the following code as the exception handler import traceback; traceback.print_exc(). This will terminate your Python program if there is an error while letting you interact with the variable afterwards.
Problem 2 can be solved by editing the fire pydev_run_in_console.py. Add this import at the beginning of the file: from pydev_console_utils import StdIn and insert sys.stdin = StdIn(interpreter, host, client_port) after what was line 61 before adding the import.
In order to solve the problem with the banner you have to download the most recent version of the PyDev source here and replace the files pydev_ipython_console and pydev_ipython_console_011 by their newer versions. In the newer version of the first file the __init__ method in line 22 has a argument called show_banner with the default value True. Change this to False.

This is probably not the answer you are searching for, but based on my experience using IPython on InteliJ products ( PyCharm, Ultimate), I don't recommend using their version of IPython. It's full of bugs, outdated and you'll lose precious time fixing problems instead of coding.
Have you tried jupyter notebook? If you installed python with anaconda, it's already installed. To run it, open the terminal and type:
jupyter notebook
If your browser doesn't open automagically, head to http://localhost:8888
Note:
You can automate this process by creating bat or sh script containing the code above inside your project directory, this way it doesn't start on you home dir, which it does by default.
Resources:
How to Install and Use IPython
nbopen project: Open a Jupyter notebook in the best available server

In all honesty, have you simply tried running a newer version of PyCharm and importing your settings? 3.5 is fairly outdated (I'm on 4.5.3 myself, newest version is 5.0) and I believe more support is offered for IPython in the newer versions. https://www.jetbrains.com/pycharm/help/ipython.html. Especially if you are a student, it might not hurt to give it a shot. I know older versions of PyCharm were more buggy than recent releases.

How to use websockify with python on windows or compiling websockify as windows executable (for more multiple client connection at a time)

I'm trying to get the websockify 0.6.0 running on windows but without any luck,
I have tried python websockify.py 1501 10.0.0.141:1501 but its not good, getting errors , like that:
Traceback (most recent call last):
File "websockify.py", line 1, in <module>
run
NameError: name 'run' is not defined
I also tried Compiling Websockify as Windows Executable , but this also didn't work
I use the following command run.exe 1501 10.0.0.141:1501 and it looks promising at the beginning, outputting the following to the console:
WARNING: no 'resource' module, daemonizing is disabled
WebSocket server settings:
- Listen on :1501
- Flash security policy server
- No SSL/TLS support (no cert file)
- proxying from :1501 to 10.0.0.141:1501
but then after trying to connect using the following from the browser ws://localhost:1501
**it outputs the following error
Traceback (most recent call last):
File "run", line 5, in <module>
File "websockify\websocketproxy.pyc", line 419, in websockify_init
File "websockify\websocket.pyc", line 1018, in start_server
UnboundLocalError: local variable 'exc' referenced before assignment
Any idea on how to use the websockify on windows / or how to use the compiled websockify as windows executable ?

The easiest way to get websockify working on Windows is to use the Node.js version of websockify (in the other/js directory). It works perfectly out of the box, with no shenanigans required.

To address this, use the modified following commands for your example source, start from the beginning of each step and see if it helps:
Firstly, install Portable Python 2.7
You then need to modify the setup.py (It looks like this is why you are getting your first error, as you may not have defined run):
from setuptools import setup, find_packages
# add (line 2):
import py2exe
setup(name=name,
# add (line 10):
console=['run'],
Ensure the above has executed correctly by inspecting setup.py and ensure it includes run.
In your local code, import the resources module to allow you to monitor, measure and control system resources utilized by your program
import resource
Inspect your local variable exc and ensure you have assigned a value to it before calling it (I'm guessing you may have attributed a system variable to it, but python was unable to do so as you did not have resources imported, and as such it was not assigned). If you like, put up an example of your code in the comment to this response and I'll take a closer look at this part.
Back to your source guide, navigate to the websockify folder in command prompt, then execute the following to compile websockify:
[Your path to Portable Python install]\App\python.exe setup.py py2exe --includes numpy
You will now see in the websockify directory a new dir 'dist' which contains the compiled exe. An example provided is:
run.exe 5901 localhost:5900
There is also a guide here to run websockify as a Windows Service if this suits (again mentioned in your source).
----Further edit for more detail----
Open up the two files that seem to be giving you issues (websockify\websocketproxy.pyc and websockify\websocket.pyc and ensure that any reference to a variable called "exc" is referenced after it has been assigned a value (small chance of an issue if you have not yet modified these files.
I believe that your code is relying upon making and monitoring changes to the system resources (such as ports etc) and you are not allowing your code to have these permissions, so it needs the resources module. If you are calling run.exe from a program (what I called your local code) then you need to import resources at the top. If you are just straight up calling the run.exe program from a command line, then try making this new program and see if this helps. If not, send me the contents of your websockify folder and run.exe and I will take a look
# Program Name: goAndRun.py
# Code:
import sys, string, os, arcgisscripting, resource
os.chdir( 'C:\\[Path_to_the_dir_containing_run.exe]' )
os.system( '"C:\\[Path_to_the_dir_containing_run.exe]\run.exe, 5901 localhost:5900"' )
And then use the command:
python goAndRun.py
Not being in your environment, I cannot tell if this will execute exactly as I have written it. The last line may also be:
os.system( '"C:\\[Path_to_the_dir_containing_run.exe]\run.exe"', '5901 localhost:5900' )

Python ImportError with Sage

Okay I am newer to python and have been researching this problem but I can't find anything like it so I am not sure what is going on.
I am creating a program that involves sage and it has a message cue. We have this set up on a development machine, so I know it works but I was wanting to set it up on my own computer so I could get a better understanding of how it all works and make it easier to develop for myself.
To start up sage, we run a script that calls sages main binary file and passes it an executable .py file. (./sage/sage ./sage_server.py) This creates an error in the sage_server.py file:
Traceback (most recent call last):
File "./sage_server.py", line 23, in <module>
from carrot.messaging import Publisher
ImportError: No module named carrot.messaging
But whenever I run that file just in the terminal (./sage_server) the import works fine and isn't until line 27 that there is an error when it tries to import something from sage.
Does anyone know what would cause the error when it is being called by something else? I am very lost as to what would be causing this.

Sage has its own python, separate from the system libraries. This "carrot" module, whatever it is, must be installed in whatever python ./sage_server.py uses, but not in Sage.
You should be able to use either
[your-sage] -sh
to start up a Sage shell and use easy_install, or you could get whatever carroty package you're using, find its setup.py file, and then run
[your-sage] -python setup.py install
where obviously your-sage is the path to your sage.
Things get a little trickier if the install process isn't setup.py-based.

Is the default configuration of re incorrect on macbooks? Or have I simply misunderstood something?

Python came pre-installed on my macbook and I have been slowly getting acquainted with the langauge. However, it seems that my configuration of the re library is incorrect, or I simply misunderstand something and things are amiss. Whenever I run a python script with "import re", I recieve the following error:
Traceback (most recent call last):
File "regex.py", line 2, in <module>
import re
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/re.py", line 4, in <module>
# re-compatible interface for the sre matching engine
AttributeError: 'module' object has no attribute 'compile'
What gives!

Pretty mysterious problem, given that line 4 in that file (and many other lines around that line number) is a comment (indeed the error msg itself shows that comment line!-) so even with the worst misconfiguration I'd be hard put to reproduce the problem as given.
Let's try to simplify things and check how they may (or may not) break. Please open a Terminal, mkdir a new empty directory somewhere and cd into it (so we know there's no filename conflict wrt modules etc), at the bash prompt unset PYTHONPATH (so we know for sure that isn't interfering), unset PYTHONSTARTUP (ditto); then type the command:
$ python -c'import re; print re.__file__'
It should emit the line:
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/re.pyc
does it? If so, then we can keep rooting around to understand what name clash (or whatever) caused your original problem. If the problem persists under such "clean" conditions then your system is jinxed and I would reinstal Mac OS X Leopard if I were in your shoes!