So this is a code snippet from python doc to demonstrate multiprocessing module.
I opened a empty file 'multi_process_test.py' and typed in these code.
from multiprocessing import Process
import sys
def f(name):
print('hello', name)
if __name__ == '__main__':
p = Process(target=f, args=('bob',))
p.start()
p.join()
And the weird part is if I run it under Spyder, the program prints nothing, but if I run the source code in powershell env by 'python multi_process_test.py', the console will prints 'hello bob' as expected, can s.b. help to explain why this happen? Or any hint to resolve this difference.
Appreciated for your help.
Related
If I run this simple code in IDLE in Python 2.7.8, it will pop a window saying "The program is still running! Do you want to kill it?".
from multiprocessing import Pool
def foo(x):
return x**2
if __name__ == '__main__':
pool = Pool(2)
pows = pool.map(foo, range(10))
print pows
Even if I do kill or not (it will ask twice) nothing will happen. I used to use Windows and I've just recently started using Mac OSX (10.9.4), and I don't know if I'm missing something here.
If I run the same code directly in the Python Shell in terminal, it will run fine. Same in iPython notebook. It just won't on IDLE, popping up that message box.
Any ideas? I'd like to keep using IDLE...
here's the log:
INFO:root:10221: Started process
INFO:root:10221: Defined foo
INFO:root:10221: __name__ == '__main__'
INFO:root:10221: pool created
Ref this:
https://docs.python.org/2/library/multiprocessing.html#introduction
Specifically, in the note:
Functionality within this package requires that the __main__ module be
importable by the children. This is covered in Programming guidelines
however it is worth pointing out here. This means that some examples,
such as the multiprocessing.Pool examples will not work in the
interactive interpreter."
Here's a similar question Child processes created with python multiprocessing module won't print
Example of logging activity to a file:
#!/usr/bin/env python
import logging
from multiprocessing import Pool
import os
logging.basicConfig(filename='example.log',level=logging.DEBUG)
def log_msg(msg):
logging.info("{}: {}".format(os.getpid(), msg))
log_msg("Started process")
def foo(x):
log_msg("running foo")
return x**2
log_msg("Defined foo")
if __name__ == '__main__':
log_msg("__name__ == '__main__'")
pool = Pool(2)
log_msg("pool created")
pows = pool.map(foo, range(10))
log_msg("map completed")
print pows
log_msg("output printed")
log_msg("Finished running")
Example output for me:
tom#fannybawz:~$ ./multiproc.py
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
tom#fannybawz:~$ cat example.log
INFO:root:22238: Started process
INFO:root:22238: Defined foo
INFO:root:22238: __name__ == '__main__'
INFO:root:22238: pool created
INFO:root:22240: {}: running foo
INFO:root:22239: {}: running foo
INFO:root:22240: {}: running foo
INFO:root:22239: {}: running foo
INFO:root:22240: {}: running foo
INFO:root:22239: {}: running foo
INFO:root:22240: {}: running foo
INFO:root:22239: {}: running foo
INFO:root:22240: {}: running foo
INFO:root:22240: {}: running foo
INFO:root:22238: map completed
INFO:root:22238: output printed
INFO:root:22238: Finished running
tom#fannybawz:~$
Try the same thing yourself with the Process version.
This was a known issue with the previous version of Pycharm. If you upgrade with the latest version now you can safely use multiprocessing within the console of the IDE without running in this issue any longer.
See here for further informations: https://youtrack.jetbrains.com/issue/PY-14969
I'm using Ubuntu 12.04 Server x64, Python 2.7.3, futures==2.1.5, eventlet==0.14.0
Did anybody hit the same problem?
import eventlet
import futures
import random
# eventlet.monkey_patch() # Uncomment this line and it will hang!
def test():
if random.choice((True, False)):
raise Exception()
print "OK this time"
def done(f):
print "done callback!"
def main():
with futures.ProcessPoolExecutor() as executor:
fu = []
for i in xrange(6):
f = executor.submit(test)
f.add_done_callback(done)
fu.append(f)
futures.wait(fu, return_when=futures.ALL_COMPLETED)
if __name__ == '__main__':
main()
This code, if you uncomment the line, will hang. I can only stop it by pressing Ctrl+C. In this case the following KeyboardInterrupt traceback is printed: https://gist.github.com/max-lobur/8526120#file-traceback
This works fine with ThreadPoolExecutor.
Any feedback is appreciated
I think this KeyboardInterrupt reaches the started process because under ubuntu and most linux systems the new process is created with fork - starting a new process from the current one with the same stdin and stdout and stderr. So the KeyboardInterrupt can reach the child process. Maybe someone who know something more can add thoughts.
We are trying to create a python script to install an app via windows shell prompt, executing our python script. We have an output prompt from the app.exe indicating
"Press Enter to Continue..."
We tried to simulate the Enter key but it doesn't work. The prompt just sits still not moving to the next wizard step.
How do we overcome this problem?
import subprocess
import win32console
APP_BIN = 'app.exe'
def main():
proc = subprocess.Popen([APP_BIN,'-i','console'],stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
proc.stdin.write("\r\n") <--- issue
output = proc.stdout.readline() <--- issue
print output
ret = proc.wait()
print ret
if __name__ == '__main__':
main()
Not entirely sure how to do it in python, but my suggestion would be simulate an actual 'enter' key press command. In your code, you are just changing caret's position and not issuing a proper return.
Have a look at this: http://win32com.goermezer.de/content/view/136/254/
import win32com.client
shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys("{ENTER}", 0)
Seems like that's exactly what you need.
The following may work (untested):
import subprocess
import win32console
APP_BIN = 'app.exe'
def main():
proc = subprocess.Popen([APP_BIN,'-i','console'],stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
stdoutdata, stderrdata = proc.communicate(input="\r\n")
output = stdoutdata.readline()
print output
ret = proc.wait()
print ret
if __name__ == '__main__':
main()
Can I programmatically launch the lamson SMTP server from inside my python code?
it cam be launched from the command line as:
lamson start
but , I want to launch it from code using the multiprocess package
from multiprocessing import Process
def f(name):
# what to put here
if __name__ == '__main__':
p = Process(target=f, args=('bob',))
p.start()
p.join()
what to put in the f function to lauch a lamson server
How this can be achieved?
thanks
The multiprocessing module in Python 3.2.1 on Windows 7 x86 seems to be defeating me.
I have two modules: servmain.py and sslserver.py. The idea is to (eventually) code an application which will communicate with clients using SSL. This is the part I already have down. However, I need the server listener to be spun off into its own separate process so the main process can do some other stuff. As a dummy test, I told the child process to print "Hello World" to stdout and write text to a non-existent text file.
Here is the code for my parent process(servmain.py):
from multiprocessing import Process
import logging
if __name__ == "__main__":
logger = multiprocessing.log_to_stderr()
logger.setLevel(logging.DEBUG)
#Fire up the server
listenerProcess = Process(target = sslserver.startServer)
listenerProcess.start()
logger.debug("Starting listener.")
listenerProcess.join()
logger.debug("Done.")
And here is the sslserver.py code:
def startServer():
print("Hello World")
f= open("testfile.txt", "w")
f.write("Hello world\n")
f.close()
When I run servmain.py, I get the following output:
[DEBUG/MainProcess] Starting listener. [DEBUG/MainProcess] Done.
Which is what I would expect. However, testfile.txt has not been created, and there is no output to stdout. Does anyone have an idea about why this might be happening?
I think I am missing a few libraries here so I had to remove your logger code because it was a problem for me. To me it would seem that you have a naming/pathing conflict. Please be sure the name 'sslserver' does not collide with any modules in python path.
Also, set python path! In my example both of these files are in the same directory.
pytest.py
#!/usr/bin/env python
from multiprocessing import Process
import sslserver
import logging
if __name__ == "__main__":
#logger = multiprocessing.log_to_stderr()
#logger.setLevel(logging.DEBUG)
#Fire up the server
listenerProcess = Process(target = sslserver.startServer)
listenerProcess.start()
#logger.debug("Starting listener.")
print "Starting Listener\n"
listenerProcess.join()
#logger.debug("Done.")
print "Done\n";
sslserver.py
#!/usr/bin/env python
def startServer():
print("Hello World")
f= open("testfile.txt", "w")
f.write("Hello world\n")
f.close()
Output
nynex#citadel:~/temp$ ./pytest.py
Starting Listener
Hello World
Done
nynex#citadel:~/temp$ cat testfile.txt
Hello world