Develop Reference

High CPU load of executable with Python thread

I use the following python-code to execute a C-program (executable):
proc = Popen(cmd, stdout=PIPE, bufsize=10)
thread = Thread(target=streamstdout, args=(proc.stdout, conn))
thread.daemon = True
thread.start()
while 1:
time.sleep(1)
conn.settimeout(0.01)
try:
data = conn.recv(1024)
except socket.timeout:
continue
except socket.error:
print 'Connection break during execution.'
if proc.poll() == None:
proc.terminate()
print 'Execution terminated'
break
data = data.strip()
if data: print 'Received: ', data
The conn is a TCP-connection.
Everything is right when I start the python script in my console with
python script.py
The CPU-load of the executable is around 30% as expected.
But if I start the python-script at boot-time in crontab or I start the script with
nohup python script.py &
and suppress with this call the output-messages of the python-script, the executable runs always with the maximum CPU-load (98%-100%).
I think the problem is the output of the python-script which can not outputted on the console...but why happens this?
I'm grateful for every idea or opinion.
Update: I appended the while-loop. But I don't think this is relevant for the problem.

Python script does not restart after it crashes

I have a python script that is doing some stuff called "python1.py". Sometimes because connection issue, it crashes. I have another script called "loop.py" that is supposed to monitor the first one when it crashes and restart it. So far, it fails to restart. Meaning, when an exception is risen ( IOError or WatsonException ( I am using a Watson API ) ) the script stops
python1.py is something like this :
def mainfunction ():
a = randrange(0, 1)
Print (' my routine is doing something')
if a = 1 :
Print ('a = 1 ')
else :
Print (' a is not equals to 1')
mainfunction ()
The other script that is supposed to restart the first one is something like this :
def loopApp():
while True :
try:
python1.mainfunction ()
except IOError :
print (' IOError y')
except WatsonException :
print (' Exception from watson API')
loopApp ()
python1.py should restart when every time the exceptions happens, but it is not.

I found the way using python subprocess. It works fine and I could get expections working.
def loopApp():
loop = 1
while loop == 1:
print ("wa_loop is starting")
try:
process = subprocess.call(['python', 'wa_dir_watch.py'])
except 'IOError':
print ("\nretrying after IOError")
continue
except KeyboardInterrupt:
print ("\nstopped by user Ctr+C")
quit()
loopApp()

Python subprocess not returning

I want to call a Python script from Jenkins and have it build my app, FTP it to the target, and run it.
I am trying to build and the subprocess command fails. I have tried this with both subprocess.call() and subprocess.popen(), with the same result.
When I evaluate shellCommand and run it from the command line, the build succeeds.
Note that I have 3 shell commands: 1) remove work directory, 2) create a fresh, empty, work directory, then 3) build. The first two commands return from subprocess, but the third hangs (although it completes when run from the command line).
What am I doing wrongly? Or, what alternatives do I have for executing that command?
# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
def ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand):
try:
process = subprocess.call(shellCommand, shell=True, stdout=subprocess.PIPE)
#process.wait()
return process #.returncode
except KeyboardInterrupt, e: # Ctrl-C
raise e
except SystemExit, e: # sys.exit()
raise e
except Exception, e:
print 'Exception while executing shell command : ' + shellCommand
print str(e)
traceback.print_exc()
os._exit(1)
# +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
def BuildApplciation(arguments):
# See http://gnuarmeclipse.github.io/advanced/headless-builds/
jenkinsWorkspaceDirectory = arguments.eclipseworkspace + '/jenkins'
shellCommand = 'rm -r ' + jenkinsWorkspaceDirectory
ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand)
shellCommand = 'mkdir ' + jenkinsWorkspaceDirectory
if not ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand) == 0:
print "Error making Jenkins work directory in Eclipse workspace : " + jenkinsWorkspaceDirectory
return False
application = 'org.eclipse.cdt.managedbuilder.core.headlessbuild'
shellCommand = 'eclipse -nosplash -application ' + application + ' -import ' + arguments.buildRoot + '/../Project/ -build myAppApp/TargetRelease -cleanBuild myAppApp/TargetRelease -data ' + jenkinsWorkspaceDirectory + ' -D DO_APPTEST'
if not ExcecuteShellCommandAndGetReturnCode(arguments, shellCommand) == 0:
print "Error in build"
return False

I Googled further and found this page, which, at 1.2 says
One way of gaining access to the output of the executed command would
be to use PIPE in the arguments stdout or stderr, but the child
process will block if it generates enough output to a pipe to fill up
the OS pipe buffer as the pipes are not being read from.
Sure enough, when I deleted the , stdout=subprocess.PIPE from the code above, it worked as expected.
As I only want the exit code from the subprocess, the above code is enough for me. Read the linked page if you want the output of the command.

How to run a shell script without having to press enter/confirm s.th. inbetween

I'm currently writing a shell script which is interfacing with numerous python scripts. In one of these Python scripts I'm calling grass without starting it explicitly. When I run my shell script I have to hit enter at the point where I call grass (this is the code I got from the official working with grass page):
startcmd = grass7bin + ' -c ' + file_in2 + ' -e ' + location_path
print startcmd
p = subprocess.Popen(startcmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode != 0:
print >>sys.stderr, 'ERROR: %s' % err
print >>sys.stderr, 'ERROR: Cannot generate location (%s)' % startcmd
sys.exit(-1)
else:
print 'Created location %s' % location_path
gsetup.init(gisbase, gisdb, location, mapset)
My problem is that I want this process to run automatically without me having to press enter everytime in between!
I have already tried numerous options such as pexpect, uinput (doesn't work that well because of problems with the module). I know that in windows you have the msvcrt module, but I am working with linux... any ideas how to solve this problem?

Use the pexpect library for expect functionnality.
Here's an example of interaction with a an application requiring user to type in his password:
child = pexpect.spawn('your command')
child.expect('Enter password:')
child.sendline('your password')
child.expect(pexpect.EOF, timeout=None)
cmd_show_data = child.before
cmd_output = cmd_show_data.split('\r\n')
for data in cmd_output:
print data

I finally found an easy and fast way for simulating a key press:
just install xdotool and then use the following code for simulating e.g. the enter key:
import subprocess
subprocess.call(["xdotool","key","Return"])

Python script to automate in linux based environment

I tried running the myview command and it ran successfully, but I am stuck after this step.
I have to choose from a list of views and have to pass in a number, for example say <1>,<2>..<10>. But when I execute the script it shows me option on the terminal window instead.
Which command should I be using? Because after this I have to run a bunch of other commands as well and basically have to execute them in a particular order. So say cmd should wait for cmd to finish. Thanks in advance for the help.
This is what I have so far.
#! /usr/bin/python
import sys
from subprocess import call
for arg in sys.argv:
print arg
call(["myview"])

Check out the doc for subprocess. I think the API call you need is check_call.

From pydoc subprocess:
try:
retcode = call("mycmd" + " myarg", shell=True)
if retcode < 0:
print >>sys.stderr, "Child was terminated by signal", -retcode
else:
print >>sys.stderr, "Child returned", retcode
except OSError, e:
print >>sys.stderr, "Execution failed:", e