Develop Reference

How can I read the error content outputs of mysql command executed from python

bash_fc = rf"mysql -u {source_user_name} -h {source_ipv4_addr} --password={source_db_password}"
When I use the following functions for the above command,
from subprocess import PIPE,run
def cons_r(cmd):
response = run(cmd, stdout=PIPE, stderr=PIPE, universal_newlines=True, shell=True)
return response.stdout
response = os.popen(bash_fc)
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: Unknown OS character set 'cp857'.
mysql: Switching to the default character set 'utf8mb4'.
ERROR 1045 (28000): Access denied for user 'root'#'pc.mshome.net' (using password: YES)
I can't read the output, is there a method you know of so I can read it?

You can read errors from standard error
import subprocess as sp
ret = sp.run(['ls', '*.bad'], stderr=sp.PIPE,
stdout=sp.PIPE, shell=True, encoding="cp857")
if ret.returncode == 0:
print(ret.stdout)
else:
print(ret.stderr)

This change fixed the problem
from subprocess import PIPE, run, STDOUT
def cons_r(cmd):
response = run(cmd, stdout=PIPE, stderr=STDOUT, universal_newlines=True, shell=True)
return response.stdout

mailutils not working via subprocess.run in python

I am trying to send a mail via python's subprocess.run methode. Unfortunately, it is not working.
import subprocess
message = "Hello World"
process = subprocess.run(["mail", "-s", "Test, "xyz#xyz.com", "<<<", message],
stdout=subprocess.PIPE,
universal_newlines=True)
print (process.stdout)
I received the following Error:
mail: Cannot parse address <<<' (while expanding<<<'): Malformed email address
mail: Cannot parse address Hello World' (while expandingHello World'): Malformed email address
The command is working in the shell though (Linux Mint > 19.0).

The <<< syntax is a feature of bash. If you want to use that you need to run your command as an argument of the bash shell:
import subprocess
message = "Hello World"
command = "mail -s Test abc#def.com <<< "+message
process = subprocess.run(
["bash","-c",command],
stdout=subprocess.PIPE,
universal_newlines=True)
print (process.stdout)
However, using shell expansion on dynamic content can be a security issue. A better way is, to use the input feature of subprocess.run ( python3 only )
import subprocess
message = "Hello World"
command = ["mail", "-s", "Test", "abc#def.com"]
process = subprocess.run(
command,
input=message,
stdout=subprocess.PIPE,
universal_newlines=True)
print (process.stdout)
See also Python - How do I pass a string into subprocess.Popen (using the stdin argument)?

popen returns error when executing python script containing logging

I have 2 python scripts where 1 is using subprocess to execute the other, see below:
main.py
import subprocess
command = ['python', 'logging_test.py']
proc1 = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc1.communicate()
print('Output returned from command: {}'.format(out))
print('Error returned from command: {}'.format(err))
logging_test.py
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger('log')
logger.info('hello')
When running main.py I get this as the output:
Output returned from command:
Error returned from command: INFO:log:hello
I would expect for the log message to be returned by stdout, not stderr... Does anyone know why it is getting returned as an error?

basicConfig, among other things, provides a StreamHandler for the root logger. By default, a new StreamHandler writes to standard error.

Python subprocess calling mysqldump with .cnf file fails when shell=False

What I want to do is call mysqldump to make a backup of a database table.
I have assembled a list of command line arguments, where db and tbl are the database and table names, respectively, and cnf is a properly formatted mysql configuration file (containing host, user and password),
args = ["mysqldump","--defaults-extra-file="+cnf,db,tbl]
print " ".join(args)
The output of the print is,
mysqldump --defaults-extra-file=dbserver.cnf test mytable
When I copy/paste the above line into the (bash) shell, it works.
When I use subprocess.Popen, the command fails,
import subprocess as sp
...
proc = sp.Popen(args, shell=False, stdin=sp.PIPE, stdout=sp.PIPE)
stdout = proc.communicate()
retcode = proc.wait()
if retcode>0: print "error",retcode
But when I join the args together, and call subprocess.Popen with shell=True,
the command works as desired,
cmd = " ".join(args)
proc = sp.Popen(cmd, shell=True, stdin=sp.PIPE, stdout=sp.PIPE)
stdout = proc.communicate()
retcode = proc.wait()
if retcode>0: print "error",retcode
Further investigation reveals that the first command does not seem to use the user/password credentials from the config file, but the second variant does.
Responses to other questions about subprocess.Popen make it clear that one should avoid the shell=True, but the first approach above fails.
Can anyone identify what I am doing wrong above?

Cannot get subprocess output in Django view

I want to emulate command line call of my script (TensorFlow neaural chatbot model) in Django view and get output from console to variable.
When i do manually in terminal of my server:
python3 var/www/engine/chatbot/udc_predict.py --model_dir=var/www/engine/chatbot/runs/1486057482/
the output is good and printed out.
So in my Django view i do:
import subprocess
answer = subprocess.check_output(['python3', 'var/www/engine/chatbot/udc_predict.py','--model_dir=var/www/engine/chatbot/runs/1486057482/'], shell=True, stderr=subprocess.STDOUT, timeout=None)
print('answer', answer)
And answer variable is printed as b'' in Apache error log.
I cannot figure out what's wrong in my call.

The answer is to use .communicate() and PIPE:
from subprocess import Popen, PIPE
proc = Popen(
"python3 var/www/engine/chatbot/udc_predict.py --model_dir=var/www/engine/chatbot/runs/1486057482/",
shell=True,
stdout=PIPE, stderr=PIPE
)
proc.wait()
res = proc.communicate()
if proc.returncode:
print(res[1])
print('result:', res[0])
answer = res[0]