This is my code :
from subprocess import Popen, PIPE
cmd = ['antiword', 'tbhocbong151.doc']
p = Popen(cmd, stdout=PIPE)
stdout, stderr = p.communicate()
print(stdout.decode('utf-8', 'ignore'))
I have content in file word like this : "Chào bạn"
but when I generated output is: "Ch?o b?n"
How can I fix it to output like input ?
Thanks for your help
I believe that the problem is that the locale is not properly set when antiword is running. Try this:
import os
from subprocess import Popen, PIPE
myenv = dict(os.environ)
if 'LC_ALL' in myenv:
del myenv['LC_ALL']
myenv['LANG'] = 'en_US.UTF-8'
cmd = ['antiword', 'tbhocbong151.doc']
p = Popen(cmd, stdout=PIPE, env=myenv)
stdout, stderr = p.communicate()
print(stdout.decode('utf-8', 'ignore'))
If that doesn't work, try setting the LANG env variable in your shell before running your python program; e.g. by doing:
export LANG=en_US.UTF-8
I am trying to run the following openssl command in python:
cmd = "openssl x509 -sha1 -in esx.crt -noout -fingerprint"
tmp = os.popen(cmd)
tmp_sha1 = tmp.readline()
This command is supposed to generate a fingerprint of the certificate. I am trying to capture the output through the file object. But when I read this file object, there's nothing in it. I have executed this command on the command line and it runs fine, generates the fingerprint. Could you tell me how can I get the fingerprint?
You achieve this natively within Python using the OpenSSL module.
from OpenSSL.crypto import load_certificate, FILETYPE_PEM
cert_file_string = open("esx.crt", "rb").read()
cert = load_certificate(FILETYPE_PEM, cert_file_string)
sha1_fingerprint = cert.digest("sha1")
print sha1_fingerprint
You can use two modules to establish what you want: subprocess and os.
Using subprocess you can check for the output for the process using communicate(), which reads data from stdout and stderr until EOF.
>>> import subprocess
>>> p = subprocess.Popen("openssl x509 -sha1 -in 17.cert -noout -fingerprint", stdout=subprocess.PIPE)
>>> out, _ = p.communicate() #return a tuple (stdout, stderr)
>>> out
b'SHA1 Fingerprint=87:68:8B:B0:6A:E2:DF:A3:E2:63:76:97:A9:2B:B4:F4:82:4E:0B:D1\n'
Using os module works fine as well, using both read() and readline() methods: (please note that os.popen() is deprecated)
>>> import os
>>> p = os.popen("openssl x509 -sha1 -in 17.cert -noout -fingerprint")
>>> p.read()
'SHA1 Fingerprint=87:68:8B:B0:6A:E2:DF:A3:E2:63:76:97:A9:2B:B4:F4:82:4E:0B:D1\n'
>>> p = os.popen("openssl x509 -sha1 -in 17.cert -noout -fingerprint")
>>> out = p.readline()
'SHA1 Fingerprint=87:68:8B:B0:6A:E2:DF:A3:E2:63:76:97:A9:2B:B4:F4:82:4E:0B:D1\n'
If you want to write the value to a file, that works as well, which you can verify by opening a file output.txt in your current working directory:
>>> with open('./output.txt', 'w') as f:
... f.write(out)
...
77
The 77 notifies us that 77 bytes were written to the file, which you can verify by opening it in your favourite text editor.
How do I encrypt file with subprocess, so the output will be string.
password = '%030x' % random.randrange(16**30)
encrypted_file = subprocess.geststatusoutput("echo "+password+
"|gpg --password-fd 0 < "+ name_of_selected_file)
I want to encrypted _file to be a string so I could use it as encrypted file to upload with post request.
What is the best way to do it with gnupg library?
You can use subprocess.Popen() to execute a gpg command like this:
import shlex
import random
from subprocess import Popen, PIPE
passphrase = '%030x' % random.randrange(16**30)
source_filename = '/tmp/somefile'
cmd = 'gpg --batch --symmetric --cipher-algo AES256 --passphrase-fd 0 --output - {}'.format(source_filename)
# error handling omitted
p = Popen(shlex.split(cmd), stdout=PIPE, stdin=PIPE, stderr=PIPE)
encrypted_data = p.communicate(passphrase.encode())[0]
# Decryption - should work for Python 2 & 3
import os
r, w = os.pipe() # pipe for sending passphrase from parent to child
try:
os.set_inheritable(r, True)
except AttributeError: # new in version 3.4
pass
cmd = 'gpg --batch --decrypt --passphrase-fd {}'.format(r)
p = Popen(shlex.split(cmd), stdout=PIPE, stdin=PIPE, stderr=PIPE, close_fds=False)
os.close(r) # closes fd in parent, child needs it
f = os.fdopen(w, 'w')
f.write(passphrase + '\n') # '\n' seems required for Python 2 ???
f.close()
decrypted_data, stderr = p.communicate(encrypted_data)
# check that data was successfully roundtripped
assert open(source_filename).read() == decrypted_data.decode()
Or, decryption for Python 3 only:
import os
r, w = os.pipe()
cmd = 'gpg --batch --decrypt --passphrase-fd {}'.format(r)
p = Popen(shlex.split(cmd), stdout=PIPE, stdin=PIPE, stderr=PIPE, pass_fds=(r,))
os.close(r) # closes fd in parent, child needs it
open(w, 'w').write(passphrase)
decrypted_data, stderr = p.communicate(encrypted_data)
# check that data was successfully roundtripped
assert open(source_filename).read() == decrypted_data.decode()
Now I am not an expert on the security of this method, however, writing the passphrase directly to the child process' stdin with communicate() is better than echoing the passphrase on the command line - which would be visible to anybody that could run ps or equivalent.
Also, relying on the shell level IO redirection of your command would require the shell=True argument to Popen which might have other security implications (see the warning in the Popen() documentation).
I have assumed in my gpg command that you are intending to use a symmetric encryption (your example does not suggest otherwise). If the remote server needs to be able to decrypt the encrypted file content, how are you going to share the generated passphrase?
You might be better off looking at using public key crypto instead.
I'm trying to do symmetric encryption with python and gnupg.
This code snippet works on my windows vista machine, on which the python gnupg module is version 0.3.2:
import gnupg
gpg = gnupg.GPG()
data = 'the quick brown fow jumps over the laxy dog.'
passphrase='12345'
crypt = gpg.encrypt(data, recipients=None,
symmetric='AES256',
passphrase=passphrase,
armor=False)
When I try to run it on my linux machine with the version 1.2.5 python gnupg module I get this error:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/gnupg/gnupg.py", line 961, in encrypt
result = self._encrypt(stream, recipients, **kwargs)
TypeError: _encrypt() got multiple values for keyword argument 'recipients'
I have done a number of searches and can't find anything on this.
This is an old ask, but I came upon this in a Google search and was unhappy with the provided answers. I found the real answer in python-gnupg's GitHub issues:
gpg.encrypt(data, symmetric='AES256', passphrase=passphrase, armor=False, encrypt=False)
So, drop the recipients=None and add an encrypt=False. Your crypt.data will then contain the encrypted data. Unintuitive, but it works.
(src: https://github.com/isislovecruft/python-gnupg/issues/110)
I think it's just broken (correct me if I'm wrong). It appears to be fixed in gnupg 0.3.6-1 for Python 3.x. You're probably better off using that or trying a workaround, such as one similar to what I'm about to describe.
This workaround uses the command-line gpg in Python instead of the gnupg module. You could do a similar workaround with gpg.encrypt_file, of course (which wouldn't have the temporarily stored passphrase I talk about in the next paragraph; that would most likely be a better choice).
If you're concerned about the passphrase showing up in the task manager, I took care of that by putting the password in a temporary file and using cat to pipe it into gpg. However, you might have to worry about malware snatching the password from the temporary file, if it can do it in time.
Note that this code is in Python 3.x, since that's what I use the most (to recreate it in 2.x you'll have to learn how temporary files work in it, and maybe a few other things).
import subprocess, tempfile, os, shutil
def gpg_encrypt(data, passphrase, alg="AES256", hash="SHA512", compress_alg="BZIP2", compress_lvl="9", iterations="1000000"):
#This is for symmetric encryption.
with tempfile.TemporaryDirectory() as directory:
filepath=os.path.join(directory, "tmp")
with open(filepath, "w") as FILE:
FILE.write(data)
iterations=str(iterations)
compress_level="--compress-level "+compress_lvl
if compress_alg.upper()=="BZIP2":
compress_level="--bzip2-compress-level "+compress_lvl
tmp_filename="tmp"
with tempfile.TemporaryDirectory() as DIR:
tmpfilepath=os.path.join(DIR, tmp_filename)
with open(tmpfilepath, "w") as FILE:
FILE.write(passphrase)
subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 --force-mdc --s2k-mode 3 --s2k-count "+iterations+" --s2k-cipher-algo "+alg+" --s2k-digest-algo "+hash+" --compress-algo='"+compress_alg+"' --compress-level "+compress_lvl+" -ac '"+filepath+"'", stdin=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=True).communicate()[0]
result=None
with open(filepath+".asc", "r") as FILE:
result=FILE.read()
return result
def gpg_decrypt(data, passphrase):
#This is for decryption.
with tempfile.TemporaryDirectory() as directory:
filepath=os.path.join(directory, "tmp")
with open(filepath, "w") as FILE:
FILE.write(data)
decrypted=None
tmp_filename="tmp"
with tempfile.TemporaryDirectory() as DIR:
tmpfilepath=os.path.join(DIR, tmp_filename)
with open(tmpfilepath, "w") as FILE:
FILE.write(passphrase)
decrypted=subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 --output='"+filepath+".gpg"+"' '"+filepath+"'", stdin=subprocess.PIPE, shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE).communicate()[0]
result=None
decrypted=not "decryption failed:" in str(decrypted)
if decrypted==True:
with open(filepath+".gpg", "r") as FILE:
result=FILE.read()
return decrypted, result #If it worked, return True. If not, return False. (And, return the decrypted data.)
test=gpg_encrypt(data="This is a test!", passphrase="enter")
print("Here is the encrypted message:\n"+test)
decrypted, data=gpg_decrypt(data=test, passphrase="enter")
if decrypted:
print("Here is the decrypted message:\n"+data)
else:
print("Incorrect passphrase to decrypt the message.")
Here's the code for encrypting/decrypting symmetrically encrypted files (just for good measure):
import subprocess, tempfile, os, shutil
def gpg_encrypt_file(filepath, passphrase, output=None, alg="AES256", hash="SHA512", compress_alg="BZIP2", compress_lvl="9", iterations="1000000"):
#This is for symmetric encryption.
filepath=filepath.replace("'", "'\\''") #This makes it so you can have apostrophes within single quotes.
iterations=str(iterations)
compress_level="--compress-level "+compress_lvl
if compress_alg.upper()=="BZIP2":
compress_level="--bzip2-compress-level "+compress_lvl
result=None
tmp_filename="tmp"
with tempfile.TemporaryDirectory() as DIR:
tmpfilepath=os.path.join(DIR, tmp_filename)
with open(tmpfilepath, "w") as FILE:
FILE.write(passphrase)
if output:
if output[0]!=os.sep and filepath[0]==os.sep:
output=os.path.join(os.path.dirname(filepath), output)
result=subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 --force-mdc --s2k-mode 3 --s2k-count "+iterations+" --s2k-cipher-algo "+alg+" --s2k-digest-algo "+hash+" --compress-algo='"+compress_alg+"' "+compress_level+" --output='"+output+"' -ac '"+filepath+"'", stdin=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=True).communicate()[0]
else:
result=subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 --force-mdc --s2k-mode 3 --s2k-count "+iterations+" --s2k-cipher-algo "+alg+" --s2k-digest-algo "+hash+" --compress-algo='"+compress_alg+"' --compress-level "+compress_lvl+" -ac '"+filepath+"'", stdin=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=True).communicate()[0]
return result.strip()
def gpg_decrypt_file(filepath, passphrase, output=None):
#This is for decryption.
filepath=filepath.replace("'", "'\\''")
result=None
tmp_filename="tmp"
with tempfile.TemporaryDirectory() as DIR:
tmpfilepath=os.path.join(DIR, tmp_filename)
with open(tmpfilepath, "w") as FILE:
FILE.write(passphrase)
if output:
if output[0]!=os.sep and filepath[0]==os.sep:
output=os.path.join(os.path.dirname(filepath), output)
result=subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 --output='"+output+"' '"+filepath+"'", stdin=subprocess.PIPE, shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE).communicate()[0]
else:
result=subprocess.Popen("cat "+tmpfilepath+"|gpg --batch --yes --passphrase-fd 0 '"+filepath+"'", stdin=subprocess.PIPE, shell=True, stderr=subprocess.STDOUT, stdout=subprocess.PIPE).communicate()[0]
return not "decryption failed:" in str(result) #If it worked, return True. If not, return False.
gpg_encrypt_file(filepath="test.txt", passphrase="myPassphrase", output="test.asc")
if gpg_decrypt_file(filepath="test.asc", passphrase="myPassphrase", output="output.txt"):
print("Successfully decrypted!")
else:
print("Incorrect passphrase.")
The "standard" python-gnupg module's (instead of pretty-bad-protocol's module) symmetric-encrypt() call makes much more sense to me:
encrypted_data = gpg.encrypt(data=msg, symmetric=True, passphrase='1234', recipients=None)
Details: https://stackoverflow.com/a/72355824/605356
Double-check the documentation, the argument isn't recipient, is is recipients (note the plural)
Docs (emphasis mine):
symmetric (defaults to False) If specified, symmetric encryption is
used. In this case, specify recipients as None. If True is specified,
then the default cipher algorithm (CAST5) is used. Starting with
version 0.3.5, you can also specify the cipher-algorithm to use (for
example, 'AES256'). Check your gpg command line help to see what
symmetric cipher algorithms are supported. Note that the default
(CAST5) may not be the best available.
This should work with python-gnupg 0.3.5 and above (adjust as necessary):
import gnupg
gpg_home = "~/.gnupg"
gpg = gnupg.GPG(gnupghome=gpg_home)
data = raw_input("Enter full path of file to encrypt: ")
phrase = raw_input("Enter the passphrase to decrypt the file: ")
cipher = raw_input("Enter symmetric encryption algorithm to use: ")
savefile = data+".asc"
afile = open(data, "rb")
encrypted_ascii_data = gpg.encrypt_file(afile, None, passphrase=phrase, symmetric=cipher.upper(), output=savefile)
afile.close()
Not sure about versions prior to 0.3.5.
The "None" for recipients does not (with current versions of the module) require "recipients=None" when the gpg.encrypt_file() is called.
gpg = gnupg.GPG()
data = 'the quick brown fow jumps over the laxy dog.'
passphrase='12345'
crypt = gpg.encrypt(
data,
recipients=None,
symmetric='AES256',
passphrase=passphrase,
armor=False,
)
print(crypt.data)
clear = gpg.decrypt(
crypt.data,
passphrase=passphrase,
)
print(clear)
As of July 28, 2022, The above works just fine for me.
Python 3.9.2 --
python-gnupg 0.4.9
I've been playing around with Python's subprocess module and I wanted to do an "interactive session" with bash from python. I want to be able to read bash output/write commands from Python just like I do on a terminal emulator. I guess a code example explains it better:
>>> proc = subprocess.Popen(['/bin/bash'])
>>> proc.communicate()
('user#machine:~/','')
>>> proc.communicate('ls\n')
('file1 file2 file3','')
(obviously, it doesn't work that way.) Is something like this possible, and how?
Thanks a lot
Try with this example:
import subprocess
proc = subprocess.Popen(['/bin/bash'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
stdout = proc.communicate('ls -lash')
print stdout
You have to read more about stdin, stdout and stderr. This looks like good lecture: http://www.doughellmann.com/PyMOTW/subprocess/
EDIT:
Another example:
>>> process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> process.stdin.write('echo it works!\n')
>>> process.stdout.readline()
'it works!\n'
>>> process.stdin.write('date\n')
>>> process.stdout.readline()
'wto, 13 mar 2012, 17:25:35 CET\n'
>>>
Use this example in my other answer: https://stackoverflow.com/a/43012138/3555925
You can get more details in that answer.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen
command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()
# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())
# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()
# use os.setsid() make it run in a new process group, or bash job control will not be enabled
p = Popen(command,
preexec_fn=os.setsid,
stdin=slave_fd,
stdout=slave_fd,
stderr=slave_fd,
universal_newlines=True)
while p.poll() is None:
r, w, e = select.select([sys.stdin, master_fd], [], [])
if sys.stdin in r:
d = os.read(sys.stdin.fileno(), 10240)
os.write(master_fd, d)
elif master_fd in r:
o = os.read(master_fd, 10240)
if o:
os.write(sys.stdout.fileno(), o)
# restore tty settings back
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)
This should be what you want
import subprocess
import threading
p = subprocess.Popen(["bash"], stderr=subprocess.PIPE,shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
exit = False
def read_stdout():
while not exit:
msg = p.stdout.readline()
print("stdout: ", msg.decode())
def read_stderro():
while not exit:
msg = p.stderr.readline()
print("stderr: ", msg.decode())
threading.Thread(target=read_stdout).start()
threading.Thread(target=read_stderro).start()
while not exit:
res = input(">")
p.stdin.write((res + '\n').encode())
p.stdin.flush()
Test result:
>ls
>stdout: 1.py
stdout: 2.py
ssss
>stderr: bash: line 2: ssss: command not found
An interactive bash process expects to be interacting with a tty. To create a pseudo-terminal, use os.openpty(). This will return a slave_fd file descriptor that you can use to open files for stdin, stdout, and stderr. You can then write to and read from master_fd to interact with your process. Note that if you're doing even mildly complex interaction, you'll also want to use the select module to make sure that you don't deadlock.
I wrote a module to facilitate the interaction between *nix shell and python.
def execute(cmd):
if not _DEBUG_MODE:
## Use bash; the default is sh
print 'Output of command ' + cmd + ' :'
subprocess.call(cmd, shell=True, executable='/bin/bash')
print ''
else:
print 'The command is ' + cmd
print ''
Check out the whole stuff at github: https://github.com/jerryzhujian9/ez.py/blob/master/ez/easyshell.py