File with non-english fileName fails to upload on SFTP? - python

I am using the following script to upload to SFTP. It works fine except if the filenames have non-english chars.
import pexpect
fileName = raw_input("Enter fileName")
#in the pexpect, send the following
put_string = "put {} /test/{}".format(fileName)
p.sendline(put_string)
My observation
Actual filename = 法嫁
fileName = \xe6\xb3\x95\xe5\xab\x81
put string = put \xe6\xb3\x95\xe5\xab\x81 /test/\xe6\xb3\x95\xe5\xab\x81
How can I resolve this issue ? Does SFTP support this type of PUT ? If no, how to upload non-eng chars file.
Versions
python2.7
Note - I have tried storing this as unicode, but same issue happens.

Related

how to do docx to pdf conversion using python library without subprocess in linux? [duplicate]

I'am tasked with converting tons of .doc files to .pdf. And the only way my supervisor wants me to do this is through MSWord 2010. I know I should be able to automate this with python COM automation. Only problem is I dont know how and where to start. I tried searching for some tutorials but was not able to find any (May be I might have, but I don't know what I'm looking for).
Right now I'm reading through this. Dont know how useful this is going to be.
A simple example using comtypes, converting a single file, input and output filenames given as commandline arguments:
import sys
import os
import comtypes.client
wdFormatPDF = 17
in_file = os.path.abspath(sys.argv[1])
out_file = os.path.abspath(sys.argv[2])
word = comtypes.client.CreateObject('Word.Application')
doc = word.Documents.Open(in_file)
doc.SaveAs(out_file, FileFormat=wdFormatPDF)
doc.Close()
word.Quit()
You could also use pywin32, which would be the same except for:
import win32com.client
and then:
word = win32com.client.Dispatch('Word.Application')
You can use the docx2pdf python package to bulk convert docx to pdf. It can be used as both a CLI and a python library. It requires Microsoft Office to be installed and uses COM on Windows and AppleScript (JXA) on macOS.
from docx2pdf import convert
convert("input.docx")
convert("input.docx", "output.pdf")
convert("my_docx_folder/")
pip install docx2pdf
docx2pdf input.docx output.pdf
Disclaimer: I wrote the docx2pdf package. https://github.com/AlJohri/docx2pdf
I have tested many solutions but no one of them works efficiently on Linux distribution.
I recommend this solution :
import sys
import subprocess
import re
def convert_to(folder, source, timeout=None):
args = [libreoffice_exec(), '--headless', '--convert-to', 'pdf', '--outdir', folder, source]
process = subprocess.run(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=timeout)
filename = re.search('-> (.*?) using filter', process.stdout.decode())
return filename.group(1)
def libreoffice_exec():
# TODO: Provide support for more platforms
if sys.platform == 'darwin':
return '/Applications/LibreOffice.app/Contents/MacOS/soffice'
return 'libreoffice'
and you call your function:
result = convert_to('TEMP Directory', 'Your File', timeout=15)
All resources:
https://michalzalecki.com/converting-docx-to-pdf-using-python/
I have worked on this problem for half a day, so I think I should share some of my experience on this matter. Steven's answer is right, but it will fail on my computer. There are two key points to fix it here:
(1). The first time when I created the 'Word.Application' object, I should make it (the word app) visible before open any documents. (Actually, even I myself cannot explain why this works. If I do not do this on my computer, the program will crash when I try to open a document in the invisible model, then the 'Word.Application' object will be deleted by OS. )
(2). After doing (1), the program will work well sometimes but may fail often. The crash error "COMError: (-2147418111, 'Call was rejected by callee.', (None, None, None, 0, None))" means that the COM Server may not be able to response so quickly. So I add a delay before I tried to open a document.
After doing these two steps, the program will work perfectly with no failure anymore. The demo code is as below. If you have encountered the same problems, try to follow these two steps. Hope it helps.
import os
import comtypes.client
import time
wdFormatPDF = 17
# absolute path is needed
# be careful about the slash '\', use '\\' or '/' or raw string r"..."
in_file=r'absolute path of input docx file 1'
out_file=r'absolute path of output pdf file 1'
in_file2=r'absolute path of input docx file 2'
out_file2=r'absolute path of outputpdf file 2'
# print out filenames
print in_file
print out_file
print in_file2
print out_file2
# create COM object
word = comtypes.client.CreateObject('Word.Application')
# key point 1: make word visible before open a new document
word.Visible = True
# key point 2: wait for the COM Server to prepare well.
time.sleep(3)
# convert docx file 1 to pdf file 1
doc=word.Documents.Open(in_file) # open docx file 1
doc.SaveAs(out_file, FileFormat=wdFormatPDF) # conversion
doc.Close() # close docx file 1
word.Visible = False
# convert docx file 2 to pdf file 2
doc = word.Documents.Open(in_file2) # open docx file 2
doc.SaveAs(out_file2, FileFormat=wdFormatPDF) # conversion
doc.Close() # close docx file 2
word.Quit() # close Word Application
unoconv (writen in Python) and OpenOffice running as a headless daemon.
https://github.com/unoconv/unoconv
http://dag.wiee.rs/home-made/unoconv/
Works very nicely for doc, docx, ppt, pptx, xls, xlsx.
Very useful if you need to convert docs or save/convert to certain formats on a server.
As an alternative to the SaveAs function, you could also use ExportAsFixedFormat which gives you access to the PDF options dialog you would normally see in Word. With this you can specify bookmarks and other document properties.
doc.ExportAsFixedFormat(OutputFileName=pdf_file,
ExportFormat=17, #17 = PDF output, 18=XPS output
OpenAfterExport=False,
OptimizeFor=0, #0=Print (higher res), 1=Screen (lower res)
CreateBookmarks=1, #0=No bookmarks, 1=Heading bookmarks only, 2=bookmarks match word bookmarks
DocStructureTags=True
);
The full list of function arguments is: 'OutputFileName', 'ExportFormat', 'OpenAfterExport', 'OptimizeFor', 'Range', 'From', 'To', 'Item', 'IncludeDocProps', 'KeepIRM', 'CreateBookmarks', 'DocStructureTags', 'BitmapMissingFonts', 'UseISO19005_1', 'FixedFormatExtClassPtr'
It's worth noting that Stevens answer works, but make sure if using a for loop to export multiple files to place the ClientObject or Dispatch statements before the loop - it only needs to be created once - see my problem: Python win32com.client.Dispatch looping through Word documents and export to PDF; fails when next loop occurs
If you don't mind using PowerShell have a look at this Hey, Scripting Guy! article. The code presented could be adopted to use the wdFormatPDF enumeration value of WdSaveFormat (see here).
This blog article presents a different implementation of the same idea.
I have modified it for ppt support as well. My solution support all the below-specified extensions.
word_extensions = [".doc", ".odt", ".rtf", ".docx", ".dotm", ".docm"]
ppt_extensions = [".ppt", ".pptx"]
My Solution: Github Link
I have modified code from Docx2PDF
I tried the accepted answer but wasn't particularly keen on the bloated PDFs Word was producing which was usually an order of magnitude bigger than expected. After looking how to disable the dialogs when using a virtual PDF printer I came across Bullzip PDF Printer and I've been rather impressed with its features. It's now replaced the other virtual printers I used previously. You'll find a "free community edition" on their download page.
The COM API can be found here and a list of the usable settings can be found here. The settings are written to a "runonce" file which is used for one print job only and then removed automatically. When printing multiple PDFs we need to make sure one print job completes before starting another to ensure the settings are used correctly for each file.
import os, re, time, datetime, win32com.client
def print_to_Bullzip(file):
util = win32com.client.Dispatch("Bullzip.PDFUtil")
settings = win32com.client.Dispatch("Bullzip.PDFSettings")
settings.PrinterName = util.DefaultPrinterName # make sure we're controlling the right PDF printer
outputFile = re.sub("\.[^.]+$", ".pdf", file)
statusFile = re.sub("\.[^.]+$", ".status", file)
settings.SetValue("Output", outputFile)
settings.SetValue("ConfirmOverwrite", "no")
settings.SetValue("ShowSaveAS", "never")
settings.SetValue("ShowSettings", "never")
settings.SetValue("ShowPDF", "no")
settings.SetValue("ShowProgress", "no")
settings.SetValue("ShowProgressFinished", "no") # disable balloon tip
settings.SetValue("StatusFile", statusFile) # created after print job
settings.WriteSettings(True) # write settings to the runonce.ini
util.PrintFile(file, util.DefaultPrinterName) # send to Bullzip virtual printer
# wait until print job completes before continuing
# otherwise settings for the next job may not be used
timestamp = datetime.datetime.now()
while( (datetime.datetime.now() - timestamp).seconds < 10):
if os.path.exists(statusFile) and os.path.isfile(statusFile):
error = util.ReadIniString(statusFile, "Status", "Errors", '')
if error != "0":
raise IOError("PDF was created with errors")
os.remove(statusFile)
return
time.sleep(0.1)
raise IOError("PDF creation timed out")
I was working with this solution but I needed to search all .docx, .dotm, .docm, .odt, .doc or .rtf and then turn them all to .pdf (python 3.7.5). Hope it works...
import os
import win32com.client
wdFormatPDF = 17
for root, dirs, files in os.walk(r'your directory here'):
for f in files:
if f.endswith(".doc") or f.endswith(".odt") or f.endswith(".rtf"):
try:
print(f)
in_file=os.path.join(root,f)
word = win32com.client.Dispatch('Word.Application')
word.Visible = False
doc = word.Documents.Open(in_file)
doc.SaveAs(os.path.join(root,f[:-4]), FileFormat=wdFormatPDF)
doc.Close()
word.Quit()
word.Visible = True
print ('done')
os.remove(os.path.join(root,f))
pass
except:
print('could not open')
# os.remove(os.path.join(root,f))
elif f.endswith(".docx") or f.endswith(".dotm") or f.endswith(".docm"):
try:
print(f)
in_file=os.path.join(root,f)
word = win32com.client.Dispatch('Word.Application')
word.Visible = False
doc = word.Documents.Open(in_file)
doc.SaveAs(os.path.join(root,f[:-5]), FileFormat=wdFormatPDF)
doc.Close()
word.Quit()
word.Visible = True
print ('done')
os.remove(os.path.join(root,f))
pass
except:
print('could not open')
# os.remove(os.path.join(root,f))
else:
pass
The try and except was for those documents I couldn't read and won't exit the code until the last document.
You should start from investigating so called virtual PDF print drivers.
As soon as you will find one you should be able to write batch file that prints your DOC files into PDF files. You probably can do this in Python too (setup printer driver output and issue document/print command in MSWord, later can be done using command line AFAIR).
import docx2txt
from win32com import client
import os
files_from_folder = r"c:\\doc"
directory = os.fsencode(files_from_folder)
amount = 1
word = client.DispatchEx("Word.Application")
word.Visible = True
for file in os.listdir(directory):
filename = os.fsdecode(file)
print(filename)
if filename.endswith('docx'):
text = docx2txt.process(os.path.join(files_from_folder, filename))
print(f'{filename} transfered ({amount})')
amount += 1
new_filename = filename.split('.')[0] + '.txt'
try:
with open(os.path.join(files_from_folder + r'\txt_files', new_filename), 'w', encoding='utf-8') as t:
t.write(text)
except:
os.mkdir(files_from_folder + r'\txt_files')
with open(os.path.join(files_from_folder + r'\txt_files', new_filename), 'w', encoding='utf-8') as t:
t.write(text)
elif filename.endswith('doc'):
doc = word.Documents.Open(os.path.join(files_from_folder, filename))
text = doc.Range().Text
doc.Close()
print(f'{filename} transfered ({amount})')
amount += 1
new_filename = filename.split('.')[0] + '.txt'
try:
with open(os.path.join(files_from_folder + r'\txt_files', new_filename), 'w', encoding='utf-8') as t:
t.write(text)
except:
os.mkdir(files_from_folder + r'\txt_files')
with open(os.path.join(files_from_folder + r'\txt_files', new_filename), 'w', encoding='utf-8') as t:
t.write(text)
word.Quit()
The Source Code, see here:
https://neculaifantanaru.com/en/python-full-code-how-to-convert-doc-and-docx-files-to-pdf-from-the-folder.html
I would suggest ignoring your supervisor and use OpenOffice which has a Python api. OpenOffice has built in support for Python and someone created a library specific for this purpose (PyODConverter).
If he isn't happy with the output, tell him it could take you weeks to do it with word.

Python - transfer files in the same network using shutil

I'm moving a .txt file between two servers (Ubuntu and Windows) in the same network.
The following code doesn't show any errors but it doesn't work:
def transfer_files_task():
source_path = r"/root/airflow/testdoc"
dest_path = f"192.168.xxx.xx\Doc-Share\Logger Output"
filename = r"/test.txt"
filenamew = f"\test.txt"
shutil.copyfile(source_path + filename, dest_path + filenamew)
Change your function to this:
import os, ntpath, posixpath
def transfer_files_task():
source_file = posixpath.join("/", "root", "airflow", "testdoc", "test.txt")
dest_file = ntpath.join("192.168.xxx.xx", "Doc-Share", "Logger Output", "test.txt")
assert os.path.exists(source_file), f"{source_file} does not exists"
shutil.copyfile(source_file, dest_file)
A small explanation: leave python format your paths, it will save you from many errors. If you do not do that, you have to know how strings work, what characters should be escaped, how to format a path on linux and windows, etc.
Also, a side note about the use of r and f prefix for strings:
r stands for raw and it roughly means that you don't have to escape special characters like the backspace. Consequently, r"\tab" == "\\tab" and print(r"\tab") gives \tab while print("\tab") gives ab
f stands for format and is the new way of formatting strings in py36+. It is used as follows:
name="john"
print(f"hello {name}")
# hello john
Finally, you might want to check this post: Cannot copy file from a remote machine using shutil

Doing a download with easywebdav results in bad files

I have a small program I created to download image files form a FlashAir webdav enabled SD Card. This works fine with Powershell but trying a Python implementation.
I have it working where easywebdav can download the file but the resulting size is 22Kb which is wrong and windows does not recognize them as a valid file.
I have applied a basestring fix to the client.py for handling Python 3.x instead of 2.x found here How to overcome Python 3.4 NameError: name 'basestring' is not defined.
I'm on Python 3.5.1.
import easywebdav
urlFlashAir = 'http://flashair'
imgLocation = '/DCIM/101MSDCF'
destLocation = 'c:\\test3\\'
remoteDirName = ''
webDavConnection = easywebdav.connect('flashair', '', '')
lsOutput = webDavConnection.ls(imgLocation)
for i in lsOutput:
if i.size == 0:
remoteDirName = i.name
if i.size > 0:
filename = (i.name).replace(remoteDirName,'')
print(i.name)
print(remoteDirName)
print(filename)
webDavConnection.download(imgLocation, destLocation + filename)
I inspected the 22KB article with notepad++ and found that it was a web page for the flashair card. Upon investigation of the code and documentation I found I was not referencing the remote filename just the location. Once I fixed the code to append a forward slash and a filename. Now properly downloads the file.

Downloading a file with a wildcard from Sharepoint URL with Python

I'm not sure if this can be done or not but I thought I'd ask!
I'm running a windows 10 PC using Python 2.7. I'm wanting to download a file form sharepoint to a folder on my C: drive.
OpenPath = "https://office.test.com/sites/Rollers/New improved files/"
OpenFile = "ABC UK.xlsb"
The downside is the file is uploaded externally & due to human error it can be saved as a .xlsx or ABC_UK. Therefor I only want to use the first 3 characters with a wildcard (ABC*) to open that file. Thankfully the first 3 Characters are unique & there only be one file in the path that should match.
to find the file in your dir:
import os, requests, fnmatch
#loop over dir
for filename in os.listdir(OpenPath):
#find the file
if fnmatch.fnmatch(filename, 'ABC_UK*'):
#download the file
# open file handler
with open('C:\dwnlfile.xls', 'wb') as fh:
#try to get it
result = requests.get(OpenPath+filename)
#check u got it
if not result.ok:
print result.reason # or text
exit(1)
#save it
fh.write(result.content)
print 'got it and saved'

How can I decrypt a PDF using PyPDF2?

Currently I am using the PyPDF2 as a dependency.
I have encountered some encrypted files and handled
them as you normally would (in the following code):
from PyPDF2 import PdfReader
reader = PdfReader(pdf_filepath)
if reader.is_encrypted:
reader.decrypt("")
print(len(reader.pages))
My filepath looks something like "~/blah/FDJKL492019 21490 ,LFS.pdf"
PDF.decrypt("") returns 1, which means it was successful. But when it hits print PDF.getNumPages(),
it still raises the error, "PyPDF2.utils.PdfReadError: File has not been decrypted".
How do I get rid of this error?
I can open the PDF file just fine by double click (which default-opens with Adobe Reader).
This error may come about due to 128-bit AES encryption on the pdf, see Query - is there a way to bypass security restrictions on a pdf?
One workaround is to decrypt all isEncrypted pdfs with "qpdf"
qpdf --password='' --decrypt input.pdf output.pdf
Even if your PDF does not appear password protected, it may still be encrypted with no password. The above snippet assumes this is the case.
The following code could solve this problem:
import os
from PyPDF2 import PdfReader
filename = "example.pdf"
reader = PdfReader(filename)
if reader.is_encrypted:
try:
reader.decrypt("")
print("File Decrypted (PyPDF2)")
except:
command = (
"cp "
+ filename
+ " temp.pdf; qpdf --password='' --decrypt temp.pdf "
+ filename
+ "; rm temp.pdf"
)
os.system(command)
print("File Decrypted (qpdf)")
reader = PdfReader(filename)
else:
print("File Not Encrypted")
To Answer My Own Question:
If you have ANY spaces in your file name, then PyPDF 2 decrypt function will ultimately fail despite returning a success code.
Try to stick to underscores when naming your PDFs before you run them through PyPDF2.
For example,
Rather than "FDJKL492019 21490 ,LFS.pdf" do something like "FDJKL492019_21490_,LFS.pdf".
It has nothing to do with whether the file has been decrypted or not when using the method getNumPages().
If we take a look at the source code of getNumPages():
def getNumPages(self):
"""
Calculates the number of pages in this PDF file.
:return: number of pages
:rtype: int
:raises PdfReadError: if file is encrypted and restrictions prevent
this action.
"""
# Flattened pages will not work on an Encrypted PDF;
# the PDF file's page count is used in this case. Otherwise,
# the original method (flattened page count) is used.
if self.isEncrypted:
try:
self._override_encryption = True
self.decrypt('')
return self.trailer["/Root"]["/Pages"]["/Count"]
except:
raise utils.PdfReadError("File has not been decrypted")
finally:
self._override_encryption = False
else:
if self.flattenedPages == None:
self._flatten()
return len(self.flattenedPages)
we will notice that it is the self.isEncrypted property controlling the flow. And as we all know the isEncrypted property is read-only and not changeable even when the pdf is decrypted.
So, the easy way to handle the situation is just add the password as key-word argument with empty string as default value and pass your password when using the getNumPages() method and any other method build beyond it
Implement qpdf using python with pikepdf library.
import pikepdf
pdf = pikepdf.open('unextractable.pdf')
pdf.save('extractable.pdf')
You can try PyMuPDF package, it can open encrypted files and solved my problems.
Reference: PyMuPDF Documentation

Categories

Resources