Develop Reference

Implement similar module of pathlib in python 2

I have created a function where it will dynamically create a json file.
Currently i am using pathlib module which can be found in python 3 . However the server that i am deploying my script will only have python 2.7 and below.
The current approach im doing now work perfectly fine with pathlib module. When this was triggered in a higher environment, error im receiving is pathlib module not found. Hence i would go without using pathlib module when locating the base path to inject json content to publish_workbook.json
import json
import os
from os import listdir
from os.path import isfile, join
from pathlib import Path
def createJson():
print("Creating Json File ....... ")
..
..
event_dict = json.loads(_json)
_jsonFile = json.dumps(event_dict, indent = 4, sort_keys=True)
#Convert path to python 2.7 library orusing os.path
base = Path('/home/reporting/job/')
_saveFile = base / ('publish_workbook' + '.json')
print("Saving json file on : " + str(_saveFile))
_saveFile.write_text(_jsonFile)
createJson()
What is the best approach i can go with this ?

Install pathlib2 on the server, which is a backported version of pathlib. It's no longer supported but then neither is Python 2.7.
You'll then only have to change the import line to
from pathlib2 import Path

Just do it as strings. It's actually fewer characters to type. Pathlib has some cool concepts, but I'm not convinced it's a net win. And, of course, for Python 2 you don't have the print function.
base = '/home/reporting/job/'
_saveFile = base + 'publish_workbook.json'
print "Saving json file on : " + _saveFile
_saveFile.write_text(_jsonFile)

Python glob: how input directories and single files

This is an easy question.
I'm using glob to print the full hierarchy of a folder using this code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import argparse
import sys
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-input', dest='input',help="input one or more files",metavar=None)
args = parser.parse_args()
def dirlist(path):
for i in glob.glob(os.path.join(path, "*")):
if os.path.isfile(i):
print (i)
elif os.path.isdir(i):
dirname = os.path.basename(i)
dirlist(i)
path = os.path.normpath(args.input)
dirlist(path)
It works pretty well, you just need to run python3 Test.py -input ..
As you can see by the argparse help description I would like to input directories but also single files.
I don't think this can be done using glob, is that right?
Can you suggest me a library that could help me print the full hierarchy of both directories and files?
I found here a long list of globe examples but they all seems to work for directories, not when you input a single file

Is nearly midnight but at least I found the solution after 2 days:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import argparse
import sys
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-input', dest='input',help="input one or more files",metavar=None)
args = parser.parse_args()
def dirlist(path):
if os.path.isfile(path):
print (path)
elif os.path.isdir(path):
for i in glob.glob(os.path.join(path, "*")):
if os.path.isfile(i):
print (i)
elif os.path.isdir(i):
dirname = os.path.basename(i)
dirlist(i)
path = os.path.normpath(args.input)
dirlist(path)

Using Python subprocess.call with crontab?

I'm running into a wall with regards to using subprocess.call in a python script running in a crontab. I have isolated this problem to be subprocess not able to find the 7z executable. I'm running this on FreeBSD 10.1, but that should not make a difference. I have tried adding PYTHONPATH=$PATH to crontab, I have tried adding shell=True to subprocess.call, and I have tried using /usr/loca/bin/7z rather than 7z. None of these have fixed the problem. The error that I get is the following:
/usr/local/bin/7z: realpath: not found
/usr/local/bin/7z: dirname: not found
exec: /../libexec/p7zip/7z: not found
Here is how I'm calling the script in crontab:
PATH=$PATH:/usr/local/bin
#every_minute $HOME/test.py >> $HOME/test.error 2>&1
Here is the contents of my script (test.py):
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import os
import subprocess
import tempfile
thing = 'blahblahblah'
errors = open('/home/myuser/error', 'wb')
with tempfile.TemporaryDirectory() as tmpdirname:
tempthing = os.path.join(tmpdirname, thing)
fh = open(tempthing, 'wb')
fh.write(b'123')
fh.close()
zipname = '{}.zip'.format(thing)
ziptempfile = os.path.join(tmpdirname, zipname)
zipper = subprocess.call(['7z', 'a', '-p{}'.format('something'), '-tzip', '-y', ziptempfile, tempthing], stdout=errors, stderr=subprocess.STDOUT)

The answer is that the PATH variable in crontab must use an absolute path like so:
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
That fixes everything.

How to read japanese filename file correctly by python in windows

import os
path="."
dirList=os.listdir(path)
for fileName in dirList:
print fileName
if the filename is japanese, print to console will be not correct(like ?????.csv,????abc.csv)
open('XXX.csv').readlines()
if the filename is japanese,
IOError:No such file or directory: \xe4\xb8\xbcABC.csv

All problems done,thanks
1)if you want to get fileNames which is not English(such as Japanese,Chinese) by os.listdir correctly(not ???.csv)
you can add u before your path string
listdir doesn't print non-english letters correctly
2)if you want to open a file,you can use file.decode('UTF-8')
#-*- coding: utf-8 -*-
import os
dirList=os.listdir(u"C:\\")
for file in dirList:
print file
file2 = file.decode('UTF-8')
count = len(open('C:\\' + file2).readlines())
print count

How to change the format of a path string to a different OS?

I need to change a file path from MAC to Windows and I was about to just do a simple .replace() of any / with \ but it occurred to me that there may be a better way. So for example I need to change:
foo/bar/file.txt
to:
foo\bar\file.txt

You can use this:
>>> s = '/foo/bar/zoo/file.ext'
>>> import ntpath
>>> import os
>>> s.replace(os.sep,ntpath.sep)
'\\foo\\bar\\zoo\\file.ext'

The pathlib module (introduced in Python 3.4) has support for this:
from pathlib import PureWindowsPath, PurePosixPath
# Windows -> Posix
win = r'foo\bar\file.txt'
posix = str(PurePosixPath(PureWindowsPath(win)))
print(posix) # foo/bar/file.txt
# Posix -> Windows
posix = 'foo/bar/file.txt'
win = str(PureWindowsPath(PurePosixPath(posix)))
print(win) # foo\bar\file.txt

Converting to Unix:
import os
import posixpath
p = "G:\Engineering\Software_Development\python\Tool"
p.replace(os.sep, posixpath.sep)
This will replace the used-os separator to Unix separator.
Converting to Windows:
import os
import ntpath
p = "G:\Engineering\Software_Development\python\Tool"
p.replace(os.sep, ntpath.sep)
This will replace the used-os separator to Windows separator.

os.path.join will intelligently join strings to form filepaths, depending on your OS-type (POSIX, Windows, Mac OS, etc.)
Reference: http://docs.python.org/library/os.path.html#os.path.join
For your example:
import os
print os.path.join("foo", "bar", "file.txt")

since path is actually a string, you can simply use following code:
p = '/foo/bar/zoo/file.ext'
p = p.replace('/', '\\')
output:
'\\foo\\bar\\zoo\\file.ext'