#Filename:backup_ver1
import os
import time
#1 Using list to specify the files and directory to be backed up
source = r'C:\Documents and Settings\rgolwalkar\Desktop\Desktop\Dr Py\Final_Py'
#2 define backup directory
destination = r'C:\Documents and Settings\rgolwalkar\Desktop\Desktop\PyDevResourse'
#3 Setting the backup name
targetBackup = destination + time.strftime('%Y%m%d%H%M%S') + '.rar'
rar_command = "rar.exe a -ag '%s' %s" % (targetBackup, ''.join(source))
##i am sure i am doing something wrong here - rar command please let me know
if os.system(rar_command) == 0:
print 'Successful backup to', targetBackup
else:
print 'Backup FAILED'
O/P:- Backup FAILED
winrar is added to Path and CLASSPATH under Environment variables as well - anyone else with a suggestion for backing up the directory is most welcome
Maybe instead of writing your own backup script you could use python tool called rdiff-backup, which can create incremental backups?
The source directory contains spaces, but you don't have quotes around it in the command line. This might be a reason for the backup to fail.
To avoid problems like this, use the subprocess module instead of os.system:
subprocess.call(['rar.exe', 'a', '-ag', targetBackup, source])
if the compression algorithm can be something else and its just to backup a directory, why not do it with python's own tar and gzip instead? eg
import os
import tarfile
import time
root="c:\\"
source=os.path.join(root,"Documents and Settings","rgolwalkar","Desktop","Desktop","Dr Py","Final_Py")
destination=os.path.join(root,"Documents and Settings","rgolwalkar","Desktop","Desktop","PyDevResourse")
targetBackup = destination + time.strftime('%Y%m%d%H%M%S') + 'tar.gz'
tar = tarfile.open(targetBackup, "w:gz")
tar.add(source)
tar.close()
that way, you are not dependent on rar.exe on the system.
Related
I have a local variable in a Python script that creates temporary files using a path in my local C:\<User> folder:
Output_Feature_Class = "C:\\Users\\<User>\\Documents\\ArcGIS\\Default.gdb\\Bnd_"
However, I want to be able to share this script with others and don't want to have to hardcode a file path for each person using it. I basically want it to go to the same folder but <insert their User Name> example: C:\\TBrown\\Documents\\ArcGIS\Default.gdb\\Bnd_. I cannot seem to get just using
Output_Feature_Class = "..\\Documents\\ArcGIS\\Default.gdb\\Bnd_"
to work. Is there something I am missing?
Same answer as #Simon but with string formatting to condense it a bit and not attempt to concatenate strings together:
import getpass
Output_Feature_Class = "C:\\Users\\%s\\Documents\\ArcGIS\\Default.gdb\\Bnd_" % getpass.getuser()
As #Matteo Italia points out, Nothing guarantees you that user profiles are under c:\users, or that the user profile directory is called with the name of the user. So, perhaps addressing it by getting the user's home directory and building the path from there would be more advantageous:
from os.path import expanduser
Output_Feature_Class = "%s\\Documents\\ArcGIS\\Default.gdb\\Bnd_" % expanduser("~")
Update
As #Matteo Italia points out again, there may be cases when the Documents directory is located somewhere else by default. This may help find the path of the Documents of My Documents folder: reference (link)
from win32com.shell import shell
df = shell.SHGetDesktopFolder()
pidl = df.ParseDisplayName(0, None, "::{450d8fba-ad25-11d0-98a8-0800361b1103}")[1]
Output_Feature_Class = "%s\\ArcGIS\\Default.gdb\\Bnd_" % shell.SHGetPathFromIDList(pidl)
Rather than asking them to input their username, why not use getpass?
For example to get their username:
import getpass
a = getpass.getuser()
Output_Feature_Class = "C:\\Users\\" + a + "\\Documents\\ArcGIS\\Default.gdb\\Bnd_"
If you work on Windows (and this will work for Windows only) the pywin module can find the path to documents:
from win32com.shell import shell, shellcon
a = shell.SHGetFolderPath(0, shellcon.CSIDL_PERSONAL, None, 0)
Output_Feature_Class = "{}\\ArcGIS\\Default.gdb\\Bnd_".format(a)
but this is not cross platform. Thanks to martineau for this solution see Finding the user's "My Documents" path for finding Documents path.
Instead of storing temporary files in a location of your choosing which may or may not exist, why not use the Windows %TEMP% environment variable? If they don't have %TEMP% set, a lot of software wont work.
import os
def set_temp_path(*args):
if os.name is 'nt':
temp_path = os.getenv('TEMP')
if not temp_path:
raise OSError('No %TEMP% variable is set? wow!')
script_path = os.path.join(temp_path, *args)
if not os.path.exists(script_path):
os.makedirs(script_path)
return script_path
elif os.name is 'posix':
#perform similar operation for linux or other operating systems if desired
return "linuxpath!"
else:
raise OSError('%s is not a supported platform, sorry!' % os.name)
You can use this code for arbitrary temporary file structures for this or any other script:
my_temp_path = set_temp_path('ArcGIS', 'Default.gdb', 'Bnd_')
Which will create all the needed directories for you and return the path for further use in your script.
'C:\\Users\\JSmith\\AppData\\Local\\Temp\\ArcGIS\\Default.gdb\\Bnd_'
This is of course incomplete if you intend on supporting multiple platforms, but this should be straightforward on linux using the global /tmp or /var/temp paths.
To get the user's home directory you can use os.path.expanduser to get a user's home directory.
Output_Feature_Class = os.path.join(os.path.expanduser('~'), "Documents\\ArcGIS\\Default.gdb\\Bnd_")
As mentioned by others, before you do this please consider if this is the best location for these files. Temporary files should be in a location reserved by OS conventions for temporary files.
I just followed totorial《A Byte of Python》,there is a problem to back up all important files.
Here is my code:
import os
import time
source = ['"C:\\My Documents"', 'C:\\Code']
target_dir = 'D:\\Backup'
target = target_dir + os.sep + \
time.strftime('%Y%m%d%H%M%S') + '.Zip'
if not os.path.exists(target_dir):
os.mkdir(target_dir)
zip_command = 'zip -r {0} {1}'.format(target,
' '.join(source))
print('Zip command is:')
print(zip_command)
print('Running:')
if os.system(zip_command) == 0:
print('Successful backup to', target)
else:
print('Backup Failed')
Although I am on windows system, I think I have added C:\Windows\GnuWin32\bin to system PATH environment variable.Is it right?
When I type zip in terminal it shows like this
But when I run the program, the result is:
Zip command is:
zip -r D:\Backup\20170310193946.Zip "C:\My Documents" C:\Code
Running:
Backup Failed
I have tried again and again, still don't know why.
Any help would be greatly appreciated. Thank you
Make sure you installed zip 3.0 from the GnuWin32 website.
Maybe you need to change system path environment variable from C:\Windows\GnuWin32\bin to C:\Program Files (x86)\GnuWin32\bin.
Restart IDE (in my case it was PyCharm) after changing the path.
I am using Python and trying to connect to SFTP and want to retrieve an XML file from there and need to place it in my local system. Below is the code:
import paramiko
sftpURL = 'sftp.somewebsite.com'
sftpUser = 'user_name'
sftpPass = 'password'
ssh = paramiko.SSHClient()
# automatically add keys without requiring human intervention
ssh.set_missing_host_key_policy( paramiko.AutoAddPolicy() )
ssh.connect(sftpURL, username=sftpUser, password=sftpPass)
ftp = ssh.open_sftp()
files = ftp.listdir()
print files
Here connection is success full. And now I want to see all the folders and all the files and need to enter in to required folder for retrieving the XML file from there.
Finally my intention is to view all the folders and files after connecting to SFTP server.
In the above code I had used ftp.listdir() through which I got output as some thing like below
['.bash_logout', '.bash_profile', '.bashrc', '.mozilla', 'testfile_248.xml']
I want to know whether these are the only files present?
And the command I used above is right to view the folders too?
What is the command to view all the folders and files?
The SFTPClient.listdir returns everything, files and folders.
Were there folders, to tell them from the files, use SFTPClient.listdir_attr instead. It returns a collection of SFTPAttributes.
from stat import S_ISDIR, S_ISREG
sftp = ssh.open_sftp()
for entry in sftp.listdir_attr(remotedir):
mode = entry.st_mode
if S_ISDIR(mode):
print(entry.filename + " is folder")
elif S_ISREG(mode):
print(entry.filename + " is file")
The accepted answer by #Oz123 is inefficient. SFTPClient.listdir internally calls SFTPClient.listdir_attr and throws most information away returning file and folder names only. The answer then uselessly and laboriously re-retrieves all that data by calling SFTPClient.lstat for each file.
See also How to fetch sizes of all SFTP files in a directory through Paramiko.
Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server"
One quick solution is to examine the output of lstat of each object in ftp.listdir().
Here is how you can list all the directories.
>>> for i in ftp.listdir():
... lstatout=str(ftp.lstat(i)).split()[0]
... if 'd' in lstatout: print i, 'is a directory'
...
Files are the opposite search:
>>> for i in ftp.listdir():
... lstatout=str(ftp.lstat(i)).split()[0]
... if 'd' not in lstatout: print i, 'is a file'
...
Here is a solution I have come up with. Based on https://stackoverflow.com/a/59109706 . My solution gives a pretty output.
Update I have modified it slightly to incorporate Martin's suggestions. Now my code is considerably fast compared to my initial version using isdir and listdir
# prefix components:
space = ' '
branch = '│ '
# pointers:
tee = '├── '
last = '└── '
def stringpath(path):
# just a helper to get string of PosixPath
return str(path)
from pathlib import Path
from stat import S_ISDIR
def tree_sftp(sftp, path='.', parent='/', prefix=''):
"""
Loop through files to print it out
for file in tree_sftp(sftp):
print(file)
"""
fullpath = Path(parent, path)
strpath = stringpath(fullpath)
dirs = sftp.listdir_attr(strpath)
pointers = [tee] * (len(dirs) - 1) + [last]
pdirs = [Path(fullpath, d.filename) for d in dirs]
sdirs = [stringpath(path) for path in pdirs]
for pointer, sd, d in zip(pointers, sdirs, dirs):
yield prefix + pointer + d.filename
if S_ISDIR(d.st_mode):
extension = branch if pointer == tee else space
yield from tree_sftp(sftp, sd, prefix=prefix + extension)
You can try it out like this using pysftp
import pysftp
with pysftp.Connection(HOSTNAME, USERNAME, PASSWORD) as sftp:
for file in tree_sftp(sftp):
print(file)
Let me know if if works for you.
Initially I was thinking of using os.path.isdir but I don't think this works for zip files. Is there a way to peek into the zip file and verify that this directory exists? I would like to prevent using unzip -l "$#" as much as possible, but if that's the only solution then I guess I have no choice.
Just check the filename with "/" at the end of it.
import zipfile
def isdir(z, name):
return any(x.startswith("%s/" % name.rstrip("/")) for x in z.namelist())
f = zipfile.ZipFile("sample.zip", "r")
print isdir(f, "a")
print isdir(f, "a/b")
print isdir(f, "a/X")
You use this line
any(x.startswith("%s/" % name.rstrip("/")) for x in z.namelist())
because it is possible that archive contains no directory explicitly; just a path with a directory name.
Execution result:
$ mkdir -p a/b/c/d
$ touch a/X
$ zip -r sample.zip a
adding: a/ (stored 0%)
adding: a/X (stored 0%)
adding: a/b/ (stored 0%)
adding: a/b/c/ (stored 0%)
adding: a/b/c/d/ (stored 0%)
$ python z.py
True
True
False
You can check for the directories with ZipFile.namelist().
import os, zipfile
dir = "some/directory/"
z = zipfile.ZipFile("myfile.zip")
if dir in z.namelist():
print "Found %s!" % dir
for python(>=3.6):
this is how the is_dir() implemented in python source code:
def is_dir(self):
"""Return True if this archive member is a directory."""
return self.filename[-1] == '/'
It simply checks if the filename ends with a slash /, Can't tell if this will work correctly in some certain circumstances(so IMO it is badly implemented).
for python(<3.6):
as print(zipinfo) will show filemode but no corrsponding property or field is provided, I dive into zipfile module source code and found how it is implemented.
(see def __repr__(self): https://github.com/python/cpython/blob/3.6/Lib/zipfile.py)
possibly a bad idea but it will work:
if you want something simple and easy, this will work in most cases but it may fail because in some cases this field will not be printed.
def is_dir(zipinfo):
return "filemode='d" in zipinfo.__repr__()
Finally:
my solution is to check file mode manually and decide if the referenced file is actually a directory inspired by https://github.com/python/cpython/blob/3.6/Lib/zipfile.py line 391.
def is_dir(fileinfo):
hi = fileinfo.external_attr >> 16
return (hi & 0x4000) > 0
You can accomplish this using the built-in library ZipFile.
import zipfile
z = zipfile.ZipFile("file.zip")
if "DirName/" in [member.filename for member in z.infolist()]:
print("Directory exists in archive")
Tested and functional with Python32.
I am new to programming. I am trying to learn Python using CH Swaroop's "Byte of Python". One example is to create a program which will backup some files from one directory to another and compress them into a .zip format. Unfortunately the example he gives is only helpful if you happen to be a linux/unix user. For windows users he says only "Windows users can use the Info-Zip program" but doesn't elaborate further. This is the code he provides ...
#!/usr/bin/python
# Filename : backup_ver1.py
import os
import time
# 1. The files and directories to be backed up are specified in a list.
source = [r'C:\Users\ClickityCluck\Documents']
# 2. The backup must be stored in a main backup directory
target_dir = r'C:\Backup'
# 3. Zip seems good
# 4. Let's make the name of the file the current date/time
target = target_dir + time.strftime('%Y%m%d%H%M%S') + '.zip'
# 5. We use the zip command to put the files in a zip archive
zip_command = "zip -qr '%s' %s" % (target, ''.join(source))
# Run Backup
if os.system(zip_command) == 0:
print "Succesful backup to", target
else:
print 'BACKUP FAILED :('
Can anyone lay out for me a way to do this in the command line on windows 7? Thank you for your time and I apologize in advance if I have failed to provide some pertinent information :)
For python 3 users and using the format method answer should be:
# 1. The files and directories to be backed up are specified in a list.
source = [r'C\Users\MySourceDir']
# 2. The backup must be stored in a # main backup directory
target_dir = r'C:\Users\MyTargetDir'
# 3. The files are backed up into a zip file.
# 4. The name of the zip archive is the current date and time
target = target_dir + os.sep + time.strftime('%Y%m%d%H%M%S') + '.7z'
# Create target directory if it is not present
if not os.path.exists(target_dir):
os.mkdir(target_dir)
# 5. We use the zip command to put the files in a zip archive
zip_command = 'C:\\"Program Files"\\7-Zip\\7z a -t7z -r "{0}" "{1}"'.format(target,' '.join(source)) # be careful with spaces on each dir specification. Problems could arise if double-quotation marks aren't between them.
# Run the backup
print('Zip command is:')
print(zip_command)
print('Running:')
if os.system(zip_command) == 0:
print('Successful backup to', target)
else:
print('Backup FAILED')
zip is command line utility to create/update/extract ZIP archives, available on Unix/Linux/Mac OS X. If you want to archive files using a command line utility, you should find and install an appropriate on (compress, for example, is a part of resource kit).
The another way is to use python's zipfile module and make a useful command-line utility for windows :)
BTW, why your question refers to MS DOS?
#!/usr/bin/python -tt
# Filename: backup_ver1.py
import os
import time
# 1. The files and directories to be backed up are specified in a list.
source = ['C:\\Documents\\working_projects\\', 'C:\\Documents\\hot_tips\\']
# 2. The back up must be stored in a main back directory
target_dir = 'C:\\temp\\backup\\'
# 3. The files are backed up into a zip file
# 4. The name of the zip archive is the current date and time
target = target_dir + time.strftime('%Y%m%d%H%M%S') + '.7z'
# 5. We use the zip command to put the files in a zip archive
#zip_command = "zip -qr '%s' %s" % (target, ' '.join(source))
zip_command = 'C:\\"Program Files"\\7-Zip\\7z a -t7z "%s" %s' % (target, ' '.join(source))
# Run the backup
if os.system(zip_command) == 0:
print 'Successful back up to', target
else:
print 'Backup FAILED'