I have a unique situation. I have a local zip file (C:\Temp\abc.zip). I want to extract this file in python to a remote drive (\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns). Please note that there is no drive letter. I have write permissions to this folder and I can access via windows explorer. I have this following code,
import zipfile, os
def main():
zfile = zipfile.ZipFile("\\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns\Zinc.zip", 'r')
for name in zfile.namelist():
(dirname, filename) = os.path.split(name)
print "Decompressing " + filename + " on " + dirname
filename = "C:/Temp/" + filename
fd = open(filename,"w")
fd.write(zfile.read(name))
fd.close()
I get the below error:
IOError: [Errno 2] No such file or directory: '\\Crprvau01n1\\Cdv_prd$\\DataDrop\\Quartz\\IMM\\DevRuns\\Zinc.zip'
Any suggestions on how to read the remote zip file is appreciated.
Thanks
Use a raw string r'...' (or double every backslash):
zipfile.ZipFile(r"\\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns\Zinc.zip", 'r')
Backslash is an escape character in normal strings.
Related
I need to open a file from a different directory without using it's path while staying in the current directory.
When I execute the below code:
for file in os.listdir(sub_dir):
f = open(file, "r")
lines = f.readlines()
for line in lines:
line.replace("dst=", ", ")
line.replace("proto=", ", ")
line.replace("dpt=", ", ")
I get the error message FileNotFoundError: [Errno 2] No such file or directory: because it's in a sub directory.
Question: Is there an os command I can use that will locate and open the file in sub_dir?
Thanks! -let me know if this is a repeat, I searched and couldn't find one but may have missed it.
os.listdir() lists only the filename without a path. Prepend these with sub_dir again:
for filename in os.listdir(sub_dir):
f = open(os.path.join(sub_dir, filename), "r")
If all you are doing is loop over the lines from the file, just loop over the file itself; using with makes sure that the file is closed for you when done too. Last but not least, str.replace() returns the new string value, not change the value itself, so you need to store that return value:
for filename in os.listdir(sub_dir):
with open(os.path.join(sub_dir, filename), "r") as f:
for line in f:
line = line.replace("dst=", ", ")
line = line.replace("proto=", ", ")
line = line.replace("dpt=", ", ")
You must give the full path if those files are not in the current directory:
f = open( os.path.join(sub_dir, file) )
I would not use file as a variable name, maybe filename, since this is used to create a file object in Python.
Code to copy files using shutil
import shutil
import os
source_dir = "D:\\StackOverFlow\\datasets"
dest_dir = "D:\\StackOverFlow\\test_datasets"
files = os.listdir("D:\\StackOverFlow\\datasets")
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
for filename in files:
if file.endswith(".txt"):
shutil.copy(os.path.join(source_dir, filename), dest_dir)
print os.listdir(dest_dir)
I am trying to download files from an ftp server to a local folder. Below I have included my code, the script is able to access the ftp succesfully and list the names of the files on the ftp server however where my issue lies is I am not sure how to download them to a local directory.
from ftplib import FTP
import os, sys, os.path
def handleDownload(block):
file.write(block)
ddir='U:/Test Folder'
os.chdir(ddir)
ftp = FTP('sidads.colorado.edu')
ftp.login()
print ('Logging in.')
directory = '/pub/DATASETS/NOAA/G02158/unmasked/2012/04_Apr/'
print ('Changing to ' + directory)
ftp.cwd(directory)
ftp.retrlines('LIST')
print ('Accessing files')
filenames = ftp.nlst() # get filenames within the directory
print (filenames)
I tried using the following code below however it gives me the error in the photo below.
for filename in filenames:
if filename != '.':
local_filename = os.path.join('U:/Test Folder/', filename)
file = open(local_filename, 'wb')
ftp.retrbinary('RETR '+ filename, file.write)
file.close()
ftp.quit()
Try the following:
for filename in filenames:
if filename not in ['.', '..']:
local_filename = os.path.join(ddir, filename)
with open(local_filename, 'wb') as f_output:
ftp.retrbinary('RETR '+ filename, f_output.write)
ftp.quit()
Your code was still attempting to download . filenames. You need to indent the writing of the valid filenames. Also by using with it will close the file automatically.
I'm trying to open a .log extension file in Python but I keep encountering an IOError. I'm wondering if this has to do with the extension because clearly, the only way to get into that loop was if 'some.log' existed in the directory.
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open('some.log', "r")
print (f.read())
Traceback:
f = open('some.log', "r")
IOError: [Errno 2] No such file or directory: 'some.log'
When attempting to open a file in a different directory, you need to supply the absolute file path. Otherwise it attempts to open a file in the current directory.
You can use os.path.join to concatenate the location and filename
import os
location = '/Users/username/Downloads'
for filename in os.listdir(location):
if filename == 'some.log':
f = open(os.path.join(location, 'some.log'), "r")
print (f.read())
I need to open a file from a different directory without using it's path while staying in the current directory.
When I execute the below code:
for file in os.listdir(sub_dir):
f = open(file, "r")
lines = f.readlines()
for line in lines:
line.replace("dst=", ", ")
line.replace("proto=", ", ")
line.replace("dpt=", ", ")
I get the error message FileNotFoundError: [Errno 2] No such file or directory: because it's in a sub directory.
Question: Is there an os command I can use that will locate and open the file in sub_dir?
Thanks! -let me know if this is a repeat, I searched and couldn't find one but may have missed it.
os.listdir() lists only the filename without a path. Prepend these with sub_dir again:
for filename in os.listdir(sub_dir):
f = open(os.path.join(sub_dir, filename), "r")
If all you are doing is loop over the lines from the file, just loop over the file itself; using with makes sure that the file is closed for you when done too. Last but not least, str.replace() returns the new string value, not change the value itself, so you need to store that return value:
for filename in os.listdir(sub_dir):
with open(os.path.join(sub_dir, filename), "r") as f:
for line in f:
line = line.replace("dst=", ", ")
line = line.replace("proto=", ", ")
line = line.replace("dpt=", ", ")
You must give the full path if those files are not in the current directory:
f = open( os.path.join(sub_dir, file) )
I would not use file as a variable name, maybe filename, since this is used to create a file object in Python.
Code to copy files using shutil
import shutil
import os
source_dir = "D:\\StackOverFlow\\datasets"
dest_dir = "D:\\StackOverFlow\\test_datasets"
files = os.listdir("D:\\StackOverFlow\\datasets")
if not os.path.exists(dest_dir):
os.makedirs(dest_dir)
for filename in files:
if file.endswith(".txt"):
shutil.copy(os.path.join(source_dir, filename), dest_dir)
print os.listdir(dest_dir)
I want to move a large number of files from a windows system to a unix ftp server using python. I have a csv which has the current full path and filename and the new base bath to send it to (see here for an example dataset).
I have got a script using os.renames to do the transfer and directory creation in windows but can figure out a way to easily do it via ftp.
import os, glob, arcpy, csv, sys, shutil, datetime
top=os.getcwd()
RootOutput = top
startpath=top
FileList = csv.reader(open('FileList.csv'))
filecount=0
successcount=0
errorcount=0
# Copy/Move to FTP when required
ftp = ftplib.FTP('xxxxxx')
ftp.login('xxxx', 'xxxx')
directory = '/TransferredData'
ftp.cwd(directory)
##f = open(RootOutput+'\\Success_LOG.txt', 'a')
##f.write("Log of files Succesfully processed. RESULT of process run #:"+str(datetime.datetime.now())+"\n")
##f.close()
##
for File in FileList:
infile=File[0]
# local network ver
#outfile=RootOutput+File[4]
#os.renames(infile, outfile)
# ftp netowrk ver
# outfile=RootOutput+File[4]
# ftp.mkd(directory)
print infile, outfile
I tried the process in http://forums.arcgis.com/threads/17047-Upload-file-to-FTP-using-Python-ftplib but this is for moving all files in a directory, I have the old and new full file names and just need it to create the intermediate directories.
Thanks,
The following might work (untested):
def mkpath(ftp, path):
path = path.rsplit('/', 1)[0] # parent directory
if not path:
return
try:
ftp.cwd(path)
except ftplib.error_perm:
mkpath(ftp, path)
ftp.mkd(path)
ftp = FTP(...)
directory = '/TransferredData/'
for File in FileList:
infile = File[0]
outfile = File[4].split('\\') # need forward slashes in FTP
outfile = directory + '/'.join(outfile)
mkpath(ftp, outfile)
ftp.storbinary('STOR '+outfile, open(infile, 'rb'))