I have some code that I wrote that downloads images from a website. The way that it currently works it needs to guess what the file extension will be for the url it will be downloading from. The block of code that does that looks like this:
for imageLink in imageLinks:
try:
urllib.request.urlretrieve(imageLink + ".png", str(threadName) + "/" + str(count) + ".png")
except:
try:
urllib.request.urlretrieve(imageLink + ".jpg",str(threadName) + "/" + str(count) + ".png")
except:
try:
urllib.request.urlretrieve(imageLink + ".gif",str(threadName) + "/" + str(count) + ".gif")
except:
urllib.request.urlretrieve(imageLink + ".webm",str(threadName) + "/" + str(count) + ".webm")
As it stands the code is relying on a fail in order to try something else.
I wanted to know if their is a way to have this functionality but to basically just look better. These methods will give identical errors if they fail so I want to just go through them sequentially until one works
for ext in ('.png', '.jpg', '.gif', '.webm'):
try:
urllib.request.urlretrieve(imageLink + ext, str(threadName) + "/" + str(count) + ext)
break
except:
pass
You can use a try/except block inside a function and return None if the control goes to the except statement. You can optimize the for loop according to your own needs. One example is here:
def get_url(link1, link2):
try:
requestData = urllib.request.urlretrieve(link1, link2)
except:
return None
return requestData
for imageLink in imageLinks:
data = urllib.request.urlretrieve(imageLink + ".png", str(threadName) + "/" + str(count) + ".png")
if data == None:
data = urllib.request.urlretrieve(imageLink + ".jpg",str(threadName) + "/" + str(count) + ".png")
if data == None:
data = urllib.request.urlretrieve(imageLink + ".gif",str(threadName) + "/" + str(count) + ".gif")
if data == None:
urllib.request.urlretrieve(imageLink + ".webm",str(threadName) + "/" + str(count) + ".webm")
Related
Is there a way to specify the whole path when running mongodump? I tried using --out but what it does at the moment is saving into a file at my_given_path/database/collection_name.json.gz
I have the following:
path = file_path + '/' + database + '/' + collection + '/'
query_input = "{\\\"metadata_id\\\": {\\\"\$oid\\\": \\\"" + metadata_id + "\\\"}}"
command = "mongodump --uri " + connection_string + database + " " \
"--collection=" + collection + " --query=\"" + query_input + "\" --gzip --out=" + path + " --quiet"
for which the file is saved at:
file_path/database/collection/database/collection.json.gz
Ideally I would like to save it into
file_path/database/collection/metadata_id.json.gz
Would this be possible?
You can use the --archive=<file> flag instead of --out
I wrote the code and now I have problem that the code finde only the first value (for example only img, we have img2 too, but it went to another picture) and copy only one, but we have 2 possibilities.
for i in df_list:
img = (filepath + i + ".jpg")
img2 = (filepath + i + "-1" + ".jpg")
img3 = (filepath + i + "-2" ".jpg")
img4 = (filepath + i + "-3" + ".jpg")
img5 = (filepath + i + "-4" + ".jpg")
img6 = (filepath + i + " -5" + ".jpg")
try:
shutil.copy(img, newpath, follow_symlinks=True)
except:
try:
shutil.copy(img6, newpath, follow_symlinks=True)
except:
try:
shutil.copy(img2, newpath, follow_symlinks=True)
except:
try:
shutil.copy(img3, newpath, follow_symlinks=True)
except:
try:
shutil.copy(img4, newpath, follow_symlinks=True)
except:
try:
shutil.copy(img5, newpath, follow_symlinks=True)
except:
with open("C:/Users/"+user+"/Desktop/J/"+datum+"/"+"Napake.txt", "a") as text_file:
print("Slika za ident {} ne obstaja.\n".format(i), file=text_file)
I neeed help, thank you for answers.
Rather than copying the try,except you can loop over the filenumbers and try copy each file. and print an error if there is an exception.
save_path = "C:/Users/" + user + "/Desktop/J/" + datum + "/" + "Napake.txt"
for folder in df_list:
for index in range(6):
if index == 0:
img = filepath + folder + ".jpg"
else:
img = f"{filepath}{folder}-{index}.jpg"
try:
shutil.copy(img6, newpath, follow_symlinks=True)
with open(save_path, "a") as text_file:
text_file.write(f"Slika za ident {folder}-{i} ne obstaja.\n")
except Exception as e:
print('could not copy file')
print(e)
I would also recommend having a look at this answer to see how to copy all files in a directory.
I have a program which has a loop in it.
In this loop I load images from a file.
But there are some not usable images, that I don`t want to sort by hand.
I have this code:
img = Image.open("downloads/parrot/" + pictures[i])
img = img.resize((150,150))
img.save("Validation/parrot/" + "picture" + str(i) + ".png")
i = i + 1
I tried to use the try except method, but it always stopped the program.
Is there any way to use an if loop, like "if imageisuseful() = false:"?
Or do you may have any other Ideas?
Thank you for your help.
If you don't want your try / except to stop your code, you can pass :
try:
img = Image.open("downloads/parrot/" + pictures[i])
img = img.resize((150,150))
img.save("Validation/parrot/" + "picture" + str(i) + ".png")
i = i + 1
except:
pass # pictures[i] raised an exception here
Depending on where the error is happening (either at the open or the resize) you might need to move things around:
img = Image.open("downloads/parrot/" + pictures[i])
try:
img = img.resize((150,150))
img.save("Validation/parrot/" + "picture" + str(i) + ".png")
i = i + 1
except:
print("FILE {} DOESN'T WORK".format(pictures[i]))
I have more than 500 xml files and each xml file should processed on FME workbench individually (iteration of FME workbench for each xml file).
For such a propose i have to run a python file (loop.py) to iterate FME workbench for each xml file.
The whole process was working in past on other PC without any problem. Now Once i run Module i got the following error:
Traceback (most recent call last):E:\XML_Data
File "E:\XML_Data\process\01_XML_Tile_1.py", line 28, in
if "Translation was SUCCESSFUL" in open(path_log + "\" + data + ".log").read():
IOError: [Errno 2] No such file or directory: 'E:\XML_Data\data_out\log_01\re_3385-5275.xml.log'
Attached the python code(loop.py).
Any help is greatly appreciated.
import os
import time
# Mainpath and Working Folder:
#path_main = r"E:\XML_Data"
path_main = r"E:\XML_Data"
teil = str("01")
# variables
path_in = path_main + r"\data_in\03_Places\teil_" + teil # "Source folder of XML files"
path_in_tile10 = path_main + r"\data_in\01_Tiling\10x10.shp" # "Source folder of Grid shapefile"
path_in_commu = path_main + r"\data_in\02_Communities\Communities.shp" # "Source folder of Communities shapefile"
path_out = path_main + r"\data_out\teil_" + teil # "Output folder of shapefiles that resulted from XML files (tile_01 folder)"
path_log = path_main + r"\data_out\log_" + teil # "Output folder of log files for each run(log_01 folder)"
path_fme = r"%FME_EXE_2015%" # "C:\Program Files\FME2015\fme.exe"
path_fme_workbench = path_main + r"\process\PY_FME2015.fmw" # "path of FME workbench"
datalists = os.listdir(path_in)
count = 0
# loop each file individually in FME
for data in datalists:
if data.find(".xml") != -1:
count +=1
print ("Run-No." + str(count) + ": with data " + data)
os.system (path_fme + " " + path_fme_workbench + " " + "--SourceDataset_XML"+ " " + path_in + "\\" + data + " " + "--SourceDataset_SHAPE" + " " + path_in_tile10 + " " + "--SourceDataset_SHAPE_COMU" + " " + path_in_commu + " " + "--DestDataset_SHAPE" +" " +path_out + " " +"LOG_FILENAME" + " " + path_log + "\\" + data + ".log" )
print ("Data processed: " + data)
shape = str(data[19:28]) + "_POPINT_CENTR_UTM32N.shp"
print ("ResultsFileName: " + shape)
if "Translation was SUCCESSFUL" in open(path_log + "\\" + data + ".log").read():
# Translation was successful and SHP file exists:
if os.path.isfile(path_out + "\\" + shape):
write_log = open(path_out + "\\" + "result_xml.log", "a")
write_log.write(time.asctime(time.localtime()) + " " + shape + "\n")
write_log.close()
print("Everything ok")
#Translation was successful, but SHP file does not exist:
else:
write_log = open(path_out + "\\" + "error_xml.log", "a")
write_log.write(time.asctime(time.localtime()) + " Data: " + shape + " unavailable.\n")
write_log.close()
# Translation was not successful:
else:
write_log = open(path_out + "\\" + "error_xml.log", "a")
write_log.write(time.asctime(time.localtime()) + " Translation " + Data + " not successful.\n")
write_log.close()
print ("Number of calculated files: " + str(count))
Most likely, the script failed at the os.system line, so the log file was not created from the command. Since you mentioned a different computer, it could be caused by many reasons, such as a different version of FME (so the environment variable %FME_EXE_2015% would not exist).
Use a workspace runner transformer to do this.
The FME version is outdated.so first check the version whether it is creating the problem.
subprocess.call(["C:/Program Files/fme/FMEStarter/FMEStarter.exe", "C:/Program Files/fme/fme20238/fme.exe", "/fmefile.fmw" "LOG_FILENAME","logfile"], stdin=None, stdout=None, stderr=None, shell=True, timeout=None)
I am trying to copy a files from different folders under a path to my usb drive.
So my source directory structure looks like this
/user/arun/Music/Songs/
under this I have different sub directories
Songs_1
Songs_2
Songs_3
the target folder is under anyone of these Songs directory
Songs_1/Kid Rock/All summer long.mp3
Songs_2/Linkin Park/In the end.mp3
Now I am constructing my src_dir in a try/except way like this.
for album,song in song_database.iteritems():
for s in song:
try:
src_dir_1 = src_dir + "/" + "Songs_1" + "/" + album + "/" + s + ".mp3"
shutil.copy2(src_dir_1,dest_dir
print src_dir_1
except IOError:
pass
try:
src_dir_1 = src_dir + "/" + "Songs_2" + "/" + album + "/" + s + ".mp3"
shutil.copy2(src_dir_1,dest_dir)
print src_dir_1
except IOError:
pass
try:
src_dir_1 = src_dir + "/" + "Songs_3" + "/" + album + "/" + s + ".mp3"
shutil.copy2(src_dir_1,dest_dir)
print src_dir_1
except IOError:
pass
try:
src_dir_1 = src_dir + "/" + "Songs_4" + "/" + album + "/" + s + ".mp3"
shutil.copy2(src_dir_1,dest_dir)
print src_dir_1
except IOError:
pass
Is there a better way to do this ?
Seems like a loop would be better:
for album,song in song_database.iteritems():
for s in song:
for sdir in 'Songs_1', 'Songs_2', 'Songs_3':
try:
src_dir_1 = src_dir + "/" + sdir + "/" + album + "/" + s + ".mp3"
shutil.copy2(src_dir_1,dest_dir)
print src_dir_1
except IOError:
pass
And, perhaps you would want to add a break statement if you succeed in copying the source to the destination...
As a side note, you might want to use os.path.join instead:
src_dir_1 = os.path.join(src_dir, sdir, album, s + ".mp3")