I'm currently facing an issue.
(started trying out python 3 hours ago)
So I tried making this discord bot where if a user sends an image, the bot would save it. My issue was the bot wasn't saving it in a certain folder (idk how also haha) so what I tried was I would copy the said image where it would create a new folder and that new folder is where the copied image would be placed. The original copies of the image would then be deleted, thus only leaving the image files in the folder.
My issue here now is that it's not consistent. It works on the first image input but it won't work if it would be attempted on the second time.
I would like to find a simpler way on being able to save an image file where it would then be directed to a folder rather than being saved in the same place as the python file.
#client.command()
async def save(ctx):
try:
url = ctx.message.attachments[0].url
except IndexError:
print("Error : No attachments")
await ctx.send("No attachments detected")
else:
imageName = str(uuid.uuid4()) + '.jpg'
r = requests.get(url, stream=True)
with open(imageName,'wb',) as out_file:
print('Saving image: ' + imageName)
shutil.copyfileobj(r.raw, out_file,)
images = [f for f in os.listdir() if '.jpg' in f.lower()]
os.mkdir('Images')
for image in images:
new_path = 'Images/' + image
shutil.move(image, new_path)```
You just need to change with open(imageName, 'wb') as out_file:
As it is it will save the image in the folder where the script is running, if you want to save them in the Images folder you just have to change that to with open("Images/" + imageName, 'wb') as out_file: or any other folder.
I think you're not giving a specific path, you can give a specific path using os.join(os.getcwd(),Images)
#client.command()
async def save(ctx):
try:
url = ctx.message.attachments[0].url
except IndexError:
print("Error : No attachments")
await ctx.send("No attachments detected")
else:
imageName = str(uuid.uuid4()) + '.jpg'
r = requests.get(url, stream=True)
if r.status_code == 200:
with open(specific_path, 'wb') as f:
f.write(r.content)
Related
i'm trying to download 200k images using their URL.
This is my code:
import requests # to get image from the web
import shutil # to save it locally
r = requests.get(image_url, stream = True)
# Check if the image was retrieved successfully
if r.status_code == 200:
# Set decode_content value to True, otherwise the downloaded image file's size will be zero.
r.raw.decode_content = True
if not os.path.isdir('images/' + filename.rsplit('/',1)[0] + '/'):
os.makedirs('images/' + filename.rsplit('/',1)[0] + '/')
with open('images/' + filename,'wb') as f:
shutil.copyfileobj(r.raw, f)
The when i run it, it downloads some images but the rest doesn't. It gives the error:
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead
I have no idea why or when this happens. Maybe when a URL is unreachable? How can I assure that everything UP will be downloaded and exceptions will be passed?
What about using a try/except?
import requests # to get image from the web
import shutil # to save it locally
try:
r = requests.get(image_url, stream = True)
# Check if the image was retrieved successfully
if r.status_code == 200:
# Set decode_content value to True, otherwise the downloaded image file's size will be zero.
r.raw.decode_content = True
if not os.path.isdir('images/' + filename.rsplit('/',1)[0] + '/'):
os.makedirs('images/' + filename.rsplit('/',1)[0] + '/')
with open('images/' + filename,'wb') as f:
shutil.copyfileobj(r.raw, f)
except urllib3.exceptions.ProtocolError as error:
print("skipped error: " + error)
Perhaps to download such a large number of images you would be interested in an asynchronous web framework like aiohttp. This would save you from having to wait for a slow site to send you its image to download more.
I am trying to download an image from an instagram media URL:
https://instagram.fybz2-1.fna.fbcdn.net/v/t51.2885-15/fr/e15/p1080x1080/106602453_613520712600632_6255422472318530180_n.jpg?_nc_ht=instagram.fybz2-1.fna.fbcdn.net&_nc_cat=108&_nc_ohc=WQizf6rhDmQAX883HrQ&oh=140f221889178fd03bf654cf18a9d9a2&oe=5F4D2AFE
Pasting this into my browser will bring up the image, but when I run the following code I get the following error which i suspect is due to issues with the URL containing a query string (running this on a simple url ending in .jpg works without issue
File "C:/Users/19053/InstagramImageDownloader/downloadImage.py", line 18, in <module>
with open(filename, 'wb') as f:
OSError: [Errno 22] Invalid argument: '106602453_613520712600632_6255422472318530180_n.jpg?_nc_ht=instagram.fybz2-1.fna.fbcdn.net&_nc_cat=108&_nc_ohc=WQizf6rhDmQAX883HrQ&oh=140f221889178fd03bf654cf18a9d9a2&oe=5F4D2AFE'
Full code as follows:
## Importing Necessary Modules
import requests # to get image from the web
import shutil # to save it locally
## Set up the image URL and filename
image_url = "https://instagram.fybz2-1.fna.fbcdn.net/v/t51.2885-15/fr/e15/p1080x1080/106602453_613520712600632_6255422472318530180_n.jpg?_nc_ht=instagram.fybz2-1.fna.fbcdn.net&_nc_cat=108&_nc_ohc=WQizf6rhDmQAX883HrQ&oh=140f221889178fd03bf654cf18a9d9a2&oe=5F4D2AFE"
filename = image_url.split("/")[-1]
# Open the url image, set stream to True, this will return the stream content.
r = requests.get(image_url, stream=True)
# Check if the image was retrieved successfully
if r.status_code == 200:
# Set decode_content value to True, otherwise the downloaded image file's size will be zero.
r.raw.decode_content = True
# Open a local file with wb ( write binary ) permission.
with open(filename, 'wb') as f:
shutil.copyfileobj(r.raw, f)
print('Image sucessfully Downloaded: ', filename)
else:
print('Image Couldn\'t be retreived')
The problem is with the filename. You need to first split by ? then take the first element then split by /
import requests # to get image from the web
import shutil # to save it locally
## Set up the image URL and filename
image_url = "https://instagram.fybz2-1.fna.fbcdn.net/v/t51.2885-15/fr/e15/p1080x1080/106602453_613520712600632_6255422472318530180_n.jpg?_nc_ht=instagram.fybz2-1.fna.fbcdn.net&_nc_cat=108&_nc_ohc=WQizf6rhDmQAX883HrQ&oh=140f221889178fd03bf654cf18a9d9a2&oe=5F4D2AFE"
filename = image_url.split("?")[0].split("/")[-1]
# Open the url image, set stream to True, this will return the stream content.
r = requests.get(image_url, stream=True)
# Check if the image was retrieved successfully
if r.status_code == 200:
# Set decode_content value to True, otherwise the downloaded image file's size will be zero.
r.raw.decode_content = True
# Open a local file with wb ( write binary ) permission.
with open(filename, 'wb') as f:
shutil.copyfileobj(r.raw, f)
print('Image sucessfully Downloaded: ', filename)
else:
print('Image Couldn\'t be retreived')
i am new on python, i wrote simple script to send all image sorted by time in folder to API. this code working on just one file(jpg), and can't send the rest of image in folder. I want to if i run this code, it's just waiting until some image added to current folder, when image is inside folder then it will send to API by time based on images that first existed. I am very confused, any helps will be appriciated! thx
import glob
import argparse
import requests
import json
import time
import os
def main():
result = []
file = glob.glob("/path/to/dir/*.jpg")
regions = ['id']
time_to_wait = 10000
time_counter = 0
while not os.path.exists(file):
time.sleep(1)
time_counter += 1
if time_counter > time_to_wait: break
print("waiting for file...")
if os.path.isfile(file):
with open(file, 'rb') as fp:
response = requests.post(
'https://GET_API/',
data=dict(regions=regions),
files=dict(upload=fp),
headers={'Authorization': 'Token ' + 'XXX'})
result.append(response.json())
resp_dict = json.loads(json.dumps(result, indent=2))
if resp_dict[0]['results']:
num=resp_dict[0]['results'][0]['plate']
print(f"DETECTED NUMBER: {num}")
os.remove(file)
else:
print("file doesn't exists!")
if __name__ == '__main__':
main()
You didn't update your file on each iteration. Maybe that's why no more new files are detected. file also need to be treated as a list, so I guess you should iterate through file. Your while loop should look like this:
while True:
files = glob.glob(os.path.join('path', 'to', 'dir', '*.jpg'))
for file in files:
if os.path.isfile(file):
with open(file, 'rb') as fp:
# Upload and delete
# sleep
I have 1k of image urls in a csv file and I am trying to download all the images from the urls. I don't know why I am not able to download all the images. Here is my code:
print('Beginning file download with requests')
path = '/home/tt/image_scrap/image2'
for idx, url in tqdm(enumerate(dataset['url']), total=len(dataset['url'])):
response = requests.get(url,stream=True)
time.sleep(2)
filename = url.split("/")[-1]
with open(path+'/'+filename, 'wb') as f:
f.write(response.content)
Try / Except statements are really good for these type of 'errors':
Try this:
try:
with open(path+'/'+filename, 'wb') as f:
f.write(response.content)
except Exception as error:
print(error)
I've a lot of URL with file types .docx and .pdf I want to run a python script that downloads them from the URL and saves it in a folder. Here is what I've done for a single file I'll add them to a for loop:
response = requests.get('http://wbesite.com/Motivation-Letter.docx')
with open("my_file.docx", 'wb') as f:
f.write(response.content)
but the my_file.docx that it is saving is only 266 bytes and is corrupt but the URL is fine.
UPDATE:
Added this code and it works but I want to save it in a new folder.
import os
import shutil
import requests
def download_file(url, folder_name):
local_filename = url.split('/')[-1]
path = os.path.join("/{}/{}".format(folder_name, local_filename))
with requests.get(url, stream=True) as r:
with open(path, 'wb') as f:
shutil.copyfileobj(r.raw, f)
return local_filename
Try using stream option:
import os
import requests
def download(url: str, dest_folder: str):
if not os.path.exists(dest_folder):
os.makedirs(dest_folder) # create folder if it does not exist
filename = url.split('/')[-1].replace(" ", "_") # be careful with file names
file_path = os.path.join(dest_folder, filename)
r = requests.get(url, stream=True)
if r.ok:
print("saving to", os.path.abspath(file_path))
with open(file_path, 'wb') as f:
for chunk in r.iter_content(chunk_size=1024 * 8):
if chunk:
f.write(chunk)
f.flush()
os.fsync(f.fileno())
else: # HTTP status code 4XX/5XX
print("Download failed: status code {}\n{}".format(r.status_code, r.text))
download("http://website.com/Motivation-Letter.docx", dest_folder="mydir")
Note that mydir in example above is the name of folder in current working directory. If mydir does not exist script will create it in current working directory and save file in it. Your user must have permissions to create directories and files in current working directory.
You can pass an absolute file path in dest_folder, but check permissions first.
P.S.: avoid asking multiple questions in one post
try:
import urllib.request
urllib.request.urlretrieve(url, filename)