Develop Reference

generate zip folder and add to storage django

Hi guys I need to generate a zip file from a bunch of images and add it as a file to the database
This is the code to generate the file
def create_zip_folder(asin):
print('creating zip folder for asin: ' + asin)
asin_object = Asin.objects.get(asin=asin)
# create folder
output_dir = f"/tmp/{asin}"
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# download images
for img in asin_object.asin_images.all():
urllib.request.urlretrieve(img.url, f"{output_dir}/{img.id}.jpg")
# zip all files in output_dir
zip_file = shutil.make_archive(asin, 'zip', output_dir)
asin_object.zip_file = zip_file
asin_object.has_zip = True
asin_object.save()
# delete folder
shutil.rmtree(output_dir)
return True
This all works and I can see the files generated in my editor but when I try to access it in the template asin.zip_file.url I get this error
SuspiciousOperation at /history/
Attempted access to '/workspace/B08MFR2DRS.zip' denied.
Why is this happening? I thought the file is to be uploaded to the file storage through the model but apparently it's in a restricted folder, this happens both in development (with local file storage) and in production (with s3 bucket as file storage)

How to upload a file to your website hosted by Heroku?

I developed a website that would take a user provided file and extract relevant data out of it, I used Python/Flask for the backend and the website is stored in Heroku.
The application is working fine on my local machine, but when I run it in Heroku, whenever I attempt to upload/process a file I get this message on Heroku's logs:
2022-02-14T17:27:48.000421+00:00 app[web.1]: FileNotFoundError: [Errno 2] No such file or directory: '/application/uploads/report.html'
My python code that goes around the file uploading is:
app.config['UPLOAD_PATH'] = './application/uploads'
#app.route('/read', methods=['GET', 'POST'])
def read():
form = ReadForm()
if form.validate_on_submit():
if request.method == 'POST':
files = request.files.getlist('read')
for file in files:
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_PATH'], filename))
return redirect(url_for('show'))
return render_template('read.html', form=form)
How do I make the uploaded files available to my application? After reading the file, the application just delete it.
I saw a question with a very similar title as mine, here:
How to upload file on website hosted by Heroku?
And one of the answers there, suggests using Amazon's S3, is it the only solution?

#buran pointed me to another post here which helped me solving the issue: Python Flask upload file to app folder on server (heroku)
I added the following to my path definition:
base_path = os.path.dirname(__file__)
So the final one would be:
# Path for the files upload part
app.config['UPLOAD_PATH'] = '/uploads'
base_path = os.path.dirname(__file__)
Then when uploading a file I refer to the path like this:
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(base_path + "/" + app.config['UPLOAD_PATH'], filename))
So, instead of:
file.save(os.path.join(app.config['UPLOAD_PATH'], filename))
I used:
file.save(os.path.join(base_path + "/" + app.config['UPLOAD_PATH'], filename))

Unable to load contents while reading a .txt file in Python3

I am intending to extract some data stored in a .txt file using python 3, however, when I tried to print out the file content, the program does not display any thing in the console. This is the code snippet I use to read the file:
def get_data(directory):
entries = os.listdir(directory)
#print(entries)
count = 0;
for file in entries:
#print(file)
if file.endswith('.txt'):
with open(file) as curr_file:
#print(curr_file)
#read data and write it to an
#excel worksheet
print(curr_file.readline())
curr_file.close()
What kind of changes am I supposed to make to let the program display contents of the file?
Update: I tried to print out all files saved in entries and the result looks fine. The following is the code snippet I used to unzip files in the directory, I am not sure whether there're anything wrong with it.
def read_zip(path):
file_list = os.listdir(path)
#print(file_list)
#create a new directory and store
#the extracted file there
directory = 'C:/Users/chent/Desktop/Test'
try:
if not os.path.exists(directory):
os.makedirs(directory, exist_ok=True)
print('Folder created')
except FileExistsError:
print ('Directory not created')
for file in file_list:
if file.endswith('.zip'):
filePath=path+'/'+file
zip_file = zipfile.ZipFile(filePath)
for names in zip_file.namelist():
zip_file.extract(names, directory)
get_data(directory)
zip_file.close()
Solution: It turns out that I didn't specify the file path when use with open() statement, which caused the program unable to locate files. To fix it, use with open(file_path, file, "r") as curr_file. See details in my updated code:
def get_data(path):
files = os.listdir(path)
for file in files:
#print(file)
try:
if file.endswith('.txt'):
print(file)
with open('C:/Users/chent/Desktop/Test/' + file, "r", ) as curr_file:
# print(curr_file.readlines())
print(curr_file)
line = curr_file.readline()
print(line)
except FileNotFoundError:
print ('File not found')
path = 'C:/Users/chent/Desktop/Test'
get_data(path)

The problem is that you use curr_file.readline() which only returns the first line.
Use curr_file.read() to get the whole file contents.

Can i have two different sets of app configurations?

I have made a flask server REST API that does an upload to a specific folder.
The upload must comply to a specific file extension, and must be in a specific folder.
So i made these two app.configs:
app.config['UPLOAD_EXTENSIONS'] = ['.stp', '.step']
app.config['UPLOAD_PATH'] = 'uploads'
However, i want to make a new route, for a new upload of a specific file extension to another folder.
So can i have two sets of app.config['UPLOAD_EXTENSIONS'] and app.config['UPLOAD_PATH']?
One set will be for extension1 in folder1 and the other set for extension2 in folder2.

Try using the extension Flask-Uploads .
Or, proceeding from the file format, form a subdirectory in your UPLOAD_PATH.
import os
def select_directory(filename: str) -> str:
file_format = filename.split('.')[1]
your_path1 = 'path1'
your_path2 = 'path2'
if file_format in ('your format1', 'your format2'):
full_path = os.path.join(app.config['UPLOAD_FOLDER'], your_path1, filename)
else:
full_path = os.path.join(app.config['UPLOAD_FOLDER'], your_path2, filename)
return full_path

Dropbox API v2 - uploading files

I'm trying to loop through a folder structure in python and upload each file it finds to a specified folder. The problem is that it's uploading a file with the correct name, however there is no content and the file size is only 10 bytes.
import dropbox, sys, os
try:
dbx = dropbox.Dropbox('some_access_token')
user = dbx.users_get_current_account()
except:
print ("Negative, Ghostrider")
sys.exit()
rootdir = os.getcwd()
print ("Attempting to upload...")
for subdir, dirs, files in os.walk(rootdir):
for file in files:
try:
dbx.files_upload("afolder",'/bfolder/' + file, mute=True)
print("Uploaded " + file)
except:
print("Failed to upload " + file)
print("Finished upload.")

Your call to dbx.files_upload("afolder",'/bfolder/' + file, mute=True) says: "Send the text afolder and write it as a file named '/bfolder/' + file".
From doc:
files_upload(f, path, mode=WriteMode('add', None), autorename=False, client_modified=None, mute=False)
Create a new file with the contents provided in the request.
Parameters:
f – A string or file-like obj of data.
path (str) – Path in the user’s Dropbox to save the file.
....
Meaning that f must be the content of the file (and not the filename string).
Here is a working example:
import dropbox, sys, os
dbx = dropbox.Dropbox('token')
rootdir = '/tmp/test'
print ("Attempting to upload...")
# walk return first the current folder that it walk, then tuples of dirs and files not "subdir, dirs, files"
for dir, dirs, files in os.walk(rootdir):
for file in files:
try:
file_path = os.path.join(dir, file)
dest_path = os.path.join('/test', file)
print 'Uploading %s to %s' % (file_path, dest_path)
with open(file_path) as f:
dbx.files_upload(f, dest_path, mute=True)
except Exception as err:
print("Failed to upload %s\n%s" % (file, err))
print("Finished upload.")
EDIT: For Python3 the following should be used:
dbx.files_upload(f.read(), dest_path, mute=True)

For Dropbox Business API below python code helps uploading files to dropbox.
#function code
import dropbox
def dropbox_file_upload(access_token,dropbox_file_path,local_file_name):
'''
The function upload file to dropbox.
Parameters:
access_token(str): Access token to authinticate dropbox
dropbox_file_path(str): dropboth file path along with file name
Eg: '/ab/Input/f_name.xlsx'
local_file_name(str): local file name with path from where file needs to be uploaded
Eg: 'f_name.xlsx' # if working directory
Returns:
Boolean:
True on successful upload
False on unsuccessful upload
'''
try:
dbx = dropbox.DropboxTeam(access_token)
# get the team member id for common user
members = dbx.team_members_list()
for i in range(0,len(members.members)):
if members.members[i].profile.name.display_name == logged_in_user:
member_id = members.members[i].profile.team_member_id
break
# connect to dropbox with member id
dbx = dropbox.DropboxTeam(access_token).as_user(member_id)
# upload local file to dropbox
f = open(local_file_name, 'rb')
dbx.files_upload(f.read(),dropbox_file_path)
return True
except Exception as e:
print(e)
return False