I'm having a problem with my project on Heroku, when I download a video from Youtube on localhost I used a code that takes the user's name and puts it in the Donwload directory, but in Heroku it doesn't put it there, I suspect it is with some problem when it comes to finding the director's place
#app.route('/', methods=['POST'])
def getvalue():
if request.method == 'POST':
name = request.form['url']
try:
url = name
youtube = pytube.YouTube(url)
video = youtube.streams.get_highest_resolution()
audio = youtube.streams.get_audio_only()
video_path = r'C:/Users/' + getpass.getuser() + '/Downloads/Youtube_Download'
audio_path = r'C:/Users/' + getpass.getuser() + '/Downloads/Youtube_Download/Audio'
if not os.path.exists(video_path):
os.makedirs(video_path)
os.makedirs(audio_path)
video.download(video_path)
audio.download(audio_path)
return render_template('index.html')
The problem is that there's no `C:/Users/' in the Heroku Dyno, since is a Linux environment.
The folder structure there will match your project folder structure.
But I don't actually recommend store any user data inside the Heroku Dyno since will be erased every time you restart the app (e.g. when deploying a new version)
You can use a Storage Bucket like Amazon S3 for this instead.
Related
I created a youtube video downloader project using pytube in python django.
in vies.py
url = request.POST.get('url')
itag = request.POST.get('itag')
yt = YouTube(url)
yt.streams.get_by_itag(itag).download()
messages.success(request, 'Download successful')
return redirect('/')
everything work good but when i run this code, video were download but it is download to the project directory.
if i hard code the download directory its work fine on local host
but when i deploy django to heroku server its does not download on cliant local storage
my question is, how can i download thie pytube video to cliant side storage not on server side?
is there anty way to grab the users download directory?
if i do like this still not download to users local storage while my code on heroku live server
its work fine on localhost.
url = request.POST.get('url')
itag = request.POST.get('itag')
path = pathlib.Path(pathlib.Path.home())
dirs = os.path.join(path, 'Downloads')
yt = YouTube(url)
yt.streams.get_by_itag(itag).download(dirs)
messages.success(request, 'Download successful'+str(dirs))
return redirect('/')
and one thing that user could be moble user or desktop user
project link on heroku
https://metube-dl.herokuapp.com/
I am trying to upload and save a file to my EC2 instance so that I can do some file manipulation.
My upload looks like this:
UPLOAD_FOLDER = os.getcwd() + '/uploads'
application = Flask(__name__)
CORS(application)
application.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
#application.route("/test-upload", methods=["POST"])
def test_upload():
file = request.files['file']
filename = secure_filename(file.filename)
file.save(os.path.join(application.config['UPLOAD_FOLDER'], filename))
# return redirect(url_for('download_file', name=filename))
return str(os.path.join(application.config['UPLOAD_FOLDER'], filename))
If I remove the file.save() line, then the function won't return errors. And as you can see I'm returning the path, so that will return the correct path. But, I get a 500 bad request from the server when I try and save the file.
I'm not sure what I'm missing here.
I can locally try this using postman, and this works fine. The file saves to the right location. I do have file size for EC2 enabled to be 100MB, but I am testing with a 50 Byte file so size can't be the issue. I know the file is being uploaded to the EC2 web server for sure.
I am having a web application running locally on my laptop. Below is the python code snippet that takes the video from the HTML and uploads it to Cloud Storage:
#app.route('/' , methods=['POST', 'GET'])
def index():
if request.method == 'POST':
video = request.form['video']
video2 = request.form['video2']
if not video:
flash('please upload your answer first')
if not video2:
flash('please upload your answer first')
else:
#store video to GCS
video_list = [video,video2]
for i in range(len(video_list)):
upload_video_to_gcs(video_list[i],i)
def upload_video_to_gcs(video,video_num):
# Setting credentials using the downloaded JSON file
client = storage.Client.from_service_account_json(json_credentials_path='sa-credentials.json')
# Creating bucket object
bucket = client.get_bucket('bucket_name')
# Name of the destination file in the bucket
gcs_file_name = "".join(("video","_",str(video_num)))
object_name_in_gcs_bucket = bucket.blob(gcs_file_name)
object_name_in_gcs_bucket.upload_from_filename(video)
return 1
It is working fine when running it locally on my laptop, in which the video is located in the same folder as the python file.
However, when I deployed this web application on GCP Cloud Run(the video is no more existing in the same folder of the python file), I am getting the below error:
FileNotFoundError: [Errno 2] No such file or directory: 'sample_video.mp4'
Do you have any idea how to upload the video(existing anywhere on my laptop) through the web service hosted on Cloud Run on GCP.
The Python FileNotFoundError: [Errno 2] No such file or directory error is often raised by the os library. This error tells you that you are trying to access a file or folder that does not exist. To fix this error, check that you are referring to the right file or folder in your program.
Here is a similar case that was fixed by providing the correct path.
Latest update: The problem had indeed to do with permission and user groups, today I learned why we do not simply use root for everything. Thanks to Jakub P. for reminding me to look into the apache error logs and thanks to domoarrigato for providing helpful insight and solutions.
What's up StackOverflow.
I followed the How To Deploy a Flask Application on an Ubuntu VPS tutorial provided by DigitalOcean, and got my application to successfully print out Hello, I love Digital Ocean! when being reached externally by making a GET request to my public server IP.
All good right? Not really.
After that, I edit the tutorial script and write a custom flask application, I test the script in my personal development environment and it runs without issue, I also test it on the DigitalOcean server by having it deploy on localhost and making another GET request.
All works as expected until I try to access it from my public DigitalOcean IP, now suddenly I am presented with a 500 Internal Server Error.
What is causing this issue, and what is the correct way to debug in this case?
What have I tried?
Setting app.debug = True gives the same 500 Internal Server Error without a debug report.
Running the application on the localhost of my desktop pc and DigitalOcean server gives no error, the script executes as expected.
The tutorial code runs and executed fine, and making a GET request to the Digital Ocean public IP returns the expected response.
I can switch between the tutorial application and my own application and clearly see that I am only getting the error wit my custom application. However the custom application still presents no issues running on localhost.
My code
from flask import Flask, request, redirect
from netaddr import IPNetwork
import os
import time
app = Flask(__name__)
APP_ROOT = os.path.dirname(os.path.abspath(__file__))
# Custom directories
MODULES = os.path.join(APP_ROOT, 'modules')
LOG = os.path.join(APP_ROOT, 'log')
def check_blacklist(ip_adress):
ipv4 = [item.strip() for item in open(MODULES + '//ipv4.txt').readlines()]
ipv6 = [item.strip() for item in open(MODULES + '//ipv6.txt').readlines()]
for item in ipv4 + ipv6:
if ip_adress in IPNetwork(item):
return True
else:
pass
return False
#app.route('/')
def hello():
ip_adress = request.environ['REMOTE_ADDR']
log_file = open(LOG + '//captains_log.txt', 'a')
with log_file as f:
if check_blacklist(ip_adress):
f.write('[ {}: {} ][ FaceBook ] - {} .\n'
.format(time.strftime("%d/%m/%Y"), time.strftime("%H:%M:%S"), request.environ))
return 'Facebook'
else:
f.write('[ {}: {} ][ Normal User ] - {} .\n'
.format(time.strftime("%d/%m/%Y"), time.strftime("%H:%M:%S"), request.environ))
return 'Normal Users'
if __name__ == '__main__':
app.debug = True
app.run()
The tutorial code:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello, I love Digital Ocean!"
if __name__ == "__main__":
app.run()
Seems like the following line could be a problem:
log_file = open(LOG + '//captains_log.txt', 'a')
if the path its looking for is: '/var/www/flaskapp/flaskapp/log//captains_log.txt'
that would make sense that an exception is thrown there. Possible that the file is in a different place, on the server, or a / needs to be removed - make sure the open command will find the correct file.
If captains_log.txt is outside the flask app directory, you can copy it in and chown it. if the txt file needs to be outside the directory then you'll have to add the user to the appropriate group, or open up permissions on the actual directory.
chown command should be:
sudo chown www:www /var/www/flaskapp/flaskapp/log/captains_log.txt
and it might be smart to run:
sudo chown -r www:www /var/www
From the client side I am sending an image via post from form enctype=multipart/form-data, and on the server side I am saving it to a directory. All of this works locally on my computer and running flask directly with python app.py.
Here is my reference for setting up file saving:
http://flask.pocoo.org/docs/patterns/fileuploads/
On the actual production server, I am running it with Apache and mod_wsgi, which I set up according to this website:
http://flask.pocoo.org/docs/deploying/mod_wsgi/
For directory permissions I have triedchown -R 777 and chown -R www-data:www-data where the relevant Apache code for users looks like this: WSGIDaemonProcess app user=www-data group=www-data threads=5.
However, after all of this I am still not able to get the file to save. I just get a 500 HTTP error back at the point where it tries to save the file.
Here is the relevant Flask code:
UPLOAD_FOLDER = '/images/'
app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
#app.route('/upload_ocr_images', methods=['GET', 'POST'])
def upload_images():
if request.method == 'POST':
files = request.files.getlist("images[]")
for file in files:
if allowed_file(file.filename):
filename = secure_filename(file.filename)
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('home'))
At this point I am wondering if there is something I need to be setting on the Apache side of things.
Youre using /uploads as your path.
That means you're trying to upload to a directory named /uploads at root level of your filesystem.
This is usually wrong and normally it's the error.
If you've the uploads folder under your flask application file structure, then you should create the path using app.root_path which holds the absolute application path.
Something like
file.save(os.path.join(app.root_path, '/uploads', filename))