my problem is that on local, my code works fine but when I pushed it on the server, the file isn't created.
Here's the code:
def write_binary_file(bfile, location):
"""Write binary file in the location"""
try:
with open(location, "wb+") as img_file:
img_file.write(bfile)
except IOError as err:
Handling error
file_url = os.path.join(settings.BASE_DIR, "dir", "dir", "dir", "dir", "user_img", filename + '.jpeg')
write_binary_file(bfile, file_url):
Difference between local and server:
_local os is windows
_server os is linux
I don't know if this matter or not since I'm using os.path.join and os.path.sep to build the url and getting file url without the first dir.
It worked before in the server but one day, somehow, it didn't work anymore till now
space left on the server: about 3Go
permission on the directory: 775 (rwxrwxr-x)
Well, figured what was the problem, it was nginx config to hide file path in the server
Related
Issue: Unable to save file in directory (/root/Notion/Image) when using Cron schedule
This is what my code is trying to do:
Check email
Download image attachment
Store in a directory - root/Notion/Image
Retrieve file path
The script is working when I run it manually in Google Cloud terminal. The problem is when I try to schedule it on Cron, it's unable to access the folder to save the file locally.
This is the error when the script failed and require permission:
Traceback (most recent call last):
File "Notion/test.py", line 121, in <module>
path = get_attachments(email.message_from_bytes(msg[0][1]))
File "Notion/test.py", line 47, in get_attachments
with open(filePath, 'wb') as f:
PermissionError: [Errno 13] Permission denied: '/root/Notion/Image/3.jpeg'
This is the code to retrieve attachment from email
def get_attachments(msg):
for part in msg.walk():
if part.get_content_maintype()=='multipart':
continue
if part.get('Content-Disposition') is None:
continue
fileName = part.get_filename()
if bool(fileName):
filePath = os.path.join(attachment_dir, fileName)
with open(filePath, 'wb') as f:
f.write(part.get_payload(decode=True))
return str(filePath)
Resolved:
The problem is that I shouldn't use root directory since it requires permission. I've changed it to home directory instead.
attachment_dir = '/home/dev_thomas_yang/folder_name/folder_name'
For people who needs to check their home direction, simply run this script.
from pathlib import Path
home= str(Path.home())
print(home)
Thanks Triplee for the patience to breakdown my issue despite my sloppy ways of presenting it!
The easiest fix hands down is to change the code so it doesn't try to write to /root. Have it write to the invoking user's home directory instead.
Your question doesn't show the relevant parts of the code, but just change attachment_dir so it's not an absolute path. Maybe separately take care of creating the directory if it doesn't already exist.
import pathlib
# ...
attachment_dir = pathlib.Path("cron/whatever/attachments").mkdir(parents=True, exist_ok=True)
# ...
for loop in circumstances:
get_attachments(something)
A better design altogether would be to have get_attachments accept the directory name as a parameter, so you can make this configurable from the code which calls it. Global variables are a nuisance and cause hard-to-debug problems because they hide information which is important for understanding the code, and tricky to change when you try to debug that code and don't know which parts of the code depend on the old value.
I'm trying to upload a file using ftp in python, but I get an error saying:
ftplib.error_perm: 550 Filename invalid
when I run the following code:
ftp = FTP('xxx.xxx.x.xxx', 'MY_FTP', '')
ftp.cwd("/incoming")
file = open('c:\Automation\FTP_Files\MaxErrors1.json', 'rb')
ftp.storbinary('STOR c:\Automation\FTP_Files\MaxErrors1.json', file)
ftp.close()
I've checked that the file exists in the location I specified, does anyone know what might be causing the issue?
The problem is that on the server, the path c:\Automation\FTP_Files\MaxErrors1.json is not valid. Instead try just doing:
ftp.storbinary('STOR MaxErrors1.json', file)
The argument to STOR needs to be the destination file name, not the source path. You should just do ftp.storbinary('STOR MaxErrors1.json', file).
you should upload file without absolute path in ftp server
for example :
import ftplib
session = ftplib.FTP('server.address.com','USERNAME','PASSWORD')
file = open('kitten.jpg','rb') # file to send
session.storbinary('STOR kitten.jpg', file) # send the file
file.close() # close file and FTP
session.quit()
im have FLASK app with
www/FlaskApp/FlaskApp/init.py file with funtion
python file wut next contains
#app.route('/')
def hello():
file = open('myfile.txt', 'w+')
os.mknod("newfile.txt")
return render_template('page2.html')
but if im run site,its return error, in file log write
PermissionError: [Errno 13] Permission denied: 'myfile.txt'
im set permision 777 for all www directories
open FileZilla
right click on www dir, and set 777 permision
Why file dont create?
Not sure if this is an optimal solution, and I don't know enough about Flask as to tell you why the relative path isn't working (I would think that it would write the file where ever your python script was) but you could get it to work by using an environment variable to specify where to store your apps data. For instance:
import os
#app.route('/')
def open_file():
filename = os.path.join(os.environ['FLASK_APP_DATA'], 'myfile.txt')
print (filename)
file = open(filename, 'w+')
file.write("This is a test")
file.close()
Then you could have the environment variable set differently on your dev box and your prod box.
I'm encountering a troublesome problem with my code and I've been unable to figure it out. Basically I am copying files from a local directory on my computer to a Dropbox folder that acts as a project repository for me and some other folks.
I keep hitting an IO Error when executing the shutil.copy line. Errno 2, N osuch file or directory. However the directory and file both exist. When I change the directory to a different/test location (test_dir in my code), the code runs perfectly fine. Any insights would be greatly appreciated.
import os, os.path
import re
import shutil
import sys
#File Location
directory_list = "path where files are located"
#Dropbox base directory:
dropbox = "path to dropbox directory"
test_dir = "path to test directory on my local machine"
sed_files = os.listdir(directory_list)
for i in sed_files:
#print i.split("BBB")[0]
#df
copy_dir = re.sub(r'XXX',r'_',i.split("BBB")[0])
copy_dir = re.sub(r'ZZZ',r'/',copy_dir)
copy_dir = dropbox + copy_dir + "/FIXED/"
if not os.path.exists(copy_dir):
os.makedirs(copy_dir)
shutil.copy(directory_list+i,copy_dir)
#print directory_list+i
#os.rename(copy_dir+i,copy_dir+i.split("BBB")[1])
Traceback error is:
Traceback (most recent call last):
File "copy_SE_files.py", line 25, in <module> shutil.copy(direcotry_list+i,copydir)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 116, in copy copyfile(src,dst)
File "C:\Python27\ArcGIS10.1\lib\shutil.py", line 82, in copyfile with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: 'C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed'
Problem solved thanks to the keen eyes of stack overflow. Modified the line to read:
shutil.copy(directory_list+i,'\\\\?\\'+os.path.abspath(copy_dir))
You're failing because the combined length of the path is greater than Window's MAX_PATH limit. C:/Users/myusername/Dropbox/NASA_HyspIRI_Project/Field_Data/Spectra/CVARS/April2014/LemonTrees/04172014_SE_LemonTreeCanopy/SE_Files/SpectraZZZCVARSZZZApril2014ZZZLemonTreesZZZZ04172014XXXSEXXXLemonTreeCanopyZZZSEXXXFilesBBBCVARS_na_LemonTrees_Bareground1_4deg_Refl_00355.sed is 274 characters long, and without going to some trouble, most Windows file manipulation APIs won't work properly with a path longer than MAX_PATH (which is 260, and one of them is reserved for the NUL terminator).
Assuming Python uses the correct APIs, you can make it work with the extended path prefix, \\?\ (and it may require you to use backslashes rather than forward slashes in your path; I'm not clear on that; read the docs).
The first thing that jumped out at me was this line:
shutil.copy(directory_list+i,copy_dir)
Consider changing it to
shutil.copy(os.path.join(directory_list,i),copy_dir)
IOW, use os.path.join() when concatenating file paths.
One work around is:
try:
shutil.copy(src, dest)
except:
try:
shutil.copy(src, "\\\\?\\" + dest)
#If Long Path as per Maximum Path limitation Windows
except:
self.failed_TC=True
print("Failed to move the script "+os.path.basename(src)+" to "+dest)
I am attempting to use python to connect to a server and upload some files from my local directory to /var/www/html but every time I try to do this I get this error:
Error: ftplib.error_perm: 553 Could not create file.
I have already did a chown and a chmod -R 777 to the path. I am using vsftpd and already set write enabled. Does anyone have any ideas?
Code:
ftp = FTP('ipaddress')
ftp.login(user='user', passwd = 'user')
ftp.cwd('/var/www/html')
for root, dirs, files in os.walk(path):
for fname in files:
full_fname = os.path.join(root, fname)
ftp.storbinary('STOR' + fname, open(full_fname, 'rb'))
I had a similar problem also getting the error 553: Could not create file. What (update: partially) solved it for me was changing this line from:
ftp.storbinary('STOR' + fname, open(full_fname, 'rb'))
to:
ftp.storbinary('STOR ' + '/' + fname, open(full_fname, 'rb'))
Notice that there is a space just after the 'STOR ' and I added a forward slash ('/') just before the filename to indicate that i'd like the file stored in the FTP root directory
UPDATE: [2016-06-03]
Actually this only solved part of the problem. I realized later that it was a permissions problem. The FTP root directory allowed writing by the FTP user but i had manually created folders within this directory using another user thus the new directories did not allow the FTP user to write to these directories.
Possible solutions:
Change the permissions on the directories such that the FTP user is
the owner of these directories, or is at least able to read and
write to them.
Create the directories using the ftp.mkd(dir_name) function, then change directory using the ftp.cwd(dir_name) function and
then use the appropriate STOR function (storlines or storbinary)
to write the file to the current directory.
As far as my understanding goes, the STOR command seems to only take a filename as a parameter (not a file path), that's why you need to make sure you are in the correct 'working directory' before using the STOR function (Remember the space after the STOR command)
ftp.storbinary('STOR ' + fname, open(full_fname, 'rb'))
Does path == '/var/www/html'? That's a local path. You need an FTP path.
The local path /var/www/html is not generally accessible by FTP. When you connect to the FTP server, the file system presented to you begins at, often, your user's home directory /home/user.
Since it sounds like you're running the ftp server (vsftpd) on the remote machine, the simplest solution might be something like:
user#server:~$ ln -s /var/www/html /home/user/html
Then you could call ftp.cwd('html'), and ftp.nlst() to get the remote directory listing, and navigate it from there.
Also, don't forget to put a space character in the 'STOR' string (should be 'STOR ').
Best of luck!
I'm sure at this point you have found a solution, but I just stumbled across this thread while I was looking for a solution. I ended up using the following:
# Handles FTP transfer to server
def upload(ftp, dir, file):
# Test if directory exists. If not, create it
if dir.split('/')[-1] not in ftp.nlst('/'.join(dir.split('/')[:-1])):
print("Creating directory: " + dir)
ftp.mkd(dir)
# Check if file extension is text format
ext = path.splitext(file)[1]
if ext.lower() in (".txt", ".htm", ".html"):
ftp.storlines("STOR " + dir + '/' + file, open(dir + '/' + file, "rb"))
else:
ftp.storbinary("STOR " + dir + '/' + file, open(dir + '/' + file, "rb"), 1024)