I'm running a Flask application on Apache2 server on Ubuntu. The application would take input from a form and save it to a text file. The file exists only for the moment when it's uploaded to S3. After that it's deleted.:
foodforthought = request.form['txtfield']
with open("filetos3.txt", "w") as file:
file.write(foodforthought)
file.close()
s3.Bucket("bucketname").upload_file(Filename = "filetos3.txt", Key = usr+"-"+str(datetime.now()))
os.remove("filetos3.txt")
but the app doesn't have permission to create the file:
[Errno 13] Permission denied: 'filetos3.txt'
I already tried to give permissions to the folder where the app is located with:
sudo chmod -R 777 /var/www/webApp/webApp
but it doesn't work
My guess is that the application is run from a different location. What output do you get from this:
import os
print(os.getcwd())
It's for that directory you need to set permissions. Better yet, use an absolute path. Since the file is temporary, use tempfile as detailed here.
foodforthought = request.form['txtfield']
with tempfile.NamedTemporaryFile() as fd:
fd.write(foodforthought)
fd.flush()
# Name of file is in the .name attribute.
s3.Bucket("bucketname").upload_file(Filename = fd.name, Key = usr+"-"+str(datetime.now()))
# The file is automatically deleted when closed, which is when the leaving the context manager.
Some final notes: You don't need to close the file, since you use a context manager. Also, avoid setting 777 recursively. The safest way is to set +wX in order to only set execute bit on directories and write bit on everything. Or better yet, be even more specific.
Related
I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?
For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.
I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)
RELATED: Python multiprocessing: Permission denied
I want to use Python's multiprocessing.Pool
import multiprocessing as mp
pool = mp.Pool(3)
for i in range(num_to_run):
pool.apply_async(popen_wrapper, args=(i,), callback=log_result)
I get OSError
File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line 178, in RLock
return RLock()
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 142, in __init__
SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1)
File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__
sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 13] Permission denied
I read in the related question that it's due to not having r/w to /dev/shm
Besides changing the permission in /dev/shm, is there a way to run as root in the code?
I initially thought you could do something like os.umask() but it didnt work
EDIT (rephrasing the question):
let's say a username A has r/w access to directory A
You are user B and your program needs access to directory A. how do you run a program as user A?
In order from the least dangerous to the most dangerous.
You can try dropping permissions as John Zwinck suggested.
Basically you would start the program with root level permissions,
immediately do what you need to do, and then switch to a non-root
user.
From this StackOverflow.
import os, pwd, grp
def drop_privileges(uid_name='nobody', gid_name='nogroup'):
if os.getuid() != 0:
# We're not root so, like, whatever dude
return
# Get the uid/gid from the name
running_uid = pwd.getpwnam(uid_name).pw_uid
running_gid = grp.getgrnam(gid_name).gr_gid
# Remove group privileges
os.setgroups([])
# Try setting the new uid/gid
os.setgid(running_gid)
os.setuid(running_uid)
# Ensure a very conservative umask
old_umask = os.umask(077)
You could also require the credentials for the root user to be
inputed into the script, and then only use them when they are
required.
subprocess.call("sudo python RunStuffWithElevatedPrivelages.py")
#From here, the main script will continue to run without root permissions
Or if you don't want the script to prompt the user for the password you can do
subprocess.call("echo getRootCredentials() | sudo -S python RunStuffWithElevatedPrivelages.py")
Or you could just run the entire program as a root user -- sudo python myScript.py.
As far as temporarily giving users root permission to /dev/shm only when they run your script, the only thing I could think of was having some script that runs in the background under the root user that can temporarily grant anyone who uses your script root privileges to /dev/shm. This could be done through using setuid to grant such permissions and then after a certain amount of time or if the script ends the privilege is taken away. My only concern would be if there is a way a user who has temporarily been given such permissions might be able to secure more permanent privileges.
i am running a flask project and i am looking for a way to create a directory ABOVE the path from which the current App is running. For example:
dirA -->
dirBinA -->
peter.py
griffin.sh
dirCinA -->
index.py <--------- this is the flask app that's running
tom.css
dick.html
harry.js
dirDinA --> <--------- this directory doesn't exist yet
anotherDir -->
turtle.py
i want to create a new directory anotherDir inside a new directory dirDinA from the flask app that's running in dirCinA/index.py
If I try with os.mkdir("../dirDinA/anotherDir/"), then flask says OSError: [Errno 2] No such file or directory: '../dirDinA/anotherDir'
You can use os.makedirs to create multiple directory levels in a single call:
os.makedirs("../dirDinA/anotherDir")
Assuming that you're running the Python script in the intended directory, you need to create a new directory and put things in it in two steps. For instance, if ../dirDinA does not exist, then
os.mkdir("../dirDinA/a_new_file.py")
produces the "No such file or directory" error (and misleadingly shows you the full path that you want to make, not the part whose non-existence is causing the problem). The following
os.mkdir("../dirDinA")
open("../dirDinA/a_new_file.py", "w").write("test")
does not produce an error.
In order to create a new 2-level-depth directory, you need to create it in TWO steps. For instance, if ../dirDinA doesn't yet exist, then the following command fails.
os.mkdir("../dirDinA/anotherDir")
It produces the OSError: No such file or directory, misleadingly, showing you the FULL path that you are trying to create, instead of highlighting on the ACTUAL part whose non-existence is producing the error.
However, the following 2 step method goes well without any error
os.mkdir("../dirDinA")
os.mkdir("../dirDinA/anotherDir")
Directory ../dirDinA needs to exist before anotherDir can be created inside it
Thanks goes to the answer by #JimPivarski.
I suggest you use os.popen:
os.popen("cd ..; mkdir dirDinA; mkdir ./dirDinA/anotherDir; --code for creating turtle.py--; chmod +x turtle.py; python turtle.py");
Hope it helpful for you.
I am using django views, I create a temp_dir using tempfile.gettempdir().
I write a gzipped text file in there, and then scp the file elsewhere. When these tasks are complete I try to delete the temp_dir.
if os.path.exists( temp_dir ):
shutil.rmtree( temp_dir )
However, occasionally I get this error back:
Operation not permitted: '/tmp/.ICE-unix'
Any ideas what this error means and how to best handle this situation?
tempfile.gettempdir() does not create a temp directory - it returns your system's standand tmp directory. DO NOT DELETE IT! That will blow everybody's temp files away. You can delete the file you created inside the temp dir, or you can create your own temp dir, but leave this one alone.
The value for temp_dir is taken from the OS environment variables, and apparently some other process is also using it to create files. The other file might be in use/locked and that will prevent you from deleting it.
Q: What is /tmp/.ICE-unix ?
A: Its a directory where X-windows session information is saved.
I am no expert but try running the python program or what your using to do this as an administrator then it will most likely allow this process to be done...
This question already has answers here:
Deleting read-only directory in Python
(7 answers)
Closed 3 years ago.
I'm trying to have python delete some directories and I get access errors on them. I think its that the python user account doesn't have rights?
WindowsError: [Error 5] Access is denied: 'path'
is what I get when I run the script.
I've tried
shutil.rmtree
os.remove
os.rmdir
they all return the same error.
We've had issues removing files and directories on Windows, even if we had just copied them, if they were set to 'readonly'. shutil.rmtree() offers you sort of exception handlers to handle this situation. You call it and provide an exception handler like this:
import errno, os, stat, shutil
def handleRemoveReadonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise
shutil.rmtree(filename, ignore_errors=False, onerror=handleRemoveReadonly)
You might want to try that.
I've never used Python, but I would assume it runs as whatever user executes the script.
The scripts have no special user, they just run under the currently logged-in user which executed the script.
Have you tried checking that:
you are trying to delete a valid path? and that
the path has no locked files?
How are you running the script? From an interactive console session? If so, just open up a DOS command window (using cmd) and type 'whoami'. That is who you are running the scripts interactively.
Ok I saw your edits just now...why don't you print the path and check the properties to see if the user account running the scripts has the required privileges?
If whoami does not work on your version of Windows, you may use the environment variables like SET USERNAME and SET DOMAINNAME from your command window.
#ThomasH : another brick to the wall.
On unix systems, you have to ensure that parent directory is writeable too.
Here is another version :
def remove_readonly(func, path, exc):
excvalue = exc[1]
if func in (os.rmdir, os.remove) and excvalue.errno == errno.EACCES:
# ensure parent directory is writeable too
pardir = os.path.abspath(os.path.join(path, os.path.pardir))
if not os.access(pardir, os.W_OK):
os.chmod(pardir, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO)
os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
func(path)
else:
raise
If the script is being run as a scheduled task (which seems likely for a cleanup script), it will probably run as SYSTEM. It's (unwise, but) possible to set permissions on directories so that SYSTEM has no access.
Simple solution after searching for hours is to check first if that folder actually exist!
GIT_DIR="C:/Users/...."
if os.path.exists(GIT_DIR):
shutil.rmtree(GIT_DIR)
This did the trick for me.