I want to create a file from within a python script that is executable.
import os
import stat
os.chmod('somefile', stat.S_IEXEC)
it appears os.chmod doesn't 'add' permissions the way unix chmod does. With the last line commented out, the file has the filemode -rw-r--r--, with it not commented out, the file mode is ---x------. How can I just add the u+x flag while keeping the rest of the modes intact?
Use os.stat() to get the current permissions, use | to OR the bits together, and use os.chmod() to set the updated permissions.
Example:
import os
import stat
st = os.stat('somefile')
os.chmod('somefile', st.st_mode | stat.S_IEXEC)
For tools that generate executable files (e.g. scripts), the following code might be helpful:
def make_executable(path):
mode = os.stat(path).st_mode
mode |= (mode & 0o444) >> 2 # copy R bits to X
os.chmod(path, mode)
This makes it (more or less) respect the umask that was in effect when the file was created: Executable is only set for those that can read.
Usage:
path = 'foo.sh'
with open(path, 'w') as f: # umask in effect when file is created
f.write('#!/bin/sh\n')
f.write('echo "hello world"\n')
make_executable(path)
If you're using Python 3.4+, you can use the standard library's convenient pathlib.
Its Path class has built-in chmod and stat methods.
from pathlib import Path
import stat
f = Path("/path/to/file.txt")
f.chmod(f.stat().st_mode | stat.S_IEXEC)
If you know the permissions you want then the following example may be the way to keep it simple.
Python 2:
os.chmod("/somedir/somefile", 0775)
Python 3:
os.chmod("/somedir/somefile", 0o775)
Compatible with either (octal conversion):
os.chmod("/somedir/somefile", 509)
reference permissions examples
Respect umask like chmod +x
man chmod says that if augo is not given as in:
chmod +x mypath
then a is used but with umask:
A combination of the letters ugoa controls which users' access to the file will be changed: the user who owns it (u), other users in the file's group (g), other users not in the file's group (o), or all users (a). If none of these are given, the effect is as if (a) were given, but bits that are set in the umask are not affected.
The goal of this is so that you don't accidentally give too many permissions. umask determines the default permissions of a new file, e.g. with a umask of 0077, touch newfile.txt produces permissions rw for the current user because the 77 would exclude group and other (x is not given by default by touch anyways though). And chmod +x would similarly only add +x for user, ignoring group and other due to the 0011 part of the mask: you would need chmod o+x, chmod g+x, chmod go+x or chmod a+x to force them to be set.
Here is a version that simulates that behavior exactly:
#!/usr/bin/env python3
import os
import stat
def get_umask():
umask = os.umask(0)
os.umask(umask)
return umask
def chmod_plus_x(path):
os.chmod(
path,
os.stat(path).st_mode |
(
(
stat.S_IXUSR |
stat.S_IXGRP |
stat.S_IXOTH
)
& ~get_umask()
)
)
chmod_plus_x('.gitignore')
See also: How can I get the default file permissions in Python?
Tested in Ubuntu 16.04, Python 3.5.2.
You can also do this
>>> import os
>>> st = os.stat("hello.txt")
Current listing of file
$ ls -l hello.txt
-rw-r--r-- 1 morrison staff 17 Jan 13 2014 hello.txt
Now do this.
>>> os.chmod("hello.txt", st.st_mode | 0o111)
and you will see this in the terminal.
ls -l hello.txt
-rwxr-xr-x 1 morrison staff 17 Jan 13 2014 hello.txt
You can bitwise or with 0o111 to make all executable, 0o222 to make all writable, and 0o444 to make all readable.
we can directly call chmod +x command in python using os.system()
import os
os.system("chmod +x somefile")
This turns on all executable bits for a file:
os.chmod("path", os.stat("path").st_mode | 0o111)
In python3:
import os
os.chmod("somefile", 0o664)
Remember to add the 0o prefix since permissions are set as an octal integer, and Python automatically treats any integer with a leading zero as octal. Otherwise, you are passing os.chmod("somefile", 1230) indeed, which is octal of 664.
Related
I am trying to make a directory that has the "set group ID on execution" set
in Python
In the shell:
$ mkdir --mode 2775 mail
$ ls -ld mail
drwxrwsr-x 1 john john 0 May 3 08:34 mail
^
In Python I am using pathlib.Path.mkdir, and it seems there some bits for mode are not taken into account,
even when the umask is cleared:
from pathlib import Path
import os
os.umask(0)
md = Path('mail')
md.mkdir(mode=0o2775)
drwxrwxr-x 1 john john 0 May 3 08:35 mail
^
Is there some cut-off at nine bits that is not documented
Ultimately, on non-Windows systems, Path.mkdir calls os.mkdir which in turn invokes the C system call mkdirat. That call's behavior is defined in terms of mkdir, which, on Linux, documents the mode with:
The argument mode specifies the permissions to use. It is modified by the process's umask in the usual way: the permissions of the created directory are (mode & ~umask & 0777)
Note that the permissions explicitly mask off all bits above 0777; this is just how mkdir system calls behave on some OSes. You'll just have to chmod it after the fact.
pathlib's mkdir essentially just calls os.mkdir(). The documentation there includes the section:
On some systems, mode is ignored. Where it is used, the current umask value is first masked out. If bits other than the last 9 (i.e. the last 3 digits of the octal representation of the mode) are set, their meaning is platform-dependent. On some platforms, they are ignored and you should call chmod() explicitly to set them.
So only the last 9 bits are guaranteed to be used.
You'll have to do (no need to set umask):
md.mkdir()
md.chmod(0o2775)
It would be better if pathlib would not silently ignore extra bits, but actually threw a ValueError.
I have a volume attached in kubernetes with path /var/www/aaa/tmp.
That volume was created using path.mkdir() and currently have 755 permissions.
It was created with code path.mkdir(parents=True, exist_ok=True) initially.
I'm trying to update its permissions without deleting the existing path.
I'm using path.mkdir(parents=True, exist_ok=True, mode=0o777). I'm still facing issues related to permissions and getting 502 Bad gateway for the flask app that is creating the above directories.
Does the path.mkdir(parents=True, exist_ok=True, mode=0o777) updates the path permissions if it already exists and have 755 permissions? Or will it ignore it completely as we've mentioned exists_ok=True ? I don't see the permissions getting updated for the path.
Should I be deleting the path completely and re-running the path.mkdir..... with mode=0o777 which creates new directories and set permissions?
Edit 1:
I've tried using os.chmod() on the path. But it's throwing PermissionError.
Here's the code snippet.
path.mkdir(parents=True, exist_ok=True)
os.chmod(path, mode=0o777)
Error:
File "./app/init.py", line 79, in create_prediction_app
create_directories(app) File "./app/init.py", line 36, in create_directories
os.chmod(path, mode=0o777) PermissionError: [Errno 1] Operation not permitted: '/var/www/aaa/tmp' unable to load app 0
(mountpoint='') (callable not found or import error) * no app
loaded. GAME OVER *
If the path is already existing, you should use the os.chmod(path, mode) instead of deleting/re-creating.
For example:
import os
os.chmod("/var/www/aaa/tmp", 0o777)
Furthermore, the chmod can get the permission from stat module.
stat.S_ISUID − Set user ID on execution.
stat.S_ISGID − Set group ID on execution.
stat.S_ENFMT − Record locking enforced.
stat.S_ISVTX − Save text image after execution.
stat.S_IREAD − Read by owner.
stat.S_IWRITE − Write by owner.
stat.S_IEXEC − Execute by owner.
stat.S_IRWXU − Read, write, and execute by owner.
stat.S_IRUSR − Read by owner.
stat.S_IWUSR − Write by owner.
stat.S_IXUSR − Execute by owner.
stat.S_IRWXG − Read, write, and execute by group.
stat.S_IRGRP − Read by group.
stat.S_IWGRP − Write by group.
stat.S_IXGRP − Execute by group.
stat.S_IRWXO − Read, write, and execute by others.
stat.S_IROTH − Read by others.
stat.S_IWOTH − Write by others.
stat.S_IXOTH − Execute by others.
For example:
import os
import stat
# Set a file write by others.
os.chmod("/var/www/aaa/tmp", stat.S_IWOTH)
You can set more permission with the bitwise operator.
For example:
import os
import stat
os.chmod(
'/var/www/aaa/tmp',
stat.S_IRUSR |
stat.S_IROTH |
stat.S_IRGRP
)
Complete testing:
>>> touch test_perm.sh
>>> ll test_perm.sh
-rw-rw-r-- test_perm.sh
>>> python -c "import os; os.chmod('test_perm.sh', 0755)"
>>> ll test_perm.sh
-rwxr-xr-x test_perm.sh
EDIT:
If you get PermissionError: [Errno 1] Operation not permitted:... exception when you want to change the permission with os.chmod, you should try the following code part to solve it. It's important to run the script with admin rights (with sudo in Linux environment).
Code:
from getpwnam import pwd
from getgrnam import grp
import os
uid = getpwnam("USERNAME")[2]
gid = grp.getgrnam("GROUPNAME")[2]
os.chown("/var/www/aaa/tmp", uid, gid)
Based on the official chown documentation:
os.chown(path, uid, gid, *, dir_fd=None, follow_symlinks=True)
Change the owner and group id of path to the numeric uid and gid. To leave one of the ids unchanged, set it to -1.
This function can support specifying a file descriptor, paths relative to directory descriptors and not following symlinks.
See shutil.chown() for a higher-level function that accepts names in addition to numeric ids.
Availability: Unix.
New in version 3.3: Added support for specifying path as an open file descriptor, and the dir_fd and follow_symlinks arguments.
Changed in version 3.6: Supports a path-like object.
Link to documentation: https://docs.python.org/3/library/os.html#os.chown
milanbalazs' answer is very nice, good explanation, but will only chmod the last directory in the path. In my case this was not good enough, because I needed to ensure every subfolder all the way down the last one was also chmodded.
I couldn't find any simple way to do this with existing bash tools, the only one close to what I needed with was chmod -R but I also didn't want to interfere without anything other than specifically each folder specified in the path.
To clarify, in the example, we have the path /var/www/aaa/tmp. I might want to start from /var/www and ensure that both /var/www/aaa and /var/www/aaa/tmp have their permissions set to whatever I want.
So I wrote the following script to solve this use case;
import os
import argparse
def base8(val):
return int(val, base=8)
def path(val):
assert os.path.exists(val)
return val
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('start_dir', type=path, help='starting point for chmod to check for subdirectories')
parser.add_argument('end_dir', help='last directory to be chmodded, relative to start_dir')
parser.add_argument('permission', type=base8, help='permission to set on each directory (must be base 8 format e.g. "0o750")')
args = parser.parse_args()
return args
def main():
args = parse_args()
directories = os.path.split(args.end_dir)
current_dir = args.start_dir
for directory in directories:
current_dir = os.path.join(current_dir, directory)
os.chmod(current_dir, args.permission)
if __name__ == '__main__':
main()
Hopefully someone finds this useful one day :)
I want to create a version of cd which will strip off a filename from a directory structure and then cd to that directory.
So for example if I put in the (tc)shell
cd /net/homes/me/myfile.jpg
it will strip off 'myfile.jpg' and cd to the directory structure. I tried this is my .cshrc:-
alias ccd '/net/homes/me/scripts/getDir.py'
Then my getDir.py file reads as:-
#! /usr/bin/python
import sys
import os
def get_dir():
the_dir = sys.argv[1]
dir_split = the_dir.split("/")
dir_count = len(the_dir.split("/"))
file_count = len(dir_split[dir_count-1])
only_dirs = the_dir[:-file_count]
#print only_dirs
os.chdir(only_dirs)
get_dir()
This strips off the filename part of the dir structure fine (I can tell that from the print statement) but the chdir command doesn't seem to work.
Thanks!
There's a standard binary called dirname which does this for you, so you can just use...
alias ccd 'cd `dirname \!:1`'
This works, can you explain the syntax?
Well, the dirname \!:1 part means to run the dirname program with the first argument passed to the aliased command, and the backticks substitute the output from that program into the cd command.
chdir doesn't change the state of the calling shell.
You need to change your alias to be something like this ( not sure of the exact tcsh syntax ):
alias ccd 'cd `getDir.py`'
Then your getDir script does nothing but print the correct path string ( no chdir needed )
In Python (tried this in 2.7 and below) it looks like a file created using tempfile.NamedTemporaryFile doesn't seem to obey the umask directive:
import os, tempfile
os.umask(022)
f1 = open ("goodfile", "w")
f2 = tempfile.NamedTemporaryFile(dir='.')
f2.name
Out[33]: '/Users/foo/tmp4zK9Fe'
ls -l
-rw------- 1 foo foo 0 May 10 13:29 /Users/foo/tmp4zK9Fe
-rw-r--r-- 1 foo foo 0 May 10 13:28 /Users/foo/goodfile
Any idea why NamedTemporaryFile won't pick up the umask? Is there any way to do this during file creation?
I can always workaround this with os.chmod(), but I was hoping for something that did the right thing during file creation.
This is a security feature. The NamedTemporaryFile is always created with mode 0600, hardcoded at tempfile.py, line 235, because it is private to your process until you open it up with chmod. There is no constructor argument to change this behavior.
In case it might help someone, I wanted to do more or less the same thing, here is the code I have used:
import os
from tempfile import NamedTemporaryFile
def UmaskNamedTemporaryFile(*args, **kargs):
fdesc = NamedTemporaryFile(*args, **kargs)
# we need to set umask to get its current value. As noted
# by Florian Brucker (comment), this is a potential security
# issue, as it affects all the threads. Considering that it is
# less a problem to create a file with permissions 000 than 666,
# we use 666 as the umask temporary value.
umask = os.umask(0o666)
os.umask(umask)
os.chmod(fdesc.name, 0o666 & ~umask)
return fdesc
I'm trying to implement my own version of the 'cd' command that presents the user with a list of hard-coded directories to choose from, and the user has to enter a number corresponding to an entry in the list. The program, named my_cd.py for now, should then effectively 'cd' the user to the chosen directory. Example of how this should work:
/some/directory
$ my_cd.py
1) ~
2) /bin/
3) /usr
Enter menu selection, or q to quit: 2
/bin
$
Currently, I'm trying to 'cd' using os.chdir('dir'). However, this doesn't work, probably because my_cd.py is kicked off in its own child process. I tried wrapping the call to my_cd.py in a sourced bash script named my_cd.sh:
#! /bin/bash
function my_cd() {
/path/to/my_cd.py
}
/some/directory
$ . my_cd.sh
$ my_cd
... shows list of dirs, but doesn't 'cd' in the interactive shell
Any ideas on how I can get this to work? Is it possible to change my interactive shell's current directory from a python script?
Change your sourced bash code to:
#! /bin/bash
function my_cd() {
cd `/path/to/my_cd.py`
}
and your Python code to do all of its cosmetic output (messages to the users, menus, etc) on sys.stderr, and, at the end, instead of os.chdir, just print (to sys.stdout) the path to which the directory should be changed.
my_cd.py:
#!/usr/bin/env python
import sys
dirs = ['/usr/bin', '/bin', '~']
for n, dir in enumerate(dirs):
sys.stderr.write('%d) %s\n' % (n+1, dir))
sys.stderr.write('Choice: ')
n = int(raw_input())
print dirs[n-1]
Usage:
nosklo:/tmp$ alias mcd="cd \$(/path/to/my_cd.py)"
nosklo:/tmp$ mcd
1) /usr/bin
2) /bin
3) ~
Choice: 1
nosklo:/usr/bin$
This can't be done. Changes to the working directory are not visible to parent processes. At best you could have the Python script print the directory to change to, then have the sourced script actually change to that directory.
For what its worth, since this question is also tagged "bash", here is a simple bash-only solution:
$ cat select_cd
#!/bin/bash
PS3="Number: "
dir_choices="/home/klittle /local_home/oracle"
select CHOICE in $dir_choices; do
break
done
[[ "$CHOICE" != "" ]] && eval 'cd '$CHOICE
Now, this script must be source'd, not executed:
$ pwd
/home/klittle/bin
$ source select_cd
1) /home/klittle
2) /local_home/oracle
Number: 2
$ pwd
/local_home/oracle
So,
$ alias mycd='source /home/klittle/bin/select_cd'
$ mycd
1) /home/klittle
2) /local_home/oracle
Number:
To solve your case, you could have the command the user runs be an alias that sources a bash script, which does the dir selection first, then dives into a python program after the cd has been done.
Contrary to what was said, you can do this by replacing the process image, twice.
In bash, replace your my_cd function with:
function my_cd() {
exec /path/to/my_cd.py "$BASH" "$0"
}
Then your python script has to finish with:
os.execl(sys.argv[1], sys.argv[2])
Remember to import os, sys at the beginning of the script.
But note that this is borderline hack. Your shell dies, replacing itself with the python script, running in the same process. The python script makes changes to the environment and replaces itself with the shell, back again, still in the same process. This means that if you have some other local unsaved and unexported data or environment in the previous shell session, it will not persist to the new one. It also means that rc and profile scripts will run again (not usually a problem).