Develop Reference

PermissionError: [Errno 13] Permission denied: '/manage.py'

I am trying to run the following command in docker-composer, to start project with django-admin:
docker-compose run app sh -c "django-admin startproject app ."
This produces the error:
Traceback (most recent call last):
File "/usr/local/bin/django-admin", line 10, in <module>
sys.exit(execute_from_command_line())
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/startproject.py", line 20, in handle
super().handle('project', project_name, target, **options)
File "/usr/local/lib/python3.7/site-packages/django/core/management/templates.py", line 155, in handle
with open(new_path, 'w', encoding='utf-8') as new_file:
PermissionError: [Errno 13] Permission denied: '/manage.py'
The Dockerfile is as follows:
FROM python:3.7-alpine
MAINTAINER anubrij chandra
ENV PYTHONUNBUFFERED 1
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN mkdir /app
COPY ./app /app
RUN adduser -D dockuser
USER dockuser
My docker-compose.yml:
version: "3"
services:
app:
build:
context: .
ports:
- "8000:8000"
volumes:
- ./app:/app
command: >
sh -c "python manage.py runserver 0.0.0.0:8000"
I applied the solution suggested in this Stack Overflow thread, but it didn't work.
I'm using Ubuntu 18.04.

ubuntu 21.04
I got here searching for PermissionError: [Errno 13] Permission denied: so i'll just leave this here.
note: the below answer doesn't work for multi user systems ... see this answer instead for another possible solution
If you want to set it and forget it for 1 user, your own user ... here's what I have on my dev machine.
I didn't own the unix socket, so I chowned it. ( this got it working straight away )
sudo chown $(whoami):$(whoami) /var/run/docker.sock
Another, more permanent solution for your dev environment, is to modify the user ownership of the unix socket creation. This will give your user the ownership, so it'll stick between restarts:
sudo nano /etc/systemd/system/sockets.target.wants/docker.socket
docker.socket:
[Unit]
Description=Docker Socket for the API
[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=YOUR_USERNAME_HERE
SocketGroup=docker
[Install]
WantedBy=sockets.target
Again another less hacky solution: https://stackoverflow.com/a/70124863/2026508

In your dockerfile, you are pointing to a new user dockuser.
RUN adduser -D dockuser
USER dockuser
Hence your container will start with user dockuser which does not seem to have proper permissions to run /manage.py.
You can either
remove the above mentioned lines where you create and point to
dockuser.
OR
provide appropriate permissions to user dockuser using chown and chmod commands in your dockerfile for /manage.py file.
I have answered such similar question here.

I had the same issue and solved it by adding my user to the docker group:
$ sudo usermod -aG docker $(whoami)

add this to your Dockerfile after RUN adduser -D dockuser:
RUN chown dockuser:dockuser -R /app/
and why you COPYthe files if you already mount them ?
if you want to keep the mount , you need to add rw persmission on the folder on the HOST system not on the Container

If you're on mac this might work for you.
After 4 days of troubleshooting this error (and other strange errors) I found out that I needed to fix dockers permissions in my file system. To do this go to:
System Preferences -> Security & Privacy -> Privacy tab -> Full Disk Access (on the left, somewhere in the list) -> Click on the + -> Docker application
Terribly frustrating problem to debug, hope it helps.

The Dockerfile that I was using was in the shared directory of my virtual machine (VirtualBox). Therefore, my issue was related to the default permission of this directory (UID=root, GID=vboxdf). I had to change it to my current user to properly run my container.
See https://superuser.com/a/640028/1655184 for a description on how to change the owner.

For me it was a WSL2 / Docker-Desktop setup issue on Windows.
What was missing was to explicitly enable the WSL distro: Docker Desktp Settings > Resources > WSL Integration > enable your distro
Then re-open WSL shell.
Before I could not even do docker run hello-world but also had a very similar error message as the OP when running docker-compose.

adding path to app like '/home/user/app:/app' in docker-compose instead of just '.:/app' resolved my problem

For ubuntu, just add sudo to your command and run as root to get the permissions.
sudo docker-compose run app sh -c "django-admin startproject app ."

ERROR: CANT_REREAD: The directory named as part of the path /home/app/logs/celery.log does not exist

I'm following a tutorial on how to use Celery on my Django production server.
When I get to the bit where it says:
Now reread the configuration and add the new process:
sudo supervisorctl reread
sudo supervisorctl update
When I perform sudo supervisorctl reread in my server (Ubuntu 16.04) terminal, it returns this:
ERROR: CANT_REREAD:
The directory named as part of the path /home/app/logs/celery.log does not exist.
in section 'app-celery' (file: '/etc/supervisor/conf.d/app-celery.conf')
I've done all of the instructions prior to this including installing supervisor as well as creating a file named mysite-celery.conf (app-celery.conf) in the folder: /etc/supervisor/conf.d
If you're curious my app-celery.conf file looks like this:
[program:app-celery]
command=/home/app/bin/celery worker -A draft1 --loglevel=INFO
directory=/home/app/draft1
user=zorgan
numprocs=1
stdout_logfile=/home/app/logs/celery.log
stderr_logfile=/home/app/logs/celery.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
stopasgroup=true
; Set Celery priority higher than default (999)
; so, if rabbitmq is supervised, it will start first.
priority=1000
Any idea what the problem is?

Somehow supervisor is not able to create the folder - /home/app/logs/.
You can create it manually using mkdir and restart the supervisor service
mkdir /home/app/logs
sudo service supervisor restart

I added my username to the superisord.conf file under the [unix_http_server] section like so:
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0770 ; sockef file mode (default 0700)
chown=appuser:supervisor ;(username:group)
This seemed to work- time will tell if it continues working after I manage solve the rest of the supervisor issues.

Cannot read environment variable in Python on Ubuntu

I have set up an environment variable which I execute locally using a .sh file:
.sh file:
#!/bin/sh
echo "environment variables"
export BROKER="amqp://admin:password#11.11.11.11:4672//"
Locally inside a virtual environment I can now read this in Python using:
BROKER = os.environ['BROKER']
However, on my production server (Ubuntu). I run the same file chmod +x name_of_file.sh and source settings.sh and can see the variable using printenv, but Python gives the error KeyError: 'BROKER' Why?
This only happens on my production machine despite the fact I can see the variable using printenv. Note my production machines does not use virtualenv.
If I run the python shell on Ubuntu and do os.environ['BROKER'] it prints out the correct value. So I have not idea what the app file does not find it.
This is the task that gets run which cannot find the variable (supervisor task)
[program:celery]
directory = /srv/app_test/
command=celery -A tasks worker -l info
stdout_logfile = /var/log/celeryd_.log
autostart=true
autorestart=true
startsecs=5
stopwaitsecs = 600
killasgroup=true
priority=998
user=ubuntu
Celery Config (which does not find the variable when executed under supervisor:
from kombu import Exchange, Queue
import os
# Celery Settings
BROKER = os.environ['BROKER']
When I restart supervisor it gives the key error.

The environment variables from your shell will not be visible within supervisor tasks.
You need to use the environment setting in your supervisor config:
[program:celery]
...
environment=BROKER="amqp://admin:password#11.11.11.11:4672//"
This requires supervisor 3.0+.

Calling to restart a "service" in Linux from Python script not working from within crontab entry

I have the following python test script (extracted from a much bigger .py file) which I'm trying to get working on an EC2 instance (haproxy OpsWorks instance).
There seems to be a major difference between the permissions applied when running as a cron entry versus running as a python script from the shell.
The closest I've found in any other posts is here but that's relating to shell scripts and, given that I'm running python OK, the paths should ( I hope ) be sufficiently configured for the cronjob.
Environment:
- Python version: 2.6.8
- Environment: OpsWorks HAProxy instance, on EC2.
- Running as user: root.
Script
import subprocess
import boto
import logging
if __name__ == '__main__':
logging.getLogger().setLevel(boto.logging.INFO)
command = ['service', 'haproxy', 'reload'];
logging.info('Executing: %s' % command)
#shell=FALSE for sudo to work.
subprocess.call(command, shell=False)
I have alternatively tried calling the subprocess command using the following to no avail.
# subprocess.call("%s %s %s %s" % ('sudo', 'service', 'haproxy', 'reload'))
Output when run from the command line:
[root#lb1 ~]# python tester.py
INFO:root:Executing: ['service', 'haproxy', 'reload']
Reloading haproxy:
[root#lb1 ~]#
Crontab entry:
[root#lb1 ~]# crontab -l
*/1 * * * * python ~/tester.py > ~/testlog 2>&1
Output when run as a crontab command
[root#lb1 ~]# cat testlog
INFO:root:Executing: ['service', 'haproxy', 'reload']
Traceback (most recent call last):
File "/root/tester.py", line 13, in <module>
subprocess.call(command, shell=False)
File "/usr/lib64/python2.6/subprocess.py", line 478, in call
p = Popen(*popenargs, **kwargs)
File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
errread, errwrite)
File "/usr/lib64/python2.6/subprocess.py", line 1228, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I'm confused as to why on earth it's throwing the "No Such File Or Directory" error!
Can anyone provide any pointers? Incidentally this is part of a much bigger EC2 integration piece - hence the boto import [it's a nice way to get logging in this instance]. It's running as root because, from browsing SO, it's bad practice to have a cron entry for a user account running with a sudo command.
Cheers,

In subprocess set shell=True
subprocess.call(command, shell=True)
I think your problem will be solved but sudo problem may be arised.

It turns out that it is indeed path related.
Path Information for Cronjobs
Commands in submodules
To solve it I provided the following entries via crontab -e
[root#lb1 ~]# crontab -l
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
Now my service restarts as expected. Yay!

how to use celeryd.conf file provided in production

This is the file provided at https://github.com/ask/django-celery/blob/master/contrib/supervisord/celeryd.conf . How can I run this conf file ?
I am running my django app using gunicorn
; =======================================
; celeryd supervisor example for Django
; =======================================
[program:celery]
command=/path/to/project/manage.py celeryd --loglevel=INFO
directory=/path/to/project
user=nobody
numprocs=1
stdout_logfile=/var/log/celeryd.log
stderr_logfile=/var/log/celeryd.log
autostart=true
autorestart=true
startsecs=10
; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
; if rabbitmq is supervised, set its priority higher
; so it starts first
priority=998
Thanks

That configuration file can't be run on its own; it is for use with supervisord.
You need to install supervisord (if you have pip, use pip install supervisor), create a configuration file for it using sudo echo_supervisord_conf > /etc/supervisord.conf and then copy paste the contents of the file above into your supervisord configuration file as described in the supervisord documentation under Adding a program.
So basically run the following at your shell:
pip install supervisor
sudo echo_supervisord_conf > /etc/supervisord.conf
sudo wget -O - -o /dev/null https://raw.github.com/ask/django-celery/master/contrib/supervisord/celeryd.conf >> /etc/supervisor.conf
sudo $EDITOR /etc/supervisor.conf
and edit the config file to your heart's content.