Develop Reference

Run the right scripts before deployment elastic beanstalk

I am editing my .ebextensions .config file to run some initialisation commands before deployment. I thought this commands would be run in the same folder of the extracted .zip containing my app. But that's not the case. manage.py is in the root directory of my zip and if I do the commands:
01_collectstatic:
command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"
I get a ERROR: [Instance: i-085e84b9d1df851c9] Command failed on instance. Return code: 2 Output: python: can't open file 'manage.py': [Errno 2] No such file or directory.
I could do command: "python /opt/python/current/app/manage.py collectstatic --noinput" but that would run the manage.py that successfully was deployed previously instead of running the one that is being deployed atm.
I tried to check what was the working directory of the commands ran by the .config by doing command: "pwd" and it seems that pwd is /opt/elasticbeanstalk/eb_infra which doesn't contain my app.
So I probably need to change $PYTHONPATH to contain the right path, but I don't know which path is it.
In this comment the user added the following to his .config file:
option_settings:
aws:elasticbeanstalk:application:environment:
DJANGO_SETTINGS_MODULE: myapp.settings
PYTHONPATH: "./src"
Because his manage.py lives inside the src folder within the root of his zip. In my case I would do PYTHONPATH: "." but it's not working.

AWS support solved the problem. Here's their answer:
When Beanstalk is deploying an application, it keeps your application files in a "staging" directory while the EB Extensions and Hook Scripts are being processed. Once the pre-deploy scripts have finished, the application is then moved to the "production" directory. The issue you are having is related to the "manage.py" file not being in the expected location when your "01_collectstatic" command is being executed.
The staging location for your environment (Python 3.4, Amazon Linux 2017.03) is "/opt/python/ondeck/app".
The EB Extension "commands" section is executed before the staging directory is actually created. To run your script once the staging directory has been created, you should use "container_commands". This section is meant for modifying your application after the application has been extracted, but before it has been deployed to the production directory. It will automatically run your command in your staging directory.
Can you please try implementing the container_command section and see if it helps resolve your problem? The syntax will look similar to this (but please test it before deploying to production):
container_commands:
01_collectstatic:
command: "source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput"

So, the thing to remember about beanstalk is that each of the commands are independent, and you do not maintain state between them. You have two options in this case, put your commands into a shell script that is uploaded in the files section of ebextensions. Or, you can write one line commands that do all stateful activities prefixed to your command of interest.
e.g.,
00_collectstatic:
command: "pushd /path/to/django && source /opt/python/run/venv/bin/activate && python manage.py collectstatic --noinput && popd"

Running python script from crontab?

I need some help running a python script from crontab:
The script looks for subfolders from current path and does something to them, also extracts a zip file located in the same folder of the script into each found subfolder.
When I go with cd /folder/folder then python script.py is all good. But when run it with crontab it runs in users home folder and not where the script is placed.
To overcome this I placed in crontab something like this:
* * * * cd /folder_of_scrpit/ && /python_path/python script.py >> log.txt
and works as needed but feels weird, is there a better way of achieving this?

You can cd in crontab the way you do it. Or you can call os.chdir() in your script. In the latter case you can write the directory in the script or pass it as a command line argument: /python path/python script.py /folder/folder.

Execute a python script as a command

I'm writing a python script which will be placed in a location. I want to execute it just like a command. for ex.
$ find_branch test
where find_branch is a script placed in anywhere in the system.
I would like to know how to achieve this. I can run it on the place where the script is present by chmod u+x on the script and removing the .py from the script

sudo nano /usr/bin/testpyscript
Then inside the script:
#!/usr/bin/python
print("I'm a python script")
Give it x permission:
sudo chmod +x /usr/bin/testpyscript
Now you can use it as a regular command:
bash-4.2$ testpyscript
I'm a python script
It doesn't have to be exactly at /usr/bin, any location that is inside your $PATH will do. Let's say you want it to be located at some folder inside your home directory, you could do something like this:
pwd
/home/brunorb
mkdir somedir
sudo mv /usr/bin/testpyscript somedir/
export PATH=$PATH:/home/brunorb/somedir/
testpyscript # from any folder in the system
I'm a python script

Make sure python has been added to your path and #!/usr/bin/python is located at the top of your script.
Note You could just try adding your script to your /usr/local/bin/ directory and give it the proper permissions.
sudo cp <your script> /usr/local/bin/

You have a number of options on how to achieve this.
Add the location where you put the script to your PATH environment variable, for example in your ~/.bashrc script:
export PATH="${PATH}:/folder/where/you/put/the/script"
Install the script to a location that is already on your path. It does not have to be a system folder like /usr/bin. Many default Bash setups will include ~/bin in your PATH.
Give the full path to your script on the command line:
/folder/where/you/put/the/script/find_branch test
Run the script through Python. This is very similar to option #2:
python /folder/where/you/put/the/script/find_branch test
Create an alias for the script in your environment. In bash you would do something like the following in your ~/.bashrc:
alias find_branch='/folder/where/you/put/the/script/find_branch'
OR
alias find_branch='python /folder/where/you/put/the/script/find_branch'
For options #1, #2, #3 and #5a to work properly, you should have a shebang with the version of python as the first line of the script. Any of the following will do, depending on how you have/want your environment set up:
#!/usr/bin/python
#!/usr/bin/python2
#!/usr/bin/python3
#!/usr/bin/env python
#!/usr/bin/env python2
#!/usr/bin/env python3
Finally, you do not have to remove the .py extension from the script if you do not want to. Many bash scripts have a .sh extension, for example, which does not prevent them from running as-is. You just have to include the extension in the name of the script when you run it.

Can't execute a cron job

I am creating a cron job to execute a python script
hello.py
a = 'a cron job was executed here'
text_file = open('output_hello.txt', 'w')
text_file.write(a)
text_file.close()
Works fine if I execute via terminal, I am on ubuntu 15.10.
My cron job file is:
* * * * * /usr/bin/python /home/rohit/hello.py
(excluding the #)
I am a root user and creating the job in /var/spool/cron
The issue is that it is not executing the script. I don't know why.

One does not simply modify the crontab, you run the command:
crontab -e
and edit from there. Execute the above command using sudo if you want it to run as root.

Assuming your paths are correct, your script may not have the right environment or it may not be executable. Ensure your script starts with:
#!/usr/bin/python
And also that you then give execute permission to that script:
chmod a+x hello.py
Ensure you use crontab -e and if you have any doubts about your syntax, you can find more info here:
https://help.ubuntu.com/community/CronHowto

running cron job with sys argument

I am trying to run a python file that takes in one argument (CSV file). It works when i run the script in the terminal but it gives errors when i run it in cron.
This is the line that i run in the terminal:
python nb2.py my_csv_file.csv
And here is my code that i am trying to run in cron:
42 13 * * * /usr/local/bin/python2.7 ~/nb/Development/code/nb2.py ~/nb/Development/code/my_csv_file.csv &> /tmp/June_QB_cat.log
The error says that it cannot find a sqlite table file which is already in the code folder.

Note that when you run the command in the terminal, you're in the ~/nb/Development/code directory already and so your current working directory is that; when you run in it cron, it is not. I would suggest either doing (in your cron job) cd ~/nb/Development/code && python nb2.py my_csv_file.csv &> /tmp/logfile.txt or doing an os.chdir("~/nb/Development/code") as the first step in your code. (Also, I'd suggest doing /home/username instead of ~ just in case you aren't running cronjob as your username at some point, but given the error you get, that sounds like it's not an issue)

You can get the path of a file relative to the current script with
import os.path
relative_path = os.path.join( os.path.dirname(__file__), "sqlitetable" )