So I'm trying to streamline my deployment process. I have a VM with Vagrant (Ubuntu Vivid64, virtualenv, django and postgres), but I still have to execute a lot of commands after executing vagrant up. I've used this vagrantfile: https://github.com/FlipperPA/django-python3-vagrant
This gets executed through the Vagrant provision script:
sudo apt-get -qq install git python3-pip > /dev/null 2>&1
sudo pip3 install --quiet virtualenv
sudo pip3 install --quiet virtualenvwrapper
mkdir ~vagrant/.virtualenvs
chown vagrant:vagrant ~vagrant/.virtualenvs
printf "\n\n# Virtualenv settings\n" >> ~vagrant/.bashrc
printf "export PYTHONPATH=/usr/lib/python3.4" >> ~vagrant/.bashrc
printf "export WORKON_HOME=~vagrant/.virtualenvs\n" >> ~vagrant/.bashrc
printf "export PROJECT_HOME=/vagrant\n" >> ~vagrant/.bashrc
printf "export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3.4\n" >> ~vagrant/.bashrc
printf "source /usr/local/bin/virtualenvwrapper.sh\n" >> ~vagrant/.bashrc
printf "alias python='/usr/bin/python3.4'\n" >> ~vagrant/.bashrc
When this is finished, I want to create a new virtualenv and install django in it. Then I want to install postgres and the related packages, create the database for postgres and then install a Python package for the postgres connection. This is the shell script I came up with:
#!/bin/bash
sudo apt-get update
# Setup virtual environment for Django
mkvirtualenv vms_django
pip3 install django
# Setup Postgres
sudo apt-get install libpq-dev postgresql postgresql-contrib
sudo su Postgres
psql postgres -c "CREATE DATABASE djangodb;
CREATE USER hhhhh WITH PASSWORD 'root';
ALTER ROLE hhhhh SET client_encoding TO 'utf8';
ALTER ROLE hhhhh SET default_transaction_isolation TO 'read committed';
ALTER ROLE hhhhh SET timezone TO 'UTC';
GRANT ALL PRIVILEGES ON DATABASE djangodb TO hhhhh;"
exit
# Setup Postgres Django connection
pip3 install psycopg2
# Migrate DB
python3 manage.py makemigrations
python3 manage.py migrate
python3 manage.py createsuperuser
...but that doesn't work. mkvirtualenv and psql commands are not found, and the last three python commands throw importerrors: No module named 'psycopg2'.
Anyone able to lend a hand? Alternative ways to solve my problem are welcome too.
Related
I'm trying to install Python 3.9 an EC2 instance that uses Amazon Linux 2. I tried following this guide: https://computingforgeeks.com/install-latest-python-on-centos-linux/, and I was able to install Python3.9 manually on the EC2 instance by SSH'ing in and running the commands. I'm now trying to setup the EC2 instance with a UserData script that calls some CloudFormationInit scripts to install dependencies, including Python 3.9, and my script is failing.
Here's part of the script that I'm using to install Python 3.9:
const installPythonString = `
#!/bin/bash
sudo amazon-linux-extras install -y epel
sudo yum -y update
sudo yum groupinstall "Development Tools" -y
sudo yum install openssl-devel libffi-devel bzip2-devel -y
gcc --version
sudo yum install wget -y
sudo mkdir -p /opt/python3.9/
sudo chown -R $USER:$USER /opt/python3.9/
wget https://www.python.org/ftp/python/3.9.9/Python-3.9.9.tgz -P /opt/python3.9
cd /opt/python3.9/
tar xvf Python-3.9.9.tgz
whoami
sudo chown -R $USER:$USER Python-3.9.9
cd Python-3.9.9/
ls -al
pwd
./configure --enable-optimizations
sudo make altinstall
python3.9 --version
pip3.9 --version
`;
init.addConfig('install_python39', new ec2.InitConfig([
ec2.InitFile.fromString('/opt/install_python39.sh', installPythonString, {
mode: '000755',
owner: 'root',
group: 'root',
}),
ec2.InitCommand.shellCommand('sudo sh install_python39.sh', {
cwd: '/opt',
key: 'install_python39',
}),
]))
I'm getting the following errors when trying to start up the EC2 instance:
Python build finished successfully!
...
WARNING: The script pip3.9 is installed in '/usr/local/bin' which is not on PATH.
install_python39.sh: line 21: python3.9: command not found
install_python39.sh: line 22: pip3.9: command not found
Is there an easier way to install Python 3.9 on Amazon Linux 2 using CloudFormationInit?
Looks like the path to the python is /usr/local/bin which is not in $PATH so the python3.9 command is not found.
run the following commands in order.
export PATH="/usr/local/bin:$PATH" or echo "export PATH='/usr/local/bin:$PATH' >> ~/.bashrc(if you do this relaunch the ssh session) to save it to bashrc so you don't have to run the export everytime you log in.
python3.9 --version
additionally if you keep having issues, follow this to install python3.9, which is what i used, and everything went flawlessly.
if you have python packages that need to be installed, i would recommend creating a requirements.txt and using pip3.9 install -r requirements.txt to install them.
Goal: Use ESPHome Flasher https://github.com/esphome/esphome-flasher
TLDR: Starting esphomeflasher I get this error message:
This program needs access to the screen. Please run with a
Framework build of python, and only when you are logged in
on the main display of your Mac.
Python Setup: https://opensource.com/article/19/5/python-3-default-mac
Mac Setup: Fresh Catalina installation.
Steps:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install pyenv
pyenv install 3.9.1
pyenv global 3.9.1
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.zshrc
pip install --upgrade pip
Install esphomeflasher: https://github.com/esphome/esphome-flasher
Steps:
pip3 install wxpython
pip3 install esphomeflasher
Problem:
Starting esphomeflasher I get the following error:
This program needs access to the screen. Please run with a
Framework build of python, and only when you are logged in
on the main display of your Mac.
I found this but don't know how to apply it in my case.
https://blurringexistence.net/wxpython-using-virtualenvwrapper-on-osx.html
My .zshrc config:
if command -v pyenv 1>/dev/null 2>&1; then
eval "$(pyenv init -)"
fi
PATH=$(pyenv root)/shims:$PATH
# We want to regularly go to our virtual environment directory
echo 'export WORKON_HOME=~/.virtualenvs' >> .zshrc
# If in a given virtual environment, make a virtual environment directory
# If one does not already exist
echo 'mkdir -p $WORKON_HOME' >> .zshrc
# Activate the new virtual environment by calling this script
# Note that $USER will substitute for your current user
echo '. ~/.pyenv/versions/3.9.1/bin/virtualenvwrapper.sh' >> .bash_profile
export WORKON_HOME=~/.virtualenvs
mkdir -p $WORKON_HOME
export WORKON_HOME=~/.virtualenvs
mkdir -p $WORKON_HOME
I have the following set of bash commands
docker pull mcr.microsoft.com/mssql/server:2017-latest
docker run -e 'ACCEPT_EULA=Y' -e 'MSSQL_SA_PASSWORD=my_password' \
--name 'sql1' -p 1401:1433 \
-v "my_space":/opt/project \
-d mcr.microsoft.com/mssql/server:2017-latest
winpty docker exec -it sql1 bash
mkdir -p /var/opt/mssql/backup
cp my_db /var/opt/mssql/backup/
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "my_password" -i /opt/project/scripts/database_import/sql_script.sql
apt-get update -y
apt-get install python3-pip -y
python3 -m pip install pymssql
python3 -m pip install pandas==0.19.2
python3 -m pip install time
python3 -m pip install sqlalchemy
python3 -m pip install sqlalchemy_utils
cd /opt/project/
python3 scripts/database_import/import_database.py
What this set of commands is essentially doing is that it pulls the mssql server, restores a database, installs some python packages and it runs a python script inside the mssql docker container.
Is there a way to run this bash script, from pycharm ?
Sure thing. If you are using the latest version, then Shell Script should be available https://www.jetbrains.com/help/idea/shell-scripts.html
So, for example, I have test.sh file:
I can just click this green run button and PyCharm will run it, or create Run Configuration for it (see the link above).
I've this block in one of the recipes which is being included in other recipe.
bash 'setup' do
cwd "#{node[:'python-virtualenv'][:home]}"
user 'ubuntu'
environment "HOME" => "#{node[:'python-virtualenv'][:home]}"
code <<-EOH
sudo add-apt-repository -y ppa:fkrull/deadsnakes
sudo apt-get update
sudo apt-get -yq install python3.5
curl -s https://bootstrap.pypa.io/get-pip.py | sudo python3.5
sudo pip3.5 install virtualenv virtualenvwrapper
echo "export WORKON_HOME=#{node[:'python-virtualenv'][:home]}.virtualenvs/" > #{node[:'python-virtualenv'][:home]}.bash_profile
echo "export VIRTUALENVWRAPPER_PYTHON=$(which python3.5)" > #{node[:'python-virtualenv'][:home]}.bash_profile
echo "source /usr/local/bin/virtualenvwrapper.sh" >> #{node[:'python-virtualenv'][:home]}.bash_profile
EOH
end
The logs say that this block got executed but when I ssh into the instance, it did not get installed.
Here's the relevant part of the log
[2018-03-07T09:59:33+00:00] INFO: Processing bash[setup] action run (app::default line 12)
[2018-03-07T09:59:40+00:00] INFO: bash[setup] ran successfully
Add set -e to the top of your shell script and you would probably see it fail on one of the intermediary commands. Why are you running as a non-root user and then trying to use sudo though? Overall this seems like a very odd shell script. Might want to add set -x to help with debugging too.
Trying to use tox to run tests before pushing, but I keep running into errors like:
ERROR: py26: InterpreterNotFound: python2.6
ERROR: py32: InterpreterNotFound: python3.2
ERROR: py34: InterpreterNotFound: python3.3
apt-cache search isn't offering any packages that look like they will help. How do you load all these versions of the interpreter for ubuntu14.04?
Obviously, Ubuntu doesn't ship all historic versions of Python. But you can use deadsnakes PPA which has everything from 2.3 to 3.4.
For one project I used drone.io CI service with, I had the following tox section I ran before actual test envs.
[testenv:setupdrone]
whitelist_externals = /bin/bash
commands =
bash -c "echo 'debconf debconf/frontend select noninteractive' | sudo debconf-set-selections"
bash -c "sudo add-apt-repository ppa:fkrull/deadsnakes &> /dev/null"
bash -c "sudo apt-get update &> /dev/null"
bash -c "sudo apt-get -y install python2.6 python3.4 &> /dev/null"