Develop Reference

How to install sqlite3 for python3.7 in seperate directory on linux without sudo commands?

I have the problem that when I run my code on a linux server I get:
ModuleNotFoundError: No module named '_sqlite3'
So after researching, I found out sqlite3 was supposed to have been installed when I installed python, however it didn't.
I think the problem comes from the way I installed python. Since I do not have sudo permissions, I installed python3.7 in a local directory using: This guide.
All solutions to this sqlite3 problem that I can find requires sudo commands.
Is there another way that I can install python3.7 together with sqlite3 in my local Linux directory without using any sudo commands?
I hope I have stated my question clearly and I would appreciate all the help I can get. Thank you!

While installing a python package in a Linux system without "sudo" privileges you can use
For Python 3
pip3 install --user pysqlite3
You can install any third party packages with the same method
pip3 install --user PACKAGE_NAME
The --user flag to pip install tells Pip to install packages in some specific directories within your home directory. For more information click here.
Hope it helps !

The solution is to first build sqlite3 into a user directory and then build python using that directory's libraries and include headers. In particular, #Ski has answered a similar question regarding python 2, which can be adopted to python 3:
$ mkdir -p ~/applications/src
$ cd ~/applications/src
$ # Download and build sqlite 3 (you might want to get a newer version)
$ wget http://www.sqlite.org/sqlite-autoconf-3070900.tar.gz
$ tar xvvf sqlite-autoconf-3070900.tar.gz
$ cd sqlite-autoconf-3070900
$ ./configure --prefix=~/applications
$ make
$ make install
$ # Now download and build python 2, same works for python 3
$ cd ~/applications/src
$ wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tgz
$ tar xvvf Python-2.5.2.tgz
$ cd Python-2.5.2
$ ./configure --prefix=~/applications
$ make
$ make install
$ ~/applications/bin/python
Alternatively, if you already have to specify a different --prefix for some reason (this has happened to me with pyenv), use LDFLAGS and CPPFLAGS when configuring python build:
$ ./configure LDFLAGS=-L/home/user/applications/lib/ CPPFLAGS=-I/home/user/applications/include/

Python application not loading graph

When I run this Python script the percentage appears in my command line but the graph does not display. I'm running this on the Raspberry Pi. All I've done so far is:
sudo apt-get install update
sudo apt-get install python-numpy
sudo apt-get install python-scipy
sudo apt-get install python-matplotlib
sudo apt-get install git
sudo git clone https://github.com/scikit-fuzzy/scikit-fuzzy.git
cd scikit-fuzzy/
sudo python setup.py install
cd ..
python tipping.py
How do I get it to open?

I have first thought about adding he following line to the code:
plt.show()
However, it can also be that Python is storing the graph information directly in the memory but not showing it. In this case you must change the "Graph visualization" preferences in Python. If you are running Python through Spyder, then you must go to Tools>Preferences>Graph visualization and tick the option related to show the graph in a different window.

This was resolved by adding:
import matplotlib
matplotlib.use('TkAgg')
before:
import matplotlib.pyplot as plt

Segmentation fault (core dumped) Error in PyQt5

I just installed PyQt 5.7.0 on my google compute engine machine which running on a ubuntu 16.04:
However, when I wanted to run PyQt and import some module, it produce Segmentation fault (core dumped) error as shown:
Can I know how do I solve it? I have been searching an answer for this for hours and still can't find an answer. Will be greatly appreciated if anyone could help.

You can try (as explained in the comments) to compile PyQt5.7 yourself, using a different version of Python (3.4.3 and 3.4.4 worked for me, everything above 3.5 did not). Note that I also compiled Qt5.7 myself, but you can use the one provided by the installer. Here is a short, hopefully exhaustive, set of commands to setup a virtual environment:
Install dependencies using apt:
sudo apt-get install -y build-essential libgl1-mesa-dev libx11-dev libxext-dev libxfixes-dev libxi-dev libxrender-dev libxcb1-dev libx11-xcb-dev libxcb-glx0-dev libfontconfig1-dev libfreetype6-dev libglu1-mesa-dev libssl-dev libcups2-dev python3-pip git
Install Python 3.4.4:
cd ~/Downloads
wget https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz
tar xf Python-3.4.4.tar.xz
cd Python-3.4.4
./configure
sudo make altinstall
Create the virtual environment:
sudo pip3 install virtualenv
virtualenv -p /usr/local/bin/python3.4 ~/python34
source ~/python34/bin/activate
Install Qt:
cd ~/Downloads
git clone git://code.qt.io/qt/qt5.git
cd ~/Downloads/qt5
git checkout 5.7
./init-repository
./configure -prefix ~/Qt/5.7/gcc_64 -opensource -nomake examples -nomake tests -release -confirm-license
make -j 5
make install
Install SIP:
cd ~/Downloads
wget http://downloads.sourceforge.net/project/pyqt/sip/sip-4.18.1/sip-4.18.1.tar.gz
tar xf sip-4.18.1.tar.gz
cd sip-4.18.1
python configure.py
make
sudo make install
Install PyQt:
cd ~/Downloads
wget http://downloads.sourceforge.net/project/pyqt/PyQt5/PyQt-5.7/PyQt5_gpl-5.7.tar.gz
tar xf PyQt5_gpl-5.7.tar.gz
cd PyQt5_gpl-5.7
python configure.py --qmake ~/Qt/5.7/gcc_64/bin/qmake --disable QtPositioning --no-qsci-api --no-designer-plugin --no-qml-plugin --confirm-license
make -j 5
sudo make install

Caveats:
I'm going to assume that you can't run any GUI programs right now in your vm.
I'm not that familiar with GCE platform and have not tried what I'm going to propose
Follow the steps at Your desktop on Google Cloud Platform to install a desktop GUI manager in the GCE vm (it will be X11-based for Linux OS), as well as vnc server.
Once you are vnc'd in to your vm using realvnc or tightvnc viewer, a GUI app will likely run. You may have to change the X11 DISPLAY variable -- eventhough the above link does not discuss this -- because AFAIR *nix systems use a separate desktop session for VNC than the one currently logged in.
I doubt it will run if there is no user logged on to the GCE vm.

If you installed PyQt5 with apt or apt-get, now do also
sudo pip3 install pyqt5

You cannot run a GUI program (PyQt) remotely, unless both machines use X11 protocol with properly set DISPLAY variables and xhost permissions. I doubt that Google compute engine is configurable to run in the X11 mode.

When you import PyQt5, it will also import PyQt5.QtCore because PyQt5.QtCore is part of PyQt5.

How do I set Django to use Python 2.7 instead of the default Python 2.4?

I am using CentOS. The default python installed is 2.4 and I also installed 2.7 in order to use Django.
How can I configure Django to use /usr/local/bin/python2.7 instead of just the default python command?
I have to leave the default Python as 2.4 because other services such as yum don't run with 2.7.
Or is there other solution?

mod_wsgi is linked with specific version of Python. You have to recompile it with Python 2.7 and make Apache load new module (by editing file like /etc/apache2/mods-enabled/wsgi.load or /etc/apache2/modules.d/70_mod_wsgi.conf).
http://code.google.com/p/modwsgi/wiki/InstallationIssues

Just run this command into your command shell: python2.7 /path/to/manage.py runserver
Then Django will run under python 2.7

The answer is to use virtualenv to set up a virtual environment for python2.7 and to install your stuff into that sandbox.
Here is some code to get you going:
$ sudo apt-get install python-setuptools python-dev build-essential
$ sudo easy_install -U pip
$ sudo pip install virtualenv virtualenvwrapper
$ mkdir ~/.virtualenvs
$ sudo cat >> ~/.bashrc << EOF
# virtualenvwrapper setup
export WORKON_HOME=~/.virtualenvs
export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--no-site-packages --python=python2.7'
source /usr/local/bin/virtualenvwrapper.sh
EOF
$ source ~/.bashrc
$ mkvirtualenv test
$ pip install django
I am assuming that CentOS uses sudo like Ubuntu does. Substitute the native call if not.

Compiling Python 2.6.6 and need for external packages wxPython, setuptools, etc... in Ubuntu

I compiled Python 2.6.6 with google-perf tools (tcmalloc) library to eliminate some of the memory issues I was having with the default 2.6.5. After getting 2.6.6 going it seems to not work becuase I think having issues with the default 2.6.5 install in Ubuntu. Will none of the binaries installed from the software channel like wxPython and setuptools work properly with 2.6.6. Do these need to be recompiled? Any other suggestions to get it working smoothly. Can I still set 2.6.5 as default without changing the Path? The path looks in usr/local/bin first.

A good general rule of thumb is to NEVER use the default system installed Python for any software development beyond miscellaneous system admin scripts. This applies on all UNIXes including Linux and OS/X.
Instead, build a good Python distro that you control, with the libraries (Python and C) that you need, and install this tarball in a non-system directory such as /opt/devpy or /data/package/python or /home/python. And why mess with 2.6 when 2.7.2 is available?
And when you are building it, make sure that all of its dependencies are in its own directory tree (RPATH) and that any system dependencies (.so files) are copied into its directory tree. Here is my version. It might not work if you just run the whole shell script. I always copy and paste sections of this into a terminal window and verify that each step worked OK. Make sure your terminal properties are set to allow lots of lines of scrollback, or only paste a couple of lines at a time.
(actually, after making a few tweaks I think this may be runnable as a script, however I would recommend something like ./pybuild.sh >pylog 2>&1 so you can comb through the output and verify that everything built OK.
This was built on Ubuntu 64 bit
#!/bin/bash
shopt -s compat40
export WGET=echo
#uncomment the following if you are running for the first time
export WGET=wget
sudo apt-get -y install build-essential
sudo apt-get -y install zlib1g-dev libxml2-dev libxslt1-dev libssl-dev libncurses5-dev
sudo apt-get -y install libreadline6-dev autotools-dev autoconf automake libtool
sudo apt-get -y install libsvn-dev mercurial subversion git-core
sudo apt-get -y install libbz2-dev libgdbm-dev sqlite3 libsqlite3-dev
sudo apt-get -y install curl libcurl4-gnutls-dev
sudo apt-get -y install libevent-dev libev-dev librrd4 rrdtool
sudo apt-get -y install uuid-dev libdb4.8-dev memcached libmemcached-dev
sudo apt-get -y install libmysqlclient-dev libexpat1-dev
cd ~
$WGET 'http://code.google.com/p/google-perftools/downloads/detail?name=google-perftools-1.7.tar.gz'
$WGET http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz
tar zxvf Python-2.7.2.tgz
cd Python-2.7.2
#following is needed if you have an old version of Mercurial installed
#export HAS_HG=not-found
# To provide a uniform build environment
unset PYTHONPATH PYTHONSTARTUP PYTHONHOME PYTHONCASEOK PYTHONIOENCODING
unset LD_RUN_PATH LD_LIBRARY_PATH LD_DEBUG LD_TRACE_LOADED_OBJECTS
unset LD_PRELOAD SHLIB_PATH LD_BIND_NOW LD_VERBOSE
## figure out whether this is a 32 bit or 64 bit system
m=`uname -m`
if [[ $m =~ .*64 ]]; then
export CC="gcc -m64"
NBITS=64
elif [[ $m =~ .*86 ]]; then
export CC="gcc -m32"
NBITS=32
else # we are confused so bail out
echo $m
exit 1
fi
# some stuff related to distro independent build
# extra_link_args = ['-Wl,-R/data1/python27/lib']
#--enable-shared and a relative
# RPATH[0] (eg LD_RUN_PATH='${ORIGIN}/../lib')
export TARG=/data1/packages/python272
export TCMALLOC_SKIP_SBRK=true
#export CFLAGS='-ltcmalloc' # Google's fast malloc
export COMMONLDFLAGS='-Wl,-rpath,\$$ORIGIN/../lib -Wl,-rpath-link,\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib -Wl,-z,origin -Wl,--enable-new-dtags'
# -Wl,-dynamic-linker,$TARG/lib/ld-linux-x86-64.so.2
export LDFLAGS=$COMMONLDFLAGS
./configure --prefix=$TARG --with-dbmliborder=bdb:gdbm --enable-shared --enable-ipv6
# if you have ia32-libs installed on a 64-bit system
#export COMMONLDFLAGS="-L/lib32 -L/usr/lib32 -L`pwd`/lib32 -Wl,-rpath,$TARG/lib32 -Wl,-rpath,$TARG/usr/lib32"
make
# ignore failure to build the following since they are obsolete or deprecated
# _tkinter bsddb185 dl imageop sunaudiodev
#install it and collect any dependency libraries - not needed with RPATH
sudo mkdir -p $TARG
sudo chown `whoami`.users $TARG
make install
# collect binary libraries ##REDO THIS IF YOU ADD ANY ADDITIONAL MODULES##
function collect_binary_libs {
cd $TARG
find . -name '*.so' | sed 's/^/ldd -v /' >elffiles
echo "ldd -v bin/python" >>elffiles
chmod +x elffiles
./elffiles | sed 's/.*=> //;s/ .*//;/:$/d;s/^ *//' | sort -u | sed 's/.*/cp -L & lib/' >lddinfo
# mkdir lib
chmod +x lddinfo
./lddinfo
cd ~
}
collect_binary_libs
#set the path
cd ~
export PATH=$TARG/bin:$PATH
#installed setuptools
$WGET http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg
chmod +x setuptools-0.6c11-py2.7.egg
./setuptools-0.6c11-py2.7.egg
#installed virtualenv
tar zxvf virtualenv-1.6.1.tar.gz
cd virtualenv-1.6.1
python setup.py install
cd ~
# created a base virtualenv that should work for almost all projects
# we make it relocatable in case its location in the filesystem changes.
cd ~
python virtualenv-1.6.1/virtualenv.py /data1/py27base # first make it
python virtualenv-1.6.1/virtualenv.py --relocatable /data1/py27base #then relocatabilize
# check it out
source ~/junk/bin/activate
python --version
# fill the virtualenv with useful modules
# watch out for binary builds that may have dependency problems
export LD_RUN_PATH='\$$ORIGIN:\$$ORIGIN/../lib:\$$ORIGIN/../../lib'
easy_install pip
pip install cython
pip install lxml
pip install httplib2
pip install python-memcached
pip install amqplib
pip install kombu
pip install carrot
pip install py_eventsocket
pip install haigha
# extra escaping of $ signs
export LDFLAGS='-Wl,-rpath,\$\$$ORIGIN/../lib:\$\$$ORIGIN/../../lib -Wl,-rpath-link,\$\$$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags'
# even more complex to build this one since we need some autotools and
# have to pull source from a repository
mkdir rabbitc
cd rabbitc
hg clone http://hg.rabbitmq.com/rabbitmq-codegen/
hg clone http://hg.rabbitmq.com/rabbitmq-c/
cd rabbitmq-c
autoreconf -i
make clean
./configure --prefix=/usr
make
sudo make install
cd ~
# for zeromq we get the latest source of the library
$WGET http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar zxvf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
make clean
./configure --prefix=/usr
make
sudo make install
cd ~
# need less escaping of $ signs
export LDFLAGS='-Wl,-rpath,\$ORIGIN/../lib:\$ORIGIN/../../lib -Wl,-rpath-link,\$ORIGIN/../lib -Wl,-z,origin -Wl,--enable-new-dtags'
pip install pyzmq
pip install pylibrabbitmq # need to build C library and install first
pip install pylibmc
pip install pycurl
export LDFLAGS=$COMMONLDFLAGS
pip install cherrypy
pip install pyopenssl # might need some ldflags on this one?
pip install diesel
pip install eventlet
pip install fapws3
pip install gevent
pip install boto
pip install jinja2
pip install mako
pip install paste
pip install twisted
pip install flup
pip install pika
pip install pymysql
# pip install py-rrdtool # not on 64 bit???
pip install PyRRD
pip install tornado
pip install redis
# for tokyocabinet we need the latest source of the library
$WGET http://fallabs.com/tokyocabinet/tokyocabinet-1.4.47.tar.gz
tar zxvf tokyocabinet-1.4.47.tar.gz
cd tokyocabinet-1.4.47
make clean
./configure --prefix=/usr --enable-devel
make
sudo make install
cd ..
$WGET http://fallabs.com/tokyotyrant/tokyotyrant-1.1.41.tar.gz
tar zxvf tokyotyrant-1.1.41.tar.gz
cd tokyotyrant-1.1.41
make clean
./configure --prefix=/usr --enable-devel
make
sudo make install
cd ..
pip install tokyo-python
pip install solrpy
pip install pysolr
pip install sunburnt
pip install txamqp
pip install littlechef
pip install PyChef
pip install pyvb
pip install bottle
pip install werkzeug
pip install BeautifulSoup
pip install XSLTools
pip install numpy
pip install coverage
pip install pylint
# pip install PyChecker ???
pip install pycallgraph
pip install mkcode
pip install pydot
pip install sqlalchemy
pip install buzhug
pip install flask
pip install restez
pip install pytz
pip install mcdict
# need less escaping of $ signs
pip install py-interface
# pip install paramiko # pulled in by another module
pip install pexpect
# SVN interface
$WGET http://pysvn.barrys-emacs.org/source_kits/pysvn-1.7.5.tar.gz
tar zxvf pysvn-1.7.5.tar.gz
cd pysvn-1.7.5/Source
python setup.py backport
python setup.py configure
make
cd ../Tests
make
cd ../Sources
mkdir -p $TARG/lib/python2.7/site-packages/pysvn
cp pysvn/__init__.py $TARG/lib/python2.7/site-packages/pysvn
cp pysvn/_pysvn_2_7.so $TARG/lib/python2.7/site-packages/pysvn
cd ~
# pip install protobuf #we have to do this the hard way
$WGET http://protobuf.googlecode.com/files/protobuf-2.4.1.zip
unzip protobuf-2.4.1.zip
cd protobuf-2.4.1
make clean
./configure --prefix=/usr
make
sudo make install
cd python
python setup.py install
cd ~
pip install riak
pip install ptrace
pip install html5lib
pip install metrics
#redo the "install binary libraries" step
collect_binary_libs
# link binaries in the lib directory to avoid search path errors and also
# to reduce the number of false starts to find the library
for i in `ls $TARG/lib/python2.7/lib-dynload/*.so`
do
ln -f $i $TARG/lib/`basename $i`
done
# for the same reason link the whole lib directory to some other places in the tree
ln -s ../.. $TARG/lib/python2.7/site-packages/lib
# bundle it up and save it for packaging
cd /
tar cvf - .$TARG |gzip >~/py272-$NBITS.tar.gz
cd ~
# after untarring on another machine, we have a program call imports.py which imports
# every library as a quick check that it works. For a more positive check, run it like this
# strace -e trace=stat,fstat,open python imports.py >strace.txt 2>&1
# grep -v ' = -1' strace.txt |grep 'open(' >opens.txt
# sed <opens.txt 's/^open("//;s/".*//' |sort -u |grep -v 'dynload' |grep '\.so' >straced.txt
# ls -1d /data1/packages/python272/lib/* |sort -u >lib.txt
# then examine the strace output to see how many places it searches before finding it.
# a successful library load will be a call to open that doesn't end with ' = -1'
# If it takes too many tries to find a particular library, then another symbolic link may
# be a good idea

I'm pretty sure you have to compile wxPython to the version of Python that you want to use it with. That's always been the case with anyone else who has done something like this on the wxPython mailing list. I think that applies to most packages and especially so if they have any C/C++ components, like wxPython does. Pure Python packages can sometimes be transferred from one version to the next intact in my experience.
There are fairly extensive wxPython build instructions here: http://wxpython.org/BUILD-2.8.html
Robin Dunn and others on the wxPython mailing list are very helpful if you run into any problems.

If you compiled 2.6.6 and installed 2.6.5 from the repos, then ubuntu is having a conflict in finding what python you're using.
I'm flagging this to move to Superuser.