Develop Reference

Can I create a pre-compiled tarball with Python 3.7 for AWS Linux AMI (RHEL)?

I'd like to create a tarball containing a pre-compiled version of Python 3.7 to be placed on Amazon S3.
On AWS Linux AMI based on Red-Hat Enterprise Linux (RHEL), I am downloading the source code and compiling Python 3.7 every time I have to initialize a new AMI.
The yum package manager does not seem to have a pre-compiled version of Python 3.7.
wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
tar xJf Python-3.7.0.tar.xz
cd Python-3.7.0
./configure
make
sudo make install
cd ..
rm -f Python-3.7.0.tar.xz
The code works fine, but it takes too long to boot up.

cmake error 'the source does not appear to contain CMakeLists.txt'

I'm installing opencv in ubuntu 16.04. After installing the necessary prerequisites I used the following command:-
kvs#Hunter:~/opencv_contrib$ mkdir build
kvs#Hunter:~/opencv_contrib$ cd build
kvs#Hunter:~/opencv_contrib/build$
kvs#Hunter:~/opencv_contrib/build$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX+/usr/local -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D BUILD_EXAMPLES=ON ..
but it produced an error:-
CMake Error: The source directory "/home/kvs/opencv_contrib" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
I used the command provided in the folder 'module' documentation. How do I solve it?
I tried the answers here at stack-overflow and a few other question but still can't figure it out.
Project Git repository here.

You should do mkdir build and cd build while inside opencv folder, not the opencv-contrib folder. The CMakeLists.txt is there.

Since you add .. after cmake, it will jump up and up (just like cd ..) in the directory. But if you want to run cmake under the same folder with CMakeLists.txt, please use . instead of ...

This reply may be late but it may help users having similar problem.
The opencv-contrib (available at https://github.com/opencv/opencv_contrib/releases) contains extra modules but the build procedure has to be done from core opencv (available at from https://github.com/opencv/opencv/releases) modules.
Follow below steps (assuming you are building it using CMake GUI)
Download openCV (from https://github.com/opencv/opencv/releases) and unzip it somewhere on your computer. Create build folder inside it
Download exra modules from OpenCV. (from https://github.com/opencv/opencv_contrib/releases). Ensure you download the same version.
Unzip the folder.
Open CMake
Click Browse Source and navigate to your openCV folder.
Click Browse Build and navigate to your build Folder.
Click the configure button. You will be asked how you would like to generate the files. Choose Unix-Makefile from the drop down menu and Click OK. CMake will perform some tests and return a set of red boxes appear in the CMake Window.
Search for "OPENCV_EXTRA_MODULES_PATH" and provide the path to modules folder (e.g. /Users/purushottam_d/Programs/OpenCV3_4_5_contrib/modules)
Click Configure again, then Click Generate.
Go to build folder
# cd build
# make
# sudo make install
This will install the opencv libraries on your computer.

An easier way to build OpenCV from source in a step by step fashion as given in this reference: Installing OpenCV from the Source
is to,
step 1: install dependencies,
sudo apt install build-essential cmake git pkg-config libgtk-
3-dev \libavcodec-dev libavformat-dev libswscale-dev
libv4l-dev \libxvidcore-dev libx264-dev libjpeg-dev
libpng-dev libtiff-dev \gfortran openexr libatlas-base-
dev python3-dev python3-numpy \libtbb2 libtbb-dev
libdc1394-22-dev
Step 2: create a directory opencv_build and Clone the necessary repositories as shown below,
mkdir ~/opencv_build && cd ~/opencv_build
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
step 3: cd into opencv directory, inside create another directory called build and cd into it,
cd ~/opencv_build/opencv
mkdir build && cd build
step 4: evoke Cmake to build OpenCV,
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \
-D BUILD_EXAMPLES=ON ..
If step 4 completes successfully you should see the following line at the end of the terminal, the build has been written to the directory created in step 3, along with the following lines above this line,
configuration done
generating done
step 5: To start the compilation process where -j
is a flag for the number of the processor inside your machine, for example -j6 means we have 6 processors available. to verify the number of processors type nproc on the terminal then use this number after -j. To start this process, we use the following command:
make -j6
step 6: install OpenCV, We use,
sudo make install
then check the version Of OpenCV to verify the installation:
pkg-config --modversion opencv4

I had a similar problem with another package and neither operating from a clean directory, to build from out, nor copy/paste the CMakeLists.txt file from the source to the clean directory worked.
I simply solved installing by conda

file structure :
example-app/
CMakeLists.txt
example-app.cpp
commands
mkdir build
cd build
cmake .. -DCMAKE_PREFIX_PATH=/absolute/path/to/libtorch..
cmake --build . --config Release

Segmentation fault (core dumped) Error in PyQt5

I just installed PyQt 5.7.0 on my google compute engine machine which running on a ubuntu 16.04:
However, when I wanted to run PyQt and import some module, it produce Segmentation fault (core dumped) error as shown:
Can I know how do I solve it? I have been searching an answer for this for hours and still can't find an answer. Will be greatly appreciated if anyone could help.

You can try (as explained in the comments) to compile PyQt5.7 yourself, using a different version of Python (3.4.3 and 3.4.4 worked for me, everything above 3.5 did not). Note that I also compiled Qt5.7 myself, but you can use the one provided by the installer. Here is a short, hopefully exhaustive, set of commands to setup a virtual environment:
Install dependencies using apt:
sudo apt-get install -y build-essential libgl1-mesa-dev libx11-dev libxext-dev libxfixes-dev libxi-dev libxrender-dev libxcb1-dev libx11-xcb-dev libxcb-glx0-dev libfontconfig1-dev libfreetype6-dev libglu1-mesa-dev libssl-dev libcups2-dev python3-pip git
Install Python 3.4.4:
cd ~/Downloads
wget https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz
tar xf Python-3.4.4.tar.xz
cd Python-3.4.4
./configure
sudo make altinstall
Create the virtual environment:
sudo pip3 install virtualenv
virtualenv -p /usr/local/bin/python3.4 ~/python34
source ~/python34/bin/activate
Install Qt:
cd ~/Downloads
git clone git://code.qt.io/qt/qt5.git
cd ~/Downloads/qt5
git checkout 5.7
./init-repository
./configure -prefix ~/Qt/5.7/gcc_64 -opensource -nomake examples -nomake tests -release -confirm-license
make -j 5
make install
Install SIP:
cd ~/Downloads
wget http://downloads.sourceforge.net/project/pyqt/sip/sip-4.18.1/sip-4.18.1.tar.gz
tar xf sip-4.18.1.tar.gz
cd sip-4.18.1
python configure.py
make
sudo make install
Install PyQt:
cd ~/Downloads
wget http://downloads.sourceforge.net/project/pyqt/PyQt5/PyQt-5.7/PyQt5_gpl-5.7.tar.gz
tar xf PyQt5_gpl-5.7.tar.gz
cd PyQt5_gpl-5.7
python configure.py --qmake ~/Qt/5.7/gcc_64/bin/qmake --disable QtPositioning --no-qsci-api --no-designer-plugin --no-qml-plugin --confirm-license
make -j 5
sudo make install

Caveats:
I'm going to assume that you can't run any GUI programs right now in your vm.
I'm not that familiar with GCE platform and have not tried what I'm going to propose
Follow the steps at Your desktop on Google Cloud Platform to install a desktop GUI manager in the GCE vm (it will be X11-based for Linux OS), as well as vnc server.
Once you are vnc'd in to your vm using realvnc or tightvnc viewer, a GUI app will likely run. You may have to change the X11 DISPLAY variable -- eventhough the above link does not discuss this -- because AFAIR *nix systems use a separate desktop session for VNC than the one currently logged in.
I doubt it will run if there is no user logged on to the GCE vm.

If you installed PyQt5 with apt or apt-get, now do also
sudo pip3 install pyqt5

You cannot run a GUI program (PyQt) remotely, unless both machines use X11 protocol with properly set DISPLAY variables and xhost permissions. I doubt that Google compute engine is configurable to run in the X11 mode.

When you import PyQt5, it will also import PyQt5.QtCore because PyQt5.QtCore is part of PyQt5.

Integrating opencv 3.0.0 in ubuntu for python and c++ and building lib using cmake

Please help me install opencv dev version 3.0.0 in ubuntu 14.04 and how to make build folder in opencv and create libraries. I earlier used 2.4.9 in ubuntu, things went fine but i need 3.0.0 as i need new commands which is included in it. so how to integrate opencv 3.0.0 and remove 2.4.9 completely?
what to do if this method does not give proper build/release folder?
Building OpenCV from Source Using CMake, Using the Command Line
Create a temporary directory, which we denote as , where you want to put the generated Makefiles, project files as well the object files and output binaries.
Enter the and type
cmake [<some optional parameters>] <path to the OpenCV source directory>
For example
cd ~/opencv
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
Enter the created temporary directory () and proceed with:
make -j8 # -j8 runs 8 jobs in parallel.
# Change 8 to number of hardware threads available.
sudo make install

Installation in Linux
These steps have been tested for Ubuntu 10.04 but should work with other distros as well.
Required Packages
GCC 4.4.x or later
CMake 2.8.7 or higher
Git
GTK+2.x or higher, including headers (libgtk2.0-dev)
pkg-config
Python 2.6 or later and Numpy 1.5 or later with developer packages (python-dev, python-numpy)
ffmpeg or libav development packages: libavcodec-dev, libavformat-dev, libswscale-dev
[optional] libtbb2 libtbb-dev
[optional] libdc1394 2.x
[optional] libjpeg-dev, libpng-dev, libtiff-dev, libjasper-dev, libdc1394-22-dev
The packages can be installed using a terminal and the following commands or by using Synaptic Manager:
[compiler] sudo apt-get install build-essential
[required] sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
[optional] sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Getting OpenCV Source Code
You can use the latest stable OpenCV version or you can grab the latest snapshot from our Git repository.
Getting the Latest Stable OpenCV Version
Go to our downloads page.
Download the source archive and unpack it.
Getting the Cutting-edge OpenCV from the Git Repository
Launch Git client and clone OpenCV repository. If you need modules from OpenCV contrib repository then clone it too.
For example
cd ~/<my_working_directory>
git clone https://github.com/Itseez/opencv.git
git clone https://github.com/Itseez/opencv_contrib.git
Building OpenCV from Source Using CMake
Create a temporary directory, which we denote as , where you want to put the generated Makefiles, project files as well the object files and output binaries and enter there.
For example
cd ~/opencv
mkdir build
cd build
Configuring. Run cmake [<some optional parameters>] <path to the OpenCV source directory>
For example
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
or cmake-gui
set full path to OpenCV source code, e.g. /home/user/opencv
set full path to <cmake_build_dir>, e.g. /home/user/opencv/build
set optional parameters
run: “Configure”
run: “Generate”
Description of some parameters
build type: CMAKE_BUILD_TYPE=Release\Debug
to build with modules from opencv_contrib set OPENCV_EXTRA_MODULES_PATH to <path to opencv_contrib/modules/>
set BUILD_DOCS for building documents
set BUILD_EXAMPLES to build all examples
[optional] Building python. Set the following python parameters:
PYTHON2(3)_EXECUTABLE = <path to python>
PYTHON_INCLUDE_DIR = /usr/include/python<version>
PYTHON_INCLUDE_DIR2 = /usr/include/x86_64-linux-gnu/python<version>
PYTHON_LIBRARY = /usr/lib/x86_64-linux-gnu/libpython<version>.so
PYTHON2(3)_NUMPY_INCLUDE_DIRS = /usr/lib/python<version>/dist-packages/numpy/core/include/
[optional] Building java.
Unset parameter: BUILD_SHARED_LIBS
It is useful also to unset BUILD_EXAMPLES, BUILD_TESTS, BUILD_PERF_TESTS - as they all will be statically linked with OpenCV and can take a lot of memory.
Build. From build directory execute make, recomend to do it in several threads
For example
make -j7 # runs 7 jobs in parallel
[optional] Building documents. Enter <cmake_build_dir/doc/> and run make with target “html_docs”
For example
cd ~/opencv/build/doc/
make -j7 html_docs
To install libraries, from build directory execute
sudo make install
[optional] Running tests
Get the required test data from OpenCV extra repository.
For example
git clone https://github.com/Itseez/opencv_extra.git
set OPENCV_TEST_DATA_PATH environment variable to <path to opencv_extra/testdata>.
execute tests from build directory.
For example
<cmake_build_dir>/bin/opencv_test_core
Credits: OpenCV Installation page.

SHOGUN on Ubuntu 12.04

What's the best way to get the latest SHOGUN / Python modular interface (http://www.shogun-toolbox.org) installed on 12.04? I tried from source without much luck (happy to post errors); Is it possible to install the Trusty Tahr package on 12.04? https://launchpad.net/ubuntu/+source/shogun/3.1.1-1

(I am the debian maintainer of this package).
You could try to
apt-get -b source shogun
but it will give you only the core libshogun library at this very moment. There is a python package on the way (in debian) but not yet accepted and not yet in ubuntu.
https://ftp-master.debian.org/new/python-shogun_3.1.1-1.html
So you are really best of installing from source. Shogun has buildbots running on ubuntu creating a python package. So you can just copy the settings from there:
https://travis-ci.org/shogun-toolbox/shogun/jobs/18605663

Following this posts here and this instruction i did the following that worked for me to install shogun directly into my conda env. Its not the most elegant way but worked out so far.
! Take care to have swig: ie. installed apt-get install swig
! I used anaconda and the shogun python infterface - so my cmake prefix was /home/user/anaconda/
You need cmake to build shogun:
SET UP CMAKE FIRST (skip if you have cmake > 3.1)
cd /path to your installation directory
workdir=$(pwd) #i.e. your home
Download and install cmake into your home:
wget http://www.cmake.org/files/v3.1/cmake-3.1.3.tar.gz
tar xzf cmake-3.1.3.tar.gz
cd cmake-3.1.3
cmake_dir=$workdir/cmake 5 ./configure --prefix=$cmake_dir
make -j 2
make install
Export cmake to your PATH so you can use it for shogun: export
PATH=$cmake_dir/bin/:$PATH
Download and install shogun into anaconda env
wget ftp://shogun-toolbox.org/shogun/releases/3.1/sources/shogun-3.1.1.tar.bz2
tar jxf shogun-3.1.1.tar.bz2 3 cd shogun-3.1.1/
mkdir build
cd build
cmake -DPythonStatic=ON -DPythonModular=ON -DCMAKE_INSTALL_PREFIX=/home/myusername/anaconda/envs/p27/ ..
make -j2 all #four processes takes a while
make install