Develop Reference

I have some problem python packages in Google Compute engine

I want to my code running for 24 hours.
So i made Google Cloude Compute Engine instance
and tried install python and other packages
but ccxt package has a error to install
this is error sentence
~$ pip3 install ccxt
Collecting ccxt
Using cached https://files.pythonhosted.org/packages/78/ec/1b33bd878c88898079e8582d01ad7f6e9d7b99377015e768537269
9cf7e5/ccxt-1.73.12-py2.py3-none-any.whl
Requirement already satisfied: aiohttp>=3.8; python_version >= "3.5.2" in ./.local/lib/python3.7/site-packages (fro
m ccxt) (3.8.1)
Requirement already satisfied: requests>=2.18.4 in ./.local/lib/python3.7/site-packages (from ccxt) (2.27.1)
Requirement already satisfied: cryptography>=2.6.1 in /usr/lib/python3/dist-packages (from ccxt) (2.6.1)
Requirement already satisfied: yarl==1.7.2; python_version >= "3.5.2" in ./.local/lib/python3.7/site-packages (from
ccxt) (1.7.2)
Requirement already satisfied: certifi>=2018.1.18 in ./.local/lib/python3.7/site-packages (from ccxt) (2021.10.8)
Collecting aiodns>=1.1.1; python_version >= "3.5.2" (from ccxt)
Using cached https://files.pythonhosted.org/packages/ab/72/991ee33a517df69c6cd6f3486cfe9b6329557cb55acaa8cefac33c
2aa4d2/aiodns-3.0.0-py3-none-any.whl
Requirement already satisfied: setuptools>=60.9.0 in ./.local/lib/python3.7/site-packages (from ccxt) (60.9.1)
Requirement already satisfied: multidict<7.0,>=4.5 in ./.local/lib/python3.7/site-packages (from aiohttp>=3.8; pyth
on_version >= "3.5.2"->ccxt) (6.0.2)
Requirement already satisfied: aiosignal>=1.1.2 in ./.local/lib/python3.7/site-packages (from aiohttp>=3.8; python_
version >= "3.5.2"->ccxt) (1.2.0)
Requirement already satisfied: asynctest==0.13.0; python_version < "3.8" in ./.local/lib/python3.7/site-packages (f
rom aiohttp>=3.8; python_version >= "3.5.2"->ccxt) (0.13.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in ./.local/lib/python3.7/site-packages (from aiohttp>=
3.8; python_version >= "3.5.2"->ccxt) (2.0.12)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in ./.local/lib/python3.7/site-packages (from aiohttp>=3
.8; python_version >= "3.5.2"->ccxt) (4.0.2)
Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in ./.local/lib/python3.7/site-pack
ages (from aiohttp>=3.8; python_version >= "3.5.2"->ccxt) (4.1.1)
Requirement already satisfied: attrs>=17.3.0 in ./.local/lib/python3.7/site-packages (from aiohttp>=3.8; python_ver
sion >= "3.5.2"->ccxt) (21.4.0)
Requirement already satisfied: frozenlist>=1.1.1 in ./.local/lib/python3.7/site-packages (from aiohttp>=3.8; python
_version >= "3.5.2"->ccxt) (1.3.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.local/lib/python3.7/site-packages (from requests>=2.18.4
->ccxt) (1.26.8)
Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in ./.local/lib/python3.7/site-packages (from re
quests>=2.18.4->ccxt) (3.3)
Collecting pycares>=4.0.0 (from aiodns>=1.1.1; python_version >= "3.5.2"->ccxt)
Using cached https://files.pythonhosted.org/packages/4c/5a/a82398e38049e689350f07b46dd8493a8a74821e39c2b904350a31
60c1cb/pycares-4.1.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.w
hl
Collecting cffi>=1.5.0 (from pycares>=4.0.0->aiodns>=1.1.1; python_version >= "3.5.2"->ccxt)
Using cached https://files.pythonhosted.org/packages/00/9e/92de7e1217ccc3d5f352ba21e52398372525765b2e0c4530e6eb2b
a9282a/cffi-1.15.0.tar.gz
Requirement already satisfied: pycparser in ./.local/lib/python3.7/site-packages (from cffi>=1.5.0->pycares>=4.0.0-
>aiodns>=1.1.1; python_version >= "3.5.2"->ccxt) (2.21)
Building wheels for collected packages: cffi
Running setup.py bdist_wheel for cffi ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-afnot
79o/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(c
ompile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-p40y4ye4 --python-tag cp37:
running bdist_wheel
running build
running build_py
running build_ext
building '_cffi_backend' extension
c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^~~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
----------------------------------------
Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: cffi, pycares, aiodns, ccxt
Running setup.py install for cffi ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-afn
ot79o/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec
(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-yai9v9gy/install-record.txt --single-version-ex
ternally-managed --compile --user --prefix=:
running install
/home/dlqudgjs12/.local/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarn
ing: setup.py install is deprecated. Use build and pip and other standards-based tools.
setuptools.SetuptoolsDeprecationWarning,
running build
running build_py
running build_ext
building '_cffi_backend' extension
c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^~~~~~~
compilation terminated.
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-afnot79o/cffi/setup.py';f=g
etattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__,
'exec'))" install --record /tmp/pip-record-yai9v9gy/install-record.txt --single-version-externally-managed --compi
le --user --prefix=" failed with error code 1 in /tmp/pip-install-afnot79o/cffi/
i tried to solve this problem.
but i couldn't find right solution
also other packages..
pip3 install TA-lib
pip3 install pandas_ta
both packages are stoped during install..

Several reasons may trigger this error. To mention a few:
Your system development libraries are missing.
You may not be running pip from your virtual environment.
You didn’t specify when you were installing PIL.
The error : x86_64-Linux-gnu-gcc failed with exit status 1 can be triggered by missing system development libraries packages, installing these library packages can fix the problem.(e.g. ffi→ libffi, ssl → libssl).
Install the appropriate packages by the following command
sudo apt-get install python3-dev
or for a specific version of Python 3, replace x with the minor version in
sudo apt-get install python3.x-dev
Try this command for installing the library package.
sudo apt-get install libffi-dev
Refer to the link for more information on why this error will trigger.

Unable to install proxybroker in PIP

I am trying to install proxybroker using PIP on Python 3.9.1, but I am getting the the following error, when I entered pip install proxybroker:-
Collecting proxybroker
Using cached proxybroker-0.3.2-py3-none-any.whl (1.6 MB)
Requirement already satisfied: aiohttp>=3.5.4 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from proxybroker) (3.7.3)
Requirement already satisfied: attrs==19.1.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from proxybroker) (19.1.0)
Requirement already satisfied: maxminddb>=1.4.1 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from proxybroker) (2.0.3)
Collecting aiodns>=2.0.0
Using cached aiodns-2.0.0-py2.py3-none-any.whl (4.8 kB)
Collecting pycares>=3.0.0
Using cached pycares-3.1.1.tar.gz (233 kB)
Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from aiohttp>=3.5.4->proxybroker) (5.1.0)
Requirement already satisfied: chardet<4.0,>=2.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from aiohttp>=3.5.4->proxybroker) (3.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from aiohttp>=3.5.4->proxybroker) (1.6.3)
Requirement already satisfied: async-timeout<4.0,>=3.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from aiohttp>=3.5.4->proxybroker) (3.0.1)
Requirement already satisfied: typing-extensions>=3.6.5 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from aiohttp>=3.5.4->proxybroker) (3.7.4.3)
Requirement already satisfied: cffi>=1.5.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from pycares>=3.0.0->aiodns>=2.0.0->proxybroker) (1.14.5)
Requirement already satisfied: pycparser in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from cffi>=1.5.0->pycares>=3.0.0->aiodns>=2.0.0->proxybroker) (2.20)
Requirement already satisfied: idna>=2.0 in c:\users\jshs\appdata\local\programs\python\python39\lib\site-packages (from yarl<2.0,>=1.0->aiohttp>=3.5.4->proxybroker) (2.10)
Using legacy 'setup.py install' for pycares, since package 'wheel' is not installed.
Installing collected packages: pycares, aiodns, proxybroker
Running setup.py install for pycares ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\jshs\appdata\local\programs\python\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\JSHS\\AppData\\Local\\Temp\\pip-install-kpsxgm2p\\pycares_9e37196811bb4121bae779a1fb0ccf7f\\setup.py'"'"'; __file__='"'"'C:\\Users\\JSHS\\AppData\\Local\\Temp\\pip-install-kpsxgm2p\\pycares_9e37196811bb4121bae779a1fb0ccf7f\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\JSHS\AppData\Local\Temp\pip-record-c7u1qn2z\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\jshs\appdata\local\programs\python\python39\Include\pycares'
cwd: C:\Users\JSHS\AppData\Local\Temp\pip-install-kpsxgm2p\pycares_9e37196811bb4121bae779a1fb0ccf7f\
Complete output (26 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\pycares
copying src\pycares\errno.py -> build\lib.win-amd64-3.9\pycares
copying src\pycares\utils.py -> build\lib.win-amd64-3.9\pycares
copying src\pycares\_version.py -> build\lib.win-amd64-3.9\pycares
copying src\pycares\__init__.py -> build\lib.win-amd64-3.9\pycares
copying src\pycares\__main__.py -> build\lib.win-amd64-3.9\pycares
running build_ext
generating cffi module 'build\\temp.win-amd64-3.9\\Release\\_cares.c'
creating build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release
building '_cares' extension
creating build\temp.win-amd64-3.9\Release\build
creating build\temp.win-amd64-3.9\Release\build\temp.win-amd64-3.9
creating build\temp.win-amd64-3.9\Release\build\temp.win-amd64-3.9\Release
creating build\temp.win-amd64-3.9\Release\deps
creating build\temp.win-amd64-3.9\Release\deps\c-ares
creating build\temp.win-amd64-3.9\Release\deps\c-ares\src
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DCARES_PULL_WS2TCPIP_H=1 -Ic:\users\jshs\appdata\local\programs\python\python39\include -Ic:\users\jshs\appdata\local\programs\python\python39\include -Ideps\c-ares\src -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\include /Tcbuild\temp.win-amd64-3.9\Release\_cares.c /Fobuild\temp.win-amd64-3.9\Release\build\temp.win-amd64-3.9\Release\_cares.obj
_cares.c
c:\users\jshs\appdata\local\programs\python\python39\include\pyconfig.h(59): fatal error C1083: Cannot open include file: 'io.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\jshs\appdata\local\programs\python\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\JSHS\\AppData\\Local\\Temp\\pip-install-kpsxgm2p\\pycares_9e37196811bb4121bae779a1fb0ccf7f\\setup.py'"'"'; __file__='"'"'C:\\Users\\JSHS\\AppData\\Local\\Temp\\pip-install-kpsxgm2p\\pycares_9e37196811bb4121bae779a1fb0ccf7f\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\JSHS\AppData\Local\Temp\pip-record-c7u1qn2z\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\jshs\appdata\local\programs\python\python39\Include\pycares' Check the logs for full command output.
It says that Microsoft build tools are missing, so I installed that too, but still not working!
Please help if possible!

have you tried to downgrade the python version?
I suspect it is because of the version that you are using.
From this link, it seems that it is compatible to python 3.5 or 3.6.
https://pypi.org/project/proxybroker/

Can't install Ball module

I am having a lot of trouble installing the Ball module, im running latest version of pip, python and im on windows 10. here is the error:
pip install ball
`Collecting ball
Using cached Ball-0.2.9.tar.gz (348 kB)
Requirement already satisfied: numpy in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from ball) (1.19.5)
Requirement already satisfied: scikit-learn in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from ball) (0.24.0)
Requirement already satisfied: pygam in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from ball) (0.8.0)
Requirement already satisfied: progressbar2 in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from pygam->ball) (3.53.1)
Requirement already satisfied: scipy in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from pygam->ball) (1.6.0)
Requirement already satisfied: future in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from pygam->ball) (0.18.2)
Requirement already satisfied: six in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from progressbar2->pygam->ball) (1.15.0)
Requirement already satisfied: python-utils>=2.3.0 in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from progressbar2->pygam->ball) (2.4.0)
Requirement already satisfied: joblib>=0.11 in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from scikit-learn->ball) (1.0.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\users\davis\appdata\local\programs\python\python39\lib\site-packages (from scikit-learn->ball) (2.1.0)
Using legacy 'setup.py install' for ball, since package 'wheel' is not installed.
Installing collected packages: ball
Running setup.py install for ball ... error
ERROR: Command errored out with exit status 1:
command: 'c:\users\davis\appdata\local\programs\python\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Davis\\AppData\\Local\\Temp\\pip-install-0abie82x\\ball_2fc4c2cf6ca545768116429af36f95b5\\setup.py'"'"'; __file__='"'"'C:\\Users\\Davis\\AppData\\Local\\Temp\\pip-install-0abie82x\\ball_2fc4c2cf6ca545768116429af36f95b5\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Davis\AppData\Local\Temp\pip-record-ly4riu7c\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\davis\appdata\local\programs\python\python39\Include\ball'
cwd: C:\Users\Davis\AppData\Local\Temp\pip-install-0abie82x\ball_2fc4c2cf6ca545768116429af36f95b5\
Complete output (18 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.9
creating build\lib.win-amd64-3.9\Ball
copying Ball\bcorsis.py -> build\lib.win-amd64-3.9\Ball
copying Ball\bcov.py -> build\lib.win-amd64-3.9\Ball
copying Ball\bd.py -> build\lib.win-amd64-3.9\Ball
copying Ball\cball.py -> build\lib.win-amd64-3.9\Ball
copying Ball\utilize.py -> build\lib.win-amd64-3.9\Ball
copying Ball\wrap_c.py -> build\lib.win-amd64-3.9\Ball
copying Ball\__init__.py -> build\lib.win-amd64-3.9\Ball
running build_ext
building 'Ball._cball' extension
swigging src/cball.i to src/cball_wrap.c
swig.exe -python -o src/cball_wrap.c src/cball.i
error: command 'swig.exe' failed: None
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\davis\appdata\local\programs\python\python39\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\Davis\\AppData\\Local\\Temp\\pip-install-0abie82x\\ball_2fc4c2cf6ca545768116429af36f95b5\\setup.py'"'"'; __file__='"'"'C:\\Users\\Davis\\AppData\\Local\\Temp\\pip-install-0abie82x\\ball_2fc4c2cf6ca545768116429af36f95b5\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\Davis\AppData\Local\Temp\pip-record-ly4riu7c\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\davis\appdata\local\programs\python\python39\Include\ball' Check the logs for full command output.`

Why is my install failing for Apache-Beam Python library?

I am running Python 2.7 and trying to install Apache-Beam.
I run this command: python -m pip install apache-beam. Output below.
The installation must be failing as I get this error when I try to run code that references the library: "No module named apache_beam"
Any help is much appreciated.
Here is the output from the installation:
(base) C:\Users\ned06>python -m pip install apache-beam
Collecting apache-beam
Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam)
Using cached https://files.pythonhosted.org/packages/20/b6/36bfb1760f6983788d9
16096193fc14c83cce512c7787c93380e09458c09/PyVCF-0.6.8.tar.gz
Requirement already satisfied: six<1.12,>=1.9 in c:\users\ned06\appdata\local\co
ntinuum\anaconda2\lib\site-packages (from apache-beam) (1.11.0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in c:\users\ned06\appdata\loc
al\continuum\anaconda2\lib\site-packages (from apache-beam) (3.12)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in c:\users\ned06\appdata\lo
cal\continuum\anaconda2\lib\site-packages (from apache-beam) (3.6.4)
Requirement already satisfied: pydot<1.3,>=1.2.0 in c:\users\ned06\appdata\local
\continuum\anaconda2\lib\site-packages (from apache-beam) (1.2.4)
Requirement already satisfied: oauth2client<5,>=2.0.1 in c:\users\ned06\appdata\
local\continuum\anaconda2\lib\site-packages (from apache-beam) (4.1.2)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from apache-beam) (0.2.6)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from apache-beam) (2018.4)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in c:\users\ned06\appdata\loca
l\continuum\anaconda2\lib\site-packages (from apache-beam) (2.1.0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in c:\users\ned06\appdat
a\local\continuum\anaconda2\lib\site-packages (from apache-beam) (3.6.0)
Requirement already satisfied: grpcio<2,>=1.8 in c:\users\ned06\appdata\local\co
ntinuum\anaconda2\lib\site-packages (from apache-beam) (1.14.2)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from apache-beam) (3.2.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in c:\users\ned06\appdata\loca
l\continuum\anaconda2\lib\site-packages (from apache-beam) (2.0.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in c:\users\ned06\appdata\
local\continuum\anaconda2\lib\site-packages (from apache-beam) (0.9.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in c:\users\ned06\appdata\local\
continuum\anaconda2\lib\site-packages (from apache-beam) (1.7)
Requirement already satisfied: avro<2.0.0,>=1.8.1 in c:\users\ned06\appdata\loca
l\continuum\anaconda2\lib\site-packages (from apache-beam) (1.8.2)
Requirement already satisfied: future<1.0.0,>=0.16.0 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from apache-beam) (0.16.0)
Requirement already satisfied: setuptools in c:\users\ned06\appdata\local\contin
uum\anaconda2\lib\site-packages (from pyvcf<0.7.0,>=0.6.8->apache-beam) (40.2.0)
Requirement already satisfied: pyparsing>=2.1.4 in c:\users\ned06\appdata\local\
continuum\anaconda2\lib\site-packages (from pydot<1.3,>=1.2.0->apache-beam) (2.2
.0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from oauth2client<5,>=2.0.1->apache-
beam) (0.2.2)
Requirement already satisfied: pyasn1>=0.1.7 in c:\users\ned06\appdata\local\con
tinuum\anaconda2\lib\site-packages (from oauth2client<5,>=2.0.1->apache-beam) (0
.4.3)
Requirement already satisfied: rsa>=3.1.4 in c:\users\ned06\appdata\local\contin
uum\anaconda2\lib\site-packages (from oauth2client<5,>=2.0.1->apache-beam) (3.4.
2)
Requirement already satisfied: pyreadline>=1.7.1 in c:\users\ned06\appdata\local
\continuum\anaconda2\lib\site-packages (from dill<=0.2.8.2,>=0.2.6->apache-beam)
(2.1)
Requirement already satisfied: docopt in c:\users\ned06\appdata\local\continuum\
anaconda2\lib\site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in c:\users\ned06\appdata\local\c
ontinuum\anaconda2\lib\site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam) (2.1
8.4)
Requirement already satisfied: enum34>=1.0.4 in c:\users\ned06\appdata\local\con
tinuum\anaconda2\lib\site-packages (from grpcio<2,>=1.8->apache-beam) (1.1.6)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in c:\users\n
ed06\appdata\local\continuum\anaconda2\lib\site-packages (from mock<3.0.0,>=1.0.
1->apache-beam) (1.0.2)
Requirement already satisfied: pbr>=0.11 in c:\users\ned06\appdata\local\continu
um\anaconda2\lib\site-packages (from mock<3.0.0,>=1.0.1->apache-beam) (4.2.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2
.1.0->apache-beam) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in c:\users\ned06\appdata\local\co
ntinuum\anaconda2\lib\site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->a
pache-beam) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in c:\users\ned06\appdata\l
ocal\continuum\anaconda2\lib\site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2
.1.0->apache-beam) (1.22)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\ned06\appdata\loca
l\continuum\anaconda2\lib\site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.
0->apache-beam) (2018.4.16)
Building wheels for collected packages: pyvcf
Running setup.py bdist_wheel for pyvcf ... error
Complete output from command C:\Users\ned06\AppData\Local\Continuum\anaconda2\
python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\ned06\\appdat
a\\local\\temp\\pip-install-qtjnbs\\pyvcf\\setup.py';f=getattr(tokenize, 'open',
open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code
, __file__, 'exec'))" bdist_wheel -d c:\users\ned06\appdata\local\temp\pip-wheel
-jrn5sw --python-tag cp27:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\vcf
copying vcf\filters.py -> build\lib.win32-2.7\vcf
copying vcf\model.py -> build\lib.win32-2.7\vcf
copying vcf\parser.py -> build\lib.win32-2.7\vcf
copying vcf\sample_filter.py -> build\lib.win32-2.7\vcf
copying vcf\utils.py -> build\lib.win32-2.7\vcf
copying vcf\__init__.py -> build\lib.win32-2.7\vcf
creating build\lib.win32-2.7\vcf\test
copying vcf\test\prof.py -> build\lib.win32-2.7\vcf\test
copying vcf\test\test_vcf.py -> build\lib.win32-2.7\vcf\test
copying vcf\test\__init__.py -> build\lib.win32-2.7\vcf\test
running egg_info
writing requirements to PyVCF.egg-info\requires.txt
writing PyVCF.egg-info\PKG-INFO
writing top-level names to PyVCF.egg-info\top_level.txt
writing dependency_links to PyVCF.egg-info\dependency_links.txt
writing entry points to PyVCF.egg-info\entry_points.txt
reading manifest file 'PyVCF.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'PyVCF.egg-info\SOURCES.txt'
copying vcf\cparse.pyx -> build\lib.win32-2.7\vcf
running build_ext
cythoning vcf/cparse.pyx to vcf\cparse.c
warning: vcf\cparse.pyx:16:14: Non-trivial type declarators in shared declarat
ion (e.g. mix of pointers and values). Each pointer declaration should be on its
own line.
warning: vcf\cparse.pyx:16:21: Non-trivial type declarators in shared declarat
ion (e.g. mix of pointers and values). Each pointer declaration should be on its
own line.
warning: vcf\cparse.pyx:16:27: Non-trivial type declarators in shared declarat
ion (e.g. mix of pointers and values). Each pointer declaration should be on its
own line.
warning: vcf\cparse.pyx:16:40: Non-trivial type declarators in shared declarat
ion (e.g. mix of pointers and values). Each pointer declaration should be on its
own line.
building 'vcf.cparse' extension
error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpytho
n27
----------------------------------------
Failed building wheel for pyvcf
Running setup.py clean for pyvcf
Failed to build pyvcf
Installing collected packages: pyvcf, apache-beam
Running setup.py install for pyvcf ... error
Complete output from command C:\Users\ned06\AppData\Local\Continuum\anaconda
2\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\ned06\\appd
ata\\local\\temp\\pip-install-qtjnbs\\pyvcf\\setup.py';f=getattr(tokenize, 'open
', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(co
de, __file__, 'exec'))" install --record c:\users\ned06\appdata\local\temp\pip-r
ecord-3ugbtb\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\vcf
copying vcf\filters.py -> build\lib.win32-2.7\vcf
copying vcf\model.py -> build\lib.win32-2.7\vcf
copying vcf\parser.py -> build\lib.win32-2.7\vcf
copying vcf\sample_filter.py -> build\lib.win32-2.7\vcf
copying vcf\utils.py -> build\lib.win32-2.7\vcf
copying vcf\__init__.py -> build\lib.win32-2.7\vcf
creating build\lib.win32-2.7\vcf\test
copying vcf\test\prof.py -> build\lib.win32-2.7\vcf\test
copying vcf\test\test_vcf.py -> build\lib.win32-2.7\vcf\test
copying vcf\test\__init__.py -> build\lib.win32-2.7\vcf\test
running egg_info
writing requirements to PyVCF.egg-info\requires.txt
writing PyVCF.egg-info\PKG-INFO
writing top-level names to PyVCF.egg-info\top_level.txt
writing dependency_links to PyVCF.egg-info\dependency_links.txt
writing entry points to PyVCF.egg-info\entry_points.txt
reading manifest file 'PyVCF.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'PyVCF.egg-info\SOURCES.txt'
copying vcf\cparse.pyx -> build\lib.win32-2.7\vcf
running build_ext
skipping 'vcf\cparse.c' Cython extension (up-to-date)
building 'vcf.cparse' extension
error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpyt
hon27
----------------------------------------
Command "C:\Users\ned06\AppData\Local\Continuum\anaconda2\python.exe -u -c "impo
rt setuptools, tokenize;__file__='c:\\users\\ned06\\appdata\\local\\temp\\pip-in
stall-qtjnbs\\pyvcf\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=
f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))"
install --record c:\users\ned06\appdata\local\temp\pip-record-3ugbtb\install-rec
ord.txt --single-version-externally-managed --compile" failed with error code 1
in c:\users\ned06\appdata\local\temp\pip-install-qtjnbs\pyvcf\

Towards the end of your error log:
error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27
This suggests that your problem stems from an out-of-date/lack of Microsoft Visual C++. You can install it here.

The Python SDK supports Python 3.6, 3.7, and 3.8. Beam 2.24.0 was the last release with support for Python 2.7 and 3.5.
# if you are still working with pathon2.7 try this,
pip install apache-beam==2.24.0
I would like to give a suggestion for you .Try use python3 with the virtual environment
# Check your version
python --version
# upgrade pip
pip install --upgrade pip
#create virtual env
python -m venv myenv
# activate your virtual env
source myenv/bin/activate
# install beam
pip3 install apache-beam==2.27.0 or
pip install apache-beam==2.27.0

Unable to insall Scrapy : "error: command 'gcc' failed with exit status 1"?

I am trying to install Scrapy using following command:
$ sudo pip install scrapy
But after this, it shows following output with an error:
Collecting scrapy
Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /usr/local/lib/python3.2/dist-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): queuelib in /usr/local/lib/python3.2/dist-packages (from scrapy)
Collecting pyOpenSSL (from scrapy)
Downloading pyOpenSSL-0.15.1-py2.py3-none-any.whl (102kB)
100% |████████████████████████████████| 106kB 49kB/s
Collecting w3lib>=1.8.0 (from scrapy)
Downloading w3lib-1.12.0-py2.py3-none-any.whl
Collecting lxml (from scrapy)
Downloading lxml-3.4.4.tar.gz (3.5MB)
100% |████████████████████████████████| 3.5MB 17kB/s
Collecting Twisted>=10.0.0 (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /usr/local/lib/python3.2/dist-packages (from scrapy)
Collecting service-identity (from scrapy)
Downloading service_identity-14.0.0-py2.py3-none-any.whl
Collecting cryptography>=0.7 (from pyOpenSSL->scrapy)
Downloading cryptography-1.1.tar.gz (348kB)
100% |████████████████████████████████| 352kB 404kB/s
Collecting zope.interface>=4.0.2 (from Twisted>=10.0.0->scrapy)
Collecting characteristic>=14.0.0 (from service-identity->scrapy)
Downloading characteristic-14.3.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->scrapy)
Downloading pyasn1_modules-0.0.8-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /usr/local/lib/python3.2/dist-packages (from service-identity->scrapy)
Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /usr/local/lib/python3.2/dist-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python3.2/dist-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): enum34 in /usr/local/lib/python3.2/dist-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Requirement already satisfied (use --upgrade to upgrade): ipaddress in /usr/local/lib/python3.2/dist-packages (from cryptography>=0.7->pyOpenSSL->scrapy)
Collecting cffi>=1.1.0 (from cryptography>=0.7->pyOpenSSL->scrapy)
Downloading cffi-1.3.0.tar.gz (347kB)
100% |████████████████████████████████| 348kB 9.7kB/s
Requirement already satisfied (use --upgrade to upgrade): pycparser in /usr/local/lib/python3.2/dist-packages (from cffi>=1.1.0->cryptography>=0.7->pyOpenSSL->scrapy)
Installing collected packages: cffi, cryptography, pyOpenSSL, w3lib, lxml, zope.interface, Twisted, characteristic, pyasn1-modules, service-identity, scrapy
Running setup.py install for cffi
Complete output from command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-mdfh82/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-lluuq6-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-i686-3.2
creating build/lib.linux-i686-3.2/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-i686-3.2/cffi
copying cffi/gc_weakref.py -> build/lib.linux-i686-3.2/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-i686-3.2/cffi
copying cffi/lock.py -> build/lib.linux-i686-3.2/cffi
copying cffi/api.py -> build/lib.linux-i686-3.2/cffi
copying cffi/__init__.py -> build/lib.linux-i686-3.2/cffi
copying cffi/model.py -> build/lib.linux-i686-3.2/cffi
copying cffi/ffiplatform.py -> build/lib.linux-i686-3.2/cffi
copying cffi/cparser.py -> build/lib.linux-i686-3.2/cffi
copying cffi/verifier.py -> build/lib.linux-i686-3.2/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-i686-3.2/cffi
copying cffi/recompiler.py -> build/lib.linux-i686-3.2/cffi
copying cffi/commontypes.py -> build/lib.linux-i686-3.2/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-i686-3.2/cffi
copying cffi/vengine_gen.py -> build/lib.linux-i686-3.2/cffi
copying cffi/_cffi_include.h -> build/lib.linux-i686-3.2/cffi
copying cffi/parse_c_type.h -> build/lib.linux-i686-3.2/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-i686-3.2
creating build/temp.linux-i686-3.2/c
gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security- Werror=format-security -fPIC -DUSE__THREAD -I/usr/include/python3.2mu -c c/_cffi_backend.c -o build/temp.linux-i686-3.2/c/_cffi_backend.o
c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-mdfh82/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-lluuq6-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-mdfh82/cffi
FAIL
I searched for any solution and on the basis of one of the recommendation I also tried to install python-dev using command :
$ sudo apt-get install python-dev
But this too showed that python-dev is the newest version:
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-dev is already the newest version.
Can anybody help me in solving this issue?

Your problem is:
c/_cffi_backend.c:14:17: error: ffi.h: No such file or directory
Try to search libffi-devel or libiffi-dev using to confirm the exact package name:
sudo apt-cache search libiffi
Then install the package using:
sudo apt-get install libiffi-dev
Hope it will help