Develop Reference

Create a Distributed Package that uses lxml

I've created a python package that uses lxml.
In setup.py I have defined:
packages=find_packages(), # Required
python_requires='>=3.8, <4',
install_requires=[
'typer[all]',
'packaging',
'lxml'
]
Tests, packaging and deployments work just fine.
However, after uploading the package to my company's PyPi repository, I'm trying to install the package with:
pip install -i <mypypirepo> <mypackage>
I get the a long error including:
fatal error C1083: Cannot open include file: 'libxml/xpath.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit code 2
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\python39\python.exe'
While trying to install manually the lxml I get the same exact error.
I found solutions on stackoverflow that talk about manually downloading a compatible wheel version of lxml and then installing it with pip.
However, the lxml is a part of the package that should be easily installed by the end-user.
Is there any way to solve the issue without forcing the user to manually install lxml?
Note: This issue only started happening after upgrading python from 3.8 to 3.9.

As of this writing 2020-10-12 there is no lxml wheel for python 3.9 on PyPI.
If using windows like I do there are unofficial wheels here:
https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

Unable to install psycopg2 on OSX Anaconda

This is what I am getting:
Adnans-MBP:~ AdnanAhmad$ easy_install psycopg2
Searching for psycopg2
Reading https://pypi.python.org/simple/psycopg2/
Downloading https://pypi.python.org/packages/7b/a8/dc2d50a6f37c157459cd18bab381c8e6134b9381b50fbe969997b2ae7dbc/psycopg2-2.6.2.tar.gz#md5=4a392949ba31a378a18ed3e775a4693f
Best match: psycopg2 2.6.2
Processing psycopg2-2.6.2.tar.gz
Writing /var/folders/d3/t5c58nx570ggzzknhm4zpjnw0000gn/T/easy_install-66rCLK/psycopg2-2.6.2/setup.cfg
Running psycopg2-2.6.2/setup.py -q bdist_egg --dist-dir /var/folders/d3/t5c58nx570ggzzknhm4zpjnw0000gn/T/easy_install-66rCLK/psycopg2-2.6.2/egg-dist-tmp-Vu8qsu
In file included from psycopg/psycopgmodule.c:27:
./psycopg/psycopg.h:31:10: fatal error: 'libpq-fe.h' file not found
#include <libpq-fe.h>
^
1 error generated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
I am using PostgreSql.app

well, since you are using anaconda anyways, might as well take advantage of their binary packages! They have one for PostgreSQL! just type
conda install psycopg2

Error installing Willie irc bot for Python

Trying to install this Python IRC bot called Willie. This is what happens when I run setup.py:
C:\Python34\python.exe "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.5.1\helpers\pycharm\pycharm_setup_runner.py" C:\Users\Quibbles\Documents\willie-5.3.0\setup.py
Testing started at 1:35 PM ...
running pycharm_test
Searching for pygeoip
Best match: pygeoip 0.3.2
Processing pygeoip-0.3.2-py3.4.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\pygeoip-0.3.2-py3.4.egg
Searching for pyenchant
Best match: pyenchant 1.6.6
Processing pyenchant-1.6.6-py3.4-win32.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\pyenchant-1.6.6-py3.4-win32.egg
Searching for praw
Best match: praw 2.1.21
Processing praw-2.1.21-py3.4.egg
Using c:\users\quibbles\documents\willie-5.3.0\.eggs\praw-2.1.21-py3.4.egg
Searching for lxml
Reading https://pypi.python.org/simple/lxml/
Best match: lxml 3.4.4
Downloading https://pypi.python.org/packages/source/l/lxml/lxml-3.4.4.tar.gz#md5=a9a65972afc173ec7a39c585f4eea69c
Processing lxml-3.4.4.tar.gz
Writing C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\setup.cfg
Running lxml-3.4.4\setup.py -q bdist_egg --dist-dir C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\egg-dist-tmp-0expfysa
Building lxml version 3.4.4.
Building without Cython.
ERROR: b"'xslt-config' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n"
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
C:\Python34\lib\distutils\dist.py:260: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
cl : Command line warning D9025 : overriding '/W3' with '/w'
lxml.etree.c
C:\Users\Quibbles\AppData\Local\Temp\easy_install-p_x9naya\lxml-3.4.4\src\lxml\includes\etree_defs.h(14) : fatal error C1083: Cannot open include file: 'libxml/xmlversion.h': No such file or directory
error: Setup script exited with error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\BIN\\cl.exe' failed with exit status 2
Process finished with exit code 1
What am I doing wrong? I downloaded the source code from https://github.com/embolalia/willie/releases/tag/v5.3.0 and tried running setup.py in PyCharm.

It should be fairly obvious by the error messages:
** make sure the development packages of libxml2 and libxslt are installed **
You need to have the dev libxml2 and libxslt installed. It appears either you have the wrong versions perhaps, or don't have them installed properly.

I'L'I's answer gives you a hint of where to start searching next, but isn't a useful answer.
After several attempts on my own, I found that to get the lxml library installed, I needed to do the following:
First, download the lxml library from http://www.lfd.uci.edu/~gohlke/pythonlibs/
This is necessary because the PiPy archives (https://pypi.python.org/pypi/lxml/3.4.4) are way out of date, only giving you the option to install for Python 3.2 for the 3+ series, which of course is useless if you're on 3.4. There's also issues if you're using 64-bit. They may be sufficient if you're using 2.7, though.
Then use the instructions here: How do I install a Python package with a .whl file? to install the .whl file.
However that only fixes half the problem. The willie package appears to be broken in other ways (on Windows) because of trying to reference a unix-like /tmp directory (absolute path) instead of the proper Windows Temp directory.
This issue describes the problem, though no fix has yet been made: https://github.com/embolalia/willie/issues/811
It indicates that it's a Windows-specific bug in the installer. To work around that, download the .tar.gz from https://pypi.python.org/pypi/willie/5.3.0
Unpack it. Go to the willie-5.3.0\willie.egg-info directory, and edit the SOURCES.txt file. The fifth line is /tmp/tmpeAhjCF/willie. Delete that line.
Repackage the directory into a .tar.gz archive. Install via pip install .\willie-5.3.0.tar.gz.
Now it's installed, and you're ready for the next round of problems of actually getting it to run on Windows. Since I haven't actually succeeded in doing that yet, that's as far as I can help for right now.

Can't install package inside virtualenv

I have the following environment:
Linux Mint17.1 which comes pre-installed with python 2.7.6.
I setup an altinstall for python 3.3.5
Using python 3.3 I created a virtualenv
I am trying to install TA-Lib in the virtual env but am not having any success.
If I try to install with:
./easy_install TA-Lib
I get the following error:
Searching for TA-Lib
Reading https://pypi.python.org/simple/TA-Lib/
Reading http://github.com/mrjbq7/ta-lib
Reading https://github.com/mrjbq7/ta-lib/releases
Best match: TA-Lib 0.4.8
Downloading https://github.com/mrjbq7/ta-lib/archive/TA_Lib-0.4.8.zip
Processing TA_Lib-0.4.8.zip
Writing /tmp/easy_install-67a3vk/ta-lib-TA_Lib-0.4.8/setup.cfg
Running ta-lib-TA_Lib-0.4.8/setup.py -q bdist_egg --dist-dir/tmp/easy_install-67a3vk/ta-lib-TA_Lib-0.4.8/egg-dist-tmp-w24p9n
/tmp/easy_install-67a3vk/ta-lib-TA_Lib-0.4.8/setup.py:49: UserWarning: Cannot find ta-lib library, installation may fail.
warnings.warn('Cannot find ta-lib library, installation may fail.')
talib/common.c:314:28: fatal error: ta-lib/ta_defs.h: No such file or directory
#include "ta-lib/ta_defs.h"
^
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
If I try to install with:
./pip install https://github.com/mrjbq7/ta-lib/archive/TA_Lib-0.4.8.tar.gz
I get the following error:
Downloading/unpacking https://github.com/mrjbq7/ta-lib/archive/TA_Lib-0.4.8.tar.gz
Downloading TA_Lib-0.4.8.tar.gz (unknown size): 544kB downloaded
Running setup.py (path:/tmp/pip-rxvyyh-build/setup.py) egg_info for package from https://github.com/mrjbq7/ta-lib/archive/TA_Lib-0.4.8.tar.gz
/tmp/pip-rxvyyh-build/setup.py:49: UserWarning: Cannot find ta-lib library, installation may fail.
warnings.warn('Cannot find ta-lib library, installation may fail.')
Installing collected packages: TA-Lib
Running setup.py install for TA-Lib
...removed to make post shorter...
compilation terminated.
/tmp/pip-rxvyyh-build/setup.py:49: UserWarning: Cannot find ta-lib library, installation may fail.
warnings.warn('Cannot find ta-lib library, installation may fail.')
error: command 'gcc' failed with exit status 1
So basically both approaches result in a final error of:
command 'gcc' failed with exit status 1
I have no problem installing other packages such as numpy, pandas, matplotlib etc in the virtualenv. It is just TA-Lib that is not working.

You don't have installed underlying ta-lib library. You can find more info on the project site https://github.com/mrjbq7/ta-lib :
Sometimes installation will produce build errors like this:
func.c:256:28: fatal error: ta-lib/ta_libc.h: No such file or directory compilation terminated.
This typically means that it can't find the underlying TA-Lib library, a dependency which needs to be installed.

Python's “easy_install” on Windows - HtmlCSSInclude

I just installed Python 2.7 and easy_install-2.7.exe on my Windows 2003 server, then set %PATH% to C:\Python27;C:\Python27\Scripts;. Then I tried to install a python package with easy_install, but easy_install failed. The failure follows:
C:\>easy_install HtmlCssInclude
Searching for HtmlCssInclude
Reading http://pypi.python.org/simple/HtmlCssInclude/
Best match: HtmlCssInclude 0.1.2
Downloading http://pypi.python.org/packages/source/H/HtmlCssInclude/HtmlCssInclu
de-0.1.2.tar.gz#md5=0b4438bf62ae66842a514dcff67c0cd7
Processing HtmlCssInclude-0.1.2.tar.gz
Running HtmlCssInclude-0.1.2\setup.py -q bdist_egg --dist-dir c:\docume~1\admini
~1\locals~1\temp\1\easy_install-qpossf\HtmlCssInclude-0.1.2\egg-dist-tmp-haku7q
C:\Python27\Lib\distutils\dist.py:267: UserWarning: Unknown distribution option:
'auther'
warnings.warn(msg)
warning: no files found matching 'testdata'
error: Setup script exited with error: can't copy 'sample.css': doesn't exist or
not a regular file
Anybody please help me.

I tried editing this package and it installed.
Downloaded the package and edited HtmlCssinclude.py. In addition i found the reference to 'sample.css' and 'testdata' is not necessary for this package installation. So i also removed those reference. This help me to install package from local folder.