Which PyPI index does Pip point to? - python

Ubuntu 20.04.1 LTS | Python 3.8.5 | Pip 20.0.2 (via python3-pip)
I accidentally overwrote the global PyPI index in my laptop with a custom index. (It was meant to be set only within a virtual environment. In case it's useful, I was using AWS CodeArtifact.)
I say this after running pip3 config -v list and seeing global.index-url assigned the other URL as well as ~/.config/.pip/pip.conf listing the same.
In an attempt to set things right, I did the following:
Set pip3's global.index-url to "https://pypi.python.org/pypi", but when I did a test to see if it worked, it stopped halfway while trying to install requests in a virtual environment (using its pip) and said some dependencies were not satisfied.
I then moved ~/.config/.pip to a backup, uninstalled python3-pip and reinstalled the package in the hope that the config would be written anew. While a new ~/config/.pip was not created, I was able to install requests into the virtual environment without any issue.
My confusion stems from the fact that I don't know where pip is picking things up from. None of the files listed below exist!
$ pip config -v list
For variant 'global', will try loading '/etc/xdg/xdg-ubuntu/pip/pip.conf'
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'user', will try loading '/home/<user>/.pip/pip.conf'
For variant 'user', will try loading '/home/<user>/.config/pip/pip.conf'
For variant 'site', will try loading '/home/<user>/test_env/venv/pip.conf'
The output is slightly different when I use pip3 with the virtual environment deactivated.
$ pip3 config -v list
For variant 'global', will try loading '/etc/xdg/xdg-ubuntu/pip/pip.conf'
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'user', will try loading '/home/<user>/.pip/pip.conf'
For variant 'user', will try loading '/home/<user>/.config/pip/pip.conf'
For variant 'site', will try loading '/usr/pip.conf'
Could someone please tell me how this all works?

https://pypi.org/simple/ packageX
let me know if its right ?
using
pip install paxckageX --verbose
not sure its right nowadays but here:
Python can't find the file pip.conf
and here
pip user guide Configuration :
Configuration
Config file
pip allows you to set all command line option defaults in a standard ini style config file.
The names and locations of the configuration files vary slightly across platforms. You may have per-user, per-virtualenv or global (shared amongst all users) configuration:
Per-user:
On Unix the default configuration file is: $HOME/.config/pip/pip.conf which respects the XDG_CONFIG_HOME environment variable.
On macOS the configuration file is $HOME/Library/Application Support/pip/pip.conf if directory $HOME/Library/Application Support/pip exists else $HOME/.config/pip/pip.conf.
On Windows the configuration file is %APPDATA%\pip\pip.ini.
There is also a legacy per-user configuration file which is also respected. To find its location:
On Unix and macOS the configuration file is: $HOME/.pip/pip.conf
On Windows the configuration file is: %HOME%\pip\pip.ini
You can set a custom path location for this config file using the environment variable PIP_CONFIG_FILE.
Inside a virtualenv:
On Unix and macOS the file is $VIRTUAL_ENV/pip.conf
On Windows the file is: %VIRTUAL_ENV%\pip.ini
Global:
On Unix the file may be located in /etc/pip.conf. Alternatively it may be in a “pip” subdirectory of any of the paths set in the environment variable XDG_CONFIG_DIRS (if it exists), for example /etc/xdg/pip/pip.conf.
On macOS the file is: /Library/Application Support/pip/pip.conf
On Windows XP the file is: C:\Documents and Settings\All Users\Application Data\pip\pip.ini
On Windows 7 and later the file is hidden, but writeable at C:\ProgramData\pip\pip.ini
Global configuration is not supported on Windows Vista.
The global configuration file is shared by all Python installations.
If multiple configuration files are found by pip then they are combined in the following order:
The global file is read
The per-user file is read
The virtualenv-specific file is read
Each file read overrides any values read from previous files, so if the global timeout is specified in both the global file and the per-user file then the latter value will be used.
The names of the settings are derived from the long command line option, e.g. if you want to use a different package index (--index-url) and set the HTTP timeout (--default-timeout) to 60 seconds your config file would look like this:
[global]
timeout = 60
index-url = https://download.zope.org/ppix
Each subcommand can be configured optionally in its own section so that every global setting with the same name will be overridden; e.g. decreasing the timeout to 10 seconds when running the freeze (pip freeze) command and using 60 seconds for all other commands is possible with:
[global]
timeout = 60
[freeze]
timeout = 10
Boolean options like --ignore-installed or --no-dependencies can be set like this:
[install]
ignore-installed = true
no-dependencies = yes
To enable the boolean options --no-compile, --no-warn-script-location and --no-cache-dir, falsy values have to be used:
[global]
no-cache-dir = false
[install]
no-compile = no
no-warn-script-location = false
For options which can be repeated like --verbose and --quiet, a non-negative integer can be used to represent the level to be specified:
[global]
quiet = 0
verbose = 2
It is possible to append values to a section within a configuration file such as the pip.ini file. This is applicable to appending options like --find-links or --trusted-host, which can be written on multiple lines:
[global]
find-links =
http://download.example.com
[install]
find-links =
http://mirror1.example.com
http://mirror2.example.com
trusted-host =
mirror1.example.com
mirror2.example.com
This enables users to add additional values in the order of entry for such command line arguments.
Environment Variables
pip’s command line options can be set with environment variables using the format PIP_<UPPER_LONG_NAME> . Dashes (-) have to be replaced with underscores (_).
For example, to set the default timeout:
Unix/macOS
export PIP_DEFAULT_TIMEOUT=60
Windows
This is the same as passing the option to pip directly:
Unix/macOS
python -m pip --default-timeout=60 [...]
Windows
For command line options which can be repeated, use a space to separate multiple values. For example:
Unix/macOS
export PIP_FIND_LINKS="http://mirror1.example.com http://mirror2.example.com"
Windows
is the same as calling:
Unix/macOS
python -m pip install --find-links=http://mirror1.example.com --find-links=http://mirror2.example.com
Windows
Options that do not take a value, but can be repeated (such as --verbose) can be specified using the number of repetitions, so:
export PIP_VERBOSE=3
is the same as calling:
pip install -vvv
Note
Environment variables set to be empty string will not be treated as false. Please use no, false or 0 instead.
Config Precedence
Command line options have precedence over environment variables, which have precedence over the config file.
Within the config file, command specific sections have precedence over the global section.
Examples:
--host=foo overrides PIP_HOST=foo
PIP_HOST=foo overrides a config file with [global] host = foo
A command specific section in the config file [<command>] host = bar overrides the option with same name in the [global] config file section

Related

Enable --no-index by default in pip configuration

Is there a way to set --no-index as default behaviour when installing packages with pip (both for Windows and Unix)?
Even though this configuration seems to work...
# /home/myaccount/.config/pip/pip.conf or C:\Users\MyAccount\AppData\Roaming\pip\pip.ini
[global]
find-urls =
/path/to/folder1
/path/to/folder2
no-index = true
I'm not able to find a reference of it in pip configuration doc.
Am I doing it right?
NOTE: I'm working offline and I don't want to add the extra-burden of a local server repository. I'm just installing offline packages by setting find-urls configuration
Any command-line option can be set using the configuration file, following some simple rules to map the switch to a name in the configuration file.
This is explained in the Naming section of the Configuration chapter:
The names of the settings are derived from the long command line option.
As an example, if you want to use a different package index (--index-url) and set the HTTP timeout (--default-timeout) to 60 seconds, your config file would look like this:
[global]
timeout = 60
index-url = https://download.zope.org/ppix
Since --no-index is a boolean flag switch, the Boolean options section in the same chapter explains how to give such switches a value in the configuration file:
Boolean options like --ignore-installed or --no-dependencies can be set like this:
[install]
ignore-installed = true
no-dependencies = yes
Using no-index = true is exactly how you'd use that setting in a config file. Putting it in the [global] section means it'll apply to all your pip commands.
Because command-line options map directly to names in the config file using these simple rules, the configuration chapter doesn't have to cover every single possible config option.

Poetry config resolution order. Do Environment variables superseed poetry config file?

I created a python package with poetry.
I can control poetry's behaviour either with a config file and the config commands
poetry config virtualenvs.in-project true
or with environment variable VIERTUALENVS_IN_PROJECT=false.
If I specify a configuration in both places, which one will be chosen?
Setting export POETRY_VIRTUALENVS_IN_PROJECT=false in the .envrc with direnv has precedence over poetry.toml.

Where is the luigi config file?

I have installed luigi by pip command and I would like to change the port for the web UI. I tried to find the config file but I couldn't. Do I need to create one?
You can start luigid with the --port option.
luigid --port 80
Configuration file locations are:
/etc/luigi/luigi.cfg
luigi.cfg (or its legacy name client.cfg) in
your current working directory
LUIGI_CONFIG_PATH environment variable
in increasing order of preference. You do need to create one. e.g.,
[core]
default-scheduler-host=www.example.com
default-scheduler-port=8088
In spite of the documentation saying otherwise, the port configuration from the config file is not used, at least in some versions or some circumstances
Until this is resolved, you should always use the --port option of luigid:
luigid --port 12345
Also see https://github.com/spotify/luigi/issues/2235
For other configuration options a config file should be used. See https://luigi.readthedocs.io/en/stable/configuration.html
For a configuration global to the host you can create a file:
/etc/luigi/luigi.cfg
Make sure it is readable by the user that runs luigid and luig.
Alternatively a local configuration file that will be recognized is
luigi.cfg
which you would have to create in the current working directory.
If you want a custom config file location you could set the environment variable LUIGI_CONFIG_PATH to the full path of your config file.

How to add passenv to tox.ini without editing the file but by running tox in virtualenv shell nature script in Jenkins behind proxy (python)

I am trying to run python unit tests in jenkins using tox's virtualenv. I am behind a proxy so I need to pass HTTP_PROXY and HTTPS_PROXY to tox, else it has problems with downloading stuff.
I found out that I can edit tox.ini and add passenv=HTTP_PROXY HTTPS_PROXY under [testenv], and than using the Create/Update Text File Plugin I can override the tox.ini(as a build step) whenever Jenkins job fetches the original file from repository. This way I can manually copy content of tox.ini from workspace, add the passenv= line below [testenv] and update the file with the plugin mentioned above. But this is not the proper solution. I don't want to edit the tox.ini file this way, because the file is constantly updated. Using this solution would force me to update the tox.ini content inside the plugin everytime it is changed on the git repository and I want the process of running unit tests to be fully automated. And no, I can't edit the original file on git repository.
So is there a way that I can pass the passenv = HTTP_PROXY HTTPS_PROXY in the Shell nature command? This is how my command in Virtualenv Builder looks like:
pip install -r requirements.txt -r test-requirements.txt
pip install tox
tox --skip-missing-interpreter module/tests/
I want to do something like this:
tox --skip-missing-interpreter --[testenv]passenv=HTTP_PROXY HTTPS_PROXY module/tests
How to solve this?
NOTE:
I think there might be a solution with using the {posargs}, but I see that there is a line in the original tox.ini containing that posargs already: python setup.py testr --testr-args='{posargs}' help...
I thought about a workaround:
Create a build step in Jenkins job, that will execute bash script, that will open the tox.ini find line [testenv] and input one line below passenv = HTTP_PROXY HTTPS_PROXY. That would solve the problem. I am working on this right now but anyway if You guys know a better solution please let me know. cheers
Ok so this is the solution:
Add a build step Execute shell
Input this: sed -i.bak '/\[testenv\]/a passenv = HTTP_PROXY HTTPS_PROXY' tox.ini This will update the tox.ini file (input the desired passenv line under [testenv] and save changes). And create a tox.ini.bak backup file with the original data before sed's change.

django i18n: Make sure you have GNU gettext tools

I try django-admin.py makemessages -l zh_CN but has error :
CommandError: Can't find msguniq. Make sure you have GNU gettext tools 0.15 or newer installed.
after I use brew install gettext,it still get wrong.
Do I need to do something? here is my terminal screenshot
Please guide me thank you.
In Ubuntu:
$ sudo apt-get install gettext
For Mac users, after installing Homebrew and gettext as #Louis Barranqueiro says (steps 1 and 2):
Install Homebrew : /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Install GNU gettext : brew install gettext
You shouldn't use brew link gettext --force in step 3, because it is risky (as Brew advises if you try). A better workaround is to set a new PATH variable for your virtual environment. So, in the postactivate file, which is located in the bin folder of your virtual environment folder, type:
export TEMP_PATH=$PATH
export PATH=$PATH:/usr/local/Cellar/gettext/0.19.7/bin
Note that you have to replace 0.19.7 by the version that is installed in your machine.
And in your predeactivate file, which is located in the same folder of postactivate file, type:
export PATH=$TEMP_PATH
unset TEMP_PATH
Now you can use the python manage.py makemessages -l <desired_language> without worries. :)
Cheers.
This procedure worked for me (OSX 10.11.2 - python v3.5 and Django 1.8)
It should work with your configuration.
Install gettext GNU tools with Homebrew using Terminal
Install Homebrew : /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Install GNU gettext : brew install gettext
Create symlink : brew link gettext --force
This solution worked for me ( win. 7, 8 and 10 )
You need to download two folders:
gettext-runtime_0.18.1.1-2_win32
gettext-tools-dev_0.18.1.1-2_win32
You can find them here.
After you download them, unzip them and add the directory of the bin file of the both folders to the system variables PATH of your pc.
You will also need a file named libstdc++-6.dll download it from here and place it in your system directory. You will find adequate details on system directory here.
And that’s it. Hope it is useful for you.
Just below solution solved my problem. I am using Windows 10 64bit
1- Go to this link :
https://mlocati.github.io/articles/gettext-iconv-windows.html
2- Download 32 or 64 bit shared and static windows installation files
3-Install both of files
4-Restart your computer
Hi first of all make sure that your virtual environment is not in your root folder. I think it's better practice to keep your virtual environment outside of the root folder. Obviously make sure your environment is activated. Of course make sure you have gettext installed as well.
If your env folder is in your root folder
To test this just make sure you add {% load i18n %} in all your templates, choose a template and do something like this:
<h1>{% trans 'My Test to be translated' %}</h1>
Now run this command
django-admin makemessages -l 'zh_CN' -i your_venv
(Make sure you replace your_venv to the name of your virtual environment.
After you run the above command, you should get this in your terminal.
processing locale zh_CN
Now you should have a locale folder like this: locale/cn/LC_MESSAGES/django.po
Now you will need to compile the messages. Run this command
django-admin compilemessages
In your locale folder, now you should get you should see django.mo file as well, but you will notice the difference in django.po file. Just add your translation there, and you can test again by setting your en language to LANGUAGE_CODE = 'zh_CN' then just refresh and test the h1 string will be translated to Chinese.
In order for the above to work in your settings.py ensure you have this here, for now most important is the LOCALE_PATHS, but please check if this ('zh_CN', _('Chinese')), is correct
LANGUAGES = [
('zh_CN', _('Chinese')),
('en', _('English')),
]
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
LOCALE_PATHS = [
os.path.join(BASE_DIR, 'locale'),
]
In this reply, the most important part is to realize where your virtual environment is located. Reason why you get all these errors.
Please make sure you refer to this video here, it's a great tutorial.
https://www.youtube.com/watch?v=xI97sLMd1rM
#max-malysh's answer solved it for me —without touching system files.
Copy and run each of the following:
brew install gettext
GETTEXT_PATH="/usr/local/Cellar/gettext/0.19.8.1/bin"
FILE="venv/bin/activate"
echo "" >> $FILE
echo "export PATH=\$PATH:$GETTEXT_PATH" >> $FILE
source venv/bin/activate
GETTEXT_PATH="/usr/local/Cellar/gettext/0.19.8.1/bin" stores gettext_path in a shell variable —adapt the version number according to what brew install gettext
FILE="venv/bin/activate" stores the path to the venv shell script
echo "" >> $FILE adds an empty line at the end of the to make sure the next command is on its own line
echo "export PATH=\$PATH:$GETTEXT_PATH" >> $FILE adds a command to the venv shell script; this command adds the path to gettext binaries to the global $PATH variable, so that they are used before OS binaries.
source venv/bin/activate runs the venv shell script so that variables are properly set. You can run this command more than once.
If you're using Docker just simply run below command:
apt-get update
Then:
apt-get install gettext
If you use fish shell, another way around is to add this path to $fish_user_paths.This variable is prepended to $PATH, so you don't have to set it in all your projects.
You can do it with the following command line :
set -U fish_user_paths /usr/local/Cellar/gettext/0.19.8.1/bin $fish_user_paths
Remember to replace 0.19.8.1 with your gettext version.
This sets $fish_user_paths as a Universal Variable. Here's what help says about Universal Variables :
A universal variable is a variable whose value is shared across all
instances of fish, now and in the future – even after a reboot. You
can make a variable universal with set -U
So setting this variable in your shell once (no need to do it in a config file) will save it even after logging out or rebooting.
that works for windows users.
i am using django 2
access this https://mlocati.github.io/articles/gettext-iconv-windows.html
2 download the static version for your system
after downloaded execute the setup.
restart your pc and it will work.
That is all. THANKS.!!!!
the problem is hinted in the output from brew...
it has installed the GNU gettext but hasn't linked it into your bin directory because OSX already provides a different version of gettext...
so Django doesn't know to run the version you installed from brew.
apparently brew is too cautious here though and you should just link it https://stackoverflow.com/a/9787791/202168

Categories

Resources