I'm trying to use pyproject.toml to exclude the venv/ directory. But it is not recognising the option.
[tool.bandit]
exclude = "/venv"
[tool.black]
exclude = "(venv)"
[tool.isort]
profile = "black"
skip = "venv"
balanced_wrapping = true
atomic = true
If I use the CLI option like so:
$ bandit -v -r . --exclude "/venv"
the directory is excluded. But if I just run bandit, it doesn't exclude the directory even though I have it in the pyproject.toml.
My bandit version is: 1.7.1.
exclude did not work for me, so I looked through official docs and found this:
We can specify dirs (and files as well) that we want to exclude in a list format
pyproject.toml:
[tool.bandit]
exclude_dirs = ["venv",]
From this documentation:
"Also you can configure bandit via pyproject.toml file. In this case you would explicitly specify the path to configuration via -c too."
Therefore, CLI option would look like this:
bandit -v -r . -c "pyproject.toml"
(will work without quotes as well)
I've never used bandit before, so if I got your question wrong - please feel free to write back, we will figure that out :D
To exclude directory venv, this command works fine for me :
bandit -r . -x */venv/*
Related
Is there a way to set --no-index as default behaviour when installing packages with pip (both for Windows and Unix)?
Even though this configuration seems to work...
# /home/myaccount/.config/pip/pip.conf or C:\Users\MyAccount\AppData\Roaming\pip\pip.ini
[global]
find-urls =
/path/to/folder1
/path/to/folder2
no-index = true
I'm not able to find a reference of it in pip configuration doc.
Am I doing it right?
NOTE: I'm working offline and I don't want to add the extra-burden of a local server repository. I'm just installing offline packages by setting find-urls configuration
Any command-line option can be set using the configuration file, following some simple rules to map the switch to a name in the configuration file.
This is explained in the Naming section of the Configuration chapter:
The names of the settings are derived from the long command line option.
As an example, if you want to use a different package index (--index-url) and set the HTTP timeout (--default-timeout) to 60 seconds, your config file would look like this:
[global]
timeout = 60
index-url = https://download.zope.org/ppix
Since --no-index is a boolean flag switch, the Boolean options section in the same chapter explains how to give such switches a value in the configuration file:
Boolean options like --ignore-installed or --no-dependencies can be set like this:
[install]
ignore-installed = true
no-dependencies = yes
Using no-index = true is exactly how you'd use that setting in a config file. Putting it in the [global] section means it'll apply to all your pip commands.
Because command-line options map directly to names in the config file using these simple rules, the configuration chapter doesn't have to cover every single possible config option.
I want to specify a GitLab job that creates a sphinx html documentation.
I am using a Python 3 alpine image (cannot specify which exactly).
the build stage within my .gitlab-ci.yml looks like this:
pages:
stage: build
tags:
- buildtag
script:
- pip install -U sphinx
- sphinx-build -b html docs/ public/
only:
- master
however, the pipeline fails: sphinx-build: command not found. (same error for make html)
According to This Tutorial, my .gitlab-ci.yml should be more or less correct.
What am I doing wrong? Is this issue related to the alpine image I am using?
As #Yasen correctly noted, the path to sphinx-build was not contained in $PATH. However, adding command in before sphinx-build did not solve the problem for me.
Anyway I found the solution in the the runner logs: The output of pip install -U sphinx produced the following warning:
WARNING: The scripts sphinx-apidoc, sphinx-autogen, sphinx-build and sphinx-quickstart are installed in 'some/path' which is not on PATH.
so I added export PATH="some/path" to the script-step in the .gitlab-ci.yml:
script:
- pip install -U sphinx
- export PATH="some/path"
- sphinx-build -b html docs/ public/
Did the command pip install -U sphinx succeed? (You should be able to tell that from the CI job log.)
If so, you may need to specify the full path to sphinx-build, as Yasen said.
If it did not succeed, you should troubleshoot the installation of Sphinx.
Most likely the reason is that $PATH doesn't contain path to sphinx-build
TL;DR try to use command
Try this:
pages:
stage: build
tags:
- buildtag
script:
- pip install -U sphinx
- command sphinx-build -b html docs/ public/
only:
- master
Explanation
GitLab runners run different way
Since GitLab CI uses runners, runner's shell profile may differ from commonly used.
So, your runner may be configured without declared $PATH to the directory that contains sphinx-build
Zsh/Bash startup files loading order (.bashrc, .zshrc etc.)
See this explanation:
The issue is that Bash sources from a different file based on what kind of shell it thinks it is in. For an “interactive non-login shell”, it reads .bashrc, but for an “interactive login shell” it reads from the first of .bash_profile, .bash_login and .profile (only). There is no sane reason why this should be so; it’s just historical.
What command does mean?
Since we don't know the path where sphinx-build installed, you may use commands like: which, type, etc.
As per this great answer(shell - Why not use "which"? What to use then? - Unix & Linux Stack Exchange, author recommends to use command <name>, or $(command -v <name>)
I am using:
mkdir -p build/python/lib/python3.6/site-packages
pipenv run pip install -r requirements.txt --target build/python/lib/python3.6/site-packages
to create a directory build with everything I need for my python project but I also need to save as much space as possible.
What can I safely remove in order to save space?
Maybe can I do find build -type d -iname "*.dist-info" -exec rm -R {} \; ?
Can I remove *.py if I leave *.pyc?
Thanks
Perhaps platform specific *.exe files, if your project doesn't need to run on Windows:
How to prevent *.exe ...
Delete *.pyc (byte-compiled files), with an impact to load-time: 100% supported, unlike your trick of the reverse: retain just *.pyc (and delete most *.py sources) in some python versions; not safe IMHO but never tried it.
I've been usually installed python packages through pip.
For Google App Engine, I need to install packages to another target directory.
I've tried:
pip install -I flask-restful --target ./lib
but it fails with:
must supply either home or prefix/exec-prefix -- not both
How can I get this to work?
Are you using OS X and Homebrew? The Homebrew python page https://github.com/Homebrew/brew/blob/master/docs/Homebrew-and-Python.md calls out a known issue with pip and a work around.
Worked for me.
You can make this "empty prefix" the default by adding a
~/.pydistutils.cfg file with the following contents:
[install]
prefix=
Edit: The Homebrew page was later changed to recommend passing --prefix on the command line, as discussed in the comments below. Here is the last version which contained that text. Unfortunately this only works for sdists, not wheels.
The issue was reported to pip, which later fixed it for --user. That's probably why the section has now been removed from the Homebrew page. However, the problem still occurs when using --target as in the question above.
I believe there is a simpler solution to this problem (Homebrew's Python on macOS) that won't break your normal pip operations.
All you have to do is to create a setup.cfg file at the root directory of your project, usually where your main __init__.py or executable py file is. So if the root folder of your project is: /path/to/my/project/, create a setup.cfg file in there and put the magic words inside:
[install]
prefix=
OK, now you sould be able to run pip's commands for that folder:
pip install package -t /path/to/my/project/
This command will run gracefully for that folder only. Just copy setup.cfg to whatever other projects you might have. No need to write a .pydistutils.cfg on your home directory.
After you are done installing the modules, you may remove setup.cfg.
On OSX(mac), assuming a project folder called /var/myproject
cd /var/myproject
Create a file called setup.cfg and add
[install]
prefix=
Run pip install <packagename> -t .
Another solution* for Homebrew users is simply to use a virtualenv.
Of course, that may remove the need for the target directory anyway - but even if it doesn't, I've found --target works by default (as in, without creating/modifying a config file) when in a virtual environment.
*I say solution; perhaps it's just another motivation to meticulously use venvs...
I hit errors with the other recommendations around --install-option="--prefix=lib". The only thing I found that worked is using PYTHONUSERBASE as described here.
export PYTHONUSERBASE=lib
pip install -I flask-restful --user
this is not exactly the same as --target, but it does the trick for me in any case.
As other mentioned, this is known bug with pip & python installed with homebrew.
If you create ~/.pydistutils.cfg file with "empty prefix" instruction it will fix this problem but it will break normal pip operations.
Until this bug is officially addressed, one of the options would be to create your own bash script that would handle this case:
#!/bin/bash
name=''
target=''
while getopts 'n:t:' flag; do
case "${flag}" in
n) name="${OPTARG}" ;;
t) target="${OPTARG}" ;;
esac
done
if [ -z "$target" ];
then
echo "Target parameter must be provided"
exit 1
fi
if [ -z "$name" ];
then
echo "Name parameter must be provided"
exit 1
fi
# current workaround for homebrew bug
file=$HOME'/.pydistutils.cfg'
touch $file
/bin/cat <<EOM >$file
[install]
prefix=
EOM
# end of current workaround for homebrew bug
pip install -I $name --target $target
# current workaround for homebrew bug
rm -rf $file
# end of current workaround for homebrew bug
This script wraps your command and:
accepts name and target parameters
checks if those parameters are empty
creates ~/.pydistutils.cfg file with "empty prefix" instruction in it
executes your pip command with provided parameters
removes ~/.pydistutils.cfg file
This script can be changed and adapted to address your needs but you get idea. And it allows you to run your command without braking pip. Hope it helps :)
If you're using virtualenv*, it might be a good idea to double check which pip you're using.
If you see something like /usr/local/bin/pip you've broken out of your environment. Reactivating your virtualenv will fix this:
VirtualEnv: $ source bin/activate
VirtualFish: $ vf activate [environ]
*: I use virtualfish, but I assume this tip is relevant to both.
I have a similar issue.
I use the --system flag to avoid the error as I decribe here on other thread where I explain the specific case of my situation.
I post this here expecting that can help anyone facing the same problem.
I want to run Python script with Jython.
the result show correctly, but at the same time there is an warning message, "sys-package-mgr*: can't create package cache dir"
How could I solve this problem?
thanks in advance~~~
You can change the location of the cache directory to a place that you have read & write access to by setting the "python.cachedir" option when starting jython, e.g.:
jython -Dpython.cachedir=*your cachedir directory here*
or:
java -jar my_standalone_jython.jar -Dpython.cachedir=*your cachedir directory here*
You can read about the python.cachedir option here:
http://www.jython.org/archive/21/docs/registry.html
1) By changing permissions to allow writing to the directory in the error message.
2) By setting python.cachedir.skip = true
You can read this:
http://www.jython.org/jythonbook/en/1.0/ModulesPackages.html#module-search-path-compilation-and-loading
for further insights.
Making directories world writable admittedly makes the problem "go away", however, it introduces a huge security hole. Anyone could introduce code to the now world writable directory that would be executed in the users' jpython environment.
Setting the cachedir to skip would presumably result in a performance drop (why implement a caching scheme other than to improve performace).
Instead I did the following:
I created a new group (in my case eclipse, but it could have been jpython). I added the users of jpython to that group.
$ sudo groupadd eclipse
I then changed the group of my eclipse plugins folder and its children to 'eclipse'.
/opt/eclipse/plugins $ sudo chgrp -R eclipse *
Then I changed the group permissions as follows
/opt/eclipse/plugins $ sudo chmod -R g+w *
/opt/eclipse/plugins $ find * -type d -print | sudo xargs chmod g+s
This added group writable, and set the S_GID bit on all directories recursively. This last bit causes new directories created to have the same group id as their parent.
The final touch was change the umask for the eclipse users set to 007.
$ sudo vi /etc/login.def
change UMASK to 007 (from 022).
UMASK=007
The easiest fix I found so far was to do:
$ sudo chmod -R 777 /opt/jython/cachedir