Develop Reference

Run vulture on Python module with CLI arguments?

Context
Suppose one has a project structure with src.projectname.__main__.py which can be executed using the following command with accompanying arguments:
python -m src.projectname -e mdsa_size3_m1 -v -x
Question
How would one run vulture whilst passing cli arguments to the script on which vulture runs?
Approach I
When I run:
python -m src.projectname -e mdsa_size3_m1 -v -x
It throws the following
usage: vulture [options] [PATH ...]
vulture: error: unrecognized arguments: -e mdsa_size3_m1 -x
because vulture tries to parse the arguments for the script that is being ran.
Notes
I am aware normally one would expect to run vulture on the script and its entirety without narrowing down the scope with arguments. However, in this case the arguments are required to specify the number of runs/duration of the code execution.
One can hack around this issue by temporarily manually hardcoding the args with (for example):
args = parse_cli_args()
args.experiment_settings_name = "mdsa_size3_m1"
args.export_images = True
process_args(args)
assuming one has such an args object, however, I thought perhaps this functionality can be realised using the CLI, without temporarily modifying the code.

Pass commandline arguments to a Python script installed with Poetry

The poetry documentation says that the script section can be used to install scripts or executable when the package is installed. But it does not show any example of how to pass arguments to the script.
How can you do to receive with argparse the arguments in the function?

First a little project setup:
Starting from a new poetry project with poetry new example_script (and creating a main.py file inside example_script dir) with a structure like this:
├── example_script
│   ├── __init__.py
│   ├── main.py
├── pyproject.toml
├── README.rst
└── tests
├── __init__.py
└── test_poetry_example.py
And adding in the pyproject.toml the config (in the section [tool.poetry.scripts]) of the script that we are going to install:
# pyproject.toml
[tool.poetry]
name = "example_script"
# some lines excluded
[tool.poetry.scripts]
my-script = "example_script.main:start"
# some lines excluded
And finally the main.py file, which has to have a start function inside (as we passed it in the toml). The arguments parser goes inside this function, since this function is the one that will end up executing when we run the script:
import argparse
def some_function(target, end="!"):
"""Some example funcion"""
msg = "hi " + target + end
print(msg)
def start():
# All the logic of argparse goes in this function
parser = argparse.ArgumentParser(description='Say hi.')
parser.add_argument('target', type=str, help='the name of the target')
parser.add_argument('--end', dest='end', default="!",
help='sum the integers (default: find the max)')
args = parser.parse_args()
some_function(args.target, end=args.end)
We can run the script with poetry, or install and run it directly:
# run with poetry
$ poetry run my-script
# install the proyect (this will create a virtualenv if you didn't have it created)
$ poetry install
# activate the virtualenv
$ poetry shell
# run the script
$ my-script --help
usage: my-script [-h] [--end END] target
Say hi.
positional arguments:
target the name of the target
optional arguments:
-h, --help show this help message and exit
--end END sum the integers (default: find the max)
$ my-script "spanish inquisition" --end "?"
hi spanish inquisition?

This question is really two separate questions:
How do I pass arguments into a script that is run using Poetry
How do I access and parse those arguments, in particular, using argparse
The initial answer (by Lucas), addresses parts of each, especially about argparse, but I'm answering to fill in some additional details and explain how to directly access the args.
Access arguments directly in any function or script
As an alternative to argparse, arguments can be directly accessed in Python at any time using sys.argv, which is a list of strings, each one is one of the arguments. Python splits up the arguments based on spaces, unless the spaces are enclosed in quotes (either single or double quotes).
This method is more direct and lightweight than argparse, with a lot less functionality.
args.py setup as a main script file with a start() function:
import sys
def start(args=sys.argv):
for i, arg in enumerate(args):
print(f'Arg #{i}: {arg}')
if __name__ == '__main__':
start()
Run it at the command-line with a variety of argument types:
$ python args.py "item 1" 'Hello Arguments!!' "i 3" 4 5 6
Arg #0: args.py
Arg #1: item 1
Arg #2: Hello Arguments!!
Arg #3: i 3
Arg #4: 4
Arg #5: 5
Arg #6: 6
The first argument is always the script that was called, in exactly the way it was called (i.e. relative or absolute path to the script file or other reference).
Adding arguments when calling with poetry run
While you can run scripts with Poetry by activating the virtual environment with poetry shell and then running the script as normal with python script.py arg1 arg2 arg3, you can also add arguments directly to the poetry run command:
At the command-line, directly running the script:
$ poetry run python args.py arg1 arg2 arg3
Arg #0: <some_path>/args.py
Arg #1: arg1
Arg #2: arg2
Arg #3: arg3
Running a python file as an installed Poetry script
Or, run it as a script, installed by Poetry. In this case the script name we assign is arg_script, and you just run it directly at a terminal prompt with the virtual environment activated (i.e. do not invoke with python):
In pyproject.toml:
[tool.poetry.scripts]
arg_script = 'args:start' # run start() function from ./args.py
After updating pyproject.toml, run poetry install at a terminal prompt to install the script in the virtual environment named as arg_script.
With Poetry, you can run a command in the virtual environment by using poetry run:
$ poetry run arg_script arg1 arg2 arg3
Arg #0: arg_script
Arg #1: arg1
Arg #2: arg2
Arg #3: arg3
Any arguments added after poetry run, act just like you were typing them into a terminal that has the virtual environment already activated. i.e. The equivalent is:
$ poetry shell
$ args_script arg1 arg2 arg3

Execute a custom bash function within python

This question is a simpler version of this question.
In simple terms, I have a custom bash function my_func defined on ~/.bash_profile and uses two more bash functions defined in the same environment.
Also, my_func accepts two arguments, let's say a and b. What my_func does is that it gets connected to a remote server and sends some files (these are determined by a and b).
If I type on the bash shell:
. my_func a b everything works fine and I get some print statements on the screen.
However, if I include:
subprocess.call(['#!/bin/bash . my_func a b'], shell=True) nothing seems to happen.
I tried to export all the bash functions that are used by my_func by including:
subprocess.call(['#!/bin/bash export -f my_func'], shell=True) and I did the same for the rest of the functions that are used by my_func.
EDIT:
If I use subprocess.call(['bash', '-c' ,'my_func a b], shell=True) the bash shell will change into bash-3.2$

You need to export the function before you start the python program:
export -f my_func
python foo.py
Well, the above example might not work if the system's default shell (/bin/sh) is not bash. To circumvent this, you may use subprocess call like this:
$ function foo() { echo "bar" ; }
$ export -f foo
$ cat foo.py
import subprocess
subprocess.call(['bash', '-c', 'foo'])
$ python foo.py
bar
Alternative:
I would put the function into a lib file, let's say:
# /usr/share/my_lib/my_lib.sh
function my_func() {
# do something
}
Then I would expose the function via a script in PATH:
#!/bin/bash
# /usr/local/bin/my_prog
source /usr/lib/my_lib/my_lib.sh
my_func "$1" "$2"
In Python you would just:
subprocess.call(['/usr/local/bin/my_prog', 'a', 'b'])
Btw: If you don't need that function somewhere else, you can just put it directly into /usr/local/bin/my_prog.

ipython not showing argparse help message

In my python script myscript.py I use argparse to pass command-line arguments. When I want to display the help information about the input arguments, I just do:
$ python myscript.py --help
If instead I want to use ipython to run my script, the help message won't be displayed. Ipython will display its own help information:
$ ipython -- myscript.py -h
=========
IPython
=========
Tools for Interactive Computing in Python
=========================================
A Python shell with automatic history (input and output), dynamic object
introspection, easier configuration, command completion, access to the
system shell and more. IPython can also be embedded in running programs.
Usage
ipython [subcommand] [options] [files]
It's not so annoying, but is there a way around it?

You need to run your .py script inside the ipython. Something like that:
%run script.py -h

This is an IPython bug, corrected in https://github.com/ipython/ipython/pull/2663.
My 0.13 has this error; it is corrected in 0.13.2. The fix is in IPthyon/config/application.py Application.parse_command_line. This function looks for help and version flags (-h,-V) in sys.argv before passing things on to parse_known_args (hence the custom help formatting). In the corrected release, it checks sys.argv only up to the first --. Before it looked in the whole array.
earlier:
A fix for earlier releases is to define an alternate help flag in the script:
simple.py script:
import argparse, sys
print(sys.argv)
p = argparse.ArgumentParser(add_help=False) # turn off the regular -h
p.add_argument('-t')
p.add_argument('-a','--ayuda',action=argparse._HelpAction,help='alternate help')
print(p.parse_args())
Invoke with:
$ ./ipython3 -- simple.py -a
['/home/paul/mypy/argdev/simple.py', '-a']
usage: simple.py [-t T] [-a]
optional arguments:
-t T
-a, --ayuda alternate help
$ ./ipython3 -- simple.py -t test
['/home/paul/mypy/argdev/simple.py', '-t', 'test']
Namespace(t='test')

How do you pass script arguments to pdb (Python)?

I've got python script (ala #! /usr/bin/python) and I want to debug it with pdb. How can I pass arguments to the script?
I have a python script and would like to debug it with pdb. Is there a way that I can pass arguments to the scripts?

python -m pdb myscript.py arg1 arg2 ...
This invokes pdb as a script to debug another script. You can pass command-line arguments after the script name. See the pdb doc page for more details.

usually I use ipython
-i
If running code from the command line, become interactive afterwards.
It is often useful to follow this with `--` to treat remaining flags as
script arguments.
ipython --pdb -i -- test.py -a

python3 -m pdb myscript.py -a val if using argparse with flag "a" and value "val"

If, like me, you prefer the more graphical pudb debugger, you can pass the arguments of your script directly by doing:
pudb myscript.py arg1 arg2 ...
Indeed, invoking:
python -m pudb myscript.py arg1 arg2 ...
won't work will return with the following error:
No module named pudb.__main__; 'pudb' is a package and cannot be directly executed