I am trying to send parallel arguments by a programmatic invocation to pytest but doesn't seem to recognize them like when parallel is not installed at all, except that I know is there because when I run py.test with a direct command line invocation including the same arguments, it will find it and run successfully.
ERROR: usage: invoke_pytest.py [options] [file_or_dir] [file_or_dir] [...]
invoke_pytest.py: error: unrecognized arguments: --workers 1 --tests-per-worker 4
This is my code:
import os
import sys
import pytest
pytest_args = [
"tests/bdd",
'--rootdir=tests/bdd',
"--workers 1",
"--tests-per-worker 4"
# ...
]
result = pytest.main(pytest_args)
print(f"RESULT {result}")
Although it seems unrelated, I am also using py.test bdd and splinter for this test suite.
Figure out it is the way py.test parse arguments programmatically and has nothing to do with parallel. The values of each argument need to be an independent position of the list!
# ...
pytest_args = [
"tests/bdd",
'--rootdir=tests/bdd',
"--workers", "1",
"--tests-per-worker", "4", # splitted argument and value
# ...
]
result = pytest.main(pytest_args)
print(f"RESULT {result}")
Related
I am trying to make the function log into a command using the following code inside simple.py:
import click
#click.command()
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
When I type the following on the command terminal:log -vvv , I get an error as : Command 'log' not found, but there are 16 similar ones.
#click.command should have converted the function log into a command? But, it doesn't work here. Could someone explain,please? Thanks!
I have tried the following commands:
log -vvv
Command 'log' not found, but there are 16 similar ones.
python3 simple.py log
Usage: simple.py [OPTIONS]
Try 'simple.py --help' for help.
Error: Got unexpected extra argument (log)
Could someone please explain what does #click.command() actually do and how's it different from running simple.py. The documentation does not make it very clear to me as well. Thanks!
import click
#click.command()
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
if __name__ == '__main__':
log()
Then calling it like
$ python simple.py
Verbosity: 0
$ python simple.py -v
Verbosity: 1
The way you try to run it, suggest you think about command group, i.e. nesting commands
import click
#click.group()
def cli():
pass
#cli.command('log')
#click.option('-v', '--verbose', count=True)
def log(verbose):
click.echo(f"Verbosity: {verbose}")
#cli.command('greet')
def greet():
click.echo("Hello")
if __name__ == '__main__':
cli()
Then
$ python simple.py greet
Hello
$ python simple.py log -v -v # or just -vv
Verbosity: 2
Next step would be setuptools integration, etc.
I am trying to explore the results of different parameter settings on my python script "train.py". For that, I use a wandb sweep. Each wandb agent executes the file "train.py" and passes some parameters to it. As per the wandb documentation (https://docs.wandb.ai/guides/sweeps/configuration#command), in case of e.g. two parameters "param1" and "param2" each agents starts the file with the command
/usr/bin/env python train.py --param1=value1 --param2=value2
However, "train.py" expects
/usr/bin/env python train.py value1 value2
and parses the parameter values by position. I did not write train.py and would like to not change it if possible. How can I get wandb to pass the values without "--param1=" in front?
Don't think you can get positional arguments from W&B Sweeps. However, there's a little work around you can try that won't require you touching the train.py file.
You can create an invoker file, let's call it invoke.py. Now, you can use it get rid of the keyword argument names. Something like this might work:
import sys
import subprocess
if len(sys.argv[0]) <= 1:
print(f"{sys.argv[0]} program_name param0=<param0> param1=<param1> ...")
sys.exit(0)
program = sys.argv[1]
params = sys.argv[2:]
posparam = []
for param in params:
_, val = param.split("=")
posparam.append(val)
command = [sys.executable, program, *posparam]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
sys.stdout.write(out.decode())
sys.stdout.flush()
sys.stderr.write(err.decode())
sys.stderr.flush()
sys.exit(process.returncode)
This allows you to invoke your train.py file as follows:
$ python3 invoke.py /path/to/train.py param0=0.001 param1=20 ...
Now to perform W&B sweeps you can create a command: section (reference) in your sweeps.yaml file while sweeping over the parameters param0 and param1. For example:
program: invoke.py
...
parameters:
param0:
distribution: uniform
min: 0
max: 1
param1:
distribution: categorical
values: [10, 20, 30]
command:
- ${env}
- ${program}
- /path/to/train.py
- ${args_no_hyphens}
I am trying to run a click cli through a bash with a different command groups run through a single command collection.
src/preprocessing_extract_data/scripts/main.py
import click
#click.group()
def run_preprocessing_extract_data():
pass
#run_preprocessing_extract_data.command()
#click.option(
"--start_date",
type=click.DateTime(formats=["%Y-%m-%d"]),
required=True,
help="Start date for the pipeline",
)
#click.option(
"--end_date",
type=click.DateTime(formats=["%Y-%m-%d"]),
required=True,
help="End date for the pipeline",
)
def main(start_date, end_date):
...
if __name__ == "__main__":
main()
src/scripts/main.py
from click import CommandCollection
from src.preprocessing_extract_data.scripts.main import run_preprocessing_extract_data
if __name__ == "__main__":
cmds = [
run_preprocessing_extract_data,
# a few more similar command groups
]
cli = CommandCollection(sources=cmds)
cli()
scripts/entrypoint.sh
#!/bin/sh
start_date="$1"
end_date="$2"
python src/scripts/main.py run_preprocessing_extract_data --start_date=$start_date --end_date=$end_date
I run it using ./scripts/entrypoint.sh --start_date="2020-11-01" --end_date="2021-12-01" --today="2021-12-10" but it keeps failing and throws the following error:
Usage: main.py [OPTIONS] COMMAND [ARGS]...
Try 'main.py --help' for help.
Error: No such command 'run_preprocessing_extract_data'.
From the docs:
The default implementation for such a merging system is the CommandCollection class. It accepts a list of other multi commands and makes the commands available on the same level.
Hence, your script now has a command main; you can check this by running your script with --help (or no arguments at all): python src/scripts/main.py --help.
Hence you can do the following:
python src/scripts/main.py main --start_date="$start_date" --end_date="$end_date"
By the way, invoking your shell script should be done without the --start_date: ./scripts/entrypoint.sh "2020-11-01" "2021-12-01".
When writing the setup.py file there is an option to add entry points in the following format:
entry_points={
'console_scripts': [
'entry_point_name0 = module.file:function0',
'entry_point_name1 = module.file:function1',
'entry_point_name2 = module.file:function2'
]
}
However when doing this, even when all three entry points are from the same project they appear as separate commands in the command line. To execute the first entry point I would simply type entry_point_name0 <args> and execute.
Would it be possible to set up a hierarchy to call those commands by first specifying a common name and later the actual entry point name? This would result in calling the first entry point using project_name entry_point_name0 <args>
Yes. Just wrote single entry point that would act as a dispatcher.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('cmd', default='help',
choices=['help', 'do-good', 'do-evil'],
help='Command to execute')
args = parser.parse_args()
cmd = args.cmd;
print(f"Your choice is {cmd}")
Works like this:
$ python cmd.py
usage: cmd.py [-h] {help,do-good,do-evil}
cmd.py: error: the following arguments are required: cmd
$ python cmd.py xyz
usage: cmd.py [-h] {help,do-good,do-evil}
cmd.py: error: argument cmd: invalid choice: 'xyz' (choose from 'help', 'do-good', 'do-evil')
$ python cmd.py do-good
Your choice is do-good
You can use argparse's subcommands for this as well: https://docs.python.org/3/library/argparse.html#sub-commands
I have following code:
class WebuiSeleniumTest(unittest.TestCase):
def setup_parser(self):
parser = argparse.ArgumentParser(description='Automation Testing!')
parser.add_argument('-p', '--platform', help='Platform for desired_caps', default='Mac OS X 10.9')
parser.add_argument('-b', '--browser-name', help='Browser Name for desired_caps', default='chrome')
parser.add_argument('-v', '--version', default='')
return parser.parse_args()
def test_parser(self):
args = self.setup_parser()
print args
if __name__ == "__main__":
unittest.main()
When i try to run this in terminal with the command "python myfile.py -b firefox" I get AttributeError: 'module' object has no attribute 'firefox' and the help output is generated.
When I isolate it and run it without the if __name__ == "__main__" it works fine. Why does it try to apply my passed argument on unittest ? I need it as a String in my code.
Calling your script with python myfile.py -b firefox does indeed go to unittest, and not to your argument parser.
Unittest tries to parse the arguments you gave, e.g. if you call your script like this:
python myfile.py --help
You see the valid options:
Usage: myfile.py [options] [test] [...]
Options:
-h, --help Show this message
-v, --verbose Verbose output
-q, --quiet Minimal output
-f, --failfast Stop on first failure
-c, --catch Catch control-C and display results
-b, --buffer Buffer stdout and stderr during test runs
Examples:
parse.py - run default set of tests
parse.py MyTestSuite - run suite 'MyTestSuite'
parse.py MyTestCase.testSomething - run MyTestCase.testSomething
parse.py MyTestCase - run all 'test*' test methods
in MyTestCase
Looking at the help output -b would buffer (I guess suppress?) stdout/stderr. The argument firefox is taken as the name of the test to run in your module. And there is no function method existing, it outputs this error:
AttributeError: 'module' object has no attribute 'firefox'
Now, what you probably want to do is to call test_parser, and if you do that with python myfile.py WebuiSeleniumTest.test_parser then you cannot pass any additional arguments. And that's probably your question in the end. There is this question which gives some possible solutions for testing argparse as unit test.