How do you add command line args to airflow python operator? - python

validation = PythonOperator(
task_id="validation",
python_callable=validation,
op_args=[
"--foo", foo,
"--bar", "null",
)
)
I'm trying to call a method from the airflow PythonOperator that requires command-line arguments which are accessed using argparse. How do I add these in to the PythonOperator?
I tried the above example but these arguments were invalid. The validation method takes no arguments.
I get this error:
ERROR - Failed to execute job 51 for task validation (main() takes 0 positional arguments but 4 were given; 10436)
[2023-02-17, 19:07:36 UTC] {{local_task_job.py:159}} INFO - Task exited with return code 1

Related

parser.parse_args does not recognize arguments in celery task

I have been Googling this problem most of the morning and find no solution.
Using the following:
parser.add_argument('-t', '--task', dest='task', action='store')
args = parser.parse_args(cmd_args)
When cmd_args contains "['-t my_task']" everything works as I expect.
When cmd_args contains "['--task my_task']" an exception is thrown.
The important part of the error says: "error: unrecognized arguments: --task mytask"

Typer python : Getting an Error "Error: Got unexpected extra argument (stringvalue)" when passing a string to my function

So i was trying to make a to-do list CLI application in python using Typer and I want it to display a message when no task is provided to it . So i set task with a default value of None
But when I pass a value to task eg "driving" I get an error "Got an unexpected extra argument"
Any suggestions on how to fix this error or maybe another solution to display an error when no argument is passed to the function
Heres my code:
#app.command()
def add(priority:int, task="None"):
if task=="None":
print("Error: Missing tasks string. Nothing added!")
else:
dict1={}
dict1.update({task:priority})
filesize1= os.stat('output.pickle').st_size
if filesize1==0:
pf=open("output.pickle","wb")
else:
pf=open("output.pickle","ab")
pickle.dump(dict1, pf)
pf.close()
pf=open("output.pickle","rb")
dict2={}
while 1:
try:
dict2.update(pickle.load(pf))
except EOFError:
break
pf.close()
dict3={k:v for k,v in sorted(dict2.items(),key= lambda v:v[1])}
print('Added task: "'+task+'" with priority '+str(priority))
file1=open("ls.txt","w+")
i=1
for key,value in dict3.items():
file1.write(str(i)+". "+key+ " ["+str(value)+"]"+"\n")
i=i+1
Recently, I've encountered the same problem with the typer version of 0.7.0
Let say we have a file called addition.py and there is your function add decorated as typer command. Then, first what I did is I've checked what --help flag would give me and I got the following:
In your terminal run:
python addition.py --help
Here you see that task argument is optional and has to be passed differently.
If you run:
python addition.py 123456 --task "SOMETHING"
Then this works perfectly when you have only one function in your addition.py file
If you have other functions with the #app.command() decorator then you have to indicate function name in terminal
python addition.py add 123456 --task "SOMETHING"

How do I execute multiple arguments in console using argparse?

I've been reading for a few hours and I'm beginning to give up so I'm here for the punishment. I am trying to run a python file which requires arguments. Help says
Usage: Eventbot.py [-h] [-id ID] [-num NUM] [--wait WAIT] url
I try typing python eventbot.py 1 1 https://www.eventbrite.com/e/cooking-at-home-with-yao-zhao-all-about-sichuan-pepper-tickets-137805759737
But it says invalid argument 1 https://www.eventbrite.com/e/cooking-at-home-with-yao-zhao-all-about-sichuan-pepper-tickets-137805759737
I've tried adding the --wait 1 at the end but that doesn't work.
if __name__ == '__main__':
try:
parser = argparse.ArgumentParser(description="Eventbrite bot to automate securing event tickets")
parser.add_argument("url", type=str, help="Eventbrite event URL")
parser.add_argument("-id", type=str, help="Ticket ID to purchase")
parser.add_argument("-num", type=str, help="Number of tickets to purchase")
parser.add_argument("--wait", type=int, help="Seconds the browser should wait for DOM to load")
args = parser.parse_args()
Since the -id and -num arguments are preceded by a -, that means that they are not positional arguments, like the url argument is. Since these arguments aren't positional, they act exactly like your --wait argument, which also is not positional. In order to pass a value to the argument, specify the desired value after the argument name, for example,
python eventbot.py -id 1 -num 1 https://www.eventbrite.com/e/cooking-at-home-with-yao-zhao-all-about-sichuan-pepper-tickets-137805759737

python argparser, argument executing function

I was wondering how coul i execute a function in python with just a call of an argument when running that command, for example,
parser = argparse.ArgumentParser(description="Database accounts manager")
parser.add_argument("-s", "--show", help="Shows all database rows and columns", dest="show",required=False)
args = parser.parse_args()
if args.show:
print("i am beautiful")
what i would like is to when i call this file : python file.py -s or python file.py --show, i would like it to execute a function like i wrote for an example : "print("i am beautiful")
because when i do that i need to give an argument so it executes the function: "print(...)"
Add action to the parameters:
parser.add_argument("-s", "--show", help="Shows all database rows and columns", dest="show",required=False, action="store_true")

Py.Test Can't find pytest-parallel when invoked programmatically

I am trying to send parallel arguments by a programmatic invocation to pytest but doesn't seem to recognize them like when parallel is not installed at all, except that I know is there because when I run py.test with a direct command line invocation including the same arguments, it will find it and run successfully.
ERROR: usage: invoke_pytest.py [options] [file_or_dir] [file_or_dir] [...]
invoke_pytest.py: error: unrecognized arguments: --workers 1 --tests-per-worker 4
This is my code:
import os
import sys
import pytest
pytest_args = [
"tests/bdd",
'--rootdir=tests/bdd',
"--workers 1",
"--tests-per-worker 4"
# ...
]
result = pytest.main(pytest_args)
print(f"RESULT {result}")
Although it seems unrelated, I am also using py.test bdd and splinter for this test suite.
Figure out it is the way py.test parse arguments programmatically and has nothing to do with parallel. The values of each argument need to be an independent position of the list!
# ...
pytest_args = [
"tests/bdd",
'--rootdir=tests/bdd',
"--workers", "1",
"--tests-per-worker", "4", # splitted argument and value
# ...
]
result = pytest.main(pytest_args)
print(f"RESULT {result}")

Categories

Resources