I'm currently discovering Prefect and I'm trying to deploy it to schedule workflows. I struggle a bit to understand how to access some data though. Here is my problem: I create a deployment and run it via the Python API and I need the ID of the flow run it creates (to cancel it, may other things happen outside of the flow).
When I run without any scheduling I can access the data I need (the flow run UUID), but I kind of want the scheduling part. It may be because the run_deployment function is asynchronous but as I am nowhere near being an expert in Python I don't know for sure (well that, and the fact that my code never exits after calling the main() function).
Here is what my code looks like:
from prefect import flow, task
from prefect.deployments import Deployment, run_deployment
from datetime import datetime, date, time, timezone
# Import the flow:
from script import my_flow
# Configure the deployment:
deployment_name = "my_deployment"
# Create the deployment for the flow:
deployment = Deployment.build_from_flow(
flow = my_flow,
name = deployment_name,
version = 1,
work_queue_name = "my_queue",
)
deployment.apply()
def main():
# Schedule a flow run based on the deployment:
response = run_deployment(
name = "my_flow/" + deployment_name,
parameters = {my_param},
scheduled_time = dateutil.parser.isoparse(scheduledDate),
flow_run_name = "my_run",
)
print(response)
if __name__ == "__main__":
main()
exit()
I searched a bit and saw in that post that it was possible to print the flow run id as it was executed, but in my case I need before the execution.
Is there anyway to get that data (using the Python API)? Or to set the flow ID myself? (I've already thoroughly checked the docs, I'm quite sure this is not possible)
Thanks a lot for your time!
Gauthier
As of 2.7.12 - released the same day you posted your question - you can create names for flows programmatically. Does that get you what you need?
As of 2.7.12 - released the same day you posted your question - you can create names for flows programmatically. Does that get you what you need?
Both tasks and flows now expose a mechanism for customizing the names of runs! This new keyword argument (flow_run_name for flows, task_run_name for tasks) accepts a string that will be used to create a run name for each run of the function. The most basic usage is as follows:
from datetime import datetime
from prefect import flow, task
#task(task_run_name="custom-static-name")
def my_task(name):
print(f"hi {name}")
#flow(flow_run_name="custom-but-fixed-name")
def my_flow(name: str, date: datetime):
return my_task(name)
my_flow()
This is great, but doesn’t help distinguish between multiple runs of the same task or flow. In order to make these names dynamic, you can template them using the parameter names of the task or flow function, using all of the basic rules of Python string formatting as follows:
from datetime import datetime
from prefect import flow, task
#task(task_run_name="{name}")
def my_task(name):
print(f"hi {name}")
#flow(flow_run_name="{name}-on-{date:%A}")
def my_flow(name: str, date: datetime):
return my_task(name)
my_flow()
See the docs or https://github.com/PrefectHQ/prefect/pull/8378 for more details.
run_deployment returns a flow run object - which you named response in your code.
If you want to get the ID before the flow run is actually executed, you just have to set timeout=0, so that run_deployment will return immediately after submission.
You only have to do:
flow_run = run_deployment(
name = "my_flow/" + deployment_name,
parameters = {my_param},
scheduled_time = dateutil.parser.isoparse(scheduledDate),
flow_run_name = "my_run",
timeout=0
)
print(flow_run.id)
Related
I am learning Python3 and I have a fairly simple task to complete but I am struggling how to glue it all together. I need to query an API and return the full list of applications which I can do and I store this and need to use it again to gather more data for each application from a different API call.
applistfull = requests.get(url,authmethod)
if applistfull.ok:
data = applistfull.json()
for app in data["_embedded"]["applications"]:
print(app["profile"]["name"],app["guid"])
summaryguid = app["guid"]
else:
print(applistfull.status_code)
I next have I think 'summaryguid' and I need to again query a different API and return a value that could exist many times for each application; in this case the compiler used to build the code.
I can statically call a GUID in the URL and return the correct information but I haven't yet figured out how to get it to do the below for all of the above and build a master list:
summary = requests.get(f"url{summaryguid}moreurl",authmethod)
if summary.ok:
fulldata = summary.json()
for appsummary in fulldata["static-analysis"]["modules"]["module"]:
print(appsummary["compiler"])
I would prefer to not yet have someone just type out the right answer but just drop a few hints and let me continue to work through it logically so I learn how to deal with what I assume is a common issue in the future. My thought right now is I need to move my second if up as part of my initial block and continue the logic in that space but I am stuck with that.
You are on the right track! Here is the hint: the second API request can be nested inside the loop that iterates through the list of applications in the first API call. By doing so, you can get the information you require by making the second API call for each application.
import requests
applistfull = requests.get("url", authmethod)
if applistfull.ok:
data = applistfull.json()
for app in data["_embedded"]["applications"]:
print(app["profile"]["name"],app["guid"])
summaryguid = app["guid"]
summary = requests.get(f"url/{summaryguid}/moreurl", authmethod)
fulldata = summary.json()
for appsummary in fulldata["static-analysis"]["modules"]["module"]:
print(app["profile"]["name"],appsummary["compiler"])
else:
print(applistfull.status_code)
I want to get all tasks from a specific task from Todoist. The documentation says. Taken from the documentation the code for getting all tasks is the following:
from todoist_api_python.api import TodoistAPI
api = TodoistAPI('someToken...')
try:
tasks = api.get_tasks()
print(tasks)
except Exception as error:
print(error)
The documentation does not contain any information about how to get all the tasks from a certain project. Due the reason that I found also another opportunity to get all tasks I am sure there is a way to call all the tasks from a project but it is just not obviously how.
Another way to get all projects from Todoist:
from todoist.api import TodoistAPI
key = "someToken..."
api = TodoistAPI(key)
api.sync()
print(api.state["projects"])
Solution
from todoist.api import TodoistAPI
key = "someToken..."
api = TodoistAPI(key)
api.sync()
for project in api.state['items']:
if(project["project_id"] == someProjectId):
print(project["content"])
Has anyone used the python wrapper for the Five9 API?
I'm trying to run one of our custom reports through the API but am unsure how to pass the time criteria.
If the report is already built:
Is there a way to run it without passing criteria?
It seems like this should be enough (since the report is already built):
client.configuration.runReport(folderName='Shared Reports', reportName='Test report')
But it didn't work as expected.
What can I do to solve this?
This works for me. You can probably get fancier with a while loop checking if the report is done but this isn't time sensitive.
from five9 import Five9
import time
client = Five9('username','password')
start = '2019-08-01T00:00:00.000'
end = '2019-08-29T00:00:00.000'
criteria = {'time':{'end':end, 'start':start}}
identifier = client.configuration.runReport(folderName='SharedReports',reportName='reportname',criteria=criteria)
time.sleep(15)
get_results = client.configuration.getReportResult(identifier)
I have a simple script shown below. I want to use the value of holiday_value to create a filter. I am thinking this could be done by putting the value into zap storage and then retrieving the value from storage and use it in a zap filter. I don't know how to get the value from the script into zap storage.
from datetime import date
import holidays
us_holidays = holidays.US()
if date.today() in us_holidays:
holiday_value='true'
else:
holiday_value='false'
I'm not really good at Python, but here's what you could do.
Trigger
If you want to trigger another Zap from this Code step, you could use the requests library and Zapier's Webhooks as a Trigger step for your other Zap (Zap that you want to Trigger).
Here are the steps:
Setup a Zap with the trigger app as Webhooks. Get the webhook URL.
From the Code step, make a request to the Webhook URL with the value of holiday_value. (Here is a sample POST request). Also helpful to refer to this example.
Filter
If you are looking for creating a filter in the same Zap instead,
You could return the value of holiday_value from this code step. Refer documentation here.
Your code would probably look like (Please check syntax, I'm not good at Python),
from datetime import date
import holidays
us_holidays = holidays.US()
if date.today() in us_holidays:
return {'holiday_value': 'true'}
else:
return {'holiday_value': 'false'}
You can now add a filter step that only allows the Zap to continue if holiday_value equals True or False. Documentation for filters here.
Hope that helps.
I am trying to learn how to use the new WebHooks service on IFTTT, and I'm struggling to figure out how it is supposed to work. Most of the projects I can find online seem to refer to a deprecated "maker" service, and there are very little resources for interpreting the new channel.
Let's say I want to check the value of "online" every ten minutes in the following json file: https://lichess.org/api/users/status?ids=thibault
I can write a Python script that extracts this value like so:
response = urlopen('https://lichess.org/api/users/status?ids=thibault')
thibault = response.read()
data = json.loads(thibault)
status = data[0]['online']
If the status is equal to "true", I would like to receive a notification via email or text message. How do I integrate the python script and the webhooks service? Or do I even need to use this script at all? I assume I need some kind of cron job that regularly runs this Python script, but how do I connect this job with IFTTT?
When I create a new applet on IFTTT I am given the ability to create a trigger with a random event name, but it's unclear what that event name corresponds to.
I have a similar setup for my IFTTT webhook service. To the best of my understanding, the answer to your question is yes, you need this script (or similar) to scrap the online value, and you'll probably want to do a cron job (my approach) or keep the script running (wouldn't be my preference).
IFTTT's webhooks a json of up to 3 values, which you can POST to a given event and key name.
Below is a very simple excerpt of my webhook API:
def push_notification(*values, **kwargs):
# config is in json format
config = get_config()
report = {}
IFTTT = {}
# set default event/key if kwargs are not present
for i in ['event', 'key']:
IFTTT[i] = kwargs[i] if kwargs and i in kwargs.keys() else config['IFTTT'][i]
# unpack values received (up to 3 is accepted by IFTTT)
for i, value in enumerate(values, 1):
report[f"value{i}"] = value
if report:
res = requests.post(f"https://maker.ifttt.com/trigger/{IFTTT['event']}/with/key/{IFTTT['key']}", data=report)
# TODO: add try/except for status
res.raise_for_status()
return res
else:
return None
You probably don't need all these, but my goal was to set up a versatile solution. At the end of the day, all you really need is this line here:
requests.post(f"https://maker.ifttt.com/trigger/{event}/with/key/{key}", data={my_json_up_to_3_values})
Where you will be placing your webhook event name and secret key value. I stored them in a config file. The secret key will be available once you sign up on IFTTT for the webhook service (go to your IFTTT webhook applet setting). You can find your key with a quick help link like this: https://maker.ifttt.com/use/{your_secret_key}. The event can be a default value you set on your applet, or user can choose their event name if you allow.
In your case, you could do something like:
if status:
push_notification("Status is True", "From id thibault", event="PushStatus", key="MysEcR5tK3y")
Note: I used f-strings with version 3.6+ (It's great!), but if you have a lower version, you should switch all the f-strings to str.format().