How to get the URL from the DAG in Airflow using templating? - python

I was trying to send to Slack an alert when the previous task fails but I would like to add the url from the DAG in the text to go easily from slack.
I tried this, but I'm getting an error:
send_slack_alert = SlackWebhookOperator(
task_id='slack_alert',
slack_webhook_conn_id="slack_connection",
message="There is an error\n"
"DAG name: {{ dag }}\n"
"Run: {{ run_id }}\n"
"url: {{ conf.base_url }}/dags/{{ dag.dag_id }}/grid```",
trigger_rule=TriggerRule.ONE_FAILED
)
Error:
'airflow.configuration.AirflowConfigParser object' has no attribute 'base_url'
I was looking this doc from airflow
https://airflow.apache.org/docs/apache-airflow/stable/templates-ref.html
Thanks in advance

The AirflowConfigParser has a get() method you can use to retrieve Airflow configuration settings. This method can still be used in a Jinja expression.
SlackWebhookOperator(
task_id='slack_alert',
slack_webhook_conn_id="slack_connection",
message="There is an error\n"
"DAG name: {{ dag.dag_id }}\n"
"Run: {{ run_id }}\n"
"url: {{ conf.get('webserver', 'BASE_URL') }}/dags/{{ dag.dag_id }}/grid",
trigger_rule=TriggerRule.ONE_FAILED
)
Rendered Template

Related

gcloud recaptcha enterprise - creating an assessment

I am working through the ReCaptcha Enterprise tutorial and have gotten stuck with a 400 Request contains an invalid argument error. I am using Django.
I am following the instructions here https://cloud.google.com/recaptcha-enterprise/docs/create-assessment#python to authenticate the recaptcha.
I have been able to load the recaptcha on the front-end with this code
{% block content %}
<script type="text/javascript">
var onloadCallback = function() {
grecaptcha.enterprise.render('html_element', {
'sitekey' : '{{ sitekey }}',
});
};
</script>
<form action="{% url 'captcha-authentication' %}" method="post">
{% csrf_token %}
<div id="html_element"></div>
<input type="submit" value="Submit">
</form>
<script src="https://www.google.com/recaptcha/enterprise.js?onload=onloadCallback&render=explicit"
async defer>
</script>
I have verified in my view that request.POST contains the 'g-recaptcha-response' key. For my site_key, I am using the key id from https://console.cloud.google.com/security/recaptcha.
def confirm_captcha(token, site_key, recaptcha_action):
parent_project = "t-commerce-321516"
client = recaptchaenterprise_v1.RecaptchaEnterpriseServiceClient()
event = recaptchaenterprise_v1.Event()
event.site_key = site_key
event.token = token
event.expected_action = recaptcha_action
assessment = recaptchaenterprise_v1.Assessment()
assessment.event = event
request = recaptchaenterprise_v1.CreateAssessmentRequest()
request.assessment = assessment
request.parent = parent_project
# this is the line that is throwing an error
response = client.create_assessment(request)
if not response.token_properties.valid:
print("The CreateAssessment() call failed because the token was " +
"invalid for the following reasons: "
+ str(response.token_properties.invalid_reason))
else:
if response.event.expected_action == recaptcha_action:
print("The reCAPTCHA score for this token is: " +
str(response.risk_analysis.score))
return True
else:
print("The action attribute in your reCAPTCHA tag does" +
"not match the action you are expecting to score")
return False
The error from creating the client assessment is 400 Request contains an invalid argument.
Is there a way to tell why Google is angry?
I am using a checkbox on the front-end and maybe that means that the recaptcha_action argument isn't needed, but I'm not sure how to do that.
Any help or pointers is appreciated!
I think you need:
request.parent = `projects/${parent_project}`
per: CreateAssessmentRequest

Show value to HTML just like how value gets display in CMD prompt when executes print() in python

I am working on speech recognition by following this tutorial and implementing in Django. I'm wondering the way we execute print("Listening...") and print("Recognizing...") for user to understand when to speak in CMD prompt, is it possible to send value (Listening, Recognizing) in HTML page each time when user speaks something ?
def takeCommand():
r = sr.Recognizer()
with sr.Microphone() as source:
print("Listening...")
r.pause_threshold = 1
r.adjust_for_ambient_noise(source)
audio = r.listen(source)
print(audio)
try:
print("Recognizing...")
query = r.recognize_google(audio, language ='en-in')
print("User said:",query)
except Exception as e:
print(e)
print("Unable to Recognizing your voice.")
return "None"
return query
You can use django.contrib.messages to "print" them to the template.
In your view:
from django.contrib import messages
messages.warning(self.request, f'Unable to Recognizing your voice')
In your template:
{% if messages %}
{% for message in messages %}
<div class="alert alert-{{ message.tags }}">{{ message }}</div>
{% endfor %}
{% endif %}
For more info about the messages framework see the: docs
how do you send the audio to the server? somehow via ajax?
When he clicks a button to start talking, you use JS/jquery to put somewhere:
$('#some_div').append('Listening...');
When he stops talking, you put there:
$('#some_div').append('Recognizing...');
And you send the audio via ajax to the server.
Then you response from the server to the ajax request and do either:
$('#some_div').append('Unable to Recognizing your voice.');
or:
$('#some_div').append('User said: ' + response.query);

Ansible: print pretty xml with Ansible debug module for the xml page saved using register

How can I print xml saved using register with debug with pretty xml format
- name: Create a JIRA issue
uri:
url: https://your.jira.example.com/rest/api/2/issue/
method: GET
user: your_username
password: your_pass
body: "{{ lookup('file','issue.json') }}"
force_basic_auth: yes
status_code: 201
body_format: json
register: test
- debug: {{ test.content }}
if test.content contains only xml you can print the data with the xml module
- xml:
xmlstring: {{ test.content }}
pretty_print : yes

Error message when submitting HIT to Amazon Mechanical Turk

I have a problem submitting a HIT to Amazon Mechanical Turk sandbox.
I'm using the following code to submit a HIT:
external_content = """"
<ExternalQuestion xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2006-07-14/ExternalQuestion.xsd">
<ExternalURL>https://MY_HOST_GOES_HERE/</ExternalURL>
<FrameHeight>400</FrameHeight>
</ExternalQuestion>
"""
import boto3
import os
region_name = 'us-east-1'
aws_access_key_id = 'MYKEY'
aws_secret_access_key = 'MYSECRETKEY'
endpoint_url = 'https://mturk-requester-sandbox.us-east-1.amazonaws.com'
# Uncomment this line to use in production
# endpoint_url = 'https://mturk-requester.us-east-1.amazonaws.com'
client = boto3.client('mturk',
endpoint_url=endpoint_url,
region_name=region_name,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key,
)
# This will return $10,000.00 in the MTurk Developer Sandbox
print(client.get_account_balance()['AvailableBalance'])
response = client.create_hit(Question=external_content,
LifetimeInSeconds=60 * 60 * 24,
Title="Answer a simple question",
Description="Help research a topic",
Keywords="question, answer, research",
AssignmentDurationInSeconds=120,
Reward='0.05')
# The response included several helpful fields
hit_group_id = response['HIT']['HITGroupId']
hit_id = response['HIT']['HITId']
# Let's construct a URL to access the HIT
sb_path = "https://workersandbox.mturk.com/mturk/preview?groupId={}"
hit_url = sb_path.format(hit_group_id)
print(hit_url)
The error message I get is:
botocore.exceptions.ClientError: An error occurred (ParameterValidationError) when calling the CreateHIT operation: There was an error parsing the XML question or answer data in your request. Please make sure the data is well-formed and validates against the appropriate schema. Details: Content is not allowed in prolog. (1493572622889 s)
What might be the reason here? The xml fully agrees with xml schema located on amazon servers.
The html returned by the external host is:
<!DOCTYPE html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>
<script src='https://s3.amazonaws.com/mturk-public/externalHIT_v1.js' type='text/javascript'></script>
</head>
<body>
<!-- HTML to handle creating the HIT form -->
<form name='mturk_form' method='post' id='mturk_form' action='https://workersandbox.mturk.com/mturk/externalSubmit'>
<input type='hidden' value='' name='assignmentId' id='assignmentId'/>
<!-- This is where you define your question(s) -->
<h1>Please name the company that created the iPhone</h1>
<p><textarea name='answer' rows=3 cols=80></textarea></p>
<!-- HTML to handle submitting the HIT -->
<p><input type='submit' id='submitButton' value='Submit' /></p></form>
<script language='Javascript'>turkSetAssignmentID();</script>
</body>
</html>
Thank you
This message "Details: Content is not allowed in prolog." is the clue. It turns out that what this is saying is that you can't have content outside of where it is expected. This is what usually happens when a junk character (think smart-quotes or non-printable ASCII value) appears in there. These can be a real pain in the butt to diagnose.
In your case, it's a little easier to debug but still just as frustrating. Check out this line:
external_content = """"
It turns out that Python only needs three quotes (""") in order to acknowledge a multi-line string definition. Thus your fourth " was actually rendering as part of the XML. Change that line to this:
external_content = """
And you're golden. I just tested it and it works. Sorry for all the frustration, but hopefully this unblocks you. Happy Sunday!

angularjs+python(Flask) file upload to server

I am developing a web app using angular, python and Flask.
In my app there is a form where the user need to enter some data and to upload his photo.
I want to implement the file upload to the server using angular.
I saw that I need to use the FormData() and to bind the data from the HTML to angular using "watch".
The javascript part is clear.
I don't understand how can I get the data from the Python side.
This is my HTML -
<form enctype="multipart/form-data" ng-submit="submitGuideDetailsForm()">
<div class="form-group">
<label for="usr">Add your photo:</label>
<input type='file' class="form-control" name='file' onchange="angular.element(this).scope().uploadFile(this.files)">
</div>
</form>
This is my angular -
$scope.uploadFile = function(files) {
$scope.file = new FormData();
$scope.file.append("file", files[0]);
};
$scope.submitGuideDetailsForm= function() {
$http.post('/uploadFile', $scope.file, {
headers: {'Content-Type': undefined },
transformRequest: angular.identity
}).success(function(results)
{
$log.log('success load file')
}).error(function(error)
{
$log.log('error load file!!!!!')
$log.log(error);
});
};
I want to get the file on the server side, using python and flask -
#app.route('/uploadFile', methods=['POST'])
def uploadFile():
json_data = request.json
print(json_data)
status = 'success'
return jsonify({'result': status})
I don't know how to get the data from the 'request' object.
request.json does not work.
Please advice me what am I doing wrong.
How can I get the data from the 'request' object?
Do I need to encode the data to a file?
How do I send the data back to client? Do I need to encode it back?
I did not find a full example using angular and python/flask uploading a file to server, saving it and than downloading it to the client.
Thanks a lot,
Dina
the file can be get byfile = request.files['file']
after that,
you can get the filename by file.filename.encode("utf8")
The command 'request.files' work
The gets me the following object -
ImmutableMultiDict([('file', <FileStorage:
u'12132638_10153196350425732_1151540092606190338_o.jpg'
('image/jpeg')>)])
I don't understand how can I save it on the server.
How do I get only the file name out of this object
And how do I send it back to the client side?
Thanks a lot, Dina
Use request.files and .filename atrribute
file = request.files['file']
fileName = file.filename
#to store it on server use getvalue() and write it to the file object
filedata = file.getvalue()
with open('abc.png') as f:
f.write(filedata)

Categories

Resources