Develop Reference

Set intersection working, however printing all output instead of only the differences

I am getting the list of AWS resources that are tagged using set.difference to give me only the duplicates. This works, however when printing instead of getting only the differences I am getting all results.
Where am I going wrong with my script?
My code:
def extract_tags():
session = boto3.Session(profile_name='dev-apps-read-only')
client = session.client('resourcegroupstaggingapi',
region_name='ap-southeast-1')
paginator = client.get_paginator('get_resources')
page_iterator = paginator.paginate()
for page in page_iterator:
for resource in page['ResourceTagMappingList']:
resource_arn = resource['ResourceARN']
tags_list = []
for key_value_pair in resource['Tags']:
key_name = key_value_pair['Key']
key_value = key_value_pair['Value']
tags_list.append(key_name)
# list to check against the tag output from AWS
mandatory_tags_list = [
"phx.tfe_workspace",
"aws.region",
"phx.bounded_context",
"phx.owner",
"phx.service_tier"
]
# tag list from AWS. Variable declared in main loop above
tags_list
non_compliant_tags = ('Non-Compliant Tags:',
(set(mandatory_tags_list).difference(tags_list)))
count = len(set(mandatory_tags_list) & set(tags_list))
print(json.dumps(f'resource ARN: {resource_arn}'))
print(json.dumps(f'{non_compliant_tags}', indent=2))
print(count)
extract_tags()
My output gives me complaint resources and non-compliant resources. I only would like non-compliant resources:
Wrong:
"resource ARN:0a895d1c83fdbd76a"
"('Non-Compliant Tags:', {'phx.service_tier', 'phx.bounded_context', 'phx.tfe_workspace', 'phx.owner', 'aws.region'})"
Correct:
"resource ARN:e5b5ccfa-db6e-e027-9fab-0415d4958c30"
"('Non-Compliant Tags:', {'phx.service_tier', 'phx.tfe_workspace'})"
3

How to prevent triples from getting mixed up while uploading to Dydra programmatically?

I am trying to upload some data to Dydra from a Sesame triplestore I have on my computer. While the download from Sesame works fine, the triples get mixed up (the s-p-o relationships change as the object of one becomes object of another). Can someone please explain why this is happening and how it can be resolved? The code is below:
#Querying the triplestore to retrieve all results
sesameSparqlEndpoint = 'http://my.ip.ad.here:8080/openrdf-sesame/repositories/rep_name'
sparql = SPARQLWrapper(sesameSparqlEndpoint)
queryStringDownload = 'SELECT * WHERE {?s ?p ?o}'
dataGraph = Graph()
sparql.setQuery(queryStringDownload)
sparql.method = 'GET'
sparql.setReturnFormat(JSON)
output = sparql.query().convert()
print output
for i in range(len(output['results']['bindings'])):
#The encoding is necessary to parse non-English characters
output['results']['bindings'][i]['s']['value'].encode('utf-8')
try:
subject_extract = output['results']['bindings'][i]['s']['value']
if 'http' in subject_extract:
subject = "<" + subject_extract + ">"
subject_url = URIRef(subject)
print subject_url
predicate_extract = output['results']['bindings'][i]['p']['value']
if 'http' in predicate_extract:
predicate = "<" + predicate_extract + ">"
predicate_url = URIRef(predicate)
print predicate_url
objec_extract = output['results']['bindings'][i]['o']['value']
if 'http' in objec_extract:
objec = "<" + objec_extract + ">"
objec_url = URIRef(objec)
print objec_url
else:
objec = objec_extract
objec_wip = '"' + objec + '"'
objec_url = URIRef(objec_wip)
# Loading the data on a graph
dataGraph.add((subject_url,predicate_url,objec_url))
except UnicodeError as error:
print error
#Print all statements in dataGraph
for stmt in dataGraph:
pprint.pprint(stmt)
# Upload to Dydra
URL = 'http://dydra.com/login'
key = 'my_key'
with requests.Session() as s:
resp = s.get(URL)
soup = BeautifulSoup(resp.text,"html5lib")
csrfToken = soup.find('meta',{'name':'csrf-token'}).get('content')
# print csrf_token
payload = {
'account[login]':key,
'account[password]':'',
'csrfmiddlewaretoken':csrfToken,
'next':'/'
}
# print payload
p = s.post(URL,data=payload, headers=dict(Referer=URL))
# print p.text
r = s.get('http://dydra.com/username/rep_name/sparql')
# print r.text
dydraSparqlEndpoint = 'http://dydra.com/username/rep_name/sparql'
for stmt in dataGraph:
queryStringUpload = 'INSERT DATA {%s %s %s}' % stmt
sparql = SPARQLWrapper(dydraSparqlEndpoint)
sparql.setCredentials(key,key)
sparql.setQuery(queryStringUpload)
sparql.method = 'POST'
sparql.query()

A far simpler way to copy your data over (apart from using a CONSTRUCT query instead of a SELECT, like I mentioned in the comment) is simply to have Dydra itself directly access your Sesame endpoint, for example via a SERVICE-clause.
Execute the following on your Dydra database, and (after some time, depending on how large your Sesame database is), everything will be copied over:
INSERT { ?s ?p ?o }
WHERE {
SERVICE <http://my.ip.ad.here:8080/openrdf-sesame/repositories/rep_name>
{ ?s ?p ?o }
}
If the above doesn't work on Dydra, you can alternatively just directly access the RDF statements from your Sesame store by using the URI http://my.ip.ad.here:8080/openrdf-sesame/repositories/rep_name/statements. Assuming Dydra has an upload-feature where you can provide the URL of an RDF document, you can simply provide it the above URI and it should be able to load it.

The code above can work if the following changes are made:
Use CONSTRUCT query instead of SELECT. Details here -> How to iterate over CONSTRUCT output from rdflib?
Use key as input for both account[login] and account[password]
However, this is probably not the most efficient way. Primarily, doing individual INSERTs for every triple is not a good way. Dydra doesn't record all statements this way (I got only about 30% of the triples inserted). On the contrary, using the http://my.ip.ad.here:8080/openrdf-sesame/repositories/rep_name/statements method as suggested by Jeen enabled me to port all the data successfully.

send multiple record to a particular user by mail

I need to send mail to particular user from the database. i have data table where 1000 of records are available. each record have some email ids. or some id has assigned to multiple record . i need to send record to the particular email only.
I have form from where i used to select company name and strt date. from these i fetch all the value into list. now worries is how to send record to particular user
let my code example
def sendmail():
form = SQLFORM.factory(Field('select_company', db.company, represent=lambda c, row:get_company_name(c), requires = IS_IN_DB(db,'company.id','%(company)s')),
Field('choose_start_date','date',requires=IS_NOT_EMPTY(error_message='Please choose Start Date')),
Field('choose_end_date','date',requires=IS_NOT_EMPTY(error_message='Please choose end Date')))
if form.accepts(request,session):
session.company = form.vars.select_company
session.strtdate = form.vars.choose_start_date
session.enddate = form.vars.choose_end_date
mailidr()
return dict(form=form)
def mailidr():
from gluon.tools import Mail,datetime
mail=Mail()
#specify server
mail=auth.settings.mailer
mail.settings.server='smtp.gmail.com:587'
mail.settings.login='comdummy09#gmail.com:critical#009'
#specify address to send as
mail.settings.sender='comdummy09#gmail.com'
#send the message
list = []
for row in db((db.tracker.company == session.company) & (db.tracker.Complience_Status != 1) & (db.tracker.periodicity_due_date >= session.strtdate) & (db.tracker.periodicity_due_date <= session.enddate)).select():
list.append('<tr><td style="padding-top:5px;padding-bottom:5px;padding-right:5px;padding-left:5px;">'+row.Compliance_Area+'</td><td style="padding-top:5px;padding-bottom:5px;padding-right:5px;padding-left:5px;">'+row.law_description+'</td><td style="padding-top:5px;padding-bottom:5px;padding-right:5px;padding-left:5px;" >'''+row.activity_name+'''</td><td style="padding-top:5px;padding-bottom:5px;padding-right:5px;padding-left:5px;">'+str(row.periodicity_due_date)+'</td></tr>')
mail.send(to=[row.client_owner],
subject='Compliance and Due_Date',
message='<html>'
'<body>'
'<h1>Test Mails</h1>'
'<h2>Dear \t' +session.company+ '</h2>'
'</br><h3>REMINDER</h3>'
'<p>Please note that the fol1ing records are due between ' +session.strtdate+ ' to ' +session.enddate+ '</p>'
'<table border=1>'
'<thead >'
'<tr>'
'<th style="padding-top:5px;padding-bottom:5px;padding-right:5px;padding-left:5px;">Compliance Area</th>'
'<th>record</th><th>date (Due Date)</th>'
'</tr>'
'</thead>'
'<tbody>'
+str(list)+
'</tbody>'
'</table>'
'</body>'
'</html>')
response.flash='mail send'
return ''
from this code i a able to send mail. but problem is that user get n no mails
if i used to take mail.send to out side the function its not work

from the docs (http://www.web2py.com/books/default/chapter/29/08/emails-and-sms#Setting-up-email)
Mail returns True if it succeeds in sending the email and False
otherwise. A complete argument list for mail.send() is as follows:
So, for a test you should try capturing the response - e.g. change:
mail.send(to=[row.client_owner],
to
mail_response = mail.send(to=[row.client_owner],
and then return locals() instead of "" so you can see in a test page what (if any) failure you are getting in the send. this is just for testing...
A permanent alternate method is shows here: i am trying send email using web2py with gmail and using smtp setting i have attached all code

Sending automatic SMS alerts Python/Django

I'm building a website using Django. And I want that users can be able to receive alerts via SMS when new topics are posted.
I tested textlocal but I had an issue when trying to send SMS to multiple numbers (numbers = ['xxxxx','xxxxx']) .(I don't want to use group_id).
Generally I want to be able to do something like this:
numbers = (SELECT number FROM users WHERE SMS_subscribe=1)
sender = 'mywebsite'
message = 'Hey, a new topic was posted'
send_sms(numbers, message, sender)
My textlocal test code:
#!/user/bin/python
# -*- coding: utf-8 -*-
from urllib2 import Request, urlopen
from urllib import urlencode
def send_sms(uname, hash_code, numbers, message, sender):
data = urlencode({
'username' : uname,
'hash' : hash_code,
'numbers' : numbers,
'message' : message,
'sender' : sender,
'test' : True
})
#data = data.encode('utf-8')
request = Request('https://api.txtlocal.com/send/?')
response = urlopen(request, data)
return response.read()
def just_one_sms_message(message, annonce_link, sender):
links_len=len(annonce_link) + len(sender) + 1
sms_max_len = 160 - links_len
if len(message)>sms_max_len:
message = message[:sms_max_len-6]+'... : '
else:
message += ' : '
return message + annonce_link + '\n' + sender
username = 'xxxxxxx#gmail.com'
hash_code = '3b5xxxxxxxxxxxxxxxxxxxxxxxxxxx8d83818'
numbers = ('2126xxxxx096','2126xxxxx888')
annonce_link = 'http://example.com/'
sender = 'sender'
message = 'New topics..'
message = just_one_sms_message(message, annonce_link, sender)
resp = send_sms(username, hash_code, numbers, message, sender)
print resp
Executing this code I get this error :
{"warnings":[{"code":3,"message":"Invalid number"}],"errors":[{"code":4,"message":"No recipients specified"}],"status":"failure"}
But if I change: numbers=('2126xxxxx096')it works.
What is the best way or web service to do this ?

There are a couple issues you're running into. The first is how tuple literals are defined.
('somenumber') is equivalent to 'somenumber' in python. It's just a string. The parentheses alone do not define a tuple literal. To define a single-element tuple literal, you need a trailing comma of the first element. E.G. ('somenumber',).
The second issue is how urlencode works. For each value in the data dictionary, it asks for the string representation.
In the case of ('2126xxxxx096','2126xxxxx888'), since it's evaluated as a tuple, it's encoded as ('2126xxxxx096','2126xxxxx888'), resulting in %28%272126xxxxx096%27%2C+%272126xxxxx888%27%29.
In the case of ('2126xxxxx096'), since it's evaluated as a string, it's encoded as 2126xxxxx096. Notice the lack of junk characters like %28 and %29.
So, in short, since the value of numbers in the urlencode dictionary is a tuple when you have multiple numbers, you need to convert the tuple into a comma-separated string. This can be accomplished via ",".join(numbers), which in the case of ('2126xxxxx096','2126xxxxx888') produces 2126xxxxx096%2C2126xxxxx888. With the fixed encoding, your message should now send to multiple numbers.

Ajax to Python Pyramid data

So far, I have managed to take a bunch of HTML elements for whose contentEditable attribute is True and join their id's and HTML data together to make an Ajax data string. I can get the serialized data back to the server, no problem. For example,
$(document).ready(function(){
$("#save").click(function(){
var ajax_string = ''
$( "[contenteditable=True]" ).each(function( intIndex ){
ajax_string = ajax_string + '&' + $("[contenteditable=True]")[intIndex].id + ':' + $(this).html();
});
$.ajax({
type:"POST",
url:"/episode_edit/{{ episode.ID_Episode }}",
data:ajax_string,
success:function(result){
<!--alert( ajax_string );-->
}
});
});
});
On the server:
for r in request.params: print r
I get strings:
AltTitle:some Alt Title
PrintTitle:The Print Title
Notes:A bunch o' notes.
My dilema now is that I need to convert each request.param string into a dictionary object, so I can map it back to my database model. I can think of some very ugly ways of doing this, but what is the best way?

You say you want to convert each request.param string into a dictionary object, but is that what you meant? It looks like each string is just a key/value pair.
You can pretty simply create a dictionary from those values using:
opts = {}
for r in request.params:
parts = r.split(':', 1)
if len(parts) == 2:
opts[parts[0]] = parts[1]
else:
# some error condition