I want so show all the students in a database based on an address the user enters. Some have qualifications, some have not. Regardless of this I would still like to show their details but I am having a difficulty working around the students who have no quailifications.
The below code throws up an error where they have no qualifications but is working fine where they have qualifications.
KeyError: 'qualifications'
How can I work around this?
def call_query(address):
with MongoClient() as client:
db = client.get_database("test")
coll = db.get_collection("docs")
for col in coll.find({"details.address": { "$regex":address}}):
print (col["_id"] , ":", col["details"]["name"],":", col["details"]["age"], col["qualifications"], )
A simple way to work it around is to use a try except in your code :
def call_query(address):
with MongoClient() as client:
db = client.get_database("test")
coll = db.get_collection("docs")
for col in coll.find({"details.address": { "$regex":address}}):
try:
print (col["_id"] , ":", col["details"]["name"],":", col["details"]
["age"], col["qualifications"], )
except KeyError:
print (col["_id"] , ":", col["details"]["name"],":", col["details"]
["age"], "No qualification", )
If the exception is raised because of a None value then it will print "No qualification" for the student
In Python, a dictionary object has a get method that allows to set a default value in case the requested key is not found. So, you can simple col.get("qualifications", "") which returns empty string if there is no "qualifications" key.
Full code:
def call_query(address):
with MongoClient() as client:
db = client.get_database("test")
coll = db.get_collection("docs")
for col in coll.find({"details.address": { "$regex":address}}):
print (
col["_id"] , ":",
col["details"]["name"],
":",
col["details"]["age"],
col.get("qualifications", ""),
)
Related
I have been trying to do some basic search queries, but I am unable to connect to an open LDAP server regardless. I tried a couple of servers, and none of them worked. I used Apache Directory Studio to make sure that the keyword was there but it did not work either way. I tried a variety of different code from different sources.
This was the first one I used
:
https://www.linuxjournal.com/article/6988
import ldap
keyword = "boyle"
def main():
server = "ldap.forumsys.com"
username = "cn=read-only-admin,dc=example,dc=com"
password = "password"
try:
l = ldap.open(server)
l.simple_bind_s(username,password)
print "Bound to server . . . "
l.protocol_version = ldap.VERSION3
print "Searching . . ."
mysearch (l,keyword)
except ldap.LDAPError:
print "Couldnt connect"
def mysearch(l, keyword):
base = ""
scope = ldap.SCOPE_SUBTREE
filter = "cn=" + "*" + keyword + "*"
retrieve_attributes = None
count = 0
result_set = []
timeout = 0
try:
result_id = l.search(base, scope, filter, retrieve_attributes)
while l != 1:
result_id = l.search(base, scope,filter, retrieve_attributes)
result_type, result_data = l.result(result_id, timeout)
if result_data == []:
break
else:
if result_type == ldap.RES_SEARCH_ENTRY:
result_set.append(result_data)
if len (result_set = 0):
print "No Results"
for i in range (len(result_set)):
for entry in result_set[i]:
try:
name = entry[1]['cn'][0]
mail = entry[1]['mail'][0]
#phone = entry[1]['telephonenumber'][0]
#desc = entry[1]['description'][0]
count = count + 1
print name + mail
except:
pass
except ldap.LDAPError, error_message:
print error_message
main()
Every time I ran this program, I received an error
{'desc': u"No such object"}
I also tried this
import ldap
try:
l = ldap.open("ldap.example.com")
except ldap.LDAPError, e:
print e
base_dn = "cn=read-only-admin,dc=example,dc=com"
search_scope = ldap.SCOPE_SUBTREE
retrieve_attributes = None
search_filter = "uid=myuid"
try:
l_search = l.search(base_dn, search_scope, search_filter, retrieve_attributes)
result_status, result_data = l.result(l_search, 0)
print result_data
except ldap.LDAPError, e:
print e
The error on this one was
{'desc': u"Can't contact LDAP server"}
I spent about 5 hours trying to figure this out. I would really appreciate it if you guys could give me some advice. Thanks.
There are several bogus things in there.
I will only comment your first code sample because it can be used by anyone with that public LDAP server.
l = ldap.open(server)
Function ldap.open() is deprecated since many years. You should use function ldap.initialize() with LDAP URI as argument instead like this:
l = ldap.initialize("ldap://ldap.forumsys.com")
l_search = l.search(..)
This is the asynchronous method which just returns a message ID (int) of the underlying OpenLDAP C API (libldap). It's needed if you want to retrieve extended controls returned by the LDAP server along with search results. Is that what you want?
As a beginner you probably want to use the simpler method LDAPObject.search_s() which immediately returns a list of (DN, entry) 2-tuples.
See also: python-ldap -- Sending LDAP requests
while l != 1
This does not make sense at all because l is your LDAPObject instance (LDAP connection object). Note that LDAPObject.search() would raise an exception if it gets an Integer error code from OpenLDAP's libldap. No need to do C-style error checks at this level.
filter = "cn=" + "" + keyword + ""
If keyword can be arbitrary input this is a prone to LDAP injection attacks. Don't do that.
For adding arbitrary input into a LDAP filter use function ldap.filter.escape_filter_chars() to properly escape special characters. Also avoid using variable name filter because it's the name of a built-in Python function and properly enclose the filter in parentheses.
Better example:
ldap_filter = "(cn=*%s*)" % (ldap.filter.escape_filter_chars(keyword))
base = ""
The correct search base you have to use is:
base = "dc=example,dc=com"
Otherwise ldap.NO_SUCH_OBJECT is raised.
So here's a complete example:
import pprint
import ldap
from ldap.filter import escape_filter_chars
BINDDN = "cn=read-only-admin,dc=example,dc=com"
BINDPW = "password"
KEYWORD = "boyle"
ldap_conn = ldap.initialize("ldap://ldap.forumsys.com")
ldap_conn.simple_bind_s(BINDDN, BINDPW)
ldap_filter = "(cn=*%s*)" % (ldap.filter.escape_filter_chars(KEYWORD))
ldap_results = ldap_conn.search_s(
"dc=example,dc=com",
ldap.SCOPE_SUBTREE,
ldap_filter,
)
pprint.pprint(ldap_results)
Use case: Want to store 0.0 as default value while no value is passed for a pewee's DoubleField. I wrote following code but didn't work for me.
class MyRelation(peewee.Model):
id = peewee.PrimaryKeyField()
percentage = peewee.DoubleField(default=0.0)
Here is an api
#api_blueprint.route('/add_data', methods=['POST'])
#http_header
def add_data():
try:
incoming = json.loads(request.data)
data = MyRelation(percentage=incoming["percentage"])
data.save()
return success_response(201,"Data has been inserted :)")
except Exception as e:
log(str(e))
return raise_error(500, str(e))
which logs following error
10-05-17 05:24:51 Line:199 Message: Error in add_data(),views.api: (1048, "Column 'percentage' cannot be null")
The difference is only enforced on the Python side -- so if you're expecting the server to enforce the default you'll need to use a constraint:
percentage = peewee.DoubleField(constraints=[SQL('DEFAULT 0.0')])
Docs: http://docs.peewee-orm.com/en/latest/peewee/models.html#default-field-values
I have this function that it's supposed should receive json and store the values on a RDS MySQL db.
def saveMetric(metrics):
cnx = RDS_Connect()
cursor = cnx.cursor()
jsonMetrics = json.loads(metrics)
#print type(jsonMetrics['Metrics'])
# Every 2000 registries, the script will start overriding values
persistance = 2000
save_metrics_query = (
"REPLACE INTO metrics "
"SET metric_seq = (SELECT COALESCE(MAX(row_id), 0) %% %(persistance)d + 1 FROM metrics AS m), "
"instance_id = \'%(instance_id)s\', "
"service = \'%(service)s\' , "
"metric_name = \'%(metric_name)s\', "
"metric_value = %(metric_value)f"
)
for metric in jsonMetrics['Metrics']:
formatData = {}
formatData['persistance'] = persistance
formatData['instance_id'] = arguments.dimensionValue
formatData['service'] = jsonMetrics['Service']
formatData['metric_name'] = metric
formatData['metric_value'] = jsonMetrics['Metrics'][metric]
print save_metrics_query % formatData
try:
cursor.execute(save_metrics_query, formatData, multi=True)
logger('info','Metrics were saved successfully!')
cnx.commit()
except mysql.connector.Error as err:
logger('error', "Something went wrong: %s" % err)
cursor.close()
cnx.close()
RDS_Connect() was already tested and it works just fine. The problem is that after running the function, the data is not saved to the DB. I think there is a problem with the commit but I don't see any errors or warning message. If I run the query manually, the data gets stored.
Here is the query that runs after parsing the json:
REPLACE INTO metrics SET metric_seq = (SELECT COALESCE(MAX(row_id), 0) % 2000 + 1 FROM metrics AS m), instance_id = 'i-03932937bd67622c4', service = 'AWS/EC2' , metric_name = 'CPUUtilization', metric_value = 0.670000
If it helps, this is the json that the function receives:
{
"Metrics": {
"CPUUtilization": 1.33,
"NetworkIn": 46428.0,
"NetworkOut": 38772.0
},
"Id": "i-03932937bd67622c4",
"Service": "AWS/EC2"
}
I'd appreciate some help.
Regards!
UPDATE:
I found that the problem was related to the formatting codes on the query template.
I re wrote the function like this:
def saveMetric(metrics):
cnx = RDS_Connect()
jsonMetrics = json.loads(metrics)
print json.dumps(jsonMetrics,indent=4)
persistance = 2000
row_id_query_template = "SELECT COALESCE(MAX(row_id), 0) % {} + 1 FROM metrics AS m"
row_id_query = row_id_query_template.format(persistance)
save_metrics_query = (
"REPLACE INTO metrics "
"SET metric_seq = (" + row_id_query + "),"
"instance_id = %(instance_id)s,"
"service = %(service)s,"
"metric_name = %(metric_name)s,"
"metric_value = %(metric_value)s"
)
for metric in jsonMetrics['Metrics']:
formatData = {}
formatData['instance_id'] = arguments.dimensionValue
formatData['service'] = jsonMetrics['Service']
formatData['metric_name'] = metric
formatData['metric_value'] = jsonMetrics['Metrics'][metric]
if arguments.verbose == True:
print "Data: ",formatData
print "Query Template: ",save_metrics_query.format(**formatData)
try:
cursor = cnx.cursor()
cursor.execute(save_metrics_query, formatData)
logger('info','Metrics were saved successfully!')
cnx.commit()
cursor.close()
except mysql.connector.Error as err:
logger('error', "Something went wrong: %s" % err)
cnx.close()
As you can see, I format the SELECT outside. I believe the whole problem was due to this line:
"metric_value = %(metric_value)f"
I changed to:
"metric_value = %(metric_value)s"
and now it works. I think the formatting was wrong, given a syntax error (tho I don't know how the exception was never thrown).
Thanks to everyone who took time to help me!
I haven't actually used MySQL, but the docs seem to indicate that calling cursor.execute with multi=True just returns an iterator. If that is true, then it wouldn't actually insert anything - you'd need to call .next() on the iterator (or just iterate over it) to actually insert the record.
It also goes on to advise against using parameters with multi=True:
If multi is set to True, execute() is able to execute multiple statements specified in the operation string. It returns an iterator that enables processing the result of each statement. However, using parameters does not work well in this case, and it is usually a good idea to execute each statement on its own.
tl;dr: remove that parameter, as the default is False.
This was the solution. I changed:
"metric_value = %(metric_value)f"
To:
"metric_value = %(metric_value)s"
Doing some troubleshooting I found a syntax error on the SQL. Somehow the exception didn't show.
I have a script that calls a list of linux guests I am trying to tidy up. Here is the code:
#!/usr/bin/python
guests = ['guest1','guest2','guest3','guest*']
def serverCheck(guestList)
for g in guestList:
server = AdminControl.completeObjectName('cell=tstenvironment,node=guest1,name=uatenvironment,type=Server,*')
try:
status = AdminControl.getAttribute(server, 'state')
print g + status
except:
print "Error %s is down." % g
serverCheck(guests)
The problem lies in this line:
server = AdminControl.completeObjectName('cell=Afcutst,node=%s,name=afcuuat1,type=Server,*') % g
How do I use my list to populate the node variable while still being able to pass the info within the parentheses to the AdminControl function?
The argument string itself is the argument to the % operator, not the return value of the function call.
server = AdminControl.completeObjectName(
'cell=Afcutst,node=%s,name=afcuuat1,type=Server,*' % (g,)
)
Peeking into the crystal ball, Python 3.6 will allow you to write
server = AdminControl.completeObjectName(
f'cell=Afcutst,node={g},name=afcuuat1,type=Server,*'
)
embedding the variable directly into a special format string literal.
can you try like this
AdminControl.completeObjectName('cell=tstenvironment,node=%s,name=uatenvironment,type=Server,*'%g)
For more readability I would suggest this and also using the same way to format strings from variables (here I chose str.format)
guests = ['guest1','guest2','guest3','guest*']
def serverCheck(guestList)
name_tpl = 'cell=tstenvironment,node={},name=uatenvironment,type=Server,*'
for g in guestList:
obj_name = name_tpl.format(g)
server = AdminControl.completeObjectName(obj_name)
try:
status = AdminControl.getAttribute(server, 'state')
print '{}: {}'.format(g, status)
except:
print 'Error {} is down'.format(g)
serverCheck(guests)
I am new programing and I developed a small webservice in python using SOAPpy.
This webservice get some data from a sql Server and it returns in a list of tuples with this format [(a,b,c),(d,e,f),...]
This is the server code:
class ws():
def recoveruserdata(self,param1,param2):
tupla_recover = ()
l_recover = []
...
cnxn = pyodbc.connect(con_string)
...
cursor.execute(query)
rows = cursor.fetchall()
for row in rows:
if row.id != ' ':
tupla_recover = (row.id,row.date)
l_recover.append(tupla_recover)
print l_recover
print "Len of lrecover ", len(l_recover)
print "first element ",l_recover[0]
return l_recover
if __name__ == '__main__':
server = SOAPServer(("localhost", 8082))
wsobj = ws()
server.registerObject(wsobj)
server.serve_forever()
The problem ocurrs when I try to consume the web service from a python client using SOAPpy too. I can get a list but it not contains the tuples as above, instead of it, you get a list of all individual elements with this format [a,b,c,d,e,f,....]
This is the client code:
from SOAPpy import SOAPProxy
server = SOAPProxy("http://localhost:8082/")
cliente = server.recoveruserdata(param1,param2)
print cliente
My question is if, is it normal this behavior or am I doing something wrong?
I apreciate your help