This is a difficult problem to explain. I have a string that looks like "system.cpu.total.pct" that I'm pulling from a json configuration file. This particular format is required elsewhere in my program so I cannot change it.
This "system.cpu.total.pct" specifies what field I'm interested in snagging out of metricbeat (in Elasticsearch).
I need to convert this into a list address (? is that what to call it ?) so that I can snag stuff out of an array of database results I'm calling 'rawData'. Right now I'm doing this:
if sourceSet == "system.cpu.total.pct":
dataArray.append(rawData['hits']['hits'][thisRecord]["_source"]['system']['cpu']['total']['pct'])
But that's no good, obviously, because the result is hard-coded.
How can I instead write something like
dataArray.append(rawData['hits']['hits'][thisRecord]["_source"]["system.cpu.total.pct"])
that will work for any arbitrary string?
Any suggestions? Thank you!
you can use:
if sourceSet == "system.cpu.total.pct":
d = rawData['hits']['hits'][thisRecord]["_source"]
for t in sourceSet.split('.'):
d = d[t]
dataArray.append(d)
Related
I have a tuple that I received after getting some data from an SQLite3 database and it looks like this (‘1’, ‘dextron’). How would I remove the brackets and add dextron to a string
After a bit of fiddling around i got it to work.
info = ("1','dextron')
user = info[1]
print(user)
Not sure if this is the best way to do it in python or not id like to hear any better ideas or imporvements
I get back an XML object from a mssql server when I call a SP from Python (2.7). I get it in the following form:
{u'XML_F52E2B61-18A1-11d1-B105-00805F49916B': 'D\x02i\x00d\x00D\x05d\x00e\x00s\x00c\x00r\x00D\x0bd\x00a\x00t\x00a\x00t\x00y\x00p\x00e\x00_\x00i\x00d\x00D\x13e\x00n\x00u\x00m\x00e\x00r\x00a\x00t\x00i\x00o\x00n\x00_\x00t\x00y\x00p\x00e\x00_\x00i\x00d\x00D\rs\x00y\x00s\x00t\x00e\x00m\x00f\x00e\x00a\x00t\x00u\x00r\x00e\x00D\x04l\x00i\x00n\x00k\x00D\x07F\x00e\x00a\x00t\x00u\x00r\x00e\x00\x01\x00\x08F\x00e\x00a\x00t\x00u\x00r\x00e\x00S\x00A\x01\x07A\x01\x01A\x03B\x01\x00\x00\x00\x81\x01\x01\x02A\x03\x11\x1a\x00r\x00e\x00s\x00p\x00o\x00n\x00d\x00e\x00n\x00t\x00_\x00i\x00d\x00\x81\x02\x01\x03A\x03B\x01\x00\x00\x00\x81\x03\x01\x05A\x03F\x01\x81\x05\x01\x06A\x03F\x00\x81\x06\x81\x07\x01\x07A\x01\x01A\x03B\x02\x00\x00\x00\x81\x01\x01\x02A\x03\x11 \x00W\x00o\x00r\x00k\x00s\x00 \x00a\x00t\x00 \x00c\x00o\x00m\x00p\x00a\x00n\x00y\x00\x81\x02\x01\x03A\x03B\x01\x00\x00\x00\x81\x03\x01\x05A\x03F\x01\x81\x05\x01\x06A\x03F\x01\x81\x06\x81\x07\x01\x07A\x01\x01A\x03B\x03\x00\x00\x00\x81\x01\x01\x02A\x03\x11\x0c\x00G\x00e\x00n\x00d\x00e\x00r\x00\x81\x02\x01\x03A\x03B\x08\x00\x00\x00\x81\x03\x01\x04A\x03B\x01\x00\x00\x00\x81\x04\x01\x05A\x03F\x00\x81\x05\x01\x06A\x03F\x00\x81\x06\x81\x07\x81\x00\x08F\x00e\x00a\x00t\x00u\x00r\x00e\x00S\x00'}
I have two questions:
1: What encoding is this?
2: What library should I use to decode this?
Addition:
The XML as it shows in the SQL Management Studio:
The SP:
ALTER PROCEDURE [dbo].[rdb_sql2python]
AS
BEGIN
SET NOCOUNT ON
SELECT * FROM [_rdb].[dbo].[features] FOR XML RAW ('Feature'), ROOT ('FeatureS'), ELEMENTS
SET NOCOUNT OFF
END
I try something like an answer, at least to the question: What is this:
At this JSON-viewer your string as you presented it did not work. But when I removed the "u", replaced the single quotes with double quotes and removed the "D" it worked somehow:
This string
{"XML_F52E2B61-18A1-11d1-B105-00805F49916B":
"\x02i\x00d\x00D\x05d\x00e\x00s\x00c\x00r\x00D\x0bd\x00a\x00t\x00a\x00t\x00y\x00p\x00e\x00_\x00i\x00d\x00D\x13e\x00n\x00u\x00m\x00e\x00r\x00a\x00t\x00i\x00o\x00n\x00_\x00t\x00y\x00p\x00e\x00_\x00i\x00d\x00D\rs\x00y\x00s\x00t\x00e\x00m\x00f\x00e\x00a\x00t\x00u\x00r\x00e\x00D\x04l\x00i\x00n\x00k\x00D\x07F\x00e\x00a\x00t\x00u\x00r\x00e\x00\x01\x00\x08F\x00e\x00a\x00t\x00u\x00r\x00e\x00S\x00A\x01\x07A\x01\x01A\x03B\x01\x00\x00\x00\x81\x01\x01\x02A\x03\x11\x1a\x00r\x00e\x00s\x00p\x00o\x00n\x00d\x00e\x00n\x00t\x00_\x00i\x00d\x00\x81\x02\x01\x03A\x03B\x01\x00\x00\x00\x81\x03\x01\x05A\x03F\x01\x81\x05\x01\x06A\x03F\x00\x81\x06\x81\x07\x01\x07A\x01\x01A\x03B\x02\x00\x00\x00\x81\x01\x01\x02A\x03\x11
\x00W\x00o\x00r\x00k\x00s\x00 \x00a\x00t\x00
\x00c\x00o\x00m\x00p\x00a\x00n\x00y\x00\x81\x02\x01\x03A\x03B\x01\x00\x00\x00\x81\x03\x01\x05A\x03F\x01\x81\x05\x01\x06A\x03F\x01\x81\x06\x81\x07\x01\x07A\x01\x01A\x03B\x03\x00\x00\x00\x81\x01\x01\x02A\x03\x11\x0c\x00G\x00e\x00n\x00d\x00e\x00r\x00\x81\x02\x01\x03A\x03B\x08\x00\x00\x00\x81\x03\x01\x04A\x03B\x01\x00\x00\x00\x81\x04\x01\x05A\x03F\x00\x81\x05\x01\x06A\x03F\x00\x81\x06\x81\x07\x81\x00\x08F\x00e\x00a\x00t\x00u\x00r\x00e\x00S\x00"}
converts to
Name: XML_F52E2B61-18A1-11d1-B105-00805F49916B
Value: "idDdescrD datatype_idDenumeration_type_idD systemfeatureDlinkDFeatureFeatureSAAABArespondent_idABAFAFAABA Works at companyABAFAFAABA GenderABABAFAFFeatureS"
This is - for sure - not the final solution, but it's clear, that this is BSON encoded JSON.
It might be a good idea to show (the relevant parts of) you(r) SP and the way you are calling this. Might be, that there is a completely different / better approach...
I am trying to write out some basic python for my kolab email server. For the primary_mail, I want it to be first initial last name, such as jdoe. The default is first name (dot) last name. john.doe#domain.com
I have came up the following:
primary_mail ='%(givenname)s'[0:1]%(surname)s#%(domain)s
Which I want to basically say jdoe#domain.com
givenname would be someone's full name. (i.e John)
surname would be someone's last name. (i.e Doe)
domain is the email domain. domain.com
When python goes to canonify it, it comes up with some mumbo jumbo like so:
'john[0:1]'doe#domain.com
Can someone help me out with correcting this? I am so close.
EDIT:
According to kolab documentation, it looks like it is something like:
"{0}#{1}": "format('%(uid)s', '%(domain)s')"
This of course doesn't work for me though....
EDIT 2:
I am getting the following in my error logs:
imaps[1916]: ptload completely failed: unable to canonify identifier: 'john'[0:1]doe#domain.com
String formatting is by far the easiest, most readable and preferred way of accomplishing this:
first_name = 'John'
surname = 'Smith'
domain = 'company.com'
primary_mail = '{initial}{surname}#{domain}'.format(initial=first_name[0].lower(), surname=surname.lower(), domain=domain)
primary_mail now equals 'jsmith#company.com'. You define a string containing named placeholders in braces, then call the format method to have those placeholders replaced at runtime with the appropriate values. Here, we take the first character of first_name and convert it to lower case, convert the entirety of surname also, and leave domain unchanged.
You can read more on string formatting at the Python 2.7 docs.
James Scholes is right that format is a better way of doing it, however reading the Kolab documentation it seems that you can only give the format string, and they use the % style formatter internally, where you can't change it. From
the Kolab 'primary_mail' documentation
primary_mail = %(givenname)s.%(surname)s#%(domain)s
The equivalent of the following Python is then executed:
primary_mail = "%(givenname)s.%(surname)s#%(domain)s" % {
"givenname": "Maria",
"surname": "Moller",
"preferredlanguage": "en_US"
}
In this case, we need a modifier to the format conversation. We have %(givenname)s, which ensures that givenname is a string. We can also specify a minimum length, followed by a . and then a precision. This is normally only used it numbers, but we can use it for strings, too. Here is a format string with no minimum length, but a maximum length (precision) of 1 character:
"%(givenname).1s"
So you probably want a string like this:
"%(givenname).1s%(surname)#%(domain)"
I have a problem that I would like to know how to efficiently tackle.
I have data that is JSON-formatted (used with dumps / loads) and contains unicode.
This is part of a protocol implemented with JSON to send messages. So messages will be sent as strings and then loaded into python dictionaries. This means that the representation, as a python dictionary, afterwards will look something like:
{u"mykey": u"myVal"}
It is no problem in itself for the system to handle such structures, but the thing happens when I'm going to make a database query to store this structure.
I'm using pyOrient towards OrientDB. The command ends up something like:
"CREATE VERTEX TestVertex SET data = {u'mykey': u'myVal'}"
Which will end up in the data field getting the following values in OrientDB:
{'_NOT_PARSED_': '_NOT_PARSED_'}
I'm assuming this problem relates to other cases as well when you wish to make a query or somehow represent a data object containing unicode.
How could I efficiently get a representation of this data, of arbitrary depth, to be able to use it in a query?
To clarify even more, this is the string the db expects:
"CREATE VERTEX TestVertex SET data = {'mykey': 'myVal'}"
If I'm simply stating the wrong problem/question and should handle it some other way, I'm very much open to suggestions. But what I want to achieve is to have an efficient way to use python2.7 to build a db-query towards orientdb (using pyorient) that specifies an arbitrary data structure. The data property being set is of the OrientDB type EMBEDDEDMAP.
Any help greatly appreciated.
EDIT1:
More explicitly stating that the first code block shows the object as a dict AFTER being dumped / loaded with json to avoid confusion.
Dargolith:
ok based on your last response it seems you are simply looking for code that will dump python expression in a way that you can control how unicode and other data types print. Here is a very simply function that provides this control. There are ways to make this function more efficient (for example, by using a string buffer rather than doing all of the recursive string concatenation happening here). Still this is a very simple function, and as it stands its execution is probably still dominated by your DB lookup.
As you can see in each of the 'if' statements, you have full control of how each data type prints.
def expr_to_str(thing):
if hasattr(thing, 'keys'):
pairs = ['%s:%s' % (expr_to_str(k),expr_to_str(v)) for k,v in thing.iteritems()]
return '{%s}' % ', '.join(pairs)
if hasattr(thing, '__setslice__'):
parts = [expr_to_str(ele) for ele in thing]
return '[%s]' % (', '.join(parts),)
if isinstance(thing, basestring):
return "'%s'" % (str(thing),)
return str(thing)
print "dumped: %s" % expr_to_str({'one': 33, 'two': [u'unicode', 'just a str', 44.44, {'hash': 'here'}]})
outputs:
dumped: {'two':['unicode', 'just a str', 44.44, {'hash':'here'}], 'one':33}
I went on to use json.dumps() as sobolevn suggested in the comment. I didn't think of that one at first since I wasn't really using json in the driver. It turned out however that json.dumps() provided exactly the formats I needed on all the data types I use. Some examples:
>>> json.dumps('test')
'"test"'
>>> json.dumps(['test1', 'test2'])
'["test1", "test2"]'
>>> json.dumps([u'test1', u'test2'])
'["test1", "test2"]'
>>> json.dumps({u'key1': u'val1', u'key2': [u'val21', 'val22', 1]})
'{"key2": ["val21", "val22", 1], "key1": "val1"}'
If you need to take more control of the format, quotes or other things regarding this conversion, see the reply by Dan Oblinger.
I would like to use a list of int to be used in a query as below:
db.define_table('customer',Field('name'),Field('cusnumber','integer'))
def custmr():
listOfNumbers=[22,12,76,98]
qry=db(db.customer.cusnumber==listOfNumbers).select(db.customer.name)
print qry
this arise an issue that the only accepted data type in the query is int or str.
Is there any way to avoid this issue (preferably by not using for loop)
Regards
It is really difficult to know what you're trying to ask, but from the syntax of db.define_table(...), I take a wild guess you're on web2py and trying to do a query which fetch any int in your listOfNumbers.
You may use contains attribute like this:
# if all=True, cusnumber will need to contains all listOfNumbers, False means any
qry=db(db.customer.cusnumber.contains(listOfNumbers, all=False)).select(db.customer.name)
You can read more in details in HERE
As OP replied that contains only works for string, I'm going to suggest using for/loop will be a better answer:
listOfNumbers=[22,12,76,98]
for each in listOfNumbers:
qry=db(db.customer.cusnumber==each).select(db.customer.name)
# ... do your stuff or whatever ...
Assuming you want the set of records for which the cusnumber is in listOfNumbers, you should use the .belongs method:
qry = db(db.customer.cusnumber.belongs(listOfNumbers)).select(db.customer.name)