Develop Reference

Is there a way to avoid double quotation in formatting strings which include quotations inside them in Python

It is not supposed to be a hard problem, but I've worked on it for almost a day!
I want to create a query which has to be in this format: 'lat':'###','long':'###' where ###s represent latitude and longitude.
I am using the following code to generate the queries:
coordinateslist=[]
for i in range(len(lat)):
coordinateslist.append("'lat':'{}','long':'-{}'".format(lat[i],lon[i]))
coordinateslist
However the result would be some thing similar to this which has "" at the beginning and end of it: "'lat':'40.66','long':'-73.93'"
Ridiculously enough it's impossible to remove the " with either .replace or .strip! and wrapping the terms around repr doesn't solve the issue.
Do you know how I can get rid of those double quotation marks?
P.S. I know that when I print the command the "will not be shown but when i use each element of the array in my query, a " will appear at the end of the query which stops it from working.
directly writing the line like this:
query_params = {'lat':'43.57','long':'-116.56'}
works perfectly fine.
but using either of the codes below will lead to an error.
aa=print(coordinateslist[0])
bb=coordinateslist[0]
query_params = {aa}
query_params = {bb}
query_params = aa
query_params = bb

Try using a dictionary instead, if you don't want to see the " from string representation:
coordinateslist.append({
"lat": lat[i],
"long": "-{}".format(lon[i])
})

It is likely that the error you are getting is something else entirely (i.e. unrelated to the quotes you are seeing in printed outputs). From the format of the query, I would guess that it is expecting a properly formatted URL parameters: reverse_geocode.json?... which 'lat':'41.83','long':'-87.68' is not. Did you try to manually call it with a fixed string, (e.g. using with postman) ?
Assuming you're calling the twitter geo/ API, you might want to ry it out with properly separated URL parameters.
geo/reverse_geocode.json?lat=41.83&long=-87.68

Turning x.x.x.x string into list address (???)

This is a difficult problem to explain. I have a string that looks like "system.cpu.total.pct" that I'm pulling from a json configuration file. This particular format is required elsewhere in my program so I cannot change it.
This "system.cpu.total.pct" specifies what field I'm interested in snagging out of metricbeat (in Elasticsearch).
I need to convert this into a list address (? is that what to call it ?) so that I can snag stuff out of an array of database results I'm calling 'rawData'. Right now I'm doing this:
if sourceSet == "system.cpu.total.pct":
dataArray.append(rawData['hits']['hits'][thisRecord]["_source"]['system']['cpu']['total']['pct'])
But that's no good, obviously, because the result is hard-coded.
How can I instead write something like
dataArray.append(rawData['hits']['hits'][thisRecord]["_source"]["system.cpu.total.pct"])
that will work for any arbitrary string?
Any suggestions? Thank you!

you can use:
if sourceSet == "system.cpu.total.pct":
d = rawData['hits']['hits'][thisRecord]["_source"]
for t in sourceSet.split('.'):
d = d[t]
dataArray.append(d)

Issue with a python string loop

So I'm having trouble getting my code to use a list of strings as inputs in a loop. Here's roughly what I have so far.
from arcgis.gis import GIS
Users = ['User01','User02','User03']
User_string = str(Users) # Have to do this as code needs input as string
gis = GIS("https://www.arcgis.com","USERNAME","PASSWORD") # This logs you into ArcGIS Online
User_role = 'org_user'
for x in User_string:
test = gis.users.get(username=x)
test.update_role(role=User_role)
print("Done! Check Web")
I just can't get the loop to work right. When I remove the for loop and put each user name in individually the get user and update role commands work just fine, it's just in the loop that is broken.
The two errors I'm getting is that the username has to be a string. I fixed that by adding the str() command, but I can't get the username to enter into the user.get loop.
Any suggestions? This code is actually looking at an excel file to produce the list of usernames so I can't just hardcode the list into the code. If it helps at all the website I've been using for the ArcGIS portion of the code is this one: https://developers.arcgis.com/python/guide/accessing-and-managing-users/
I should mention that I also tried just printing
test=gis.users.get(username=User_string)
And it came back as None. So I guess my question is how do I get 'User01' to go into the username=x spot?
Thanks much!

You're doing a for with the list as a string so it's looping on each character of the string. You need to do it with the original list.

Based on your code, you are telling your for loop to iterate on a String, since you converted your list to a string with User_string = str(Users); So the loop is going over each character on the string, which now it is User01User02User03
What you need to do is to iterate the list Users, like:
from arcgis.gis import GIS
Users = ['User01','User02','User03']
gis = GIS("https://www.arcgis.com","USERNAME","PASSWORD") # This logs you into ArcGIS Online
User_role = 'org_user'
for x in Users:
test = gis.users.get(username=x)
test.update_role(role=User_role)
print("Done! Check Web")

How to do parsing in python?

I'm kinda new to Python. And I'm trying to find out how to do parsing in Python?
I've got a task: to do parsing with some piece of unknown for me symbols and put it to DB. I guess I can create DB and tables with help of SQLAlchemy, but I have no idea how to do parsing and what all these symbols below mean?
http://joxi.ru/YmEVXg6Iq3Q426
http://joxi.ru/E2pvG3NFxYgKrY
$$HDRPUBID 112701130020011127162536
H11127011300UNIQUEPONUMBER120011127
D11127011300UNIQUEPONUMBER100001112345678900000001
D21127011300UNIQUEPONUMBER1000011123456789AR000000001
D11127011300UNIQUEPONUMBER200002123456987X000000001
D21127011300UNIQUEPONUMBER200002123456987XIR000000000This item is inactive. 9781605600000
$$EOFPUBID 1127011300200111271625360000005
Thanks in advance those who can give me some advices what to start from and how the parsing is going on?

The best approach is to first figure out where each token begins and ends, and write a regular expression to capture these. The site RegexPal might help you design the regex.

As other suggest take a look to some regex tutorials, and also re module help.
Probably you're looking to something like this:
import re
headerMapping = {'type': (1,5), 'pubid': (6,11), 'batchID': (12,21),
'batchDate': (22,29), 'batchTime': (30,35)}
poaBatchHeaders = re.findall('\$\$HDR\d{30}', text)
parsedBatchHeaders = []
batchHeaderDict = {}
for poaHeader in poaBatchHeaders:
for key in headerMapping:
start = headerMapping[key][0]-1
end = headerMapping[key][1]
batchHeaderDict.update({key: poaHeader[start:end]})
parsedBatchHeaders.append(batchHeaderDict)
Then you have list with dicts, each dict contains data for each attribute. I assume that you have your datafile in text which is string. Each dict is made for one found structure (POA Batch Header in example).
If you want to parse it further, you have to made a function to parse each date in each attribute.
def batchDate(batch):
return (batch[0:2]+'-'+batch[2:4]+'-20'+batch[4:])
for header in parsedBatchHeaders:
header.update({'batchDate': batchDate( header['batchDate'] )})
Remember, that's an example and I don't know documentation of your data! I guess it works like that, but rest is up to you.

using list instead of number or string in the query

I would like to use a list of int to be used in a query as below:
db.define_table('customer',Field('name'),Field('cusnumber','integer'))
def custmr():
listOfNumbers=[22,12,76,98]
qry=db(db.customer.cusnumber==listOfNumbers).select(db.customer.name)
print qry
this arise an issue that the only accepted data type in the query is int or str.
Is there any way to avoid this issue (preferably by not using for loop)
Regards

It is really difficult to know what you're trying to ask, but from the syntax of db.define_table(...), I take a wild guess you're on web2py and trying to do a query which fetch any int in your listOfNumbers.
You may use contains attribute like this:
# if all=True, cusnumber will need to contains all listOfNumbers, False means any
qry=db(db.customer.cusnumber.contains(listOfNumbers, all=False)).select(db.customer.name)
You can read more in details in HERE
As OP replied that contains only works for string, I'm going to suggest using for/loop will be a better answer:
listOfNumbers=[22,12,76,98]
for each in listOfNumbers:
qry=db(db.customer.cusnumber==each).select(db.customer.name)
# ... do your stuff or whatever ...

Assuming you want the set of records for which the cusnumber is in listOfNumbers, you should use the .belongs method:
qry = db(db.customer.cusnumber.belongs(listOfNumbers)).select(db.customer.name)