I have a script that works fine in Python2 but I can't get it to work in Python3. I want to base64 encode each item in a list and then write it to a json file. I know I can't use map the same way in Python3 but when I make it a list I get a different error.
import base64
import json
list_of_numbers = ['123456', '234567', '345678']
file = open("orig.json", "r")
json_object = json.load(file)
list = ["[{\"number\":\"" + str(s) + "\"}]" for s in list_of_numbers]
base64_bytes = map(base64.b64encode, list)
json_object["conditions"][1]["value"] = base64_bytes
rule = open("new.json", "w")
json.dump(json_object, rule, indent=2, sort_keys=True)
rule.close()
I'm not sure if your error is related to this, but here's what I think might be the problem. When you map a function, the returned value becomes a map object. To get the results as a list again, you need to cast it back to a list after you map your function. In other words:
base64_bytes = list(map(base64.b64encode, list))
P.S. It's better to avoid list as your variable name since it's the name of the built-in function list.
Related
I want to read a dictionary from a text file. This dictionary seems like {'key': [1, ord('#')]}. I read about eval() and literal_eval(), but none of those two will work due to ord().
I also tried json.loads and json.dumps, but no positive results.
Which other way could I use to do it?
So Assuming you read the text file in with open as a string and not with json.loads you could do some simple regex searching for what is between the parenthesis of ord e.g ord('#') -> #
This is a minimal solution that reads everything from the file as a single string then finds all instances of ord and places the integer representation in an output list called ord_. For testing this example myfile.txt was a text file with the following in it
{"key": [1, "ord('#')"],
"key2": [1, "ord('K')"]}
import json
import re
with open(r"myfile.txt") as f:
json_ = "".join([line.rstrip("\n") for line in f])
rgx = re.compile(r"ord\(([^\)]+)\)")
rgd = rgx.findall(json_)
ord_ = [ord(str_.replace(r"'", "")) for str_ in rgd]
json.dump() and json.load() will not work because ord() is not JSON Serializable (meaning that the function cannot be a JSON object.
Yes, eval is really bad practice, I would never recommend it to anyone for any use.
The best way I can think of to solve this is to use conditions and an extra list.
# data.json = {'key': [1, ['ord', '#']]} # first one is function name, second is arg
with open("data.json") as f:
data = json.load(f)
# data['key'][1][0] is "ord"
if data['key'][1][0] == "ord":
res = ord(data['key'][1][1])
i have following string in python
b'{"personId":"65a83de6-b512-4410-81d2-ada57f18112a","persistedFaceIds":["792b31df-403f-4378-911b-8c06c06be8fa"],"name":"waqas"}'
I want to print the all alphabet next to keyword "name" such that my output should be
waqas
Note the waqas can be changed to any number so i want print any name next to keyword name using string operation or regex?
First you need to decode the string since it is binary b. Then use literal eval to make the dictionary, then you can access by key
>>> s = b'{"personId":"65a83de6-b512-4410-81d2-ada57f18112a","persistedFaceIds":["792b31df-403f-4378-911b-8c06c06be8fa"],"name":"waqas"}'
>>> import ast
>>> ast.literal_eval(s.decode())['name']
'waqas'
It is likely you should be reading your data into your program in a different manner than you are doing now.
If I assume your data is inside a JSON file, try something like the following, using the built-in json module:
import json
with open(filename) as fp:
data = json.load(fp)
print(data['name'])
if you want a more algorithmic way to extract the value of name:
s = b'{"personId":"65a83de6-b512-4410-81d2-ada57f18112a",\
"persistedFaceIds":["792b31df-403f-4378-911b-8c06c06be8fa"],\
"name":"waqas"}'
s = s.decode("utf-8")
key = '"name":"'
start = s.find(key) + len(key)
stop = s.find('"', start + 1)
extracted_string = s[start : stop]
print(extracted_string)
output
waqas
You can convert the string into a dictionary with json.loads()
import json
mystring = b'{"personId":"65a83de6-b512-4410-81d2-ada57f18112a","persistedFaceIds":["792b31df-403f-4378-911b-8c06c06be8fa"],"name":"waqas"}'
mydict = json.loads(mystring)
print(mydict["name"])
# output 'waqas'
First you need to convert the string into a proper JSON Format by removing b from the string using substring in python suppose you have a variable x :
import json
x = x[1:];
dict = json.loads(x) //convert JSON string into dictionary
print(dict["name"])
I am trying to get a json response decode with utf and access the dictionaries in the list. The following is the JSON response
'[{"id":26769687,"final_price":58.9,"payment_method_cost":"\\u003cem\\u003e+ 0,00 €\\u003c/em\\u003e \\u003cspan\\u003eΑντικαταβολή\\u003c/span\\u003e","net_price":53.9,"net_price_formatted":"53,90 €","final_price_formatted":"58,90 €","shop_id":649,"no_credit_card":false,"sorting_score":[-5.0,-156,-201,649,20],"payment_method_cost_supported":true,"free_shipping_cost_supported":false,"shipping_cost":"\\u003cem\\u003e+ 5,00 €\\u003c/em\\u003e \\u003cspan\\u003eΜεταφορικά\\u003c/span\\u003e","link":"/products/show/26769687"},
{"id":26771682,"final_price":55.17,"payment_method_cost":"\\u003cem\\u003e+ 2,83 €\\u003c/em\\u003e \\u003cspan\\u003eΑντικαταβολή\\u003c/span\\u003e","net_price":48.5,"net_price_formatted":"48,50 €","final_price_formatted":"55,17 €","shop_id":54,"no_credit_card":false,"sorting_score":[-3.6,-169,-84,54,10],"payment_method_cost_supported":true,"free_shipping_cost_supported":false,"shipping_cost":"\\u003cem\\u003e+ 3,84 €\\u003c/em\\u003e \\u003cspan\\u003eΜεταφορικά\\u003c/span\\u003e","link":"/products/show/26771682"}]'
which is produce by the following
url2besearched = 'https://www.skroutz.gr/personalization/20783507/product_prices.js?_=1569161647'
Delays = [25,18,24,26,20,22,19,30]
no_of_pagedowns= 20
RandomDelays = np.random.choice(Delays)
#WAIT TIME
time.sleep(RandomDelays)
fp = urllib.request.urlopen(url2besearched)
mybytes = fp.read()
post_elems =[]
mystr = mybytes.decode("utf8")
fp.close()
mystr1 = mystr.rsplit('=')
mystr2 = mystr1[1].split(";")
#I ADD THE FOLLOWING BECAUSE THE INITIAL DOES NOT HAVE ENDING BRACKETS
mystr3 = mystr2[0]+"}"+"]"
for d in mystr3:
for key in d:
post_elems.append([d[key],d['final_price'],d['shop_id']])
When I do the for loop is getting character by character the mystr3 variable and not as a dictionary
How can I have a list with the key of dictionary and final_price with shop_id
My desired output needs to be a list like
post_elems =['26769687','58.9','649']
First the API you are calling for some reason gives a weird response. So .json() on response will not work as there is a field in front. It would be good to understand why or check the URL query strings are correct. Anyway. You have removed them. So I'll copy that code:
import requests, json
mystr = requests.get('https://www.skroutz.gr/personalization/20783507/product_prices.js?_=1569161647').text
mystr1 = mystr.rsplit('=')
mystr2 = mystr1[1].split(";")[0]
json.loads(mystr2)
This works. However. there are two things that are not great here. mystr1 is a systems Hugarian notation, this is very unpythonic. Use type-hinting to help remind what class something belongs to, not the variable name. Also your mystr2 gives a list, a nice example why Hugarian notation is bad.
My json file looks like this:
{"07/01/2015-08/01/2015":
{"ABC": [
["12015618727", "2015-07-29 02:32:01"],
["12024079732", "2015-07-24 13:04:01"],
["12024700142", "2015-07-02 00:00:00"]
]
}
}
I want to extract the numbers 12015618727, 12024079732, 12024700142 from here in python.
I wrote this code:
import json
numbers=set()
input_file=open('filename', 'r')
json_decode=json.load(input_file)
for item in json_decode["07/01/2015-08/01/2015"]["ABC"]:
for j in item:
numbers.add(j[0])
print " ".join(str(x) for x in numbers)
But this doesn't print the numbers.
Python has a json parsing library, see https://docs.python.org/2/library/json.html for details.
Usage:
import json
text = open("file.txt", "r").read()
obj = json.loads(text)
where obj is a python native dict object with nested arrays and dicts.
Edit:
This is the code you want.
import json
numbers=set()
input_file=open('filename.json', 'r')
json_decode=json.load(input_file)
for item in json_decode["07/01/2015-08/01/2015"]["ABC"]:
numbers.add(item[0])
print " ".join(str(x) for x in numbers)
You iterated through each item (the two strings) and added the first letter of each string, hence 1 and 2. Next time, please provide the output you got.
Also, you should attempt to debug your code first. I added a print at the beginning of each loop, and that made the problem pretty clear.
I am converting an XML file to a JSON file. I do this by opening the xml, use the xmltodict module and then use the .get method to traverse the tree to the level I want. This level is the parents to the leaves. I then check on a certain condition that some of the leaves for each of these task is true and if it is then I use json.dumps() and write it to the file. The issue is (I think this is where it is stemming from) that when I only append one JSON object to the file, it doesn't append a comma to the end of the object because it thinks it is the only object. I tried combating this by appending a ',' at the end of each JSON object but then when I try to use the json.loads() method it gives me an error saying "No JSON object could be decoded". However when I manually append the '[' and ']' to the file it doesn't give me an error. My code is below and I'd appreciate any help/suggestions you have.
def getTasks(filename):
f = open(filename, 'r')
a = open('tasksJSON', 'w')
a.write('[')
d = xmltodict.parse(f)
l = d.get('Project').get('Tasks').get('Task')
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
j = json.dumps(task)
a.write (str(j))
a.write(',')
a.write(']')
f.close()
a.close()
This works and puts everything in tasksJSON but like I said, when I call
my_file = open('tasksJSON', 'r')
data = json.load(my_file) # LINE THAT GIVES ME ERROR
I get an error saying
ValueError: No JSON object could be decoded
and the output file contains:
[{"UID": "4", "ID": "14", "Name": "Design"},{"UID": "5", "ID": "15", "Name": "Basic Skeleton"}]
^
this is the comma I manually inserted
make it this way:
def getTasks(filename):
f = open(filename, 'r')
a = open('tasksJSON', 'w')
x = []
d = xmltodict.parse(f)
l = d.get('Project').get('Tasks').get('Task')
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
#j = json.dumps(task)
x.append(task)
#a.write (str(j))
#a.write(',')
a.write(json.dumps(x))
f.close()
a.close()
JSON doesn't allow extra commas at the end of an array or object. But your code adds such an extra comma. If you look at the official grammar here, you can only have a , before another value. And Python's json library conforms to that grammar, so:
>>> json.loads('[1, 2, 3, ]')
ValueError: Expecting value: line 1 column 8 (char 7)
To fix this, you could do something like this:
first = True
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
if first:
first = False
else:
a.write(',')
j = json.dumps(task)
a.write(str(j))
On the other hand, if memory isn't an issue, it might be simpler—and certainly cleaner—to just add all of the objects to a list and then json.dumps that list:
output = []
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
output.append(task)
a.write(json.dumps(output))
Or, more simply:
json.dump([task for task in l if task['Name'] == 'dinner'], a)
(In fact, even if memory is an issue, you can extend JSONEncoder, as shown in the docs, to handle iterators by converting them lazily into JSON arrays, but this is a bit tricky, so I won't show the details unless someone needs them.)
It seems, that you put into a file several json objects and add your own square brackets. Hence, it can not load as single obj