I am trying to load a string (the actual program read this line from a file and it is a very large file that I can not manually modify) formatted as a dictionary.
I need to convert the string line into a json object so I can check value of specific key, e.g. myJson[Date] .
This is the script:
import json
mystring = "{'Date': 'Fri, 19 Apr 2019 03:58:04 GMT', 'Server': 'Apache/2.4.39', 'Accept-Ranges': 'bytes'}"
mystring = json.dumps(mystring)
myJson = json.loads(mystring)
print(str(myJson.keys()))
print(str(myJson))
I am getting this error:
AttributeError: 'str' object has no attribute 'keys'
I suspect that the mystring format is not conforming and that the single quotes should be double quotes? Given that I have a large data, and I can not simply replace single colons with double one using simple search/replace as single colons may be included in the values which I should not modify. If this is the cause of the problem, is there any way to replace the colons of the key/value pair only without touching the colons in the values? I am hoping that this is not the problem.
Rather than dealing with the single quoted string and struggling to convert it into json, just use ast package to convert it into a valid dict.
import ast
mystring = "{'Date': 'Fri, 19 Apr 2019 03:58:04 GMT', 'Server': 'Apache/2.4.39', 'Accept-Ranges': 'bytes'}"
my_dict = ast.literal_eval(mystring)
the result is:
> print(my_dict["Date"])
Fri, 19 Apr 2019 03:58:04 GMT
This code stores the string as a dictionary in a variable called "Tempvar"
From that variable you can just use the keys like a regular dictionary.
import json
mystring = "{'Date': 'Fri, 19 Apr 2019 03:58:04 GMT', 'Server': 'Apache/2.4.39', 'Accept-Ranges': 'bytes'}"
exec("tempvar = " + mystring)
mystring = json.dumps(mystring)
myJson = json.loads(mystring)
print(str(tempvar['Date']))
print(str(myJson))
Hope this helps
Yes json decoder likes double quotes for keys and values, and I think you can use python to do the replacement, try if applies:
mystring = "{'Date': 'Fri, 19 Apr 2019 03:58:04 GMT', 'Server': 'Apache/2.4.39', 'Accept-Ranges': 'bytes'}"
json_string = mystring.replace("'", "\"")
d = json.loads(json_string)
dstring = json.dumps(d)
myJson = json.loads(dstring)
print(str(myJson.keys()))
Related
I am trying to convert the date formats and make them uniform throughout the document using Python 3.6.
Here is the sample of the dates in my document:(There can be other formats as the document is large.)
9/21/1989
19640430
6/27/1980
5/11/1987
Mar 12 1951
2 aug 2015
I have checked the datetime lbrary. But could not understand hoow to detect and change the format of the dates automatically. Here is what I have checked till now:
>>> from datetime import datetime
>>> oldformat = '20140716'
>>> datetimeobject = datetime.strptime(oldformat,'%Y%m%d')
>>> newformat = datetimeobject.strftime('%m-%d-%Y')
>>> print (newformat)
07-16-2014
But I am not getting how I can make the program detect the date patterns automatically and convert them to one single uniform pattern of dates as mm/dd/yyyy
Kindly, suggest what I need to do, so as to achieve my goal using Python 3.6.
There is no universal Python way of doing this, but I'd recommend using regex to identify the type and then converting it correctly:
Example Python
import re
from datetime import datetime
with open("in.txt","r") as fi, open("out.txt","w") as fo:
for line in fi:
line = line.strip()
dateObj = None
if re.match(r"^\d{8}$", line):
dateObj = datetime.strptime(line,'%Y%m%d')
elif re.match(r"^\d{1,2}/", line):
dateObj = datetime.strptime(line,'%m/%d/%Y')
elif re.match(r"^[a-z]{3}", line, re.IGNORECASE):
dateObj = datetime.strptime(line,'%b %d %Y')
elif re.match(r"^\d{1,2} [a-z]{3}", line, re.IGNORECASE):
dateObj = datetime.strptime(line,'%d %b %Y')
fo.write(dateObj.strftime('%m-%d-%Y') + "\n")
Example Input
9/21/1989
19640430
6/27/1980
5/11/1987
Mar 12 1951
2 aug 2015
Example Output
09-21-1989
04-30-1964
06-27-1980
05-11-1987
03-12-1951
08-02-2015
I have tried using the dateutil library in my code to detect the date strings in any format. and then used the datetime library to convert it into the appropriate format.
Here is the code:
>>> import dateutil.parser
>>> yourdate = dateutil.parser.parse("May 24 2016")
>>>
>>> print(yourdate)
2016-05-24 00:00:00
>>> from datetime import datetime
>>> oldformat = yourdate
>>> datetimeobject = datetime.strptime(str(oldformat),'%Y-%m-%d %H:%M:%S')
>>> newformat = datetimeobject.strftime('%m-%d-%Y')
>>> print (newformat)
05-24-2016
This works.
See the image of the output:
(There can be other formats as the document is large.)
Unfortunately, Python does not provide "guess what I mean" functionality (although you might be able to repurpose GNU date for that, as it is quite flexible). You will have to make a list of all of the formats you want to support, and then try each in turn (using datetime.strptime() as you've shown) until one of them works.
Python does not try to guess because, in an international context, it is not generally possible to divine what the user wants. In the US, 2/3/1994 means "February 3rd, 1994," but in Europe the same string means "The 2nd of March, 1994." Python deliberately abstains from this confusion.
The output of the openssl command looks like this:
serial=CABCSDUMMY4A168847703FGH
notAfter=Oct 21 16:43:47 2024 GMT
subject= /C=US/ST=WA/L=Seattle/O=MyCo/OU=TME/CN=MyCo.example.com
How do I convert this string to JSON?
I tried these:
temp_txt_bytes = subprocess.check_output (["openssl", "x509", "-serial", "-enddate", "-subject", "-noout", "-in", pem_file_name])
temp_txt_strings = temp_txt_bytes.decode("utf-8")
test = json.loads(temp_txt_strings) #json.parse, json.dump, and json.load also failing
You can split every line with "=" as a separator, put the two parts in an ordered dictionary and then dump it to json:
my_list = "serial=CABCSDUMMY4A168847703FGH".split("=")
ordered_dict = OrderedDict()
ordered_dict[my_list[0]] = my_list[1]
print(json.dumps(ordered_dict))
the output would be like this:
{"serial": "CABCSDUMMY4A168847703FGH"}
you can do it for all lines.
PS don't forget to import json and OrderedDict
On a switch, i run ntpq -nc rv and get an output:
associd=0 status=0715 leap_none, sync_ntp, 1 event, clock_sync,
version="ntpd 4.2.6p3-RC10#1.2239-o Mon Mar 21 02:53:48 UTC 2016 (1)",
processor="x86_64", system="Linux/3.4.43.Ar-3052562.4155M", leap=00,
stratum=2, precision=-21, rootdelay=23.062, rootdisp=46.473,
refid=17.253.24.125,
reftime=dbf98d39.76cf93ad Mon, Dec 12 2016 20:55:21.464,
clock=dbf9943.026ea63c Mon, Dec 12 2016 21:28:03.009, peer=43497,
tc=10, mintc=3, offset=-0.114, frequency=27.326, sys_jitter=0.151,
clk_jitter=0.162, clk_wander=0.028
I am attempting to create a bash shell command using Python's subprocess module to extract only the value for "offset", or -0.114 in the example above
I noticed that I can use the subprocess replacement mod or sh for this such that:
import sh
print(sh.grep(sh.ntpq("-nc rv"), 'offset'))
and I get:
mintc=3, offset=-0.114, frequency=27.326, sys_jitter=0.151,
which is incorrect as I just want the value for 'offset', -0.114.
Not sure what I am doing wrong here, whether its my grep function or I am not using the sh module correctly.
grep reads line by line; it returns every line matching any part of the input. But I think grep is overkill. Once you get shell output, just search for the thing after output:
items = sh.ntpq("-nc rv").split(',')
for pair in items:
name, value = pair.split('=')
# strip because we weren't careful with whitespace
if name.strip() == 'offset':
print(value.strip())
I am currently trying to make a text file with numbers into a list
the text file is
1.89
1.99
2.14
2.51
5.03
3.81
1.97
2.31
2.91
3.97
2.68
2.44
Right now I only know how to read the file. How can i make this into a list?
afterwards how can I assign the list to another list?
for example
jan = 1.89
feb = 1.99
etc
Code from comments:
inFile = open('program9.txt', 'r')
lineRead = inFile.readline()
while lineRead != '':
words = lineRead.split()
annualRainfall = float(words[0])
print(format(annualRainfall, '.2f'))
lineRead = inFile.readline()
inFile.close()
months = ('jan', 'feb', ...)
with open('filename', 'rb') as f:
my_list = [float(x) for x in f]
res = dict(zip(months, my_list))
This will however work ONLY if there are the same number of lines than months!
A file is already an iterable of lines, so you don't have to do anything to make it into an iterable of lines.
If you want to make it specifically into a list of lines, you can do the same thing as with any other iterable: call list on it:
with open(filename) as f:
lines = list(f)
But if you want to convert this into a list of floats, it doesn't matter what kind of iterable you start with, so you might as well just use the file as-is:
with open(filename) as f:
floats = [float(line) for line in f]
(Note that float ignores trailing whitespace, so it doesn't matter whether you use a method that strips off the newlines or leaves them in place.)
From a comment:
now i just need to find out how to assign the list to another list like jan = 1.89, feb = 1.99 and so on
If you know you have exactly 12 values (and it will be an error if you don't), you can write whichever of these looks nicer to you:
jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec = (float(line) for line in f)
jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec = map(float, f)
However, it's often a bad idea to have 12 separate variables like this. Without knowing how you're planning to use them, it's hard to say for sure (see Keep data out of your variable names for some relevant background on making the decision yourself), but it might be better to have, say, a single dictionary, using the month names as keys:
floats = dict(zip('jan feb mar apr may jun jul aug sep oct nov dec'.split(),
map(float, f))
Or to just leave the values in a list, and use the month names as just symbolic names for indices into that list:
>>> jan, feb, mar, apr, may, jun, jul, aug, sep, oct, nov, dec = range(12)
>>> print(floats[mar])
2.14
That might be even nicer with an IntEnum, or maybe a namedtuple. Again, it really depends on how you plan to use the data after importing them.
I'm trying to turn this object into something I can work with in my program. I need to store the message-id to a database. Short of flat-out parsing it, can I just turn this whole thing into a dictionary somehow?
Here is where I'm at with some IDLE testing:
>>> response = urllib.request.urlopen(req)
>>> response.getheaders()
[('Server', 'nginx/1.6.0'), ('Date', 'Sat, 07 Jun 2014 00:32:45 GMT'), ('Content-Type', 'application/json;charset=ISO-8859-1'), ('Transfer-Encoding', 'chunked'), ('Connection', 'close'), ('Cache-Control', 'max-age=1')]
>>> response.read()
b'{"message-count":"1","messages":[{"to":"11234567890","message-id":"02000000300D8F21","status":"0","remaining-balance":"1.82720000","message-price":"0.00620000","network":"302220"}]}'
After a half hour of sifting through google I was able to turn it into a string with:
>>> response.response.decode('utf-8')
>>> print(response)
'{"message-count":"1","messages":[{"to":"11234567890","message-id":"02000000300D8F21","status":"0","remaining-balance":"1.82720000","message-price":"0.00620000","network":"302220"}]}'
>>> type(response)
<class 'str'>
I found this post but here is what I get:
>>> a_dict = dict([response.strip('{}').split(":"),])
Traceback (most recent call last):
File "<pyshell#79>", line 1, in <module>
a_dict = dict([response.strip('{}').split(":"),])
ValueError: dictionary update sequence element #0 has length 9; 2 is required
Maybe I'm going about this all wrong. What's the quickest way to turn this object into a dictionary or something else I can easily work with?
Load it via json.load():
import json
response = urllib.request.urlopen(req)
result = json.loads(response.readall().decode('utf-8'))
message_id = result['messages'][0]['message-id']