I'm creating a mail "bot" for one of my web services that will periodically collect a queue of e-mail messages to be sent from a PHP script and send them via Google's SMTP servers. The PHP script returns the messages in this format:
test#example.com:Full Name:shortname\ntest2#example.com:Another Full Name:anothershortname\ntest#example.com:Foo:bar
I need to "convert" that into something like this:
{
"test#example.com": [
[
"Full Name",
"shortname"
],
[
"Foo",
"bar"
]
],
"test2#example.com": [
[
"Another Full Name",
"anothershortname"
]
]
}
Notice I need to have only one key per e-mail, even if there are multiple instances of an address. I know I can probably do it with two consecutive loops, one to build the first level of the dictionary and the second to populate it, but there should be a way to do it in one shot. This is my code so far:
raw = "test#example.com:Full Name:shortname\ntest2#example.com:Another Full Name:anothershortname\ntest#example.com:Foo:bar"
print raw
newlines = raw.split("\n")
print newlines
merged = {}
for message in newlines:
message = message.split(":")
merged[message[0]].append([message[1], message[2]])
print merged
I'm getting a KeyError on the last line of the loop, which I take to mean the key has to exist before appending anything to it (appending to a nonexistent key will not create that key).
I'm new to Python and not really familiar with lists and dictionaries yet, so your help is much appreciated!
May work as:
for message in newlines:
message = message.split(":")
temp = []
temp.append(message[1])
temp.append(message[2])
merged[message[0]] = temp
Actually maybe:
for message in newlines:
message = message.split(":")
temp = []
temp.append(message[1])
temp.append(message[2])
if message[0] not in merged:
merged[message[0]] = []
merged[message[0]].append(temp)
I see that you've already accepted an answer, but maybe you're anyhow interested that what you're doing can be easily achieved with defaultdict:
from collections import defaultdict
raw = "test#example.com:Full Name:shortname\ntest2#example.com:Another Full Name:anothershortname\ntest#example.com:Foo:bar"
merged = defaultdict(list)
for line in raw.split('\n'):
line = line.split(':')
merged[line[0]].append(line[1:])
You are right about the error. So you have to check if the key is present. 'key' in dict returns True if 'key' is found in dict, otherwise False. Implementing this, here's your full code (with the debugging print statements removed):
raw = "test#example.com:Full Name:shortname\ntest2#example.com:Another Full Name:anothershortname\ntest#example.com:Foo:bar"
newlines = raw.split("\n")
merged = {}
for message in newlines:
message = message.split(":")
if message[0] in merged:
merged[message[0]].append([message[1], message[2]])
else:
merged[message[0]]=[[message[1], message[2]]]
print merged
Notice the extra brackets for the nested list on the second last line.
Just check for presence of key, if it is not present, create the key,
if it is present, then append the data to existing list.
if(messsage[0] in merged):
merged[message[0]] = [message[1],[message[2]]
else:
merged[message[0]].append([[message[1], message[2]])
Related
I need to get some data from a file .py
Inside the file we have a list like this one
authorized=["somenick", "someid", 45345090, "deadeye", 324234 ]
I want to split every item inside the list authorized like:
Somenick
Someid
45345090
324234
deadeye
But I'm also using all this information for a script which work with telethon...
basically I need to retrieve those info from that list and send it via client_messages(chat, text)
So I will need to define the Text too and the text should be:
text =''' Somenick \nSomeid \n45345090 \n324234 \ndeadeye '''
so my problem at the moment with the actual code are:
async def botadminlist(e):
ciao = open('admins.py', 'r')
for line in ciao:
x= line.split()
for i in x:
y = str(i)
m16 = await helper.control_panel.send_message(config.chat , y)
But it will send 5 messages with every item.. I want only one message with all information like:
text =''' Somenick \nSomeid \n45345090 \n324234 \ndeadeye '''
so I can have a nice output into telegram chat.
With 5 times I mean like for every item in the list will send a new message like
somenick is a message
some id is another message
etc..
I want all information inside the list being into a single variable called text with \n after every item.
DO NOT try to solve using those information, are for example. but in the list I will have int and str as the example. and more the 50 items probably.
You can use ast to turn the list representation into an actual list and join to do your formating.
import ast
file = open('test.txt','r')
lst = ast.literal_eval(file.readline().strip().split('=')[1])
print('\n'.join(list(map(str,last))))
output
somenick
someid
45345090
deadeye
324234
I'm currently working on a program in python 3 that reads in JSON strings and modifies them. I need to search for a value that contains and certain word and then reverse it. Here is the code I have:
msg = '{"company": "Happy, Inc.", "agent": "007"}'
message = json.loads(msg)
for k , v in message.items():
if not k.startswith("_"):
if 'Happy' in v:
message[k] = v[::-1]
print (json.dumps(message))
This reverses the string containing Happy but also reverses the tuples producing this:
{"agent": "007", "company": ".cnI ,yppaH"}
anyone know a way to just reverse the string without reversing the tuples as well?
**I'm teaching myself python and this is my first program using it
If you want to preserve the order of the keys in the JSON, you can parse it into an OrderedDict instead:
>>> message = json.loads(msg, object_pairs_hook=OrderedDict)
>>> message["company"] = message["company"][::-1]
>>> message
OrderedDict([(u'company', u'.cnI ,yppaH'), (u'agent', u'007')])
>>> json.dumps(message, sort_keys=False)
'{"company": ".cnI ,yppaH", "agent": "007"}'
I am converting an XML file to a JSON file. I do this by opening the xml, use the xmltodict module and then use the .get method to traverse the tree to the level I want. This level is the parents to the leaves. I then check on a certain condition that some of the leaves for each of these task is true and if it is then I use json.dumps() and write it to the file. The issue is (I think this is where it is stemming from) that when I only append one JSON object to the file, it doesn't append a comma to the end of the object because it thinks it is the only object. I tried combating this by appending a ',' at the end of each JSON object but then when I try to use the json.loads() method it gives me an error saying "No JSON object could be decoded". However when I manually append the '[' and ']' to the file it doesn't give me an error. My code is below and I'd appreciate any help/suggestions you have.
def getTasks(filename):
f = open(filename, 'r')
a = open('tasksJSON', 'w')
a.write('[')
d = xmltodict.parse(f)
l = d.get('Project').get('Tasks').get('Task')
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
j = json.dumps(task)
a.write (str(j))
a.write(',')
a.write(']')
f.close()
a.close()
This works and puts everything in tasksJSON but like I said, when I call
my_file = open('tasksJSON', 'r')
data = json.load(my_file) # LINE THAT GIVES ME ERROR
I get an error saying
ValueError: No JSON object could be decoded
and the output file contains:
[{"UID": "4", "ID": "14", "Name": "Design"},{"UID": "5", "ID": "15", "Name": "Basic Skeleton"}]
^
this is the comma I manually inserted
make it this way:
def getTasks(filename):
f = open(filename, 'r')
a = open('tasksJSON', 'w')
x = []
d = xmltodict.parse(f)
l = d.get('Project').get('Tasks').get('Task')
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
#j = json.dumps(task)
x.append(task)
#a.write (str(j))
#a.write(',')
a.write(json.dumps(x))
f.close()
a.close()
JSON doesn't allow extra commas at the end of an array or object. But your code adds such an extra comma. If you look at the official grammar here, you can only have a , before another value. And Python's json library conforms to that grammar, so:
>>> json.loads('[1, 2, 3, ]')
ValueError: Expecting value: line 1 column 8 (char 7)
To fix this, you could do something like this:
first = True
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
if first:
first = False
else:
a.write(',')
j = json.dumps(task)
a.write(str(j))
On the other hand, if memory isn't an issue, it might be simpler—and certainly cleaner—to just add all of the objects to a list and then json.dumps that list:
output = []
for task in l:
if (task['Name'] == 'dinner'): #criteria for desirable tasks
output.append(task)
a.write(json.dumps(output))
Or, more simply:
json.dump([task for task in l if task['Name'] == 'dinner'], a)
(In fact, even if memory is an issue, you can extend JSONEncoder, as shown in the docs, to handle iterators by converting them lazily into JSON arrays, but this is a bit tricky, so I won't show the details unless someone needs them.)
It seems, that you put into a file several json objects and add your own square brackets. Hence, it can not load as single obj
I am running a server with cherrypy and python script. Currently, there is a web page containing data of a list of people, which i need to get. The format of the web page is as follow:
www.url1.com, firstName_1, lastName_1
www.url2.com, firstName_2, lastName_2
www.url3.com, firstName_3, lastName_3
I wish to display the list of names on my own webpage, with each name hyperlinked to their corresponding website.
I have read the webpage into a list with the following method:
#cherrypy.expose
def receiveData(self):
""" Get a list, one per line, of currently known online addresses,
separated by commas.
"""
method = "whoonline"
fptr = urllib2.urlopen("%s/%s" % (masterServer, method))
data = fptr.readlines()
fptr.close()
return data
But I don't know how to break the list into a list of lists at where the comma are. The result should give each smaller list three elements; URL, First Name, and Last Name. So I was wondering if anyone could help.
Thank you in advance!
You can iterate over fptr, no need to call readlines()
data = [line.split(', ') for line in fptr]
You need the split(',') method on each string:
data = [ line.split(',') for line in fptr.readlines() ]
lists = []
for line in data:
lists.append([x.strip() for x in line.split(',')])
If you data is a big 'ole string (potentially with leading or trailing spaces), do it this way:
lines=""" www.url1.com, firstName_1, lastName_1
www.url2.com, firstName_2 , lastName_2
www.url3.com, firstName_3, lastName_3 """
data=[]
for line in lines.split('\n'):
t=[e.strip() for e in line.split(',')]
data.append(t)
print data
Out:
[['www.url1.com', 'firstName_1', 'lastName_1'], ['www.url2.com', 'firstName_2',
'lastName_2'], ['www.url3.com', 'firstName_3', 'lastName_3']]
Notice the leading and trailing spaces are removed.
Going to re-word the question.
Basically I'm wondering what is the easiest way to manipulate a string formatted like this:
Safety/Report/Image/489
or
Safety/Report/Image/490
And sectioning off each word seperated by a slash(/), and storing each section(token) into a store so I can call it later. (Reading in about 1200 cells from a CSV file).
The answer for your question:
>>> mystring = "Safety/Report/Image/489"
>>> mystore = mystring.split('/')
>>> mystore
['Safety', 'Report', 'Image', '489']
>>> mystore[2]
'Image'
>>>
If you want to store data from more than one string, then you have several options depending on how do you want to organize it. For example:
liststring = ["Safety/Report/Image/489",
"Safety/Report/Image/490",
"Safety/Report/Image/491"]
dictstore = {}
for line, string in enumerate(liststring):
dictstore[line] = string.split('/')
print dictstore[1][3]
print dictstore[2][3]
prints:
490
491
In this case you can use in the same way a dictionary or a list (a list of lists) for storage. In case each string has a especial identifier (one better than the line number), then the dictionary is the option to choose.
I don't quite understand your code and don't have too much time to study it, but I thought that the following might be helpful, at least if order isn't important ...
in_strings = ['Safety/Report/Image/489',
'Safety/Report/Image/490',
'Other/Misc/Text/500'
]
out_dict = {}
for in_str in in_strings:
level1, level2, level3, level4 = in_str.split('/')
out_dict.setdefault(level1, {}).setdefault(
level2, {}).setdefault(
level3, []).append(level4)
print out_dict
{'Other': {'Misc': {'Text': ['500']}}, 'Safety': {'Report': {'Image': ['489', '490']}}}
If your csv is line seperated:
#do something to load the csv
split_lines = [x.strip() for x in csv_data.split('\n')]
for line_data in split_lines:
split_parts = [x.strip() for x in line_data.split('/')]
# do something with individual part data
# such as some_variable = split_parts[1] etc
# if using indexes, I'd be sure to catch for index errors in case you
# try to go to index 3 of something with only 2 parts
check out the python csv module for some importing help (I'm not too familiar).