I have some Python code that is generated dynamically and stored in a text file. It basically consists of various variables like lists and strings that store data. This information is fed to a class to instantiate different objects. How can I feed the data from the text files into the class?
Here is my class:
class SomethingA(Else):
def construct(self):
// feed_data_a_here
self.call_method()
class SomethingB(Else):
def construct(self):
// feed_data_b_here
self.call_method()
Here is some sample content from the text_a file. As you can see, this is some valid Python code that I need to feed directly into the object. The call the call_method() depends on this data for the output.
self.height = 12
self.id = 463934
self.name = 'object_a'
Is there any way to load this data into the class without manually copying and pasting all of its from the text file one by one?
Thanks.
I would probably write a parser for your files which would delete 'self.' at the beginning and add the variable to the dictionary:
import re
# You could use more apprpriate regex depending on expected var names
regex = 'self\.(?P<var_name>\D+\d*) = (?P<var_value>.*)'
attributes= dict()
with open(path) as file:
for line in file:
search = re.search(regex, line)
var_name = search.group(var_name)
var_value = search.group(var_value).strip() # remove accidentalwhite spaces
attributes[var_name] = var_value
foo = classA(**attributes)
example of the regex in work
Edit
If you use the code I've proposed, all items in the dictionary will be of the string type. Probably you can try:
eval(), as proposed by #Welgriv but with small modification:
eval(f'attributes[{var_name}] = {var_value}')
If your data consists of standard python data and properly formated you can try using json:
import json
x = '12'
y = '[1, 2, 3]'
z = '{"A": 50.0, "B": 60.0}'
attributes = {}
for i, v in enumerate([x, y, z]):
attributes[f'var{i+1}'] = json.loads(v)
print(attributes)
# Prints
# {'var1': 12, 'var2': [1, 2, 3], 'var3': {'A': 50.0, 'B': 60.0}}
You probably look for the eval() function. It evaluate and try to execute a python expression as text. For example:
eval('a = 3')
Will create a variable named a equal to 3. In your case you should open the text file and then evaluate it.
Remarks
eval() function present some security issues because the user can potentially execute any code.
I'm not sure what is the overall context of what you try to implement but you might prefer to store your data (name, id, height...) in another way than python code such as key-values or something because it will make your application extremely dependent of the environment. As an example, if there is a python update and some code are deprecated your application will not work anymore.
Related
I am new to python and I know I can make the following code more concise using iteration (such as a for loop), I am just not sure how to
this is what I have so far
# Open file for reading
dataFileRead = open(filename, "r")
# Read file content into a list - to be completed - Part 1
SampleData = [line.rstrip('\n') for line in open(filename)]
print(SampleData)
variables = [mazeWidth, mazeHeight, aNumOfTreasures, aNumOfBombs, emptyCell, treasure, bomb, exitGate, boundary, boundarySide]
mazeWidth = SampleData[0]
mazeHeight = SampleData[1]
aNumOfTreasures = SampleData[2]
aNumOfBombs = SampleData[3]
emptyCell = SampleData[4]
treasure = SampleData[5]
bomb = SampleData[6]
mario = SampleData[7]
exitGate = SampleData[8]
boundary = SampleData[9]
boundarySide = SampleData[10]
any input helps! thank you
You can use a dictionary to hold the variables' names and values instead of having separate variables:
variable_names = ['mazeWidth', 'mazeHeight', 'aNumOfTreasures', 'aNumOfBombs', 'emptyCell', 'treasure', 'bomb', 'exitGate', 'boundary', 'boundarySide']
variables = {
name: SampleData[i] for i, name in enumerate(variable_names)
}
Later, if you want the value of the variable exitGate for example, you can use:
variables['exitGate']
And for assignment, use:
variables['exitGate'] = "some value"
If you want separate variables however, you can use this:
for i, name in enumerate(variable_names):
globals()[name] = SampleData[i]
And later you can access (get and set) the variables just as you normally would do (print(exitGate); exitGate = "some value").
If you really need these 11 variables to exist with their own names, abstain from any trick. 11 lines is no big deal.
Otherwise, keep working with the list SampleData.
If I were to use your method (which I wouldn't – I'll detail that later), I would edit your code as follows to combine SampleData with variables using zip() and dict().
# Open file for reading
dataFileRead = open(filename, "r")
# Read file content into a list - to be completed - Part 1
SampleData = [line.rstrip('\n') for line in open(filename)]
print(SampleData)
#previously known as variables
variable_names = ["mazeWidth", "mazeHeight", "aNumOfTreasures", "aNumOfBombs", "emptyCell", "treasure", "bomb", "exitGate", "boundary", "boundarySide"]
variables = dict(zip(variable_names, SampleData))
print(variables)
This will combine both lists into one dictionary. This way, if you want to access the number of bombs, or the width of the maze, all you have to do is write:
print(variables["aNumOfBombs"])
Dictionaries are useful like that. However, I would redo the system entirely. As your reading from a file anyway, I think you should make use of the json module and store the data that way. All of the code above would instead look like:
import json
with open(filename, "r") as var_file:
variables = json.load(var_file)
The only difference is how you structure the file you read from, which would instead look something like this:
{
"mazeWidth": 5,
"mazeHeight": 10,
"aNumOfTreasures": 4,
"aNumOfBombs": 16,
"emptyCell": "whatever",
"treasure": true,
"bomb": true,
"exitGate": false,
"boundary": "red",
"boundarySide": "NW"
}
Look into it!
I have large .GEOJSON files that I parse using ijson. One data I load is coordinates listed as for example: "coordinates": [[[47335.8499999996, 6571361.68], [47336.2599999998, 6571360.54], [47336, 6571335.4]]]
I'm able to load this data, having changed its type from Decimal.decimal() to float in the ijson object class. I use the following to parse the JOSN file.
class ReadJSON:
def __init__(self, filename, name=""):
self.name = name
self.f = open(datafolder+filename)
self.objects = ijson.items(self.f, 'features')
def load_file(self):
for obj in self.objects:
final_list = list()
for entry in obj:
temp_list = list()
col_names = list()
for key in entry.keys():
for col in entry[key]:
temp_list.append(entry[key][col])
col_names.append(self.name+'.'+col)
final_list.append(temp_list)
df = pd.DataFrame(final_list, columns=col_names)
return df
Everything ends up where it should, but the list of coordinates is string type. I need to be able to work with the individual points and xy-coordinates. I will have for example: df_rivers, where df_rivers["coordinates"] will contain such lists.
I have tried
temp_list = "[[[47335.8499999996, 6571361.68], [47336.2599999998, 6571360.54], [47336, 6571335.4]]]"
t_list = temp_list.split('],')
print(temp_list[0])
out: [[[47335.8499999996, 6571361.68
type(temp_list[0]) is 'str'
point = temp_list[0].split(',')
print(point[0]):
[[[47335.8499999996
type(point[0]) is 'str
So I am able to access each point and coordinate, however it is quite cumbersome. In addition, point[1] suddenly became out of bounds in the middle of a temp_list. I have many of these lists, which are in reality much longer, and I need to be able to work easily with them.
I don't care whether the fix lies in the loading of the data, or if I can apply it afterwards on the whole column as the script will rarely be run once finished. However, I will have 153 files with up to 60000 rows it will have to run through, so efficiency would be nice.
I'm using Python 3.6.3
You can use ast.literal_eval to obtain the list object from the string: Here is a demo :
>>> temp_list = "[[[47335.8499999996, 6571361.68], [47336.2599999998, 6571360.54], [47336, 6571335.4]]]"
>>> import ast
>>> li = ast.literal_eval(temp_list)
>>> li
[[[47335.8499999996, 6571361.68], [47336.2599999998, 6571360.54], [47336, 6571335.4]]]
>>> type(li)
<class 'list'>
Here is the Python documentation : Doc
http://www.learnpython.org/Serialization_using_JSON_and_pickle
Here are the instructions:
The aim of this exercise is to print out the JSON string with key-value pair "Me" : 800 added to it.
And below is the starting code, which we should modify.
#Exercise fix this function, so it adds the given name and salary pair to the json it returns
def add_employee(jsonSalaries, name, salary):
# Add your code here
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
I'm completely lost. The JSON lesson was brief, at best. The issue I seem to be running in to here is that orginalJsonSalaries is defined as a string (containing all sort of unnecessary symbols like brackets. In fact, I think if the single quotes surrounding its definition were removed, originalJsonSalaries would be a dictionary and this would be a lot easier. But as it stands, how can I append "Me" and 800 to the string and still maintain the dictionary-like formatting?
And yes, I'm very very new to coding. The only other language I know is tcl.
EDIT:
OK, thanks to the answers, I figured out I was being dense and I wrote this code:
import json
#Exercise fix this function, so it adds the given name and salary pair to the json it returns
def add_employee(jsonSalaries, name, salary):
# Add your code here
jsonSalaries = json.loads(jsonSalaries)
jsonSalaries["Me"] = 800
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
This does not work. For whatever reason, the original dictionary keys are formatted as unicode (I don't know where that happened), so when I print out the dictionary, the "u" flag is shown:
{u'Jane': 301, 'Me': 800, u'Alfred': 300}
I have tried using dict.pop() to replace the key ( dict("Jane") = dict.pop(u"Jane") ) but that just brings up SyntaxError: can't assign to function call
Is my original solution incorrect, or is this some annoying formatting issue and how to resolve it?
The page you linked to says exactly how to do this:
In order to use the json module, it must first be imported:
import json
[...]
To load JSON back to a data structure, use the "loads" method. This method takes a string and turns it back into the json object datastructure:
print json.loads(json_string)
They gave you a string (jsonSalaries). Use json.loads to turn it into a dictionary.
Your last question is a new question, but... When you print a dictionary like that you are just using the fact that python is nice enough to show you the contents of its variables in a meaningful way. To print the dictionary in your own format, you would want to iterate through the keys and print the key and value:
for k in newJsonSalaries:
print("Employee {0} makes {1}".format(k, newJsonSalaries[k]))
There are other problems in your code....
It is weird to load the JSON inside the add employee function. That should be separate...
Also, in your add_employee() function you are hardwired always to add the same values of Me and 800 instead of using the name and salary variables that are passed in, so that line should be:
jsonSalaries[name] = salary
Use this:
import json
def add_employee(jsonSalaries, name, salary):
# Add your code here
jsonSalaries = json.loads(jsonSalaries)
jsonSalaries[name] = salary
jsonSalaries = json.dumps(jsonSalaries)
return jsonSalaries
#Test code - shouldn't need to be modified
originalJsonSalaries = '{"Alfred" : 300, "Jane" : 301 }'
newJsonSalaries = add_employee(originalJsonSalaries, "Me", 800)
print(newJsonSalaries)
Add this before return jsonSalaries:
jsonSalaries = json.dumps(jsonSalaries)
I am confused about classes in python. I don't want anyone to write down raw code but suggest methods of doing it. Right now I have the following code...
def main():
lst = []
filename = 'yob' + input('Enter year: ') + '.txt'
for line in open(filename):
line = line.strip()
lst.append(line.split(',')
What this code does is have a input for a file based on a year. The program is placed in a folder with a bunch of text files that have different years to them. Then, I made a class...
class Names():
__slots__ = ('Name', 'Gender', 'Occurences')
This class just defines what objects I should make. The goal of the project is to build objects and create lists based off these objects. My main function returns a list containing several elements that look like the following:
[[jon, M, 190203], ...]
These elements have a name in lst[0], a gender M or F in [1] and a occurence in [3]. I'm trying to find the top 20 Male and Female candidates and print them out.
Goal-
There should be a function which creates a name entry, i.e. mkEntry. It should be
passed the appropriate information, build a new object, populate the fields, and return
it.
If all you want is a handy container class to hold your data in, I suggest using the namedtuple type factory from the collections module, which is designed for exactly this. You should probably also use the csv module to handle reading your file. Python comes with "batteries included", so learn to use the standard library!
from collections import namedtuple
import csv
Person = namedtuple('Person', ('name', 'gender', 'occurences')) # create our type
def main():
filename = 'yob' + input('Enter year: ') + '.txt'
with open(filename, newlines="") as f: # parameters differ a bit in Python 2
reader = csv.reader(f) # the reader handles splitting the lines for you
lst = [Person(*row) for row in reader]
Note: If you're using Python 2, the csv module needs you to open the file in binary mode (with a second argument of 'rb') rather than using the newlines parameter.
If your file had just the single person you used in your example output, you' get a list with one Person object:
>>> print(lst)
[Person(name='jon', gender='M', occurences=190203)]
You can access the various values either by index (like a list or tuple) or by attribute name (like a custom object):
>>> jon = lst[0]
>>> print(jon[0])
jon
>>> print(jon.gender)
M
In your class, add an __init__ method, like this:
def __init__(self, name, gender, occurrences):
self.Name = name
# etc.
Now you don't need a separate "make" method; just call the class itself as a constructor:
myname = Names(lst[0], etc.)
And that's all there is to it.
If you really want an mkEntry function anyway, it'll just be a one-liner: return Names(etc.)
I know you said not to write out the code but it's just easier to explain it this way. You don't need to use slots - they're for a specialised optimisation purpose (and if you don't know what it is, you don't need it).
class Person(object):
def __init__(self, name, gender, occurrences):
self.name = name
self.gender = gender
self.occurrences = occurrences
def main():
# read in the csv to create a list of Person objects
people = []
filename = 'yob' + input('Enter year: ') + '.txt'
for line in open(filename):
line = line.strip()
fields = line.split(',')
p = Person(fields[0], fields[1], int(fields[2]))
people.append(p)
# split into genders
p_m = [p for p in people if p.gender == 'M']
p_f = [p for p in people if p.gender == 'F']
# sort each by occurrences descending
p_m = sorted(p_m, key=lambda x: -x.occurrences)
p_f = sorted(p_f, key=lambda x: -x.occurrences)
# print out the first 20 of each
for p in p_m[:20]:
print p.name, p.gender, p.occurrences
for p in p_f[:20]:
print p.name, p.gender, p.occurrences
if __name__ == '__main__':
main()
I've used a couple of features here that might look a little scary, but they're easy enough once you get used to them (and you'll see them all over python code). List comprehensions give us an easy way of filtering our list of people into genders. lambda gives you an anonymous function. The [:20] syntax says, give me the first 20 elements of this list - refer to list slicing.
Your case is quite simple and you probably don't even really need the class / objects but it should give you an idea of how you use them. There's also a csv reading library in python that will help you out if the csvs are more complex (quoted fields etc).
I have a function which generates a large amount of output, like such
A: {B:1,C:0,D:3}
B: {A:1,C:0,D:3}
C: {B:1,A:0,D:3}
D: {B:1,C:0,A:3}
min = 0, x = A, y = C, new = AC
AC: {B:1,D:3}
B: {AC:1,D:3}
D: {B:1,AC:0}
min = 0, x = B, y = AC, new = ACB
Essentially, Im using a function (for arguments sake we'll call it 'fun1') which generates the type of output like shown above.
Im looking at doing something like data1 = fun1(input), and then using this variable as input to another function, along with some other input too.
So how do I utilize this information? I'm used to dealing with lists, dictionaries and what not, as output from a function. Im trying to access the information so that I can find out certain things about the relationship of a certain letter to others. Also it the amount of output above is just 2 lists of letters, but thats totally arbitrary.
Help please :)
Its not clear to me why you have a function that prints out tons of data as opposed to returning it as a useable python object. If you have control of this function (its yours and you wrote it), then it should be doing something like this (keep in mind that this is regardless of efficiency or whatever. Just trying to make it simple for you)...
def fun1():
allresults = []
# do you loop and generate objects like
results = {
"A": {"B":1,"C":0,"D":3},
"B": {"A":1,"C":0,"D":3},
"C": {"B":1,"A":0,"D":3},
"D": {"B":1,"C":0,"A":3},
"min": 0,
"x": "A",
"y": "C",
"new": "AC"
}
# and add them to your list
allresults.append(results)
return allresults
Then when you ran your function, you would have a return value:
data = fun1()
If you do not have control over the source of this function (its part of some other strangely written library), then the only clean way would be to run it in another process and read the stdout out. Then parse that into something useable. Otherwise you are talking about redirecting the stdout pipe temporarily while running this function, but I am not even gunna go there.
Update
In your comments you said that this is part of some other library. Your best bet is to make changes to this code and get it to properly return its data so that its reusable. Your biggest issue here is that what is being printed isnt even in a common format. At the very least it could have dumped out JSON or something like that to be parsed by another process.