I'm learning to code and I got this task to create a program. (It is from Starting out with Python, 4th edition, Chapter 8, Exercise 14.)
I have created all functions and called them all yet the program will only execute one and return none for the rest. What am I missing?
def main():
gas_file=open('GasPrices.txt', 'r')
average_per_year(gas_file)
print()
average_per_month(gas_file)
print()
highest_lowers(gas_file)
print()
lowest_to_highest(gas_file)
print()
highest_to_lowest(gas_file)
gas_file.close()
Here is full code: https://pastecode.xyz/view/b007ccb0
I get no error warning or anything btw
Kind regards, Juls
The first function you call reads to the end of gas_file until it's at the end. Since you pass the same file handle to all the other functions, they just see an empty file.
Your code would work (and be a lot simpler and faster than what you're trying to do) if instead of copying+pasting the code for reading the file into all of these functions, so that each is trying to re-read the file and do the same work repeatedly, you had a single function that parses the file into some useful data structure (like a list or a dict of values), and then had all the other functions just read the data from that.
Something like:
from typing import List, NamedTuple
class GasPrice(NamedTuple):
month: int
year: int
price: float
def read_prices(gas_file: str) -> List[GasPrice]:
"""Load a list of GasPrices from a file."""
with open(gas_file, 'r') as f:
return [
GasPrice(
int(line[:2]), # month
int(line[6:10]), # year
float(line[11:]) # price
)
for line in f
]
def main() -> None:
prices = read_prices('GasPrices.txt')
# ... do stuff with prices
Related
I have a small python program to edit a row only in a csv file, I do this by getting all the rows and columns of the file and replacing the selected row with what the user entered.
The point is that I have a function that is activated depends on which row has been selected but it cannot be activated, I have already tried many ways, I have already called it clearly and this function is not executing yet.
I already have the code but I just need to activate the function.
This is a small example of my code:
import pandas as pd
my_variable = "ninguna"
new_variable1 = "hello"
new_variable2 = "How are you"
new_variable3 = "brother"
if my_variable == "none":
with open("priv/productos.csv", 'r') as edit:
edit = edit.read()
def edit_row(self):
def tryloc(df, col, idx, default=None):
try:
return df.iloc[col, idx]
except IndexError:
return default
print("the variable if it is running")
edit = pd.read_csv("priv/products.csv")
product_2 = tryloc(edit, 1, 0)
brand_2 = tryloc(edit, 1, 1)
price_2 = tryloc(edit, 1, 2)
with open("priv/products.csv", 'w') as login_main:
login_main.write("PRODUCTS" +",")
login_main.write("BRANDS" +",")
login_main.write("PRICES" +"\n")
login_main.write(str(new_variable1) +",")
login_main.write(str(new_variable2) +",")
login_main.write(str(new_variable3) +"\n")
login_main.write(str(product_2) +",")
login_main.write(str(brand_2) +",")
login_main.write(str(price_2) +"\n")
edit_row(self)
When I run the program everything works except the function :(
Does anyone know what this problem is?
Thank you
Your function have to be written before your condition. Inside the condition should be only edit_row(data) and this will be a call of your function. Also self is something that you can use in functions when you define Classes (I don't see classes in your code). Also I don't understand why did you include one function inside another one. What it will return in the case of error. What is default?
Call of function have to be outside of function.
I'd recommend laying out your code more like this:
import pandas as pd
def edit_row(filename):
with open(filename, 'r') as edit:
data = edit.read()
# lines to edit data here
# lines to save edited data here
edit_row("priv/productos.csv")
What you have right now is that my_variable contains the string 'ninguna' and the lines to define and call edit_row() only execute if my_variable contains the string 'none'. So when you run your code, the definition and call to edit_row() are all skipped.
Your code also assumes that edit_row() is a method on an object named edit whose class is never defined. For that to work, you'd need to define a class with edit_row() as a method. For now, just go procedural.
Admittedly I am not sure how to ask this, as I know how to handle this in R (code execution in a new environment), but equivalent searches for the python solution are not yielding what I was hoping.
In short, I will receive a spreadsheet (or csv) where the contents of the column will contain, hopefully, valid python code. This could be the equivalent of a script, but just contained in the csv/workbook. For a use case, think teaching programming and the output is an LMS.
What I am hoping to do is loop over the file, and for each cell, run the code, and with the results in memory, test to see if certain things exist.
For example: https://docs.google.com/spreadsheets/d/1D-zC10rUTuozfTR5yHfauIGbSNe-PmfrZCkC7UTPH1c/edit?usp=sharing
When evaluating the first response in the spreadsheet above, I would want to test that x, y, and z are all properly defined and have the expected values.
Because there would be multiple rows in the file, one per student, how can I run each row separately, evaluate the results, and ensure that I isolate the results to only that cell. Simply, when moving on, I do not retain any of the past evaluations.
(I am unaware of tools to do code checking, so I am dealing with it in a very manual way.)
It is possible to use Python's exec() function to execute strings such as the content in the cells.
Ex:
variables = {}
exec("""import os
# a comment
x = 2
y = 6
z = x * y""", variables)
assert variables["z"] == 12
Dealing with the csv file:
import csv
csv_file = open("path_to_csv_file", "rt")
csv_reader = csv.reader(csv_file)
iterator = iter(csv_reader)
next(iterator) # To skip the titles of the columns
for row in iterator:
user = row[0]
answer = row[1]
### Any other code involving the csv file must be put here to work properly,
### that is, before closing csv_file.
csv_file.close() # Remember to close the file.
It won't be able to detect whether some module was imported (Because when importing from an exec() function, the module will remain in cache for the next exec's). One way to test this would be to 'unimport' the module and test the exec for Exceptions.
Ex:
# This piece of code would be before closing the file,
# INSIDE THE FOR LOOP AND WITH IT IDENTED (Because you want
# it to run for each student.).
try:
del os # 'unimporting' os (This doesn't 'unimport' as much as deletes a
# reference to the module, what could be problematic if a 'from
# module import object' statement was used.)
except NameError: # So that trying to delete a module that wasn't imported
# does not lead to Exceptions being raised.
pass
namespace = dict()
try:
exec(answer, namespace)
except:
# Answer code could not be run without raising exceptions, i.e., the code
# is poorly written.
# Code you want to run when the answer is wrong.
else:
# The code hasn't raised Exceptions, time to test the variables.
x, y, z = namespace['x'], namespace['y'], namespace['z']
if (x == 2) and (y == 6) and (z == x * y):
# Code you want to run when the answer is right.
else:
# Code you want to run when the answer is wrong.
I sense that this is not the best way to do this, but it is certainly an attempt.
I hope this helped.
EDIT: Removed some bad code and added part of Tadhg McDonald-Jensen's comment.
I have a problem to make my code more self-healable. Eg: I execute a method 1 to load the data from a CSV into the Vertica database. I have another method 2 to check if the number of rows in the database and the number of lines in CSV file is same. If the number of lines doesn't match, then I was thinking of calling the method 2 from the point where it called the query to load data from CSV into the database.
I was thinking of a checkpointing strategy for this problem. like, maintain some points in the code where the errors usually occur and recalling them at other points.
I already tried using pickle module in python, but came to know that pickle can only save objects, classes, variables etc. can't save the point from where I can actually execute a method.
i have provided some demo code:
import pickle
class Fruits:
def apple(self):
filehandler= open ("Fruits.obj","wb")
print "apple"
pickle.dump(self,filehandler)
print "mapple"
filehandler.close()
def mango(self):
filehandler = open("Fruits.obj","rb")
print "mango"
obj=pickle.load(filehandler)
obj.apple()
general = Fruits()
general.apple()
general.mango()
the output of above program is:
apple
mapple
mango
apple
mapple
I want my code to execute such that when mango method calls apple method, it must execute from the point of only print "mapple". it must not execute the whole method.
please do provide me some insight on how to solve this problem.
thanks in advance
Note:
Your code doesn't work at all. filehandler in def mango(... IS NOT the same as filehandler in def apple(.... Therfore, the file opend in def mango(... are never closed.
Add a if condidtion to def apple, you don't need pickle at all.
def apple(self, mango=False):
if not a´mango:
filehandler= open ("Fruits.obj","wb")
...
print "mapple"
...
def mango(self):
filehandler = open("Fruits.obj","rb")
...
obj.apple(True)
Just trying to learn and I"m wondering if multiprocessing would speed
up this for loop ,.. trying to compare
alexa_white_list(1,000,000 lines) and
dnsMISP (can get up to 160,000 lines)
Code checks each line in dnsMISP and looks for it in alexa_white_list.
if it doesn't see it, it adds it to blacklist.
Without mp_handler function the code works fine but it takes
around 40-45 minutes. For brevity, I've omitted all the other imports and
the function that pulls down and unzips the alexa white list.
The below gives me the following error -
File "./vetdns.py", line 128, in mp_handler
p.map(dns_check,dnsMISP,alexa_white_list)
NameError: global name 'dnsMISP' is not defined
from multiprocessing import Pool
def dns_check():
awl = []
blacklist = []
ctr = 0
dnsMISP = open(INPUT_FILE,"r")
dns_misp_lines = dnsMISP.readlines()
dnsMISP.close()
alexa_white_list = open(outname, 'r')
alexa_white_list_lines = alexa_white_list.readlines()
alexa_white_list.close()
print "converting awl to proper format"
for line in alexa_white_list_lines:
awl.append(".".join(line.split(".")[-2:]).strip())
print "done"
for host in dns_misp_lines:
host = host.strip()
host = ".".join(host.split(".")[-2:])
if not host in awl:
blacklist.append(host)
file_out = open(FULL_FILENAME,"w")
file_out.write("\n".join(blacklist))
file_out.close()
def mp_handler():
p = Pool(2)
p.map(dns_check,dnsMISP,alexa_white_list)
if __name__ =='__main__':
mp_handler()
If I label it as global etc I still get the error. I'd appreciate any
suggestions!!
There's no need for multiprocessing here. In fact this code can be greatly simplified:
def get_host_form_line(line):
return line.strip().split(".", 1)[-1]
def dns_check():
with open('alexa.txt') as alexa:
awl = {get_host_from_line(line) for line in alexa}
blacklist = []
with open(INPUT_FILE, "r") as dns_misp_lines:
for line in dns_misp_lines:
host = get_host_from_line(line)
if host not in awl:
blacklist.append(host)
with open(FULL_FILENAME,"w") as file_out:
file_out.write("\n".join(blacklist))
Using a set comprehension to create your Alexa collection has the advantage of being O(1) lookup time. Sets are similar to dictionaries. They are pretty much dictionaries that only have keys with no values. There is some additional overhead in memory and the initial creation time will likely be slower since the values you put in to a set need to be hashed and hash collisions dealt with but the increase in performance you gain from the faster look ups should make up for it.
You can also clean up your line parsing. split() takes an additional parameter that will limit the number of times the input is split. I'm assuming your lines look something like this:
http://www.something.com and you want something.com (if this isn't the case let me know)
It's important to remember that the in operator isn't magic. When you use it to check membership (is an element in the list) what it's essentially doing under the hood is this:
for element in list:
if element == input:
return True
return False
So every time in your code you did if element in list your program had to iterate across each element until it either found what you were looking for or got to the end. This was probably the biggest bottleneck of your code.
You tried to read a variable named dnsMISP to pass as an argument to Pool.map. It doesn't exist in local or global scope (where do you think it's coming from?), so you got a NameError. This has nothing to do with multiprocessing; you could just type a line with nothing but:
dnsMISP
and have the same error.
I want to write a tool in Python to prepare a simulation study by creating for each simulation run a folder and a configuration file with some run-specific parameters.
study/
study.conf
run1
run.conf
run2
run.conf
The tool should read the overall study configuration from a file including (1) static parameters (key-value pairs), (2) lists for iteration parameters, and (3) some small code snippets to calculate further parameters from the previous ones. The latter are run specific depending on the permutation of the iteration parameters used.
Before writing the run.conf files from a template, I need to run some code like this to determine the specific key-value pairs from the code snippets for that run
code = compile(code_str, 'foo.py', 'exec')
rv=eval(code, context, { })
However, as this is confirmed by the Python documentation, this just leads to a None as return value.
The code string and context dictionary in the example are filled elsewhere. For this discussion, this snippet should do it:
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
I have done this before in Perl and Java+JavaScript. There, you just give the code snippet to some evaluation function or script engine and get in return a value (object) from the last executed statement -- not a big issue.
Now, in Python I struggle with the fact that eval() is too narrow just allowing one statement and exec() doesn't return values in general. I need to import modules and sometimes do some slightly more complex calculations, e.g., 5 lines of code.
Isn't there a better solution that I don't see at the moment?
During my research, I found some very good discussions about Pyhton eval() and exec() and also some tricky solutions to circumvent the issue by going via the stdout and parsing the return value from there. The latter would do it, but is not very nice and already 5 years old.
The exec function will modify the global parameter (dict) passed to it. So you can use the code below
code_str="""import math
Result1 = math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
exec(code_str, context)
print (context['Result1']) # 31.6
Every variable code_str created will end up with a key:value pair in the context dictionary. So the dict is the "object" like you mentioned in JavaScript.
Edit1:
If you only need the result of the last line in code_str and try to prevent something like Result1=..., try the below code
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = { 'width' : 30, 'height' : 10 }
lines = [l for l in code_str.split('\n') if l.strip()]
lines[-1] = '__myresult__='+lines[-1]
exec('\n'.join(lines), context)
print (context['__myresult__'])
This approach is not as robust as the former one, but should work for most case. If you need to manipulate the code in a sophisticated way, please take a look at the Abstract Syntax Trees
Since this whole exec() / eval() thing in Python is a bit weird ... I have chose to re-design the whole thing based on a proposal in the comments to my question (thanks #jonrsharpe).
Now, the whole study specification is a .py module that the user can edit. From there, the configuration setup is directly written to a central object of the whole package. On tool runs, the configuration module is imported using the code below
import imp
# import the configuration as a module
(path, name) = os.path.split(filename)
(name, _) = os.path.splitext(name)
(file, filename, data) = imp.find_module(name, [path])
try:
module = imp.load_module(name, file, filename, data)
except ImportError as e:
print(e)
sys.exit(1)
finally:
file.close()
I came across similar needs, and finally figured out a approach by playing with ast:
import ast
code = """
def tf(n):
return n*n
r=tf(3)
{"vvv": tf(5)}
"""
ast_ = ast.parse(code, '<code>', 'exec')
final_expr = None
for field_ in ast.iter_fields(ast_):
if 'body' != field_[0]: continue
if len(field_[1]) > 0 and isinstance(field_[1][-1], ast.Expr):
final_expr = ast.Expression()
final_expr.body = field_[1].pop().value
ld = {}
rv = None
exec(compile(ast_, '<code>', 'exec'), None, ld)
if final_expr:
rv = eval(compile(final_expr, '<code>', 'eval'), None, ld)
print('got locals: {}'.format(ld))
print('got return: {}'.format(rv))
It'll eval instead of exec the last clause if it's an expression, or have all execed and return None.
Output:
got locals: {'tf': <function tf at 0x10103a268>, 'r': 9}
got return: {'vvv': 25}