I have a small python program to edit a row only in a csv file, I do this by getting all the rows and columns of the file and replacing the selected row with what the user entered.
The point is that I have a function that is activated depends on which row has been selected but it cannot be activated, I have already tried many ways, I have already called it clearly and this function is not executing yet.
I already have the code but I just need to activate the function.
This is a small example of my code:
import pandas as pd
my_variable = "ninguna"
new_variable1 = "hello"
new_variable2 = "How are you"
new_variable3 = "brother"
if my_variable == "none":
with open("priv/productos.csv", 'r') as edit:
edit = edit.read()
def edit_row(self):
def tryloc(df, col, idx, default=None):
try:
return df.iloc[col, idx]
except IndexError:
return default
print("the variable if it is running")
edit = pd.read_csv("priv/products.csv")
product_2 = tryloc(edit, 1, 0)
brand_2 = tryloc(edit, 1, 1)
price_2 = tryloc(edit, 1, 2)
with open("priv/products.csv", 'w') as login_main:
login_main.write("PRODUCTS" +",")
login_main.write("BRANDS" +",")
login_main.write("PRICES" +"\n")
login_main.write(str(new_variable1) +",")
login_main.write(str(new_variable2) +",")
login_main.write(str(new_variable3) +"\n")
login_main.write(str(product_2) +",")
login_main.write(str(brand_2) +",")
login_main.write(str(price_2) +"\n")
edit_row(self)
When I run the program everything works except the function :(
Does anyone know what this problem is?
Thank you
Your function have to be written before your condition. Inside the condition should be only edit_row(data) and this will be a call of your function. Also self is something that you can use in functions when you define Classes (I don't see classes in your code). Also I don't understand why did you include one function inside another one. What it will return in the case of error. What is default?
Call of function have to be outside of function.
I'd recommend laying out your code more like this:
import pandas as pd
def edit_row(filename):
with open(filename, 'r') as edit:
data = edit.read()
# lines to edit data here
# lines to save edited data here
edit_row("priv/productos.csv")
What you have right now is that my_variable contains the string 'ninguna' and the lines to define and call edit_row() only execute if my_variable contains the string 'none'. So when you run your code, the definition and call to edit_row() are all skipped.
Your code also assumes that edit_row() is a method on an object named edit whose class is never defined. For that to work, you'd need to define a class with edit_row() as a method. For now, just go procedural.
Related
Admittedly I am not sure how to ask this, as I know how to handle this in R (code execution in a new environment), but equivalent searches for the python solution are not yielding what I was hoping.
In short, I will receive a spreadsheet (or csv) where the contents of the column will contain, hopefully, valid python code. This could be the equivalent of a script, but just contained in the csv/workbook. For a use case, think teaching programming and the output is an LMS.
What I am hoping to do is loop over the file, and for each cell, run the code, and with the results in memory, test to see if certain things exist.
For example: https://docs.google.com/spreadsheets/d/1D-zC10rUTuozfTR5yHfauIGbSNe-PmfrZCkC7UTPH1c/edit?usp=sharing
When evaluating the first response in the spreadsheet above, I would want to test that x, y, and z are all properly defined and have the expected values.
Because there would be multiple rows in the file, one per student, how can I run each row separately, evaluate the results, and ensure that I isolate the results to only that cell. Simply, when moving on, I do not retain any of the past evaluations.
(I am unaware of tools to do code checking, so I am dealing with it in a very manual way.)
It is possible to use Python's exec() function to execute strings such as the content in the cells.
Ex:
variables = {}
exec("""import os
# a comment
x = 2
y = 6
z = x * y""", variables)
assert variables["z"] == 12
Dealing with the csv file:
import csv
csv_file = open("path_to_csv_file", "rt")
csv_reader = csv.reader(csv_file)
iterator = iter(csv_reader)
next(iterator) # To skip the titles of the columns
for row in iterator:
user = row[0]
answer = row[1]
### Any other code involving the csv file must be put here to work properly,
### that is, before closing csv_file.
csv_file.close() # Remember to close the file.
It won't be able to detect whether some module was imported (Because when importing from an exec() function, the module will remain in cache for the next exec's). One way to test this would be to 'unimport' the module and test the exec for Exceptions.
Ex:
# This piece of code would be before closing the file,
# INSIDE THE FOR LOOP AND WITH IT IDENTED (Because you want
# it to run for each student.).
try:
del os # 'unimporting' os (This doesn't 'unimport' as much as deletes a
# reference to the module, what could be problematic if a 'from
# module import object' statement was used.)
except NameError: # So that trying to delete a module that wasn't imported
# does not lead to Exceptions being raised.
pass
namespace = dict()
try:
exec(answer, namespace)
except:
# Answer code could not be run without raising exceptions, i.e., the code
# is poorly written.
# Code you want to run when the answer is wrong.
else:
# The code hasn't raised Exceptions, time to test the variables.
x, y, z = namespace['x'], namespace['y'], namespace['z']
if (x == 2) and (y == 6) and (z == x * y):
# Code you want to run when the answer is right.
else:
# Code you want to run when the answer is wrong.
I sense that this is not the best way to do this, but it is certainly an attempt.
I hope this helped.
EDIT: Removed some bad code and added part of Tadhg McDonald-Jensen's comment.
I'm learning to code and I got this task to create a program. (It is from Starting out with Python, 4th edition, Chapter 8, Exercise 14.)
I have created all functions and called them all yet the program will only execute one and return none for the rest. What am I missing?
def main():
gas_file=open('GasPrices.txt', 'r')
average_per_year(gas_file)
print()
average_per_month(gas_file)
print()
highest_lowers(gas_file)
print()
lowest_to_highest(gas_file)
print()
highest_to_lowest(gas_file)
gas_file.close()
Here is full code: https://pastecode.xyz/view/b007ccb0
I get no error warning or anything btw
Kind regards, Juls
The first function you call reads to the end of gas_file until it's at the end. Since you pass the same file handle to all the other functions, they just see an empty file.
Your code would work (and be a lot simpler and faster than what you're trying to do) if instead of copying+pasting the code for reading the file into all of these functions, so that each is trying to re-read the file and do the same work repeatedly, you had a single function that parses the file into some useful data structure (like a list or a dict of values), and then had all the other functions just read the data from that.
Something like:
from typing import List, NamedTuple
class GasPrice(NamedTuple):
month: int
year: int
price: float
def read_prices(gas_file: str) -> List[GasPrice]:
"""Load a list of GasPrices from a file."""
with open(gas_file, 'r') as f:
return [
GasPrice(
int(line[:2]), # month
int(line[6:10]), # year
float(line[11:]) # price
)
for line in f
]
def main() -> None:
prices = read_prices('GasPrices.txt')
# ... do stuff with prices
Just trying to learn and I"m wondering if multiprocessing would speed
up this for loop ,.. trying to compare
alexa_white_list(1,000,000 lines) and
dnsMISP (can get up to 160,000 lines)
Code checks each line in dnsMISP and looks for it in alexa_white_list.
if it doesn't see it, it adds it to blacklist.
Without mp_handler function the code works fine but it takes
around 40-45 minutes. For brevity, I've omitted all the other imports and
the function that pulls down and unzips the alexa white list.
The below gives me the following error -
File "./vetdns.py", line 128, in mp_handler
p.map(dns_check,dnsMISP,alexa_white_list)
NameError: global name 'dnsMISP' is not defined
from multiprocessing import Pool
def dns_check():
awl = []
blacklist = []
ctr = 0
dnsMISP = open(INPUT_FILE,"r")
dns_misp_lines = dnsMISP.readlines()
dnsMISP.close()
alexa_white_list = open(outname, 'r')
alexa_white_list_lines = alexa_white_list.readlines()
alexa_white_list.close()
print "converting awl to proper format"
for line in alexa_white_list_lines:
awl.append(".".join(line.split(".")[-2:]).strip())
print "done"
for host in dns_misp_lines:
host = host.strip()
host = ".".join(host.split(".")[-2:])
if not host in awl:
blacklist.append(host)
file_out = open(FULL_FILENAME,"w")
file_out.write("\n".join(blacklist))
file_out.close()
def mp_handler():
p = Pool(2)
p.map(dns_check,dnsMISP,alexa_white_list)
if __name__ =='__main__':
mp_handler()
If I label it as global etc I still get the error. I'd appreciate any
suggestions!!
There's no need for multiprocessing here. In fact this code can be greatly simplified:
def get_host_form_line(line):
return line.strip().split(".", 1)[-1]
def dns_check():
with open('alexa.txt') as alexa:
awl = {get_host_from_line(line) for line in alexa}
blacklist = []
with open(INPUT_FILE, "r") as dns_misp_lines:
for line in dns_misp_lines:
host = get_host_from_line(line)
if host not in awl:
blacklist.append(host)
with open(FULL_FILENAME,"w") as file_out:
file_out.write("\n".join(blacklist))
Using a set comprehension to create your Alexa collection has the advantage of being O(1) lookup time. Sets are similar to dictionaries. They are pretty much dictionaries that only have keys with no values. There is some additional overhead in memory and the initial creation time will likely be slower since the values you put in to a set need to be hashed and hash collisions dealt with but the increase in performance you gain from the faster look ups should make up for it.
You can also clean up your line parsing. split() takes an additional parameter that will limit the number of times the input is split. I'm assuming your lines look something like this:
http://www.something.com and you want something.com (if this isn't the case let me know)
It's important to remember that the in operator isn't magic. When you use it to check membership (is an element in the list) what it's essentially doing under the hood is this:
for element in list:
if element == input:
return True
return False
So every time in your code you did if element in list your program had to iterate across each element until it either found what you were looking for or got to the end. This was probably the biggest bottleneck of your code.
You tried to read a variable named dnsMISP to pass as an argument to Pool.map. It doesn't exist in local or global scope (where do you think it's coming from?), so you got a NameError. This has nothing to do with multiprocessing; you could just type a line with nothing but:
dnsMISP
and have the same error.
I want to write a tool in Python to prepare a simulation study by creating for each simulation run a folder and a configuration file with some run-specific parameters.
study/
study.conf
run1
run.conf
run2
run.conf
The tool should read the overall study configuration from a file including (1) static parameters (key-value pairs), (2) lists for iteration parameters, and (3) some small code snippets to calculate further parameters from the previous ones. The latter are run specific depending on the permutation of the iteration parameters used.
Before writing the run.conf files from a template, I need to run some code like this to determine the specific key-value pairs from the code snippets for that run
code = compile(code_str, 'foo.py', 'exec')
rv=eval(code, context, { })
However, as this is confirmed by the Python documentation, this just leads to a None as return value.
The code string and context dictionary in the example are filled elsewhere. For this discussion, this snippet should do it:
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
I have done this before in Perl and Java+JavaScript. There, you just give the code snippet to some evaluation function or script engine and get in return a value (object) from the last executed statement -- not a big issue.
Now, in Python I struggle with the fact that eval() is too narrow just allowing one statement and exec() doesn't return values in general. I need to import modules and sometimes do some slightly more complex calculations, e.g., 5 lines of code.
Isn't there a better solution that I don't see at the moment?
During my research, I found some very good discussions about Pyhton eval() and exec() and also some tricky solutions to circumvent the issue by going via the stdout and parsing the return value from there. The latter would do it, but is not very nice and already 5 years old.
The exec function will modify the global parameter (dict) passed to it. So you can use the code below
code_str="""import math
Result1 = math.sqrt(width**2 + height**2)
"""
context = {
'width' : 30,
'height' : 10
}
exec(code_str, context)
print (context['Result1']) # 31.6
Every variable code_str created will end up with a key:value pair in the context dictionary. So the dict is the "object" like you mentioned in JavaScript.
Edit1:
If you only need the result of the last line in code_str and try to prevent something like Result1=..., try the below code
code_str="""import math
math.sqrt(width**2 + height**2)
"""
context = { 'width' : 30, 'height' : 10 }
lines = [l for l in code_str.split('\n') if l.strip()]
lines[-1] = '__myresult__='+lines[-1]
exec('\n'.join(lines), context)
print (context['__myresult__'])
This approach is not as robust as the former one, but should work for most case. If you need to manipulate the code in a sophisticated way, please take a look at the Abstract Syntax Trees
Since this whole exec() / eval() thing in Python is a bit weird ... I have chose to re-design the whole thing based on a proposal in the comments to my question (thanks #jonrsharpe).
Now, the whole study specification is a .py module that the user can edit. From there, the configuration setup is directly written to a central object of the whole package. On tool runs, the configuration module is imported using the code below
import imp
# import the configuration as a module
(path, name) = os.path.split(filename)
(name, _) = os.path.splitext(name)
(file, filename, data) = imp.find_module(name, [path])
try:
module = imp.load_module(name, file, filename, data)
except ImportError as e:
print(e)
sys.exit(1)
finally:
file.close()
I came across similar needs, and finally figured out a approach by playing with ast:
import ast
code = """
def tf(n):
return n*n
r=tf(3)
{"vvv": tf(5)}
"""
ast_ = ast.parse(code, '<code>', 'exec')
final_expr = None
for field_ in ast.iter_fields(ast_):
if 'body' != field_[0]: continue
if len(field_[1]) > 0 and isinstance(field_[1][-1], ast.Expr):
final_expr = ast.Expression()
final_expr.body = field_[1].pop().value
ld = {}
rv = None
exec(compile(ast_, '<code>', 'exec'), None, ld)
if final_expr:
rv = eval(compile(final_expr, '<code>', 'eval'), None, ld)
print('got locals: {}'.format(ld))
print('got return: {}'.format(rv))
It'll eval instead of exec the last clause if it's an expression, or have all execed and return None.
Output:
got locals: {'tf': <function tf at 0x10103a268>, 'r': 9}
got return: {'vvv': 25}
First, sorry for my stupid title :) And here is my problem.. Actually it's not a problem. Everything works, but I want to have better structure...
I have a python script with a loop "looped" each second.
In the loop there are many many IFs. Is it possible to put each IF in a separate file and then to include it in the loop? So this way every time the loop is "looped", all the IFs will be passed, too..
There are too many conditions in my script and all of them are different generally from the otheres so I want to have some kind of folder with modules - mod_wheather.py, mod_sport.py, mod_horoscope.py, etc..
Thanks in advance. I hope I wrote everything understandable..
EDIT:
Here is a structural example of what I have now:
while True:
if condition=='news':
#do something
if condition=='sport':
#so something else
time.sleep(1)
It will be good if I can have something like this:
while True:
import mod_news
import mod_sport
time.sleep(1)
And these IFs from the first example to be separated in files mod_news.py, mod_sport.py...
perhaps you wonder how to work with your own modules in general.
make one file named 'weather.py' and have it contain the appropriate if-statements like:
""" weather.py - conditions to check """
def check_all(*args, **kwargs):
""" check all conditions """
if check_temperature(kwargs['temperature']):
... your code ...
def check_temperature(temp):
-- perhaps some code including temp or whatever ...
return temp > 40
same for sport.py, horoscope.py etc
then your main script would look like:
import time, weather, sport, horoscope
kwargs = {'temperature':30}
condition = 'weather'
while True:
if condition == 'weather':
weather.check_all(**kwargs)
elif condition == 'sport':
sport.check_all()
elif condition == 'horoscope':
horoscope.check_all()
time.sleep(1)
edit: edited according to the edit in your question. Note that I suggest importing all modules only one time, at the beginning of the script, and using its functions. This is better than executing code by importing. But if you insist, you could use reload(weather), which actually performs a reload including code execution. But I cannot stress too much that using functions of external modules is a better way to go!
Put them in functions in separate files and then Import them:
"""thing1.py
A function to demonstrate
"""
def do_things(some_var):
print("Doing things with %s" % (some_var))
``
"""thing2.py
Demonstrates the same thing with a condition
"""
def do_things(some_var):
if len(some_var) < 10:
print("%s is < 10 characters long" % (some_var))
else:
print("too long")
``
"""main_program.py"""
import thing1, thing2
myvar = "cats"
thing1.do_things(myvar)
thing2.do_things(myvar)
I believe you are looking for some kind of PHP-like include() or C prepocessor #include. You would have a file such as the included.py below:
a = 2
print "ok"
and another file which has the following code:
for i in values:
import included
and you want the result to be equivalent to
for i in values:
a = 2
print "ok"
Is it what you are looking for? If so... no, it is not possible. Once Python imports a module, the code of the module is executed and following imports of the same mode only retrieve the already imported instance of the module. The code of a module is not executed everytime it is imported.
I can invent some crazy ways of doing it (let us say, file.read() + eval(), or calling reload() in an imported module.) but it would be a bad idea anyway. I bet we can think of a better solution to your real problem :)
Perhaps all you need is to call functions in your loop; and have those functions in other modules, which you import as needed.
while true:
if condition:
from module_a import f
f()
if condition2
from module_b import g
g()
Though the above is legal Python, and so answers your question, you should in practice write all the imports at the top of your file.
You could import the needed modules if they're needed, for example:
if condition:
import weather
... do something
However I'm not sure if that's what you really want.
I have a python script with a loop "looped" each second. In the loop
there are many many IFs.
Then you must optimize the repeatedly executed tests. Suppose there are 50 IFs blocks in your code and that in a turn of the for-loop, the N th condition is True: that means that the N-1 other conditions must be tested before the N th is tested and triggers the execution of the corresponding code.
It would be preferable to do so:
# to_include.py
def func_weather(*args,**kwargs):
# code
return "I'm the weather"
def func_horoscope(*args,**kwargs):
# code
return "Give me your birth'date"
def func_gastronomy(*args,**kwargs):
# code
return 'Miam crunch'
def func_sports(*args,**kwargs):
# code
return 'golf, swimming and canoeing in the resort station'
didi = {'weather':func_weather, 'horoscope':func_horoscope,
'gastronomy':func_gastronomy, 'sports':func_sports}
and the main module:
# use_to_include.py
import to_include
x = 'sports'
y = to_include.didi[x]()
# instead of
# if x =='weather' : y = func_weather()
# elif x=='horoscope' : y = func_horoscope()
# elif x=='gastronomy': y = func_gastronomy()
# elif x=='sports' : y = func_sports()
print y
result
golf, swimming and canoeing in the resort station