Python: Logical flow for setup and validation - python

I am having trouble figuring out how to structure the logical flow of my python application's initial setup module. I have a number of functions that perform small steps in finding that required config files are present, validating them if they are there, creating them if they are not, testing a login with the validated credentials, and finally, if everything checks out, starting the main application.
A simplified list of the functions is below:
def config_files_present(config_path):
config present = is_the_file_there
return config_present
def validate_config_file(config_file):
valid_email = code validating email returns True or False
valid_fqdn = code validating fqdn returns True or False
valid_config = valid_email is True and valid_fqdn is True
return valid_config
def write_config(config_path):
code with user input to collect credentials and write config.ini
config_file_written = True
return config_file_written
def test_login(config_file):
code to login with config_file and
return mailbox_object
etc
The (simplified) happy path is easier for me to visualize:
ready = False
while not ready:
config_present = config_files_present(config_path)
if not config_present:
write_config_file = write_config(config_path)
if write_config_file:
valid_config = validate_config_file(config_file)
if valid_config:
login_test = test_login(config_file)
if login_test:
ready = True
run the main app
The complicated path is more complicated. I have mapped out how it should work, but can't figure out how to program it.
Trying to represent the above flow in the else clauses of each if feels like it will turn into a bottomless recursive spiral. I suspect that there is probably an established best practice or structure to do this kind of thing, but I don't know what it is.
Do I power through this recursive mess, or is there a better way to structure this logical flow?
EDIT: Going to close this question. Because each function has only 1 possible next step if it evaluates to True, I should be able to call the next function in sequence within the function, provided the original logical test evaluates to true. Then, in my else clauses, I will just kick it back to the step that needs to be repeated.

Going to close this question. Because each function has only 1 possible next step if it evaluates to True, I should be able to call the next function in sequence within the function, provided the original logical test evaluates to true. Then, in my else clauses, I will just kick it back to the step that needs to be repeated.

Related

how can i trigger an action when a variable change it value in this case time?

I need to detect when the minutes of the clock/time change and do something,
This is mi code so far, for the clock but still can figuruate out in python how to detect the value has change and do action after. Any help will be apreciated i come from a c++ backgorund my implementations seems so far not working.
while True:
now = datetime.now()
print(now.strftime("%M), end = " ", flush = true)
time.sleep(1)
currentMin = now.srtftime("%M")
that worked for me:
from datetime import datetime
import time
past_min = None
while True:
#current min
now_min = int(datetime.now().strftime("%M"))
#first iteration
if not past_min:
past_min = now_min
if now_min != past_min:
#call your function here
print("Min change detected")
past_min = now_min
#print the seconds
print(datetime.now().strftime("%S"))
time.sleep(1.5)
I think you can create a class (in the below example Minute) with a property currenMin to store the current minute value. By using #<property>.setter function, when the property <property> is changed, it will trigger the setter function
from datetime import datetime
import time
class Minute(object):
def __init__(self):
self._currenMin = ''
#property
def currentMin(self):
return self._currenMin
#currentMin.setter
def currentMin(self, value):
if value != self._currenMin:
# ACTION CODE BELOW
print('Minute changed')
self._currenMin = value
minute = Minute()
while True:
now = datetime.now()
print(now.strftime("%M"), end=" ", flush = True)
time.sleep(1)
minute.currentMin = now.strftime("%M")
Well, for the general case with simple variables, you can't simply do it. There are two simple options to do something similar:
if you control EVERYTHING that writes it, make them trigger that action
write code that regularly checks it and triggers the action when it changes
use language tools like a custom setter (see #user696969's answer)
The first case needs you to control everything that could modify that value. At that point, you might not even need a variable, and just pass the new value (and you can reverse this by having a variable that is always updated). This is a very common pattern, called Event-driven programming, and heavily used for example in UIs, websites (client-side, see a list of DOM events for example) and game frameworks (see pygame's documentation on events)
The second-case of writing a loop or checking it regularly can also work, however, there are some downsides to it as well. You probably don't want to write an infinite loop waiting for it to change, especially not in a way that also blocks the changing of that variable, and thus dead-locking the entire program as it's preventing something it's waiting for. If you just check it regularly between other, it might be hard to ensure it will be checked regardless of what else is the program doing. You might use multiple threads for it, but that brings it's own set of problems. You also have to store and update the previous value, so you can compare it. This might be slow or memory-consuming if the variable holds too much data.
You can also use language tools with custom setters. This is clean, but can not be used for any variable, just for class attributes, so you still need some control over the rest of the program.
Generally I'd use the event-driven approach or the setter one, depending on the wider context. However, for such simple cases, the checking is also fine. The simplest solution might event be to remove the need for this entirely.

How to test while-loop (once) with nosetest (Python 2.7)

I'm pretty new to this whole "programming thing" but at age 34 I thought that I'd like to learn the basics.
I unfortunately don't know any python programmers. I'm learning programming due to personal interest (and more and more for the fun of it) but my "social habitat" is not "where the programmers roam" ;) .
I'm almost finished with Zed Shaws "Learn Python the Hard Way" and for the first time I can't figure out a solution to a problem. The last two days I didn't even stumble upon useful hints where to look when I repeatedly rephrased (and searched for) my question.
So stackoverflow seems to be the right place.
Btw.: I lack also the correct vocabular quite often so please don't hesitate to correct me :) . This may be one reason why I can't find an answer.
I use Python 2.7 and nosetests.
How far I solved the problem (I think) in the steps I solved it:
Function 1:
def inp_1():
s = raw_input(">>> ")
return s
All tests import the following to be able to do the things below:
from nose.tools import *
import sys
from StringIO import StringIO
from mock import *
import __builtin__
# and of course the module with the functions
Here is the test for inp_1:
import __builtin__
from mock import *
def test_inp_1():
__builtin__.raw_input = Mock(return_value="foo")
assert_equal(inp_1(), 'foo')
This function/test is ok.
Quite similar is the following function 2:
def inp_2():
s = raw_input(">>> ")
if s == '1':
return s
else:
print "wrong"
Test:
def test_inp_2():
__builtin__.raw_input = Mock(return_value="1")
assert_equal(inp_1(), '1')
__builtin__.raw_input = Mock(return_value="foo")
out = StringIO()
sys.stdout = out
inp_1()
output = out.getvalue().strip()
assert_equal(output, 'wrong')
This function/test is also ok.
Please don't assume that I really know what is happening "behind the scenes" when I use all the stuff above. I have some layman-explanations how this is all functioning and why I get the results I want but I also have the feeling that these explanations may not be entirely true. It wouldn't be the first time that how I think sth. works turns out to be different after I've learned more. Especially everything with "__" confuses me and I'm scared to use it since I don't really understand what's going on. Anyway, now I "just" want to add a while-loop to ask for input until it is correct:
def inp_3():
while True:
s = raw_input(">>> ")
if s == '1':
return s
else:
print "wrong"
The test for inp_3 I thought would be the same as for inp_2 . At least I am not getting error messages. But the output is the following:
$ nosetests
......
# <- Here I press ENTER to provoke a reaction
# Nothing is happening though.
^C # <- Keyboard interrupt (is this the correct word for it?)
----------------------------------------------------------------------
Ran 7 tests in 5.464s
OK
$
The other 7 tests are sth. else (and ok).
The test for inp_3 would be test nr. 8.
The time is just the times passed until I press CTRL-C.
I don't understand why I don't get error- or "test failed"-meassages but just an "ok".
So beside the fact that you may be able to point out bad syntax and other things that can be improved (I really would appreciate it, if you would do this), my question is:
How can I test and abort while-loops with nosetest?
So, the problem here is when you call inp_3 in test for second time, while mocking raw_input with Mock(return_value="foo"). Your inp_3 function runs infinite loop (while True) , and you're not interrupting it in any way except for if s == '1' condition. So with Mock(return_value="foo") that condition is never satisfied, and you loop keeps running until you interrupt it with outer means (Ctrl + C in your example). If it's intentional behavior, then How to limit execution time of a function call in Python will help you to limit execution time of inp_3 in test. However, in cases of input like in your example, developers often implement a limit to how many input attempts user have. You can do it with using variable to count attempts and when it reaches max, loop should be stopped.
def inp_3():
max_attempts = 5
attempts = 0
while True:
s = raw_input(">>> ")
attempts += 1 # this is equal to "attempts = attempts + 1"
if s == '1':
return s
else:
print "wrong"
if attempts == max_attempts:
print "Max attempts used, stopping."
break # this is used to stop loop execution
# and go to next instruction after loop block
print "Stopped."
Also, to learn python I can recommend book "Learning Python" by Mark Lutz. It greatly explains basics of python.
UPDATE:
I couldn't find a way to mock python's True (or a builtin.True) (and yea, that sounds a bit crazy), looks like python didn't (and won't) allow me to do this. However, to achieve exactly what you desire, to run infinite loop once, you can use a little hack.
Define a function to return True
def true_func():
return True
, use it in while loop
while true_func():
and then mock it in test with such logic:
def true_once():
yield True
yield False
class MockTrueFunc(object):
def __init__(self):
self.gen = true_once()
def __call__(self):
return self.gen.next()
Then in test:
true_func = MockTrueFunc()
With this your loop will run only once. However, this construction uses a few advanced python tricks, like generators, "__" methods etc. So use it carefully.
But anyway, generally infinite loops considered to be bad design solutions. Better to not getting used to it :).
It's always important to remind me that infinite loops are bad. So thank you for that and even more so for the short example how to make it better. I will do that whenever possible.
However, in the actual program the infinite loop is how I'd like to do it this time. The code here is just the simplified problem.
I very much appreciate your idea with the modified "true function". I never would have thought about that and thus I learned a new "method" how tackle programming problems :) .
It is still not the way I would like to do it this time, but this was the so important clue I needed to solve my problem with existing methods. I never would have thought about returning a different value the 2nd time I call the same method. It's so simple and brilliant it's astonishing me :).
The mock-module has some features that allows a different value to be returned each time the mocked method is called - side effect .
side_effect can also be set to […] an iterable.
[when] your mock is going to be
called several times, and you want each call to return a different
value. When you set side_effect to an iterable every call to the mock
returns the next value from the iterable:
The while-loop HAS an "exit" (is this the correct term for it?). It just needs the '1' as input. I will use this to exit the loop.
def test_inp_3():
# Test if input is correct
__builtin__.raw_input = Mock(return_value="1")
assert_equal(inp_1(), '1')
# Test if output is correct if input is correct two times.
# The third time the input is corrct to exit the loop.
__builtin__.raw_input = Mock(side_effect=['foo', 'bar', '1'])
out = StringIO()
sys.stdout = out
inp_3()
output = out.getvalue().strip()
# Make sure to compare as many times as the loop
# is "used".
assert_equal(output, 'wrong\nwrong')
Now the test runs and returns "ok" or an error e.g. if the first input already exits the loop.
Thank you very much again for the help. That made my day :)

how can I detect infinite loops in python

I am learning Python 3 and working on an exercise that calls for writing a Python program which simulates/reads a BASIC program as input. I am stuck on writing the part of the Python program that should detect infinite loops. Here is the code I have so far:
def execute(prog):
while True:
location = 0
if prog[location] == len(prog) - 1:
break
return "success"
getT = prog[location].split()
T = len(getT) - 1
location = findLine(prog, T)
visited = [False] * len(prog)
Here, prog is a list of strings containing the BASIC program (strings are in the form of 5 GOTO 30, 10 GOTO 20, etc.).
T is the target string indicated in prog[location].
If the BASIC program has an infinite loop, then my Python program will have an infinite loop. I know that if any line is visited twice, then it loops forever, and my program should return "infinite loop".
A hint given by the tutorial assistant says "initialize a list visited = [False] * len(prog) and change visited[i] to True when prog[i] is visited. Each time through the loop, one value updates in visited[]. Think about how you change a single value in a list. Then think about how you identify which value in visited[] needs to change."
So this is the part I am stuck on. How do I keep track of which strings in prog have been visited/looped through?
I'm not sure I agree that visiting a line twice proves an infinite loop. See the comments under the question. But I can answer the actual question.
Here's the hint:
A hint given by the tutorial assistant says "initialize a list visited = [False] * len(prog) and change visited[i] to True when prog[i] is visited. Each time through the loop, one value updates in visited[]. Think about how you change a single value in a list. Then think about how you identify which value in visited[] needs to change."
This is saying you should have two lists, one that contains the program, and one that contains true/false flags. The second one is to be named visited and initially contains False values.
The Python code is just like the hint says:
visited = [False] * len(prog)
This uses the * list operator, "list repetition", to repeat a length-1 list and make a new list of a longer length.
To change visited[i] to True is simple:
visited[i] = True
Then you can do something like this:
if visited[i]:
print("We have already visited line {}".format(i))
print("Infinite loop? Exiting.")
sys.exit(1)
Note that we are testing for the True value by simply saying if visited[i]:
We could also write if visited[i] == True: but the shorter form is sufficient and is customary in the Python community. This and other customary idioms are documented here: http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
For a program this small, it's not too bad to keep two lists like this. For larger and complex programs, I prefer to keep everything together in one place. This would use a "class" which you might not have learned yet. Something like this:
class ProgramCode(object):
def __init__(self, statement):
self.code = statement
self.visited = False
prog = []
with open(input_basic_program_file, "rt") as f:
for line in f:
prog.append(ProgramCode(line))
Now instead of two lists, we have a single list where each item is a bit of BASIC code and a visited flag.
P.S. The above shows an explicit for loop that repeatedly uses .append() to add to a list. An experienced Python developer would likely use a "list comprehension" instead, but I wanted to make this as easy to follow as possible.
Here's the list comprehension. Don't worry if it looks weird now; your class will teach this to you eventually.
with open(input_basic_program_file, "rt") as f:
prog = [ProgramCode(line) for line in f]
I know of no automatic way of infinite loop detection in Python, but by using divide and conquer methods and testing individual functions, you can find the offending function or block of code and then proceed to debug further.
If the Python program outputs data, but you never see that output, that's a good indicator you have an infinite loop. You can test all your functions in the repl, and the function that does "not come back" [to the command prompt] is a likely suspect.
You can write output under a debug variable of some sort, to be shut off when everything works. This could be a member variable of a Python class to which your code would have to have access to at any time, or you could have a module-scoped variable like Debug=1 and use debug levels to print varying amounts of debug info, like 1 a little, 2 more, 3, even more, and 4 verbose.
As an example, if you printed the value of a loop counter in a suspected function, then eventually that loop counter would keep printing well beyond the count of data (test records) you were using to test.
Here is a combination I came up with using parts of J. Carlos P.'s answer with the hints that steveha gave and using the hint that the instructions gave:
def execute(prog):
location = 0
visited = [False] * len(prog)
while True:
if location==len(prog)-1:
return "success"
findT = prog[location].split()
T = findT[- 1]
if visited[location]:
return "infinite loop"
visited[location] = True
location = findLine(prog, T)

Forcing a callback to be executed first

I've got the following codesnippet in a program using secure multiparty computation:
c = self.runtime.open(b) # open result
c.addCallback(self.determine)
j = self.compute(i)
return j
Now the function determine sets a boolean to either false or true, depending on the value of c. This boolean is then used by the function compute.
I thought that callbacks are always executed first, before the rest of the program is. However, I'm getting an error from compute that the boolean is undefined.
How can I force the callback to be executed before compute is executed?
Because I'm working within a secure multiparty computation framework, I have to work with callbacks since the value for c is a shared secret. However, the problem would also appear without secret shares I think. The language is Python.
The code for determine and compute would be something like this:
def determine(c):
global computeB
computeB = False
if c == 1:
computeB = True
else:
computeB = False
return c
def compute(i):
if computeB:
do this
else:
do this
return result
The callback gets executed when it gets executed. There is no point in "making" it execute earlier.
I guess you are dealing with twisted so heres a tutorial http://krondo.com/?page_id=1327 but it is helpfull even for understanding async programming in general, which you obviously need.
I'm not a pro in async but I think you want to yield your first function and tell your routine to wait before it goes on.
yield self.runtime.open(b) # open result
j = self.compute(i)
return j

Return continuous result from a single function call

I have got stuck with a problem.
It goes like this,
A function returns a single result normally. What I want is it to return continuous streams of result for a certain time frame(optional).
Is it feasible for a function to repeatedly return results for a single function call?
While browsing through the net I did come across gevent and threading. Will it work if so any heads up how to solve it?
I just need to call the function carry out the work and return results immediately after every task is completed.
Why you need this is not specified in the question, so it is hard to know what you need, but I will give you a general idea, and code too.
You could return in that way: return var1, var2, var3 (but that's not what you need I think)
You have multiple options: either blocking or non-blocking. Blocking means your code will no longer execute while you are calling the function. Non-blocking means that it will run in parallel. You should also know that you will definitely need to modify the code calling that function.
That's if you want it in a thread (non-blocking):
def your_function(callback):
# This is a function defined inside of it, just for convenience, it can be any function.
def what_it_is_doing(callback):
import time
total = 0
while True:
time.sleep(1)
total += 1
# Here it is a callback function, but if you are using a
# GUI application (not only) for example (wx, Qt, GTK, ...) they usually have
# events/signals, you should be using this system.
callback(time_spent=total)
import thread
thread.start_new_thread(what_it_is_doing, tuple(callback))
# The way you would use it:
def what_I_want_to_do_with_each_bit_of_result(time_spent):
print "Time is:", time_spent
your_function(what_I_want_to_do_with_each_bit_of_result)
# Continue your code normally
The other option (blocking) involves a special kind of functions generators which are technically treated as iterators. So you define it as a function and acts as an iterator. That's an example, using the same dummy function than the other one:
def my_generator():
import time
total = 0
while True:
time.sleep(1)
total += 1
yield total
# And here's how you use it:
# You need it to be in a loop !!
for time_spent in my_generator():
print "Time spent is:", time_spent
# Or, you could use it that way, and call .next() manually:
my_gen = my_generator()
# When you need something from it:
time_spent = my_gen.next()
Note that in the second example, the code would make no sense because it is not really called at 1 second intervals, because there's the other code running each time it yields something or .next is called, and that may take time. But I hope you got the point.
Again, it depends on what you are doing, if the app you are using has an "event" framework or similar you would need to use that, if you need it blocking/non-blocking, if time is important, how your calling code should manipulate the result...
Your gevent and threading are on the right track, because a function does what it is programmed to do, either accepting 1 var at a time or taking a set and returning either a set or a var. The function has to be called to return either result, and the continuous stream of processing is probably taking place already or else you are asking about a loop over a kernel pointer or something similar, which you are not, so ...
So, your calling code which encapsulates your function is important, the function, any function, eg, even a true/false boolean function only executes until it is done with its vars, so there muse be a calling function which listens indefinitely in your case. If it doesn't exist you should write one ;)
Calling code which encapsulates is certainly very important.
Folks aren't going to have enough info to help much, except in the super generic sense that we can tell you that you are or should be within in some framework's event loop, or other code's loop of some form already- and that is what you want to be listening to/ preparing data for.
I like "functional programming's," "map function," for this sort of thing. I think. I can't comment at my rep level or I would restrict my speculation to that. :)
To get a better answer from another person post some example code and reveal your API if possible.

Categories

Resources