Assume I have a list my_list, a variable var, and a block of code that modifies the list using the variable
my_list = ['foo']
var = 'bar'
my_list.append(var)
In the actual task I have a lot of variables like var and a lot of commands like append which modify the list. I want to relegate those commands to another module. In the case at hand I would like to have two modules: modify.py which contains the modifying commands
my_list.append(var)
and main.py which defines the list and the variable and somehow uses the code from the modify.py
my_list = ['foo']
var = 'bar'
import_and_run modify
The goal is to make the main file more readable. Modifying commands in my case can be nicely grouped and would really be good as separate modules. However, I am only aware of the practice when one imports a function from a module, not a block of code. I do not want to make the whole modify.py module a function because
1) I don't want to pass all the arguments needed. Rather, I want modify.py to directly have access to main.py name space.
2) code in modify.py is not really a function. It runs only once. Also, I do not the whole module to be a body of a function, that just does not feel right.
How do I achieve that? Or the whole attitude is wrong?
If your goal is to make the code more readable, I'd suggest taking these steps.
Decompose your problem into a series of separate actions.
Give these actions names.
Define a function main in your module that calls functions named
after the actions:
def main():
do_setp1()
do_step2()
# etc
return
Separate you existing code into the functions that you're calling in
main()
As #flaschbier suggested, collect related, common parameters into dictionaries to make passing the around easier to manage.
Consider repeating these steps on your new functions, decomposing
them into sub-functions.
Done well, you should be left with a file that's easier to look at, because the function definitions and their indented bodies break up the flow of text.
The code should be easier to reason about because you only need to understand one function at a time, instead of the entire script.
Generally you want to keep all the code related to a particular task in a single module, unless there's more than say 500 lines. But before moving code into separate modules see if you can reduce the total lines of code by factoring repeated code into functions, or making your code more succinct: for example see if for loops can be replaced by list comprehensions.
Consider using code linting tools to help you make the code well-formatted.
So in summary: don't go against the grain of Python by hiding code in another
module and going down the import_and_run route. Instead use good code organisation and Python's inherent good visual structure to make your code readable.
By the way, seems like you still haven't grasped the concept of Python modules.
Well, modules in Python are the .py files. Each function, class or even variables in a .py file can be imported into another program.
Consider a (perhaps crazy) example like this crazy.py:
class crazyCl:
# crazy stuffs
pass
def crazyFn():
# some another crazy stuffs
crazyVar = 'Please do not try this at home'
Now, to import any of these, into another program, say goCrazy.py in the same folder, simply do this
import crazy # see ma, no .py
if __name__ == '__main__':
print crazy.crazyVar # Please do not try this at home
This is a simple introduction to Python modules. There are many other features like packages that have to be tried out.
As a simple introduction, this should do. Hope you got some idea.
Related
In other languages, like Java, you can do something like this:
String path;
if (exists(path = "/some/path"))
my_path = path;
the point being that path is being set as part of specifying a parameter to a method call. I know that this doesn't work in Python. It is something that I've always wished Python had.
Is there any way to accomplish this in Python? What I mean here by "accomplish" is to be able to write both the call to exists and the assignment to path, as a single statement with no prior supporting code being necessary.
I'll be OK with it if a way of doing this requires the use of an additional call to a function or method, including anything I might write myself. I spent a little time trying to come up with such a module, but failed to come up with anything that was less ugly than just doing the assignment before calling the function.
UPDATE: #BrokenBenchmark's answer is perfect if one can assume Python 3.8 or better. Unfortunately, I can't yet do that, so I'm still searching for a solution to this problem that will work with Python 3.7 and earlier.
Yes, you can use the walrus operator if you're using Python 3.8 or above:
import os
if os.path.isdir((path := "/some/path")):
my_path = path
I've come up with something that has some issues, but does technically get me where I was looking to be. Maybe someone else will have ideas for improving this to make it fully cool. Here's what I have:
# In a utility module somewhere
def v(varname, arg=None):
if arg is not None:
if not hasattr(v, 'vals'):
v.vals = {}
v.vals[varname] = arg
return v.vals[varname]
# At point of use
if os.path.exists(v('path1', os.path.expanduser('~/.harmony/mnt/fetch_devqa'))):
fetch_devqa_path = v('path1')
As you can see, this fits my requirement of no extra lines of code. The "variable" involved, path1 in this example, is stored on the function that implements all of this, on a per-variable-name basis.
One can question if this is concise and readable enough to be worth the bother. For me, the verdict is still out. If not for the need to call the v() function a second time, I think I'd be good with it structurally.
The only functional problem I see with this is that it isn't thread-safe. Two copies of the code could run concurrently and run into a race condition between the two calls to v(). The same problem is greatly magnified if one fails to choose unique variable names every time this is used. That's probably the deal killer here.
Can anyone see how to use this to get to a similar solution without the drawbacks?
I'm working in a large Python codebase which has its own functions defined for running certain sections of code called "rules". We have these rules defined in a certain place, and to run them we call the following function with their name as an argument:
runRule(<name of rule>)
Under the hood I don't know how it works, and don't have access to that information (sadly), but I think it does something like copying and pasting the rule code in the current position. I know it's not ideal and messy, but I don't have a choice but to work with it as it is.
My concern is that using runRule could potentially modify the local/global variables in the current namespace in an undesirable/unexpected way.
My question is whether something like the following code would make sense to avoid my concern?
# (using deepcopy didn't work when I tried... so will have to settle for a shallow copy)
globals_temp = dict(globals())
runRule("my_edit_rule")
# restore globals to what it was before running the rule
globals().update(globals_temp)
# this was added to globals originally and is no longer needed
del globals()["globals_temp"]
I know it's yuck code, but it may be my only option given then codebase I'm working with...
I am trying to write a function that can take a sentence as an input and output the same list of words in a scrambled order. So far I have the following:
import random
def scramble(sentence):
split = sentence.split()
shuffle(split)
return ' '.join(split)
print scramble(sentence)
The issue is that Jython is returning an error vice the output I'm looking for. I've been tweaking with the code for hours with no luck.
Thanks!
You need to use
random.shuffle(split)
When you import from a module, that module's name becomes a namespace. In order to use any of its functions, you need to direct the Python interpreter that those functions reside in a certain namespace. Larger programs can have dozens (or more!) of import statements, and namespaces help keep everything clear, so you're not polluting the main (global) namespaces with potentially multiple functions with the same name.
When you do from module import function the name function is now in the global namespace, and you can use it without prefixing its originating module name on the front. This isn't always a great idea in larger programs, as you can lose track of things, but here if you wanted to change
import random
to
from random import shuffle
then your program would work as-is.
One other potential issue with your program - at least in the code you've shown, you haven't assigned a value to the variable sentence, so when you call print scramble(sentence) you'll get an error because scramble() has no input.
As a bit of advice, I wouldn't name a variable after a built-in function, as that can screw things up down the line. Maybe call it shuffled or something...
Your problem is on this line:
shuffle(split)
It is part of the random module, thus you need to change the line to this:
random.shuffle(split)
Alternatively, you can adjust your import line to be this:
from random import shuffle
Then you don't need to adjust the shuffle(split) line
When writing Python code, I often find myself wanting to get behavior similar to Lisp's defvar. Basically, if some variable doesn't exist, I want to create it and assign a particular value to it. Otherwise, I don't want to do anything, and in particular, I don't want to override the variable's current value.
I looked around online and found this suggestion:
try:
some_variable
except NameError:
some_variable = some_expensive_computation()
I've been using it and it works fine. However, to me this has the look of code that's not paradigmatically correct. The code is four lines, instead of the 1 that would be required in Lisp, and it requires exception handling to deal with something that's not "exceptional."
The context is that I'm doing interactively development. I'm executing my Python code file frequently, as I improve it, and I don't want to run some_expensive_computation() each time I do so. I could arrange to run some_expensive_computation() by hand every time I start a new Python interpreter, but I'd rather do something automated, particularly so that my code can be run non-interactively. How would a season Python programmer achieve this?
I'm using WinXP with SP3, Python 2.7.5 via Anaconda 1.6.2 (32-bit), and running inside Spyder.
It's generally a bad idea to rely on the existence or not of a variable having meaning. Instead, use a sentinel value to indicate that a variable is not set to an appropriate value. None is a common choice for this kind of sentinel, though it may not be appropriate if that is a possible output of your expensive computation.
So, rather than your current code, do something like this:
# early on in the program
some_variable = None
# later:
if some_variable is None:
some_variable = some_expensive_computation()
# use some_variable here
Or, a version where None could be a significant value:
_sentinel = object()
some_variable = _sentinel # this means it doesn't have a meaningful value
# later
if some_variable is _sentinel:
some_variable = some_expensive_computation()
It is hard to tell which is of greater concern to you, specific language features or a persistent session. Since you say:
The context is that I'm doing interactively development. I'm executing my Python code file frequently, as I improve it, and I don't want to run some_expensive_computation() each time I do so.
You may find that IPython provides a persistent, interactive environment that is pleasing to you.
Instead of writing Lisp in Python, just think about what you're trying to do. You want to avoid calling an expensive function twice and having it run two times. You can write your function do to that:
def f(x):
if x in cache:
return cache[x]
result = ...
cache[x] = result
return result
Or make use of Python's decorators and just decorate the function with another function that takes care of the caching for you. Python 3.3 comes with functools.lru_cache, which does just that:
import functools
#functools.lru_cache()
def f(x):
return ...
There are quite a few memoization libraries in the PyPi for 2.7.
For the use case you give, guarding with a try ... except seems like a good way to go about it: Your code is depending on leftover variables from a previous execution of your script.
But I agree that it's not a nice implementation of the concept "here's a default value, use it unless the variable is already set". Python does not directly support this for variables, but it does have a default-setter for dictionary keys:
myvalues = dict()
myvalues.setdefault("some_variable", 42)
print some_variable # prints 42
The first argument of setdefault must be a string containing the name of the variable to be defined.
If you had a complicated system of settings and defaults (like emacs does), you'd probably keep the system settings in their own dictionary, so this is all you need. In your case, you could also use setdefault directly on global variables (only), with the help of the built-in function globals() which returns a modifiable dictionary:
globals().setdefault("some_variable", 42)
But I would recommend using a dictionary for your persistent variables (you can use the try... except method to create it conditionally). It keeps things clean and it seems more... pythonic somehow.
Let me try to summarize what I've learned here:
Using exception handling for flow control is fine in Python. I could do it once to set up a dict in which I can store what ever I want.
There are libraries and language features that are designed for some form of persistence; these can provide "high road" solutions for some applications. The shelve module is an obvious candidate here, but I would construe "some form of persistence" broadly enough to include #Blender's suggest to use memoization.
I've been thinking about this far too long and haven't gotten any idea, maybe some of you can help.
I have a folder of python scripts, all of which have the same surrounding body (literally, I generated it from a shell script), but have one chunk that's different than all of them. In other words:
Top piece of code (always the same)
Middle piece of code (changes from file to file)
Bottom piece of code (always the same)
And I realized today that this is a bad idea, for example, if I want to change something from the top or bottom sections, I need to write a shell script to do it. (Not that that's hard, it just seems like it's very bad code wise).
So what I want to do, is have one outer python script that is like this:
Top piece of code
Dynamic function that calls the middle piece of code (based on a parameter)
Bottom piece of code
And then every other python file in the folder can simply be the middle piece of code. However, normal module wouldn't work here (unless I'm mistaken), because I would get the code I need to execute from the arguement, which would be a string, and thus I wouldn't know which function to run until runtime.
So I thought up two more solutions:
I could write up a bunch of if statements, one to run each script based on a certain parameter. I rejected this, as it's even worse than the previous design.
I could use:
os.command(sys.argv[0] scriptName.py)
which would run the script, but calling python to call python doesn't seem very elegant to me.
So does anyone have any other ideas? Thank you.
If you know the name of the function as a string and the name of module as a string, then you can do
mod = __import__(module_name)
fn = getattr(mod, fn_name)
fn()
Another possible solution is to have each of your repetitive files import the functionality from the main file
from topAndBottom import top, bottom
top()
# do middle stuff
bottom()
In addition to the several answers already posted, consider the Template Method design pattern: make an abstract class such as
class Base(object):
def top(self): ...
def bottom(self): ...
def middle(self): raise NotImplementedError
def doit(self):
self.top()
self.middle()
self.bottom()
Every pluggable module then makes a class which inherits from this Base and must override middle with the relevant code.
Perhaps not warranted for this simple case (you do still have to import the right module in order to instantiate its class and call doit on it), but still worth keeping in mind (together with its many Pythonic variations, which I have amply explained in many tech talks now available on youtube) for cases where the number or complexity of "pluggable pieces" keeps growing -- Template Method (despite its horrid name;-) is a solid, well-proven and highly scalable pattern [[sometimes a tad too rigid, but that's exactly what I address in those many tech talks -- and that problem doesn't apply to this specific use case]].
However, normal module wouldn't work here (unless I'm mistaken), because I would get the code I need to execute from the arguement, which would be a string, and thus I wouldn't know which function to run until runtime.
It will work just fine - use __import__ builtin or, if you have very complex layout, imp module to import your script. And then you can get the function by module.__dict__[funcname] for example.
Importing a module (as explained in other answers) is definitely the cleaner way to do this, but if for some reason that doesn't work, as long as you're not doing anything too weird you can use exec. It basically runs the content of another file as if it were included in the current file at the point where exec is called. It's the closest thing Python has to a source statement of the kind included in many shells. As a bare minimum, something like this should work:
exec(open(filename).read(None))
How about this?
function do_thing_one():
pass
function do_thing_two():
pass
dispatch = { "one" : do_thing_one,
"two" : do_thing_two,
}
# do something to get your string from the command line (optparse, argv, whatever)
# and put it in variable "mystring"
# do top thing
f = dispatch[mystring]
f()
# do bottom thing