Develop Reference

Python how to access parent's modules and functions from child module

I have three files that all contain classes with the same names but slightly different definitions. Some methods in these classes are identical across all three files, so I abstracted them out to another file, utils.py, where they are defined within a "template" version of the original class. The problem is that these methods invoke functions and modules that exist in the original files but not this new one.
My original approach was to use multiple class inheritance, which would initialize the template class within the scope of the parent class, allowing access to all the functions and modules it requires. However, I was instructed to avoid multiple class inheritance and to simply import the utils file.
Importing does not apply the same scoping logic as mentioned above with inheritance. So here arises my problem. I have created a small example to show what I mean. I am using a module called datajoint. You don't need to know much about it except that a schema is basically a table or collection of tables in a database.
schemas.py
import datajoint as dj
from datetime import datetime
import utils
dj.conn()
schema = dj.Schema('adib_example1')
schema.drop()
schema = dj.Schema('adib_example1')
def test_print():
print("test")
#schema
class Subject(dj.Lookup):
definition = """
subject_id: int
"""
contents = [dict(subject_id=1)]
#schema
class Session(dj.Computed):
definition = """
-> Subject
time: varchar(30)
"""
def make(self, key):
utils.SessionTemplate.make(self,key)
Session.populate() # invokes Session's make(), passing Subject's primary key
Approach 1
Import scoping not working like inheritance
utils.py
class SessionTemplate():
#staticmethod
def make(table, key):
test_print() # parent function usage example
table.time = f"{datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
error
Traceback (most recent call last):
File "/home/.anaconda/imported_make/schemas.py", line 30, in <module>
Session.populate() # invokes Session's make(), passing Subject's primary key
File "/opt/conda/lib/python3.9/site-packages/datajoint/autopopulate.py", line 153, in populate
make(dict(key))
File "/home/.anaconda/imported_make/schemas.py", line 28, in make
utils.SessionTemplate.make(self,key)
File "/home/.anaconda/imported_make/utils.py", line 5, in make
test_print() # parent function usage example
NameError: name 'test_print' is not defined
Approach 2
Importing schemas.py into utils.py works, but requires including schemas. before every imported function and module, which is not practical in my case.
utils.py
import schemas
class SessionTemplate():
#staticmethod
def make(table, key):
schemas.test_print() # parent function usage example
table.time = f"{schemas.datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
Approach 3
Import using * to avoid having to add schemas. before each parent function/module somehow does not provide access to the parents modules and functions.
from schemas import *
class SessionTemplate():
#staticmethod
def make(table, key):
test_print() # parent function usage example
table.time = f"{datetime.now()}" # parent module usage example
new_entry = dict(**key, time=table.time)
table.insert1(new_entry)
error
Traceback (most recent call last):
File "/home/.anaconda/imported_make/run.py", line 1, in <module>
import schemas
File "/home/.anaconda/imported_make/schemas.py", line 30, in <module>
Session.populate() # invokes Session's make(), passing Subject's primary key
File "/opt/conda/lib/python3.9/site-packages/datajoint/autopopulate.py", line 153, in populate
make(dict(key))
File "/home/.anaconda/imported_make/schemas.py", line 28, in make
utils.SessionTemplate().make(self,key)
File "/home/.anaconda/imported_make/utils.py", line 7, in make
test_print() # parent function usage example
NameError: name 'test_print' is not defined
I know import * is bad practice, but it would have been fine in this instance if it worked, and I'm not sure why it doesn't.

boss.py
class tasks():
def job1(input):
// do something
return output
def job2(input):
// do something
return output
worker.py
import boss.tasks
from boss.tasks import job1, job2
input_value = "xyz"
output1 = boss.tasks().job1(input_value)
output2 = boss.tasks().job2(input_value)

Python - ModuleNotFoundError when changing name of flask application to app.py. Possible solutions?

I have recently changed my main flask app file from inputServer.py to app.py to increase ease of use.
My code that throws the error is as follows:
def readBackupPlayers(objectFile):
with open(objectFile, 'rb') as openedFile:
manager.playerList = pickle.load(openedFile)
print('Backup of players retrieved')
^
Function that read a custom pickle file used for backing up a list stored inside a custom object
class PlayerManager:
def __init__(self):
self.playerList = []
self.ID = len(self.playerList) + 1
self.currentGame = None
self.tournament = None
manager = PlayerManager()
^
Code that declares the class and creates an instance of it to use to store variables across the program.
Error message:
File "c:\users\simon\appdata\local\programs\python\python38-32\lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\simon\OneDrive\Documents\AtomTesting\app.py", line 356, in setup
readBackupPlayers('Backups/playerBackup')
File "C:\Users\simon\OneDrive\Documents\AtomTesting\app.py", line 72, in readBackupPlayers
manager.playerList = pickle.load(openedFile)
ModuleNotFoundError: No module named 'inputServer'
Directory:
AtomTesting -
Backups -
playerBackup
tournamentBackup
Static -
static Flask files (not relevant)
templates -
Flask templates (not relevant)
app.py (renamed from inputServer.py)
config.py
README.md

Here's what's happening. Python's pickle module serializes and deserializes (dumps and loads) Python objects based on their name. Since the name of the module where your class lives changed, Pickle can't figure out how to re-instanciate those objects.
There is, however, a solution which is detailed on the Python wiki. It involves mapping old names to new names:
import pickle
renametable = {
'inputServer': 'app',
}
def mapname(name):
if name in renametable:
return renametable[name]
return name
def mapped_load_global(self):
module = mapname(self.readline()[:-1])
name = mapname(self.readline()[:-1])
klass = self.find_class(module, name)
self.append(klass)
def loads(str):
file = StringIO(str)
unpickler = pickle.Unpickler(file)
unpickler.dispatch[pickle.GLOBAL] = mapped_load_global
return unpickler.load()
Alternatively, and this is probably a better approach, you shouldn't use pickle for any serious data store, especially when something might be used between versions of your app.
You might consider using an ORM with a lightweight database, like SQLite.

Class super() unable to inherit "NameError: name 'in_dir' is not defined

I'm trying to pass an input directory path from a separate script into the main module "qc".
My first attempt at passing an argument from parent to child in the qc module looks like this:
import os
class Data:
def __init__(self, in_dir):
# get all inputs
self.in_dir = in_dir # get input directory
# get files from input directory
self.files = os.listdir(self.in_dir) # get file list
class LAS(Data):
"""Takes in log files from parent class Data's file list"""
def __init__(self):
# inherit the directory from Data parent
super().__init__(LAS, in_dir)
# target only the .las files in the directory
ext_las = ['.las', '.LAS', '.Las', '.LAs']
self.lasfiles = [lasfile for lasfile in self.files if any(match in lasfile for match in ext_las)]
def out_las(self):
return self.lasfiles
When I call the classes from my separate script it looks like this:
in_dir = 'C:\\......directory path...'
Data(in_dir=in_dir)
output = LAS().out_las()
Which raises the error:
File "C:\......", line 62, in __init__
super().__init__(LAS, in_dir)
NameError: name 'in_dir' is not defined
I can't figure out how to properly call this parent class' inputs.
What am I doing incorrectly?

Difficult to point you in the right direction, but it looks like you're expecting the child class to inherit from an instance of the parent class - but this is not the case.
The following may just work close to what you expect, hope it helps.
class LAS(Data):
"""Takes in log files from parent class Data's file list"""
def __init__(self, in_dir):
# inherit the directory from Data parent
super().__init__(in_dir)
# target only the .las files in the directory
ext_las = ['.las', '.LAS', '.Las', '.LAs']
self.lasfiles = [lasfile for lasfile in self.files if any(match in lasfile for match in ext_las)]
def out_las(self):
return self.lasfiles
las = LAS(in_dir='C:\\......directory path...')
output = las.out_las()

Namespace for class declaration in file executed at runtime

I wrote a program to allow users to give me a class name and the file that defines that class like so:
input = raw_input("Input the new classname with a space and then the file name defining the class")
class_name, file_name = input.split(" ")
I then execute the file defining the class:
execfile(file_name)
and try to create an instance of the newly defined class:
d[k] = eval(new_class)(**d[k])
However this creates the following error:
d[i] = eval(new_class)(**d[i])
File "<string>", line 1, in <module>
NameError: name 'MeasureDay' is not defined
This is after I fed in the following class definition that was executed:
class MeasureDay:
print("defining measureday")
def __init__(self, **kargs):
self.day = kwargs.get('day', -1)
self.measurement_value = kwargs('measurement_value', -1)
def __str__(self):
return "here is a custom defined string for MeasureDay"
I know the class definition was executed because I see the logged statement "defining measureday" as desired.
What am I doing wrong? I am guessing this has to do with some namespace convention. Do I need to define a namespace when I define classes like so, and if so how would I do it. Otherwise, what is the problem?
Thanks for any suggestions.

eval and execfile are to be avoided, and are unnecessary here. importlib.import_module can import the module file from a string, and you can then use getattr to get the class for instantiation.
class_name, file_name = input.split(" ")
mod = import_module(file_name)
cls = getattr(mod, class_name)
d[k] = cls(**d[k])

Python Help: Accessing static member variable from another class

I'll do my best to describe the issue I am having. I am building a Python program that is built on multiple classes and uses the unittest framework. In a nutshell, the Main.py file has a "ValidateDriver" class that defines a "driver" variable as an ElementTree type. If I point this directly to the XML file I need to parse, (i.e. driver = ElementTree.parse(rC:\test.xml)) then I can access it from another class. However, in reality I don't have the actual XML file that is passed in from the command-line until you get to the Main function in the ValidateDriver class. So under the ValidateDriver class driver would really be driver = ElementTree and then in the main function I would reassign that variable to ValidateDriver.driver = ElementTree.parse(args.driver). However, this is the crux. When I go to the other class and try to call ValidateDriver.driver I don't have the "findall" method/attribute available. Again, the only way it will work is to do something like: ElementTree.parse(rC:\test.xml)). If I did this in C# it would work, but I am new to Python and this is kicking my butt. Any help/suggestions is appreciated. I've included the code for both classes.
Main Function:
import sys
import argparse
import xml.etree.ElementTree as ElementTree
import unittest
import Tests.TestManufacturer
class ValidateDriver:
driver = ElementTree
def main(argv):
parser = argparse.ArgumentParser(description='Validation.')
parser.add_argument('-d', '--driver', help='Path and file name xml file', required=True)
parser.add_argument('-v', '--verbosity',
help='Verbosity for test output. 1 for terse, 2 for verbose. Default is verbose',
default=2, type=int)
#args = parser.parse_args()
args = r'C:\test.c4i'
#print ("Validate Driver: %s" % args.driver)
#print ("Verbosity Level: %s" % args.verbosity)
ValidateDriver.driver = ElementTree.parse(r'C:\test.c4i')
loader = unittest.TestLoader()
suite = loader.loadTestsFromModule(Tests.TestManufacturer)
runner = unittest.TextTestRunner(verbosity=2) # TODO Remove this...
# TODO Uncomment this...
runner = unittest.TextTestRunner(verbosity=args.verbosity)
result = runner.run(suite)
if __name__ == "__main__":
main(sys.argv[1:])
Other Class, Test Manufacturer:
import unittest
import Main
manufacturer = ['']
class Tests(unittest.TestCase):
# Test to see if Manufacturer exists.
def test_manufacturer_exists(self):
for m in Main.ValidateDriver.driver.findall('./manufacturer'):
print m.text
Producing the following error:
C:\Python27\python.exe C:\Users\test\PycharmProjects\Validator\Main.py
Traceback (most recent call last):
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 22, in <module>
class ValidateDriver:
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 65, in ValidateDriver
main(sys.argv[1:])
File "C:\Users\test\PycharmProjects\Validator\Main.py", line 36, in main
ValidateDriver.driver = ElementTree.parse(r'C:\test.c4i')
NameError: global name 'ValidateDriver' is not defined
Process finished with exit code 1

The main problem seems to be that your main script is wrapped in a class. There's really no reason for this, and is quite confusing.
if __name__ == "__main__":
main_object = ValidateDriver()
main_object.main(sys.argv[1:])
This should go outside the class definition

This has nothing to do with "findall" being available. The issue is that the class itself hasn't yet been completely declared at the time you try to access it. In python, the file is read top to bottom. For example, this is not allowed:
if __name__ == "__main__":
f()
def f():
...
The call to f must happen at the bottom of the file after it is declared.
What you're doing with ValidateDriver is similar, because the class isn't defined until the statements directly in its body are executed (this is different from functions, whose bodies of course aren't executed until they are called). You call main(sys.argv[1:]) inside the class body, which in turn tries to access ValidateDriver.driver, which doesn't exist yet.
Preferably, the main function, as well as the code which calls it, should be outside the class. As far as I can tell, the class doesn't need to exist at all (this isn't C# or Java -- you can put code directly at the module level without a class container). If you insist on putting it in a class as a static method, it must be defined as a class method:
#classmethod
def main(cls, argv):
...
which can then be called (outside the class definition) like:
ValidateDriver.main(sys.argv[1:])
But I stress that this is non-standard and should not be necessary.