Develop Reference

Importing a class with its dependencies

I am trying to import a class I have created from another file, into my current working file. The class is as follows in this example:
ClassFile
class example_class:
def __init__(self, ticker):
self.df = yf.download(ticker, start='2020-12-31')
def do_something(self):
return something
Then when I go to import it into my working file, and run the code:
WorkingFile
import ClassFile
import yfinance as yf
instance = example_class('MSFT')
I get the following error:
name 'yf' is not defined
I understand that yfinance is not being imported somewhere but, after researching, I have not been able to understand how to import it.
Essentially, I am trying to understand how to import a class from another file alongside the dependencies within said class.

You need to import the symbol where you use it; in ClassFile.
import yahoofinance as yf
class example_class:
def __init__(self, ticker):
self.df = yf.download(ticker, start='2020-12-31')
def do_something(self):
return something
When you import something, it is imported into the current scope, i.e. the current file of code.
In theory, if you left import ... yf in WorkingFile.py, you could say
import WorkingFile
...
self.df = WorkingFile.yf.download(ticker, start='2020-12-31')
though of course in this case, that would create a circular import; and anyway, you don't want to do this - it is completely insane. The entire job of import is to bring in something where you actually need it.
The other way around is how you are supposed to use it; the user of ClassFile should not need to know or care that the implementation uses yfinance internally. A proper class should encapsulate its internals from its users, and only expose functionality that they directly need.
In some scenarios, you need to compromise; for example, if ClassFile is a really thin wrapper around yfinance (let's say it inherits from it and adds a new method sell_MSFT but otherwise works identically) then of course the user needs to be familiar with yfinance anyway, and you can defer to its documentation for most of the functionality. But still, in that case, the import yfinance needs to go where it is actually used; in ClassFile.

You either specifically import the class if you want to use its name without the file classifier (ClassFile) or use the Classifier with the function name. You do NOT have to import the depenency modules. ie
import ClassFile
instance = ClassFile.example_class('MSFT')
or:
from ClassFile import example_class
instance = example_class('MSFT')
I assume yf is imported or declared somehwere in ClassFile. That is put you r yf import statement in ClassFile

Access parent object from submodule

I'm writing a module for accessing some data from a local SQLite Database, and would like the end-user functionality to work like this:
import pyMyDB
myDBobj = pyMyDB.MyDB( '/path/to/DB/file' ) # Instance of the class MyDB
User has simply created an object that connects to a database.
Then, depending on the need, I want to have different submodules the user can access to work on the data from the database. For example, Driving provides certain functions while Walking provides others, which I'd love to be accessed like so:
result = myDBobj.Driving.GetTachometerReading()
or
result = myDBobj.Walking.GetShoeType()
My question is about how to make the submodules Driving & Walking have access to the parent Object (not just the Parent module's class/functions, but the instantiated object).
If I make my file heirarchy something like this:
pyMyDB/
__init__.py # imports the below modules as so:
MyDB_class.py # imported as *, contains MyDB class definition
Driving.py # imported normally via `import Driving`
Walking.py # imported normally
where __init__.py imports MyDB_class.py (which contains the Database class) and also imports Driving/Walking.py, then
the user can do pyMyDB.Driving.some_func(), but some_func() won't actually have access to an instantiated MyDB object, right?
Has anyone come across a way to have the sub-modules access the instantiated object?
Here is my current solution. It seems overly complicated.
First, I have to use an additional globals.py file to overcome circular module imports (Like this hint).
In the Driving module I make a new class called Drv, which is used to make a duplicate of the MyDBobj, but also has the extra methods I want, such as Driving.GetTachometerReading().
Driving.py:
class DrvClass( MyDB ):
def __init__(self, MyDBobj):
self.attribute1 = MyDBobj.attr1
self.attribute2 = MyDBobj.attr2
.... # copy all attribute values!
def GetTachometerReading(self):
.... #some function here
Then, to accomplish the sub-indexing (MyDBobj.Driving.GetTach()), from within the Driving.py module, I add a Driving() method to the MyDB class via
def temp_driving(self):
return DrvClass( self ) # pass the MyDBobj
MyDB.Driving = temp_driving # add the above method to the MyDB class
So now a user can do: MyDBobj.Driving().GetTachometerReading(), where MyDBobj.Driving() returned the new DrvClass object that has the new GetTachometer() function. I don't like the fact that I must call Driving() as a function.
What do you think - is there a better/simpler way?
Btw the reason I want the MyDB class to be separate is because our access methods may change, without changing the analysis functions (Driving/Walking), or vice-versa. Thus I don't want to just add the MyDB access techniques directly to separate Driving & Walking modules.
Thanks in advance for your sage advice!

I think I might use a different access approach on the client side. If the clients used a protocol like this:
result = myDBobj.Driving_GetTachometerReading()
then it's straightforward to add this name into the class in the sub-module Driving.py:
import pyMyDB
def GetTachometerReading(self):
# self will be the MyDB instance
pyMyDB.MyDB.Driving_GetTachometerReading = GetTachometerReading
However, if you are set on your approach then I think it could be improved.As it is, you create a new instance of Driving for every call to a Driving function which is not good. However, we do need to create an instance when the MyDB itself is instantiated. Yet that seems tricky because at that time we don't seem to know which sub-modules to include. I have fixed that by having each sub-module inject a list of its methods into MyDB. Like this:
pyMyDB:
import functools
class MyDB(object):
submodule_methods = {}
def __init__(self):
for k,v in self.submodule_methods.items():
self.__dict__[k] = SubmodulePointer(self,v)
self.db_stuff = "test"
class SubmodulePointer(object):
def __init__(self,db,methods):
self.db = db
for name,func in methods.items():
self.__dict__[name] = functools.partial(func,db)
Then for each of the sub-modules, eg Driving we do:
import pyMyDB
def GetTachometerReading(self):
# self here is the db instance, as if this was a method on the db
print(self.db_stuff)
pyMyDB.MyDB.submodule_methods["Driving"] = {"GetTachometerReading":GetTachometerReading}
Then the client can just do:
import pyMyDB
import Driving
m = pyMyDB.MyDB()
m.Driving.GetTachometerReading()
As you wanted. A simpler approach, with a slightly more conventional style would be to make Driving a class of its own:
pyMyDB:
import functools
class MyDB(object):
submodule_classes = []
def __init__(self):
for c in self.submodule_classes:
self.__dict__[c.__name__] = c(self)
self.db_stuff = "test"
Then for each of the sub-modules, eg Driving we do:
import pyMyDB
class Driving(object):
def __init__(self,db):
self.db = db
def GetTachometerReading(self):
# self here is the Driving instance,
# but we have access to the db
print(self.db.db_stuff)
pyMyDB.MyDB.submodule_classes.append(Driving)
This means that the methods in the Driving class don't look like methods on MyDB but that could be an advantage. I'm not sure if that was something you wanted.

Python Multiple file program

For larger programs, in order to be more organized, I have been looking into dividing my code up into different .py files and having the main file that calls upon those files when needed. I have looked around and seen lots of remarks about creating a directory and a SystemPath for Python. Are those reasonable options for a program that could be distributed between a few computers? As a test, I tried to assemble an example:
This is the class named grades in the same directory as main
class student:
def __init__(self):
self.name = ""
self.score = 0
self.grade = 0
def update(self,name,score,grade):
self.score = score
self.name = name
self.grade = grade
print self.score,self.name,self.grade
s = student()
s.update(name,score,grade)
This is my main script currently:
from grades import score
import random
name = 'carl'
score = random.randrange(0,100)
grade = 11
s = student()
s.score(name,score,grade)
There are some questions I have generally about this method:
Is there a way to import all from different files or do I need to specify each individual class?
If I just had a function, is it possible to import it just as a function or can you only import via a class?
Why is it when I call upon a class, in general, I have to make a variable for it as in the example below?
# way that works
s = student()
s.update(name,score,grade)
# incorrect way
student.update(name,score,grade)
Thank you for your time and thought towards my question.

Yes.
You can import instance of student from other script to main script like this:
from grades import s
# if your first script is called grades.py
import random
name = 'carl'
score = random.randrange(0,100)
grade = 11
# you can directly use it without initializing it again.
s.score(name,score,grade)
2.
If you have a function called test() in grades.py, you can import it in this way:
from grades import test
# then invoke it
test()
3.
This variable stands for the instance of class student. You need this instance to invoke the function inside.

Generally, to divide the source code of a program, Python use module to do that, which corresponds to a *.py file. Then for your 3 questions:
You can import a whole "module content" (function, class, global variables, ...) through import module_name.*
for a function, if it is a function in a class(member method, class method or static method) you can not only import the function, you should import class to use the method; if it is a function under module, you can separately import the function through import module_name.function_name
update is a member function of the student class, so you should use it through an instance. if it is a class method or static method, you can use it through the class name you wrote.

1: Is there a way to import all from different file or do i need to
specify each individual class?
You can use the "wildcard import", but you probably shouldn't. See
Should wildcard import be avoided?
If i just had a function, is it possible to import it just as a
function or can you only import via a class?
Functions can be totally independent of classes in Python.
3.Why is it when i call upon a class in general i have to make a variable for it as in the example below?
You should read up on object-oriented programming. In the basic cases, you have to instantiate instances of a class in order to use that class's functionality. In your example, the class student describes what it means to be a student, and the statement
s = student()
creates a student and names it "s".
I think this should be clear after reading a bit about object-oriented programming.

First, you can use from module import * to import everything like:
hello.py:
def hello():
print 'hello'
def bye():
print 'Bye'
main.py:
from hello import *
hello()
bye()
But it's not a good way, if you have two files, two functions have the same name,
so use
from hello import hello, bye
hello()
bye()
is better, it an example for function ,as same as class.
Third before Second, student is a class, so you have to use an instance object to use the function which with self parameter. If you want to use student.function, the function must be a static method like this:
class Person:
def __init__():
pass
#staticmethod
def Count():
return 1
print Person.Count()
Second, you can import the function in a class file which is independent of the class.

Is there a way to import all from different file or do i need to specify each individual class?
the answer is yes , as python import statement use sys.path (A list of strings that specifies the search path for modules ) you need to add the patht of your modules in sys.path , for example if you want to interact between different computers you can put your modules in public folder and add the path of folder to sys.path :
import sys
sys.path.append( path to public )
If i just had a function, is it possible to import it just as a function or can you only import via a class?
you just need to use from ... import function_name.
Why is it when i call upon a class in general i have to make a variable for it as in the example below?
for this question you just need to read the python Class objects documentation :
Class objects support two kinds of operations: attribute references and instantiation.

What could be a reason why a class can't be accessed from within another class's function?

I have 2 classes:
import follow
class User(object):
def __init__(self):
pass
import user
class Follow(object):
def doSomething(self):
u = User()
>> f = Follow()
>> f.doSomething()
>> NameError: global name 'User' is not defined
Is this causing a problem because both classes import each other? Is there the equivalent of an import once? Any other potential causes? Thanks.

u = user.User()
or, alternatively:
from user import User
u = User()

user.py
class User(object):
def __init__(self):
pass
follow.py
from user import User
class Follow(object):
def doSomething(self):
u = User()

The problem seems to be that you're are importing user and then accessing User. Use user.User or (not as nice) from user import User.
Python treats modules as namespaces and so anything defined in the global scope of a module is only available as an attribute of the module in other namespaces that import it. There is no true 'global scope' in Python the way there is in some other languages.
The from foo import bar syntax allows one to bring an identifier directly into whatever scope it is executed in but later attempts to reload the module will no longer update that reference. There is also the problem of keeping track of where identifiers come from.
Also, you don't actually seem to be using follow in the user module. This shouldn't be a problem but if you are actually using it, then you should probably extend your example to include the use. How modules import each other can determine if this is actually allowed or not.

You can try to use either
from user import *
(which is usually not recommended)
or create User objects using qualified name:
...
u = user.User()
Correct import
from user import User
doesn't work in your case, I guess, because of recursive imports.

There will always be a problem no matter.
If this a theoretical question then fine.
But if this an actual design then stop. following is an action. It should be a function in the class User to make any sence.
class User(object):
def __init__(self):
print "bleh"
def follow(self,otheruser)
following.append(otheruser)
this makes much more sense. following obviously is a list pulled from somewhere either a variable or directly from the database.

How do I extend a python module? Adding new functionality to the `python-twitter` package

What are the best practices for extending an existing Python module – in this case, I want to extend the python-twitter package by adding new methods to the base API class.
I've looked at tweepy, and I like that as well; I just find python-twitter easier to understand and extend with the functionality I want.
I have the methods written already – I'm trying to figure out the most Pythonic and least disruptive way to add them into the python-twitter package module, without changing this modules’ core.

A few ways.
The easy way:
Don't extend the module, extend the classes.
exttwitter.py
import twitter
class Api(twitter.Api):
pass
# override/add any functions here.
Downside : Every class in twitter must be in exttwitter.py, even if it's just a stub (as above)
A harder (possibly un-pythonic) way:
Import * from python-twitter into a module that you then extend.
For instance :
basemodule.py
class Ball():
def __init__(self,a):
self.a=a
def __repr__(self):
return "Ball(%s)" % self.a
def makeBall(a):
return Ball(a)
def override():
print "OVERRIDE ONE"
def dontoverride():
print "THIS WILL BE PRESERVED"
extmodule.py
from basemodule import *
import basemodule
def makeBalls(a,b):
foo = makeBall(a)
bar = makeBall(b)
print foo,bar
def override():
print "OVERRIDE TWO"
def dontoverride():
basemodule.dontoverride()
print "THIS WAS PRESERVED"
runscript.py
import extmodule
#code is in extended module
print extmodule.makeBalls(1,2)
#returns Ball(1) Ball(2)
#code is in base module
print extmodule.makeBall(1)
#returns Ball(1)
#function from extended module overwrites base module
extmodule.override()
#returns OVERRIDE TWO
#function from extended module calls base module first
extmodule.dontoverride()
#returns THIS WILL BE PRESERVED\nTHIS WAS PRESERVED
I'm not sure if the double import in extmodule.py is pythonic - you could remove it, but then you don't handle the usecase of wanting to extend a function that was in the namespace of basemodule.
As far as extended classes, just create a new API(basemodule.API) class to extend the Twitter API module.

Don't add them to the module. Subclass the classes you want to extend and use your subclasses in your own module, not changing the original stuff at all.

Here’s how you can directly manipulate the module list at runtime – spoiler alert: you get the module type from types module:
from __future__ import print_function
import sys
import types
import typing as tx
def modulize(namespace: tx.Dict[str, tx.Any],
modulename: str,
moduledocs: tx.Optional[str] = None) -> types.ModuleType:
""" Convert a dictionary mapping into a legit Python module """
# Create a new module with a trivially namespaced name:
namespacedname: str = f'__dynamic_modules__.{modulename}'
module = types.ModuleType(namespacedname, moduledocs)
module.__dict__.update(namespace)
# Inspect the new module:
name: str = module.__name__
doc: tx.Optional[str] = module.__doc__
contents: str = ", ".join(sorted(module.__dict__.keys()))
print(f"Module name: {name}")
print(f"Module contents: {contents}")
if doc:
print(f"Module docstring: {doc}")
# Add to sys.modules, as per import machinery:
sys.modules.update({ modulename : module })
# Return the new module instance:
return module
… you could then use such a function like so:
ns = {
'func' : lambda: print("Yo Dogg"), # these can also be normal non-lambda funcs
'otherfunc' : lambda string=None: print(string or 'no dogg.'),
'__all__' : ('func', 'otherfunc'),
'__dir__' : lambda: ['func', 'otherfunc'] # usually this’d reference __all__
}
modulize(ns, 'wat', "WHAT THE HELL PEOPLE")
import wat
# Call module functions:
wat.func()
wat.otherfunc("Oh, Dogg!")
# Inspect module:
contents = ", ".join(sorted(wat.__dict__.keys()))
print(f"Imported module name: {wat.__name__}")
print(f"Imported module contents: {contents}")
print(f"Imported module docstring: {wat.__doc__}")
… You could also create your own module subclass, by specifying types.ModuleType as the ancestor of your newly declared class, of course; I have never personally found this necessary to do.
(Also, you don’t have to get the module type from the types module – you can always just do something like ModuleType = type(os) after importing os – I specifically pointed out this one source of the type because it is non-obvious; unlike many of its other builtin types, Python doesn’t offer up access to the module type in the global namespace.)
The real action is in the sys.modules dict, where (if you are appropriately intrepid) you can replace existing modules as well as adding your new ones.

Say you have an older module called mod that you use like this:
import mod
obj = mod.Object()
obj.method()
mod.function()
# and so on...
And you want to extend it, without replacing it for your users. Easily done. You can give your new module a different name, newmod.py or place it by same name at a deeper path and keep the same name, e.g. /path/to/mod.py. Then your users can import it in either of these ways:
import newmod as mod # e.g. import unittest2 as unittest idiom from Python 2.6
or
from path.to import mod # useful in a large code-base
In your module, you'll want to make all the old names available:
from mod import *
or explicitly name every name you import:
from mod import Object, function, name2, name3, name4, name5, name6, name7, name8, name9, name10, name11, name12, name13, name14, name15, name16, name17, name18, name19, name20, name21, name22, name23, name24, name25, name26, name27, name28, name29, name30, name31, name32, name33, name34, name35, name36, name37, name38, name39
I think the import * will be more maintainable for this use-case - if the base module expands functionality, you'll seamlessly keep up (though you might shade new objects with the same name).
If the mod you are extending has a decent __all__, it will restrict the names imported.
You should also declare an __all__ and extend it with the extended module's __all__.
import mod
__all__ = ['NewObject', 'newfunction']
__all__ += mod.__all__
# if it doesn't have an __all__, maybe it's not good enough to extend
# but it could be relying on the convention of import * not importing
# names prefixed with underscores, (_like _this)
Then extend the objects and functionality as you normally would.
class NewObject(object):
def newmethod(self):
"""this method extends Object"""
def newfunction():
"""this function builds on mod's functionality"""
If the new objects provide functionality you intend to replace (or perhaps you are backporting the new functionality into an older code base) you can overwrite the names

May I suggest not to reinvent the Wheel here? I'm building a >6k line Twitter Client for 2 month now, at first I checked python-twitter too, but it's lagging a lot behind the recent API changes,, Development doesn't seem to be that active either, also there was(at least when I last checked) no support for OAuth/xAuth).
So after searching around a bit more I discovered tweepy:
http://github.com/joshthecoder/tweepy
Pros: Active development, OAauth/xAuth and up to date with the API.
Chances are high that what you need is already in there.
So I suggest going with that, it's working for me, the only thing I had to add was xAuth(that got merge back to tweepy :)
Oh an a shameless plug, if you need to parse Tweets and/or format them to HTML use my python version of the twitter-text-* libraries:
http://github.com/BonsaiDen/twitter-text-python
This thing is unittestetd an guaranteed to parse Tweets just like Twitter.com does it.

Define a new class, and instead of inherit it from the class you want to extend from the original module, add an instance of the original class as an attribute to your new class.
And here comes the trick: intercept all non-existing method calls on your new class and try to call it on the instance of the old class.
In your NewClass just define new or overridden methods as you like:
import originalmodule
class NewClass:
def __init__(self, *args, **kwargs):
self.old_class_instance = originalmodule.create_oldclass_instance(*args, **kwargs)
def __getattr__(self, methodname):
"""This is a wrapper for the original OldClass class.
If the called method is not part of this NewClass class,
the call will be intercepted and replaced by the method
in the original OldClass instance.
"""
def wrapper(*args, **kwargs):
return getattr(self.old_class_instance, methodname)(*args, **kwargs)
return wrapper
def new_method(self, arg1):
"""Does stuff with the OldClass instance"""
thing = self.old_class_instance.get_somelist(arg1)
# returns the first element only
return thing[0]
def overridden_method(self):
"""Overrides an existing method, if OldClass has a method with the same name"""
print("This message is coming from the NewClass and not from the OldClass")
In my case I used this solution when simple inheritance from the old class was not possible, because an instance had to be created not by its constructor, but with an init script from an other class/module. (It is the originalmodule.create_oldclass_instance in the example above.)