I come from Java background and most of my thinking comes from there. Recently started learning Python. I have a case where I want to just create one connection to Redis and use it everywhere in the project. Here is how my structure and code looks.
module: state.domain_objects.py
class MyRedis():
global redis_instance
def __init__(self):
redis_instance = redis.Redis(host='localhost', port=6379, db=0)
print("Redus instance created", redis_instance)
#staticmethod
def get_instance():
return redis_instance
def save_to_redis(self, key, object_to_cache):
pickleObj = pickle.dumps(object_to_cache)
redis_instance.set(key, pickleObj)
def get_from_redis(self, key):
pickled_obj = redis_instance.get(key)
return pickle.loads(pickled_obj)
class ABC():
....
Now I want to use this from other modules.
module service.some_module.py
from state.domain_objects import MyRedis
from flask import Flask, request
#app.route('/chat/v1/', methods=['GET'])
def chat_service():
userid = request.args.get('id')
message_string = request.args.get('message')
message = Message(message_string, datetime.datetime.now())
r = MyRedis.get_instance()
user = r.get(userid)
if __name__ == '__main__':
global redis_instance
MyRedis()
app.run()
When I start the server, MyRedis() __init__ method gets called and the instance gets created which I have declared as global. Still when the service tries to access it when the service is called, it says NameError: name 'redis_instance' is not defined I am sure this is because I am trying to java-fy the approach but not sure how exactly to achieve it. I read about globals and my understanding of it is, it acts like single variable to the module and thus the way I have tried doing it. Please help me clear my confusion. Thanks!
Related
Suppose I have a module data_provider.py that keeps some code responsible for connecting to external API. In order to establish a connection, an API token needs to be retrieved from the database.
API_TOKEN = get_token_from_database()
class DataProvider:
def __init__(self):
self.api_token = API_TOKEN
def make_query(self):
'''make some request using self.api_token'''
Assuming the code needs to stay in such shape more or less (API_TOKEN being a global variable), what would be a good pattern for retrieving the API_TOKEN from the database?
Obiously now it is not ideal because the module cannot be imported without the database being turned on. Also, I would like to retrieve it only once, not per DataProvider creation.
Should I for example make API_TOKEN load lazily or turn it into a function?
If you have one single DataProvider object in your application, you could simply remove the global variable and load the token in the __init__ method:
class DataProvider:
def __init__(self):
self.api_token = get_token_from_database()
Another possible lazy loading would be to initialize the global token to None and then set it at first instanciation of a DataProvider object
API_TOKEN = None
class DataProvider:
def __init__(self):
global API_TOKEN
if API_TOKEN is None:
API_TOKEN = get_token_from_database()
self.api_token = API_TOKEN
Error processing still omitted for brievety...
I'd probably go with a simple singleton pattern
class DataProvider:
def __init__(self):
self.__api_token = None
def get_token(self):
if self.__api_token is None:
self.__api_token = .....
return self.__api_token
You might want to make get_token(self) into a property token instead. That's a matter of taste.
I have a problem with mongo. When I ended my automation tests, I need trash all data and object which I created. I create a script. In this script I delete a rows from a few table. But when I start this, This class doesn't start, where is my problem?
In consol I haven't any message, zero value.
from pymongo import MongoClient
class deleteMongo():
def deleteFirst(self):
client = MongoClient('databaseaddress')
db = client.TableData
db.Employe.delete_one({"name": "EmployeOne"})
def deleteSecond(self):
client = MongoClient('databaseaddress')
db = client.PersonData
db.Person.delete_one({"name": "PersonOne"})
def deleteThird(self):
client = MongoClient('databaseaddress')
db = client.AutoData
db.Auto.delete_one({"name": "AutoThird"})
If I am understanding your question correctly, you are trying to run the script above and it's not doing anything?
If this is your complete module, you are not calling the class at all, but defining the class object.
also the parenthesis in class deleteMongo(): are redundant in a class, since it always inherits the object. On the current setup of this class object, you should use def instead, or setup your class to initialize shared objects of the class.
Based on your current code, try this:
from pymongo import MongoClient
class deleteMongo:
def __init__(self, databaseAddress):
# if the databseAddress is always the same, you can remove it from the arguments
# and hardcode it here
self.client = MongoClient(databaseAddress)
def deleteFirst(self):
db = self.client.TableData
db.Employe.delete_one({"name": "EmployeOne"})
def deleteSecond(self):
db = self.client.PersonData
db.Person.delete_one({"name": "PersonOne"})
def deleteThird(self):
db = self.client.AutoData
db.Auto.delete_one({"name": "AutoThird"})
and then when you need to call one of the class functions, call it like this:
deleteMongo(databaseAddress='someaddress').deleteFirst()
I'm new to Python and tried similar suggestions from here and failed.
I'm writing a script that consists of few functions, the first function will create some of the variables that will be used in other functions (it can't global variables).
When I've tried my script I kept getting NameError for undefined vars.
import boto3
import json
from awsretry import AWSRetry
from botocore.exceptions import ClientError
#AWSRetry.backoff(tries=5)
def instance_details(event, context):
client = boto3.client('ec2')]
ec2_resource = boto3.resource('ec2')`
alert = event['Records'][0]['Sns']['Message']
instance_id = alert['Trigger']['Dimensions'][0]['value']
instance = ec2_resource.Instance(instance_id)
return client
#AWSRetry.backoff(tries=5)
def tagging():
instance_type = instance['Reservations'][0]['Instances'][0]['InstanceType']
Why I can't pass the values of instance and client to other functions?
Thanks in advance and sorry for duplicates.
intance_details i believe is lambda handler method. Since you are returing client I believe you should be able to see client value in the variable in which you will be capturing return of this method.
Apart from that, you can try to use Class here and declare these variables in __init__ method. Then create instance of that class in the lambda handler and access these variables. Then you would be able to use these variables in the whole class.
import boto3
class Answer:
def __init__(self):
self.instance = None
self.client = boto3.client('ec2')]
self.ec2_resource = boto3.resource('ec2')
def meth1(self):
# suppose here we want to use the value of instance
# using self.instance you can use the value of instance here
# you can pass the alert from lambda_handler to this method
# as well and do all the computation here too.
print(self.client) # example how to use class variables.
def lambda_handler(event, context):
ans = Answer()
alert = event['Records'][0]['Sns']['Message']
instance_id = alert['Trigger']['Dimensions'][0]['value']
ans.instance = ans.ec2_resource.Instance(instance_id)
# if you want to pass instance id, you can pass in the arguments and
# change the definition of meth1 accordingly.
# Apart form that you can pass the alert in the meth1 too and do all the computation there.
ans.meth1()
if __name__ == "__main__":
lambda_handler(event, "")
I am changing some code to spin up VMs in ec2 instead of openstack. Main starts a thread per VM, and then various modules perform tasks on these VM. Each thread controls it's own VM. So, instead of either having to add parameters to all of the downstream modules to look up information, or having to change all of the code to unpickle the class instance that created the vm, I am hoping that I can have the class itself decide whether to start a new VM or return the existing pickle. That way the majority of the code wont need to be altered.
This is the general idea, and closest I have gotten to getting it to work:
import os
import sys
import pickle
if sys.version_info >= (2, 7):
from threading import current_thread
else:
from threading import currentThread as current_thread
class testA:
def __init__(self, var="Foo"):
self.class_pickle_file = "%s.p" % current_thread().ident
if os.path.isfile(self.class_pickle_file):
self.load_self()
else:
self.var = var
pickle.dump(self, open(self.class_pickle_file, "wb"))
def test_method(self):
print self.var
def load_self(self):
return pickle.load(open(self.class_pickle_file, "rb"))
x = testA("Bar")
y = testA()
y.test_method()
But that results in: NameError: global name 'var' is not defined
But, If I do y = pickle.load(open("140355004004096.p", "rb")) it works just fine. So the data IS getting in there by storing self inside the class, it's a problem of getting the class to return the pickle instead of itself...
Any ideas? Thanks.
It looks to me like you create a file named by the current thread's ident, then you instantiate another TestA object using the same thread (!!same ident!!), so it checks for a pickle file (and finds it, that's bad), then self.var never gets set.
In test_method, you check for a variable that was never set.
Run each item in its own thread to get different idents, or ensure you set self.var no matter what.
the last few weaks, I am playing a little bit with the Web.py framework. As my application is now getting bigger and bigger, I want to restructure the sourcecode and put code fragments in different classes. Now, I don't really know where I should create my object instances if I need them in different web.py classes. Let us assume, my sourcecode looks like:
import web
import myclass
urls = (
'/', 'index',
'/test', 'test'
)
#should i make my instance global...
my = myclass.myClass()
class test:
def __init__(self):
#...or should i make my instance local: my = myclass.myClass()
pass
def GET(self):
item = my.getItem()
return item
def POST(self):
pass
class index:
def __init__(self):
#...or should i make my instance local: my = myclass.myClass()
pass
def GET(self):
date = my.getDate()
return date
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
Now, I want to access the methods getItem() and getDate() (which belong to the instance my), if the appropriate sites in my webbrowser are called. My question is now: Should I make the instance global or is it better, if I make it local? I really don't like global instances, but I don't see any other way as to make it global. Sure, it would be possible, to create a local instance, but then, every time the page loads, a new instance would be created, right? Normally, this wouldn't be a problem, but myclass accesses a serial port, so I need to make sure, that only one instance is created.
Am I missing something or is a global instance the only possible solution to accomplish this?
After some research, I came to the conclusion, that global instances are the way to go here. However, one must be careful with global instances if they are used together with the web.py auto reload mode. In auto reload mode, a global instance is created every time a new page loads. If you want to avoid that, you have to use something like this:
import web
import serial
urls = ("/(.*)", "test"
)
web.config.debug = True
if(web.config.debug):
#in debug mode, make sure that global serial instance is only created at start up
if not hasattr(serObj, "_email"):
serObj = serial.Serial(0, 9600, parity=serial.PARITY_NONE)
web._serObj = serObj
else:
serObj = web._serObj
class test:
def GET(self):
return "Test"
def POST(self):
pass
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()