I am a beginner Python user and I have come across an output to a function that I don't understand. I can't give all the code because some of it is IP at my company.
I am basically using a library written by one of our devs to pull a metric from out data warehouse. I want to then use this metric value in another application to when i get the value i will pass it to my own DB.
My issue is I dont understand the output of the function I am using to actually extrapolate the value I want.
If someone with more Python experience could tell me what the return of the function is doing as the best I can tell it is building a dict, but I don't fully understand how and where. I must add this is the function from inside the lib
def get(self, **kwargs):
if 'SchemaName' not in kwargs:
kwargs['SchemaName'] = self.find_schema_by_params(**kwargs)
if 'Stat' in kwargs and kwargs['Stat'] not in MWS.VALID_Stat:
raise MWSException("Incorrect Stat value: %s" % kwargs['Stat'])
if 'Period' in kwargs and kwargs['Period'] not in MWS.VALID_Period:
raise MWSException("Incorrect Period value: %s" % kwargs['Period'])
self._validate_schema(kwargs, MWS.DEFAULT_GET_PARAMETERS)
self._encode_start_time(kwargs)
if 'EndTime' not in kwargs:
if kwargs['StartTime'].startswith('-P'):
kwargs['EndTime'] = '-P00D'
else:
kwargs['EndTime'] = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.000Z")
return self._mws_action('GetMetricData', **kwargs)['StatisticSeries']
Apparently, _mws_action() is a method that is passed a string, 'GetMetricData' and the same keyword arguments as your get method (with a few modifications). _mws_action() returns a dictionary, and you return the 'StatisticSeries' element of that dictionary.
**kwargs converts a dictionary to/from keyword arguments. So you can call get as
get(SchemaName='schema', Stat='somestat', EndTime="-P00D")
and kwargs will be:
{'SchemaName': 'schema', 'Stat':'somestat', 'EndTime':"-P00D"}
Related
So I'm currently trying to create a dynamic function, that will take 1 kwarg, and use that in it's filter_by.
This is what the function currently looks like:
def Get_from_var(session, **data):
for entry in data:
if bool(session.query(Tutor).filter_by(entry=data[entry]).first()):
return session.query(Tutor).filter_by(entry=data[entry]).first()
return "No tutor found"
I know that my function will only recieve 1 kwarg, so I'm not worried about some weird exceptions. But currently it seems that the 'entry' paramter in both of the filter_by functions, are interpreted literally, and not as variables? Because It tells me that "tutor has no property 'entry'"
This is how I'm currently calling it:
tutor = Tutor.Get_from_var(session, tutor_id=filters["id"])
You can use **data in filter_by directly to get the correct result.
session.query(Tutor).filter_by(**data).first()
** unpacks your data into key-value pairs and passes them to the filter_by function.
Also, you probably dont want to query twice just to check your if statement.
I am new to python and came across a problem.
I have to get a dictionary from the user via a function that has few elements (key-value). At the same time, if the user is not providing the dictionary, they can provide individual elements as arguments.
How do I check if user has provided dictA, if not, src msg. If the user doesn't provides either one of them, return an error back to the calling function.
Lets say:
def myfunc(dictA, src, msg, e=True)
where dictA is the dictonary and src, msg are arguements if the user didn't provide dictA.
I wouldn't support them passing in dictA or the other arguments. I would just have them pass in dictA as kwargs if that want to use that. Then you can just check if the values have been there
New function prototype might be
def myfunc(src, msg, e=True)
And then usages could be
myfunc(a_source, a_msg)
Or
myfunc(**dictA)
This way your function will automatically check that those values are at least present. And any additional value checking can be done on only one input
For example the following call would fail (without you needing to do anything else) because it would still be missing the positional argument of msg.
myfunc(**{'src': 'a_source'})
Similarly it will fail if you send too many positional arguments as well (i.e. passing in a positional argument and a kwarg via a dictionary)
I recently had to use a function conditionally dispatching tasks to other functions, with a lot of mandatory and optional named arguments (e.g. manipulating connection strings, spark connectors configs and so on), and it occurred to me that It would have been really much "cleaner" (or "pythonesque") to have a syntax allowing me to pass every arguments from a function to another similar to this :
def sisterFunction(**kwargs) : # Doing things with a bunch of mandatory and optional args
<do various things/>
def motherFunction(a,b,**kwargs) :
<do various things/>
sisterFunction(**allArgs)
where allArgs would be a dictionary containing keys a,b, and everything in kwargs. This sounds like something python would be inclined to allow and ease but I can't seem to find something similar to a "super kwargs" implemented. Is there a straightforward way to do this ? Is there an obvious good reason it's not a thing ?
def sisterFunction(**kwargs):
pass
def motherFunction(a, b, **kwargs):
sisterFunction(a=a, b=b, **kwargs)
kwargs in sisterFunction will contain a and b keys with corresponding values.
UPDATE
If you don't want to pass long list of function parameters via a=a, there is some workaround to get allArgs:
def motherFunction(a, b, **kwargs):
allArgs = locals().copy()
allArgs.update(allArgs.pop('kwargs', {}))
sisterFunction(**allArgs)
I would probably go with just using kwargs
def sisterFunction(**kwargs):
pass
def motherFunction(**kwargs):
# use the values directly from 'kwargs'
print(kwargs['a'])
# or assign them to local variables for this function
b = kwargs['b']
sisterFunction(**kwargs)
This will probably be the option with the least code in your function signatures (the definitions of all the parameters to the function).
A KeyError will be raised if some parameters were not passed to the function and the function tries to use them.
(Python 3)
First of all, I feel my title isn't quite what it should be, so if you stick through the question and come up with a better title, please feel free to edit it.
I have recently learned about Python Decorators and Python Annotations, and so I wrote two little functions to test what I have recently learned.
One of them, called wraps is supposed to mimic the behaviour of the functools wraps, while the other, called ensure_types is supposed to check, for a given function and through its annotations, if the arguments passed to some function are the correct ones.
This is the code I have for those functions:
def wraps(original_func):
"""Update the decorated function with some important attributes from the
one that was decorated so as not to lose good information"""
def update_attrs(new_func):
# Update the __annotations__
for key, value in original_func.__annotations__.items():
new_func.__annotations__[key] = value
# Update the __dict__
for key, value in original_func.__dict__.items():
new_func.__dict__[key] = value
# Copy the __name__
new_func.__name__ = original_func.__name__
# Copy the docstring (__doc__)
new_func.__doc__ = original_func.__doc__
return new_func
return update_attrs # return the decorator
def ensure_types(f):
"""Uses f.__annotations__ to check the expected types for the function's
arguments. Raises a TypeError if there is no match.
If an argument has no annotation, object is returned and so, regardless of
the argument passed, isinstance(arg, object) evaluates to True"""
#wraps(f) # say that test_types is wrapping f
def test_types(*args, **kwargs):
# Loop through the positional args, get their name and check the type
for i in range(len(args)):
# function.__code__.co_varnames is a tuple with the names of the
##arguments in the order they are in the function def statement
var_name = f.__code__.co_varnames[i]
if not(isinstance(args[i], f.__annotations__.get(var_name, object))):
raise TypeError("Bad type for function argument named '{}'".format(var_name))
# Loop through the named args, get their value and check the type
for key in kwargs.keys():
if not(isinstance(kwargs[key], f.__annotations__.get(key, object))):
raise TypeError("Bad type for function argument named '{}'".format(key))
return f(*args, **kwargs)
return test_types
Supposedly, everything is alright until now. Both the wraps and the ensure_types are supposed to be used as decorators. The problem comes when I defined a third decorator, debug_dec that is supposed to print to the console when a function is called and its arguments. The function:
def debug_dec(f):
"""Does some annoying printing for debugging purposes"""
#wraps(f)
def profiler(*args, **kwargs):
print("{} function called:".format(f.__name__))
print("\tArgs: {}".format(args))
print("\tKwargs: {}".format(kwargs))
return f(*args, **kwargs)
return profiler
That also works cooly. The problem comes when I try to use debug_dec and ensure_types at the same time.
#ensure_types
#debug_dec
def testing(x: str, y: str = "lol"):
print(x)
print(y)
testing("hahaha", 3) # raises no TypeError as expected
But if I change the order with which the decorators are called, it works just fine.
Can someone please help me understand what is going wrong, and if is there any way of solving the problem besides swapping those two lines?
EDIT
If I add the lines:
print(testing.__annotations__)
print(testing.__code__.co_varnames)
The output is as follows:
#{'y': <class 'str'>, 'x': <class 'str'>}
#('args', 'kwargs', 'i', 'var_name', 'key')
Although wraps maintains the annotations, it doesn't maintain the function signature. You see this when you print out the co_varnames. Since ensure_types does its checking by comparing the names of the arguments with the names in the annotation dict, it fails to match them up, because the wrapped function has no arguments named x and y (it just accepts generic *args and **kwargs).
You could try using the decorator module, which lets you write decorators that act like functools.wrap but also preserve the function signature (including annotations).
There is probably also a way to make it work "manually", but it would be a bit of a pain. Basically what you would have to do is have wraps store the original functions argspec (the names of its arguments), then have ensure_dict use this stored argspec instead of the wrapper's argspec in checking the types. Essentially your decorators would pass the argspec in parallel with the wrapped functions. However, using decorator is probably easier.
I am writing a script at the moment that will grab certain information from HTML using dom4j.
Since Python/Jython does not have a native switch statement I decided to use a whole bunch of if statements that call the appropriate method, like below:
if type == 'extractTitle':
extractTitle(dom)
if type == 'extractMetaTags':
extractMetaTags(dom)
I will be adding more depending on what information I want to extract from the HTML and thought about taking the dictionary approach which I found elsewhere on this site, example below:
{
'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags
}[type](dom)
I know that each time I run the script the dictionary will be built, but at the same time if I were to use the if statements the script would have to check through all of them until it hits the correct one. What I am really wondering, which one performs better or is generally better practice to use?
Update: #Brian - Thanks for the great reply. I have a question, if any of the extract methods require more than one object, e.g.
handle_extractTag(self, dom, anotherObject)
# Do something
How would you make the appropriate changes to the handle method to implemented this? Hope you know what I mean :)
Cheers
To avoid specifying the tag and handler in the dict, you could just use a handler class with methods named to match the type. Eg
class MyHandler(object):
def handle_extractTitle(self, dom):
# do something
def handle_extractMetaTags(self, dom):
# do something
def handle(self, type, dom):
func = getattr(self, 'handle_%s' % type, None)
if func is None:
raise Exception("No handler for type %r" % type)
return func(dom)
Usage:
handler = MyHandler()
handler.handle('extractTitle', dom)
Update:
When you have multiple arguments, just change the handle function to take those arguments and pass them through to the function. If you want to make it more generic (so you don't have to change both the handler functions and the handle method when you change the argument signature), you can use the *args and **kwargs syntax to pass through all received arguments. The handle method then becomes:
def handle(self, type, *args, **kwargs):
func = getattr(self, 'handle_%s' % type, None)
if func is None:
raise Exception("No handler for type %r" % type)
return func(*args, **kwargs)
With your code you're running your functions all get called.
handlers = {
'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags
}
handlers[type](dom)
Would work like your original if code.
It depends on how many if statements we're talking about; if it's a very small number, then it will be more efficient than using a dictionary.
However, as always, I strongly advice you to do whatever makes your code look cleaner until experience and profiling tell you that a specific block of code needs to be optimized.
Your use of the dictionary is not quite correct. In your implementation, all methods will be called and all the useless one discarded. What is usually done is more something like:
switch_dict = {'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags}
switch_dict[type](dom)
And that way is facter and more extensible if you have a large (or variable) number of items.
The efficiency question is barely relevant. The dictionary lookup is done with a simple hashing technique, the if-statements have to be evaluated one at a time. Dictionaries tend to be quicker.
I suggest that you actually have polymorphic objects that do extractions from the DOM.
It's not clear how type gets set, but it sure looks like it might be a family of related objects, not a simple string.
class ExtractTitle( object ):
def process( dom ):
return something
class ExtractMetaTags( object ):
def process( dom ):
return something
Instead of setting type="extractTitle", you'd do this.
type= ExtractTitle() # or ExtractMetaTags() or ExtractWhatever()
type.process( dom )
Then, you wouldn't be building this particular dictionary or if-statement.