Develop Reference

Override Standard Assert Messaging in Pytest Assert

I'm using Pytest to test some SQL queries my team runs programmatically over time.
My SQL queries are lists of JSONs - one JSON corresponds to one row of data.
I've got a function that diffs the JSON key:value pairs so that we can point to exactly which values are different for a given row. Ideally, I'd output a list of these diffs instead of the standard output of an assert statement, which ends up looking clunky and not-very-useful for the end user.

You can use Python built-in capability to show custom exception message:
assert response.status_code == 200, "My custom message: actual status code {}".format(response.status_code)
Check it out: https://wiki.python.org/moin/UsingAssertionsEffectively

Pytest give us the hook pytest_assertrepr_compare to add a custom explanation about why an assertion failed.
You can create a class to wrap the JSON string and implement your comparator algorithm overloading the equal operator.
class JSONComparator:
def __init__(self, lst):
self.value = value
def __eq__(self, other):
# Here your algorithm to compare two JSON strings
...
# If they are different, save that information
# We will need it later
self.diff = "..."
return True
# Put the hook in conftest.py or import it in order to make pytest aware of the hook.
def pytest_assertrepr_compare(config, op, left, right):
if isinstance(left, JSONComparator) and op == "==":
# Return the diff inside an array.
return [left.diff]
# Create a reference as an alias if you want
compare = JSONComparator
Usage
def test_somethig():
original = '{"cartoon": "bugs"}'
expected = '{"cartoon": "bugs"}'
assert compare(original) == expected

list of functions Python

I have a list of patterns:
patterns_trees = [response.css("#Header").xpath("//a/img/#src"),
response.css("#HEADER").xpath("//a/img/#src"),
response.xpath("//header//a/img/#src"),
response.xpath("//a[#href='"+response.url+'/'+"']/img/#src"),
response.xpath("//a[#href='/']/img/#src")
]
After I traverse it and find the right pattern I have to send the pattern as an argument to a callback function
for pattern_tree in patterns_trees:
...
pattern_response = scrapy.Request(...,..., meta={"pattern_tree": pattern_tree.extract_first()})
By doing this I get the value of the regex not the pattern
THINGS I TRIED:
I tried isolating the patterns in a separate class but still I have the problem that I can not store them as pattern but as values.
I tried to save them as strings and maybe I can make it work but
What is the most efficient way of storing list of functions
UPDATE: Possible solution but too hardcoded and it's too problematic when I want to add more patterns:
def patter_0(response):
response.css("#Header").xpath("//a/img/#src")
def patter_1(response):
response.css("#HEADER").xpath("//a/img/#src")
.....
class patternTrees:
patterns = [patter_0,...,patter_n]
def length_patterns(self):
return len(patterns)

If you're willing to consider reformatting your list of operations, then this is a somewhat neat solution. I've changed the list of operations to a list of tuples. Each tuple contains (a ref to) the appropriate function, and another tuple consisting of arguments.
It's fairly easy to add new operations to the list: just specify what function to use, and the appropriate arguments.
If you want to use the result from one operation as an argument in the next: You will have to return the value from execute() and process it in the for loop.
I've replaced the calls to response with prints() so that you can test it easily.
def response_css_ARG_xpath_ARG(args):
return "response.css(\"%s\").xpath(\"%s\")" % (args[0],args[1])
#return response.css(args[0]).xpath(args[1])
def response_xpath_ARG(arg):
return "return respons.xpath(\"%s\")" % (arg)
#return response.xpath(arg)
def execute(function, args):
response = function(args)
# do whatever with response
return response
response_url = "https://whatever.com"
patterns_trees = [(response_css_ARG_xpath_ARG, ("#Header", "//a/img/#src")),
(response_css_ARG_xpath_ARG, ("#HEADER", "//a/img/#src")),
(response_xpath_ARG, ("//header//a/img/#src")),
(response_xpath_ARG, ("//a[#href='"+response_url+"/"+"']/img/#src")),
(response_xpath_ARG, ("//a[#href='/']/img/#src"))]
for pattern_tree in patterns_trees:
print(execute(pattern_tree[0], pattern_tree[1]))
Note that execute() can be omitted! Depending on if you need to process the result or not. Without the executioner, you may just call the function directly from the loop:
for pattern_tree in patterns_trees:
print(pattern_tree[0](pattern_tree[1]))

Not sure I understand what you're trying to do, but could you make your list a list of lambda functions like so:
patterns_trees = [
lambda response : response.css("#Header").xpath("//a/img/#src"),
...
]
And then, in your loop:
for pattern_tree in patterns_trees:
intermediate_response = scrapy.Request(...) # without meta kwarg
pattern_response = pattern_tree(intermediate_response)
Or does leaving the meta away have an impact on the response object?

Looping through list of functions in a function in Python dynamically

I'd like to see if it's possible to run through a list of functions in a function. The closest thing I could find is looping through an entire module. I only want to use a pre-selected list of functions.
Here's my original problem:
Given a string, check each letter to see if any of the 5 tests fulfill.
If a minimum of 1 letter passes a check, return True.
If all letters in the string fails the check, return False.
For each letter in the string, we will check these functions: isalnum(), isalpha(), isdigit(), islower(), isupper()
The result of each test should print to different lines.
Sample Input
qA2
Sample Output (must print to separate lines, True if at least one letter passes, or false is all letters fail each test):
True
True
True
True
True
I wrote this for one test. Of course I could just write 5 different sets of code but that seems ugly. Then I started wondering if I could just loop through all the tests they're asking for.
Code for just one test:
raw = 'asdfaa3fa'
counter = 0
for i in xrange(len(raw)):
if raw[i].isdigit() == True: ## This line is where I'd loop in diff func's
counter = 1
print True
break
if counter == 0:
print False
My fail attempt to run a loop with all the tests:
raw = 'asdfaa3fa'
lst = [raw[i].isalnum(),raw[i].isalpha(),raw[i].isdigit(),raw[i].islower(),raw[i].isupper()]
counter = 0
for f in range(0,5):
for i in xrange(len(raw)):
if lst[f] == True: ## loop through f, which then loops through i
print lst[f]
counter = 1
print True
break
if counter == 0:
print False
So how do I fix this code to fulfill all the rules up there?
Using info from all the comments - this code fulfills the rules stated above, looping through each method dynamically as well.
raw = 'ABC'
functions = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper]
for func in functions:
print any(func(letter) for letter in raw)
getattr approach (I think this is called introspection method?)
raw = 'ABC'
meths = ['isalnum', 'isalpha', 'isdigit', 'islower', 'isupper']
for m in meths:
print any(getattr(c,m)() for c in raw)
List comprehension approach:
from __future__ import print_function ## Changing to Python 3 to use print in list comp
raw = 'ABC'
functions = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper]
solution = [print(func(raw)) for func in functions]

The way you are looping through a list of functions is slightly off. This would be a valid way to do it. The functions you need to store in the list are the generic string functions given by str.funcname. Once you have those list of functions, you can loop through them using a for loop, and just treat it like a normal function!
raw = 'asdfaa3fa'
functions = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper] # list of functions
for fn in functions: # iterate over list of functions, where the current function in the list is referred to as fn
for ch in raw: # for each character in the string raw
if fn(ch):
print(True)
break
Sample outputs:
Input Output
===================================
"qA2" -----> True True True True True
"asdfaa3fa" -----> True True True True
Also I notice you seem to use indexing for iteration which makes me feel like you might be coming from a language like C/C++. The for in loop construct is really powerful in python so I would read up on it (y).
Above is a more pythonic way to do this but just as a learning tool, I wrote a working version that matches how you tried to do it as much as possible to show you where you went wrong specifically. Here it is with comments:
raw = 'asdfaa3fa'
lst = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper] # notice youre treating the functions just like variables and aren't actually calling them. That is, you're writing str.isalpha instead of str.isalpha()
for f in range(0,5):
counter = 0
for i in xrange(len(raw)):
if lst[f](raw[i]) == True: # In your attempt, you were checking if lst[f]==True; lst[f] is a function so you are checking if a function == True. Instead, you need to pass an argument to lst[f](), in this case the ith character of raw, and check whether what that function evaluates to is true
print lst[f]
counter = 1
print True
break
if counter == 0:
print False

Okay, so the first question is easy enough. The simple way to do it is just do
def foo(raw):
for c in raw:
if c.isalpha(): return True
if c.isdigit(): return True
# the other cases
return False
Never neglect the simplest thing that could work.
Now, if you want to do it dynamically -- which is the magic keyword you probably needed, you want to apply something like this (cribbed from another question):
meths = [isalnum, isalpha, isdigit, islower, isupper]
for c in raw:
for m in meths:
getattr(c, m)()
Warning, this is untested code meant to give you the idea. The key notion here is that the methods of an object are attributes just like anything else, so, for example getattr("a", "isalpha")() does the following:
Uses getattr to search the attributes dictionary of "a" for a method named isalpha
Returns that method itself -- <function isalpha>
then invokes that method using the () which is the function application operator in Python.
See this example:
In [11]: getattr('a', 'isalpha')()
Out[11]: True

All the other answers are correct, but since you're a beginner, I want to point out the problem in your code:
lst = [raw[i].isalnum(),raw[i].isalpha(),raw[i].isdigit(),raw[i].islower(),raw[i].isupper()]
First: Not sure which value i currently has in your code snipped, but it seems to point somewhere in the string - which results in single characters being evaluated, not the whole string raw.
Second: When you build your list, you are already calling the methods you want to insert, which has the effect that not the functions themself get inserted, but their return values (that's why you're seeing all those True values in your print statement).
Try changing your code as follows:
lst = [raw.isalnum, raw.isalpha, raw.isdigit, raw.islower, raw.isupper]

I'm going to guess that you're validating password complexity, and I'm also going to say that software which takes an input and says "False" and there's no indication why is user-hostile, so the most important thing is not "how to loop over nested char function code wizardry (*)" but "give good feedback", and suggest something more like:
raw = 'asdfaa3fa'
import re
def validate_password(password):
""" This function takes a password string, and validates it
against the complexity requirements from {wherever}
and returns True if it's complex enough, otherwise False """
if not re.search('\d', password):
print("Error: password needs to include at least one number")
return False
elif not re.search('[a-z]', password):
print("Error: password must include at least one lowercase letter")
return False
elif not re.search('[A-Z]', password):
print("Error: password must include at least one uppercase letter")
return False
print("Password is OK")
return True
validate_password(raw)
Try online at repl.it
And the regex searching checks ranges of characters and digits in one call, which is neater than a loop over characters.
(PS. your functions overlap; a string which has characters matching 'isupper', 'islower' and 'isnumeric' already has 'isadigit' and 'isalnum' covered. More interesting would be to handle characters like ! which are not upper, lower, digits or alnum).
(*) function wizardry like the other answers is normally exactly what I would answer, but there's so much of that already answered that I may as well answer the other way instead :P

To answer the original question:
raw = 'asdfa3fa'
functions = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper]
isanything = [func(raw) for func in functions]
print repr(isanything)

Since you are looping through a list of simple items and trying to find if all of the functions has any valid results, you can simply define the list of functions you want to call on the input and return that. Here is a rather pythonic example of what you are trying to achieve:
def checker(checks, value):
return all(any(check(r) for r in value) for check in checks)
Test it out:
>>> def checker(checks, value):
... return all(any(check(r) for r in value) for check in checks)
...
>>> checks = [str.isalnum, str.isalpha, str.isdigit, str.islower, str.isupper]
>>> checker(checks, 'abcdef123ABC')
True
>>> checker(checks, 'abcdef123')
False
>>>

You can use introspection to loop through all of an object's attributes, whether they be functions or some other type.
However you probably don't want to do that here, because str has lots of function attributes, and you're only interested in five of them. It's probably better to do as you did and just make a list of the five you want.
Also, you don't need to loop over each character of the string if you don't want to; those functions already look at the whole string.

Check out this one-line solution for your problem. That problem is from HackerRank. I loop through a list of functions using the built-in getattr function.
s='qA2'
[print(bool(list(filter(lambda x : getattr(x, func)(),s)))) for func in ['isalnum','isalpha','isdigit','islower','isupper']]

Python "if" statement - if xpath is true

im trying to code in python (very new to it) and need to check if an xpath is there then variable = the xpath but if not variable = string.
An example is below
if tree.xpath('//*#id="page"]/div[1]/div/main/div/article/div/div[1]/section[2]/p[1]/span/text()') = true
$value = tree.xpath('//*#id="page"]/div[1]/div/main/div/article/div/div[1]/section[2]/p[1]/span/text()')
else
$value = ''

You should really start by doing the whole official tutorial before anything else, as it will anwser your question.
First point : Python objects all have a 'truth' value in a boolean context, which is defined by the object's type and actual value. For builtin types, all empty containers (lists, dicts, sets etc), the empty string, all numerical zeros and the None object are false, everything else is true. For non builtin types you'll have to check the package's documentation.
The builtin type bool will also tell you the boolean value of a given object, so all of the below tests are equivalent:
if myobj:
xxx
if bool(myobj):
xxx
if bool(myobj) == True:
xxx
BUT keep in mind that it doesn't imply that bool(myobj) is the same as myobj - the first one is the boolean value of your object, so the following is NOT equivalent (unless myobj is one of True, 1 or 1.0):
if myobj == True:
xxx
Now wrt/ your actual code snippet: it's not valid Python (bad indentation, invalid identifier $value, invalid use of the assignment operator =, missing : after the if and else statements, wrong capitalization for True...)
Assuming you meant:
# let's make this at least readable:
path = '//*#id="page"]/div[1]/div/main/div/article/div/div[1]/section[2]/p[1]/span/text()'
if tree.xpath(path) == True:
value = tree.xpath(path)
else:
value = ''
The obvious error is the explicit test against True (tree.xpath() doesn't return a boolean). You either have to explicitely cast the return of tree.xpath() to a boolean (which is quite verbose, totally useless and definitly unpythonic) or just let Python do the right thing by removing the == True part of your test.
As a side note: calling tree.xpath twice in a row with the same argument is a waste of processor cycle (both calls will return the same value), so use a variable instead - it will also make your code much more readable and maintainable. The Pythonic version of your code would look something like:
path = '//*#id="page"]/div[1]/div/main/div/article/div/div[1]/section[2]/p[1]/span/text()'
found = tree.xpath(path)
value = found if found else ''
or even more simply:
path = '//*#id="page"]/div[1]/div/main/div/article/div/div[1]/section[2]/p[1]/span/text()'
value = tree.xpath(path) or ''
since the or operator will not yield a boolean value but either the first of it's operand that has a true value or the last operand if none has a true value.

#No need to test == if value is bool. and you not check, you assign value with one =
if anything:
#Then do this
else:
#Do this

code runs else statement repeatedly

every time the below code runs, it goes straight through to the first else statement, then runs the next else statement four times. what is going on and how do I fix it so it calls the movement() module?
class square(object):
def __init__(self,updown,leftright,residence,name):
self.updown = updown
self.leftright = leftright
self.residence = bool
self.name = name
a1 = square(1,1,True,"a1")
a2 = square(2,1,False,"a2")
b1 = square(1,2,False,"b1")
b2 = square(2,2,False,"b2")
square_dict = {a1:"a1",a2:"a2",b1:"b1",b2:"b2"}
movement_select()
def movement_select():
response = raw_input("where would you like to move?")
if response in square_dict:
moveTo = square_dict[response]
else:
print "this runs once"
for a in square_dict:
if a.residence == True:
moveFrom = a
movement(moveFrom,moveTo)
else:
print "this runs four times"
movement_select()

Look at how you're defining residence:
self.residence = bool
So, for any square a, a.residence will be the type bool, never the boolean value True (or anything else). So this test will always fail:
if a.residence == True:
To fix it, change that first line to:
self.residence = residence
While we're at it, you rarely need == True, so you can also change the second line to:
if a.residence:
But that isn't a necessary fix, just a way of simplifying your code a bit.
Meanwhile, your squares_dict is a bit odd. I'm not sure whether it's incorrect or not, but let's take a look:
It maps from square objects to their names. That could be a useful thing to do. It means you can iterate over the dict and get all the squares—as you correctly do in your code. And if you later had a square and wanted to get its name, you could use square_dict for that. Then again, you could probably get the same benefit with just a square_list, and using the name already available as an attribute of the square objects (unless you need the same square to have different names in different contexts).
And meanwhile, a mapping in this direction can't be used for looking up squares_dict[response], because response is a name, not a square. So, you definitely need a mapping in the opposite direction, either in addition to or instead of this one.
If you scrap the square-to-name mapping and only keep the name-to-square mapping, you can still iterate over the squares; you'd just have to do for square in squares_dict.values(): instead of just for square in squares_dict:.

First problem: your dictionary appears to be backwards: you want to look up the square objects from their locations, rather than the other way around. This is why your first conditional is never true. You also might as well strip() the response to ensure that you don't have any hidden whitespace in there.
square_dict = {"a1":a1, "a2":a2, "b1":b1, "b2":b2}
# Later...
response = raw_input("where would you like to move?").strip()
# Later still...
for a in square_dict.values(): # Because we switched the dictionary around!
If you don't want to silently strip off the whitespace, I'd suggest that you at least echo their input back to them (print('"{}"'.format(response))) in the case that it's not found in your dictionary, so they (you) can be sure that at least the input was correct.
The second problem is because of how you define residence. You set the variable equal to bool, which is not what you want at all. Line five ought to read:
self.residence = residence
Finally, some other thoughts on your code! You check whether a value is True by checking if a.residence == True:. The preferred form of this comparison is the simpler version:
if a.residence:
Your methods could also be named more descriptively. Generally speaking, it's always nice to begin a function or method name with a verb, to improve readability. This is of course a question of style, but for instance, the two functions we see, movement_select and movement aren't extremely clear as to their function. It would be much easier to read if they used a standardized form, e.g. choose_move and perform_move.

self.residence = bool
should be
self.residence = residence

You don´t set residence right, this is wrong:
class square(object):
def __init__(self,updown,leftright,residence,name):
self.updown = updown
self.leftright = leftright
self.residence = bool
self.name = name
it has to be
class square(object):
def __init__(self,updown,leftright,residence,name):
self.updown = updown
self.leftright = leftright
self.residence = residence # not bool!!
self.name = name

Your response contains \n symbol, just strip() it
You should also swap places between keys and values in your dictionary