Develop Reference

Is there a way to ignore empty lines in the ndiff lib and to print only the + and the number of the line? PYTHON

So guys, here is my code:
import io
import difflib
import re
with io.open('textest.txt', mode="r", encoding="utf_8_sig") as file:
lines1 = file.readlines()
with io.open('minitext.txt', mode="r", encoding="utf_8_sig") as file:
lines2 = file.readlines()
def prefilter(line):
return re.sub("\s+"," ",line.strip())
for d in difflib.ndiff([prefilter(x) for x in lines1],[prefilter(x) for x in lines2]):
print(d)
the textest.txt is the full song and the minitext.txt is just a part of it. The output is this (I know, it's a justin bieber song, it's just an example)
+ somethin' I don't wanna hold back
- For all the times that you rained on my parade
- And all the clubs you get in using my name
- You think you broke my heart, oh, girl, for goodness' sake
- You think I'm crying on my own, well, I ain't
- And I didn't wanna write a song
- 'Cause I didn't want anyone thinkin' I still care, I don't, but
- You still hit my phone up
- And baby, I'll be movin' on
- And I think you should be somethin' I don't wanna hold back
Maybe you should know that
My mama don't like you and she likes everyone
And I never like to admit that I was wrong
And I've been so caught up in my job
Didn't see what's going on, but now I know
+
+
+
I'm better sleeping on my own
+ 'Cause if you like the wa
- 'Cause if you like the way you look that much
- Oh, baby, you should go and love yourself
- And if you think that I'm still holdin' on to somethin'
The thing is: I wanted to print only the + (The different lines on the lines2, that is the minitext.txt), and the number of the line which is different. I also wanted to ignore the completely empty lines so the output is just like:
somethin' I don't wanna hold back (Number of line in minitext.txt)
'Cause if you like the wa (Number of line in minitext.txt)
or anything similiar. Is there a way I could do that?

if you go into difflib.py in your system (difflib.__file__ (get path for this module in your system))
you find that ndiff -> return Differ(linejunk, charjunk).compare(a, b)
after that inside Differ class check compare function....
this module work with _dump for yield (tag + string -->>> you see in print)
we overwrite this function for our own purpose
we filter tag(we need + tag)
filter string (we need not empty)
you can check(source code) and retrieve everything that you like.
import difflib
# over write Differ class (difflib.ndiff work with compare function that exist in this class)
class Differ(difflib.Differ):
# this function used for + tag
def _dump(self, tag, x, lo, hi):
"""Generate comparison results for a same-tagged range."""
for i in range(lo, hi):
# only yield if tag == "+"
if tag == '+':
# if second list not empty yield result
if x[i] != "":
# change format (item of second list string, i==index (number line start from 0))
yield '%s(%s)' % (x[i], i)
# example of two list of string
a = ["first", "second", "three", "four"]
b = ["number_one", "second", "number_three", "four"]
differ_object = Differ()
result = differ_object.compare(a, b)
for _ in result:
print(_)
# result
"""
number_one(0)
number_three(2)
"""

Python - .insert() method only replaces the first letter of word?

I've posted about this before and I've been able to whittle the program down to a single function for testing purposes. I'm not getting any errors, but I am getting a bug that is driving me up the wall. I'm able to replace the shortcut, but it only replaces the first letter of the shortcut.
Here's the code:
from tkinter import *
root = Tk()
text = Text(root)
text.pack(expand=1, fill=BOTH)
syntax = "shortcut = sc" # This will be turned into a function to return the shortcut
# and the word, I'm only doing this for debugging purposes.
def replace_shortcut(event=None):
tokens = syntax.split()
word = tokens[:1]
shortcut = tokens[2:3]
index = '1.0'
while 1:
index = text.search(shortcut, index, stopindex="end")
if not index: break
last_idx = '%s + %dc' % (index, len(shortcut))
text.delete(index, last_idx)
text.insert(index, word)
last_idx = '%s + %dc' % (index, len(word))
text.bind('<space>', replace_shortcut)
text.mainloop()
The shortcut given, in our case 'sc' will turn into 'shortcutc' after the space is typed. Any help is appreciated!

You have two problems.
You have the variable shortcut defined to be ['sc'] instead of 'sc'. So len(shortcut) will always be 1 (the length of the array) rather than 2 (the length of the string). You'll always end up deleting just one character. Probably you want len(shortcut[0])
[You also have the same problem with len(word). You'll always get 1, the length of the array.]
Also, the last line of your while loop should set index rather than last_idx, since that's the variable that's going to be used in the next search.

Python - how to loop over the elements returned by this function?

I just started learning python. I am now trying to apply my knowledge to modify some code I found online
The function shape.part(i) returns a point (x,y) on an image. I am now trying to write each of those points on an individial line to a file. But for some reason the for loop always generates runtime errors, no matter whether I used
- range(len(shape.part() ))
- enum(len(shape.part() ))
- enum(shape.part() ))
- ....
Could somebody show me what is the correct way to do this and explain why what I've tried didn't work?
Thanks
The piece of code I am trying to get to work:
filePts = open('myFile.pts','w')
filePts.write('version: 1'+'\n'+'n_points: 68'+'\n'+'{')
for k, d in enumerate(dets):
print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(k, d.left(), d.top(), d.right(), d.bottom()))
# Get the landmarks/parts for the face in box d.
shape = predictor(img, d)
print("Part 0: {}, Part 1: {} ...".format(shape.part(0),shape.part(1)))
# Write output to file
for i in enumerate(( shape.part() )): # <----- This is what fails and am trying to fix
print('i: '+ str(i))
coordLandMark = str(shape.part(i))
coordLandMark = coordLandMark[1:]
coordLandMark = coordLandMark.replace(", ", " ")
coordLandMark = coordLandMark.replace(")", "")
filePts.write(coordLandMark+'\n')
# Draw the face landmarks on the screen.
win.add_overlay(shape)
filePts.write('}\n')

hi try this may be you looking for it
for row in range(0,len(shape.part() )):
print row

As I said in the comment, in Python you iterate over the thing itself. So, assuming shape.part() returns some kind of sequence:
for coordLandMark in shape.part():
coordLandMark = coordLandMark[1:]
coordLandMark = coordLandMark.replace(", ", " ")
coordLandMark = coordLandMark.replace(")", "")
filePts.write(coordLandMark+'\n')
Note, I don't actually know what shape.part() returns. If you need more help, you'll need to post a link to the documentation of whatever that object is. There might well be a different method you need to call on shape to get something you can iterate over.

You cannot iterate over a function like that in Python, you can only iterate over lists, tuples, strings, and other iterable objects. I believe what you want to do is run the function for many values? Like suggested by your print("Part 0: {}, Part 1: {} ...".format(shape.part(0), shape.part(1))) line. Therefore what you should do is something like:
for i in range(some_value):
coordLandMark = str(shape.part(i))
# Do other things you need to do
The problem now is for you to find how many items you need to iterate over (the some_value in my code). I can't really help more on that matter without more information...

how can i speed up my code?

It's a program that suggests to the user a player's name if the user made a typo. It's extremely slow.
First it has to issue a get request, then checks to see if the player's name is within the json data, if it is, pass. Else, it takes all the players' first and last names and appends it to names. Then it checks whether the first_name and last_name closely resembles the names in the list using get_close_matches. I knew from the start this would be very slow, but there has to be a faster way to do this, it's just I couldn't come up with one. Any suggestions?
from difflib import get_close_matches
def suggestion(first_name, last_name):
names = []
my_request = get_request("https://www.mysportsfeeds.com/api/feed/pull/nfl/2016-2017-regular/active_players.json")
for n in my_request['activeplayers']['playerentry']:
if last_name == n['player']['LastName'] and first_name == n['player']['FirstName']:
pass
else:
names.append(n['player']['FirstName'] + " " + n['player']['LastName'])
suggest = get_close_matches(first_name + " " + last_name, names)
return "did you mean " + "".join(suggest) + "?"
print suggestion("mattthews ", "stafffford") #should return Matthew Stafford

Well, since it turned out my suggestion in the comments worked out, I might as well post it as an answer with some other ideas included.
First, take your I/O operation out of the function so that you're not wasting time making the request every time your function is run. Instead, you will get your json and load it into local memory when you start the script. If at all possible, downloading the json data beforehand and instead opening a text file might be a faster option.
Second, you should get a set of unique candidates per loop because there is no need to compare them multiple times. When a name is discarded by get_close_matches(), we know that same name does not need to be compared again. (It would be a different story if the criteria with which the name is being discarded depends on the subsequent names, but I doubt that's the case here.)
Third, try to work with batches. Given that get_close_matches() is reasonably efficient, comparing to, say, 10 candidates at once shouldn't be any slower than to 1. But reducing the for loop from going over 1 million elements to over 100K elements is quite a significant boost.
Fourth, I assume that you're checking for last_name == ['LastName'] and first_name == ['FirstName'] because in that case there would have been no typo. So why not simply break out of the function?
Putting them all together, I can write a code that looks like this:
from difflib import get_close_matches
# I/O operation ONCE when the script is run
my_request = get_request("https://www.mysportsfeeds.com/api/feed/pull/nfl/2016-2017-regular/active_players.json")
# Creating batches of 10 names; this also happens only once
# As a result, the script might take longer to load but run faster.
# I'm sure there is a better way to create batches, but I'm don't know any.
batch = [] # This will contain 10 names.
names = [] # This will contain the batches.
for player in my_request['activeplayers']['playerentry']:
name = player['FirstName'] + " " + player['LastName']
batch.append(name)
# Obviously, if the number of names is not a multiple of 10, this won't work!
if len(batch) == 10:
names.append(batch)
batch = []
def suggest(first_name, last_name, names):
desired_name = first_name + " " + last_name
suggestions = []
for batch in names:
# Just print the name if there is no typo
# Alternatively, you can create a flat list of names outside of the function
# and see if the desired_name is in the list of names to immediately
# terminate the function. But I'm not sure which method is faster. It's
# a quick profiling task for you, though.
if desired_name in batch:
return desired_name
# This way, we only match with new candidates, 10 at a time.
best_matches = get_close_matches(desired_name, batch)
suggestions.append(best_matches)
# We need to flatten the list of suggestions to print.
# Alternatively, you could use a for loop to append in the first place.
suggestions = [name for batch in suggestions for name in batch]
return "did you mean " + ", ".join(suggestions) + "?"
print suggestion("mattthews ", "stafffford") #should return Matthew Stafford

Amazon Alexa's card system and Flask-Ask

Ok im new to Alexa skill development but its going well, my skill is coming on well however, id like to add some more content in the form of a card. The data id like to populate the card with is in the form of a list. So I thought id try passing the list directly (didnt think it would work but worth a shot). There is nothing in the docs that explains passing lists to the card system. Can anyone explain how to achieve this?
The intent function looks like this:
#ask.intent('TopTenCounties')
def top_ten():
top_countries = get_top_ten_countries()
stats = []
for item in top_countries[1]:
stat = str(item[0]) + ' ' + str(item[1])
stats.append(stat)
msg = "The top ten countries are, {}".format(top_countries[0])
return statement(msg).standard_card(title='Top Ten Usage Stats:',
text=stats,
large_image_url='url.com/img.png')

Alexa's cards only take text, no other rich format is currently supported (https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/providing-home-cards-for-the-amazon-alexa-app). What's happening here is that the list is being converted to string automatically. You probably want to make your own little function to do it yourself so you have more fine control on how it's done.
def lst2str(lst, last='and'):
if len(lst)==1:
return lst[0]
else:
return ', '.join(lst[0:-1]) + ' ' + last + ' ' + lst[-1]