import concurrent.futures
import urllib.request
import json
myurls = {}
for x in range(1, 15):
for y in range(1, 87):
strvar1 = "%s" % (x)
strvar2 = "%s" % (y)
with open("C:\\Python33\\NASDAQ Stock Strings\\NASDAQ_Config_File_{}_{}.txt".format(x,y),"r") as f:
myurls[x,y] = f.read().replace('\n', '')
print("myurls_" + str(strvar1) + "_" + str(strvar2) + "=", myurls[x,y])
def myglob():
global myurls
URLS = [myurls2_1_1,myurls2_1_2,myurls2_1_3,myurls2_1_4,myurls2_1_5 ETC>>>ETC >>>]
The code below here works fine. The idea is:
Define multiple strings from multiple source .txt files into a dictionary.
Pass these multiple variables to the URLS = [] statement at the same time so that they may be processed in parallel using the concurrent.futures module for python.
You say
def myglob():
global myurls
works syntax wise but the list of variables I then want to use i.e. myurls2_1_1, myurls2_1_2 etc are throwing up an error saying that they are not defined.
That is right. But you are quite close: just access them exactly the way you defined them: as myurls[1,1] or how you defined them.
Defining variables in a dynamic way is almost never the way to go; normally you can achieve that by any dict key or list index access.
As you now provided an example, I can exactly show how you'd do:
myurls = {}
for x in range(1, 15):
for y in range(1, 87):
strvar1 = "%s" % (x) # you can omit these.
strvar2 = "%s" % (y)
with open("C:\\Python33\\NASDAQ Stock Strings\\NASDAQ_Config_File_{}_{}.txt".format(x,y),"r") as f: # This is NOT the right place to put an application-specific config file.
myurls[x,y] = f.read().replace('\n', '')
print("myurls[%d,%d] = %s" % (x, y, myurls[x,y]))
def myglob(): # this function is completely pointless.
global myurls
# Construct your urls list only after you have everything that belongs in it, i. e. on the correct indentation level:
urls = [myurls[1,1], myurls[1,2], myurls[1,3], myurls[1,4], ...]
This would be one way to go. It looks very complicated and weird, this last, very long line.
But you can shorten it with a list comprehension:
urls = [myurls[x,y] for x in range(1, 15) for y in range(1, 87)]
But then, you might be asked: "Why not construct it on the way already?"
Well, here we go:
myurls = {}
urls = []
for x in range(1, 15):
for y in range(1, 87):
with open("C:\\Python33\\NASDAQ Stock Strings\\NASDAQ_Config_File_{}_{}.txt".format(x,y), "r") as f:
thisfile = f.read().replace('\n', '')
myurls[x,y] = thisfile
urls.append(thisfile)
print("myurls[%d,%d] = %s" % (x, y, thisfile))
and there you are.
There is no need to put them all at once into the list, as this has nothing to do with parallelization, which only seems to come later.
All that counts is that you have the urls present at the start of the parallelization. If this happens at once or item by item does not matter.
Related
I need to have different results for different "i"s
so whenever i call for example a_8, it should return me the exact value. but python
doesnt assign diferrent "i"s to it and just knows a_i.
1.for i in range(10): a_i = i + 5
You can use a list:
a = []
for i in range(10): a.append(i+5)
or a dictionary:
a = {}
for i in range(10): a[i] = i+5
In both cases, you can access the value later with
for i in range(10): print(a[i])
What I should use on my case?
Take a look at this answer.
for i in range(10):
globals()['a_' + str(i)] = i + 5
print(a_6)
but i don't think you should use it, better use dicts instead.
I have an idea but don't know how to execute it.
i have a variable value EG: 5
now i want my python program to create 5 different list all with differnt names
EG: list1 = []
list2 = []
...
I don't know if this is possible. When you say list1, you mean that list1 is an identifier for a variable which is a list. I don't think it is possible that the program creates identifiers within itself (or at least I don't know how).
Of course, you can write a Python program which writes another Python program where there are variables with these identifiers, then get creative with it:
VAL = 5 # can change this
f = open("lists.py", "w")
for i in range(1, VAL + 1):
f.write("list" + str(i) + " = []\n")
f.close()
Output (lists.py):
list1 = []
list2 = []
list3 = []
list4 = []
list5 = []
Morning Jacob,
I have a little bit of difficulties reading your question. But let me try to answer your question.
Direct answer
There are no simple way to generate python variables. The only way I know is to generate the code as str and then execute it.
def generate_code(n):
ret = ""
for i in range(1, n + 1):
var_name = "lst_" + str(i)
ret += var_name + " = []\n"
return ret
code = generate_code(5)
print(code)
exec(code)
where the printed code will be
lst_1 = []
lst_2 = []
lst_3 = []
lst_4 = []
lst_5 = []
The idiomatic solution
The idiomatic solution would be to create a list of list. Something like bellow.
lsts = [[] for i in range(5)]
then, you could access your sub lists using indexes in a very similar fashion then with the variables you asked for. lsts[0] would be equivalent of the above lst_1, lsts[1] would be equivalent of the above lst_2, lsts[2] would be equivalent of the above lst_3, ...
I am making a serial key generator for a school exercise, and I am running into a small problem. When I run the program, and type in I want to make 20 serial keys, the program will make twenty of the same keys. The cause of this is in the for loop making variables every time it passes by. (The v{0} thing)
I can not figure out how to use another method to read all the values in a list, and create variables out of them.
My code:
import random, sys
class Application:
def __init__(self):
global i
i = int(input("How many serial codes do you want to create?\n"))
print("")
self.main(i)
def main(self, i):
seq = "ABCDFGHJIKLMNOPQRSTUVWXYZ1234567890"
store = []
for z in range(0, i):
for x in range(0, 5):
first = random.choice(seq)
second= random.choice(seq)
third = random.choice(seq)
fourth =random.choice(seq)
fifth = random.choice(seq)
serial = first + second + third + fourth + fifth
store.append(serial)
for y, item in enumerate(store):
setattr(sys.modules[__name__], 'v{0}'.format(y), item)
create = v0 + "-" + v1 + "-" + v2 + "-" + v3 + "-" + v4
print(create)
print("\nCreated", i, "serial keys!")
if __name__ == '__main__':
app = Application()
You are doing some very funky stuff there that is totally not necessary.
This loop is your problem:
for y, item in enumerate(store):
setattr(sys.modules[__name__], 'v{0}'.format(y), item)
You are setting the same 5 global variables over and over again, overwriting the previous version. Your use of setattr(sys.modules[__name__], ..) is totally not needed when you have the globals() function giving you the global namespace as a dictionary. But setting global variables to solve a simple problem is like using a shotgun to catch one gnat.
Your code could be vastly simplified:
def main(self, count):
seq = "ABCDFGHJIKLMNOPQRSTUVWXYZ1234567890"
for i in range(count):
print('-'.join(''.join(random.choice(seq) for _ in range(5)) for _ in range(5)))
print("\nCreated {} serial keys!".format(count))
Sample printed output for count set to 5:
LWLGX-F6MNR-9YIZC-H23TK-TIGB9
YALYY-4ZARO-8H9BV-YMHVD-HFFGP
JNA5R-65GT1-TZ3BM-PNMZI-56NM3
39VCY-MLZ98-SU1PP-PYMX7-RZJQZ
76LGC-VF5MG-LK7C4-XPUSO-H0B97
You may find it useful to print store at the end for debugging purposes. You continually append to the list, but only ever read the first five elements.
The use of global variables is a bit odd to me, but the underlying problem is the way you're using the list.
The issue I'm having right now is loading a bunch of sound files as their own objects in Pygame. You load a sound with this syntax:
sound1 = pygame.mixer.Sound('file.wav')
Say I have seven files, and I want them loaded and named sound1 - sound7. And I didn't want to load them each individually. If I didn't know it was flawed, I would try something like this:
for i in range(1, 8):
new = 'sound' + str(i)
new = pygame.mixer.Sound(str(new) + 'wav')
How would I go about making 'new' it's own variable, and not a string? I've read about getattr, but it's confusing. I would really like to know how to use functions and loops to dynamically create code, but so far I can't find anything useful to a beginner like myself. Using this as an example, would anybody like to explain in a simple way the ways you could go about creating code inside of code and turn strings into usable variables/objects?
Thank you!
sounds = [] # list
for i in range(1, 8):
sounds.append(pygame.mixer.Sound('sound' + str(i) + 'wav'))
Or
sounds = {} # dictionary
for i in range(1, 8):
sounds[i] = pygame.mixer.Sound('sound' + str(i) + 'wav')
At first it seems you use the dictionary approach the same as the list approach, e.g. sounds[1] sounds[2] and so on, but you can also do this:
sounds = {} # dictionary
for i in range(1, 8):
sounds['sound' + str(i)] = pygame.mixer.Sound('sound' + str(i) + 'wav')
And now sounds["sound1"] and so on work, for example.
You can use arrays for this:
sound = []
for i in range(1,8):
sound.append (pygame.mixer.Sound("sound%d.wav" % i))
# Now use sound[0..6] to reference sound[1..7].wav
That will load the files sound1.wav through sound8.wav - if your files are named differently, you just change the range and/or string formatting.
There are 2 kinds of loops in python the for-loop and the while-loop. The for-loop is used to repeat something n number of times. The while-loop is used to repeat until something happens. For-loops are useful for game programming as they often handle the frames the game displays. Each frame runs one time through a loop. The way you store a for-loop is with a list. Here is an example of a basic loop you can familiarize yourself with:
he_count = [1, 2, 3, 4, 5]
fruits = ['apples', 'oranges', 'pears', 'apricots']
change = [1, 'pennies', 2, 'dimes', 3, 'quarters']
# this first kind of for-loop goes through a list
for number in the_count:
print "This is count %d" % number
# same as above
for fruit in fruits:
print "A fruit of type: %s" % fruit
# also we can go through mixed lists too
# notice we have to use %r since we don't know what's in it
for i in change:
print "I got %r" % i
# we can also build lists, first start with an empty one
elements = []
# then use the range function to do 0 to 5 counts
for i in range(0, 6):
print "Adding %d to the list." % i
# append is a function that lists understand
elements.append(i)
# now we can print them out too
for i in elements:
print "Element was: %d" % i
You can learn more about loops and game programming in python here:
programarcadegames.com/index.php?lang=en&chapter=loops
As part of the python twitter crawler I'm creating, I am attempting to make a "hash-table" of sorts to ensure that I don't crawl any user more than once. It is below. However, I am running into some problems. When I start crawling at the user NYTimesKrugman, I seem to crawl some users more than once. When I start crawling at the user cleversallie (in another completely independent crawl), I don't crawl any user more than once. Any insight into this behavior would be greatly appreciated!!!
from BeautifulSoup import BeautifulSoup
import re
import urllib2
import twitter
start_follower = "cleversallie"
depth = 3
U = list()
api = twitter.Api()
def add_to_U(user):
U.append(user)
def user_crawled(user):
L = len(L)
for x in (0, L):
a = L[x]
if a != user:
return False
else:
return True
def turn_to_names(users):
names = list()
for u in users:
x = u.screen_name
names.append(x)
return names
def test_users(users):
new = list()
for u in users:
if (user_crawled):
new.append(u)
return new
def crawl(follower,in_depth): #main method of sorts
if in_depth > 0:
add_to_U(follower)
users = api.GetFriends(follower)
names = turn_to_names(users)
select_users = test_users(names)
for u in select_users[0:5]:
crawl(u, in_depth - 1)
crawl(start_follower, depth)
for u in U:
print u
print("Program done.")
EDIT
Based on your suggestions (thank you all very much!) I have rewritten the code as the following:
import re
import urllib2
import twitter
start_follower = "NYTimesKrugman"
depth = 4
searched = set()
api = twitter.Api()
def crawl(follower, in_depth):
if in_depth > 0:
searched.add(follower)
users = api.GetFriends(follower)
names = set([str(u.screen_name) for u in users])
names -= searched
for name in list(names)[0:5]:
crawl(name, in_depth-1)
crawl(start_follower, depth)
for x in searched:
print x
print "Program is completed."
You have a bug where you set L = to len(L), not len(U). Also, you have a bug where you will return false if the first user does not match, not if every user does not match. In Python, the same function may be written as either of the following:
def user_crawled(user):
for a in l:
if a == user:
return True
return False
def user_crawled(user):
return user in a
The test_users function uses a user_crawled as a variable, it does not actually call it. Also, it seems you are doing the inverse of what you intend, you wish new to be populated with untested users, not tested ones. This is that function with the errors corrected:
def test_users(users):
new = list()
for u in users:
if not user_crawled(u):
new.append(u)
return new
Using a generator function, you can further simplify the function (provided you intend on looping over the results):
def test_users(users):
for u in users:
if not user_crawled(u):
yield u
You can also use the filter function:
def test_users(users):
return filter(lambda u: not user_crawled(u), users)
Your using a list to store users, not a hash-based structure. Python provides sets for when you need a list-like structure which can never have duplicates and requires fast existence tests. Sets can also be subtracted to remove all the elements in one set from the other.
Also, your list (U) is of users, but you are matching it against user names. You need to store just the user name of each added user. Also, you are using u to represent a user at one point in the program and to represent a user name at another, you should use more meaningful variable names.
The syntactic sugar of python ends up eliminating the need for all of your functions. This is how I would rewrite the entire program:
import twitter
start_follower = "cleversallie"
MAX_DEPTH = 3
searched = set()
api = twitter.Api()
def crawl(follower, in_depth=MAX_DEPTH):
if in_depth > 0:
searched.add(follower['screen_name'])
users = api.GetFriends(follower)
names = set([u['screen_name'] for u in users])
names -= searched
for name in list(names)[:5]:
crawl(name, in_depth - 1)
crawl(start_follower)
print "\n".join(searched)
print("Program done.")
The code sample you've given just plain doesn't work for starters, but I would guess your problem has something to do with not even making a hashtable (dictionary? set?).
You call L = len(L) when I cannot see anywhere else that L is defined. You then have a loop,
for x in (0, L):
a = L[x]
if a != user:
return False
else:
return True
which will actually just execute twice, once with x = 0 and once with x = L, where L is the len(L). Needless to say when you attempt to index into L the loop will fail. That won't even happen because you have an if-else that returns either way and L is not defined anywhere.
What you are most likely looking for is a set with a check for the user, do some work if they're absent, then add the user. This might look like:
first_user = 'cleversallie'
crawled_users = {first_user} #set literal
def crawl(user, depth, max_depth):
friends = get_friends(first_user)
for friend in friends:
if friend not in crawled_users and depth < max_depth:
crawled_users.add(friend)
crawl(friend, depth + 1, max_depth)
crawl(first_user, 0, 5)
You can fill in the details of what happens in get friends. Haven't tested this so pardon any syntax errors but it should be a strong start for you.
Let's start by saying there's lots of errors in this code a lot of non-python isms.
For instance:
def user_crawled(user):
L = len(U)
for x in (0, L):
a = L[x]
if a != user:
return False
else:
return True
This iterates only once through the loop... So you really ment something like [adding range() and the ability to check all the users.
def user_crawled(user) :
L = len(U)
for x in range(0, L) :
a = L[x]
if a == user :
return True
return False
Now of course a slightly more python way would be to skip the range and just iterate over the loop.
def user_crawled(user) :
for a in U :
if a == user :
return True
return False
Which is nice an simple, but now in true python you would jump on the "in" operator and write:
def user_crawled(user) :
return user in U
A few more python thoughts - list comprehensions.
def test_user(users) :
return [u for u in users if user_crawled(u)]
Which could also be applied to turn_to_names() - left as an exercise to the reader.