import praw
import time
from selenium import webdriver
driver = webdriver.Chrome()
r = praw.Reddit(client_id='XXXXXXXXXX',
client_secret='XXXXXXXXXXXXX', password='XXXXXXXXX',
user_agent='Grand Street Tech', username='grandstreetsupreme')
subreddit = r.subreddit('supremeclothing')
submissions = []
users = []
for submission in r.subreddit('supremeclothing').new(limit=999):
for comment in submission.comments:
author = comment.author
users.append(author)
It takes like 10 minutes to complete and when it does it doesn't do anything.
There is no print statement for the users right, put a statement like below.
print users
This is because you just created the list users, you need to tell python to print it.
After your for loop, put print users
Related
I am using instaloader to scrape instagram posts as part of a study project.
To avoid getting shut down by instagram, I use sleep function to sleep between 1-20 sec between each round. This works well.
I don't want to have to go through all posts each time I scrape, and therefore i want the loop to run 5 times. Which will give me 5 posts. But I don't seem to manage to get it to do it.
I had written the following function to try to scrape the profile and return the first 5 posts:
## importing and creating instance
from instaloader import Instaloader
from instaloader import Profile
import instaloader
import time
from random import randint
L = instaloader.Instaloader()
#random time for sleep
vent = randint(1,20)
# function:
def get2posts(profile_name):
profile = Profile.from_username(L.context, profile_name)
POSTS = profile.get_posts()
for post in POSTS:
for i in range(2):
L.download_post(post, profile_name)
time.sleep(vent)
break
print('scrape done')
This code returns 5 of the same posts though, and I simply can't figure out a way to get it to return the first 5 posts of an account.
The working function, which harvests all posts of a profile is:
# the original function (without range)
def get_posts(profile_name):
profile = Profile.from_username(L.context, profile_name)
POSTS = profile.get_posts()
for post in POSTS:
L.download_post(post, profile_name)
time.sleep(vent)
print('I am done')
Hope you can help :)
The problem is that the inner for loop runs download_post twice (range(2)) on the same post, and then the outer loop breaks. If POSTS is a list, you can use slicing to loop only over the first 5 items like so: for post in POSTS[:5]:. A safer method though would be to count the posts as you go, which should work for most types of iterables (not just lists), like so:
def get2posts(profile_name):
profile = Profile.from_username(L.context, profile_name)
POSTS = profile.get_posts()
for i, post in enumerate(POSTS):
L.download_post(post, profile_name)
if i == 4:
break
time.sleep(vent)
print('scrape done')
Essentially, I've written a program for a reddit bot designed to list down certain apsects of a reddit post, such as the title or poster, as long as they fit a certain criteria. I want it to be able to automatically run once every hour. I also want it to be able to make a post once every 7 days. Could someone share code for these please?
#!/usr/bin/python
import base64
import praw
#Enter your correct Reddit information into the variable below
userAgent = 'RRBot-Beta'
cID = 'Enter your so and so'
cSC = 'Enter your secret'
userN = 'Enter your Reddit username'
userP = 'Enter your Reddit password'
unfilled_post_URL = [""]
unfilled_post_url_B64 = [""]
submission_title_and_poster = {}
filled_requests = 0
unfilled_requests = 0
requests = 0
reddit = praw.Reddit(user_agent=userAgent,
client_id=cID,
client_secret=cSC,
username=userN,
password=userP)
subreddit = reddit.subreddit('riprequestsnew') #any subreddit you want to monitor
title_keywords = {'requests', 'request'} #makes a set of keywords to find in subreddits
comment_keyword = "share"
for submission in subreddit:
lowercase_title = submission.title.lower() #makes the post title lowercase so we can compare our keywords with it.
for title_keyword in title_keywords: #goes through our keywords
if title_keyword in lowercase_title: #if one of our keywords matches a title in the subreddit
requests = requests + 1 #tracks the number of requests
for post in requests:
comments = subreddit.submission.comment.lower() #makes the comment text lowercase
if comment_keyword in comments: #checks the comment text for our keyword
filled_requests += 1 #if someone has shared something, this post will be marked as filled
elif comment_keyword not in comments: #if no one has shared anything, the post url will be added to a list
submission_title_and_poster.update({subreddit.submission.title: subreddit.submission.name})
unfilled_post_URL.append(subreddit.submission.url)
for url in unfilled_post_URL: #B64 encodes each url and adds it to a new list
text = open(url, "rb")
text_read = text.read()
url_encoded = base64.encodestring(text_read)
unfilled_post_url_B64.append(url_encoded)
unfilled_requests += 1
Schedule (https://pypi.python.org/pypi/schedule) seemed to be what you need.
You will have to install their Python library:
pip install schedule
then modify the sample script :
import schedule
import time
def job():
schedule.every(10).seconds.do(job)
schedule.every(10).minutes.do(job)
schedule.every().hour.do(job)
schedule.every().day.at("10:30").do(job)
schedule.every(5).to(10).minutes.do(job)
schedule.every().monday.do(job)
schedule.every().wednesday.at("13:15").do(job)
schedule.every().minute.at(":17").do(job)
while True:
schedule.run_pending()
time.sleep(1)
With your own function in job() and use the needed call for your timing.
Then you can run it with nohup.
Be advised that yu will need to start it again if you reboot.
here is the docs
I'm using Spotipy and LyricsGenius to open lyrics on a web browser from a terminal.
I can open a url for one song, but have to run the script each time to run consecutively. What are some ways to detect the end of a song using Spotipy?
import spotipy
import webbrowser
import lyricsgenius as lg
...
# Create our spotifyObject
spotifyObject = spotipy.Spotify(auth=token)
# Create out geniusObject
geniusObject = lg.Genius(access_token)
...
while True:
currently_playing = spotifyObject.currently_playing()
artist = currently_playing['item']['artists'][0]['name']
title = currently_playing['item']['name']
search_query = artist + " " + title
# if (currently_playing has changed):
song = geniusObject.search_songs(search_query)
song_url = song['hits'][0]['result']['url']
webbrowser.open(song_url)
webbrowser.open(song_url)
I was reading relevant threads such as this, this, and read through documentation but could not find an answer to my question if this could be handled by Spotipy. I would appreciate any suggestions, thank you.
I used time.sleep(length) with the argument 'length' standing for the remaining duration of a current track.
I have a problem. I am writing a bot in Selenium. I'm working with 3 arrays but for loop not working. All elements of the array are written to an input. You can see the codes below.
How can I fix?
Best regards.
bot.py:
from selenium import webdriver
import time
import list
browser = webdriver.Firefox()
browser.get("")
time.sleep(4)
sayi=1
for posta, isim, kadi in zip(instagramList.email,instagramList.fullName,instagramList.userName):
browser.find_element_by_css_selector("input[name='emailOrPhone']").send_keys(posta)
browser.find_element_by_css_selector("input[name='fullName']").send_keys(isim)
browser.find_element_by_css_selector("input[name='username']").send_keys(kadi)
browser.find_element_by_css_selector("input[name='password']").send_keys("+1gP5xc!")
time.sleep(2)
browser.find_element_by_partial_link_text('Sign').click()
print(str(sayi)+". "+ "Kayit olustu.")
sayi = sayi+1
list.py
email=["abidinkandemir#evtsoft.com", "asd#asd.com", "sdsd#asd.com"],
fullName=["abidin kandemir","asd asd", "asdd asd"],
userName=["abidinkandemir102","asdas", "asdd"]
For example, you have renamed the file with lists to "instagramList.py".
In "bot.py" file you need to import that file like: import instagramList.
After these steps, your script should work one iteration (If you are correctly intended your code):
bot.py:
from selenium import webdriver
import time
import instagramList
browser = webdriver.Firefox()
browser.get("your_url")
time.sleep(4)
sayi = 1
for posta, isim, kadi in zip(instagramList.email, instagramList.fullName, instagramList.userName):
browser.find_element_by_css_selector("input[name='emailOrPhone']").send_keys(posta)
browser.find_element_by_css_selector("input[name='fullName']").send_keys(isim)
browser.find_element_by_css_selector("input[name='username']").send_keys(kadi)
browser.find_element_by_css_selector("input[name='password']").send_keys("+1gP5xc!")
time.sleep(2)
browser.find_element_by_partial_link_text("Sign").click()
print(str(sayi) + ". Kayit olustu.")
sayi = sayi + 1
instagramList.py
email = ["abidinkandemir#evtsoft.com", "asd#asd.com", "sdsd#asd.com"],
fullName = ["abidin kandemir","asd asd", "asdd asd"],
userName = ["abidinkandemir102","asdas", "asdd"]
PS: Even after this your code will not work as you expected. Why? You need after each login does "logout"-action and open the Login page with browser.get("your_url").
Hope it helps you!
I am new to Facebook API. Currently, I am trying to print out ALL the comments that have been posted for this facebook page called 'leehsienloong'. However, I could only print out a total of 700+ comments. I'm sure there are more than 700+ comments in total.
I find out that the problem is, I did not request to go to another page to print out the comments. I read about paging Facebook API, but I still do not understand how to do the code for paging.
Is there anyone out there who will be able to help/assist me? I really need help. Thank you.
Here is my code, without paging:
import facebook #sudo pip install facebook-sdk
import itertools
import json
import re
import requests
access_token = "XXX"
user = 'leehsienloong'
graph = facebook.GraphAPI(access_token)
profile = graph.get_object(user)
posts = graph.get_connections(profile['id'], 'posts')
Jstr = json.dumps(posts)
JDict = json.loads(Jstr)
count = 0
for i in JDict['data']:
allID = i['id']
try:
allComments = i['comments']
for a in allComments['data']:
count += 1
print a['message']
except (UnicodeEncodeError):
pass
print count
You can use the limit parameter to increase the number of comments to be fetched. The default is 25. You can increase it like this:
posts = graph.get_connections(profile['id'], 'posts', limit=100)
But more convenient way would be get the previous and next pages from paging and do multiple requests.
to get all the comments of a post the logic should be something like
comments = []
for post in posts["data"]:
first_comments = graph.get_connections(id=post["id"], connection_name="comments")
comments.extend(first_comments["data"])
while True:
try:
next_comments = requests.get(post_comments["paging"]["next"]).json()
comments.extend(next_comments["data"])
except KeyError:
break