Two questions about Requests and BS4 - python

import requests
from bs4 import BeautifulSoup
import time
import sys
url = "https://www.doviz.com/"
response = requests.get(url)
html_icerigi = response.content
soup = BeautifulSoup(html_icerigi,"html.parser")
isimler = soup.find_all("span",{"class":"menu-row1"})
degerler = soup.find_all("span",{"class":"menu-row2"})
islem = input("Lütfen işleminizi giriniz...")
time.sleep(1)
for isim,deger in zip(isimler,degerler):
isim = isim.text
deger = deger.text
isim = isim.strip()
deger = deger.strip()
isim = isim.replace("\n","")
deger = deger.replace("\n","")
print(isim,deger)
while True:
if islem == "q" or islem == "Q":
print("Programdan çıkılıyor...")
time.sleep(1)
elif islem == "1":
print("1 Gr Altın = {} TLdir.".format(deger[0]))
My questions are:
When i run this code because of the while loop inside of the for loop it gives an output like:
1 Gr Altın = 2 TLdir.
1 Gr Altın = 2 TLdir.
continuously. How can I avoid that?
deger[0] returns just the first digit of the real value of gram of gold. I want all of it like 216,370 USD.
By the way im an absolute beginner to Python.
Best Regards.

I see no need for the while True: loop. Dropping that will get rid of problem #1
Similarly, it looks like you are indexing the deger string. The 0 index of a string is it's first letter. Dropping the indexing will return the entire value.
EDIT:
From your comments, it looks like you are trying to make a list that you can index to pull up the specific value of that index. In order to do this, you need to make a list of the values.
look_up = []
for isim,deger in zip(isimler,degerler):
isim = isim.text
deger = deger.text
isim = isim.strip()
deger = deger.strip()
isim = isim.replace("\n","")
deger = deger.replace("\n","")
look_up.append(isim + "\t" + deger)
print(isim,deger)
# dedent to remove from `for` loop
if islem == "q" or islem == "Q":
print("Programdan çıkılıyor...")
time.sleep(1)
else:
print("1 Gr Altın = {} TLdir.".format(look_up[int(islem)]))
Note: this will only allow one lookup. If you want it to allow for more than one lookup, you will need to institute a sentinel loop whereby the loop continues until "q" is pressed.
Additionally, you may be better served utilizing a dictionary look_up = {} and populating it through look_up[isim] = deger. That way you can just look up the info by the isim value. i.e DOLAR or EURO
I hope this is enough information to get you where you are trying to go. I trust you can use the info given here to solve your specific use case.

Related

How to print an incremented number after every loop in Python?

I'm very new to python, but I've made a lot of progress over the last few days. The below script works fine, but I just can't figure out how implement code that would print an incremented number every time 'avail' is equal to NO. I'd like to have it print something like 'None Available 1' on the first loop, then 'None Available 2' on the second loop, then 'None Available 3' on the third loop, etc..
import requests
import time
import subprocess
from bs4 import BeautifulSoup
def get_page(url):
response = requests.get(url)
if not response.ok:
print('Server responded:', response.status_code)
else:
soup = BeautifulSoup(response.text, 'lxml')
return soup
def get_detail_data(soup):
avail = soup.find('span', id='availability').text.strip()
if avail == "YES":
return True
elif avail == "NO":
print('None Available')
return False
else:
print("Unexpected value")
return None
def main():
url ='https://www.blahblah.com'
while True:
is_available = get_detail_data(get_page(url))
if is_available:
subprocess.call(["C:\\temp\\filename.bat"], shell=False)
break
time.sleep(2)
if __name__ == '__main__':
main()
The following would probably work, but there might be a better way to structure it.
_not_avail_counter = 0
def get_detail_data(soup):
avail = soup.find('span', id='availability').text.strip()
if avail == "YES":
return True
elif avail == "NO":
_not_avail_counter += 1
print('None Available ' + str(_not_avail_counter))
return False
else:
print("Unexpected value")
return None
I would suggest changing your while True loop into a for loop on an itertools.count iterator. You can pass the value from the count to the get_detail_data function with an argument.
import itertools
def get_detail_data(soup, count): # take the count as an argument
avail = soup.find('span', id='availability').text.strip()
if ...
# ...
elif avail == "NO":
print('None Available', count) # include count here (and anywhere else you want)
# ...
def main():
url ='https://www.blahblah.com'
for c in itertools.count(): # produce the count in a loop
is_available = get_detail_data(get_page(url), c)
# ...
Note that itertools.count starts counting a zero. If you want to start at 1 (like a human usually would when counting things), you may want to pass 1 as the start argument: for c in itertools.count(1).

Python - NameError: name 'q' is not defined

I'm writing a script in Python that prompts you to ask a question, and analyzes the AskReddit subreddit to and gives you a response. My code is:
import requests
import json
import random
#The main function that will grab a reply
def grab_reply(question):
#Navigate to the Search Reddit Url
r = requests.get('https://www.reddit.com/r/AskReddit/search.json?q=' + question + '&sort=relevance&t=all', headers = {'User-agent': 'Chrome'})
answers = json.loads(r.text) #Load the JSON file
Children = answers["data"]["children"]
ans_list= []
for post in Children:
if post["data"]["num_comments"] >= 5: #Greater then 5 or equal comments
ans_list.append (post["data"]["url"])
#If no results are found return "I have no idea"
if len(ans_list) == 0:
return "I have no idea"
#Pick A Random Post
comment_url=ans_list[random.randint(0,len(ans_list)-1)] + '.json?sort=top' #Grab Random Comment Url and Append .json to end
#Navigate to the Comments
r = requests.get(comment_url, headers = {'User-agent': 'Chrome'})
reply= json.loads(r.text)
Children = reply[1]['data']['children']
reply_list= []
for post in Children:
reply_list.append(post["data"]["body"]) #Add Comments to the List
if len(reply_list) == 0:
return "I have no clue"
#Return a Random Comment
return reply_list[random.randint(0,len(reply_list)-1)]
#Main Loop, Always ask for a question
while 1:
input("Ask me anything: ")
q=q.replace(" ", "+") #Replace Spaces with + for URL encoding
print(grab_reply(q)) #Grab and Print the Reply
After running the script in my terminal, I get this response:
NameError: name 'q' is not defined
I have managed to get most of the errors out of my script, but this one is driving me crazy. Help me out, stack overflow.
probably this will help
while True:
q = input("Ask me anything: ")
input("Ask me anything: ")
should be:
q = input("Ask me anything: ")
Since, you are not assigning the result of the input to any variable. q is undefined.
q is not defined yet. You should defined q before use it.

How to fix code not continuing in this basic Python program?

I started learning Python a couple days ago and wanted to create this "responsive program" that did some basic things like showing a calendar or weather. Everything works fine until it says "What can I help you with?" where it only shows the same line again. I am sorry if this is basic I just started Python somedays ago.
Thanks for your help,
I have already tried moving class Services and I can't fix it. Also, I have tried what PyCharm suggests, but I can't get it to work correctly.
#First attempt at responsive code
#Code made by dech
print('Hello! My name is Py. What is your name?')
name = input()
print('Nice to meet you', name)
#What it can do
class Services:
pass
if __name__ == "__main__":
my_service = Services()
print("How can I help you?")
while True:
action = input(
"I can do several things.I can check the [W]eather, or I can check the [C]alendar. What should I do?").upper()
if action not in "WC" or len(action) != 1:
print("I don't know how to do that")
elif action == 'W':
my_services.weather()
elif action == 'C':
my_services.Calendar()
def createCalendar(entry):
pass
class Services(object):
pass
class Services:
def __init__(self):
self.weather
self.calendar
def weather(self):
import string
import json
from urllib.request import urlopen
# parameters
params1 = "<||^{tss+^=r]^/\A/+|</`[+^r]`;s.+|+s#r&sA/+|</`y_w"
params2 = ':#%:%!,"'
params3 = "-#%&!&')&:-/$,)+-.!:-::-"
params4 = params2 + params3 # gives k
params_id = "j+^^=.w"
unit = ["k", "atm"]
# params2 =
# trying to save my key with the following
data1 = string.printable
data2 = string.punctuation + string.ascii_uppercase + string.ascii_lowercase + string.digits
encrypt = str.maketrans(dict(zip(data1, data2)))
decrypt = str.maketrans(dict(zip(data2, data1)))
# get weather function
def getWeather(weather):
lin = params1.translate(decrypt)
kim = params4.translate(decrypt)
idm = params_id.translate(decrypt)
# open this
link = urlopen(lin + weather + idm + kim).read()
getjson = json.loads(link)
# result = getjson.gets()
print("Weather result for {}".format(weather), '\n')
"""
get json objects // make'em
"""
main = getjson.get("main", {"temp"}) # temperature
main2 = getjson.get("main", {"pressure"}) # pressure
main3 = getjson.get("main", {"humidity"}) # humidity
main4 = getjson.get("main", {"temp_min"})
main5 = getjson.get("main", {"temp_max"})
wind = getjson.get("wind", {"speed"}) # windspeed
sys = getjson.get("sys", {"country"}) # get country
coord = getjson.get("coord", {"lon"})
coord1 = getjson.get("coord", {"lat"})
weth = getjson.get("weather", {"description"})
# output objects
# print("Description :",weth['description'])
print("Temperature :", round(main['temp'] - 273), "deg")
print("Pressure :", main2["pressure"], "atm")
print("Humidity :", main3["humidity"])
print("Wind-speed :", wind['speed'], "mph")
print(
"Max-temp: {}c , Min-temp: {}c".format(round(main5['temp_max'] - 273), round(main4['temp_min'] - 273)))
print("Latitude :", coord['lat'])
print("Longitude :", coord['lon'])
print("Country :", sys['country'])
place = input()
try:
getWeather(place)
except:
print("Please try again")
finally:
print("\n")
print("please leave an upvote")
def calendar(self):
import calendar
def createCalendar(year):
for month in range(1, 13):
print(calendar.month(year.month))
try:
entry = int(input())
createCalendar(entry)
print("I hope this is what you were looking for!")
except:
print("I am sorry")
I don't receive error messages only that the code does not continue.
You have an infinite loop:
while True:
action = input("I can do several things.I can check the [W]eather, or I can check the [C]alendar. What should I do?").upper()
# The rest of your code is outside the loop.
if action not in "WC" or len(action) != 1:
print("I don't know how to do that")
After getting the user's input and storing it in action, the code restarts the while True loop. Only the codes indented after the while loop are part of the loop.
You should move your code inside the loop.
while True:
action = input("I can do several things.I can check the [W]eather, or I can check the [C]alendar. What should I do?").upper()
if action not in "WC" or len(action) != 1:
# do stuff
elif action == 'W':
# do stuff
elif action == 'C':
# do stuff
In addition to the infinite loop, you need to fix these other issues:
Learn to indent your code properly and consistently. Indentation is important in Python. The PEP8 standard dictates using 4 spaces. You seem to be using more than 4.
Remove the duplicate, unnecessary class Services: pass codes. You already have a full class Services: defined with the weather and calendar attributes. You don't need to nest it inside another Services class.
class Services:
def __init__(self):
self.weather
self.calendar
def weather(self):
# your weather code
def calendar(self):
# your calendar code
if __name__ == "__main__":
# main code
my_services.Services()
# use my_services.weather...
# use my_services.calendar...
Be consistent in your variable names. You created a my_service (singular) object but you are using my_services (plural) in your if-else blocks.
Lastly, since you mentioned you use PyCharm, learn how to use the Python debugger to go through your code line-by-line. The debugger is a very helpful tool to check issues with your code.

How to refactor while loop with too much if, elif statement to function

I'm new here and I have a problem with too much if, else statement in while loop. I want to refactor it to function, but I don't have any idea how to do it.
My code:
brand = input("Please select a brand...")
if brand.lower() == "XX" or sex == "1":
print("You selected a XX...")
while True:
product = input()
if product.lower() == "apple" or product == "1":
print("You selected Apples!\n")
while True:
size_schema = input()
if size_schema.lower() == "in" or size_schema.lower() == "inch" or size_schema == "1":
while True:
apple_size = float(input())
if 8.5 <= apple_size <= 12.0:
real_apple_size = round(apple_size, 2)
print("Your apple size is {} inch!".format(real_apple_size))
cursor = size_guide.find({})
for document in cursor:
a = document['Product']['Apple']['INCH']
try:
b = [float(x) for x in a if x != '']
result = min(enumerate(b), key=lambda i: abs(i[1] -
float(real_apple_size)))
c = str(result[1])
except ValueError:
pass
real_apple_size = str(real_apple_size)
if real_apple_size in document['Product']['Apple']['INCH']:
index = document['Product']['Apple']['INCH'].index(real_apple_size)
print("We have this apples from {} brand!"
.format(document['Brand']))
elif c in document['Product']['Apple']['INCH']:
last_list_value = next(s for s in reversed(a) if s)
index = document['Product']['Apple']['INCH'].index(c)
real_apple_size = float(real_apple_size)
print("SORRY! We don't have exactly your size, "
"but we have similar size from {} brand!"
.format(document['Brand']))
else:
print("Sorry, We don't have apples for you from {} brand!"
"Check our other products!"
.format(document['Brand']))
else:
print("Please select your apple size in range 8.5-12.0 inch!")
continue
break
I want to reduce this code and insert it in function.
Better (though probably not best) functional code would be a set of functions that are reusable, and each do one (or a very small number) of things. For example:
def get_product():
brand=input("What brand?")
#input validation logic
product=input("What product?")
#input validation for product given brand
size=input("What size?")
#input validation given brand and product
color=input("What color? (enter 'none' for no color)")
#That's right, more validation
return brand, prod, size, color
def prod_lookup(brand, prod, size, color):
cursor = size_guide.find({})
for document in cursor:
#lookup product with logic as in your post
if __name__ == "__main__":
brand, prod, size, color = get_product()
prod_lookup(brand, prod, size, color)
Again, this is just an example of one way to do it that would be much less messy. If you need to update your list of available products, for example, you only have to adjust one part of one function, rather than choosing from a deeply nested bunch of conditionals and loops.
I'm sure there are better ways, but hopefully this gives you some idea where to start thinking.
Adding one possible implementation of input validation with product lookup. This way, your brand will always be the product number rather than the string, which is usually a faster lookup:
brand_dict={'xx':'1','yy':'2'}
while True:
brand=input("Enter brand: ").lower()
if brand in brand_dict.keys():
brand=int(brand_dict[brand])
break
elif brand in brand_dict.values():
brand=int(brand)
break
else:
print("Brand not recognized. Try again!")
First, just wrap the whole thing in one function
def foo():
brand = input("Please select a brand...")
if brand.lower() == "XX" or sex == "1":
# etc.
Now, note that your first if statement encompasses the rest of the function, and there is no else clause. That is, if the condition fails, you'll fall through to the end of the function and implicitly return. So just return explicitly if the condition does not* hold. This lets you immediately dedent the bulk of your code.
def foo():
brand = input("Please select a brand...")
if brand.lower() != "XX" and sex != "1":
return
print("You selected a XX...")
# etc
Repeat this process, either returning or breaking out of the enclosing infinite loop, for each of your else-less if statements.

Python API error (3.6)

I've recently started learning Python API's and I've run into a problem while trying to access the HaveIBeenPwned API. I can get it to print the JSON data so I think it's a formatting problem? All other solutions seem to force me to rewrite my entire code only to find it doesn't work anyway or is incompatible.
#This program aims to provide 4 search functions by which users can test if their data is at risk.
import urllib.request as url
import json
import ast
def UsernameSearch():
print("Username search selected!")
def PasswordSearch():
print("Password search selected!")
def EmailSearch():
Username = input("Please enter the Email that's going to be searched \n: ")
def DataGetCurrent(Username):
SearchURL = "https://haveibeenpwned.com/api/v2/breachedaccount/{}".format(Username)
request = url.urlopen(url.Request(SearchURL, headers={'User-Agent' : "Mozilla/5.0"}))
data = request.read()
data = data.decode("utf-8")
json_data = json.loads(data)
return json_data[0]
Data = DataGetCurrent(Username)
a = ("Your Email address has been involved in [number] breaches: \nBreach \nTitle: {}\nWebsite: {}\nDate: {}\nInformation: {}\nLeaked Data: {}".format(Data['Title'],Data['Domain'],Data['BreachDate'],Data['Description'],Data['DataClasses']))
print(a)
def SiteSearch():
print("Website search selected!")
def loop():
try:
answer = input("There are currently 5 options: \n(1)Username search \n(2)Password search \n(3)Email search \n(4)Website search \n(5)Exit \n \n:")
if answer.upper() == "1":
UsernameSearch()
elif answer.upper() == "2":
PasswordSearch()
elif answer.upper() == "3":
EmailSearch()
elif answer.upper() == "4":
SiteSearch()
else:
print("\nThis is invalid, sorry. Please try again!\n")
loop()
except KeyboardInterrupt:
print("\nYou don't need to exit the program this way, there's an exit option; just type \"exit\"!\n")
loop()
loop()
The error it throws is:
TypeError: string indices must be integers
Edit:
Updated now and it does call some information however it only calls the first dictionary entry whereas I need it to call as many as there are (and preferably have a count variable sometimes).
I'm also having trouble selecting the "DataClasses" entry and printing the individual entities within.
All help is appreciated, thanks.
To convert a json string to dictionary, use json module (standard library):
import json
data_str = '{"index" : 5}'
json_dict = json.loads(data_str)
In your example:
import json
# ...
def DataGetCurrent(Username):
SearchURL = "https://haveibeenpwned.com/api/v2/breachedaccount/{}".format(Username)
request = url.urlopen(url.Request(SearchURL, headers={'User-Agent' : "Mozilla/5.0"}))
data = request.read()
data = data.decode("utf-8")
return json.loads(data)
EDIT
Apparently HaveIBeenPwned returns a list of dictionaries. Therefore, to get the results, you need to get the dictionary in the 0th index of the list:
def DataGetCurrent(Username):
SearchURL = "https://haveibeenpwned.com/api/v2/breachedaccount/{}".format(Username)
request = url.urlopen(url.Request(SearchURL, headers={'User-Agent' : "Mozilla/5.0"}))
data = request.read()
data = data.decode("utf-8")
json_list = json.loads(data)
return json_list[0]
EDIT 2
0th element of the list is only one of the results. To process all the results, the list itself should be returned and used accordingly.

Categories

Resources