Since I am using a for loop to request a URL which changed with a constant number, how can I parsed the variable in?
total_page = 4
for y in range (total_page):
variable = 20*y+1
base_url = 'https://abcd.com/r='
url = ''.join([base_url, variable])
finviz1 = requests.get(url)
However, an error occurred
url = ''.join([base_url, variable])
TypeError: sequence item 1: expected string or Unicode, int found
How to eliminate the error?
It expects all elements of the array to be a string, whereas you are passing variable as an integer. Convert it into a string before passing it, like this:
url = ''.join([base_url, str(variable)])
Related
I'm using Python \ Selenium \ Chrome driver to perform webscraping. I want to pass an INT variable (id) to a URL - how do I do this? I have tried all the below but Python errors on this line:
id = 2000
# Part 1: Customer:
#urlg = 'https://mythirteen.co.uk/customerRest/show/?id=2000' working but need to pass variable
#urlg = 'https://mywebsite.com/customerRest/show/?id=' %(id)
#urlg = 'https://mywebsite.com/customerRest/show/?id={id}'
# urlg = 'https://mywebsite.com/customerRest/show/?id='.format(id)
# urlg = 'https://mywebsite.com/customerRest/show/?id='+id
# urlg = "https://mywebsite.com/customerRest/show/?id=".format(id)
# urlg = 'https://mywebsite.com/customerRest/show/?id=' % id
driver.get(urlg)
I receive errors such as:
TypeError: not all arguments converted during string formatting
I know it its not a string though - id is INT.
Ultimately, I will need to loop through and increase the id + 1 each time, but for now I just want to be able pass the actual variable.
If you want to concatenate strings and integer, for most methods you have to cast the integer to a string with str(id).
Otherwise I really like using f-strings:
urlg = f'https://mywebsite.com/customerRest/show/?id={id}'
Edit
As #chitown88 mentioned, using id as a variable is not a good idea, as it is reserved internally. Better use something like customer_id.
The problem with the methods you are trying is you essentially aren't telling it where to put the string, or trying to concat a string and an int
so this 'https://mywebsite.com/customerRest/show/?id=' %(id) needs to be 'https://mywebsite.com/customerRest/show/?id=%s' %(id)
So all these would work:
Also, I would not use id as the variable, as it's a reserved function in python. It also could make more sense to make it more descriptive:
customerId = 2000
urlg = 'https://mythirteen.co.uk/customerRest/show/?id=2000' working but need to pass variable
urlg = 'https://mywebsite.com/customerRest/show/?id=%s' %(customerId)
urlg = 'https://mywebsite.com/customerRest/show/?id=%s' %customerId
urlg = f'https://mywebsite.com/customerRest/show/?id={customerId}'
urlg = 'https://mywebsite.com/customerRest/show/?id={}'.format(customerId)
urlg = 'https://mywebsite.com/customerRest/show/?id='+str(customerId)
I have this api : https://api.publicapis.org/entries
And I wat to iterate key entries of it. I tried as it follows :
r = requests.get('https://api.publicapis.org/entries')
entries = [] #initializing the vector entries
for i in entries: #iterating it
return(i.text) #trying to print the entries
Then I received the following error :
TypeError: The view function did not return a valid response. The function either returned None or ended without a return statement.
How can I solve this problem ?
For that particular API endpoint, you should be fine with
resp = requests.get('https://api.publicapis.org/entries')
resp.raise_for_status() # raise exception on HTTP errors
entries = resp.json()["entries"] # parse JSON, dig out the entries list
# ... do something with entries.
You can use json.loads to parse response text.
Let me add the full code.
import requests
import json
r = requests.get('https://api.publicapis.org/entries')
entries = json.loads(r.text)['entries'] #initializing the vector entries
for i in entries: #iterating it
API = i['API']
Description = i['Description']
Im trying to loop through a list and insert the 'extensionid' into the URL
extensionid = ['1234','12356']
url = '/restapi/v1.0/account/~/extension/'+extensionid+'/presence'
params = {
'dndStatus': "TakeAllCalls",
}
resp = platform.get(url, params,)
print ((resp.text()))
But I get the error
url = '/restapi/v1.0/account/~/extension/'+extensionid+'/presence'
TypeError: can only concatenate str (not "list") to str [Finished in
1.121s
What am I doing wrong?
Thanks!
You probably need.
extensionid = ['1234','12356']
url = '/restapi/v1.0/account/~/extension/{}/presence'
params = {
'dndStatus': "TakeAllCalls",
}
for id in extensionid: #iterate each id
resp = platform.get(url.format(id), params) #form URL and query.
print ((resp.text()))
extensionid is a list of strings.
So, you cannot concatenate a list with a string as error is telling you.
To access a specific item of the list you have to use an index.
For example:
extensionid[0] // it contains 1234
extensionid[1] // it contains 12356
So, your url can be written like this:
url = '/restapi/v1.0/account/~/extension/'+extensionid[0]+'/presence'
In this case it will be evaluated by python as:
url = '/restapi/v1.0/account/~/extension/1234/presence'
Please consider this simple documentation about lists:
https://www.programiz.com/python-programming/list
To iterate the elements of a list you can use:
for element in extensionid:
url='/restapi/v1.0/account/~/extension/'+ element +'/presence'
print(url)
url = 'https://www.shanbay.com/read/article/97936/'
temp = filter(str.isdigit,url) #slect numbers
Instead of using for...in expression, can I convert temp to str directly?
filters are lazy in Python 3, you'll need to feed them to a function that will consume them and force them to produce their elements.
url = 'https://www.shanbay.com/read/article/97936/'
temp = filter(str.isdigit,url) # filter object
If you feed this to another function that consumes it, you can get back the elements. If you need a string, just join it:
nums = "".join(temp)
"".join will get all the elements from the filter instance and join them on the empty string "" producing 97936; use that in an int call if you require an int result.
filter returns an iterator, so you'd have to join its elements.
url = 'https://www.shanbay.com/read/article/97936/'
temp = "".join(filter(str.isdigit,url))
How about re.findall()?
import re
url = 'https://www.shanbay.com/read/article/97936/'
temp = re.findall('[0-9]+', url)
I am trying to decode a JSON with Python. Here is a little snippet of what the JSON looks like.
b'{"success":true,"data":[{"id":26,"name":"A","comment":"","start_time_plan":null,"start_time_actual":"2016-09-13 00:00:00","start_time_delta":null,"start_time_score":null,"start_time_score_achievement":null,"start_time_traffic_light":null,"end_time_plan":null,"end_time_actual":"2016-09-13 00:00:00","end_time_delta":null,"end_time_score":null,"end_time_score_achievement":null,"end_time_traffic_light":null,"status":0,"measure_schedule_revision_id":63,"responsible_user_id":3,"created_time":"2016-09-13 11:29:14","created_user_id":3,"modified_time":"2016-09-21 16:33:41","modified_user_id":3,"model":"Activity"},{"id":27,"name":"B","comment":"","start_time_plan":null,"start_time_actual":"2016-09-13 00:00:00","start_time_delta":null,"start_time_score":null,"start_time_score_achievement":null,"start_time_traffic_light":null,"end_time_plan":null,"end_time_actual":"2016-09-13 00:00:00","end_time_delta":null,"end_time_score":null,"end_time_score_achievement":null,"end_time_traffic_light":null,"status":0,"measure_schedule_revision_id":63,"responsible_user_id":3,"created_time":"2016-09-13 11:29:48","created_user_id":3,"modified_time":"2016-10-16 18:14:36","modified_user_id":1,"model":"Activity"}
I am trying to get a hold of start_time_deltaand end_time_delta and produce a little scatter plot. But somehow I can't decode the JSON.
Here is what I do:
#falcon api
u = 'https://myurl.com'
#urllib3 + poolmanager for requests
import urllib3
http = urllib3.PoolManager()
import json
r = http.request('GET', u)
json.loads(r.data.decode('utf-8'))
end = json.loads(r.data['end_time_delta'])
start = json.loads(r.data['start_time_delta'])
This is the error I get: byte indices must be integers or slices, not str
How come? And how do I solve the problem?
You are ignoring the return value of json.loads() here:
json.loads(r.data.decode('utf-8'))
You then try to decode the same raw again and try to use that as the decoded Python result. Call json.loads() once, and use the resulting Python dictionary:
result = json.loads(r.data.decode('utf-8'))
start = result['data'][0]['start_time_delta']
end = result['data'][0]['end_time_delta']
Because top-level dictionary 'data' key points to a list of results, I used 0 to get to the first of those and extract the data you want.
If you need to extract those data points for every dictionary in that list, you'd have to use a loop:
for entry in result['data']:
start = entry['start_time_delta']
end = entry['end_time_delta']
# do something with these two values, before iterating to the next