Develop Reference

Cannot scrape using python.requests() but work when loading on browser

I want to scrape data from this page: https://raritysniffer.com/viewcollection/primeapeplanet
The API request works on the browser but returns 403 ERROR when I use python.requests.
requests.get("https://raritysniffer.com/api/index.php?query=fetch&collection=0x6632a9d63e142f17a668064d41a21193b49b41a0&taskId=any&norm=true&partial=true&traitCount=true")
I understand it is possible that I have to pass on specific headers to make it work, but as a python novice, I have no idea how to make it work. Please advise. Thanks!

If you check the response, you can see that the website uses Cloudfare and which indeed returns the 403. To bypass this, try cloudscraper. (be mindful)
import cloudscraper
url = 'https://raritysniffer.com/api/index.php?query=fetch&collection=0x6632a9d63e142f17a668064d41a21193b49b41a0&taskId=any&norm=true&partial=true&traitCount=true'
scraper = cloudscraper.create_scraper(browser = 'firefox')
print(scraper.get(url).text)

How to start with the InstagramAPI in Python?

i want to play with the InstagramAPI and write some code for like getting a list of my follower and something like that. I am really new to that topic.
What is the best way to do this? Is there a Python-Lib for handle those json request or should I send them directly to the (new? graphAPI, displayAPI) InstagramAPI?
Appreciate every advice I can get. Thanks :)

LevPasha's Instagram-API-python, instabot, and many other API's are no longer functional as of Oct 24, 2020 after Facebook deprecated the legacy API and now has a new, authentication-required, API. It now requires registering your app with Facebook to be able to get access to many of the API features (via oembed) that were previously available without any authentication.
See https://developers.facebook.com/docs/instagram/oembed/ for more details on the new implementation and how to migrate.
You should still be able to get a list of your followers, etc. via the new oEmbed API and python--it will require registering the app, making a call to the new GET API with your authentication key via the python requests package, and then processing the result.

There is one library called instabot. This useful library has all the necessary functions/methods to interact with your insta account. Read its documentation here.
The pip installation is: pip install instabot
To get started with, lets say you want to simply login to your account.
from instabot import Bot
bot = Bot()
bot.login(username="YOUR USERNAME", password="YOUR PASSWORD")
To get the list of your followers,
my_followers = bot.followers()
If you want to upload a photo or get your posts,
bot.upload_photo(image, caption="blah blah blah") #the image variable here is a path to that image
all_posts = bot.get_your_medias() #this will return all of your medias of the account
#to get the info of each media, use
for post in all_posts:
print(bot.get_media_info(post))
and there are many other functions/methods available in this library.
It actually very fun to interact with instagram using python. You will have a great time. Enjoy :)

You can use https://github.com/LevPasha/Instagram-API-python to call Instagram APIs
and also if you want to call API directly You can use the requests package.
It also supports graphql APIs.
Here you can see an example:
https://gist.github.com/gbaman/b3137e18c739e0cf98539bf4ec4366ad

It seems like in 2022 this is the only active working and maintained python solution:
https://github.com/adw0rd/instagrapi

How to post a message to Stocktwits through python script?

I have created an app, and have the token, the keys, and all that information from stocktwits.com. I want to know how i can integrate code into my python script where i am able to post a message to stocktwits via Python.
Thanks for your help.
Regards,
PM

Using Python's request module, you could make a request like this:
import requests
payload={"access_token":"YOURACCESSTOKEN", "body":"YOURMESSAGE"}
requests.post("https://api.stocktwits.com/api/2/messages/create.json", data=payload)
Be sure to look at Stocktwit's auth docs so you can make sure that you have the right API keys.

Is python-oauth2 the right tool for my issue?

I'm coding an app which has to use this api. So I want to do at a certain point a search on their database. Now I'm struggling with which python library is the right one to use in order to authenticate about oAuth2? I couldn't find any by now, where I was sure, it would offer the necessary functions.
I wonder if this library (python-oauth2) offers, what I need. But this isn't a library for the client, is it? It seems it is for the server...
I'd be really grateful, if someone could just give me an advice, with what I should work.

Method 1
You will need to use the following modules. No need to use oauth. Just need to get the token before performing any search using the api.
requests, json, urllib
Here's a short Example code for that
import requests, json, urllib
BASE_URL = "http://scoilnet.com/grants/apikey/"
r = requests.post(_BASE_URL+"user/token/", data={'username': username , 'password': password })
print r.getcontent
The above code will show you how to request a token from the api. Using that token you will be making get and post requests to the api which will give a json response. That json response will be shown as a Dictionary from which you will load you data in your program.
Method 2
You can also use urllib or urllib2 or urllib3

GData and OAuth in Python: Unable to convert request token to access token

I am trying to implement a button on a web-based dashboard that allows a user to export the current data to a Google Spreadsheet using OAuth and GData API. Currently, I can get the user to a login/grant access page, but if I add the line to convert the request token to an access token, I receive:
"RequestError: Unable to upgrade OAuth request token to access token: 400, parameter_absent
oauth_parameters_absent:oauth_token"
I am following the instructions for OAuth 2 on this page:
https://developers.google.com/gdata/docs/auth/oauth
and have read both PyDocs for the Google APIs and found no details on this issue:
http://gdata-python-client.googlecode.com/hg/pydocs/gdata.docs.client.html#DocsClient
(Won't let me post a this hyperlink but other Pydoc is same URL but replace the piece after pydocs/ with gdata.gauth.html#ClientLoginToken)
This is the code that works:
def createDocsClient(self, oauth_callback_url):
docsClient = gdata.docs.client.DocsClient(source='RiskOps-QualityDashboard')
request_token = docsClient.GetOAuthToken(SCOPES, oauth_callback_url, CONSUMER_KEY, consumer_secret=CONSUMER_SECRET)
domain = None
auth_url = request_token.generate_authorization_url(google_apps_domain=domain)
self.redirect(str(auth_url))
request_token = gdata.gauth.AuthorizeRequestToken(request_token, self.request.uri
With the above code, I get to a grant access page and if you click the grant access page, you get a 404 error because it doesn't know where to go after (as expected), but the page has the proper URL displayed listing an oauth_verifier and oauth_token. The "AuthorizeRequestToken" line is supposed to use that URL to authorize the token so up to this line, everything seems to work.
When I add the following line right after the code above, I get the "RequestError" I wrote about:
access_token = docsClient.GetAccessToken(request_token)
I've tried different combinations of nesting the calls within each other, using the AeSave and AeLoad (as the instructions mention might be needed but I'm not sure if my case calls for it) and many other random and unsuccessful ideas and nothing is really giving me a good idea of what I'm missing or doing wrong.
Would really appreciate and help or any ideas anyone has.(If you can't tell, I'm fairly inexperienced when it comes to real-world code (as opposed to academic code). Thanks so much.