Without having a twitter-account I can search tweets, e.g. with the URL https://twitter.com/search?q=stackoverflow or https://twitter.com/StackOverflow or https://twitter.com/hashtag/stackoverflow.
To access twitter using Python, there are several API wrappers, maybe the most famous ones being tweepy and twitter.
It does not seem to be possible to access tweets without authentication.
How can I access tweets from Python without logging in to twitter?
ps. I know that I can download the page and parse the html, but this looks like a last resort rather than an elegant solution.
From the docs
Authentication on all endpoints
We require applications to
authenticate all of their requests with OAuth 1.0a or Application-only
authentication. This visibility allows us to prevent abusive behavior,
and it also helps us to further understand how categories of
applications are using the API. We apply this understanding to better
meet the needs of developers as we continue to evolve the platform.
They need to apply rate limiting (and may be other security precautions) to minimize abuse, so, they do not allow public access to APIs.
Searching tweets using Twitter's website is manual. So there are less chances of abuse. Moreover, even if you try to parse it from web, most probably they will ban your IP address as soon as you start sending more requests than they think are not abusive.
Why not to create an account? Keep in mind different circumstances.
accounts are occassionally very hard to create. For instance, it has been reported that Twitter doesn't support phone numbers from certain locations. What would you do if you cannot authenticate with your personal mobile?
account features are heavily limited. For instance, under the standard dev account you will not retrieve past data for your educational project (e.g. a sentiment analysis of celebrities tweets).
accounts will not be free Twitter has just decided to discontinue supporting free accounts
Related
How does e-commerce usually handle integrations with ERP software?
We are working on a project for a client, who previously planned to use an ERP system that had a REST API.
This API allowed us to:
Place orders
Inform the ERP if the order was paid for
Get order status
Get all of the items available
Check item availability
Get user data
That would allow us to build a fairly complex online store with a lot of features.
Now the client wants to use another ERP system:
http://www.netsuite.com/portal/platform.shtml
I researched it, and the difficulty of integration surprised me. No REST API, some weird SOAP protocol to communicate with the system, and you have to write a lot of logic using SuiteScript. A whole new, different programming language just to build an integration with an online store? Why not just give developers access to an API to place orders and fetch items? And there are absolutely no docs available online for the thing. People on forums are saying that the system lacks in documentation and one has figure it out himself, along the way.
Magento and Shopify integration is done by third parties and looks dodgy. Same thing with SAP ERP. Am I missing something? Why is such a basic thing as a REST API for e-commerce not available for those systems?
Why develop using Python Django for the back-end and using React.js for the frontend. What is the right way to integrate them with the ERP system?
NetSuite does have a REST API and webservices. "you have to write a lot of logic using SuiteScript" is true but it's just JavaScript and there are many talented developers out there.
I'm not sure there is a "right way" but there are many ways to connect to the data.
My suggestion would be to contact a partner company, such as SWK Technologies. http://swktech.com
NetSuite has two main APIs, SuiteTalk and SuiteScript.
SuiteTalk is the Web Services API, which is SOAP based and allows for pulling data from and updating NetSuite. The SuiteScript API is JavaScript based and allows you to customize accounts and export data at the appropriate event during your business process. The term "SuiteCloud" encompasses all APIs and integration tools.
As for documentation, this is mostly only available to clients and partners. If you have a client who provides you with access to their account, you will gain access to the NetSuite Help Center and all relevant documentation.
Your options for integrating with the e-commerce platform depends on the exact platform. This ranges from Webhooks to HTTP requests.
You can't say NetSuite is delimiting developers in any way. It depends on how you look at it. As I see it, NetSuite provides two main method for developers - SuiteTalk and SuiteScript.By this, developer can create his/her own API, define what kind of acces those API should have.
SuiteTalk is SOAP based.
I would suggest using SuiteScript to create your own API using either NS RESTlet or NS Suitelet.
They have the feature for External URL. By sending request to this external URL you can trigger your own custom functions written on the SuiteScript. By SuiteScript, you can create your own API and define your own function. Ie, developer is in full control.
The only problem I see with NetSuite is its higher barrier for entry. There is no way you can access NetSuite Help Centre without having a Client/Partner/Test account.
But obviously, those who need some kind of integration with NetSuite have NS account.
I am new to web programming- I've recently been familiarizing myself with the webapp2 framework. I'm trying to start building a website, and would like users to login to the site with Facebook and I'll need access to their friends list. I've been trying to find a way to do this- I found out about OAUTH2, and I think this may be a way to do this. All the tutorials for python and OAUTH2 that I've found have been using the google API, I'm not sure if it's any different, but I haven't been able to get it to work.
Does anyone have sample code they can post that uses OAUTH2 (or anything else) to get users to sign in through Facebook? Or any good resources that can help me with this?
Your app needs to authorize users with Facebook, since there's where the resources you need are (e.g. friend lists).
This is a classic use of OAuth2 and you don't have a way around it, because FB implements this protocol.
My suggestion is that you look at the Google sample and then adjust it for FB API. The important changes are:
The endpoint URLs (e.g. authorize, token and user profile
The scopes that define the extent of permissions you are requesting (e.g. list of friends)
The user profile (e.g. the information returned by FB on a user: name, e-mail, etc)
This is a very simple sample that does this in Python. It was meant to run in Google App Engine. The only caveat is that it uses our own library to encapsulate the flow. But you can use it to study how the basic protocol works. Run the live demo and turn on dev tools on your browser to see the network activity.
You will notice that OAuth2 is a rather simple protocol, using simple HTTP requests.
I'm trying to understand OAuth, and I'm having a hard time figuring this basic thing out...
I have developed a service (with Python and Flask), which supports classic authentification through a dedicated login & password combination, and an "official" client in the form of a webapp. I would like my service to support OAuth and looked into flask-oauthprovider, which seems like a perfect fit for this task, but I can't seem to understand how everything should articulate.
My questions are:
Today, all my API entry points required the user to be logged in: once my service supports OAuth, should every entry points become "oauth_required" rather than "login_required"?
What is the proper way to support my "official" webapp front-end? I'd rather not have it go through the regular OAuth flow (with the extra redirections to login on the service). Should it go through OAuth with automatically granted access tokens, or should it bypass OAuth and directly use the "resource owner" login & password?
I think one of the problems with the concept behind oauthlib is that it tries too hard to be everything and the result is a difficult-to-reason-about set of abstractions (this is very similar to the python-oauth2 approach). OAuth providers in particular are tricky because you implicitly need to persist things like tokens not to mention the assumption of some kind of pre-exisiting user management. As such a "good" or idiomatic implementation tends to be more opinionated from framework to framework. In my opinion this is part of why we don't see a single Python OAuth provider implementation as an abstraction: there just aren't great solutions, but plenty of messy ones. Looking at flask-oauthprovider and we see some direct examples of these issues. I've had similar problems with flask-login, which I maintain. At any rate, this past weekend I wrote a very rough first pass of a OAuth provider in Flask that "just works"; feel free to take a look and adapt it to your needs. It assumes some things like, like MongoDB but with minimal work I think any datastore could be used.
1) Protect whichever endpoints you want to be accessible via a third-party, e.g. your public API.
2) I would avoid automatic access tokens, that defeats the person of negotiating authorization on a per-user basis, unless of course you have a different scheme, e.g. a predefined set of clients. I believe the second option you're talking about is xauth, in which case, why not just use OAuth 2.0 and grant_type=password? Bearer tokens are similar in concept but may be a little easier to implement so long as you can provide HTTPS.
Essentially the same problem as this question but looking for a solution in Python. How to work around Twitter OAuth?
Ideally, I dont want to have to go through the hoops of setting up a user/login interface and backend since the work I'm doing is for internal purposes.
I would also like to bypass the part where I need to re-direct the user to Twitter for authorization.
Thanks
You'll want to use Twitter's OOB flow. This is explained nicely in this answer
Twitter API - OOB Flow
So, reading between the lines a little, you have a twitter account and a password because this is internal, so you don't want to go with an auth process that requires a user to interact with it?
The idea behind OAuth is that you don't ever find out what the user's password is; I agree that if I'm right about what you are trying to do that it isn't the right thing. The OOB Flow suggested by JohnD has the same problem.
If you do have an account/password, then you can work with submissions to the website directly, using the login form and the tweet form. Unfortunately this means you don't have access to the API (they nuked basic authentication via the API last year) -- depending on what you're trying to do that may or may not be a problem.
Edit:
Use OAuth and remember the token. It never expires, according to the twitter API docs, and since you presumably have some limited number of accounts that you care about, you can just jump through the OAuth hoops once for each account and you're done until you need another account. You can even do the whole thing programmatically given the username and password, assuming they don't stick a captcha in there at some point. But I suspect your best bet is to just use OAuth and store the tokens.
I just found this bash script that works, tested personally, just change --ssl to --sslv3.
It's based on a simpler auth method used on mobile.twitter.com, you can use the same principle to deal with it using urllib2 and re modules.
Otherwise you can consider to lean against a site like http://www.supertweet.net/
SuperTweet.net provides a safe
mechanism to use Basic Authentication
with the Twitter API in your scripts
and other Twitter apps. Simply Sign-up
via Twitter to authorize the MyAuth
API Proxy SuperTweet.net Application
and then assign a password of your
choosing (not your real Twitter
password) that your applications can
use with the http://api.supertweet.net
API.
edit: I see now this site was cited in an article linked in an answer of How to work around Twitter OAuth?, if you already read about it ignore this part.
If you're using a desktop or mobile application, then you can use xAuth. From the user perspective it's the same as basic auth for getting the original OAuth credentials, and there's no going to external pages. Note you have to be approved by the Twitter API team to get xAuth access.
You might consider looking at Mechanize. It automates browser activity.
So you could give your username/password to your script. Then the script should pass on those credentials to http://twitter.com/#!/login.
conventionally, if you manually login from that webpage, the response will be another page based on whether the credentials you used were correct.
Same thing here: Based on whether the credentials are correct, the response is another page.
You can then check whether the response is a "login failed" page or a "login passed" page, and do what you need to do from there.
Hope this helps
I am building a website for a comedy group which uses Facebook as one of their marketing platforms; one of the requirements for the new site is to display all of their Facebook events on a calendar.
Currently, I am just trying to put together a Python script which can pull some data from my own Facebook account, like a list of all my friends. I presume once I can accomplish this I can move to pulling more complicated data out of my clients account (since they have given me access to their account).
I have looked at many of the posts here, and also went through the Facebook API documentation, including Facebook Connect, but am really beating my head against the wall. Everything I have read seems like overkill, as it involves setting up a good deal of infrastructure to allow my app to set up connections to any arbitrary user's account (who authorizes me). Shouldn't it be much simpler, given I only ever need to access 1 account?
I cannot find a way to retrieve data without having to display the Facebook login window. I have a script which will retrieve all my friends, but it includes a redirect where I have to physically log myself in to Facebook.
Would appreciate any advice or links, I just feel like I must be missing something simple.
Thank you!
Just posting up my notes on the successful advice, should others find this post;
Per Daniel and William's advice, I obtained the right permissions using the Connect options. From William, this link explains how the Facebook connection works
https://developers.facebook.com/docs/authentication/
This section on setting up the actual authentication was most helpful to me.
http://developers.facebook.com/docs/api
Basically, it goes as follows:
Post a link to the following URL. A user will need to physically click on it (even if that user is just you, the site admin).
https://graph.facebook.com/oauth/authorize?client_id=YOUR_CLIENT_ID&redirect_uri=http://www.example.com/HANDLER
This will redirect to a Facebook login, which will return to http://www.example.com/HANDLER after the user authenticates. If you wish to do more than basic reads and news feed updates you will need to include this variable in the above link: scope=offline_access,user_photos. The scope variable just includes a comma separated list of values, which Facebook will explicitly tell the authenticating user about during the login process, and they will have to OK. Most helpful for me was the offline_access flag (user_photos lets you get at their photos too), so I can pull content without someone logging in regularly (so long as I store the access token obtained later)
Have a script located at http://www.example.com/HANDLER that will take a variable from the request (so facebook will redirect to http://www.example.com/HANDLER&code=YOUR_CODE after authentication). Your handler needs to pull out the code variable, and then send the following request:
https://graph.facebook.com/oauth/access_token?
client_id=YOUR_CLIENT_ID&
redirect_uri=http://www.example.com/oauth_redirect&
client_secret=YOUR_SECRET_KEY&
code=YOUR_CODE
This request will return a string of the form access_token=YOUR_ACCESS_TOKEN.
Just parse off the 'access_token=', and you will have a token that you can use to access the facebook graph API, in requests like
http://graph.facebook.com/me/friends?access_token=YOUR_ACCESS_TOKEN
This will return a JSON object containing all of your friends
Hope this saves someone else some not fun time straining through documentation. Thanks for the help!
It is true, that Facebook's API is targeted at developers who are creating apps that will be used by many users.
Thankfully, the new Graph API is much simpler to use than its predecessor, and shouldn't be terribly difficult for you to work with without using or creating a lot of underlying infrastructure.
You will need to implement authorization, but this is not difficult, and as long as you prompt the user for the offline_access permission, it'll only need to be done once.
The documentation on Desktop Authentication would probably be most relevant to you at this point, though you might want to move to the javascript-based authentication once you've got a web app up and running.
Once the authentication is done, all you're doing is making GET requests to various urls and working with the resulting JSON.
Here's the documentation about Events, and you can get a list of friends from the friends connection of a User.
I'm not expert on Facebook/Facebook Connect, however I've seen it used/used applications with it and it seems there's really only the 'official' way to do it. I'm afraid it looks like your best bet would probably be something along the lines of this.
http://wiki.developers.facebook.com/index.php/Connect/Authentication_and_Authorization
Regardless of how you actually 'use' it, you'll still need to authorize the application to connect to the account and this means having a Facebook App as well.
The answer to Facebook application authentication is hard to find but is actually found within the "Analytics" page of the Graph API.
Specify the following: https://graph.facebook.com/oauth/access_token?client_cred&client_id=yourappid&client_secret=yourappsecret , you will then be given an access_token that you may use on all other calls.
The Facebook provided APIs do NOT currently provide this level of functionality.