Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I'm trying to make a simple post method with the requests module, like this :
s=requests.Session()
s.post(link,data=payload)
In order to do it properly, the payload is an id from the page itself, and it's generated in every access to the page.
So I need to POST without reloading the entire webpage so i can get the right id .
Is this code in a view?
If yes, I'd advise you to create an Ajax view receiving this POST and send the payload from the page using javascript as Ajax.
This way on success, you can get the right id in the response of that view without reloading the page.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I am trying to scrape this website with scrapy and I have had to search for each link extracting the information from each one, I would like to know if there is an API of the site that I can use (I don't know how to find it).
I would also like to know how I can obtain the latitude and longitude? Currently the map is shown but I do not know how to obtain the numbers
I appreciate any suggestions
The website may be loading the data dynamically using Javascript. Use your browser dev tools and look at the networking tab, look for any XHR calls which may be accessing an API. Then you can scrape from that directly.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Is there a way to scrape data from a popup? I'd like to import data from the site tennisinsight.com.
For example, http://tennisinsight.com/match-preview/?matchid=191551201
This is a sample data extraction link. When clicking "overview" there is a button with "Match Stats", I'd like to be able to import those data from many links in a text or CSV file.
What's the best way to accomplish this? Is Scrapy able to do this? Is there software able to do this?
You want to open the network analyzer in your browser (e.g. in Web Developer in Firefox) to see what requests are sent when you click the "match stats" button in order to replicate them using python.
When I do it, a POST request is sent to http://tennisinsight.com/wp-admin/admin-ajax.php with action and matchID parameters.
You presumably already know the match ID (see URL you posted above), so you just need to set up a POST request for each matchID you have.
import requests
r = requests.post('http://tennisinsight.com/wp-admin/admin-ajax.php', data={'action':'showMatchStats', 'matchID':'191551201'})
print r.text #this is your content of interest
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I want to create a little program with following features.
Use proxy in format proxy:port:username:password
Choose a proxy sequentialially from list
Open http://example.com
Fill Details choosing data from data.txt ( CSV )
Export Cookie,username,password,email address --> cookie.txt
Delete Cookies
Log into associated email account and confirm account by visiting
link sent to that email address.
Then cycle through Step1 again.
I read several similar question on stackoverflow.
I planned to use Selenium for this program, but reading comment here How to save and restore all cookies with Selenium RC?
the get_cookie method doesn't provide the path, domain, and expiry
date for each cookie, so it isn't possible to fully restore those
parameters with create_cookie. any other ideas
And i won't be able to manipulate cookies using method as describe here http://hub.tutsplus.com/tutorials/how-to-build-a-python-bot-that-can-play-web-games--active-11117
I want to know easiest way to tackle this problem. I plan to run single threaded application.
I don't know selenium, but why not use mechanize and requests ? Both are awesome.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I need to design a API where a request is made to the URL and in turn return a response.
Now i do not want to use any framework as such.
So my question is how can i make this happen.I want to send a request from the web browser and get the response.I have the apache server setup.So where should my python code leave to receive the request and how to respond back
EDIT
import urllib2
request = urllib2.Request('http://localhost/')
request.get_method = lambda : 'HEAD'
response = urllib2.urlopen(request)
response.info().gettype()
Try Flask, it's a microframework and setting it up should be a one-liner.
http://flask.pocoo.org/
Here're the docs for configuring Apache to run it:
http://flask.pocoo.org/docs/deploying/mod_wsgi/
I know you said no frameworks, but this is a lot more simple than doing it yourself.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Is there a Python program where you can post a status update without going to the website itself?
Where or how do i get started?
You would wanna get started here, https://github.com/facebook/python-sdk
unless there's a library I don't know about.
You can checkout my fork of the python sdk. It's customized to work better with django.
You can check here on how to use it to authenticate a user.
After that you can use the python sdk to post a news entry.
fb.put_wall_post("hello")
Or do it yourself, it's only a matter of doing a POST request to http://graph.facebook.com/FBID/feed where FBID is the ID of the page or profile you want to publish to, more info in the facebook doc.