i am trying to get my image hosted online and for that i am using python
import requests
url = 'http://imgup.net/'
data = {'image[image][]':'http://www.webhost-resources.com/wp-content/uploads/2015/01/dedicated-hosting-server.jpg'}
r = requests.post(url, files=data)
i am not able to get the response url of the hosted image from the response .
Please help !
The files parameter of requests.post needs a:
Dictionary of 'name': file-like-objects (or {'name': ('filename', fileobj)}) for multipart encoding upload.
There's more data you'll need to send than just the file, most importantly the "authenticity token". If you look at the source code of the page, it'll show you all other parameters as <input type="hidden"> tags.
The upload URL is http://imgup.net/upload, as you can see from the action attribute of <form>.
So what you need to do is:
Download the image you want to upload (I'll call it dhs.jpg).
Do a GET request of the main page, extracting the authenticity_token.
Once you have that, send the request with files= and data=:
url = "http://imgup.net/upload"
data = {'utf8': '✓', 'authenticity_token': '<put your scraped token here>', '_method': 'put'}
f = open("dhs.jpg", "rb") # open in binary mode
files = {'image[image][]': f}
r = requests.post(url, files=files, data=data)
f.close()
print(r.json()["image_link"]
Final note: While I couldn't find any rule against this behaviour in their T&C, the presence of an authenticity token makes it seem likely that imgup doesn't really want you to do this automatically.
Related
;TLDR
I want to send a file with requests.send() using multipart/form-data request without storing the file on a hard drive. Basically, I'm looking for an alternative for open() function for bytes object
Hello, I'm currently trying to send multipart/form-data request and pass in-memory files in it, but I can't figure out how to do that.
My app receives images from one source and sends them to another. Currently it sends get request directly to file, (e.g. requests.get('https://service.com/test.jpeg')), reads image's bytes and writes them into new file on the hard drive. The sending code that works looks like this:
def send_file(path_to_image: str)
url = get_upload_link()
data = {'photo': open(path_to_image, 'rb')}
r = requests.post(url, files=data)
send_file("test.jpeg")
The main issue I have with this approach is that I have to keep files on my hard drive. Sure, I can use my drive as some sort of a "temporary buffer" and just delete them after I no longer need these files, but I believe there's much more simple way to do that.
I want my function to receive bytes object and then send it. I actually tried doing that, but the backend doesn't accept them. Here's what I tried to do
Attempt 1
def send_file(image: bytes)
url = get_upload_link()
data = {'photo': open(image, 'rb')}
r = requests.post(url, files=data)
I get "ValueError: embedded null byte"
Attempt 2
def upload_photo(image: bytes):
url = get_upload_link()
file = BytesIO(image)
data = {'photo': file}
r = requests.post(url, files=data)
Backend server doesn't process my files correctly. It's like passing files=None, same response
I also tried:
sending the returning value of the methods: file.getbuffer() and file.read()
file.write(image) and then sending file
StringsIO object
etc.
Final notes
I noticed, that open() returns _io.BufferedReader object. I also looked for a way to construct its instance, but couldn't fund a way. Can someone help me, please?
UPD:
If anyone is interested, the receiving api is this
From the official documentation:
POST a Multipart-Encoded File
...
If you want, you can send strings to
be received as files:
url = 'https://httpbin.org/post'
files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}
r = requests.post(url, files=files)
I'm trying to download some videos from a website using Selenium.
Unfortunately I can't download it from source cause the video is stored in a directory with restricted access, trying to retrieve them using urllib, requests or ffmpeg returns a 403 Forbidden error, even after injecting my user data to the website.
I was thinking of playing the video in its entirety and store the media file from cache.
Would it be a possibility? Where can I find the cache folder in a custom profile? How do I discriminate among files in cache?
EDIT: This is what I attempted to do using requests
import requests
def main():
s = requests.Session()
login_page = '<<login_page>>'
login_data = dict()
login_data['username'] = '<<username>>'
login_data['password'] = '<<psw>>'
login_r = s.post(login_page)
video_src = '<<video_src>>'
cookies = dict(login_r.cookies) # contains the session cookie
# static cookies for every session
cookies['_fbp'] = 'fb.1.1630500067415.734723547'
cookies['_ga'] = 'GA1.2.823223936.1630500067'
cookies['_gat'] = '1'
cookies['_gid'] = 'GA1.2.1293544716.1631011551'
cookies['user'] = '66051'
video_r = s.get(video_src, cookies=cookies)
print(video_r.status_code)
if __name__ == '__main__':
main()
The print() function returns:
403
This is the network tab for the video:
Regarding video_r = s.get(video_src, cookies=cookies) Have you try to stream the response ? which send correct byte-range headers to download the video. Most websites prevent downloading the file as "one" block.
with open('...', 'wb') as f:
response = s.get(url=link, stream=True)
for chunk in response.iter_content(chunk_size=512):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
You can send a head request before if you want, in that way you can create a progress bar, you will retrieve the full content length from header.
Also a 403 is commonly use by anti-bot system, may be your selenium is detected.
You blocked because you forgot about headers.
You must use:
s.get('https://httpbin.org / headers', headers ={'user-agent': <The user agent value (for example: last line of your uploaded image)>})
or:
s.headers.update({'user-agent': <The user agent value (for example: last line of your uploaded image)>})
before sending a request
I am having a real headache with the way of sending a pdf file to a Telegram Bot.
Apparently I am following the documentation but never get it sent.
I am using the url: https://api.telegram.org/botBOTID/sendDocument?chat_id=CHATID&document=/home/lix/Downloads/2.pdf
It is a pdf file storaged locally, but I think it is just the way I am presenting it.
The error getting is:
{"ok":false,"error_code":400,"description":"Bad Request: URL host is empty"}
Does anybody knows how to send a pdf local file?
Many thanks
You should send a POST request, with the PDF as a payload, using the Python Requests library, your code should look something like this:
import requests
# Url with bot token + user id
url = "https://api.telegram.org/bot<MY-BOT-TOKEN>/sendDocument?chat_id=<MY_CHAT_ID>"
# Create payload with PDF file
payload = {}
files = [
('document', open('/home/lix/Downloads/2.pdf','rb'))
]
headers= {}
# Request
response = requests.request("POST", url, headers=headers, data = payload, files = files)
# Log reponse as UTF-8
print(response.text.encode('utf8'))
I'm trying to scrape image URLs from a website and display the images on another site (using BeautifulSoup), but the website in question (yupoo.com) has some sort of protection against loading images from their server if you don't browse their site.
How to reproduce my problem:
You can't load this image:
https://photo.yupoo.com/0832club_v/0058b582/96f83ddb.jpeg
Now visit this site: https://0832club.x.yupoo.com/29611853?uid=1
Now open the link above
"https://photo.yupoo.com/0832club_v/0058b582/96f83ddb.jpeg" and for
some reason it now works...
I checked for cookies and stuff, but I seriously don't understand how they protect their images
You have to send header referer and then server think that it is load from page https://0832club.x.yupoo.com/29611853?uid=1
import requests
url = 'https://photo.yupoo.com/0832club_v/0058b582/96f83ddb.jpeg'
headers = {
'referer': 'Referer: https://0832club.x.yupoo.com/29611853?uid=1'
}
r = requests.get(url, headers=headers)
print(r.content[:100]) # you can see string `JFIF` or `GIF` in content
f = open('output.jpg', 'wb')
f.write(r.content)
f.close()
With referer I see string JFIF in content so it sends JPG. Without referer I see string GIF in content so it sends GIF
You can also check
print(r.headers['Content-Type'])
and with referer it returns image/jpeg, without referer it returns image/gif
I am using Python's Requests library to POST a PDF to a document store, the uploaded PDF is thereafter used in a signature process. However when uploading the PDF using Python (in stead of CURL) the signing environment doesnt work. On comparing different files, I found out that Requests adds some data to the PDF:
--ca9a0d04edf64b3395e62c72c7c143a5
Content-Disposition: form-data; name="LoI.pdf"; filename="LoI.pdf"
%%Original PDF goes here%%
--ca9a0d04edf64b3395e62c72c7c143a5--
This data is accepted perfectly fine by different PDF readers, but not by the Signature API. Is there a way to prevent Requests from adding this data to the PDF? I used the following code:
myfile = request.FILES['myfile']
url = %%documentstoreURL%%
resp = requests.request('post', url, files={myfile.name:myfile}, headers={'Content-Type':'application/pdf'}, auth=(%%auth details%%))
Thanks!
You're sending the file as binary data with curl, but attaching it in requests.
I read over the source code, and I believe resp = requests.request('post', url, data={myfile.name:myfile}, headers={'Content-Type':'application/pdf'}, auth=(%%auth details%%)) (data instead of files) will avoid the multipart encoding.
At the very least, it should be differently broken.
Being guided in the right direction, I found a working solution based on Python requests - POST data from a file
In the end I did it as follows:
myfile = request.FILES['myfile']
payload = request.FILES['myfile'].read()
headers = {'content-type': 'application/pdf'}
url = "%%DocumentServiceURL"
r = requests.post(url, auth=(%%auth_details%%), data=payload, verify=False, headers=headers)