I have a list of Url of files that open as Download Dialogue box with an option to save and open.
I'm using the python requests module to download the files. While using Python IDLE I'm able to download the file with the below code.
link = fileurl
r = requests.get(link,allow_redirects=True)
with open ("a.torrent",'wb') as code:
code.write(r.content)
But when I use this code along with for loop, the file which gets downloaded is corrupted or says unable to open.
for link in links:
name = str(links.index(link)) ++ ".torrent"
r = requests.get(link,allow_redirects=True)
with open (name,'wb') as code:
code.write(r.content)
If you are trying to download a video from a website, try
r = get(video_link, stream=True)
with open ("a.mp4",'wb') as code:
for chunk in r.iter_content(chunk_size=1024):
file.write(chunk)
Related
I am trying to download a file using python from a URL. However its not working and instead I am getting index.html.
Please help on same.
import requests
target_url = "https://transparency-in-coverage.uhc.com/?file=2022-07-01_United-HealthCare-Services_Third-Party-Administrator_EP1-50_C1_in-network-rates.json.gz&origin=uhc"
filename = "2022-07-01_United-HealthCare-Services_Third-Party-Administrator_EP1-50_C1_in-network-rates.json.gz"
with requests.get(target_url, stream=True) as r:
r.raise_for_status()
with open(filename, "wb") as f:
for chunk in r.iter_content(chunk_size=1024):
f.write(chunk)
That's because you're the URL you specified is for an HTML page that subsequently starts the download for the .gz file you want.
This is the link for the file:
https://mrfstorageprod.blob.core.windows.net/mrf-even/2022-07-01_ALL-SAVERS-INSURANCE-COMPANY_Insurer_PS1-50_C2_in-network-rates.json.gz?sv=2021-04-10&st=2022-07-05T22%3A19%3A13Z&se=2022-07-09T22%3A19%3A13Z&skoid=89efab61-5daa-4cf2-aa04-ce3ba9d1d1e8&sktid=db05faca-c82a-4b9d-b9c5-0f64b6755421&skt=2022-07-05T22%3A19%3A13Z&ske=2022-07-09T22%3A19%3A13Z&sks=b&skv=2021-04-10&sr=b&sp=r&sig=NaLrw2KG239S%2BpfZibvw7%2B25AAQsf9GYZ1gFK0KRN20%3D&rscd=attachment
To find it, you need to have the inspector open on the 'Network' tab whilst loading the page (or you can click on the file in the list when it loads the list of files on the page). When the download starts you'll see two files pop-up, one of which is the actual URL of the .gz file.
It does look the URL has a timestamp in it, so it might not work at a later time, I don't know.
Using a simple code to download zip files
import requests
def download_url(url, save_path, chunk_size=128):
r = requests.get(url, stream=True)
with open(save_path, 'wb') as fd:
for chunk in r.iter_content(chunk_size=chunk_size):
fd.write(chunk)
url = 'https://www1.nseindia.com/content/historical/EQUITIES/1994/NOV/cm03NOV1994bhav.csv.zip'
save_path = 'D:/folder/Programming/Python/trading/Bhavcopy/bhavcopy.csv.zip'
download_url(url,save_path)
The end result is the creation of an invalid zip file. I tried to open the website by manually pasting the url on browser and got this
But when I open link via the original website i.e going the nse website and clicking button to download, the link works.
Additional data
Here is the link from where you try downloading the file for yourself. https://www1.nseindia.com/products/content/equities/equities/archieve_eq.htm
I'm donwloading the files from first option(Bhavcopy) for the first date for which it is available (3rd Nov 1994)
You need to send referer headers:
headers = {'Referer':'https://www1.nseindia.com'}
...
r = requests.get(url, stream=True,headers=headers)
I want to download a file, there is a hyper link in html page which does not include the file name and extension. How can I download the file using python?
For example the link is http://1.1.1.1:8080/tank-20/a/at_download/file,
but whenever I click on it the file will download and open with browser.
Use python requests to get the body of the response and write to file, this is essentially what the browser is doing when you click the link.
Try the below:
import requests
# define variables
request_url = "http://1.1.1.1:8080/tank-20/a/at_download/file"
output_file = "output.txt"
# send get request
response = requests.get(request_url)
# use 'with' to write to file
with open(output_file, 'w') as fh:
fh.write(response.content)
fh.close()
I've downloaded some files using requests
url = 'https://www.youtube.com/watch?v=gp5tziO5lXg&feature=youtu.be'
video_name = url.split('/')[-1]
print("Downloading file:%s" % video_name)
# download the url contents in binary format
r = requests.get(url)
# open method to open a file on your system and write the contents
with open('saved.mp4', 'wb') as f:
f.write(r.content)
and using urllib.requests
url = 'https://www.youtube.com/watch?v=gp5tziO5lXg&feature=youtu.be'
video_name = url.split('/')[-1]
print("Downloading file:%s" % video_name)
# Copy a network object to a local file
urllib.request.urlretrieve(url, "saved2.mp4")
When I then try to open the .mp4 file I get the following error
Cannot play
This file cannot be played. This can happen because the file type is
not supported, the file extension is incorrect or the file is
corrupted.
0xc00d36c4
If I test it with pytube it works fine.
What's wrong with the other methods?
To answer your question, with the other methods it is not downloading the video but the page. What you may be obtaining is an html file with an mp4 file extension.
Therefore, it gives that error when trying to open the file.
If pytube works for what you need, I would suggest using that one.
If you want to download videos from other platforms, you might consider youtube-dl.
Hello you can import IPython.display for audio diplay
import IPython.display as ipd
ipd.Audio(video_name)
regards
I hope I can have solved your problem
Perhaps I'm misunderstanding how .torrent files work, but is there a way in python to download the actual referenced content the .torrent file points to that would be downloaded say using a torrent client such as utorrent, but from the shell/command line using python?
The following works for simply downloading a .torrent file and sure I could open the torrent client as well do download the .torrent, but I'd rather streamline the process in the command line. Can't seem to find much online about doing this...
torrent = torrentutils.parse_magnet(magnet)
infohash = torrent['infoHash']
session = requests.Session()
session.headers.update({'User-Agent': 'Mozilla/5.0'})
url = "http://torcache.net/torrent/" + infohash + ".torrent"
answer = session.get(url)
torrent_data = answer.content
buffer = BytesIO(torrent_data)
gz = gzip.GzipFile(fileobj = buffer)
output = open(torrent['name'], "wb")
output.write(torrent_data)
As far as I know i can't use libtorrent for python3 on a 64 bit windows os.
If magnet: links work in your web browser then a simple way to start a new torrent download from your Python script is to open the url using your web browser:
import webbrowser
webbrowser.open(magnet_link)
Or from the command-line:
$ python -m webbrowser "magnet:?xt=urn:btih:ebab37b86830e1ed624c1fdbb2c59a1800135610&dn=StackOverflow201508.7z"
The download is performed by your actual torrent client such as uTorrent.
BitTornado works on Windows and has a command line interface. Take a look at btdownloadheadless.py. But this written in Python 2.
http://www.bittornado.com/download.html