Downloading a File from URL is a very common task in Python scripts. A real life example is to download images from a website to the local system and then process it in our Python program.
In this tutorial, we will learn different ways to download file from a URL in Python.
Using requests library to download file from URL in Python Scripts
If your requirement is to get the file from a given URL using GET HTTP request, then the Python requests module is perfect for you.
|
1 2 3 4 5 6 7 8 |
import requests file_url="https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png" file_object = requests.get(file_url) with open('Python-Tutorial.png', 'wb') as local_file: local_file.write(file_object.content) |
The file will be downloaded in the same directory as the Python script. If you want to change the directory location, you can provide a complete path or relative path in the open() function call.
Recommended Reading: Python with Statement
Linux aficionado? Use Python wget library to download file from URL
If you love Linux commands and want to have similar flavor in your Python program, you can use wget library to download the file from a URL.
Python wget library is not part of the default installation, so you can install it using the PIP package manager.
|
1 |
# pip install wget |
Here is the Python program to download a file from URL using wget library.
|
1 2 3 4 5 6 7 |
import wget file_url="https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png" dest_file="/Users/pankaj/pt.png" wget.download(file_url, dest_file) |
The destination file argument is optional. If we don’t provide that then the file will be saved in the same directory as the script and filename will be the same as the remote file name.
Downloading File from a URL that Redirects
Sometimes we get short URLs that redirect to the actual file. The requests library get() method automatically follows the redirect and download the actual file. If you look at the get() implementation, it sets allow_redirects parameter as True.
|
1 2 3 |
def get(url, params=None, **kwargs): kwargs.setdefault('allow_redirects', True) return request('get', url, params=params, **kwargs) |
Downloading a Large File from URL in Python
If the file is large, then it’s not a good idea to get all the content in one go. It will require a lot of memory and might cause out of memory error.
We can pass stream=True to requests get() method to open a file stream and download it in chunks. Then we can use a for loop to read the chunks and write it into the local file.
|
1 2 3 4 5 6 7 8 9 10 11 |
import requests file_url="https://www.journaldev.com/wp-content/uploads/2019/08/Python-Tutorial.png" file_stream = requests.get(file_url, stream=True) with open('Python-Tutorial.png', 'wb') as local_file: for data in file_stream: local_file.write(data) print('Done') |
Conclusion
It’s very easy to download a file from URL in Python. The requests module is perfect for it. We can also specify the HTTP methods to download the file.
Reference: Requests Module Official Docs