Integrating Http with Python using Requests
Python is one of the most robust languages in modern computing. Though it may not result in the fastest computational time, I find that the enumerable packages its community provides make solving specialized problems quicker to solve. One package I recently discovered is Requests. This powerful package allows python to access and download any resource over https in just a few lines of code. By following this guide, you will gain basic understanding on this module and how to install it.
Installing the Package
This tutorial assumes that you have a basic understanding of Python, and have the Python Package Manager (PIP) already installed. With this being the case, simply install the package by typing:
pip install requests
Since the module does not require any dependencies, it should install without any issues. You can test if it installed correctly by importing it in the shell.
Using the Request Module
Now that the module is installed, it is time to have fun. We will only cover the basics of the module, so feel free to read up on other functions in the module documentation http://docs.python-requests.org/en/master/.
Like any python package we need to import it before using it, this can be done with the standard:
Next, we need a resource to play with. The easiest way is to use the standard format of :
If the resource needs credentials to be accesses, add the function auth= as a parameter:
requests.get('http://path/to/resource' , auth=('user','password'))
Now that we know the format let's create our own instance by using the following:
bbsRes = requests.get('https://blackbricksoftware.com/images/bitsonbytes/request.txt)
This line gets a text file from our website and saves it in a variable. It should be noted that you can use any resource from the web, but we are just going to be using resources from our own site.
The next step is to confirm that there are no issues with the file. Fortunately, the requests module has a nifty function called raise_for_status. We can call it by the following line
Once this is called, you should receive a http response code. If you have never worked with response codes before, fret not! These are just messages the server sends whenever you communicate with it. Though we won't cover all the codes, here are some that you should know when working with requests.
|200||Ok - This is the code we want|
|400||Bad Request - There was something wrong. Check your code for syntax errors.|
|401||Unauthorized - Use the auth= method as a parameter and include your credentials.|
|403||Forbidden - You don't have the rights to use this resource|
|500||Internal Server Error - The server is probably down for maintenance. Try again later.|
If everything was input correctly you should have the response of 200. If this is the case, we are good to go!
Playing with the Resource
We now have the ability to use python to accomplish many cool things with the request. Though this tutorial is meant to get your feet wet with requests, pretty much anything is possible. Maybe we will revisit this topic in a later post.
We can still do some pretty cool things with basic python as well. For example, we can count the number of characters in the file by using:
We can also read in the first 60 characters of the file by using:
Downloading the Request and Saving It to the Hard Drive
Though manipulation is pretty neat, the main point to requests is downloading the resource. Combining the request with open function, makes it really easy to save the resource to a file. The basic snippet to do so is the following:
saveFile = open('BlackBrickSoftwareRequests.txt', 'wb')
for chunk in bbsRes.iter_content(FileSize):
It should be noted that FileSize is a variable representing the total size of the file. You can easily adjust it to fit the size of the resource.