How do modules work in Python?

Question

I was wondering how does a module really work in Python.For example I have this code

import requests
r = requests.get("http://www.google.com")
print r.status_code

Now according to my understanding, the requests module should have a python file which would be containing a class called "get" and within the "get" class there must be a member variable called "status_code" So when I create the object "r", I get the variable status_code for it.

However, when I looked at all the files that come in the package, I could not find any class named "get". I could however find a function called "get", under a class called "response". But since we did not create the object as an instance of the "response" class, how can we access the "get" function inside it?

I think I am missing a key concept here, can someone point it out for me please?

Thanks

• falsetru: Read the code.

Answer 1

When you import requests file __init__.pyis executed, if you examine that file in your case, you will find this line:

from .api import request, get, head, post, patch, put, delete, options

Which means that from api.py you are importing get() function:

def get(url, **kwargs):
 kwargs.setdefault('allow_redirects', True)
 return request('get', url, **kwargs)

And as you can see it calls request function from api.py that looks like:

def request(method, url, **kwargs):
 session = sessions.Session()
 return session.request(method=method, url=url, **kwargs)

That creates an object Session defined inside session.py, then calls its method request. This method will call method send() which returns a Responseobject which is defined in the class Response inside models.py (I copy the first lines):

class Response(object):
 def __init__(self):
 super(Response, self).__init__()
 self._content = False
 self._content_consumed = False
 #: Integer Code of responded HTTP Status.
 self.status_code = None
...

Here is where status_code is defined, so when you invoke r = requests.get("http://www.google.com") you are retrieving this object and then you can access to status_code

• positivelydoped23: So if I am getting it right, the get function uses the request function which in turn creates a sessions object and calls the request function defined in the sessions class. The output of that request function is what we get finally. So in my code "r" now gets the output from the request function, but how does it get the status_code variable?
• jabaldonedo: I have updated the post so now "how status_code is got", is already explained".
• positivelydoped23: Thanks, I understand it now. But I have another question. Did you manually open each of these files to check in which file what function is called? Or is there a tool which we can use for this? I am currently using sublime text and I have to open each file individually.
• jabaldonedo: I opened each file manually, it's a little bit tedious. Remember accept the answer :)

Answer 2

Your understanding is not entirely correct.

The requests module is an object; .get is then an attribute lookup on that object; it has to be a callable object because you try to call it with the (...) syntax. That means it can be a class, or a function or any other object with a __call__ method.

That callable returns something; all callables do. For a class, generally an instance is returned, but for a function that can be any Python object. Whatever .get() does, it returns an object that has a .status_code attribute, or has a .__getattr__ method that returns something when called with the name status_code.

In this specific case, get() is a function, which has been imported into the requests/__init__.py package initializer module. This function, indirectly, creates a Session() instance, and calls the .request() method on that instance. That method, eventually, returns a Response instance, which does have a .status_code attribute.

• Martijn Pieters: @glglgl: where a descriptor is a special kind of attribute; just because the .__get__ method is called doesn't mean we need to complicate things even more here. :-P
• glglgl: You are right, I just wanted to mention this further option for completeness, as it is nevertheless slightly different from the "standard". But as it doesn't apply here, it is not relevant.