Adding a reference domain¶

The objective of this tutorial is to illustrate roles, directives and domains. Once complete, we will be able to use this extension to describe a recipe and reference that recipe from elsewhere in our documentation.

Note

This tutorial is based on a guide first published on opensource.com and is provided here with the original author’s permission.

Overview¶

We want the extension to add the following to Sphinx:

A recipe directive, containing some content describing the recipe steps, along with a :contains: option highlighting the main ingredients of the recipe.
A ref role, which provides a cross-reference to the recipe itself.
A recipe domain, which allows us to tie together the above role and domain, along with things like indices.

For that, we will need to add the following elements to Sphinx:

A new directive called recipe
New indexes to allow us to reference ingredient and recipes
A new domain called recipe, which will contain the recipe directive and ref role

Prerequisites¶

We need the same setup as in the previous extensions. This time, we will be putting out extension in a file called recipe.py.

Here is an example of the folder structure you might obtain:

└── source
  ├── _ext
 │  └── recipe.py
  ├── conf.py
  └── index.rst

Writing the extension¶

Open recipe.py and paste the following code in it, all of which we will explain in detail shortly:

 1fromcollectionsimport defaultdict
 2
 3fromdocutils.parsers.rstimport directives
 4
 5fromsphinximport addnodes
 6fromsphinx.applicationimport Sphinx
 7fromsphinx.directivesimport ObjectDescription
 8fromsphinx.domainsimport Domain, Index
 9fromsphinx.rolesimport XRefRole
 10fromsphinx.util.nodesimport make_refnode
 11fromsphinx.util.typingimport ExtensionMetadata
 12
 13
 14classRecipeDirective(ObjectDescription):
 15"""A custom directive that describes a recipe."""
 16
 17 has_content = True
 18 required_arguments = 1
 19 option_spec = {
 20 'contains': directives.unchanged_required,
 21 }
 22
 23 defhandle_signature(self, sig, signode):
 24 signode += addnodes.desc_name(text=sig)
 25 return sig
 26
 27 defadd_target_and_index(self, name_cls, sig, signode):
 28 signode['ids'].append('recipe' + '-' + sig)
 29 if 'contains' in self.options:
 30 ingredients = [x.strip() for x in self.options.get('contains').split(',')]
 31
 32 recipes = self.env.get_domain('recipe')
 33 recipes.add_recipe(sig, ingredients)
 34
 35
 36classIngredientIndex(Index):
 37"""A custom index that creates an ingredient matrix."""
 38
 39 name = 'ingredient'
 40 localname = 'Ingredient Index'
 41 shortname = 'Ingredient'
 42
 43 defgenerate(self, docnames=None):
 44 content = defaultdict(list)
 45
 46 recipes = {
 47 name: (dispname, typ, docname, anchor)
 48 for name, dispname, typ, docname, anchor, _ in self.domain.get_objects()
 49 }
 50 recipe_ingredients = self.domain.data['recipe_ingredients']
 51 ingredient_recipes = defaultdict(list)
 52
 53 # flip from recipe_ingredients to ingredient_recipes
 54 for recipe_name, ingredients in recipe_ingredients.items():
 55 for ingredient in ingredients:
 56 ingredient_recipes[ingredient].append(recipe_name)
 57
 58 # convert the mapping of ingredient to recipes to produce the expected
 59 # output, shown below, using the ingredient name as a key to group
 60 #
 61 # name, subtype, docname, anchor, extra, qualifier, description
 62 for ingredient, recipe_names in ingredient_recipes.items():
 63 for recipe_name in recipe_names:
 64 dispname, typ, docname, anchor = recipes[recipe_name]
 65 content[ingredient].append((
 66 dispname,
 67 0,
 68 docname,
 69 anchor,
 70 docname,
 71 '',
 72 typ,
 73 ))
 74
 75 # convert the dict to the sorted list of tuples expected
 76 content = sorted(content.items())
 77
 78 return content, True
 79
 80
 81classRecipeIndex(Index):
 82"""A custom index that creates an recipe matrix."""
 83
 84 name = 'recipe'
 85 localname = 'Recipe Index'
 86 shortname = 'Recipe'
 87
 88 defgenerate(self, docnames=None):
 89 content = defaultdict(list)
 90
 91 # sort the list of recipes in alphabetical order
 92 recipes = self.domain.get_objects()
 93 recipes = sorted(recipes, key=lambda recipe: recipe[0])
 94
 95 # generate the expected output, shown below, from the above using the
 96 # first letter of the recipe as a key to group thing
 97 #
 98 # name, subtype, docname, anchor, extra, qualifier, description
 99 for _name, dispname, typ, docname, anchor, _priority in recipes:
100 content[dispname[0].lower()].append((
101 dispname,
102 0,
103 docname,
104 anchor,
105 docname,
106 '',
107 typ,
108 ))
109
110 # convert the dict to the sorted list of tuples expected
111 content = sorted(content.items())
112
113 return content, True
114
115
116classRecipeDomain(Domain):
117 name = 'recipe'
118 label = 'Recipe Sample'
119 roles = {
120 'ref': XRefRole(),
121 }
122 directives = {
123 'recipe': RecipeDirective,
124 }
125 indices = {
126 RecipeIndex,
127 IngredientIndex,
128 }
129 initial_data = {
130 'recipes': [], # object list
131 'recipe_ingredients': {}, # name -> object
132 }
133 data_version = 0
134
135 defget_full_qualified_name(self, node):
136 return f'recipe.{node.arguments[0]}'
137
138 defget_objects(self):
139 yield from self.data['recipes']
140
141 defresolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
142 match = [
143 (docname, anchor)
144 for name, sig, typ, docname, anchor, prio in self.get_objects()
145 if sig == target
146 ]
147
148 if len(match) > 0:
149 todocname = match[0][0]
150 targ = match[0][1]
151
152 return make_refnode(builder, fromdocname, todocname, targ, contnode, targ)
153 else:
154 print('Awww, found nothing')
155 return None
156
157 defadd_recipe(self, signature, ingredients):
158"""Add a new recipe to the domain."""
159 name = f'recipe.{signature}'
160 anchor = f'recipe-{signature}'
161
162 self.data['recipe_ingredients'][name] = ingredients
163 # name, dispname, type, docname, anchor, priority
164 self.data['recipes'].append((
165 name,
166 signature,
167 'Recipe',
168 self.env.current_document.docname,
169 anchor,
170 0,
171 ))
172
173
174defsetup(app: Sphinx) -> ExtensionMetadata:
175 app.add_domain(RecipeDomain)
176
177 return {
178 'version': '0.1',
179 'parallel_read_safe': True,
180 'parallel_write_safe': True,
181 }

Let’s look at each piece of this extension step-by-step to explain what’s going on.

The directive class

The first thing to examine is the RecipeDirective directive:

 1classRecipeDirective(ObjectDescription):
 2"""A custom directive that describes a recipe."""
 3
 4 has_content = True
 5 required_arguments = 1
 6 option_spec = {
 7 'contains': directives.unchanged_required,
 8 }
 9
10 defhandle_signature(self, sig, signode):
11 signode += addnodes.desc_name(text=sig)
12 return sig
13
14 defadd_target_and_index(self, name_cls, sig, signode):
15 signode['ids'].append('recipe' + '-' + sig)
16 if 'contains' in self.options:
17 ingredients = [x.strip() for x in self.options.get('contains').split(',')]
18
19 recipes = self.env.get_domain('recipe')
20 recipes.add_recipe(sig, ingredients)

Unlike Extending syntax with roles and directives and Extending the build process, this directive doesn’t derive from docutils.parsers.rst.Directive and doesn’t define a run method. Instead, it derives from sphinx.directives.ObjectDescription and defines handle_signature and add_target_and_index methods. This is because ObjectDescription is a special-purpose directive that’s intended for describing things like classes, functions, or, in our case, recipes. More specifically, handle_signature implements parsing the signature of the directive and passes on the object’s name and type to its superclass, while add_target_and_index adds a target (to link to) and an entry to the index for this node.

We also see that this directive defines has_content, required_arguments and option_spec. Unlike the TodoDirective directive added in the previous tutorial, this directive takes a single argument, the recipe name, and an option, contains, in addition to the nested reStructuredText in the body.

The index classes

 1classIngredientIndex(Index):
 2"""A custom index that creates an ingredient matrix."""
 3
 4 name = 'ingredient'
 5 localname = 'Ingredient Index'
 6 shortname = 'Ingredient'
 7
 8 defgenerate(self, docnames=None):
 9 content = defaultdict(list)
10
11 recipes = {
12 name: (dispname, typ, docname, anchor)
13 for name, dispname, typ, docname, anchor, _ in self.domain.get_objects()
14 }
15 recipe_ingredients = self.domain.data['recipe_ingredients']
16 ingredient_recipes = defaultdict(list)
17
18 # flip from recipe_ingredients to ingredient_recipes
19 for recipe_name, ingredients in recipe_ingredients.items():
20 for ingredient in ingredients:
21 ingredient_recipes[ingredient].append(recipe_name)
22
23 # convert the mapping of ingredient to recipes to produce the expected
24 # output, shown below, using the ingredient name as a key to group
25 #
26 # name, subtype, docname, anchor, extra, qualifier, description
27 for ingredient, recipe_names in ingredient_recipes.items():
28 for recipe_name in recipe_names:
29 dispname, typ, docname, anchor = recipes[recipe_name]
30 content[ingredient].append((
31 dispname,
32 0,
33 docname,
34 anchor,
35 docname,
36 '',
37 typ,
38 ))
39
40 # convert the dict to the sorted list of tuples expected
41 content = sorted(content.items())
42
43 return content, True

 1classRecipeIndex(Index):
 2"""A custom index that creates an recipe matrix."""
 3
 4 name = 'recipe'
 5 localname = 'Recipe Index'
 6 shortname = 'Recipe'
 7
 8 defgenerate(self, docnames=None):
 9 content = defaultdict(list)
10
11 # sort the list of recipes in alphabetical order
12 recipes = self.domain.get_objects()
13 recipes = sorted(recipes, key=lambda recipe: recipe[0])
14
15 # generate the expected output, shown below, from the above using the
16 # first letter of the recipe as a key to group thing
17 #
18 # name, subtype, docname, anchor, extra, qualifier, description
19 for _name, dispname, typ, docname, anchor, _priority in recipes:
20 content[dispname[0].lower()].append((
21 dispname,
22 0,
23 docname,
24 anchor,
25 docname,
26 '',
27 typ,
28 ))
29
30 # convert the dict to the sorted list of tuples expected
31 content = sorted(content.items())
32
33 return content, True

Both IngredientIndex and RecipeIndex are derived from Index. They implement custom logic to generate a tuple of values that define the index. Note that RecipeIndex is a simple index that has only one entry. Extending it to cover more object types is not yet part of the code.

Both indices use the method Index.generate() to do their work. This method combines the information from our domain, sorts it, and returns it in a list structure that will be accepted by Sphinx. This might look complicated but all it really is is a list of tuples like ('tomato', 'TomatoSoup', 'test', 'rec-TomatoSoup',...). Refer to the domain API guide for more information on this API.

These index pages can be referenced with the ref role by combining the domain name and the index name value. For example, RecipeIndex can be referenced with :ref:`recipe-recipe` and IngredientIndex can be referenced with :ref:`recipe-ingredient`.

The domain

A Sphinx domain is a specialized container that ties together roles, directives, and indices, among other things. Let’s look at the domain we’re creating here.

 1classRecipeDomain(Domain):
 2 name = 'recipe'
 3 label = 'Recipe Sample'
 4 roles = {
 5 'ref': XRefRole(),
 6 }
 7 directives = {
 8 'recipe': RecipeDirective,
 9 }
10 indices = {
11 RecipeIndex,
12 IngredientIndex,
13 }
14 initial_data = {
15 'recipes': [], # object list
16 'recipe_ingredients': {}, # name -> object
17 }
18 data_version = 0
19
20 defget_full_qualified_name(self, node):
21 return f'recipe.{node.arguments[0]}'
22
23 defget_objects(self):
24 yield from self.data['recipes']
25
26 defresolve_xref(self, env, fromdocname, builder, typ, target, node, contnode):
27 match = [
28 (docname, anchor)
29 for name, sig, typ, docname, anchor, prio in self.get_objects()
30 if sig == target
31 ]
32
33 if len(match) > 0:
34 todocname = match[0][0]
35 targ = match[0][1]
36
37 return make_refnode(builder, fromdocname, todocname, targ, contnode, targ)
38 else:
39 print('Awww, found nothing')
40 return None
41
42 defadd_recipe(self, signature, ingredients):
43"""Add a new recipe to the domain."""
44 name = f'recipe.{signature}'
45 anchor = f'recipe-{signature}'
46
47 self.data['recipe_ingredients'][name] = ingredients
48 # name, dispname, type, docname, anchor, priority
49 self.data['recipes'].append((
50 name,
51 signature,
52 'Recipe',
53 self.env.current_document.docname,
54 anchor,
55 0,
56 ))

There are some interesting things to note about this recipe domain and domains in general. Firstly, we actually register our directives, roles and indices here, via the directives, roles and indices attributes, rather than via calls later on in setup. We can also note that we aren’t actually defining a custom role and are instead reusing the sphinx.roles.XRefRole role and defining the sphinx.domains.Domain.resolve_xref method. This method takes two arguments, typ and target, which refer to the cross-reference type and its target name. We’ll use target to resolve our destination from our domain’s recipes because we currently have only one type of node.

Moving on, we can see that we’ve defined initial_data. The values defined in initial_data will be copied to env.domaindata[domain_name] as the initial data of the domain, and domain instances can access it via self.data. We see that we have defined two items in initial_data: recipes and recipe_ingredients. Each contains a list of all objects defined (i.e. all recipes) and a hash that maps a canonical ingredient name to the list of objects. The way we name objects is common across our extension and is defined in the get_full_qualified_name method. For each object created, the canonical name is recipe.<recipename>, where <recipename> is the name the documentation writer gives the object (a recipe). This enables the extension to use different object types that share the same name. Having a canonical name and central place for our objects is a huge advantage. Both our indices and our cross-referencing code use this feature.

The setup function

As always, the setup function is a requirement and is used to hook the various parts of our extension into Sphinx. Let’s look at the setup function for this extension.

1defsetup(app: Sphinx) -> ExtensionMetadata:
2 app.add_domain(RecipeDomain)
3
4 return {
5 'version': '0.1',
6 'parallel_read_safe': True,
7 'parallel_write_safe': True,
8 }

This looks a little different to what we’re used to seeing. There are no calls to add_directive() or even add_role(). Instead, we have a single call to add_domain() followed by some initialization of the standard domain. This is because we had already registered our directives, roles and indexes as part of the directive itself.

Using the extension¶

You can now use the extension throughout your project. For example:

index.rst¶

Joe's Recipes
=============
Below are a collection of my favourite recipes. I highly recommend the
:recipe:ref:`TomatoSoup` recipe in particular!
.. toctree::
 tomato-soup

tomato-soup.rst¶

The recipe contains `tomato` and `cilantro`.
.. recipe:recipe:: TomatoSoup
 :contains: tomato, cilantro, salt, pepper
 This recipe is a tasty tomato soup, combine all ingredients
 and cook.

The important things to note are the use of the :recipe:ref: role to cross-reference the recipe actually defined elsewhere (using the :recipe:recipe: directive).

Navigation

On this page

Adding a reference domain¶

Overview¶

Prerequisites¶

Writing the extension¶

Using the extension¶

Further reading¶