BK-tree

A BK-tree (short for Burkhard-Keller tree) is a metric tree suggested by Walter Austin Burkhard and Robert M. Keller ^[1] specifically adapted to discrete metric spaces. For simplicity, given a way to measure the distance between any two elements of a set, a BK-tree is built with a single root node and several subtrees, each connected to the root as a child. All nodes in a subtree has an equal distance to the root node, and the edge weight of the edge connecting the subtree to the root is equal to the distance. As shown in the picture. Also, each subtree of a BK-tree is a BK-tree.

BK-trees can be used for approximate string matching in a dictionary ^[2] . The problem is formulated as follows: Given a pattern string ${\displaystyle P=p_{1}p_{2}...p_{m}}${\displaystyle P=p_{1}p_{2}...p_{m}} and a text string ${\displaystyle T=t_{1}t_{2}\dots t_{n}}${\displaystyle T=t_{1}t_{2}\dots t_{n}}, we need to find a substring ${\displaystyle T_{j',j}=t_{j'}\dots t_{j}}${\displaystyle T_{j',j}=t_{j'}\dots t_{j}} in ${\displaystyle T}${\displaystyle T}, which, of all substrings of ${\displaystyle T}${\displaystyle T}, has the smallest edit distance to the pattern ${\displaystyle P}${\displaystyle P}.

An ordinary way using BK-tree is to insert all ${\displaystyle \Theta (n^{2})}${\displaystyle \Theta (n^{2})} substrings of ${\displaystyle T}${\displaystyle T} and the pattern string ${\displaystyle P}${\displaystyle P} into the BK-tree, and find the subtree containing nodes with the smallest distance from the root. This will lead to a time complexity of ${\displaystyle \Theta (n^{2}\log n)}${\displaystyle \Theta (n^{2}\log n)}. However, this algorithm is very accurate and can find every substrings with the smallest edit distance to ${\displaystyle P}${\displaystyle P}.

This picture depicts the BK-tree for the set ${\displaystyle W}${\displaystyle W} of words {"book", "books", "cake", "boo", "boon", "cook", "cape", "cart"} obtained by using the Levenshtein distance

• each node ${\displaystyle u}${\displaystyle u} is labeled by a string of ${\displaystyle w_{u}\in W}${\displaystyle w_{u}\in W};
• each arc ${\displaystyle (u,v)}${\displaystyle (u,v)} is labeled by ${\displaystyle d_{uv}=d(w_{u},w_{v})}${\displaystyle d_{uv}=d(w_{u},w_{v})} where ${\displaystyle w_{u}}${\displaystyle w_{u}} denotes the word assigned to ${\displaystyle u}${\displaystyle u}.

The BK-tree is built so that:

• for all node ${\displaystyle u}${\displaystyle u} of the BK-tree, the weight assigned to its egress arcs are distinct;
• for all arc ${\displaystyle e=(u,v)}${\displaystyle e=(u,v)} labeled by ${\displaystyle k}${\displaystyle k}, each descendant ${\displaystyle v'}${\displaystyle v'} of ${\displaystyle v}${\displaystyle v} satisfies the following equation: ${\displaystyle d(w_{u},w_{v'})=k}${\displaystyle d(w_{u},w_{v'})=k}:
  • Example 1: Consider the arc from "book" to "books". The distance between "book" and any word in {"books", "boo", "boon", "cook"} is equal to 1;
  • Example 2: Consider the arc from "books" to "boo". The distance between "books" and any word in {"boo", "boon", "cook"} is equal to 2.

The insertion primitive is used to populate a BK-tree ${\displaystyle t}${\displaystyle t} according to a discrete metric ${\displaystyle d}${\displaystyle d}.

Input:

• ${\displaystyle t}${\displaystyle t}: the BK-tree;
  • ${\displaystyle d_{uv}}${\displaystyle d_{uv}} denotes the weight assigned to an arc ${\displaystyle (u,v)}${\displaystyle (u,v)};
  • ${\displaystyle w_{u}}${\displaystyle w_{u}} denotes word assigned to a node ${\displaystyle u}${\displaystyle u};
• ${\displaystyle d}${\displaystyle d}: the discrete metric used by ${\displaystyle t}${\displaystyle t} (e.g. the Levenshtein distance);
• ${\displaystyle w}${\displaystyle w}: the element to be inserted into ${\displaystyle t}${\displaystyle t};

Output:

• The node of ${\displaystyle t}${\displaystyle t} corresponding to ${\displaystyle w}${\displaystyle w}

Algorithm:

• If the ${\displaystyle t}${\displaystyle t} is empty:
  • Create a root node ${\displaystyle r}${\displaystyle r} in ${\displaystyle t}${\displaystyle t}
  • ${\displaystyle w_{r}\leftarrow w}${\displaystyle w_{r}\leftarrow w}
  • Return ${\displaystyle r}${\displaystyle r}
• Set ${\displaystyle u}${\displaystyle u} to the root of ${\displaystyle t}${\displaystyle t}
• While ${\displaystyle u}${\displaystyle u} exists:
  • ${\displaystyle k\leftarrow d(w_{u},w)}${\displaystyle k\leftarrow d(w_{u},w)}
  • If ${\displaystyle k=0}${\displaystyle k=0}:
    • Return ${\displaystyle u}${\displaystyle u}
  • Find ${\displaystyle v}${\displaystyle v} the child of ${\displaystyle u}${\displaystyle u} such that ${\displaystyle d_{uv}=k}${\displaystyle d_{uv}=k}
  • If ${\displaystyle v}${\displaystyle v} is not found:
    • Create the node ${\displaystyle v}${\displaystyle v}
    • ${\displaystyle w_{v}\leftarrow w}${\displaystyle w_{v}\leftarrow w}
    • Create the arc ${\displaystyle (u,v)}${\displaystyle (u,v)}
    • ${\displaystyle d_{uv}\leftarrow k}${\displaystyle d_{uv}\leftarrow k}
    • Return ${\displaystyle v}${\displaystyle v}
  • ${\displaystyle u\leftarrow v}${\displaystyle u\leftarrow v}

Given a searched element ${\displaystyle w}${\displaystyle w}, the lookup primitive traverses the BK-tree to find the closest element of ${\displaystyle w}${\displaystyle w}. The key idea is to restrict the exploration of ${\displaystyle t}${\displaystyle t} to nodes that can only improve the best candidate found so far by taking advantage of the BK-tree organization and of the triangle inequality (cut-off criterion).

Input:

• ${\displaystyle t}${\displaystyle t}: the BK-tree;
• ${\displaystyle d}${\displaystyle d}: the corresponding discrete metric (e.g. the Levenshtein distance);
• ${\displaystyle w}${\displaystyle w}: the searched element;
• ${\displaystyle d_{max}}${\displaystyle d_{max}}: the maximum distance allowed between the best match and ${\displaystyle w}${\displaystyle w}, defaults to ${\displaystyle +\infty }${\displaystyle +\infty };

Output:

• ${\displaystyle w_{best}}${\displaystyle w_{best}}: the closest element to ${\displaystyle w}${\displaystyle w} stored in ${\displaystyle t}${\displaystyle t} and according to ${\displaystyle d}${\displaystyle d} or ${\displaystyle \perp }${\displaystyle \perp } if not found;

Algorithm:

• If ${\displaystyle t}${\displaystyle t} is empty:
  • Return ${\displaystyle \perp }${\displaystyle \perp }
• Create ${\displaystyle S}${\displaystyle S} a set of nodes to process, and insert the root of ${\displaystyle t}${\displaystyle t} into ${\displaystyle S}${\displaystyle S}.
• ${\displaystyle (w_{best},d_{best})\leftarrow (\perp ,d_{max})}${\displaystyle (w_{best},d_{best})\leftarrow (\perp ,d_{max})}
• While ${\displaystyle S\neq \emptyset }${\displaystyle S\neq \emptyset }:
  • Pop an arbitrary node ${\displaystyle u}${\displaystyle u} from ${\displaystyle S}${\displaystyle S}
  • ${\displaystyle d_{u}\leftarrow d(w,w_{u})}${\displaystyle d_{u}\leftarrow d(w,w_{u})}
  • If ${\displaystyle d_{u}<d_{best}}${\displaystyle d_{u}<d_{best}}:
    • ${\displaystyle (w_{best},d_{best})\leftarrow (w_{u},d_{u})}${\displaystyle (w_{best},d_{best})\leftarrow (w_{u},d_{u})}
  • For each egress-arc ${\displaystyle (u,v)}${\displaystyle (u,v)}:
    • If ${\displaystyle |d_{uv}-d_{u}|<d_{best}}${\displaystyle |d_{uv}-d_{u}|<d_{best}}: (cut-off criterion)
      • Insert ${\displaystyle v}${\displaystyle v} into ${\displaystyle S}${\displaystyle S}.
• Return ${\displaystyle w_{best}}${\displaystyle w_{best}}

Consider the example 8-node B-K Tree shown above and set ${\displaystyle w=}${\displaystyle w=}"cool". ${\displaystyle S}${\displaystyle S} is initialized to contain the root of the tree, which is subsequently popped as the first value of ${\displaystyle u}${\displaystyle u} with ${\displaystyle w_{u}}${\displaystyle w_{u}}="book". Further ${\displaystyle d_{u}=2}${\displaystyle d_{u}=2} since the distance from "book" to "cool" is 2, and ${\displaystyle d_{best}=2}${\displaystyle d_{best}=2} as this is the best (i.e. smallest) distance found thus far. Next each outgoing arc from the root is considered in turn: the arc from "book" to "books" has weight 1, and since ${\displaystyle |1-2|=1}${\displaystyle |1-2|=1} is less than ${\displaystyle d_{best}=2}${\displaystyle d_{best}=2}, the node containing "books" is inserted into ${\displaystyle S}${\displaystyle S} for further processing. The next arc, from "book" to "cake," has weight 4, and since ${\displaystyle |4-2|=2}${\displaystyle |4-2|=2} is not less than ${\displaystyle d_{best}=2}${\displaystyle d_{best}=2}, the node containing "cake" is not inserted into ${\displaystyle S}${\displaystyle S}. Therefore, the subtree rooted at "cake" will be pruned from the search, as the word closest to "cool" cannot appear in that subtree. To see why this pruning is correct, notice that a candidate word ${\displaystyle c}${\displaystyle c} appearing in "cake"s subtree having distance less than 2 to "cool" would violate the triangle inequality: the triangle inequality requires that for this set of three numbers (as sides of a triangle), no two can sum to less than the third, but here the distance from "cool" to "book" (which is 2) plus the distance from "cool" to ${\displaystyle c}${\displaystyle c} (which is less than 2) cannot reach or exceed the distance from "book" to "cake" (which is 4). Therefore, it is safe to disregard the entire subtree rooted at "cake".

Next the node containing "books" is popped from ${\displaystyle S}${\displaystyle S} and now ${\displaystyle d_{u}=3}${\displaystyle d_{u}=3}, the distance from "cool" to "books." As ${\displaystyle d_{u}>d_{best}}${\displaystyle d_{u}>d_{best}}, ${\displaystyle d_{best}}${\displaystyle d_{best}} remains set at 2 and the single outgoing arc from the node containing "books" is considered. Next, the node containing "boo" is popped from ${\displaystyle S}${\displaystyle S} and ${\displaystyle d_{u}=2}${\displaystyle d_{u}=2}, the distance from "cool" to "boo." This again does not improve upon ${\displaystyle d_{best}=2}${\displaystyle d_{best}=2}. Each outgoing arc from "boo" is now considered; the arc from "boo" to "boon" has weight 1, and since ${\displaystyle |2-1|=1<d_{best}=2}${\displaystyle |2-1|=1<d_{best}=2}, "boon" is added to ${\displaystyle S}${\displaystyle S}. Similarly, since ${\displaystyle |2-2|=0<d_{best}}${\displaystyle |2-2|=0<d_{best}}, "cook" is also added to ${\displaystyle S}${\displaystyle S}.

Finally each of the two last elements in ${\displaystyle S}${\displaystyle S} are considered in arbitrary order: suppose the node containing "cook" is popped first, improving ${\displaystyle d_{best}}${\displaystyle d_{best}} to distance 1, then the node containing "boon" is popped last, which has distance 2 from "cool" and therefore does not improve the best result. Finally, "cook" is returned as the answer ${\displaystyle w_{best}}${\displaystyle w_{best}} with ${\displaystyle d_{best}=1}${\displaystyle d_{best}=1}.

The efficiency of BK-trees depends strongly on the structure of the tree and the distribution of distances between stored elements.

In the average case, both insertion and lookup operations take ${\displaystyle \Theta (\log n)}${\displaystyle \Theta (\log n)} time, assuming the tree remains relatively balanced and the distance metric distributes elements evenly.^{[3] [4]}

In the worst case, when the data or distance function causes the tree to become highly unbalanced (for example, when many elements are at similar distances), both insertion and lookup can degrade to ${\displaystyle \Theta (n)}${\displaystyle \Theta (n)}.^{[5] [6]}

In practical applications, the actual performance depends on the choice of distance metric (e.g., the Levenshtein distance) and on the allowed search radius during approximate matching.^[7]

• Levenshtein distance – the distance metric commonly used when building a BK-tree
• Damerau–Levenshtein distance – a modified form of Levenshtein distance that allows transpositions

• ^ W. Burkhard and R. Keller. Some approaches to best-match file searching, CACM, 1973
• ^ R. Baeza-Yates, W. Cunto, U. Manber, and S. Wu. Proximity matching using fixed queries trees. In M. Crochemore and D. Gusfield, editors, 5th Combinatorial Pattern Matching, LNCS 807, pages 198–212, Asilomar, CA, June 1994.
• ^ Ricardo Baeza-Yates and Gonzalo Navarro. Fast Approximate String Matching in a Dictionary. Proc. SPIRE'98

• ^ W. A. Burkhard and R. M. Keller. Some approaches to best match file searching. Communications of the ACM, 16(4):230–236, 1973.
• ^ R. Baeza-Yates and G. Navarro. Fast Approximate String Matching in a Dictionary. In Proceedings of SPIRE'98, 1998.
• ^ H. Hyyro. Explaining and Extending the Burkhard–Keller Tree for Fast String Searching. In Proceedings of the 7th Symposium on String Processing and Information Retrieval (SPIRE), pages 1–10, 2003.
• ^ S. Mihov and K. Schulz. Fast approximate search in large dictionaries. Computational Linguistics, 30(4):451–477, 2004.

• A BK-tree implementation in Common Lisp with test results and performance graphs.
• An explanation of BK-Trees and their relationship to metric spaces [8]
• An explanation of BK-Trees with an implementation in C# [9]
• A BK-tree implementation in Lua [10]
• A BK-tree implementation in Python [11]

• Trees (data structures)

• Articles with short description
• Short description is different from Wikidata