So to do this, we're first going to go ahead and start with unwinding that array field, and here we go. And one thing you'll notice now is that every entry in ancestors is an ancestor of Cat Toys, and these have been separated out into separate documents. So now we can go ahead and try and group by ancestors.name on these different ancestor categories to go ahead and build out a descendant graph for every item. Here we're going to go ahead and group by every ancestor, and then we're going to go ahead and add the original root node's name to its list of descendants. And it works, great. Let's go ahead and now do the same thing but on our original query where we looked for any category containing the word cat. We're also going to look for the Pet Supplies category by adding a match stage at the end, because we know that Pet Supplies is an ancestor of Cat Supplies. And we're also going to maintain that parental information so that we can use that later to construct a list of immediate children for each node. And you'll see that in just a moment. Great, it nearly works. We now have a small issue where we can see that the node's name is duplicated in its list of descendants. But we can fix that, and we already know how to do that by using a set difference expression. And so we're going to go ahead and use set difference again. Now on the _id field, because we used group by. And then we're going to ahead and difference that against its descendants where it has the same name. We're going to also add a field called children where we're going to look for any immediate children of a node. And we're just going to filter over this descendants array for items where the parent is equal to the name of the node that we're on. And this should give us all of our immediate children. And before executing this pipeline we're going to make a copy of it, so that we can use it again later. And then so on this copy, we're going to go ahead and add our match at the beginning and at the end, like before. And here you go, now we have Cat Toys, which has no descendants because it's a leaf node. Same thing with Cat Litter Boxes. And so with Cat Supplies, we're going to see all of its descendants. All of its descendants are also all of its children. But then when we look at Pet Supplies we're going to all of its descendants, including leaf nodes like Cat Toys. But then in its immediate children we're just going to see Cat Supplies, because that's its only immediate child. Here I've written a function to descend the graph from any node and generate the appropriate string to visualize the tree. So now we can see if the tree looks like the way we described that we want it to at the beginning of the lesson. And we can see that now it does, which is awesome. Now, in inverting the tree to get it looking this way, we destroyed those parental references. And so let's go ahead and now build a pipeline where we go ahead and get those back. We're just going to go ahead and do a simple set difference on the ancestor's parent against its name. I'm calling that ancestors and also keeping its original parent. And now we can go ahead and take our pipeline that inverts our tree and our pipeline that keeps our parents, and we can run them in parallel using facets. We'll then unwind our parent_tree and pair up document entries in the child_tree output by name. And then the rest here is really just extracting the different important pieces of the child_tree that we care about and assigning it to the top level document. And then cleaning up the output a little bit. And then, of course, sorting so we can have a better visualization. And we're sorting by the number of descendants, and here is really the key part. Here we're now able to to calculate some really interesting heuristics, like the number of children, the total number of descendants, and the total number of ancestors. And calculating this information before would have been impossible without first inverting the tree using graph lookup. Excellent, and now you can see we have a truly doubly linked tree structure, where each node has parental information if it has it. As well as any ancestor information, as well as immediate children, and finally a list of all its descendants. And then, moreover, we're able to now calculate all these interesting statistics, like the number of ancestors, number of children, and number of descendants. Let's go and see what this tree actually looks like. And here you can see a slightly different view of the entire product tree beginning with Pet Supplies. Now, having the tree represented in this way is very advantageous. Instead of having to crawl the tree one node at a time, we can take shortcuts to determine information such as if one item is an ancestor or descendant of another. Also, because the parent and children information is here, we can find common ancestors of two elements, if there is one. And here's a basic example for how we can perform some interesting analysis. Here I'm creating functions to determine, is_descendant, is_ancestor, or is_common_ancestor. And here we can see if Bird Cage Food & Water Dishes is a descendant of Pet Supplies, which it is. Here we can see if Pet Supplies is an ancestor of Bird Cage Accessories, which it is. And here we can see if Small Animal Food and Pet Food Containers have a common ancestor, which they do and it is Pet Supplies. Answering questions like this would have been very, very difficult with the way that our data was modeled before. And now we're able to use graph lookup to model our data in a way where we can easily answer these questions. Now, for example, imagine if we paired this information with transactional information. We'd be able to easily identify categories that could be removed in an effort to streamline business. We can also use this type of structure for recommendation. If a customer is buying a product in the Cat Toys category, we may recommend products from other categories under the parent, Cat Supplies. Without a common ancestor and without the full ancestry or descendant information, doing this in a timely manner would be infeasible. Lastly, I wanted to give you the ability to see the entire product tree. We're not going to display this here because there's over 17,000 entries in this collection and the image is very, very large. Feel free to uncomment on the last line in the cell to render the image and open it in an external image viewer with really good zoom capabilities. But beware, this image is very, very large. Okay, we covered a lot of information this lesson, so let's discuss what we've learned. We learned how to work with simple trees and we saw how simple trees often don't contain enough information for efficient traversal, which makes them harder to work with when trying to explore relationships. We also saw how we can transform these simple, singly linked trees into more complex, doubly linked trees using graphLookup. Moreover, we saw how we combine two graphLookup stages in a facet to invert a bottom-up tree into a top-down tree, and then combine the two. And you saw the advantages of these more complex trees with double links. We had much better entity relationship descriptions. We were able to do much faster traversal. Do keep in mind that complex trees use up slightly more space because of that richer entity relationship information. And that's the lesson on tree-like data with MongoDB.
