Okay. So, let's talk about how to do this type of update the right way. What we're going to look at here is something called, Bulk Rights in MongoDB. This is essentially a way of performing a number of update operations with a single request to the database. So, rather than update each document one at a time, we're going to batched together updates for in this case, a thousand documents at a time and then make a single request to the database for all thousand updates at once. So, the update code that we see here is exactly the same as what we looked at in our lesson on the use of Update One. The only difference is that as I said, instead of calling Update One, we are instead going to use this Update One class which I've imported from the Pymongo library here. We're going to use Update One which is an operation class to create and update one operation object here, and add it to a list of updates that we're keeping track of. Then, each time through the loop, we're simply checking how many updates we have accumulated to this point, how many update objects we've accumulated, and once we've hit our batch size limit which as we saw above is 1000, then I make a call to the bulk write method passing it this list of updates. Take a look in the handouts for this lesson to study this code in detail, if you have any questions. But the upshot is that we are creating 1000 Update One operation objects, passing that list of Update One objects to the bulk write method. Bulk right will construct the necessary request to the MongoDB to the server and the server then will receive that entire list of updates. It will then apply the updates one at a time. Again, the update code, the way we're selecting what document to update and the specific updates to apply for each document are exactly the same as we looked at in detail in our lesson on the use of Update One. Here, we're doing exactly those same updates but doing it in a much more efficient manner. Outside this loop, we have one statement here and that's merely to catch the final list of updates that need to be applied because it's entirely possible and in fact probable that we will finish this loop with some number less than 1000 updates that need to be applied. In order to catch all of those, whatever remains, we call Bulk Right here, if in fact our list does contain any updates to be applied. Okay. So, let's actually test this code and in order to do that, what I'm going to do is first, create a brand new collection in my database that is the original movie's dataset that we began working with in this course. So, when I make this call to Mongo Import this will load that initial movies dataset into a new collection called, Movies in my database. Now, I'm doing this just so I don't mistakenly update the movies underscore initial collection. So, I'm going to try and execute this, and once it finishes, then I can go into compass and I can take a look at that data set. And here, we can see that it is in fact has the same shape and this data should look very familiar to you. Now, let's go ahead and execute this bulk update code and once that finishes, then if I look again at my movies collection, once the script is finished running, I can see that the data set now contains arrays for all of the fields that I've split from strings into arrays. I have an IMDB embedded document and as we scan through, we can see that there are no empty fields or fields that have as their value the empty string and finally, for runtime for each document I have an integer. So, that gives us a pretty good overview of how to go about doing data cleaning in a collection in MongoDB. We've looked at several different approaches and in the process given you a pretty detailed overview of some of the key aspects of the MongoDB aggregation framework, as well as the MongoDB query language and use of the Pymongo driver in the Python programming.
