Let's say you want to store your application's data, and the data is structured or semi-structured, but not relational. You want to get up and running quickly, but you want to make sure that the database option that you choose will be highly scalable and meets the needs of your application. Datastore is the ideal solution for this use case. Datastore can scale from zero to millions of requests per second without you changing configuration or adding notes to a cluster. In this module, best practices for using Datastore, you'll learn key concepts about data objects, such as kinds, entities, keys, and attributes. You'll learn and apply best practices related to queries, built-in, and composite indexes, inserting and deleting data, transactions, and error handling. A common question application developers have is, how do I bulk load data into Datastore? We will explore how to use Dataflow to bulk load data into Datastore. Firestore in Datastore mode uses Datastore system behavior, but stores and reads data from Firestores and improved storage layer. Firestore and Datastore mode is backward compatible with Datastore. But Firestore's Native data model, real-time updates, and mobile and web client library features cannot be used with Datastore mode. To access all of the new Firestore features, you must use Firestore in Native mode. When you create a new Firestore database, you must select a database mod, and you cannot use both Native mode and Datastore mode in the same project. Datastore mode should typically be used for server applications. Native mode should be used for new mobile and web apps, or when requiring real-time and offline features. Data objects in Datastore are called entities, and they are made up of one or more properties. Properties can have one or more values. Each entity has a key that uniquely identifies it, composed of a name space, the entity's kind, an identifier, and an optional ancestor path. Operations on one or more entities are called transactions and are atomic. When you create an entity, you can specify another entity as its parent. An entity without a parent is a root entity. An entity's parents, the parents parent, and so on, are the entity's ancestors. An entity's child, the child's child, etc, are the entity's descendants. The sequence of entities from the root entity to a specific entity forms the ancestor path. The example shows the root entity, John Doe with kind customer. The complete key for the pencil entity includes the kind identifier pairs, customer, John Doe, invoice, June, and product, pencil. Datastore has two types of indexes, built-in and composite indexes. By default, in ascending and descending index is predefined for each property of each entity kind. These are called built-in indexes. The single property indexes are sufficient to perform many simple queries, such as equality only queries and simple inequality queries. These indexes are assumed and do not appear in the indexes page of the Google Cloud Console. For more complex queries, you must manually define composite indexes. Composite indexes are composed of multiple properties. If a property will never be needed for a query, exclude the property from indexes. Unnecessarily indexing a property will result in increased latency to achieve consistency and increased storage costs for indexes. You should avoid having too many composite indexes. Excessive use of composite indexes also results in increased latency and storage costs. If you need to execute ad-hoc queries on large data sets without previously defined indexes, use BigQuery. Do not index properties with monotonically increasing values, such as a now Timestamp. Maintaining such an index could lead to hotspots that impact in-store latency for applications with high read and write rates. Composite indexes are defined in the applications index configuration file, index.yaml. Composite indexes are viewable but not editable through the Cloud Console. To deploy a composite index, modify the index.yaml configuration file to include all properties you want to index. Run gCloud Datastore indexes create to create the new index. Depending on how much data is already in Datastore, creating the index may take a significant amount of time. Queries that are run before the index is finished building will result in an exception. When you change or remove an existing index from the index configuration, the original index is not deleted from Datastore automatically. When you're sure that old indexes are no longer needed, use gCloud Datastore indexes cleanup. That command deletes all indexes for the production Datastore Instance that are not mentioned in the local version of index.yaml. You can check the status of indexes in the Cloud Console. The table compares concepts and Datastore with standard relational database concepts. One key difference between Datastore and a relational database is that Datastore is designed to automatically scale to handle very large datasets, allowing applications to maintain high-performance as they receive more traffic. Datastore right-scale by automatically distributing data as necessary. Datastore reads scale by only supporting queries whose performance scales are the size of the result set as opposed to the dataset. A query whose result set contains a 100 entities, performs the same, whether it searches over a 100 entities or a million. All queries are served by previously built indexes, so the types of queries that can be executed are more restrictive than those allowed on a relational database with SQL. Datastore does not include support for joint operations, inequality filtering on multiple properties, or filtering on data based on the results of a subquery. Unlike relational databases, Datastore is schema-less. It doesn't require entities of the same kind to have a consistent property set. If you wish to enforce a specific set of properties for particular kind, you must enforce that in your own application code.
