Google I/O 2010 - Next gen queries

What people said

Recommends npdoty · 2010-06 · Stack Overflow · on “How to store and search GeoData in AppEngine?”

I've had success using the geomodel [LINK] open-source project for storing and querying lat/lon data in the App Engine datastore. It uses a geohash [LINK] to enable querying on a single property, it's available for both Python and Java and since it's open source you can modify the code to fit your particular case if need be. It's also worth noting that it was announced at Google I/O a couple weeks back that the datastore would someday (hopefully soon) provide native support for geospatial and date queries, using a Hilbert curve internally. This would allow fast querying for bounding box or even in arbitrary polygon queries. You can watch the video [this video] of that talk online for more detail,

Recommends Jaime Gomez · 2014-08 · Stack Overflow · on “Appengine Datastore, one index for multiple queries”

You can't use only one index, they have to be really specific to the query in order to be fast. However for most of your example queries automatic indexes will work fine, thanks to the zigzag merge join algorithm [this video] , and those don't count towards your 200 index limit. Sorting will require a custom index, but only per property, for example: Index(User, domain, -age) Index(User, enabled, -age) Index(User, name, -age) Index(User, phone, -age) This 4 indexes should enable you to use any query that orders by age, no matter what properties are you filtering by (if you were to add composite indexes for every permutation you would need 16). The only caveat is that the shape of your data will d

Recommends Patrice · 2015-02 · Stack Overflow · on “How to build datastore indexes (PHP GAE)”

As Alex Martelli mentioned in a comment, most of the time, your indexes are built when you run your app on the devserver and have your datastore get queried there (this adds the required indexes for any question into your index.yaml file. So you have two ways you can go at it. 1- Run your app on local devserver, go to your dev " Developer Console [LINK] " to add one or two entities to your datastore. Run your queries, that'll populate your index.yaml with all required indexes. You can then run appcfg.py update_indexes to just deploy your index.yaml (bottom of this page [LINK] ) 2- Your other solution would be to read this [LINK] , a page on how datastore indexes work. Then read this [LINK] a

Recommends Patrick Costello · 2019-08 · Stack Overflow · on “Are built-in index based ancestor queries efficient?”

Cloud Datastore built-in indexes are always split into a prefix and a postfix at query time. The prefix portion is the part that remains the same (eg equalities or ancestors), the postfix portion is the part that changes (sort order). Builtin indexes are laid out: Kind, PropertyName, PropertyValue, Key For example, a query: FROM MyKind WHERE A > 1 Would divide the prefix/postfix as: MyKind,A | range<1, inf> In the case you're asking about (ancestor with equality), FROM MyKind WHERE __key__ HAS ANCESTOR Key('MyAncestor', 1) AND A = 1 the first part of the prefix is easy: MyKind,A,1 To understand the ancestor piece, we have to consider that Datastore keys are a hierarchy. In the case of MyKind

Recommends Kato · 2021-01 · Stack Overflow · on “Firestore query usage”

Firestore uses sorted indices and a binary search pattern. This makes reads extremely fast and basically makes it impossible to create a slow query. An overview of Firestore queries is published here [LINK] . And there's a detailed explanation of the indices in Todd's How do queries work in Cloud Firestore? video at time 5:30 [LINK] . You pay for exactly how many documents are returned. So only the matches and only up to the limit you set. There's a detailed explanation and example billing scenario in the docs here [LINK] . Note that compound queries are also extremely fast since we use a ZigZag merge join (explained in this I/O talk [this video] and summarized in Todd's video at time 9:22 [LINK]

Recommends Frank van Puffelen · 2023-08 · Stack Overflow · on “When index merging is possible with Firestore”

From the documentation on Queries supported by single-field indexes [LINK] : Using these automatically created single-field indexes, you can run simple queries like the following: const stateQuery = citiesRef.where("state", "==", "CA"); const populationQuery = citiesRef.where("population", "<", 100000); const nameQuery = citiesRef.where("name", ">=", "San Francisco"); You can also create in and compound equality (==) queries: citiesRef.where('country', 'in', ["USA", "Japan", "China"]) // Compound equality queries citiesRef.where("state", "==", "CO").where("name", "==", "Denver") citiesRef.where("country", "==", "USA") .where("capital", "==", false) .where("state", "==", "CA") .where("populat

Cites Drew Sears · 2010-06 · Stack Overflow · on “how to effectively run two inequality filters on queries in app engine”

According to Alfred Fuller's recent Google I/O talk [this video] , they're working on support for multiple inequality filters on numeric properties.

Cites David Underhill · 2010-07 · Stack Overflow · on “datastore.Cursor - paging in reverse supported?”

No, there is no way to go backwards with cursors with the current release. You could "fake" it, however, by caching previous cursors. Note: According to Alfred Fuller's Google IO talk [this video] , at some point in the future app engine will hopefully support reverse cursors too.

Cites Prashanth Babu · 2011-02 · Stack Overflow · on “Inequality Filters on a date and a number”

Am trying to query my Google App Engine datastore [Python], which has a item_name, manufacturing_date and number_of_items_shipped. There are ~1.0 million records in the datastore and ever increasing. The scenario: Get all the item_names which have been shipped more than x_items [user input] and manufactured after some_date [user input]. Basically, kind of an inventory check. Effectively 2 inequalities on properties. But due to restrictions on queries [LINK] in GAE, am not able to do this. Searched SO for this issue. But, no luck till now. Did you come across this issue? If so, were you able to resolve this? Please let me know. Also in Google I/O 2010, Next Gen Queries [this video] , Alfred Fuller

Cites Patrice · 2015-04 · Stack Overflow · on “App Engine Datastore - Indexes optimization”

The datastore never computes anything, it literally just reads in the order presented in the index. The presence of cost in the middle of the index for Q2 means that Q1 can't use it. It's that simple. It's ordered by type, then cost, then invented_at. I know it would be possible to have the system read just what it needs, but it won't skip over a column for that. There is the zigzag merge join [this video] algorithm that helps minimize the number of your indexes if you want, but not in this situation. You can also have more info here [LINK]