Google App Engine Shim? NDB Middleware receiving an error

Question

I have the following sync service written as a microservice.

Per the google cloud instructions I could add the middleware? shown at the bottom as a type of shim? I don't have a good mental model of what this does, could I ask for some help clarifying.

Should I expect that code at the bottom to be available and in memory when the /keepalive/ path is traversed?

app.route('/keepalive/', methods=['GET'])
def keepalive():
 collection_dbs, collection_cursor = model.Collection.get_dbs(
 order='name'
 )
 ...
 return 'ok'
...
import google.appengine.api
client = ndb.Client()
def ndb_wsgi_middleware(wsgi_app):
 def middleware(environ, start_response):
 with client.context():
 return wsgi_app(environ, start_response)
 return middleware
app.wsgi_app = ndb_wsgi_middleware(google.appengine.api.wrap_wsgi_app(app.wsgi_app))
Thanks

Answer 1

1. Since you're using the cloud version of ndb, you need a client - which is why you're doing

   client = ndb.Client()
   

2. The documentation says any use of ndb must be within the context of a call to context() i.e.

   client = ndb.Client()
   with client.context():
    # Use NDB for some stuff
    pass
   

3. I'm not sure what model.collection means in your code but if it's coming from ndb, then you would ordinarily have had to encase it within (wrap it) with client.context() i.e.

   with client.context():
    collection_dbs, collection_cursor = model.Collection.get_dbs(
    order='name'
    )
   

4. Google documentation says that if your webapp uses a WSGI framework, you can prevent having to manually wrap each of your calls to ndb with a with client.context(), by creating a middleware to automatically do this for you. Since it's a middleware, I believe you have to (should) initialize/define it at the top of your code so that it applies to the rest of your code

   from flask import Flask
   from google.cloud import ndb
   client = ndb.Client()
   def ndb_wsgi_middleware(wsgi_app):
    def middleware(environ, start_response):
    with client.context():
    return wsgi_app(environ, start_response)
    return middleware
   app = Flask(__name__)
   app.wsgi_app = ndb_wsgi_middleware(app.wsgi_app) # Wrap the app in middleware.
   app.route('/keepalive/', methods=['GET'])
   def keepalive():
    collection_dbs, collection_cursor = model.Collection.get_dbs(
    order='name'
    )
    ...
    return 'ok'
   

5. If you want something similar to your Python 2 code (where you didn't have to create clients or worry about context), you can use the bundled services SDK

• ffejrekaburb: thank you for the reply. This was awesome and clears up a lot of doubt in my mind. I moved it to the top as I don't have the fully python implementaton in my head. I'm still getting the same error though, I asked another question if you could help that'd great. I don't know how to interpret the _retry or unpack. Do we want urllib3 to be using sockets?
• NoCommandLine: You're doing app.wsgi_app = ndb_wsgi_middleware(google.appengine.api.wrap_wsgi_app(app.wsgi_app)). I don't know if that's the correct thing in this situation (I know that code is for when you're using the bundled services SDK which you aren't doing here). Why not try the middleware code in my solution (it's directly from Google's documentation) and is for Cloud NDB
• ffejrekaburb: Thank you, I ran that experiment and replied in the other question. I don't know how this code is/should be traversing in the network to get to the datastore.