Orchestrate.io Client API Design

Question

I've written a Python client for a new NoSQL database as a service product called Orchestrate.io. The client is very straightforward and minimal. It uses the requests library, making the underlying code even more streamlined.

I've been using the service as part of the private beta. However, Orchestrate.io went live to the public today. They have added a few new features to the API and I would like to include them in the Python client. With these updates, I'm considering other design choices as well.

I am relatively new to writing such an API and I would like to get feedback on the current design and perhaps suggestions for how it can be improved. Personally, I like where it is now because it is super simple/minimal. That said, I am open to making changes that will make the code most useful to the community.

Below are few things that I'm considering in the next round of updates:

• Putting the client and each service (Key/Value, Search, Events, ect...) into their own classes
• Implementing optional success and error callbacks
• Providing an asynchronous option (currently requests are blocking)
• Improved error handling

Here is the code in its current state (also available here):

'''
A minimal implementation of an Orchestrate.io client
'''
import requests
# Settings
logging = False
auth = ('YOUR API KEY HERE', '')
root = 'https://api.orchestrate.io/v0/'
header = {'Content-Type':'application/json'}
# Auth
def set_auth(api_key):
 global auth
 auth = (api_key, '')
# Collections
def delete_collection(collection):
 '''
 Deletes an entire collection
 '''
 return delete(root + collection + '?force=true') 
# Key/Value
def format_key_value_url(collection, key):
 '''
 Returns the url for key/value queries
 '''
 return root + '%s/%s' % (collection, key) 
def get_key_value(collection, key):
 '''
 Returns the value associated with the supplied key
 '''
 return get(format_key_value_url(collection, key))
def put_key_value(collection, key, data):
 '''
 Sets the value for the supplied key
 '''
 return put(format_key_value_url(collection, key), data)
def delete_key_value(collection, key):
 '''
 Deletes a key value pair
 '''
 return delete(format_key_value_url(collection, key))
# Search
def format_search_query(properties = None, terms = None, fragments = None):
 '''
 propertes - dict: {'Genre' : 'jazz'}
 terms - list, tuple: ['Monk', 'Mingus']
 fragments - list, tuple: ['bari', 'sax', 'contra']
 '''
 def formatter(items, pattern=None):
 result = ''
 for i in range(0, len(items)):
 item = items[i]
 if pattern: result += pattern % item 
 else: result += item
 if i < len(items) - 1: result += ' AND '
 return result
 query = '' 
 if properties:
 query += formatter(properties.items(), '%s:%s') 
 if terms:
 if properties: query += ' AND '
 query += formatter(terms)
 if fragments:
 if properties or terms: query += ' AND '
 query += formatter(fragments, '*%s*')
 return query
def format_event_search(span, start, end, start_inclusive = True, end_inclusive = True):
 '''
 Formats a query string for event searches.
 Example output: Year:[1999 TO 2013}
 span - string: YEAR, TIME
 start - string: beginning date or time
 end - string: ending date or time
 start_inclusive - boolean: whether or not to include start
 end_inclusive - boolean: whether or not to include end
 '''
 result = span + ':'
 result += '[' if start_inclusive else '{'
 result += start + ' TO ' + end
 result += ']' if end_inclusive else '}'
 return result 
def search(collection, query):
 '''
 Searches supplied collection with the supplied query
 '''
 return get(root + "%s/?query=%s" % (collection, query))
# Events
def format_event_url(collection, key, event_type):
 '''
 Returns the base url for events
 '''
 return root + '%s/%s/events/%s' % (collection, key, event_type)
def get_event(collection, key, event_type, start='', end=''):
 '''
 Returns an event
 '''
 return get(format_event_url(collection, key, event_type) + '?start=%s&end=%s' % (start, end))
def put_event(collection, key, event_type, time_stamp, data):
 '''
 Sets an event
 '''
 return put(format_event_url(collection, key, event_type) + '?timestamp=%s' % (time_stamp), data)
def delete_event(collection, key, event_type, start='', end=''):
 '''
 Delets an event
 '''
 return delete(format_event_url(collection, key, event_type) + '?start=%s&end=%s' % (start, end))
# Graph
def format_graph_url(collection, key, relation):
 '''
 Returns the base url for a graph
 '''
 return root + '%s/%s/relations/%s/' % (collection, key, relation)
def get_graph(collection, key, relation):
 ''' 
 Returns a graph retlationship
 '''
 return get(format_graph_url(collection, key, relation))
def put_graph(collection, key, relation, to_collection, to_key):
 '''
 Sets a graph relationship
 '''
 return put(format_graph_url(collection, key, relation) + ('%s/%s') % (to_collection, to_key))
def delete_graph(collection, key, relation):
 '''
 Deletes a graph relationship
 '''
 return delete(format_graph_url(collection, key, relation))
'''
Convenience methods used by client for generic, get, put and delete. 
'''
def get(url):
 log('GET', url) 
 return requests.get(url, headers=header, auth=auth) 
def put(url, data=None):
 log('PUT', url)
 return requests.put(url, headers=header, auth=auth, data=data)
def delete(url):
 log('DEL', url)
 return requests.delete(url, auth=auth)
def log(op, url):
 if logging: print '[Orchestrate.io] :: %s :: %s' % (op, url.replace(root, ""))

Answer 1

1. This is a very "thin" layer around the Orchestrate.io database. By "thin" I mean that it provides no abstraction and no mapping of concepts between the Orchestrate and Python worlds. Without your module, someone might have written a sequence of operations like this:

   import requests
   value = requests.get(root + collection_name + '/' + key, headers=header, auth=auth)
   requests.delete(root + collection_name + '/' + key, headers=header, auth=auth)
   

   but with your module they can write it like this:

   import orchestrate
   orchestrate.auth = auth
   orchestrate.root = root
   value = orchestrate.get(collection_name, key)
   orchestrate.delete(collection_name, key)
   

   which you have to admit is not much of an improvement. All you've done is factor out a bit of boilerplate, which any Python programmer could easily have done for themselves.

   What you should do is figure out some way to map concepts back and forth between the Orchestrate and Python worlds. For example, a key-value store is very like a Python dictionary, so wouldn't it be nice to be able to write the above sequence of operations like this:

   import orchestrate
   conn = orchestrate.Connection(root, api_key)
   collection = conn.Collection(collection_name)
   value = collection[key]
   del collection[key]
   

   The advantage of this kind of approach is not just that it results in shorter code, but that it interoperates with other Python functions. For example, you'd be able to write:

   sorted(data, key=collection.__getitem__)
   

   or:

   "{product} has {stock_count} items.".format_map(collection)
   

2. By using the requests module, you require all your users to install that module. If you are trying to write something for general use, you should strive to use only features from Python's standard library. Even if requests is easier to use than urllib.request, a bit of inconvenience for you could save a lot of inconvenience for your users if it would enable them to run your code on a vanilla Python installation.

3. There doesn't seem to be any attention to security or validation. You should strive to make your interface robust against erroneous or malicious data. Some examples I spotted:

   1. What if root doesn't end with a /? It would be safer to use urllib.parse.urljoin instead of string concatenation.

   2. What if collection or key contains a / or a ? or a %? You might consider using urllib.parse.quote_plus.

   3. Instead of appending ?force=true, why not use the requests module's params interface?

   4. Similarly for ?query=%s. Using the params interface would ensure that the query is properly encoded.

   5. format_search_query and format_event_search look vulnerable to code injection attacks.

• JeremyFromEarth: This is great feedback and I really appreciate you taking the time to reply. I really like the idea of building a mapping to the Python language. I think that I will stick w/ the requests module. I have included a setup.py file that installs the dependency. The security/validation points are very helpful and I will address each one. Thank you!