cl-zotero-connector

Common Lisp implementation of the zotero connector backend.

Motivation

The killer feature of Zotero is it's excellent collection of translators for scraping bibliographic data directly from publishers websites, library cataloges and similar sources. It's possible to run the standalone zotero translation server to make use of these powers in other software, but it remains limited compared to zoteros browser integrations, because it is not capable of accessing sources that require some kind of authentification via the browser.

This library provides an alternative backend server for the zotero connector browser plugin, allowing to export bibliographic data from it to any program without needing to run zotero.

API

Start the connector server by creating a server object:

connector:server

(make-instance 'lou.zotero.connector:server)

• initargs
  • :prefs: instance of connector:prefs. Can be used to (de)activate features of the http API.
  • :session-class: subclass of connector:save-session the server will use it as the class used for sessions.
  • :handler: function called after each api call by the browser extension. Takes one argument of the type stored in session-class. The default handler just prints a json representation to stdout.
  • :logger: function handling the http access log. The default logger prints the logs to stderr.

To stop the server:

(lou.zotero.connector:stop *)

connector:save-session

The save-session class is both used internally to store import data and as the primary format in which data is passed to the user via the handler function specified on the server class. The class is modeled after Zoteros internals this may change.

detailed documentation tbd.

TODO

This library is still WIP. The API may change.

• API for collection data
• handle timeouts/invalid requests more gracefully.
• improve documentation
• write tests

Zotero Connector API

Zoteros implementation of the Connector Server can be found in

zotero/chrome/content/zotero/xpcom/server/server_connector.js

Zotero is listening for http requests on port 23119

Endpoints

• implemented
  • /connector/ping
  • /connector/delaySync
  • /connector/saveItems
  • /connector/saveSnapshot
  • /connector/updateSession
  • /connector/saveSingleFile
  • /connector/saveAttachment
  • /connector/getSelectedCollection (no userfacing api)
  • /connector/saveStandaloneAttachment
  • /connector/hasAttachmentResolvers (always returns false)
• not implemented
  • /connector/detect
  • /connector/import
  • /connector/proxies
  • /connector/request
  • /connector/installStyle
  • /connector/getTranslators
  • /connector/getTranslatorCode
  • /connector/getRecognizedItem
  • /connector/getClientHostnames
  • /connector/saveAttachmentFromResolver

ping

Test connection
Accepts:
 Nothing
Returns:
 Nothing (200 OK response)

getSelectedCollection

Returns the full serialized collection tree (excluding non-editable libraries)
and the selected collection tree item.
Accepts:
 Nothing
Returns:
 libraryID
 libraryName
 collectionID
 collectionName

saveItems

Saves items to DB
Accepts:
 items - an array of JSON format items
Returns:
 201 response code with item in body.

saveAttachment

Attaches an PDF/EPUB attachment to an item saved with /saveItems or /saveSnapshot
URI params:
 sessionID
Expected headers:
 X-Metadata:
 - parentItemID
 - title
 - url
Returns:
 400 - Bad params
 200 - Non-writable library
 201 - Created

saveSnapshot

Creates a webpage item top-level item in Zotero
Called by the Connector when no translators are detected on the page
Accepts:
 uri - The URI of the page to be saved
 html - document.innerHTML or equivalent
 cookie - document.cookie or equivalent
Returns:
 Nothing (200 OK response)

saveSingleFile

Attaches a singlefile attachment to an item saved with /saveItems or /saveSnapshot
If data.snapshotContent is empty, it means the save failed in the Connector
And we fallback to saving in Zotero
Accepts:
 sessionID
 snapshotContent
 url - The URI of the page to be saved
 title
 cookie - document.cookie or equivalent
 detailedCookies
 proxy
Returns:
 Nothing (200 OK response)

saveStandaloneAttachment

Saves a standalone attachment
URI params:
 sessionID
Expected headers:
 X-Metadata:
 - parentItemID
 - title
 - url
Returns:
 400 - Bad params
 200 - Non-writable library
 201 - Created

updateSession

Accepts:
 sessionID - A session ID previously passed to /saveItems
 target - A treeViewID (L1, C23, etc.) for the library or collection to save to
 tags - A string of tags separated by commas
 note - A string to turn into a child note
Returns:
 200 response on successful change
 400 on error with 'error' property in JSON

getTranslators

Lists all available translators, including code for translators that should be run on every page
Accepts:
 Nothing
Returns:
 Array of Zotero.Translator objects

delaySync

detect

Detects whether there is an available translator to handle a given page
Accepts:
 uri - The URI of the page to be saved
 html - document.innerHTML or equivalent
 cookie - document.cookie or equivalent
Returns a list of available translators as an array

getRecognizedItem

Gets the top-level item created for a standalone attachment
Accepts:
 sessionID - A session ID previously passed to /saveItems
Returns:
 200

hasAttachmentResolvers

Checks if the item has OA attachments (in case PDF saving in connector failed).
Also checks custom resolvers.
Accepts:
 sessionID - A session ID previously passed to /saveItems
 itemID - The ID of the item to save alternative attachment for
Returns:
 200 - json boolean

saveAttachmentFromResolver

Accepts:
 sessionID - A session ID previously passed to /saveItems
 itemID - The ID of the item to save alternative attachment for
Returns:
 400 - Bad params
 201 - Created and attachment title
 500 - Failed to save

import

 Translates resources using import translators
 Returns:
 - Object[Item] an array of imported items

installStyle

Install CSL styles
Returns:
 - {name: styleName}

getTranslatorCode

Get code for a translator
Accepts:
 translatorID
Returns:
 code - translator code

getClientHostnames

Get a list of client hostnames (reverse local IP DNS)
Accepts:
 Nothing
Returns:
 {Array} hostnames

proxies

Get a list of stored proxies
Accepts:
 Nothing
Returns:
 {Array} hostnames

request

Test connection
Accepts:
 Nothing
Returns:
 Nothing (200 OK response)

Copytright Note

The documentation of the Zotero API above, is in part copied from the Zotero source repository. The Common Lisp implementation of the Zotero Connector API was possible because Zotero is free software, allowing me to discover API behaviour by reading the source code.

Zotero is Copyright © 2018 Corporation for Digital Scholarship http://digitalscholar.org.

Copyright © 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 Roy Rosenzweig Center for History and New Media, George Mason University, Fairfax, Virginia, USA http://zotero.org

The Corporation for Digital Scholarship distributes the Zotero source code under the GNU Affero General Public License, version 3 (AGPLv3).