Consuming Zotero Libraries into Drupal with Feeds/Xpath Parser: Revisited

For the original post, visit https://www.utsc.utoronto.ca/digitalscholarship/content/blogs/consuming-zotero-libraries-drupal-feedsxpath-parser

Recently we found that we needed to revisit our old friend the Drupal Feeds module and get it to play nice with Zotero's API. This time we wanted to use it with Culinaria's Zotero library. Our goal was to pull all of the items and their content from the library into their UTSC website (much like how an RSS feed works). In the original post, Kirsta brought up that she was having trouble when the processor was set to periodically import items to keep the feed up to date with the Zotero Library. What happened was that every time the import ran, the Processor wouldn't just add any new items and update existing ones , but it would create a new item every single time from the API. This meant that there were lots of duplicate entries in Drupal that needed to be removed. We got it to work now, but first:

A Recap

How to set up a Zotero feed in Drupal:
1. Create 2 new content types: Custom Feeds Processor, Custom Feed Item (under Structure)
2. Add a new Feed Importer (under Structure)
3. Add a new Custom Feed Processor (under Add Content)
4. Configure the new Feed Importer: attach it to the Custom Feed Processor, map it to the fields in the Custom Feed Item which will be pased with XPath (under Structure)
5. Go to the Custom Feed Processor-> use XPath as your field output (under Content)
6. Run the Import in your Custom Feed Processor. You should see that every entry in the Zotero library has created a new custom feed item in Drupal in your Content view. You can then create a page with all of your feed items using the Views module.

Results

You can see the final page here: https://www.utsc.utoronto.ca/digitalscholarship/culinaria/resources

We needed to set a unique key that we can use to match up with any existing feed item in Drupal. It seemed to work best when we used the element zapi:key in the Title field. That way every time the import runs, it checks if that key exists and if it does it will update (but not create) a new feed item with that key.

These are the fields we selected in the Feed Importer  to map to the processor.  We also set Title as the unique key in the target configuration column. 

Here are the XPaths we used in our Custom Feeds Processor:

 

Other Tips

The Zotero API maxes out at 100 items, but we had 115 items. Our workaround was to import all the items in the library by running your processor twice, once by sorting your items in descending order, once by sorting your items in ascending order. Then we set the API to sort by descending order from hereon out so it will only grab the 100 most recent items.

Earlier we used Oxygen to get the XPaths, but Google Chrome has an XML tree extension that you can use that will also quickly get you automatic XPaths: https://chrome.google.com/webstore/detail/xml-tree/gbammbheopgpmaagmckhp...

In your Feed Importer, it's useful to use the Tamper settings to clean up your feeds.  We used HTML entity decode  and URL decode which converts hex values such as "&" into "&".  You can also use plugins such as Find and Replace, Filter empty values, or Explode.

You can turn tags in the Zotero into a taxonomy in Drupal, then create a Menu for those terms.  First you'll need to create the new terms from your Feed Importer: