Relational APIs with Node.js Tastypie and RethinkDB

Eric Satterwhite
2016 Aug12

ne of the more tricky and debated topics in API circles is how to handle relational data. How deep into your data tree should it go, how to accept it from the client and how much to return to the client. One could choose to treat each resource as a silo and force the client to individually create each object and piece all of the relations together. However, that is not a great user experience. It also leads to a very large number of requests and data transfer, which has been a blemish on the idea of a REST API for a while. The emergence of modern mobile devices which limited bandwidth and performance profiles need a more streamlined way doing things.

Take, for example, a photo gallery. A Gallery would have zero or more Photos and each Photo would have a set of Tags for categorizing. Lately, I have been enamored with RethinkDB - a beautiful blend of Document storage and relational databases, which allows to model our example in any number of ways. Let's take a traditional approach. In the silo-ed Resource world we would have to create each of those entities from the lowest level up:

Issue a POST request for each Tag
Issue a POST request to create each Photo
Issue a PUT for each Photo to assign tags
Issue a POST request to create the Gallery
Issue a PUT request to add Photos to it

For a new Gallery with 10 Photos with 3 Tags each, that is pushing 50 requests. With the v3.0.0 release of Tastypie-Rethink, you link any number of resources together to handling the relationship management for you. More importantly, expose an API that makes this process significantly easier for end clients. Install the required NPM packages

npm install tastypie@^5.0.0 thinky @tastypie/rethink

Assuming we have 3 thinky models Gallery, Photo, Tag:

Gallery = thinky.createModel('gallery',{
  title:thinky.type.string().required()
});

Photo = thinky.createModel('photo',{
  title:thinky.type.string(),
  src: think.type.string()
});

Tag = thinky.createModel('tag',{
   name: thinky.type.string().required(),
   slug: thinkt.type.string()
   photo_id: thinky.type.string()
});

Tag.pre('save', function( next ){
   this.slug = slugify( this.name ),
});

Gallery.hasAndBelongsToMany(Photo,'id,'id');
Photo.hasMany(Tag, 'id','id');

API [eɪ.piˈaɪ], -n, --noun

abbreviation for application programming interface: a way of communicating with a particular computer program or internet service

Simple - it isn't pretty or perfect, but it will illustrate the example. Now, we need to expose the API by setting up and linking our resource. The Tastypie Rethink package ships with 3 new field types to do this: hasone, hasmany, and document

Field Types

HasOne: The hasone field handles direct relations, or foreign key relations. In the thinky world this is the hasOne and belongsTo relation type.

HasMany: the hasmany field handles one-to-many and many-to-many relations. In the thinky world the is the hasMany and the hasAndBelongsToMany relation types

Document: The document field is for dealing with is commonly referred to as subdocuments or nested objects. These do not get saved to the database in a separate table, but provide a layer of validation, formatting and consistency over user input. They are also included in the endpoint schema definition for greater introspection be clients.

// Tag Resource
var RethinkResource = require('@tastypie/rethink')
  , Tag = require('../models/tag')
  , TagResource
  ;

TagResource = RethinkResource.extend({
  options:{
    name:'tag'
   ,queryset:Tag.filter({})
  }
 ,fields:{
    name : { type: 'char', nullable: false }
   ,slug : { type: 'char', readonly: true }
 }
});

// Photo Resource
var RethinkResource = require('@tastypie/rethink')
  , Photo = require('../models/photo')
  , TagResource = require('./tag')
  , PhotoResource
  ;

PhotoResource = RethinkResource.extend({
  options:{
    name:'photo'
   ,queryset: Photo.getJoin({tags:true}).filter({})
  }
 ,fields:{
    name : { type: 'char' }
   , src: { type: 'char' }
   , tags : { type: 'hasmany', to: TagResource }
 }
});

Pretty simple. As far as our ReST API is concerned, A Gallery has many Photos, and a Photo has many Tags - Let's plug it into a hapi server:

const server = new hapi.Server();
const api = new tastypie.Api( 'api/v1' );

server.connection({port:3000});

api.use( new GalleryResource() );
api.use( new PhotoResource() );
api.use( new TagResource() );

server.register([ api ], function( err ){
  server.start( console.log )
})

Done. That's it. We have a Photo Gallery API that can handle all sorts of incoming data. Let's create a new photo gallery with 3 photos with 3 tags each. To do that, we need to post all of the data as a single json document with all of the data inlined :

curl -XPOST -H "Content-Type: application/json" http://localhost:3000/api/v1/gallery -d '{
  "title":"This is a gallery",
  "photos":[{
      "name":"photo 1",
      "src":"http://photos.com/random-1.jpg",
      "tags":[{
        "name":"a"
      },{
        "name":"b"
      },{
        "name":"c"
      }]
   },{
      "name":"photo 2",
      "src":"http://photos.com/random-2.jpg",
      "tags":[{
        "name":"d"
      },{
        "name":"e"
      },{
        "name":"f"
      }]
   },{
      "name":"photo 3",
      "src":"http://photos.com/random-3.jpg",
      "tags":[{
        "name":"g"
      },{
        "name":"h"
      },{
        "name":"i"
      }]
   }]
}'

By default Relations will be returned as URIs, which in can be used in place of full objects to form relations. This also cuts down on the payload sizes. So for the above example, you would get a response like the following:

{
   "title":"This is a gallery"
   "id":"8da953a2-4bb4-4364-8ee4-ae742de62ce6",
   "uri":"/api/v1/gallery/8da953a2-4bb4-4364-8ee4-ae742de62ce6",
   "photos":[
     "/api/v1/photo/479c597a-d246-46c6-ab2a-292adcea7e53",
     "/api/v1/photo/5189c19d-97f6-494a-8641-f34095bb4e60",
     "/api/v1/photo/c91564ac-c16a-43aa-ad20-b07dfb101e65"
   ]
}

If you wanted to remove the last two photos, you would just have to issue a patch request with the list of one uri:

You can use URIs, objects or any combination of the two to create relations between new and existing objects

curl -XPATCH -H "Content-Type: application" http://localhost:3000/api/v1/gallery/8da953a2-4bb4-4364-8ee4-ae742de62ce6 -d '{
   "photos":[
     "/api/v1/photo/479c597a-d246-46c6-ab2a-292adcea7e53"
   ]
}'

Sending a URI will create a new relationship between the objects in question. Sending an object will result in a new object being created as well as the relation.

If you want all of the data for the photo returned, you would need to set the full option on that field to true

fields:{
  photos:{ type: 'hasmany', full: true } 
}

This will fully render the photo field, but the tags will still remain as an array of URIs.

{
   "title":"This is a gallery"
   "id":"8da953a2-4bb4-4364-8ee4-ae742de62ce6",
   "uri":"/api/v1/gallery/8da953a2-4bb4-4364-8ee4-ae742de62ce6",
   "photos":[{
       "id":"479c597a-d246-46c6-ab2a-292adcea7e53"
       "name":"photo 1",
       "src":"http://photos.com/random-1.jpg",
       "url":"/api/v1/photo/479c597a-d246-46c6-ab2a-292adcea7e53",
     },{
       "id":"5189c19d-97f6-494a-8641-f34095bb4e60"
       "name":"photo 2",
       "src":"http://photos.com/random-2.jpg",
       "uri":"/api/v1/photo/5189c19d-97f6-494a-8641-f34095bb4e60",     
     },{
       "id":"c91564ac-c16a-43aa-ad20-b07dfb101e65"
       "name":"photo 3",
       "src":"http://photos.com/random-3.jpg",
       "uri":"/api/v1/photo/c91564ac-c16a-43aa-ad20-b07dfb101e65"
   }]
}

The RethinkDB resource for tastypie exposes a great deal of functionality in 3 new field types ( hasone, hasmany and document ) and some additional filters to dealing with dates ( year, month and day ) making it dead simple to stand up expressive and usable ReST APIs in minutes

api tastypie hapi rethinkdb node