elational Databases typically make use of a rigid schema - predefined tables containing typed columns allowing for a rich set of functionality that would otherwise be impossible. It is both a major strength as well as a major weakness. On one hand strong typing allows databases to expose a rich set of operators, functions and functionality for each of the types. For postgres, this usually presents itself in the form of column types sql syntax to interact with them. On the other hand it means that all of the data in the table is uniform and deviations or alterations are rather difficult to do.

At the day job, I am in the process of migrating a number of applications

Read More

lasticsearch is an extremely powerful searchable document store. The more you use it, the more you learn about the more realize how deep the rabbit hole of possibility goes. Except for when it comes null values. The Achilles heel of elasticsearch. What it really boils down to is elasticsearch doesn't index null values or fields that are missing from a document. Even if you have set up an index mapping and told elasticsearch about your field.

As you might think it can get a little tricky searching document for a field value or where that field is null / missing - especially when combined with other field queries. For example lets say I have some documents that looks something like

Read More

lasticsearch is an amazing piece of technology. Built on top of luecine it offers all of he incredible search facilities that you'd expect from a full featured search. What makes elasticsearch so powerful, however, is the fact that it stores the actual data that was originally index as JSON documents. Basically, it is a Full Text Search Database more so than a search engine. This allows elastic search to do things that other search engines can't do like aggregations, scripted queries, multi-query searches, etc; All in addition to the expected searching capabilities like suggestions, spelling corrections, faceting, and so on. For these reasons people are using elasticsearch as the primary data store for massive amounts of data.

One thing

Read More

ately I have been using [Docker](http://docker.com) for development. It makes it really easy to stand up services and package entire stacks for projects. Most recently, I set up an elastic search instance, and the first thing I wanted to do was install the insanely helpful [Head](http://mobz.github.io/elasticsearch-head) and [Big Desk](http://bigdesk.org/) plugins. The official [elastic search Dockerfile](https://github.com/dockerfile/elasticsearch) is just elastic search, no plugins. And of course, containers are pretty much a black box. Every time it restart, everything is back to square one. Luckily, The image exposes a volume for persistent data, and that is all we really need to have to install plugins. Here

Read More

one of the things that makes ghost great is it's almost forced simplicity. However, with that comes a bit of rigidity. If you want something more than the provided simplicity, you might find your self pulling your hair out. Search is one of the thing that is lacking from the default ghost set up. The last incarnation of my blog had everything indexed in a Xapian search index. I really wanted to bring back a local search index. Luckily, in the 0.5.x series, they pulled some string so that you can use ghost as a plain npm module. This means you can build your own app around ghost. This should make search possible.

Xapian is a search

Read More

t has been about 3 years since I launched my blog. At the time the tech was pretty interesting. It was a Django project on mongrel2 app server backed by ZeroMQ and xapian as the search engine. I built out a modular Web UI with a WYSIWYG editor using mootools. But it was rather clunky in how it actually generated HTML and usually required some manual tweaking.

Out With The Old

The WYSIWYG Editor was probably the biggest problem. It took serious liberties in cleaning the HTML and usually resulted in me editing the code by hand in the end. Interesting code blocks with highlighting was a 3 step process and was almost always formatted wrong.

Managing Images was

Read More

o consider myself to be an equal opportunity coder. I like investigating different tools built in different languages to solve different types of problems. Recently at work I had the opportunity to play around with [elasticsearch](http://www.elasticsearch.org). Elasticsearch brings the whole distributed / non relational data craze to the world of search. It is actually a rather impressive piece of tech. If you are at all interested in search as a problem, I recommend you give it a look. In any event, I need to get some data into the search engine to play around with. Where could I get a good deal of data without doing the heavy lifting myself? Twitter! Sure the twitter activity stream.

Read More