Polyglot Persistence and Event Store Database

With the advent of polyglot persistence, the rise of specialized databases seemed inevitable. If your system already has more than one database, no reason to stop at just two. I’m interested in working with more different kinds of databases and experimenting with the variety of what is available. I’ve worked with a bunch of different SQL databases, a couple of different document databases, some key value databases and a full text database; I’ve talked about a my experiences with a graph database some before.

I hadn’t tried a spatial database, time series database, columnar database, or an event store database. The spatial database clearly needs a specific kind of data to work, and is good for solving a specific kind of problem. Time series databases also need a specific kind of data, and solve particular kinds of problems so you need to find the right kind of problem to give one a fair shake. Columnar databases seem harder to get a feeling for since it can do most things that a regular SQL database can do, but the performance characteristics are very different. This means you could build a toy system based on one that looks perfectly reasonable, but the real test would be how it performs under load; you would need to try that to understand how it really works. So when I ran across an event store database I felt like it was time to give it a try and better understand what it could do.

Event Store is a database designed to store event data and not do anything else. It doesn’t have any significant search capabilities, and no update or delete. It does have a built in feature to create an atom feed. While I haven’t used the feature, it is an interesting idea. It’s all about event streams and subscriptions.

I worked on a toy project with it and it fit the event store niche well. I can’t say much about its performance and scalability. But it seems designed with those aspects in mind, and due to the limited feature set, outlined above, it can make significant design tradeoffs to accomplish this. The .net client was written with modern features and idioms, which is nice in comparison to a lot of other database libraries that aren’t written with the .net specific features in mind.

I was using it to store events that one microservice was creating, and used it implement web service endpoints that other services could poll to get this event data. You could implement a similar system using a variety of message buses. Compared to a message bus-based design, this particular design means that the consumers need to know more about where the events are coming from, but it also makes it easier for consumers to go back and reprocess historical events. This design also reduces coupling to any particular technology.

Overall my experience with polyglot persistence have been positive. There are a couple of considerations I would suggest if you were looking to do this as well. Any particular piece of data should have one store that is the owner of the data. Other data stores can cache the data for ease of use or performance, but one is always the truth everything else is just a representation. The polyglot persistence approach does ratchet up complexity for getting started, but in the long term it can significantly improve the overall design.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s