Building materialised views for linked data systems using microservices

In the BBC’s Content Distribution Services division, we build and maintain systems that provide content metadata to a wide range of audience-facing products.

Our current architecture for distributing tagging metadata consists mainly of two RDF-based read and write APIs feeding off a central triplestore. This single storage setup for all operations imposes restrictions on performance and scalability.

I will talk about our work done to create an event-driven distribution pipeline that generates materialised views of tagging metadata.

The new microservices architecture comprises of small, single-purpose services, lambda functions, event stores, queues, streams etc. The views are built on data stores that are optimised to serve specific query profiles thus improving the overall performance and scalability of the system.

Augustine Kwanashie
Software Engineer, BBC

Augustine works as a Senior Software Engineer in the BBC’s Content Distribution Services division. He helps build and maintain APIs (and supporting infrastructure) that store and serve metadata for BBC content in the Linked Data space. Previously he was a PhD researcher at the University of Glasgow working on designing efficient algorithms for matching problems.

Video
Presentation


This is what Connected Data London 2018 brought to the fore. Connected Data London 2019 is on! Secure your chance to learn from experts and innovators, get your ticket early! Limited number early bird tickets available.