The challenge

A solid data warehouse is of crucial importance for a Fintech organization that is supervised by AFM. If the data that is requested or displayed is incorrect, it has a great impact on many people. BUX’s existing data warehouse was built on-premise and was becoming obsolete. Innovating the existing warehouse would be a very costly affair and was also not in line with BUX’s cloud-first strategy.

The solution

After an orientation phase, in which BUX listed the pros and cons of the various cloud providers, BUX decided to transfer the entire data stack to Google Cloud Platform. The flagship BUX Zero was already running on GCP, and they were very satisfied and familiar with it. Moreover, the cultural fit of Google - in which open source and freedom of choice are of paramount importance - is also best in line with BUX's vision. All in all, it was therefore a logical choice to move the BI environment to Google's Cloud Platform.

The choice for the data warehouse fell on Snowflake. BigQuery might seem more logical - since it is a product of Google itself - but turned out not to be entirely suitable for BUX's use case (read more about Snowflake vs Bigquery).

BUX engaged g-company to help with the transfer of data from the various data sources. g-company laid the foundation for a good infrastructure for the new data stack. This was a fairly complex project, in which the entire network first had to be devised and designed. Then the application landscape was built with Kubernetes running AirFlow (a scheduler for data pipelines). Everything is connected, so that BUX only has to write the data pipelines.

It is quite unique that so much effort is put into modernizing the infrastructure where a data project is fundamentally involved. But for a modern company operating on a scale like BUX, it was essential to get everything right, from the ground up.

In the meantime, BUX has hired two data engineers and g-company has transferred the knowledge to BUX, so that they are in control themselves. All the code that g-company has written is now completely mastered by the engineers.

The result

BUX now has an infrastructure that makes it possible to set up all their data pipelines in a scalable, testable, repeatable and manageable way. Three Kubernetes clusters have been created - for development, testing and production - with which BUX can put all data streams in a development process. Everything runs in the same way through infrastructure-as-code, which also makes it very maintainable. In short: the entire data stack is repeatable and scalable, which is quite exceptional. infrastructure-as-code, which also makes it very maintainable. In short: the entire data stack is repeatable and scalable, which is quite exceptional.

In short: the entire data stack is repeatable and scalable, which is quite exceptional.