Our Journey to an Event-Sourcing Middleware @ Eurostar

Data access in Thalys and Eurostar was already a bit of a tangled web, with legacy systems, proprietary solutions, different versions of the same information and quality issues. Some processes even lacked digitalization. The merger with Eurostar and Thalys only added to the complexity, making it clear that we needed better, more interconnected tools.

We decided to tackle these challenges head-on by building "the data hub"—a solution designed to address both technical and organizational issues and reshape our approach to data. Our journey began a year ago with a team dedicated to this vision, incorporating various key ingredients:

• Event sourcing for traceability, full-ordering, and strong modeling, which are particularly beneficial for train operations.
• Pushing left on data quality and governance to ensure high-quality analytics.
• Developing our own message store in Go and Protobuf to handle cross-tech stack performance issues.
• Creating an inner-sourced TypeScript framework for quick projections and better collaboration with other teams.
• Implementing a data lakehouse based on Databricks to elevate our analytics capabilities.

This talk will cover our journey so far: the vision, the easy parts, the difficult parts, the wins, the losses, the bumps, the pains, and some successes.