Ruter AS is the public transport authority for the Oslo and Akershus counties in Norway. Formally a limited company – 60% of its shares are owned by the Oslo county municipality and 40% by that of Akershus – it is responsible for the administration, funding, and marketing (but not direct operation) of public transport in the two counties, including bus, Metro, Tram and ferry services. Ruter also has agreements with the Norwegian State Railway, concerning the regulation of fares on local and regional train services within the two counties.
The development of digital services at Ruter involves many actors. At the heart of Ruter’s digital platform is Ruter’s Core Mobility Platform, a service which is fully managed by Basefarm. The Core Mobility Platform itself consists of a Fleet Logistics service and a Reporting Service built on Amazon EC2 instances where Basefarm operates the key components Kafka, Cassandra, MQ (Message Queue) and several database services on Amazon RDS.

Solution

A number of complex requirements exist for monitoring the Core Mobility Platform, since a number of different consumers each need visibility of the components that they are responsible for. Ruter and Basefarm chose Datadog as the primary monitoring tool, because of its range of plugin extensions and the capability to correlate logs and Application Performance Management data within a single tool.

Datadog’s plugin approach enables Ruter and Basefarm to add-on different modules as needed. In this case study we focus on the elements of the platform where Basefarm has monitoring responsibility, which include the underlying AWS infrastructure, EC2 compute, storage and networking. All infrastructure and monitoring components are defined as IaC using HashiCorp Terraform. This enables Ruter and Basefarm to easily manage and monitor different environments. Resources are tagged as belonging to clusters and groups, which enables monitoring events and logs to be automatically associated with the correct services as scaling events are triggered and new resources are launched.

Using Datadog to forecast threshold breach

Datadog collects a wide range of metrics from heterogenous sources, it employs predictive algorithms to predict how specific metrics will continue to rise or fall, based on identified trends.

Basefarm is responsible for managing the different Cassandra Clusters for Ruter, forecasting growth in compute or storage of the cluster enables Basefarm to plan scaling and maintenance activity, before performance begins to degrade, this ensures responsiveness of the cluster, without hugely overprovisioning cluster resources.

Basefarm is using the AWS GuardDuty threat detection service to continuously monitor Ruter’s productions accounts for malicious activity and unauthorized behavior. GuardDuty uses Machine Learning based on a set of AWS log sources, when an anomaly is detected, the issue is displayed in AWS Security Hub for full transparency with the customer. Basefarm has integrated AWS Security Hub alerts with our service management system, and any alerts will be handled 24/7 by Basefarm’s Operations Center.