Technology that can adapt to the ever-changing needs of ESco. Technology = people, processes and tools
Phase 3 ยท Late 2023 to July 2025

Hybrid build and buy: interchangeable adaptable infrastructure

Pivoted

Hypothesis

Could we build a hybrid, adaptable infrastructure, the Harbour and Crane, to process data specifically for the Single Customer View product, and in doing so prove the CRM-agnostic approach that Phases 1 and 2 pointed towards?

What we explored

We set up Google Cloud Platform (GCP) and MongoDB cloud environments and designed two core components:

  • The Crane, a custom ESco integration service to extract, transform, and load data from Myriad via APIs.
  • The Harbour, a central MongoDB data warehouse to receive and store that data for downstream use.

The intended use case was the Single Customer View product, which uses Orbit (Apteco) to present a consolidated view of subscriber data drawn from multiple sources. The environments were stood up and the architecture designed, but the full SCV proof of concept was not completed before SCV was paused in July 2025.

Phase 3 planning whiteboard showing internal phases of work across POC, environments, and infrastructure
Phase 3 planning whiteboard

What we found

GCP and MongoDB environments were configured and functioning. The Crane and Harbour pattern was designed and the infrastructure was in place โ€” but the full SCV proof of concept was not completed. Phase 3 got the environments ready; the POC work happened in Phase 4 against different use cases.

It also became apparent that the market appetite for the Single Customer View service was not strong enough to justify ongoing, exclusive investment. The commercial case for SCV as a standalone product did not hold up.

Decision and reasoning

Phase 3 ran from late 2023 through to July 2025. At that point, two factors prompted a pause rather than a full pivot:

  1. Weak commercial demand for SCV as a standalone product had been apparent for some time.
  2. News of Intermedia's acquisition of Gordon and Gotch (the company behind Myriad) in July 2025 โ€” with the transaction completing in September 2025 โ€” raised an urgent question: should the team continue focused on SCV, or prove that the Harbour and Crane pattern could work for the wider business?

SCV was paused while the team spent the summer and autumn of 2025 running a broader proof of concept, exploring whether the middle layer architecture could support other use cases beyond SCV. Two micro services were chosen as test cases: Books/Backs and Comm Docs (Commercial Documents). That work validated the approach at broader scope.

At the end of 2025, the decision was made to apply the architecture to a specific, high-value user need: promo code creation. Phase 4 began in December 2025.

Phase 3 architecture

The CRM-agnostic approach

The central objective of the architecture is to ensure ESco is not locked to Myriad, or any single CRM. By making backend systems interchangeable, ESco dramatically reduces vendor lock-in risk. If a new client wants to use their own CRM, or if ESco decides to replace Myriad entirely, the core microservices continue to function without a complete system rebuild.

This matters because the acquisition of Gordon and Gotch (the company behind Myriad) by Intermedia created genuine uncertainty about Myriad's long-term roadmap and pricing. The architecture is designed so that Myriad can be swapped out, or supplemented, without disruption to the systems around it.

Core components

The Crane
ETL / RTL Engine

The intermediary tool built to extract, receive, transform, and load data. Acts as the universal translator between the CRM and the microservices. ESco is currently evaluating whether to continue building this in-house or procure an iPaaS.

The Harbour
Process Queue System

The central data repository, built on MongoDB. Initially used in Phase 3 as a static data warehouse with a flexible schema. In Phase 4, repurposed to act as a process queue system.

Microservices
Modular business processes

Independent services (such as Aimee, eCommerce, document generation) that interact with the Harbour. Designed to be replaceable and independent of the core CRM.

Google Cloud Platform
Cloud infrastructure

Chosen to host the new architecture, including API endpoints and AI integrations, shifting ESco away from on-premises servers.

How data flows

Data processing is bi-directional. The flow works as follows:

  1. An event occurs in the CRM (for example, a subscription update).
  2. A webhook fires in real time and passes the data to the Crane.
  3. The Crane translates and loads the data into the Harbour as a queued item.
  4. An independent microservice reads from the queue, performs its specific task (such as fulfilment, reporting, or document generation), and writes the result back to the Harbour.
  5. The Crane detects the update and pushes the result back to the CRM.

This replaces the old approach of manual or scheduled polling, where systems were checked periodically rather than updated in real time.

Integration layer

The integration layer relies on an ESco API and webhooks. Webhooks allow the architecture to respond to real-time events in the CRM rather than batch-processing data on a schedule. This is a significant shift from how ESco systems historically operated and is central to making the architecture scalable and responsive.

Design principles

  • CRM agnostic: no component should be permanently tied to Myriad or any specific subscription management platform.
  • Interchangeable: individual components (Crane, Harbour, microservices, CRM) should be replaceable without rebuilding the whole.
  • Modular: each microservice does one thing and does it independently.
  • Momentum over perfection: accepted tools can be swapped via standard procurement once the proof of concept is validated, the priority is to prove the concept first.

Known risks and open questions

MongoDB as a queue system. External consultants flagged that document databases are not naturally optimised for high-throughput queueing semantics. The internal team accepted this risk and proceeded with MongoDB to maintain momentum, noting that the underlying tools can be swapped out if the proof of concept validates the approach but MongoDB buckles under production queue loads.

Build or buy for the Crane. Whether ESco builds the ETL/RTL Crane entirely in-house or procures an external iPaaS platform is still unresolved and sits as an open procurement decision.

Key supporting documents

  • Single Customer View - Definition V1.0.docx
  • SCV Business Case v1.docx
  • Tech Transformation Phase 3 Interval 1_BoardMeetingUpdate_08072025.pdf