Hexagonal architecture

Decision

Each team decides on their own to adopt the Hexagonal Architecture (also known as Ports and Adapters Architecture). We will establish clear guidelines and standards to maintain consistency after gaining experience inside the teams.

Problems

Need for dedicated teams for domain-specific services

We have identified the importance of having dedicated teams that can independently develop and maintain domain-specific services. However, our current software structure does not adequately support this segregation of responsibilities, leading to inefficiencies and difficulties in managing domain-specific functionalities.

Mixing of business logic and framework-specific code

Our backend monolith contains a significant amount of code where business logic and framework-specific code are intertwined. This integration makes it challenging to distinguish between the two and poses difficulties in maintaining, testing, and modifying the codebase.

Complex and inadequate testing

Some portions of our codebase, specifically Filch and Hermione related modules and apps, are exceptionally difficult to test properly. The lack of clear separation and dependencies on external systems make it challenging to isolate these components for thorough testing, leading to potential quality issues and increased debugging efforts.

For example the createCampaignsFromAutomation.test.js requires us to spy and mock other executors, and we have a lot of database dependencies we need to insert before running the tests. Big parts of the tests are testing implementation details instead of the business logic itself.

Inability to scale with increasing campaign volumes

As the volume of campaigns grows, our current infrastructure struggles to scale efficiently to meet the rising demands. This limitation hampers our ability to deliver a reliable and performant service to our users, resulting in potential disruptions and customer dissatisfaction.

Transitioning while accessing existing data

We are embarking on building the core of our application from scratch, but we still need to access and utilize data that resides in various modules of the existing monolith. This requirement presents a significant challenge as we need a way to seamlessly integrate and access the existing data while building the new core.

Flexible technology selection during infrastructure migration

As part of our infrastructure migration, we anticipate the need to adopt new technologies and frameworks that best suit our evolving requirements. However, our current software structure lacks the necessary flexibility to easily incorporate and experiment with different technologies, potentially limiting our ability to make informed decisions.

Non predictable server

The current API we have does not always deliver consistent results and there are no clear CRUD endpoints since we have very specific routes for each user interaction in the frontend. It is unclear which database model is used due to the way we inherit our models. The main source for this is orderDefinition which always needs to communicate to Product Engine in order to get the correct results.

Context

Currently, our software system is structured as a monolithic application, primarily located within the shared/monorepo folder. This monolith encompasses multiple services and modules, all accessing the same database tables. While efforts have been made to apply Test-Driven Development (TDD) principles, the tight coupling between business logic and frameworks/technologies often makes it challenging to achieve comprehensive test coverage.

Teams face coordination challenges when modifying database-related code, as multiple modules rely on the same database tables. This dependency creates a need for synchronized changes and complicates the process of introducing modifications to the database schema or query logic.

Furthermore, we have deployed the Filch job runner via ECS. However, parallelizing the processing of jobs within Filch requires significant effort, hindering its efficiency and scalability.

To address these issues, the Collect team has undertaken the migration of the recipient import process. This migration involves leveraging Lambda functions and applying the Hexagonal Architecture to improve the design and functionality of the recipient import functionality.

Options

The following architectural styles were considered:

1. Hexagonal architecture
2. Layered Architecture
3. Microservice architecture

Reasoning

We have chosen the Hexagonal Architecture for the following reasons:

Modularity

By structuring the system into hexagonal layers, we achieve a clear separation of concerns. The core application is shielded from external concerns, such as user interfaces, databases, or external services. This modularity improves maintainability and enables incremental development and deployment.

Testability

The Hexagonal Architecture supports extensive testing by allowing easy substitution of external dependencies with test doubles. This enables thorough unit testing, integration testing, and end-to-end testing of the core application logic, resulting in improved software quality.

Flexibility and adaptability

The loose coupling between the application core and external components ensures that changes in one part of the system have minimal impact on other parts. This flexibility allows us to evolve and adapt the system over time without disrupting the entire architecture.

Domain-driven design

The Hexagonal Architecture aligns well with domain-driven design principles. By emphasizing the core domain and use cases, it helps us create a software system that closely reflects the problem space, improving maintainability and enabling effective collaboration between domain experts and developers.

The Layered Architecture approach separates components into layers (such as presentation, business logic, and data access). However, it can lead to tight coupling between layers and makes testing and modifying individual components more challenging.

Microservices Architecture decomposes the system into small, independent services. While it provides scalability and isolation benefits, it introduces complexities related to service discovery, network communication, and distributed transaction management. Before splitting into microservices, we first need to decouple the business logic from the rest of the codebase and identify the domains so that the teams can work individually inside their own domain(s).

Consequences

How do we implement this change?

Each team individually can implement the Hexagonal Architecture in small iterations. For now mainly by focussing on services that are extracted out of the monolith already.

Who will implement the change?

The Collect team started using hexagonal architecture in the recipient-importer, and share their learnings afterwards. Also Create team looked into the topic to reduce dependencies between modules when working on Hermione endpoints.

How do we teach this change?

Philip will run a workshop in one of the following Learning Fridays.

What could go wrong?

Over-engineering

It's possible to overcomplicate the system by introducing unnecessary abstractions or layers. This can lead to additional complexity and decreased development velocity. Care should be taken to strike the right balance between modularity and simplicity.

Learning curve

Adopting the Hexagonal Architecture may require developers to familiarize themselves with new concepts and patterns. This learning curve could potentially slow down initial development efforts until the team becomes proficient in applying the architectural principles effectively.

Performance considerations

The use of additional layers and abstractions can introduce a performance overhead. Critical performance-sensitive components should be carefully identified and optimized to ensure that the architecture does not negatively impact the overall system performance. Since our application is not as performance critical, this should not affect us too much.

Maintaining consistency

With the flexibility of the Hexagonal Architecture, there is a risk of inconsistent implementation across different components or modules. It's essential to establish clear guidelines and standards to maintain consistency and ensure that all parts of the system adhere to the architectural principles.

What do we do if something goes wrong?

Rolling out hexagonal architecture in small iterations enables us rolling back the parts where it proved problematic without needing to roll back everything.

If consistency problems arise after specifying guidelines and standard, they should be addressed in cross-team retro.

But overall, hexagonal architecture is so fundamental that it might be hard to undo once we transitioned a service into that format.

What is still unclear?

Hexagonal architecture does not solve the problem we have with the non-predictable server. Therefore, as mentioned by the Price team, we need to start with refactoring Product Engine and make our endpoints in Hermione more consistent. It's unclear if any team will focus on this in one of the following quarters.

There are certain aspects that may still be unclear when it comes to Hexagonal Architecture. These include adapting existing systems, defining boundaries, managing data flow, and understanding trade-offs and best practices. To overcome these uncertainties, it is important to seek guidance, conduct experiments, and engage in knowledge-sharing within the team.