(1) Building a complete Tweet index by Twitter Team
- On the application of the pipe-and-filter architectural pattern to build a system that processes a stream of hundreds of billions of Tweets.
(2) Handling five billion sessions a day – in real time by Twitter Team
- The description of a software architecture for a reliable and highly performant system.
(3) Data Lake by Martin Fowler
- Looking at big data from a software engineering perspective
- Some requirements:
- Never loose data, which means that it should not be filtered, aggregated or cleaned.
- Separate operational from analytical systems, data is immutable and operational systems should not communicated through the lake
- Separate competences, data scientist work on the lake and specialized applications are built on the lake shore
- Security (confidentially) of the data in the lake is an open problem
(4) Microservices by Martin Fowler
- Small services can be implemented using different languages and can be independently deployed
- It is easier to enforce visibility restrictions because another microservice can only access the microservice published interface. Most programming languages have flaws in terms of restriction of visibility, the modularity constructs do not allow several levels of encapsulation
- A development team can be in charge of a microservice, which assigns the full lifecycle of the service to a single team and reduces communication between teams, cross-functional teams
- Interesting the distinction from event bus based systems
- The use transactionless coordination between services require the explicit management of consistency by developers
- Due to the increase of the types of failures it is necessary to have sophisticated monitoring
(5) Refactoring Module Dependencies by Martin Fowler
- A set of mechanisms that allow defer binding of modules, namely Service Locator and Dependency Injection
(6) Feature Toggles by Martin Fowler
- On the modifiability quality and the defer binding tactic. It discusses the different “business” contexts where this tactic can be applied when dealing with the support of variations of a feature.
- Discusses some implementations of defer binding by applying techniques like inversion of control, strategies, and factories.
(7) Serverless Architectures by Mike Roberts, and Serverless by Badri Janakiraman
- An architecture which supports stateless, short duration, services
- An alternative platform for microservices which, instead of platform as a service (PaaS), provides a completely transparent support of scalability and elasticity, and it also differs from Software as a Service (SaaS) being referred as Function as a Service (FaaS), see previous bullet
- Development and Unit testing become harder
Other systems
- Elements Of Scale: Composing And Scaling Data Platforms by Ben Stopford