Architecture and Design 1:
– Intro to Design Principles – Technical Debt
There are many well-known design principles for software. Here are a few at several levels:
Make good abstractions Separate concerns Encapsulate well
Hide information
Don’t leak information
Increase cohesion
Reduce coupling
No redundancy (DRY)
No needless complexity (YAGNI)
Respect the Law of Demeter
Decompose into single responsibilities
Make open for extension, closed for modification
Conform to Liskov substitutability Segregate interfaces
Invert high-to-low level dependencies
Old? Mostly, but that’s because they are the survivors.
Most derive from the same influential idea: modularity
Most help us deal with our two big enemies
COMP3297: Software Engineering
2
Fighting the enemy
We saw we have two big enemies:
Complexity Change
Our software process helps us organize our projects to deal with those enemies.
But we also need to tackle them in actual code. There, in the software, they are related:
Complex software is hard to change.
We need design techniques that minimize complexity and support change.
COMP3297: Software Engineering 3
Complexity
Recognizing complexity when you see it is a key design skill
You need to know it to fight it.
Helps you determine whether a design is good or bad. Helps you decide between design alternatives
4
A complex design is one with so many components, and so many interactions among those components that it is:
Practically impossible to understand the software. Very difficult to modify the software safely.
This works as a useful, practical definition:
Complexity is anything related to the structure of a software system that makes it hard to understand and modify the system.
COMP3297: Software Engineering
Ousterhout 2018
Signs
Small changes become a big tasks
What should just be a simple change requires you to modify the code in many places.
Too much to remember when coding
Takes more time to learn how to work with the software and there’s more chance of missing some step that was needed.
Not clear how to make a change
Problem here is that something might be missed because there was no way to find out it actually needed to be modified as part of a change. You don’t find out until tests fail or there are bug reports from users.
You don’t know what you don’t know without reading every line of code.
COMP3297: Software Engineering 5
Major cause
Dependencies
Can’t avoid dependencies. Code depends on other code to perform its responsibilities – functions call other functions, methods call other methods and methods of other classes.
But don’t create dependencies unless they are necessary.
And when you do create them, keep them simple and make them obvious. What’s worse than a dependency? A dependency you didn’t even know existed.
Most of the design principles listed earlier help developers avoid creating problematic dependencies.
COMP3297: Software Engineering 6
But why?
Programmers are often coding fast against a deadline. They take a short-term approach and focus only on getting the code working.
“I’ll add this little dependency to get it working quickly. It won’t do much harm”
It will
“I’ll fix it later”
You won’t
“We’ll refactor it when we need to work with the code again”
Grows until it’s too big a task. You’ll just patch it up
COMP3297: Software Engineering 7
Sometimes it is the “heroes” who do this!
Technical Debt
Working code isn’t enough. Tactical vs. Strategic Programming Primary goal should be great design (which also works)
“Working” is not a high enough standard.
If you compromise to get things done fast, think of it as going into debt.
Pay it back soon, or the interest will mount up quickly
8
What’s the “interest”?
The extra effort it takes to make the next changes
Until fixed, cost per change continues to rise
COMP3297: Software Engineering
Some other kinds of Debt
Design that no longer works for the current product
Defects we haven’t fixed
Poor test coverage – we know we should improve it but we don’t Too much reliance on manual testing
They all slow us down over the long term
COMP3297: Software Engineering 9
Where it comes from
Naive debt
Poor technical practices and/or training
Unavoidable debt
The world changes. Dependencies change.
Never have perfect knowledge up-front of how system will evolve.
May need to revisit design decisions made earlier.
Strategic debt: when being tactical becomes the strategy
Accept debt to achieve short-term goal. Common in startups.
Strong Definition of Done helps stop debt escaping the sprint.
COMP3297: Software Engineering 10
Debt you don’t need to repay
Product at end of its life – maybe because of too much debt.
Retire it or re-engineer it.
Throwaway prototypes
Sometimes they really are short-term solutions
COMP3297: Software Engineering 11
When you do repay Repay:
High interest debt first
Where the code needs to be changed often
And not where it isn’t changing and unlikely to fail
When we touch existing code
COMP3297: Software Engineering 12
Change: Evaluating Maintainability:
ISO Quality Model Maintainability Sub-characteristics
Modularity:
To what extent is the program composed of discrete components such that a change to one component has minimal impact on others?
Reusability:
How well can an asset be used in more than one system, or in building other assets?
Analysability:
How easy is it to find deficiencies or the cause of failure if the software does not function correctly? How easy is it to work out what needs to be modified when we want to change or add functionality?
Modifiability:
How easy is it to actually make modifications without introducing defects or degrading performance?
Testability:
How easy is it to establish test criteria and to perform tests to determine whether those criteria
have been met?
COMP3297: Software Engineering 13
Change: Evaluating Maintainability:
ISO Quality Model Maintainability Sub-characteristics
Modularity:
To what extent is the program composed of discrete components such that a change to one component has minimal impact on others?
Reusability:
How well can an asset be used in more than one system, or in building other assets?
Analysability:
How easy is it to find deficiencies or the cause of failure if the software does not function correctly? How easy is it to work out what needs to be modified when we want to change or add functionality?
Modifiability:
How easy is it to actually make modifications without introducing defects or degrading performance?
Testability:
How easy is it to establish test criteria and to perform tests to determine whether those criteria
have been met? Examples? COMP3297: Software Engineering
14
The key to good design
Modularity
What is a module?
Many definitions at different scales. Simplest: a logical grouping of related code.
General context: some overall piece of functionality is provided by a collection of independent, self-contained, reusable modules, where each module provides one part of the functionality. That is, each module provides a well-defined service.
A service is a coherent unit of functionality – again, can exist at different scales. An email service may include other, more specific services for creating mail, sending mail, etc.
Ideally, modules would be completely independent. Then can a developer can work on any module without knowing anything about the others.
But already saw: there will always be dependencies. A goal of modular design is to minimize the dependencies.
COMP3297: Software Engineering
15
16
Interface and Implementation
Modules use each others’ services, and access them through the module’s API. Good to think of a module in terms of an:
Reveals what the module does.
Everything that needs to be known to use the module.
and an:
interface implementation
How the module provides its service. No knowledge of this is needed to use the module.
If you’re maintaining this module, this should be the only implementation you need to understand.
Two main indicators for doing it right: coupling and cohesion COMP3297: Software Engineering
The Big Two: 1. Increase cohesion
Concerns how closely the operations within a module are related to one
another.
Guideline: aim for high cohesion by grouping strongly related functionality
together while also keeping everything else out.
Relevant to design at any level, from individual functions up to entire
subsystems.
Methods/functions with the strongest cohesion have a clear single purpose.
They execute that single purpose extremely well and do nothing else.
Modules with low cohesion do the opposite: they group many unrelated responsibilities together and, as a result, are difficult to understand and bring a lot of junk along when we want to reuse some part.
Modules with low cohesion change for many different and unrelated reasons.
COMP3297: Software Engineering 17
The Big Two : 2. Reduce coupling
Goal is to reduce the number and strength of dependencies – the connections to other
modules, knowledge of other modules, or reliance on other modules.
High coupling among modules has a negative effect on all characteristics of maintainability:
Reusability
Analyzability
Modularity and Modifiability
Testability
More difficult to reuse a module in isolation
More difficult to understand a module in isolation
Modifying a module takes effort and care to ensure corresponding modifications are made to dependent modules
Less likely to make all the needed modifications
More difficult to test a module in isolation
COMP3297: Software Engineering 18
The interface consists of two kinds of information:
formal
and: informal.
Make the informal parts clear – they are usually larger and more complex than the formal parts.
For a method, this is its signature.
For a class, it is the collection of signatures of all its public methods, plus names and types of public variables
At higher levels, the API.
“Formal” because it is specified explicitly in the code
Behaviour, including side-effects. The contract. Plus constraints on how the module must be used (say methods must be invoked in a certain order).
“Informal” because it isn’t specified in a way that can be enforced by most languages.
The interface
COMP3297: Software Engineering 19
Deep modules
Good modules are deep. They provide their functionality through a comparatively simple interface, an abstraction – a simplified view – of the functionality.
From the point of view of managing complexity and change:
Simple interfaces minimize the complexity a module introduces into a system.
If the interface is much simpler than the implementation, then there is a lot of opportunity for change without affecting dependent modules.
COMP3297: Software Engineering 20
Other General Principles
Make Good Abstraction and Separate Concerns
Key to good decomposition into modules. How to achieve high cohesion:
Group similar concerns together and separate them from other groupings
Effect:
majority of interactions will occur within modules
interactions between modules can be kept simple. So: low coupling.
Result:
each module tends to capture a single abstraction or feature of a problem and can export the corresponding services through a simple interface .
complexity is managed. Can deal with modules in relative isolation.
Can understand a module and its interactions with other modules rather than
needing to understand every internal aspect of the software.
COMP3297: Software Engineering 21
Good Abstraction and Separation of Concerns
Important at all levels of design. Here are some familiar separations:
Separation of data and its presentation as in Model View Controller and similar patterns.
Django projects as collections of many tightly-focused apps.
Site or large scale service as a collection of small independently-deployable
service components as in Microservice architectures.
Architectural layering separating Presentation, Application Services, Domain,
and Infrastructure/Persistence concerns.
Class design based on the Domain Model such that software classes abstract real-world concepts. The Order class from last week’s exercise groups together things to do with real-world orders.
COMP3297: Software Engineering 22
Encapsulation and Information Hiding
Keep data and the operations that manage that data bound together as modules.
Prefer modules that export only a minimum amount of detail through their interfaces: as we saw, deep modules.
Or, if there is an indication that you shouldn’t depend on a particular detail (as in Python), then don’t. Don’t couple to it.
How much should you reveal in an interface? A module should reveal only the information that a client absolutely must know in order to use the module’s services.
In particular, the way the services are actually provided – for example, the data structures used within the module – should be hidden from client objects.
Example from the Process Sale domain model: Clients shouldn’t care how a Sale knows its total and certainly will not depend on those details
COMP3297: Software Engineering 23
Encapsulation and Information Hiding
Emphasizes the important separation between interface and implementation we saw earlier Hence:
Program to an interface, not to an implementation
Clients depend on the specification or contract of the module and not how the module fulfills that contract.
Results in another important feature of interactions in OO and dynamic languages: Clients don’t need to know the specific types of objects they use. They only know about the interfaces. This greatly reduces implementation dependencies and makes systems much more modifiable.
Example: in Django any callable that accepts an HttpRequest object and returns an HttpResponse object can serve as a View.
COMP3297: Software Engineering 24
No redundancy (DRY)
Every piece of knowledge must have a single, unambiguous, authoritative representation within a system
COMP3297: 25
:What’s wrong with duplication?
To make a change, must find where that particular piece of knowledge is
embedded in the project.
If it is embedded in only one place we can be certain we have found every occurrence.
If it has been duplicated in many places, we must be sure we change all of them.
If we don’t, then we’ll introduce a contradiction into the project.
Django’s design emphasizes DRY everywhere:
E.g. Models => Schema, Models => ModelForms
Software Engineering
How about your own Django designs?
You can appreciate these principles in the design of Django itself, but you didn’t need them to design a good application.
Why? Because if you develop the way Django wants, you end up with good modules, good separation of concerns and loose coupling. Following Django’s design philosophy:
“Models know nothing about data display”.
“Views don’t care which template system is used”. “Template system knows nothing about web requests”.
But some approaches that work for small projects can begin to violate principles of good design if they continue to be used as those projects increase in size.
COMP3297: Software Engineering 26
Fixing Violations of the Principles:
Fixing Fat Apps
27
Don’t try to write the site as a single app with wide focus.
Best approach for app design? The UNIX way: Encapsulate a single
task. Do that one thing and do it well.
For example, a project might feature Tagging or be required to support Reviewing. These are likely to be useful in other projects – separate them into apps that can be reused. Or, better, use existing third-party apps because others have already done exactly that.
Signs your apps may be unfocused:
You can’t explain the app in a short sentence.
You can change some feature and it doesn’t affect how other features of the app work.
Your complex project settings don’t have many apps in INSTALLED_APPS.
Apps have too many models.
COMP3297: Software Engineering
Fixing Fat Views
Common approach in small projects is to put most business logic into View functions and keep the Models as a data access layer.
Can work when there is not much business logic – for example, only CRUD functionality. There, the application acts like a simple interface to data.
But as the logic gets more complicated, the View functions/methods can become very long and complex.
Often find you repeat the same logic in multiple views, violating DRY.
Views do too much, cohesion gets worse and code becomes difficult to maintain, debug, test and reuse.
COMP3297: Software Engineering 28
Fixing Fat Views
29
What is the role of a View?
Better if they don’t contain domain logic. If views make heavy use of models’ built-in query methods, they become strongly coupled to the details of the models.
Models should contain data and all relevant logic.
In the design for getting the total of an Order, the logic doesn’t belong in the
view. It belongs in a get_total() method in the Order model. Push logic into Model Manager and Model methods :
Effectively provides an API over the model layer that provides services that views, etc, need.
Reduces duplication.
Easier to find common logic than if it was just separated out into a helper functions.
Easier to test.
Better self-documentation.
COMP3297: Software Engineering
Fixing Fat Models
The Thin View, Fat Model approach to design is better, but there is danger that models get too fat.
Keep methods, classmethods and properties, as part of the model interface, but just as wrappers for utility/helper functions that contain the actual logic. (You’ll find them in xxx/utils.py or xxx/helpers.py in large projects). Now very easy to test.
Make fat models thinner by factoring out behaviours, particularly if they are reusable, into mixins.
Or take them all the way back to their original responsibilities by adding a service layer.
COMP3297: Software Engineering 30