The first law of thermodynamics states something like this: the total energy of the system plus the surroundings is constant. Or basically energy is conserved; it cannot be created or destroyed, just moved around. This first law is an important realization for chemists/physicists, and underpins the way they look at problems.
I think there is a similar realization that can be made regarding the complexity of software systems, and that it can change the way you look at and design software. It would go something like this:
The underlying complexity of a given problem is constant. It can be hidden, but it does not go away.
Or:
Complexity is conserved by abstractions. In fact, apparent complexity can be increased by abstractions, but the underlying complexity can never be reduced.
Complex problems remain complex, now matter how much good design and architecture you heap on top. Abstractions hide the complexity. But this is a good thing! A good abstraction divides a problem in two, putting part of the complexity on either side. A good abstraction puts most of the complexity where it can be solved by an existing solution to that subproblem.
But complex problems remain complex, and abstractions leak. If you come up with a good design that lets you hide the complexity of a problem, and then you forget about that complexity, rest assured that it will eventually come back to bite you. Hide complexity, but don’t ignore it.
You may have easily implemented a huge, performant web application
because you used off-the-shelf components for presentation,
persistence, messaging, clustering and fail-over. You applied well
known abstractions like J2EE, RDBMSes, Web Services, MVC… the list
goes on. But abstractions leak. Clustering and fail-over is inherently
hard no matter what the vendors tell you, and the complexity often leaks
through under load. HTTP and TCP do a great job of getting bytes from
one place to another, until a bug feature
in the OS or router shows itself
(like MTU size or sockets closing on DHCP lease re-acquisition).
Use abstractions to move complexity to where it is more easily dealt with. But don’t forget that it is still there.
Mathematicians know a great trick that can be used to sidestep my
first law of complexity: redefine the problem. The underlying
complexity of a given problem is constant, so change the
problem. This can be the single most useful outcome of a
requirements gathering exercise. Someone may come up and ask
you to solve this problem: “we need an enterprise-bus messaging
system”. But if you dig a little bit, you may find out they would be
happy with occasional automated emails, a .forward script
and a CGI.
Use abstractions to move complexity to where it is more easily dealt with. But remember complexity is conserved by abstractions; and abstractions leak.
