How many businesses still use spreadsheets as part of their business operations? The reality is that they are used even at the highest levels in multinational companies. Is this some kind of accident? Did they not receive the memo about the security risks they present, their limitations, their complexity?
Spreadsheets still exist because they put the power to adapt and evolve in the hands of those who need it most. Spreadsheet users are not banging down the door of IT every week requesting that new features be added to Excel so they can get their job done. They have the power to do what they need to do no matter what domain they work in.
But the professional information technology people know better, or so they believe. They want to write software that is designed from the ground up to do a specific narrowly defined job. It begins with the foundation stones, the data structures and schema on top of which is built the object models and object relational mappings into which is stuffed the domain specific business logic.
A more recent development has been Microservices where a system is broken down into separate modules with specific mini applications with a well defined subset of functionality. This is a similar concept to encapsulation with classes, that every class has a well defined contract, and that only the contract is visible to the outside. Furthermore these modules are related to domain specific aspects of the system.
Modularity and clear separation of concerns is a critical part of complex systems which allows you to break applications into sections with well defined responsibilities. But should we design our module architecture around a specific domain? The answer is an emphatic no!
Building in Inflexibility
The first reason design driven development is a bad idea is that it encourages you to build business logic into the application. Imagine if database software was domain specific, where the database language was specific to a single industry. Imagine there were different databases for each industry, rather than a standard which allowed all kinds of different systems to be built on top of them. Yes, database servers are an example of good architectural design, where their purpose of storing data is not confused with the domain the database will be used for.
Building in domain level business logic into your software can only result in the software being more brittle and less adaptable to change. Think of the spreadsheet where a user can modify how it works in real time, add a new field or add a new graph. Can your users add new fields? Can they create new reports and queries? To add new features do you need to rebuild and deploy the software? Spreadsheets never require a rebuild or redeploy. Databases do not need to be rebuilt and redeployed simply because you added a column. But business software developers have been inculcated with the fixed schema static domain model. But it does not need to be that way.
Domain Derived Modules
With the advent of MicroServices it appears that we are making the same mess as the early java EJB’s, in that it is trying to scale by breaking services up based on their front end contracts. Let us say you have clients, suppliers and partners. Each of these entities has different fields and business rules. The classic approach would be at the very least to have separate tables for each of these in the database, to have separate classes for each, which would be mapped from to the tables, and for web services to be exposed for each individual type.
The MicroService approach takes this a step further, breaking the system into separate concerns. Perhaps all these entities will end up in separate services, running on different containers that talk to their own databases. The predictable result is a nightmare trying to do things like create cumulative data sets across multiple services. One justification for taking this approach is scalability, but what actually occurs is a maze of services calling services, all the time increasing latency.
What usually happens when people start building real systems and run into this problem is the gurus come in and tell them they have been doing it wrong. They didn’t quite understand the one true path. It seems our guru never really evaluated whether breaking an architecture apart based on the domain was a clever idea in the first place, or indeed whether using the domain as the foundation at all was a good plan.
Root of the Disease
Introductory books on Object Oriented Programming often use real world objects as examples. It encourages developers to model the real world in terms of code. I know they probably meant well, but what this did was inculcate modern developers with the idea that the business domain belongs in the binary.
While databases themselves are cleverly universal they did at the same time expect a static schema to be defined, which in reality was just as inflexible as any binary. The ongoing mess with trying to keep data structures up to date as the software is updated is a common experience for developers. It wasn’t exactly their fault; after all the popular dBase language was rooted in binding the database and the code together.
Ok, clever clogs, what’s the answer?
It is all very well waxing lyrical about the failure of modern software architectural approaches without presenting some kind of working example of how else it could be done. It would also be dishonest to simply expect people to believe me on a hunch. Bottom line is that I have implemented a successful solution based on my architectural principles.
In 2006 I worked on the a healthcare data integration application. That sounds pretty domain specific, but the reality was that health specific functionality was a very small part of the system, limited primarily to several connectors specifically for a protocol used in hospitals and healthcare providers. The rest of the system was a universal integration engine that would work for any company in need of integrating different systems.
The application was an excellent example of a technology called OSGi, which separates the application into different concerns. None of the core modules were related to health at all. All were well defined and had a contract or API. These modules were based on the functionality required by the application, but the actual configuration of the routing and transforms was all in dynamic configuration that users could modify in real time with no deployment. Importantly the modules were not in any way influenced by the intended domain. This domain independence would inform my later experiences.
In a subsequent role I wrote a process orchestration system for a Internet Service Provider. Originally I was brought in to complete a project that had been developed in a way that was tightly coupled with the domain. In fact it was tightly coupled to a specific business process. After a period of time it became clear that it was so inflexible that we needed a different solution. So we developed a plan for another system which would address all the shortfalls of the first attempt.
One of the key architectural principles that we adopted for the new project was universality. The domain would be evicted from the application and into configuration. This project would never be a domain specific hack to address an immediate need with no thought for adaptability or flexibility. It took three months to complete the first version of the core functionality, another three to complete plug in components and write the dynamic configuration for the business processes.
Just like with the healthcare integration application we ended up with a universal system. It was able to meet the needs of any business with workflow automation needs. Soon we were developing new business processes faster that we were ever able to before. Then the company was sold and I was able to negotiate an agreement that would see the code base released as open source.
A couple of years ago I introduced a fortune 500 company to this software. Today it is being used successfully in a totally different domain from the original purpose it was developed. Ironically it is replacing spreadsheets because while it has many of the flexible features of spreadsheets it is also secure, central and easy to use. It delivers an excellent user experience thanks to the UI experts there, while also delivering the flexibility and adaptability of spreadsheets.
Complexity is the enemy of Scalability and Availability
Let us do a little thought experiment about how to scale based on two competing models; functional architecture and domain driven architecture. In the functional architecture we have three web servers fronted by a load balancer and a cluster of three database servers. The function of the web servers is to service API calls. The function of the database cluster is to store data in a single logical database replicated across all three servers regardless of API call.
In the domain architecture we have a router which routes API calls to different web servers depending on the required service. Each of these web servers are connected to a database server to store the data for that service.
So let us think about scalability and availability in these two systems. Imagine that there is a head crash on on of the database servers. In functional architecture the other two servers take over and everything trucks along as normal while the damaged server is brought back into service. What happens to the domain driven architecture? Well one of the API will go totally offline. Not only that but you will need to restore the database server from backup, potentially with data loss.
Okay, but surely domain driven architecture is better for scaling? Well lets think about it. With the domain driven architecture all the hits for a specific service will go to the same servers. That means that if you are unfortunate enough to get a rush on one of the API you will get over loaded. But with a functional architecture any web server can service a call and the load is spread across multiple web servers and database servers.
And if you have multiple interdependent services I can’t even begin to imagine the mess you will get into. We need services which are both universal and well defined. The ‘application’ should be a dynamic veneer.
Is your solution just bad old Monolithic Architecture?
Not at all. A monolithic architecture attempts to include all the domain specific rules within itself.
Imagine all the uses that spreadsheets have been put. Do spreadsheets contain all the inherent complexity that they have enabled? No. What I am proposing here is that we decouple our applications from the domain and make the user experience a consequence of runtime configuration in data. This is the same model that spreadsheets used, but we are using this principle in business software over the web. We should not be building in all the business rules and domain entities into our code. The domain still needs to exist, but it can be expelled from code into dynamic runtime configurations. This is a fundamental change of mindset.
Wrapping up the case against Domain Driven Development
Putting business logic and rules into an application sets them in stone. It undermines the power of managers to use the software in ways a developer might not imagine. How many companies have a toxic culture where managers and users are always in conflict with the software developers as any and all changes no matter how small need to be designed, planned and rolled out as part of a deployment. Under such circumstances is it any wonder they have a death grip of their beloved spreadsheet?
This is more than a design principle, it is a fundamental philosophy about the relationship between developers and the user; that we should be empowering users rather than creating barriers. Rather than holding onto the keys to the castle, the domain, we should let it go.