Holistic Engineering: Organic Problem Solving for Complex Evolving Systems

Write for InfoQ

Feed your curiosity. Help 550k+ global
senior developers
each month stay ahead.Get in touch

Like

Late projects. Architectures that drift from their original design. Code that mysteriously evolves into something nobody planned. These persistent problems in software development often stem not from technical failures, but from forces we pretend don't exist, reward systems that incentivize the wrong behaviors, organizational structures that ignore domain boundaries, and human dynamics we consider 'too petty' to factor into our technical decisions.

Holistic engineering is the practice of deliberately factoring these non-technical forces into our technical decisions, designs, and strategies. Just as geological forces leave visible layers in canyon walls, organizational and human forces leave their marks in our codebases. The difference is that we can see the canyon's layers clearly, while the forces shaping our software remain invisible until we learn to look for them.

Image source: Credit to Tomáš Malík on Pexels

Throughout my career, as a software engineer, architect, tech lead, engineering manager, principal engineer, and consultant, I've observed the same patterns recurring across different companies, domains, and tech stacks. These patterns reveal how non-technical forces consistently shape our technical systems, whether we acknowledge them or not.

Phenomena in the Wild

Late projects are a persistent problem in our business. It's very challenging to achieve perfect alignment with deadlines. Another persistent issue occurs when, at the end of the project, after so much strategy, so much design, the final result differs completely from the original design, not for good reasons. Not because we're evolving the architecture, but more because some funding was cut, some other problem was there, or there was such a misunderstanding that, at some point, teams must work with existing constraints.

These patterns emerge because technologists operate within isolated perspectives. Like many other disciplines, technologists view our problem space from our own perspective, considering our responsibilities, and we rarely have the opportunity to examine the whole picture as thoroughly as we could.

A 'phenomenon' represents an event or situation that everybody recognizes exists, but people might have different opinions of why that happens. These patterns are referred to as 'phenomena in the wild’, patterns that emerge repeatedly. The same applies to natural forces, which generally are forces that influence the path in one way or another, and teams must find ways to either accept, mitigate, or solve them.

The Shared Kitchen Sink Phenomenon

There's an idiom that says "everything but the kitchen sink", which means you've stuffed all sorts of things inside your bag or inside your suitcase, containing everything except the kitchen sink. This phenomenon appears in almost every organization in the form of a utilities library. Organizations typically name these libraries 'common' or use the company name. It’s a library that is used by all the services of your company, where all your developers from all the different teams are adding small features with the intent to help each other.

This approach creates several significant issues. The first issue is that if you have a library that everybody contributes to, if there is an issue in that library, all the services that are consuming it might be part of the biggest imaginable blast radius if something goes wrong. Another issue is that if multiple people from different teams constantly add small features, you end up with a very big library, and at some point, developers lose visibility into the library's contents.

The bigger it becomes, and the more people collaborate on it, the more it becomes increasingly difficult for people to refactor it and actually make it more efficient and more flexible. This inevitably increases complexity, which increases bug rates, and then, generally, whenever there's complexity, it's not going to be well-tested.

Two main forces contribute to this phenomenon: the people management process and the accompanying reward system. Additionally, poor planning or delivery pressure are contributing factors.

Career frameworks often include sensible-sounding criteria that create perverse technical incentives. HR departments and senior leadership frequently design evaluation systems that measure engineering seniority by "area of impact", where company-wide influence indicates higher maturity than team-level contributions. While conceptually sound, this influence drives engineers to game the system by adding functionality to shared libraries and services, allowing them to claim broad organizational impact in performance reviews rather than focusing on appropriate technical solutions.

Another driving force is poor planning due to delivery pressure. Common phrases such as 'We need this by tomorrow' or 'We need this next week' create delivery pressure. Then developers in the team will have no choice but to use the delivery mechanism they already have, which is this library that's already there.

These represent two forces that have actually shaped the presence of a library. The decisions behind it are not technical whatsoever. Different departments make the decisions for different reasons, but they have a very direct influence on your code.

The Domain Identity Crisis Phenomenon

Organizations often create shared domain libraries containing classes like Customer or Order that attempt to represent core business entities for all teams across the company. For example, a single Customer class might need to satisfy the marketing team (who needs demographic data and campaign preferences), the billing team (who needs payment history and credit scores), the support team (who needs ticket history and communication preferences), and the shipping team (who needs addresses and delivery preferences). Rather than recognizing these as different bounded contexts requiring separate representations, teams stuff all these concerns into one massive Customer class.

This approach violates domain-driven design principles by assuming the entire company shares a single domain model. The resulting bloated classes become impossible to maintain and modify. Teams begin referring to the same concepts using different terminology, revealing their fundamentally different perspectives on what should be separate domain representations. When human understanding fragments this way, the software inevitably fails to correctly model the business reality it's supposed to represent.

This phenomenon emerges when the product organization lacks the maturity to fully understand and communicate domain boundaries. Without proper domain mapping and communication skills from the product team, engineering departments cannot accurately represent business reality in code.

Teams lacking dedicated business analysts face additional challenges. Business analysts bridge requirements and development teams, ensuring requirements don't overlap and all details are captured. When teams lack this role, developers assume these responsibilities as secondary duties, inevitably producing inferior domain modeling due to time constraints and divided attention.

The strongest contributing factor is organizational design that ignores domain boundaries when forming teams. Organizations might group services by technology rather than domain, assigning two Java services to the same team while other teams use different languages. While this approach optimizes resource logistics, it forces teams to view disparate domains as unified, leading them to model separate business areas as a single domain in their code.

The Cirque du Soleil Coding Phenomenon

Cirque du Soleil is renowned for skilled acrobatic performances that showcase technical mastery. Similarly, some codebases exhibit what can be called "Cirque du Soleil coding", implementations where simple features incorporate every available design pattern, library feature, and external dependency. These solutions prioritize demonstrating technical range over identifying the simplest effective approach.

This over-engineering pattern creates significant maintenance overhead. Complex solutions cost more to debug, modify, and test. The resulting code becomes fragile and difficult for teams to understand, leading to increased bug rates and reduced development velocity. A "culture of coolness" emerges where developers equate technical sophistication with professional competence, driving engineers to use every available tool rather than selecting appropriate solutions.

Career frameworks compound this problem by including criteria like "practitioner is highly skilled in the company's tech stack". When organizations lack the technical depth to properly evaluate engineering work, developers respond by creating elaborate technical demonstrations in their pull requests. These demonstrations allow developers to showcase the breadth of knowledge in concentrated examples rather than distributed across multiple simpler implementations.

Influential technical leaders inadvertently reinforce this pattern. When senior engineers or architects promote specific libraries or approaches, teams interpret these recommendations as mandates. Casual endorsements become organizational standards applied universally rather than contextually.

Toxic organizational cultures exacerbate these dynamics. In environments where developers feel pressure to constantly prove their competence, over-engineering becomes a defensive mechanism. Engineers create unnecessarily complex solutions to demonstrate their technical capabilities and secure their position within the team.

Making the Unconscious Conscious

Engineers often avoid acknowledging organizational problems and dynamics that fall outside their direct technical responsibilities. Like the proverbial three wise monkeys – see no evil, hear no evil, speak no evil – technical teams close their eyes and ears to dysfunctional patterns, refusing to discuss forces that inevitably shape their work.

Carl Jung observed: "When an inner situation is not made conscious, it happens outside as fate". Engineers frequently recognize human dynamics and organizational problems within their companies, yet when projects run late or drift from their intended direction, they attribute these outcomes to unforeseeable circumstances rather than predictable organizational forces.

Gloria Steinem noted: "The truth will set you free, but first it will piss you off". Technical teams often possess awareness of dysfunctional dynamics, but dismiss them as too petty or unprofessional to factor into technical decisions. They consider it beneath their professional standards to account for stakeholder conflicts, misaligned career frameworks, or toxic team dynamics. This deliberate blindness to non-technical forces doesn't eliminate their influence; it simply ensures that teams remain unprepared for the inevitable consequences that manifest directly in their code architecture and project outcomes.

Holistic Engineering

The solution requires examining the problem holistically. The term holistic describes a construct whose parts represent the entire organism and cannot be properly understood without examining the complete system. Holistic medicine can serve as an example. When people are treated in holistic medicine, practitioners examine not only illnesses and symptoms, but also overall well-being. Treatment considers the complete context of illness, including environmental and lifestyle factors.

Holistic engineering involves considering, during technical design, among the factors, not only traditional technical factors, but also all the other non-technical forces that will be influencing your system anyhow. By acknowledging these forces, teams can view the problem as an organic system and influence, to some extent, various parts of the system.

How your project sits in the intersection of multiple systems

External Forces

• World events
• Business changes and trends
• Technology trends

External forces are factors beyond organizational control that influence technical decisions and project outcomes. These forces operate outside the company, but directly impact system design, architecture choices, and project timelines.

External factors that influence your technical decisions( in green)

Teams building e-commerce systems typically consider the ability to support holiday periods. Few teams consider how elections or regulatory changes will influence the future of their projects, especially within the timeframe of bigger strategic projects.

Business trends present similar challenges. Without understanding where the product sits within the broader business context, teams struggle to identify which components will change and which will remain stable. Understanding what could completely disrupt the market or destroy the product becomes essential for prioritizing technical decisions.

Technical trends represent another category of external forces, though teams more commonly acknowledge these when making architectural decisions.

Internal Forces

• Organization
• Technology
• Product
• People

Internal forces are organizational factors within the company that influence technical decisions and project outcomes. Unlike external forces, teams can potentially influence, mitigate, or work around internal forces, though these forces often feel immutable due to their institutional nature.

Internal factors that influence your technical decisions( in green)

The triad of people, product, and engineering operates within a pre-existing organizational structure. This organizational foundation was established before current team members joined and will likely persist after they leave. Teams must recognize that their technical solutions are built upon and constrained by this organizational substrate.

Consider the actual information structure within your organization. Understanding actual workflow patterns and communication channels reveals how work truly gets accomplished. These communication patterns often differ significantly from the formal hierarchy. Next, identify which processes could block your progress. For example, some organizations require approval from twenty people, including the CTO, to decide on a release.

Reward systems, career frameworks, and bonus structures represent powerful internal forces that influence team behavior at critical project moments. These mechanisms shape how individuals prioritize their work, often in ways that conflict with optimal technical decisions but align with personal advancement goals.

Product

Teams often lack time to examine product requirements beyond their immediate project scope. Without understanding the full product strategy and mission of the company, teams cannot make informed decisions about what will remain stable, where to invest effort, and what will likely change. This disconnect between local project requirements and broader product direction significantly influences technical decisions. Marketing strategies can reveal upcoming changes that help teams prevent architectural issues.

People

Analyzing team members requires looking beyond technical skills and role counts. Key characteristics of project participants, their career motivations, personal investment in the project outcome, and individual circumstances directly impact project success. People may be assigned to projects outside their career plans, affecting their engagement and performance. Personal life changes, such as relationship changes, relocations, family additions, or job searches, create availability and focus constraints that require specific mitigation strategies.

Engineering

Technical decisions must align with the company's technical strategy and capabilities. Teams that design solutions exceeding their organization's operational maturity, in deployment frequency, rollback capabilities, or architectural complexity, create implementations that their companies cannot sustain. Path to production quality, observability systems, and operational maturity directly constrain which architectures teams can successfully deploy. Security and privacy requirements, when addressed late in projects, frequently become blockers requiring additional funding and causing delays.

Putting Holistic Engineering into Practice

Identify Organizational Forces

Every organization contains dysfunctional patterns that influence technical decisions. Systematically identifying these forces allows teams to predict where projects will encounter resistance, delays, or unexpected changes. This analysis reveals which technical approaches will succeed within existing constraints and which require organizational changes to implement effectively.

Make Implicit Knowledge Explicit

Transform informal observations about organizational dysfunction into concrete documentation. Model communication patterns, map actual decision-making processes, and visualize how work flows through the organization. These artifacts convert informal observations into actionable intelligence that teams can use for project planning. Documentation makes previously invisible forces measurable and addressable.

Socialize Information Organizationally

Share force analysis beyond immediate stakeholders to create organization-wide awareness. When dysfunction becomes an acknowledged fact rather than private knowledge, it enables coordinated improvement efforts. Public recognition that "our deployment process creates bottlenecks" transforms individual frustration into an organizational mandate for change. This approach builds coalitions for addressing root causes rather than working around symptoms.

Design Two Architectures

Create both an ideal "North Star" architecture and a pragmatic current-state solution. The North Star represents optimal design without organizational constraints, while the pragmatic design works within existing limitations. This dual approach provides concrete evidence of the gap between potential and reality, enabling informed discussions about whether to address organizational constraints or accept technical compromises.

Define Evolution Pathways

Document specific milestones between the current and ideal architectures. These transition plans demonstrate that pragmatic initial solutions can evolve toward better designs as organizational maturity improves. Clear evolution paths prevent teams from permanently settling for suboptimal architectures while providing realistic timelines for improvement. This approach transforms "we wish we could build X, but we're stuck with Y" into "we're building Y now and here's how we reach X".

Conclusion

Organizations that embrace holistic engineering gain predictable control over forces that typically derail technical projects. Instead of reacting to "unforeseen" delays and architectural drift, teams can anticipate and plan for organizational constraints that inevitably influence technical outcomes.

Holistic engineering transforms engineering teams from passive victims of organizational dysfunction into active partners in systematic improvement. When teams make organizational forces explicit and address both technical and human factors simultaneously, they create sustainable solutions that thrive within organizational constraints while gradually improving those constraints over time.

The alternative is predictable: continued cycles of late projects, architectural drift, and technical solutions that work on paper, but fail in organizational reality.

Related Topics:

• Development
• Architecture & Design
• QCon London 2025
• Platform Engineering
• Agile
• QCon Software Development Conference
• DevOps
• Onboarding New People
• Software Engineering
• Architecture

Enter your e-mail address

Select your country Select a country I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.