What a systematic literature review reveals about how EA is assessed — and what is missing
A peer-reviewed paper co-authored with Martin Henkel and Erik Perjons of Stockholm University — "Improving EA Evaluations Through the Work-Oriented Approach" — has been published in Springer Nature's Communications in Computer and Information Science (CCIS) series.
The central finding is blunt: enterprise architecture research predominantly evaluates EA through the eyes of those who produce it. The stakeholders whose work EA is supposed to improve are largely absent from the evaluations that claim to assess whether it does.
What the systematic review found
The paper presents a systematic literature review of 35 empirical research articles on EA evaluation. Four questions structured the review: which stakeholders contribute, how practices and their work are represented, how relationships between practices are characterised, and how accommodations — the mechanisms by which EA delivers value — are analysed. The answers are consistent across the literature.
Stakeholder voices are not equal. EA experts and IT personnel with prior EA knowledge dominate the evaluations. Business leaders, process owners, and end users of architectural work — the people EA is intended to serve — are significantly underrepresented. In several studies, EA experts explicitly spoke on behalf of stakeholder groups not present in the data. Evaluations that rely on one group to represent the perspectives of others have a validity problem, regardless of how well they are otherwise designed. The same structural question is worth asking of the commercial analyst: whose practices and voices are represented in the research that Gartner, Forrester, and their peers publish behind paywalls?
Practices are not represented. The work that stakeholders actually do — their day-to-day use of architectural information products — is rarely characterised with any precision. Most evaluations rely on factor-oriented approaches — identifying, linking, and assessing factors related to EA, information systems, or organisational structures — without grounding those factors in the specific work contexts where they are supposed to manifest. A representative type of evaluative statement from the literature asserts that EA aligns business strategies with IT resources to create a competitive advantage. What it does not specify is who does what, with which architectural output, in which practice, and through what mechanism. That level of specificity is almost never made explicit.
Relationships between practices are absent. EA does not operate in isolation. It is produced in one practice and consumed in others. It generates value — or fails to — through those relationships. Yet the reviewed literature rarely characterises the producer-consumer relationships that define how architectural work circulates and how value is transferred between practices. EA evaluations largely treat EA as a self-contained object rather than as something that lives in use.
The mechanisms of value delivery are opaque. Very few articles examine how EA generates organisational benefit — the actual causal connection between an architectural output and a change in how work is done. The field frequently measures whether EA "delivers value" without being explicit about what the delivery mechanism is.
What the Work-Oriented Approach contributes
The Work-Oriented Approach (WOA) provides a set of constructs — practices, relationships, practice roles, accommodation, and alternatives — designed precisely to make these things explicit. WOA treats architectural information (products) as participants in stakeholders' work practices rather than as objects with inherent properties. Value is not a property of an architectural model; it is a relationship between that model, the work it participates in, and the stakeholders doing that work.
The paper proposes that future EA evaluations adopt WOA constructs to include underrepresented stakeholder voices at the right level of specificity, characterise practices clearly enough to anchor evaluation factors in them, represent producer-consumer relationships between practices, and examine the actual mechanisms by which architectural outputs deliver value — rather than asserting that they do.
WOA does not replace existing evaluation frameworks. It grounds them where they currently float.
What this means for the field
If EA evaluations consistently overrepresent the producers of EA and underrepresent its consumers, the evidence base for EA's value is structurally biased. This is not a criticism of individual researchers — it reflects how the field has developed and who has been most willing to participate in studies. But the consequence is real: the accumulated research on EA effectiveness is less reliable than it appears, because it has been heard primarily from one side of the producer-consumer relationship.
Closing this gap requires more than adding stakeholder surveys to existing studies. It requires redesigning how evaluations are structured — starting with a clear picture of whose work the evaluation is actually about, the practices those stakeholders engage in, and the mechanisms that are supposed to connect architectural outputs to their work.
The paper is available on Springer Nature:
Read the paper → Improving EA Evaluations Through the Work-Oriented Approach
DOI: 10.1007/978-3-032-19050-5_6
If you evaluate EA in your organisation — or if you are evaluated by it — whose work is the evaluation actually grounded in?