There's an odour in enterprise IT, perhaps multiple. A faint worrying sound of last gasps of Enterprise Architecture. Well, no. Of course, this isn't the case. But let me try to explain why we might at least consider it.
Let me first suggest what I mean by EA. Very obliquely it is the practices, skills, organisational structures, ideas, that are embraced in frameworks like TOGAF, practiced by a significant number of people in this industry, sold by consulting organisations, and set as one of the primary mechanism by which organisations connect IT with the business. In fact, it is probably fair to suggest that as EA has embedded itself over the last 15 years (25 years?), TOGAF has become the badge of choice. This is not to say I'm just getting at TOGAF; rather that TOGAF exemplifies the current state of the "art" of EA, and that it serves as context.
What are the external forces that are driving a re-think?
Pace. Long "EA thinking", OK lets call it planning, cycles are just not going to cut it anymore.
A reduction in Accidental Complexity. A decade ago, the up-front design effort required in all but the simplest enterprise technology was significant. In software engineering the time required to establish appropriate frameworks, procure enterprise products, establish programmes structures was annoying long; and all required before taking on the business domain. Complex business problems with complex technologies evolved complex planning approaches to deal with them, and a reliance on what is now only sarcastically called "enterprise technology". In today's world, we see radical simplification of technologies and processes. The rise of open source, web software frameworks (and perhaps the "web as an architecture"), and agile delivery disciplines has dramatically reduced the level of non-business-related complexity. In IT, lean is everywhere. Our industry is lowering the "cost of failure" to nearly zero.
Lean is everywhere. Whether it be called "efficency", and driven directly from the ideas of the "Lean Startup", there is no escaping both the critique and the practices of lean/agile. This is not a "kool-aid" response; rather experience (ours and others) suggests that lean/agile produces results, with the added side-effect (or perhaps the main benefit) that should programmes are simply more fun to work on.
The lean critique is, perhaps, most damaging to EA. Reduce waste! More specifically, and if you have ever spent any time with "hard-core" agilists you will be familar with these; the YAGNI (you aint going to need it) and its lesser known cousin, "they aint going to read it".
Whether we talk solution/software or enterprise architecture, the number of strategies, whitepapers, architecture documents, reference models, etc, etc, that have been produced and simply not read should be frightening for, if nothing else, our own sanity as architects. A document is just a poor representation of the architect anyway.
But YAGNI! This might be rephrased as this. If the world of technology is moving faster (meaning that the cost/time of doing is higher than the cost/time invested in the EA "paper production machine") than EA is able to plan, we should all be worried. [I am, course limiting this discussion to those organisations who don't believe that planning is an end in itself]
Why Can't the current world of EA respond
There are certain qualities of EA that constrain it.
The frameworks are (TOGAF et al) are unweildy. They require significant investment in skills, and specialists, and if you are going to "do it right" (whatever that means) you're going to need tooling (which is also prohibitively expensive, because, actually, it doesn't sell well). EA is a closed discipline. It demands those who enter to use its discourse (perhaps all the IT disciplines are like this), and to use the discource of the frameworks that seek to perpetuate.
EA seems to breed model curators; those who make little concrete impact on the organisation; who precide over ever-increasing abstract models defined in notations that only they understand, that add less and less to the over all enterprise and technology delivery, and have less bearing on the real-world they are modeling. The model is always "a myth" of some sort. The more abstract, the long they take to curate, the more specialised the reader has to be, the less useful they are for doing architecture; they are, of course, always useful for that organisational phenomena, "the tick in the box"--but anything is potentially useful there.
EA is deskilling our talented designers, developers, analysts, etc. Why? Because EA, as a discipline, is where the money is. It has been a fact of the paid IT professional's work life that they will be paid more to work at increasing levels of abstraction than to become better and better at what they are good at.
No one is doing EA anyway (which is of course the point)! Typicaly when architects says they're "doing TOGAF", in reality they mean that they have picked certain abstract parts (say the ADM) and labelled the deliverables in a TOGAF fashion, without actually doing TOGAF. No organisation, part from perhaps a well funded Defence Department, would ever, even consider following TOGAF in its entirety So, what we get, is a bunch of TOGAF-certified architects operating locally (ie within a specific business context) to their own experiences and interpretations of the frameworks; which is obvious to all working in EA.
EA is both formalised, and hierarchical. It also has wiffs of command and control built in. If this is the structure of the organisation, then perhaps we have a match. Today, most organisations don't look like this (and certainly even your command and control organisation has the barest sniff of lean thinking). With the engagement with lean principles at the highest levels, most organisations do not even had this as a stated strategy.
Those large, heavy, formal disciplines that used to characterise IT; PRINCE, CMMI, COBIT, Waterfall, to name a few, were founded around a network of assumptions that aligned quality with robust processes (where robust processes meant the production of lots of work products that had no direct bearing on the actual product being produced). I've heard it stated that developers need detailed architecture/designs because they are "stupid", they have to be told what to do. I don't think this has ever been the case, and it certainly isn’t now. Of course, if we take stupidity as our base assumption, and instigate controlling processes drive a certain banal adherence to a way of understanding position, skills, pay, then the process might create this helplessness, not the other way around.
Just Stop Doing It
We have lost our way. As EA's we are no longer technologists, we align to the business, we are the business. But the business sees us as technologists. Our peers in the IT engine room see us as, at best, architectural astronauts, and at worst, model curators. What if we just stopped doing it?
Well, a quick survey of any organisation and you might find the same "architectural critiques" that existing 10-15 years go; that is before investment in EA. Systems in silos. Poor processes. Limited integration. Too much lock-in. High costs of IT. Etc etc. Of course there has been advances; of course there has. For instance we don't buy as many mainframes as we used to. But there is worryingly much to do. And all this during a period of EA!
By "stop doing it" I mean something like this. Lets get those stable, perennial architectural disciplines that exist "lower down" in the architecture stack up into the enterprise; loose coupling, high cohesion, interfaces, cost-cutting concerns, etc. What might be call "enterprise engineering". And, perhaps, lets move those (IT-focused) business architects into the business, and get them focused on driving business outcomes and not just describing business processes. And lets start caring that our practices have too much weight in them; thats it harder to connect 2 systems within our firewall than to mash-up two completely disparate systems on the web.
I'd really like to start by focusing on this idea of "enterprise engineering"; a poor term, I know, but perhaps worth considering in an IT world where the best organisations proudly hail their engineering, and the rest of us stare into the void wishing we were elsewhere.
So, perhaps next time….
Doing Enterprise Architecture by Stopping!
"Of course, if we take stupidity as our base assumption, and instigate controlling processes drive a certain banal adherence to a way of understanding position, skills, pay, then the process might create this helplessness, not the other way around."
This is precisely what we've been doing, with the expected result.
" It has been a fact of the paid IT professional's work life that they will be paid more to work at increasing levels of abstraction than to become better and better at what they are good at."
You nailed it right there. And this hasn't changed.