Monday, July 21, 2014

Agile and the Fairy Godmother

Once upon a time, a long, long time ago, well some 20 years actually, some clever folk figured out a way of structuring work that was quite revolutionary. So revolutionary in fact that most people didn’t understand it. Now a few folk in a parallel world worked hard over the years to make this method work, and they invented more ideas, frameworks and tools. And every day they are constantly improving their productivity and quality; and many of them also use Agile methods, as they are entirely complementary to the revolutionary method. The method – Design by Contract.

However the mainstream of the market seems to be fixated on Agile methods, as being the metaphorical silver bullet. Yet we all can’t help noticing that Agile Land is not such a happy place. There is continuing debate about the difficulties of making the methods work in the enterprise; the fragmentation of the original Agile principles and the outbreak of religious wars. And I am minded to comment, again, that the root problem with Agile methods is that they are one-dimensional - solely focused on people and process to the exclusion of all the other opportunities to create agile businesses.

At the heart of the conundrum is the need to focus on some different questions. Such as:
- How can we reduce the amount of work that has to be done?
- How can we structure the outcomes (deliverables) so they are inherently agile?
- How can we structure the work so that there is real traceability between the intrinsic business model and the delivered systems and services?
- How can we ensure that overall technical debt is always reducing faster than new functionality is being delivered?
- . . .
You get the idea.

So back to my parallel world. In Design by Contract the business problem, typically the Use Case, Services and Operations are attributed with Pre-conditions, Post-conditions and Invariants (rules that must remain constant). These artifacts provide us with functional and design level specifications that can be produced in an Agile, iterative manner, that
a) Deliver implementation independent service specifications (descriptions if you prefer)
b) And therefore also for publishing API specifications
c) Form the basis for structuring the code that is fully traceable to the business model
d) Create inherently agile systems and service structures
e) Create the structure (stubs) for Unit, Integration, Functional and Regression testing
(e.g all conditions and rules within a given test scope must be tested)
f) Depending on the technology employed to define the conditions and invariants, (rules engines, pseudo languages etc) both code and test cases can be produced automatically.

I was prompted to write this blog because I happened to read Rob Marvin’s useful blog on testing, and his very interesting ideas for improving test productivity to keep up with Agile projects. In his piece he talks about automation, but actually seems to miss the opportunity to auto-generate the test cases. Even more important, he seems to believe that the current state of Agile projects is his benchmark that he has to keep up with. I would comment that state of the art Design by Contract projects together with model driven frameworks are delivering order of magnitude greater productivity with exceptional quality, and this ought to be the where testers should set the bar.

Sometimes it seems to me that the Agile methods community is rather in the situation of hoping a Fairy Godmother will appear, wave a magic wand and all will be well. Everyone will use Scrum like they ought to; enterprises will waive their awkward little requirements for inter-project coordination and all will be well. But while the Agile community continue to ignore the broader scope of agile enablers this day won’t come.

If you are interested in an example of Design by Contract at work take a look at the Agile Service Factory.

Wednesday, June 11, 2014

Mapping Agile Architecture

Jason Bloomberg recently published a mind map for Agile Architecture. It’s a nice map that sketches top level thinking and I welcome that. It prompted me to do a drill down.
Mind maps are useful in that they are, by definition free form and intended to support brain storming. The downside is obvious – they are generally inconsistent and cause modelers’ intense frustration! Caveat emptor over, I fully agree with Jason that we need a dual interpretation of Agile  - that is Agile practices and Agile Architecture, and I have written about this on many occasions. Also that the entire motivation is about business agility. On this last point my mind map is clearly a little more technical than Jason’s, and on reflection I think that is because it’s essential to converge the business and technology concerns.

For example, the map suggests a strong capability centric approach to interpret the business morphology. However this is insufficient; the technology must also establish appropriate levels of implementation independence that will facilitate the pluggability of business capabilities. Similarly you might think that considerations regarding the platform and delivery technology (such as MDA/MDD) are irrelevant to business concerns. However the platform and platform delivery technology are potentially massive drivers of rapid iteration and ongoing change, because they encapsulate common application level infrastructure and common services, so understanding the “business” standardization and localization model is crucial to delivering agility through this structure.

     Jason's Mind Map
Related posts: 

Tuesday, May 27, 2014

Open Architecture for the Public Sector

In his blog Richard Veryard relates how the public sector, in the UK at least, is moving slowly towards understanding the impact of Digital Government. He reports on a workshop he attended recently to discuss some of the architectural aspects of Digital Government, hosted by Skyscape. One purpose of this discussion was to feed into the Labour Party Digital Government Review, and possibly into the Labour manifesto for the next election.

I very much agree with Richard that the key to success in this area is multi-agency collaboration that delivers joined up government. Sadly the rate of progress is evidently very slow. Richard and I responded to a UK Government call for information on Shared Services way back in 2005. See below for link. Much of what we said at that time, based on work for the Danish Government, is still highly relevant - the need to develop joined up government architecture based on business driven SOA.

Today we might add:
1. While information sharing is clearly needed between silos, an (implemented) business capability architecture would be a better way to rationalize inter-government complexities, and lay a componentized, federated foundation for collaborations. The business capability based architecture also defines units of integrity that are ideal units of provisioning or acquisition which is essential in today's Cloud based, multi-source environment.

2. In his post Richard mentioned that at the workshop there was "considerable discussion about the role of Government in providing a platform, and whether the platform should be a Minimum Viable Platform (similar to the Internet) or provide added value." The idea of an MVP is very interesting as it parallels work that we (Everware-CBDI) are doing. In our Agile Service Factory we guide organizations to develop what we now call a Common Core, which handles all the standardized services and all non-functional behaviors. We are finding this is 80 - 95% of the code/effort/cost. This Common Core becomes the enterprise (or domain or capability) platform, which is managed as one or more product lines concurrently evolving with solutions, and providing continuous change with massive reduction in cost. But the primary value is the ability to exert standardization and governance over parts of federated solutions while actively facilitating localization that does not compromise the standardized components.

3. By focusing multi-agency inter-working on capability, we have a business driven approach that is inherently service oriented, and clarity of understanding around what aspects of the common platform should be shared both within and across agency.

Richard Veryard: Towards Open Architecture for the Public Sector
2005 CBDI Report Shared Services for the UK Public Sector

Thursday, May 8, 2014

Not Another Framework? Part 2

In my last post Oh No! We need another Practice Framework,  I developed the earlier theme commenced in “Beware the New Silos”. I argued that the widely used frameworks are narrowly discipline centric and actually inhibit cross discipline working. I described how my own firm’s experiences have led to the development of a de facto framework, (we call it SOAM) and illustrated how this is essentially a value chain commencing with customer demand and finishing with value add to some enterprise.

I ended by sketching some basic principles concluding that we need a new framework that is goal driven and incorporates the entire value chain of capabilities, which of course may selectively reuse some parts of existing frameworks. In this post, I suggest a strawman that covers a) principles and b) capability model.

Before diving into principles, it will be useful to declare some scope. Our framework has developed from working with larger enterprises, both commercial and government in the area of business service and solution delivery. All of these enterprises share common issues that they have extensive legacy application assets that act as a serious inhibitor to business change, and successive, narrowly scoped solution projects over many years have often resulted in great complexity and technical debt. It is also common in my experience that enterprise architecture functions are routinely bypassed or ignored; that Agile methods have been attempted and found useful on narrow focused projects, but because of the constrained focus, tend to increase overall complexity of the ongoing application asset base; that consistent customer experiences are commonly compromised by narrow focused projects; and line of business managers in large enterprises are frequently dissatisfied with IT application service support.

The objectives of the framework are to:

- describe practices relevant to service and solution delivery in the digital business environment
- achieve a balance between short term goals and longer term objectives
- support progressive transformation to an enterprise comprised of independent business capabilities
- facilitate continuous, short cycle time evolution of business capability
- progressively and continuously resolve legacy portfolio complexities
- enable rapid delivery at low cost without compromise in quality

Principles are foundational for any framework. 

Principles should be enduring and lead to both excellent policy communication and policy interpretation in everyday situations. I also find it useful to classify principles by subject.

Capability Model

In business architecture the capability model has become ubiquitous. And in thinking organizations I observe delivery of highly independent service and solution components that reduce dependencies and the impact of change, as well as mirroring the IT architecture on the business organization. Why wouldn't we use the same approach in defining a set of activities to deliver services and solutions?

If you are uncertain about the capability concept, it’s important to appreciate that the optimum business capability is one that enables:
- maximum cohesion of internal functional capability, plus consistency of life cycle, strategic class (core, context, innovating . . . ), business partition (global, local, LoB . . ), standardization, customizability, stability, metrics and drivers
- defined, stable dependencies that are implemented as services
[Further reading on capability optimization ]

In the capability dependency model below, the arrows are dependencies. For example, Demand Shaping is dependent upon Conceptual Business Modeling and Portfolio Management.  So this is not a flow diagram, rather all the capabilities should be regarded as iterative, I will come back and discuss how Lean principles operate in a framework like this, and as discussed above, highly independent.

Most of the capabilities in the model are self-explanatory. However some need explanation:
1. The Conceptual Business Modeling capability is the ability for business stakeholders to describe business improvement in conceptual terms. Many business people speak in solution terms. Most business requirements therefore surface as solutions, some more baked than others. Because the business stakeholder generally has the budget, the solution vision frequently drives and shapes the project with outcomes that frequently compromise the existing and planned portfolio. By educating business stakeholders to communicate in concepts, the opportunity is created to develop the business improvement idea without preconceptions of implementation or product, and to optimize architectural and portfolio integration. 
2. Demand Management is reasonably well understood. Demand Shaping is best regarded as a complementary capability that takes raw customer demand and decomposes into components such as pre-existing or planned services/APIs, considers opportunities for modernization and provisioning, and reassembles as a set of projects or project components that optimize the progressive development of the portfolio. Demand Shaping is primarily an architectural task, but should be run by a cross functional team including architect, product management, business design and technical expert roles. 
3. The Architecture Capability is shown as a decomposition of sub-capabilities, essentially one for each View, plus modernization. Whilst modernization is not classically an architecture view, there is commonly a specialist requirement for modernization architecture that will include identification of appropriate transformation and transitional architecture patterns. The primary objective of all of the architecture sub-capabilities is to define realizable structure to meet the demand and, as discussed above, to optimize opportunities for modernization and provisioning. While there is no explicit enterprise architecture View called out, each architecture capability should be executed separately and iteratively for reference, portfolio, program, project and module, thereby defining progressive layers of standard functionality that will be common to the defined scope, as well as situation specific business functionality. 
I will detail all the capabilities in a subsequent post.

Final remarks. 

This high level view of the framework has attempted to list a set of principles and associated capabilities required to support the value chain illustrated in Part 1 of this extended blog post. What will hopefully have become clear is the need for architecture capabilities particularly to be involved throughout the value chain. This approach integrates all types of architecture (enterprise, service, solution, deployment  . . . ) into the business improvement value chain and creates better opportunity to demonstrate the ROI on architecture. Further the approach prevents enterprise architecture particularly becoming divorced from mainstream business improvement and encourages a better balance of short term and strategic goals. What will not yet be fully clarified is how the framework is very strongly focused on realizing architecture in delivered services and solutions, as a series of successive collaborations. I will describe how this is done using a Lean approach in a subsequent post. 

                  Beware the New Silos

Monday, May 5, 2014

Oh No! We need another Practice Framework?

A few years ago I commented [in Beware the New Silos, ref 1 below] that in a complicated world we cope by specialization - and across the industry in general and in individual enterprises specifically we have created silos of our primary disciplines. The widely used frameworks and methods illustrate that common practice of discipline centricity. We don’t have to look too far for examples such as Enterprise Architecture (TOGAF), Governance (Cobit), SOA (separately by OASIS, OMG, Open Group), Agile methods (many and various), Business Process Management (BPM) and IT Service Management (ITIL). All of these disciplines, whether de jure or de facto standards, are all very narrowly focused with minimal treatment of how the disciplines might work together.

A good example is how Agile methods are tightly focused on application development and architecture is assumed to be an integral part of the Agile delivery project. Yet the reality in all enterprises is that significant aspects of architecture must be consistent at the domain or enterprise level. Another good example is how the three standards bodies OASIS, OMG and the Open Group were so divergent in their treatment of SOA, they commissioned a report [ref 2 below] to explain how these inherently duplicating standards and specifications relate to each other.

The level of adoption by enterprises or service providers of all these and similar practice frameworks and standards is of course highly variable. However it must be said that the very existence of the discipline based materials will frequently have some considerable influence on how enterprises organize themselves.

The thinking IT or business professional might also like to question just how up to date these frameworks are, and how they support today’s business goals, which for most of us have changed dramatically over the past few years. We might also speculate whether the education and certification ecosystems that feed off some discipline based frameworks may discourage rapid evolution. A good example is how TOGAF maintains the core architecture style as application centric and treats SOA as a style extension. This is really quite extraordinary because by now everyone knows and many accept the digital business is going to be inherently service oriented. In practice of course the TOGAF specifications are so extensive that making fundamental changes may be very difficult, but it demonstrates neatly how legacy capabilities become “part of the furniture”, not just in frameworks but also in delivered systems and services.

Which brings me right back to the question – do we really need another practice framework? 

For several years now I and my colleagues have been evolving and implementing a different transformation approach. Initially we focused on SOA. And as many will know, we were fundamentalists in this area and we published detailed and open meta models for the service architecture and delivery life cycle based on “everything is a Service”. This approach has been very successful, particularly when the service architecture conforms with a strong layered reference architecture and rigorous componentization of services and business capabilities. But because we always knew that there was no such thing as a green field, we worked to harvest knowledge from existing systems. And over time we made a virtue of this,  focusing on continuous modernization as a matter of principle. More recently we have further evolved the approach to embrace Agile and MDD, establishing an agile service factory and generating code from rigorously defined service specifications.

But we found many of our customers struggled to implement a strategic SOA because business change was implemented project by project. And sure enough, project specific services and SOA have become ubiquitous; you might say almost a contradiction in terms. To counter this we advise that the demand management process should be complemented with demand shaping that decomposes the customer solution based demand to discover requirements for services and other sharable components and then re-aggregates the raw component demand into related projects that coordinate the delivery of business solutions and strategic services. 

But even though this approach works well at a project and technical level, we frequently encounter difficulty in persuading business stakeholders to balance short term advantage with more strategic goals. And we recognize this is because business stakeholders habitually make investment decisions on flawed criteria, because the ROI model only looks at the solution in isolation, rather than looking at the strategic opportunities to implement better architecture, address portfolio rationalization and reduce technical debt. This prompted us to find ways of working more effectively with business stakeholders. To encourage them to understand and indeed influence the conceptual business model and to understand a richer underlying business model that reflects a more comprehensive cost benefit model. And this helps to bring LoB managers into the conversation on demand shaping – balancing immediate solution requirements with longer term goals. 

In effect what we did was to establish a service and solution delivery value chain that commences with the raw customer demand and finishes with the delivery and integration of some useful business capability, but crucially with a much improved balance between immediate solution needs and longer term strategic goals. And it’s this balance that many enterprises find extremely difficult to achieve.
The core problem is that disciplines are vertically integrated; set up to optimize the discipline at varying levels of abstraction. In contrast the value chain approach optimizes across disciplines in pursuit of overall value chain outcomes. And this is only achieved by value chain activities that encourage effective collaborations between multiple capabilities and disciplines.
In the beginning we (Everware-CBDI) had a framework - Service Architecture & Engineering (SAE). The name makes it clear this is a forward engineering approach, and whilst it was very strongly business driven, it would be fair to say that the business modeling components were less well worked than the architecture, design and delivery. And as described we have very naturally, as part of the process of supporting large enterprise initiatives, expanded the framework capabilities and embraced an inherently Agile approach.

The principles underlying the framework now include:
- business capability based modularity
- pervasive service architecture
- continuous modernization
- service evolution not (one time) delivery
- model driven architecture and development
- everything is inherently agile - iterative, evolving, and narrowly focused on specific business capability delivery.

So to answer my own question, we clearly need a new framework. But it's a very different practice framework to the ones that we are are accustomed to.
In our natural evolutionary process we recognized that the original (SAE) framework was merely one component of a much broader body of knowledge and practices. The new framework is goal driven, not discipline driven and incorporates the entire value chain of capabilities. But the capabilities are not standalone, they are effectively services that are executed in a way that supports the overall business goals of the enterprise, domain or program. We refer to this as  Service Oriented Application Modernization (SOAM).

Interestingly this is not a monolithic framework. It's vital to treat the framework as a set of capabilities with defined services and dependencies. Some might say, "eating our own dog food". In this way we don't make the same mistake as legacy frameworks such as TOGAF, that are very difficult to keep current.

Finally what happens to the existing discipline based frameworks and standards? Of course they can be used in conjunction with the SOAM framework. But we do need to be careful not to just inherit monolithic capabilities. Increasingly we find it necessary to do this very selectively in order to use capabilities that fit and support the value chain. Perhaps in time the various disciplines will recognize the monolithic nature of their practices, and decompose and modernize into more goal oriented components. Meanwhile, in SOAM we have demonstrated that it is possible to reinvent the wheel.

Some SOAM Components:
    Agile Service Factory
    Agile Modernization
    Conceptual (Agile) Business Modeling
    SOA Reference Framework

Ref 1: Beware the New Silos

Wednesday, March 19, 2014

Agile is not Dead, it's Morphing

I note healthy discussion around whether Agile is Dead [ref 1]. And while I may sympathize (sic) with many of the comments, particularly the commercial trivialization of education, the core issue must surely be the difficulty of adopting de facto Agile practices to support real world enterprise programs and projects. My experience is most of the advice and guidance out there is predicated on scaling the de facto Agile development methods. And this isn’t the best place to start.

An exception is Dean Leffingwell’s SAFe, [ref 2] which does introduce the idea of portfolio, program and project perspectives and intentional architecture. I recommend this framework as an intelligent set of practices, but for me it doesn't go far enough because it is still primarily about development practices. This is the core problem - that Agile is development specific and practices only. In the enterprise, Agile development needs to be an integral part of a bigger ecosystem spanning business design, architecture, requirements, modernization and operational transformation practices plus architecture, delivery and modernization disciplines.

I am indebted to Dave Thomas whose recent blog, [ref 1] includes a worthy successor to the Agile Manifesto, reducing the original, development specific values and principles to a minimalist, more generic set. He says; “here is how to do something in an agile fashion
- Find out where you are
- Take a small step towards your goal
- Adjust your understanding based on what you learned
- Repeat
And this is more useful in the enterprise context because it is relevant to a broader set of activities than purely software development.

The diagram below is an outline maturity model template for Agile in the enterprise. It suggests there are four key views that need to be part of the transformation.  In addition to agile practices we need to be equally focused on what elements of agile architecture are required for an enterprise. What the agile delivery framework is and how the existing application portfolio will be modernized to progressively eliminate the duplication and complexity present in every enterprise on the planet.

People & Process. Much of the dissatisfaction with Agile arises from the limitations of the basic Agile practices, and the need to compromise these in an enterprise context. Both DAD and RUP (yes it’s an iterative method) are examples of extended or hybrid practices that introduce coordination, phasing and other disciplines that are more acceptable in enterprises that require traceability, governance and compliance with pre-existing life cycle practices. Enterprise frameworks such as Leffingwell’s SAFe as discussed and Everware-CBDI’s SOAM are examples of frameworks that adhere more closely to the purity of Agile principles while addressing enterprise specific needs. SAFe provides a framework which is more strongly Lean, coordinating portfolio, program and project activity to meet agile release train demand. SOAM provides a complementary, full life cycle process framework for software service modernization and delivery.

Agile Architecture. There is a requirement to articulate the enterprise requirements for agility as a reference architecture for business agility. In today’s fast moving world core architecture for the business, services, implementations, technology and deployments needs to be:
- under continuous development using Agile principles
- derived from the assessment of business needs for response to change, and constantly updated to reflect competitive and technology opportunities and threats.
- mapped to service architectures, patterns, policies and modernization strategies
- modeled using MDA/MDD to allow delivery as consistent architecture runways for portfolio and demand management, programs and projects.

Agile Delivery Framework. Most enterprises have a well-defined delivery framework of tools, repositories, templates etc that are designed to support well established QA and delivery policies. This is one of the most common inhibitors that Agile projects in the enterprise have to  overcome. In an enterprise Agile context that framework must be realigned to provide maximum automation of life cycle management and governance so that key enterprise requirements for integrity can be met without loss of productivity. Similarly the development  activity must be structured so that developers can extend the architecture runway with business solution specific rules and behaviors in a managed fashion which preserves the integrity of the architecture. Everware-CBDI has pioneered this runway extension capability implemented as a model driven (MDA/MDD) capability in which the runway code is generated and provided to developers enabling very significant productivity gains in both forward engineering and even more so in iteration.

Agile Modernization.  Finally in an enterprise context the elephant in the room is the existing or legacy portfolio. Unless this elephant is addressed, the enterprise will continue to create more and more complexity, increase costs and reduce response times to change. What’s required is a discipline of continuous, Agile modernization. That means, using Dave Thomas’s minimalist manifesto [above] every portfolio item, program and project must include steps to find out the current situation and address minimum goals that reduce complexity and support a progressive modernization strategy. Without that, all enterprise Agile projects will remain narrow focus and simply add technical debt.

I suggest that while Agile is not dead in the enterprise it is certainly struggling to survive. This is because Agile practices alone will be suffocated at birth by enterprise realities of consistency and integrity; or turned into narrow focus, standalone projects; or morphed into BAU. I really don’t want to enter into a debate about nouns or verbs, life is too short.  IMO Agile has considerable momentum and it can be morphed into a ground breaking concept that delivers enterprise business agility.

1 Agile is Dead (Long Live Agility)
2 Dean Leffingwell SAFe

Thursday, January 16, 2014

Architecture for the Digital Business

There’s another metaphorical asteroid approaching the world. The last one (the Web) hit around 1991 and we are still struggling to come to terms with it. But the next one is an order of magnitude bigger - it’s often referred to as Digital Business. It’s not a bad name, but it doesn’t immediately signal some of the awesome impacts that will result. Essentially Digital Business is about Web enabling everything we do. Don’t be fooled into thinking this is about Google Glass, or the Smart Watch. It’s about Web enabling “everything”, from vehicles, power and water meters, appliances, houses, offices, driverless cars to traffic management systems, personal healthcare and anything you can think about. All these nodes become devices using sensors to communicate on a peer to peer basis via Machine to Machine (M2M) networks, and generate vast amounts of data that will revolutionize the way the world works! Like the Web, adoption will be slow at first, but the initial stages are already happening and we can expect it will be another 15 to 20 year change cycle which follows an exponential growth curve.

Don’t believe me? A really good example I just happened to see as I wrote this blog, is the smart bike lock from Lock8. The lock is paired with a mobile app and, lo and behold it’s paired with a peer-to-peer marketplace that a) alerts the owner if someone tries to steal the bike and b) tracks it if the thief manages to take it away, and c) even allows owners to loan or rent their bikes to friends or others registered on the service, and make a little cash on the side. Of course Lock8 also works in lots of other use cases. And just to reinforce the message, Lock8 have been hugely successful in raising significant funding through crowdsourcing!

It’s interesting to observe the media debate about Google Glass, which is strongly focused on privacy issues. Yet there is little or no debate yet about the Web enablement of almost everything, which has far more implications, not just for privacy, but also security, reliability and risk, as well as impacts on employment and changes in long established cost assumptions. It’s likely that Digital Business will trigger huge cultural changes. And that’s before you get to talk about humanoid robots to do our housework, look after children and older citizens. And yes they are coming also.

So what’s happening to the archetypal enterprise in this process of change. And what does enterprise architecture look in this changing environment? Of course technology is the primary stimulus, but we need to look at the social and economic effects of new technology enabled capabilities to understand how the enterprise may respond.

By coincidence I noted the Drucker Form met last month in Vienna and one of their major topics was Complexity. And there was fascinating debate about what complexity is, and whether it is increasing over time. And surely the Digital Business with the expected exponential growth in nodes, information and dependencies is going to be a major driver of complexity increase.

Roger Martin, of the University of Toronto, told the forum that one reason we feel overwhelmed by complexity . . . is because of a growing tendency to see and study problems within silos. The complexity of our world has actually not increased. It is just the unaddressed “inter-domain complexity” which makes us feel like everything has become more complex.

Don Tapscott, (of New Paradigm) was clearly responding to this issue of inter-domain complexity when he told the  conference - first, our institutions need to radically decentralize. Further, the future is not to be predicted, but to be achieved through a new dynamic paradigm including self-organizing, emergent and sometimes resilient networks involving millions of stakeholders. These networks embrace, active participation, uncertainty and constantly changing conditions, and they show great promise for solving global problems and governing an volatile and complex planet in the future. Command and control being replaced with self-organizing networks. Linear and non-linear organizations.

I particularly liked Tapscott’s analogy of murmurations of flocking starlings. Starlings somehow organizing themselves en masse to see off predators are at the opposite end of the (command and control) leadership spectrum from the arch bureaucrat. It reminds me also of the March of the Penguins. Did you ever see the huddle of thousands of penguins, their backs to the ice cold wind, continuously moving inwards and then outwards to allow all the group members to share the extreme exposure and protect the inner circle.

And of course this is what’s happening today. Self-organizing networks are forming everywhere, facilitated by the Web, and enabling collaboration and information sharing and transactions on a local and global scale without any central command or control.

So what does the Enterprise model of the future look like? In the diagram below I suggest the enterprise is a network of networks, a series of Capabilities that are supporting many Networks. The scope of the enterprise is the extent of interest in the Networks and Capabilities, not the boundaries of the enterprise as a legal entity. Participants are of course any form of Node, Device, Person, Entity that has a relationship with the Capability through some form of Experience and /or Channel. That Experience is provided by one or more Solutions, but we should plan that distributed Capabilities collaborate to share common assets on many levels. Note this is a very high level diagram, and you can safely assume that all the relationships are many to many. Further there is much more detail that is required particularly to make this a business AND IT model. But it is sufficient for my purpose in showing a core enterprise.

So what does the Enterprise Architecture look like in the fully distributed, self-organizing enterprise. The diagram below shows that the new EA needs to establish the basis for the federated business with architectures that are about managing the interconnections and dependencies, while allowing the distributed business to have as much freedom of action as is possible. The Reference Architecture , therefore defines the minimum necessary alignment. The Platform Architecture manages the essential business and IT interoperability and consistency that ensures the federation has minimum necessary cohesion. The Common Asset Architecture is then a portfolio of pluggable service components – that comply with the platform and facilitate sharing of common behaviors where appropriate, and enable solutions that have the necessary level of enterprise consistency together with required level of localization. Each of these architectures of course has the five views – Business, Service and Application, Implementation, Technology and Deployment. But this is a Federated Enterprise Architecture, that’s purpose is not to exert command and control over the leaf nodes, rather to facilitate the minimum necessary consistency to ensure the enterprise is a) in regulatory compliance and b) able to optimize cross enterprise dependencies, information and customer experience.

 In the Digital Business world Enterprise Architecture is vital, but it needs to be tightly focused on optimizing the distributed, federated business work, which means delegation. And here we have another cultural problem, because we have to embrace subsidiarity, the notion that all matters ought to be handled by the smallest, lowest or least centralized authority capable of addressing that matter effectively. And in general we are very bad at subsidiarity. Enterprises tend to prefer command and control as the default modus operandi.

The diagram below suggests that EA needs to undergo a significant transformation, moving from the As-Is in which the objective is frequently to seek the highest level of commonality to a To-Be model in which the aim is to organize the business as independent capabilities. Clearly this is much more than an architecture issue. It must be reflected in the way the business and IT are organized, transferring and delegating responsibility to leaf nodes, while establishing and governing the key policies that make the enterprise a coherent whole.

In my experience many enterprises have frequently responded to Web based business opportunities in a tactical, technology led manner. This has led to duplication of core business capabilities and, I observe, critical weaknesses in customer experience and lost opportunities. Many enterprises particularly telecos, publishers, video stores, music and media firms, have through competitive pressure come to recognize the strategic importance of the Web. In embracing the Digital Business, enterprises may be tempted to go down the tactical route, but this is likely to be a big mistake, as the network effect will become an integral part of the enterprise’s product and services.

It’s true that Asteroids don’t hit the earth very often, and when they do the impact is immediate and very dramatic. Digital Business isn’t going to be instantaneous, but the impact is likely to be colossal.  So the key message is - figure out your new reference model, get rid of command and control, but manage the cross network dependencies at all levels of the reference architecture - that's how you deliver real agility in the new world.

Drucker Forum
Blogpost: Agile Architecture
CBDI Journal Report: Capability Planning and Analysis