As far as we can tell, most organisations that are using Agile development methods (and most are) started doing so because someone in IT thought it was a good idea, gave it a try, and it spread. That is to say, it was (is) an emergent strategy (and none the worse for that, of course). The problem with emergent strategies is that they tend not to take much account of the impact they're having on the rest of the strategy, or the rest of the business - usually because the folks adopting the new strategy don't have any influence on or responsibility for strategy or the rest of the business.
Prior to the use of Agile methods, there was never a time when the majority of IT projects were regarded as successful in terms of time, cost and quality (fortunately, these are not the final arbiters of success). Unfortunately, a recent Agile survey (10th Annual State of AgileTM Report source www.versionone.com) indicates that things haven't really improved making AGILE mainstream. What has changed, though, is the list of reasons for lack of success, and these focus on the failure of management to adapt to the adoption of Agile. While that may sound a little like blaming someone else (or, indeed, anyone else), it does have some merit. And, usefully, we can actually do something about it, if we identify what's different in making AGILE mainstream, and so a candidate for a management response.
Firstly, the cycle time of Agile development is much shorter. This demands that project governance must be able to respond in a similar timescale. Instead of having a monthly project board meeting to review progress, spend and the risk log, you'll need one to review the outputs of the recent iteration, decide whether a further iteration is required, and if so approve the priorities for the next iteration. If the Agile development is part of a programme, there won't be time for both project and programme board reviews - and more generally, if you have more than one level of governance, Agile demands that you go straight to the highest level required for the budget / timeline / criticality / other measure of importance to the business, without stopping off at any intermediate levels - you need to cut out the middle man.
Secondly, the amount of business knowledge demanded by Agile on a day-to-day basis is much higher (because requirements are being identified, fleshed out and validated as the work proceeds). This means that the availability of expertise in operations must be higher - there needs to be a 'go to' person available to the project as well as to operational staff.
Thirdly, development teams need to be small (seven plus or minus two is often the recommended number) and stable (being more cross-functional, it's harder for Agile teams to 'gel', but much more effective when they do). This has some serious consequences for how the organisation structures its portfolio of work and how individual projects are structured, and also implies strongly that the practice of raiding existing projects for resources is highly counter-productive (it probably always has been, by the way).
There are other things for making AGILE mainstream, too, of course. Contact CITI to see how you might adapt to be better at adopting Agile, or I can always be reached directly via email me at HSmall@citi.co.uk
Managing Consultant, CITI Limited