Traditional waterfall-based projects promise successful outcomes through meticulous planning and tight control of the scope and risks, but it’s a promise that isn’t always fulfilled. The famous quote that, “No battle plan survives first contact with the enemy” neatly sums up the problem with waterfall in situations where the detail of requirements isn’t entirely clear or where there are likely to be many externalities not easy to control along the way. Nevertheless, many software buyers still yearn for the certainty of a fixed-price waterfall project.
Conversely, an agile methodology acknowledges the fact that often we don’t have a full understanding of the requirements at the start or know the best way to achieve our desired outcomes before we start. Agile believes that it’s better to figure things out as we go along but this too can lead to disappointing results, especially when agile teams lose sight of non-negotiable deadlines or ignore expectations that even though the project is agile, its scope is fixed; as the famous quote has it, “Blessed are the flexible, for they can tie themselves in knots.”
Both approaches work, in different circumstances: waterfall when requirements are very well understood and the deadline is set in stone; agile if service innovation is a desired outcome and requirements can evolve, as is often the case with business transformation programmes.
But what if that same business transformation programme encompasses both challenges?
- A clear need to move from one system to another at a given point in the future;
- An objective to create innovative processes spanning different business functions.
We faced exactly this challenge while working with a leading housing provider in Wales that needed to replace its core finance system but recognising that this would be a ‘once in a generation’ opportunity to innovate. To meet the challenge, we adopted a hybrid approach, implementing a new finance system (Microsoft Business Central) using waterfall, while simultaneously taking an agile approach to the innovation programme. This approach enabled our client to control its costs and plan with certainty in the critical accounting areas (waterfall) while shaping business change through continuous review and feedback (agile).
The project tradition
The ‘iron triangle of project management’, understood since the 1950s, lays out the inescapable fact that the outcome of any project will depend on:
- How much you want to achieve (scope);
- How quickly you want it delivered (time);
- How much you want to spend on resources (cost).
Setting objectives for any two of these inexorably determines what you will get for the third. This has been expressed succinctly as, “Quick, cheap, good – pick two!” Waterfall methodologies seek to maximise the probability of successful outcomes by creating frameworks to make sure that the three choices are in equilibrium before any significant effort or expense is committed to a project.
Waterfall therefore seeks to manage risk and uncertainty from the start, tracking progress through a series of stages (such as analysis, design, build, test and go live) towards a predefined goal. This approach delivers projects as a sequential set of activities. However, despite the emergence of sophisticated project management methodologies such as Prince2, waterfall has a chequered history and after each high-profile failure, organisations have sought ever more control in an attempt to avoid similar fates. Eventually, projects became scary things, shrouded in a mysterious set of rituals that could only be understood by ordained members of the Prince2 priesthood.
The project reformation
An alternative approach emerged following the publication of the ‘Manifesto for Agile Software Development’ in 2001. In contrast to waterfall, agile prides itself on being a flexible approach that tackles planning, design, implementation and testing through a series of short iterative cycles. The manifesto opens with a declaration of its values:
- Individuals and interactions are more valuable than processes and tools;
- Working software is more valuable than comprehensive documentation;
- Customer collaboration is more valuable than contract negotiation;
- Responding to change is more valuable than following a plan.
The second half of each of these statements is anathema to the risk-averse methodologies associated with waterfall. Nevertheless, the prospect of speed, freedom and rapid progress has seen agile gain significant traction in the project community. Indeed, in the UK, agile has gained sufficient momentum to displace Prince2 as the default project methodology in many organisations over the past 20 years, particularly when delivering bespoke line-of-business systems.
The counter-reformation
Although Agile was principally formulated with software development in mind, its evangelists recognised the need for additional tools and techniques if it was going to become mainstream. These were needed to maintain focus, sustain collaboration over an extended period and create checks and balances to safeguard the project team and its stakeholders. The desire for credibility in these areas led to the emergence of agile methodologies such as Scrum, Kanban and Lean.
Die-hards in the waterfall camp, though, still yearned for the certainty of fixed-scope, fixed-term, and fixed-price projects. Eventually, sequential and iterative methodologies were synthesised in the 2015 publication of Prince2 Agile, a methodology that seeks to combines Prince2 management controls with a broad toolset of agile delivery techniques and frameworks. For some, this is the best of both worlds; for others, an incoherent and illusory mishmash of fundamentally-opposed attitudes.
The dilemma
How do you approach a project when some aspects lend themselves to waterfall, some to agile? Where some requirements are fixed and clear, but there is also a strong desire to innovate? Choosing one methodology over the other can’t be the answer and despite its promise, Prince2 Agile still requires proficiency in a baffling array of processes and frameworks that remains the preserve of an anointed few.
We recently encountered this situation with a large Welsh housing provider that was looking to replace its accounting system while simultaneously exploring opportunities to redesign cross-functional processes between finance and operations. The accounting system was, in effect, an important part of a wider finance transformation programme. This gave us the following dilemma:
- Waterfall was the natural choice for the accounting system due to the pressure to maintain control, support monthly accounting deadlines, handle a mid-year cutover, satisfy external auditors and meet non-negotiable deadlines.
- Agile was more appropriate where the objective was to revolutionise performance across functional boundaries through service innovation and end-to-end process transformation.
The answer
Focusing only on the replacement of finance processes on a like-for-like basis would squander the innovation opportunity. The transformation programme would then require ‘digging the road up twice’ and lose momentum, so we needed a more fluid approach.
After ruling out Prince2 Agile, we knew we had to find a way to achieve the desired outcome without compromising either the rigour required for the accounting system replacement or the flexibility required for process innovation. The approach we chose was to incorporate both agile and waterfall work packages, clearly identifiable as such, nested within fixed deliverables.
In practice, this involved the rapid implementation of a baseline version of Business Central pre-integrated with the client’s HMS (Microsoft Dynamics CRM) which, in turn, enabled the finance team to go hands-on in the very early stages of the project.
In a radical departure from the conventional approach to finance systems implementations, this allowed members of the finance team to take an active role in the configuration and design of core accounting processes through a series of ‘learning sprints’, with the benefits quickly becoming apparent.
Firstly, the team was able to understand the transformational potential of moving to a single, unified data architecture as something real and tangible, rather than as something abstract and conceptual.
Secondly, the team could confidently decide which processes could be made cross-functional as part of the move to Business Central and which could be deferred. Finally, and perhaps most importantly of all, much of the thinking has already been done before the next phase of the transformation programme starts and, almost subconsciously, the finance team has embraced cross-functional working as ‘the new normal’.
This model has enabled us to implement Business Central as a well-defined and tightly-controlled project while simultaneously allowing the finance team space and time to rethink repairs, asset management and rent accounting as cross-functional processes within the client’s broader transformation programme.
Nick Hill is a director of Esuasive.
