Tuesday, 24 January 2012

The Basis of the Requirements-Driven Business

About a month ago I started working for a new three man startup business, Advised ICT Finland Oy (site is only in Finnish, for now) which offers primarily Enterprise Architecture and Information Architecture consultancy services.

Here's the thing, though: for many Enterprise Architecture seems to be something that only Big Businesses should care about: something massive, time consuming and expensive that does not even fit to concept of agile business. True enough, it can be all this - but it does not have to be. The concept of Information Architecture seems a little confusing, too as the definition tends to vary depending on the context.

At the heart of it all things are simpler than some might think.

Identify - Specify - Satisfy
The way I see it business is always driven by requirements. To have a requirements is to have a need; if there is no need making investments and changes simply does not make any sense.

The Requirement-Driven Business thinking is based on three logical steps:

Simple Basis of the Requirements-Driven Approach
Assume that there is a company that has existed for years and business has been reasonably good. Now the management has come to realise that the world is changing and if the company intends to stay afloat in the coming years it must change along with the world.

Identify Requirements
The company has served its customers well over the years, but it is no longer enough just to have a good customer service. Customers have begun to expect more, and the company also wants to have a more direct relationship with the customers in order to improve customer loyalty and the quality of products and services the company is offering, both of which would be good for business.

The management comes up with a list of what the company needs: an online service that integrates with the company's Customer Relationship Management system, offers variety of community features and is customisable over time. In other words, the company has just identified requirements.

In other words, identifying a requirement is as simple as saying "in order for our business to be successful we need..."

Specify Requirements
To say "I need this" is easy. To explain what it actually means in practical terms i.e. to specify a requirement takes a bit more effort.

The first step was to identify requirements. The next step is to specify requirements which will transform few high-level requirements into a list of specific, detailed requirements. A good requirement has qualitative and/or quantitative aspects that can be measured in a way of determining if the requirement has been satisfied.

There are three common types of requirements.
  • Business Requirements: the new online service must benefit the company's business; otherwise there is no reason to invest in it. The investment in time, money and resources must be justified (e.g. increase revenue by X per cent, increase productivity by reducing time it takes to get something done). On the other hand, business requirements must be realistic in order to have any value.

  • Functional Requirements: the new online service must enable users to accomplish specific tasks in an efficient and practical manner. There are also things the service must not allow the users to do. In some cases the service itself must be able to function autonomously. If the service does not satisfy functional requirements it is about as useful a broken window during winter.

  • Non-Functional Requirements: the new online service must operate and perform above specific limits. It is simply not enough to support 10 concurrent users if the service has 10 000 users with a thousand users logged in any given time. These requirements also address multitude of concerns such as reliability, maintainability, security, inter-system compatibility and so on.

Identifying Risks
Any system or operation involves risks which can be handled in one of two ways. Many choose to ignore the risks in order to save time and money in short term only to end up spending that time and money many times over when a risk inevitably becomes a reality. I recommend the opposite: spend some time and money now so that when things (again, inevitably) go wrong the fallout can be controlled.

For example, the Failure Mode and Effect Analysis (FMEA) can be used to identify potential risks and what effects realised risks might have. Risks have three metrics:
  1. The likelihood that a risk becomes realised.
  2. The severity of effects when a risk is realised.
  3. The difficulty of detecting what has happened in time.
Each metric receives an integer value which often is between 1 and 5. By combining these three values the risk priority value is determined.

Through risk analysis an additional set of requirements can be identified and specified. Most risks can be eliminated through careful planning and implementation, and by refining operational procedures. At very least there must be ways to exercise damage control to limit the effects.

Satisfy Requirements
Once the requirements have been identified and then specified in practical terms they can be satisfied. This is to say the hands-on implementation work can be started. This is when the new online service is developed, tested and finally taken to production.

With well-defined requirements the architects, developers and testers know exactly what the new system must do and how it must perform. Well-defined requirements will also make it more likely that when the new system is taken to production it becomes a valuable tool and resource for the company instead of colossal waste of time, money and resources.

However, this is not the end but just a new beginning: as the online service is used, world changes and the company evolves over time new requirements are identified and the cycle begins anew.

The iterative requirement driven cycle.

The Requirement-Driven Business is an iterative cycle of continuously identifying, specifying and satisfying requirements through various means. This is simple in principle but there are many ways to do this in practice:

Enterprise Architecture methodologies such as TOGAF help to control the business as a whole and through this identify various requirements. Through Enterprise Architecture modelling of business management can, for example, see how everything fits together, what could be improved, and how changes in one aspect of the business would affect other aspects of the business.

Information Architects often involve themselves with requirements specification work and risk analysis, but this is only a part of their work, which also involves activities such as high-level architecture design, data modelling, service concept design, and technical team leading. There is no single definition for Information Architecture that would be independent of context, but at least I personally prefer to accept a holistic view of it.

Technical Architects such as Software Architects, Network Architects, and Database Architects along with similar specialties of developers and testers come along when the practical, technology specific implementation work begins in order to satisfy requirements and create something that can be of value. Multitude of methodologies, processes, technologies and frameworks are available at this point and admittedly it is a measure of professionalism to be able to select the ones that best serve the purpose.


  1. Aaahhh.... TL;DR....

    What about your own thoughts? This was great copy/paste from the textbooks. Are all these different architects necessary? Where's the added value?

    I mean - there's real possibility of 'death by powerpoint' with this approach :)

  2. Actually, these are my own thoughts. No copy/paste from any textbook was used :) I may have misrepresented my point, though.

    My main point here is that all business is requirement driven: if something is not needed, there is no reason to invest in it. When something is needed one must first understand what the practical requirements are before the implementation work is started (as opposed to all-too-common approach of "we'll figure it out in future iterations"). Without requirements it not possible to say if a system works as intended.

    True enough that there are many and more methodologies and frameworks. In the end it really does not matter which ones to use. Pick one - or many - that is practical and serves a purpose while avoiding unnecessary complexity. The value comes from having a commonly understood way of working; a common frame of reference that helps people with different backgrounds to understand each other better.

    Another benefit is that when same things does not need to be figured out repeatedly (as in re-invent the wheel) following some well-known methodology, process or framework should improve productivity, at least when kept within reasonable scale to avoid the "death by Powerpoint" :)

  3. Nice big cyclic models look good on PP. And reference models are just references. But What i would like to see is to actually study couple of enterprises and see how what kind of cycles there really is. Not to start out unconnected reference model but to actually map out decision making flows and what not and model them without baggage from reference models, that would be way to get out of pipe dream and maybe bring out some value.

    1. I wrote this blog entry while sorting out my thoughts related to a work assignment I'm currently working on. We have been talking a lot about enterprise architectures and possible ways to handle customer cases, but as we all have different professional backgrounds we sometimes end up doing the classical thing: talk about same thing by using different terms and concepts, or talk by using the same terms and concepts while meaning different things. People make life interesting :)

      I will probably write more about this topic later and those posts would bemore specific and practical. Generally speaking I prefer to move from generic to specific without making assumptions about what a reader might already know unless I'm writing for specific people (I'm a firm believer that assumption is the mother of all fuckups).

  4. So by this blog post you state that in your mind this will result in a good piece of software? Or better yet - good specification for implementation work?

    Okay, how long should this requirement gathering take? Let's calculate on man-hours. Couple? a day? several days? even weeks? quarters? a year?

    1. Let's put it this way. I have worked in software business for the last twelve years or so and during this time I've been in quite a few development project. Some of them were successful, others less so. Few were downright failures.

      One of the main reasons when a project went wrong was that the development team did not really know what the work was about: the requirements were unclear or in most cases there were no requirements at all.

      Instructions might have been along the lines "the system needs to send emails" and sure enough during the development it was shown that emails are sent alright. But when the system was taken to production it failed because it could not handle the amount of emails that was needed. So the developers had done what had been asked, it just wasn't what was needed.

      This is such an unnecessary reason to fail when a reasonably small investment in time and effort (and money) could save many times more time, effort (and money) down the road. So I made a career decision to focus on design and planning side of software development. I want to understand better what each project was about: who would use it, why they would use it, for what they would use it, how it should benefit the organisation, how it should integrate with other systems and so on...

      So doing at least proper requirements specifications and identifying risks will without a doubt significantly improve a project's chances of producing something that actually works as intended. It also helps to keep the customer happy, which is also nice. If possible one should also work on Use Case design as it helps to keep functionalities consistent and simplifies development as does few other things.

      Requirements and needs that drive development projects do not materialise out of thin air: they originate from the business. Each and every project must benefit the business in some way or another in order to justify necessary investments. This is why I'm now interested in Enterprise Architecture methodologies such as TOGAF as with these the focus is on the business as a whole.

      Finally to answer your question about how long requirement gathering should take, the simple answer is "I don't know". It varies from case to case. Sometimes I've got all necessary information in a two short meetings after which I needed few days to get the documentation ready. In some other cases it has taken few weeks or even longer.

      The scope is agreed with the customer at the beginning of the project. It is important to keep things simple and avoid unnecessary details: if it is not specific to the project at hand, skip it. On average I would say a requirements specification project takes about 2-3 weeks, but again depending on the case it might take more or less time. One just needs to make sure to get the job done within agreed time period :)

  5. Congratulations, you've just re-discovered the waterfall methodology!

    1. If you like to think it that way, but I think you might have missed the point.

      Some might point out that the classic Waterfall flow is Requirements -> Design -> Implementation -> Verification -> Maintenance. If having steps in logical order is equal to the Waterfall then by all means, but I think there is more than that to the Waterfall methodology :)

      In any case, many seem think Waterfall as evil incarnate but I see it as an approach among many: the needs of a project should determine which approach to take instead of trying to do everything with a "one trick works for everything" mentality. For example, Scrum probably would not be the best approach for designing and building a commercial aeroplane, agree?

      Oh, it crashed and burned because the wings fell off? Oh well, we'll fix it in the next iteration...

  6. The problem with model you're describing is the effort it takes. Sure requirements should be gathered but there's no sane human being on a planet who can read through these 200+ pages of requirements/use-cases and at the same time have firm understanding of the big picture.

    That's also why agile methods and tools like cucumber (for java, grails, ror) are so excellent. This hard-core requirements gathering you're describing works only if you can gather all the requirements up-front, and they won't change much. Both are cases which very very very very rarely happen in software project.

    And being agile doesn't rule out more traditional industries - read W. Edwards Deming theories. Although related to quality and manufacturing many of them are very agile. And who considers traditional manufacturing as being agile?

    As said earlier - these kind of processes sound nice but usually suck big time on practice. This is also what agile manifesto says 'Individuals and interactions over processes and tools' and 'working software over comprehensive documentation'.

    You claim to have several years of software experience as well as architectural experience. How many successfull projects have you honestly been part of where you get 100+ page document specifying requirements and additional documents detailing use cases in a level you described in earlier posts?

    1. A very good comment, thank you for that.

      The thing is, I never talked about writing hundreds of pages of documentation :) I have worked in agile projects (mainly Scrum) where the planning and designing has been incorporated with the agile development and it does work really well although it sometimes does take a little extra effort to stay a step or two ahead of the developers.

      The agile and lean principles apply even when making a point of identifying and specifying requirements as I talked about in this post. The point is not to try and identify everything at once, as new requirements are inevitably discovered during the development. But when a requirement is discovered it is good to study it: like in Scrum a customer comes up with a feature request that is essentially an epic, but which through little extra thinking is turned into handful of user stories.

      One of main reasons - in my experience - for changing requirements is the lack of initial understanding what is actually needed. So instead of coming up with few obscure requirements that are sure to change during development, or going into too much detail with 100+ page requirements documentation, one needs to find the middle ground.

      I need to write a proper blog entry about this soon as I know people tend to misunderstand this thinking it's something awfully heavy and extensive, or the opposite, not really doing it at all.

      Mind you, I'm not trying to describe or invent any methodologies or approaches of my own. I'm just writing based on my own experiences and sorting out my thoughts.

    2. You didn't talk - but explaining use cases like this: http://rambling-finn.blogspot.com/2010/08/thoughts-about-writing-use-cases-and.html makes even trivial system seem far too complex. Take even moderately complex system like.....simple document management with some approvals, document lifecycles, archiving etc. So very basic, implement your model on that and you are guaranteed to have easily 100-200 pages of requirements, use cases and some such non-sense. Then add in couple architects thinking software, network architect and database architect. Suddenly you are talking project with 100k-200k euros.

      And this is only the software side. I guess it would be great to include bunch of non-functional requirements? Perhaps even some hype.... federated identity supporting moden IdM systems, Web Services (add some cryptic standards, SOAP messages and what not), tiered architecture with DAO implementation (hibernate, ejb etc) with dependency injection leveraging (spring, guice, whatever) with pluggable service-interfaces that enables synergy in modern SOA-environment?

      As stated before - what you are writing may sound good on paper but can easily lead to CCCP type of mammoth. Instead of pouring these software development 101 into your writings try to squeeze real life experiences here. What has worked for you - what hasn't. Reflect those on existing methodologies.

      You mention TOGAF... great, it's just one more giant sinking into swamp of other such frameworks. I mean... why not take OBASHI, IDABC, DoDAF, Rational method, MODAF....

      There's a claim that you do not think heavy-weight solutions but everything you write points the other way. Reading these posts gives immediate feeling of traditional, waterfall, heavy-weight project. Do you really think all this stuff is neccessary for good enterprise architecture? Please write some posts also about agile projects, light-weight solutions, KISS (keep it.... and so on. You do know the acronym?)

      Many developers have turned from the dark path back into the light. It's not too late - you can still do the right thing.

    3. As said, I'll write a separate blog entry about this but suffice to say that I understand plannig ahead is not everybody's cup of tea.

      You seem to assume that I'm writing about something that I do not have any practical experience and I can understand that too. I have not made an effort write extensively or even consistently about the work I do so I can't expect people to get the right idea just by reading these few entries that I have written over the last few years. That's alright.

      That being said, I have met people who talk just like you. Worked with them, too. In most cases all forms of planning and design was to be shunned as it was not agile but just some pointless waste of time. Unfortunately I often ended up having similar thoughts about our software after a while when the implementation began losing consistency and clarity after repeated direction changes as work was planned only one or two sprints ahead.

      The quality was poor as without measurable requirements we could not prove by testing that the software was truly production ready. True enough stuff seemed to work in dev environment, but that has very little to do with real life. Without consistent documentation (if any existed) bringing up new people took extra time as they often had to figure out things by the hard way unless somebody took the time to work as a mentor for them in addition to their own work.

      I could continue but I think you might get the idea. So I started doing things a little differently, trying to find a place somewhere between "too much" and "not enough".

      thinking about the same thought of the same about the software we worked on

  7. Oh no... Now you've misunderstand me.

    I'm by no means saying that planning ahead is bad. I'm trying to point out that the way you are describing it sounds awfully heavy.

    I assume you are familiar with TDD and BDD? Let's say - cucumber, steak for example. Now, let's return to your example as use case. Have you explored for example Pivotal Tracker? Take a tool, write user stories, get those accepted by the client.

    Now you've established not only requirements but also verified the process with customer in almost plain english. I can quarantee that 'domain experts' ie the users prefer talking with 'Given .... Then... should...' format that user stories represent instead of writing 'actor: user. System does. Actor sees this. Exception: blaa. Comments: This is use case for login'.

    If needed throw in light-weight balsmiq mockups from UI. Client says Ok - there's your spec.

    I'm not saying 'no, don't plan - that's not agile'! What I'm saying is - do not drown yourself. Quickly write follow-up. Explain yourself. Anyone reading your blog is afraid that's you end up drowning on useless specs/stories/documents!