Problem Statement

Requirements are a special issue for Engineering management because it is where their relationship with the Business teams really gets tested. The problem domain determines the nature and stability of the requirements, which in turn influences the relationship and the processes used by both business and technical groups. In heavily constrained problem domains, such as highway construction, requirements are extremely detailed and stable, which led to the evolution of the traditional development processes. Software is often on the opposite extreme, with requirements which are comparatively vague and fluid. This has motivated the work toward agile processes, which attempt to change the ways that Business and Engineering teams work to accommodate this uncertainty.

We will go over some typical management problems that occur with respect to requirements definition. Then we will look at an ideal model for generating requirements and how to apply that in different settings.

Case Study: Wyse Technology

When I worked at Wyse Technology in the Systems Division we were constantly bombarded to support new operating systems, drivers, databases and applications on the range of Wyse hardware ranging from laptops to 8 processor machines. As a result, we supported approximately 8 different Operating systems on 6 – 7 different hardware platforms with all of the device driver and database development and testing that goes along with each new operating systems. The combination of supported platforms was enormous, on the other hand, there was no-one in Marketing or Sales for that matter who was accountable for the ROI on a per-platform basis. No one’s job was on the line if a particular platform failed to become profitable, as a result, Product Marketing wish lists strained engineering resources with requirements from poorly managed product configurations, thus de-focusing engineering from more profitable product combinations.

Management Problems

Sometimes requirements are described as a contract between the business teams and the technical teams. However requirements documents are seldom written with the detail of a legal document, and they don’t always require review or signatures to make them active, unless you’re dealing in a government context or have adopted strict software engineering methodologies like CMM levels3 – 5. Viewed in this context, a lot of the mis-understandings and problems with requirements come from the fact they attempt to instantiate the agreement between groups without the detail and formality of a legal document. With the relationship between groups is not strictly dictated by binding agreements, it is absolutely critical that an understanding be reached between the groups on the nature and scope of the requirements. Without ground rules in this area, either team is free to criticize the other team for not doing their part, and product development will proceed badly. It’s equally important that Product Management (typically the creator of the MRDs and Use cases) consider the requirements from all possible sources, including existing customers, prospects, engineering (technologies/refactoring), Sales Engineering, etc. In addition, there must be an open forum for the relative prioritization of these collective requirements that balances short term sales needs with long-term strategic goals. Please refer to the Customer Understanding chapter for additional information related to requirements management.

Larger Companies

Large corporations can have the problem that the requirements become too much like legal documents. In an attempt to legislate away any misunderstandings or disagreements, the requirements documents become nearly as formal as contracts, with extremely detailed sections and sign off at every stage. In high constrained problem domains, or when the problem is extremely well defined or involve many vendors and sub contractors, these detailed requirements documents can make sense. Some industries, such as biomedical devices and pharmaceutical products, or civil engineering can require both detailed requirements and detailed design documentation. However for industries and problem domains that are not heavily regulated and have short product lifetimes, this kind of requirements can kill innovation. These larger companies are less likely to adopt agile practices, such as a ‘living backlog list’ or 1 – N prioritized feature list. Also be aware of the squeaky wheel symptom. This is typically a more vocal/passionate advocate for a given product feature that disproportionately influences the priority of a given feature over more worthy features.

At a large multinational computer manufacturer, the requirements for all products were long and detailed. When they started to purchase OEM products for re-sale, they didn’t adjust their requirements. The requirements made it so difficult to approve the product that it couldn’t be released until past the normal end of life for a PC product. This meant the company was reselling the older generation of product, losing out to their own supplier, who could provide the current generation to the market. The requirements process was so broken that it even broke products that they didn’t develop at all.

Startup Problems

Another classic problem is when the business teams have no experience with requirements, which is common at startups. In those cases, their view of requirements as marching orders that business teams give technical teams, rather than as a joint effort to find the best tradeoff between business and technical issues. This problem is especially bad in the software industry because the work appears to be so easy to define. With hardware, manufacturing or regulated industries, such as biomedical devices, there are enough constraints in the problem domain to convince most business people that requirements require a joint effort. However in software, it can appear that the software directly implements the business vision. This can lead to business teams mistaking the GUI for the product, so that the requirements document become an attempt at a user interface design, from people without any training in the area.

Every startup has the problem of changing requirements. It’s typical that during a 4 to 6 month development effort new requirements come in from sales prospects or the existing customer base that are important for revenue. At ACME corporation where I worked in the late 90’s, a ‘high priority’ feature added by the CTO late in the development cycle of a 300+ person staff year release had the unintended affect of delaying the overall release by several weeks at the cost of investor confidence. In the past, these new requirements were simply added to the head of the list without any evaluation of the opportunity cost or ROI of other affected features. The ‘backlog list’ concept promoted in Scrum and Agile practices addresses many of these issues. the ‘backlog list’ is a living document that maintains a 1 to N prioritized list of all candidate features, typically driven by marketing but with input from Sales and Engineering. A new requirement ‘candidate’ is inserted at the proper ‘priority level’, based on ROI, effort and timing. The next release is easily defined as the top ‘x’ items on the backlog list. This provides instant visibility to the E-Staff of the trade-off decisions regarding ‘creeping elegance’. Assuming that each requirement line item has an associated high-level estimate of effort, the rough timing of the release can be easily calculated. At the same time, it’s important to document within the 1-N list what I call ‘horizontal goals’ and not just features. These include quality targets, time-to-market, interactivity, scalability, platform support, human factors, 3rd party application or tool dependency etc. many software releases have missed customer expectations by not looking at release/product requirements from all relevant perspectives.

It’s important to weed out of the MRD the ‘how’ statements and leave only the ‘what’ statements. Too often a somewhat tech savvy marketing manager will try and assert design level information in the MRD, when this is best left to the expert(s), the engineers in this case. In the end, potentially creative alternative implementations may not get considered.

The best requirements come from a partnership between the business and engineering teams, and should produce a documents that allow both sides latitude for creativity and innovation that leads to the mutually agreed upon goals. The business teams need to do their part in understanding the industry, the market, and what the customers want. The engineering teams need to focus on the technology challenges and production issues. Each side should be free to learn more about other areas and both sides need to engage in open and honest discussion. Executives need to do their part in insuring that requirements process is done effectively and with the right spirit of cooperation, because problems in this area can cause not only a product but an entire company to fail.

A challenge common to startups is where the requirements of a single big customer will command all the engineering resources in a release to the exclusion of all other requirements particularly those of smaller customers or those requested by sales for prospects. Our requirements management chapter should include some discussion on how to define a release project from a collection of (hopefully well articulated) requirements. Common practice is to identify the source of a requirement, e.g. client X or prospect Y or CEO/CTO. A release goal can then be set as, e.g. 60% existing customers, 20% new customers, 20% product vision (i.e. CEO/CTO request, so long term strategic product plans). When the release goal is specified in this manner it adds transparency to the the effects of scope creep. It also makes clear where customer facing parts of the company may need to engage in tactical customer management to support strategic goals for the business.

Startup Anecdote: Confusing Defining requirements with designing the GUI

Inexperienced business people often mistake defining requirements with designing the software, especially the GUI design. I was VPE at a startup that was designing a simple new feature for a desktop application. This feature was to display a list of names, which could be clicked for a detail view of each name. Engineering wanted requirements which would define what data would be in each screen, and how to deal with missing data, however the business team felt they should design the two screens.Engineering would build prototypes which were reviewed by different subsets of the business managers, who would often disagree and reverse the decision of whoever wasn’t present. While business managers enjoyed their feeling of control, the Engineering team felt frustrated and demeaned. Worst, the GUI discussion was so distracting that none of the more subtle issues, such as problems of data quality were ever addressed. GUI ideas were proposed which were technically impossible and time was spent “proving” this to the business managers. In the end, it took nearly 2 months and over 25 person-hours of meetings before the business team felt they had given their “requirements”, and Engineering decided the business rules on it’s own.

Requirements Management Infrastructure

A good requirements management process must have proper support infrastructure. I’ve never seen a good requirements management process based on just Microsoft Word, Excel, or even a wiki. It must support the ability to slice and dice the requirements repository by priority, customer, product area, degree of difficulty / level of effort estimate, time frame desired, etc. Wiki’s don’t do this. Word can’t do this. Excel isn’t centrally sharable, or guaranteed current.

• A good RMS must be
• centrally sharable,
• allow slice and dice,
• always be current and authoritative,
• support what-if scenarios for release planning,
• be able to carry associations
  • between customers and application scenarios,
  • between application scenarios and use cases,
  • between use cases and user stories
  • between user stories and implementation tasks
  • between user stories and acceptance criteria
  • between user stories and acceptance tests

Requirements Management for Visibility and Traceability

Overview

Motivation: Many companies specify products with PRD’s (or just MRD’s), and expect engineers to respond directly to these documents, or spreadsheet forms of them. These organizations and processes implicitly assume that engineers do not require detailed problem understanding. In practice, this leads to engineering tradeoffs that are frequently non-optimal, as PRD’s (and product managers) cannot anticipate every possible issue the engineer may encounter.

Problem: The development engineering team makes best-possible guesses when issues and tradeoffs are encountered. These guesses aren’t based in customer problem context, because such problem context often doesn’t exist. This paper describes a lightweight process that allows an organization (including its product management staff) to capture customer problem context, and to use this context to form the basis of product decision-making from high-level strategic direction to detailed feature specifications.

Approach: We (re-)introduces an artifact (the “application scenario”) which is a simple way to capture the job function or responsibility area of a customer stakeholder. Using a corpus of many such application scenarios allows a product planning / management team to readily identify the specific target user / market that a product is intended to serve. (Even more, it is readily apparent what markets and users are *not* the intended customers for a given product.) Each application scenario motivates a set of use cases. Each use case motivates functional and non-functional requirements. These artifacts are an epistemological foundation for designs, test plans, user documentation, and other product artifacts.

Results: Projects that adopt such user-modeling artifacts and use this understanding to systematically capture customer understanding provide plan and status visibility and traceability throughout product development. Moreover, capturing this information allows product roadmap benefit analysis with respect to specific customers or markets, since each engineering task is traceable to a specific customer need.

Conclusion: If an organization feels their product development team lacks sufficient end-user-context a/o user-problem understanding, or if customers indicate that specific product releases don’t quite meet their needs, the organization may want to consider adopting processes based on explicitly modeling customer problems and needs. (Such situations are more commonly found in product version 1.0, ie new products.) Application scenarios are very useful artifacts to capture such context.

Basic Concepts

Use Cases are a good organizing tool for requirements, but what collects use cases?

Application scenarios address a problem space.

Application scenarios motivate multiple use cases that together resolve the problem of the main stakeholder.

Application scenarios are derived from real people, with real problems, with real issues, in real organizations, with real budgets, with real decision authority.

Application scenarios are a good way to collect information about every stakeholder a company representative comes in contact with

Sales can characterize their contacts by application scenario.

Support can characterize their users by application scenario.

Each can review each others’ representations of application scenario, and fill in missing understanding .

Application scenarios drive use cases.

Core use cases are the basic necessary use cases in order to viably resolve the problem.

Nice to have use cases are those that address the 2nd order issues.

Here’s a requirements object model for information flow:

• Customer => Application Scenario
• Application Scenario => Use Case(s)
• Use Cases => Functional Requirements
• Use Cases => Non-Functional Requirements (and informed by Customer and Application Scenario)
• Requirements => Design (and informed by Application Scenario and Use Cases)
• Use Cases => Acceptance Tests
• Use Cases =>
• Requirements => Unit Tests
• Etc.

Application Scenarios

Application scenarios are described in Crossing the Chasm (around page 95). They are simply these 5 questions:

• Who is the primary stakeholder?
• What’s the problem?
• How does this person address the problem without us? What’s the typical day in the life of this person?
• How does this person address / resolve the problem using our product(s)? What’s the new day in the life of this person?
• What are the differences?

Discussion: When discussing the primary stakeholder, this must be a real person. Do not make up this individual. There are plenty of times we can use our imagination, but this is not one of them. Understanding real customers means you have to find one or more of them. (Prospects will do.) This is the most important part of the application scenario, so it typically will take 3 or more passes through the application scenario before you feel you’ve properly characterized the person’s:

• Skillset
• Education
• Likes / dislikes
• Responsibilities
• Collaborators
• Respected inner circle
• Budget
• Decision-making style
• Etc

The problem is the second most important part of the application scenario. If solving the problem isn’t going to get the individual a nice promotion or nice raise, or if failure to solve the problem doesn’t risk the person his/her job, the problem arguably isn’t important enough to motivate the individual to find a solution (the one you’re selling). The problem is more than a single task. The problem typically is the achievement of some goal, or maintaining some metric. The problem must clearly be important to both the individual ***and*** the organization where he/she is employed. There must be substantial pain to the individual and the organization that your product will eliminate.Thirdly, what are the methods employed currently to either address the problem, or partially address the problem? Why are the methods deficient? What resources are required to utilize the current methods? On what metrics are these methods falling short? Characterize all dimensions. Look at:

• Skills required
• Staff
• Time
• Budget
• Space
• Communications
• Approvals
• Additional supporting infrastructure
• What is being done
• Etc.
• What is not being done

Next, imagine the individual’s life and environment once our product is in place successfully addressing the problem. Note the same dimensions as described above. Describe the new situation.What should immediately apparent are the differences between the previous 2 points. Our product may have some disadvantages (short-term costs?), but should have an overwhelmingly large set of compelling advantages over the current situation. Our product could be eliminating staff, saving cost (over the long run), saving time, reducing required skill levels, etc. These are important advantages that will eventually work their way into our product positioning and messaging once our product is ready for prime time.

When writing application scenarios, reflecting on the other stakeholders that interact with the scenario’s central figure is a good way to generate other application scenarios. Clearly, every individual could be the subject of at least one application scenario, as typically an individual targeted by a high-tech product holds more than one area of responsibility. So not only can one document multiple scenarios per individual (it’s sometimes surprising how different scenario authors will have important and different interpretations of an individual’s responsibilities… hence the need to document the scenario author as part of the scenario production.By examining one scenario’s subject, and understanding the collaborators (direct reports, boss, peers, interdepartmental collaborators, etc), the set of application scenarios can grow nicely. It is important to realize that a purchaser of a system has very different product evaluation criteria than the user, or the user’s boss, or the CEO, etc. These stakeholder concerns are very important to capture, as what may be compelling product for one may raise new problems for another.Application scenarios are just a fancy way to describe how each individual in an organization could think about the customers and prospects they come in contact with every day. Asking these questions (in your own mind at least, if not in actual direct interaction) puts one in a conscious mental framework of learning about the business and the business’s impacts on its customers. Asking these questions needs to be a way of doing business for every employee of a company. Wanting to know this kind of information about customers should be a burning desire for every employee of an organization. What’s the potential if everyone in an organization had this level of understanding about the company’s customers? Now.. what do you do with application scenarios? They are not an end in themselves.

Another interesting aspect of application scenarios is: According to whom? Make sure the source of the application scenario and its story is well tracked and note how well the scenario writer knows the central subject of the scenario.

Use Cases and Application Scenarios

Application Scenarios describe an entire responsibility, or problem area, that must be well-managed by a stakeholder. For example, a CIO has a responsibility to provide 24×7 operations for all of an enterprise’s business-critical applications. This is many use cases, not just a single use case. Clearly, s/he must:

• Interact with other stakeholders
• Recruit administrators
• Acquire equipment
• License software
• Assign work
• etc.

Application scenarios are an organizing conceptual structure (“container”) for use cases. They relate use cases so that together, they form a product that delivers value to a specific application scenario.

A company accustomed to thinking in terms of application scenarios to understand its customers, prospects, partners, users, and all with whom it comes in contact, ends up with a rich knowledge base of many application scenarios. This enables many data-drive (evidence-driven, if you’re into Harvard-speak) decisions:

• Easily sort out which application scenarios are in the “sweet spot” of your intended or delivered application or upgrade.
• Identify key and compelling value propositions that will communicate your product’s advantages vs your competition.
• Based on understanding the aspects of the individuals described in the application scenarios, you can determine what media through which to advertise in order to reach these stakeholders (potential customers)
• Determine many operational and environmental guidelines, compatibility & interoperability requirements
• Determine what application scenarios are to be set aside for the next version, or minor release
• Make proper tradeoffs in the course of product development, because your engineers have a good understanding of the key stakeholder concerns and needs, as encoded in these application scenarios

Prioritization by Use Case

Use Cases are an important artifact introduced by Ivar Jacobson to organize and motivate requirements. In many organizations, independent requirements would be prioritized independently, based on degree of difficulty, availability of skillset for development, clarity of specifications, etc. However, what got lost in this practice is the realization that many requirements are inter-dependent. IE in order to perform a function (use case), it may be necessary for 14 requirements all to be implemented. If 2 of the critical requirements (our of these 14) are omitted, the use case wouldn’t be functional, and the product may as well not have the other 12. So time spent development those 12 would be wasted if the critical 2 were left out. Without explicitly encoding these relationships among these requirements, the team loses such understanding of these prioritization tradeoffs, and then delivers a non-optimal product, because the effort spent on those 12 requirements could have been spent implementing the complete form of another smaller use case.

So the summary recommendation: Prioritize by use case, not by requirement. It’s more effective, easier, faster, and results in better product.

Working With Application Scenarios

Once you have the application scenarios defined, a clear algorithm emerges for decision making.

• For every stakeholder you come across, seek to understand their problem to be solved, their upside and downside for solving / not solving the problem, and their particular constraints
• Create “Application Scenarios” (per Chasm) that simply and effectively capture a key stakeholder’s interest
  • See Application Scenario Template for details of what an application scenario looks like
• Use application scenarios to understand your serviceable market
  • Weed out those specific AS’s that don’t represent the sweet spot of your product offering
    • Examine these for possible follow-on product offerings, if the immediate product succeeds
  • Focus on those specific AS’s that are smack in the center of your product sweet spot
    • Examine these for characteristics of stakeholders that assist in determination of your market
      • How many such stakeholders are there?
      • How much do they wield? (budget authority, decision making authority, staff influence, executive influence, etc)
      • What are their key decision criteria?
      • Whom else must they convince?
      • What must they avoid at all cost?
      • How is your solution going to get them their next raise or promotion? If this isn’t the case, it’s likely too weak to be actionable
• Use the surviving AS’s to derive a set of use cases (UCs) that together address the AS they are motivated from
• Use the Use Cases to drive a set of user stories (if you are doing agile)
• Use the user stories to drive sprints / iterations
• Use the application scenario to drive a set a NFR’s (non-functional requirements)
• Use the application scenarios to refine your understanding of each stakeholder at each contact point
  • Every individual in the company who encounters a stakeholder (customer, prospect, ex-customer, lost prospect, etc) should author an AS
  • The production of AS’s should become part of the way a company does business… it’s just the way you think about the market you serve
  • Examine each AS to learn about the stakeholder’s
    • budget
    • decision authority
    • manager’s concerns / key decision criteria
    • upside for success
    • downside for failure
    • ability to create options
  • Other side-effects that come out of these ASs are:
    • What materials / newsletters / magazines / periodicals / reviews does this stakeholder read? hold credible? disbelieve? scoff at?
    • What education level has this stakeholder achieved?
    • What are the deployment constraints imposed on a potential solution, in this stakeholder’s environment?
    • What other systems / actors must the solution interact with?
    • What are the stated success factors?
    • What are the implicit / hidden success factors?
• Use the use cases and application scenarios to drive user documentation
• Use the use cases and user stories to drive test planning

Customer Understanding and Application Scenarios

We can now return to the question of Customer Understanding and recast it in terms of application scenarios.

• In the best of all possible worlds, you have a customer-partner-program (more than just a CAB, which meets infrequently). A CPP meets at least once a month, for 2 hours, in a group setting. In this way, a CPP’s members can themselves share their application scenarios, specific stakeholder needs, most important features, usage concerns, etc. This is fertile ground for capturing application scenarios and their derivatives.
• Second choice, if one can’t create and maintain a CPP, is to have your customer touchpoints (sales, support, marketing, etc) all capture their specific customer interactions in Application Scenarios. They should get used to thinking of these five questions for every new individual they come across.
• Third choice, if one can’t actually get your sales and support people to create these application scenarios, is to regularly debrief them (at least once per week) and you (as product management role, or engineering management) will ask the basic questions that define Application Scenarios and try to document them, as a way to assist the field in retaining this kind of customer understanding.
• Fourth choice, if you don’t have the option to do any of the above, is to realize that you are going to make lots of tradeoffs in the “wrong” direction, and be comfortable with this reality. Don’t expect to make good decisions in the absence of good customer understanding, and then be surprised when your products miss the mark(et).

Note: This is an excerpt from “Leading and Managing in Silicon Valley“, a book co-authored by Sam Hahn and other VPEs.