It's probably not a surprise to you, but regardless of the software development lifecycle you use in your project, design plays a role. There are no SDLCs that avoid design, but it is the case that design is manifest in different ways. Design can be formal and elaborate, resulting in meetings of very expensive people and documents with the expense of people's signatures on them. I only have personal experience in the industrial world with a couple of variations of Waterfall and Agile. I will summarize my experience there and I will talk about the role of design in other SDLCs from a more academic point of view. Waterfall and its variants were the software development lifecycle of choice in government contracts for decades. The strict version of Waterfall specified that design wouldn't and couldn't happen until all requirements were complete. In some groups it appeared to be contractually illegal to work on design before the requirements phase was done. The US government Waterfall process had a lot of ceremony. So typically the end of the requirements phase was marked by meetings where the requirements were approved. These meetings would involve several days of presentations, collection of requirements, and finally a sign off. Once the representative of the customer and the representative of the development team signed off the requirements, the requirements became contractual obligations. Design teams were formed from often the same people who participated in the requirements process. The good part of this was that the designers knew the requirements pretty well. The drawback was that the design talent among the workers was inconsistent. Some had long experience using UML, Unified Modeling Language, and others had to learn it as they were going along. Similar to the requirements phase, the design phase was expected to occupy a certain amount of time and end with a set of meetings, presentations, and a couple of signatures. While the requirements were locked down, the designers quickly recognized that what was written in the requirements document wasn't what they meant to write. This, by the way, was before the domain specific language I talked about previously. Layers of editing, changed words, and meanings of requirements. There was a formal process to get the requirements changed. A person would submit a suggested change, a justification, and periodically a group of people would meet to approve or disapprove the changes. The problem with this process was that approving large numbers of changes would admit that the requirements weren't really ready to be signed off at that big meeting. Even if the change could be justified in terms of changing technology or some other external influence, the development leadership was reluctant to approve changes for fear of customer criticism. Interestingly, designers did what they thought they needed to do and in some cases they largely disregarded the requirements or created designs which were necessary but weren't based at all on any written requirement. What we're seeing here is the effect of rigidity on what happens during a design, something has to give. Of course the end product design, is something which tells the implementers what to do. In the organization where all the requirements people became all the design people, and all the design people became all the implementers, there are some advantages and drawbacks as mentioned before. The implementers would be intimately familiar with the design but their experience in implementing would vary widely. What this meant was that the designers recognized that whatever they wrote down as a design would only be a rough guide as to what to do. They knew the requirements, they knew what was intended. And so if the design was inaccurate, they could just correct that when they wrote the code. This meant that the design didn't have to be particularly rigorous. I will talk about a situation in a few minutes where the design had to be rigorous, but in this Waterfall project the cynicism had already set in. Ceremoniously signed off requirements were routinely disregarded or augmented in the design. There was no audit trail between a requirement and the resulting design. The way the design was validated against the requirements was to hire a third party to look at the requirements and decide whether or not the design was adequate. The result in this situation was a huge waste of money and effort, although the effort was contractually obliged. The third party could not absorb the sense of the requirements which had been worked for several years by the requirements slash design slash implementation team. The third party needed to come up with discrepancies to justify their employment but they focused on the most superficial and obvious of inconsistencies. These were often remedied with an explanation and occasionally with an actual fix. In the end, no one knew if the design was good. The designers thought it was and didn't care if it wasn't because they would be the implementers and the previous phases of the project demonstrated that if something was wrong they could just change it and not say anything. This sounds like a disaster in the making. It wasn't. It was about breaking the rules to get things done and it was about wasted time and money. In this rigid Waterfall project, design was a monolithic effort; bounded at one end by the requirements phase and at the other, by the beginning of the implementation phase. The clear and inflexible boundaries meant that the reality of changing requirements could not easily be accommodated. Inconsistent skills among designers meant that designs of varying quality were produced but this variation in quality was not detected by the group hired to verify that design. Can design be effectively implemented in a Waterfall process? Yes, absolutely but it's rare. Fortunately, we don't have to deal with that much anymore. An interesting variation on this was the design process I implemented when working for Briggs and Stratton. Yes, the gasoline engine company. They had outsourced software implementation of a business to business e-commerce product to Kirloskar, a large industrial conglomerate with a software shop in Bangalore India. The SDLC was a miniaturized version of Waterfall. A basic framework of the software existed but customer specific requirements had to be elicited, designs perhaps a dozen modules, would be created and then sent to Bangalore for implementation. The development cycle for a new customer was on the order of six months to a year, depending on complexity. The 11 and a half hour time difference between Colorado and Bangalore meant that if there were questions on the design documents, the implementers in Bangalore would call us on their lunch break which would be about midnight in Colorado. For this reason the the design documents needed to be as clear as possible. This meant that the design documents needed to be so detailed that the implementation process was almost deterministic, the same way that the same design would always produce the same product. This level of detail, while achievable and actually laudable, was expensive. The only way to verify the accuracy and completeness of the design document was to create a piece of software based on it. Often we found ourselves thinking that a few tweaks to the module that we created to verify our design would be the product that could be shipped rather than the one that was produced in India. This of course was not the fault of the coders in Bangalore. It was simply that to produce a near-perfect design document we pretty much had to code the result that we wanted. While this isn't the goal of the course, let me take a look at the business side of the situation. The reason why coding was offshore was because the coders in Bangalore were costing about $15 an hour and coders in Colorado were costing about $75 an hour. That's cost to the company, Briggs and Stratton. A lot of money was saved in the process, even with the designers duplicating the coding effort to make sure that the design was right. But what we had unknowingly created in Colorado was an Agile system before Agile was really a thought or a word. The very best people were kept on as designers, the development team size was reduced by about 80%, the handful of people doing designs were extremely capable coders. The designs were produced in cycles taking new customer requirements and turning them into designs in nearly usable software. An Agile-like process would have been only a step away if the decision were made to replace the offshore coding effort with incrementally more implementation effort at home to turn the verification modules into actual deliverables. But the step was never taken. Now let's take a look at design in the Agile Software development lifecycle. Agile is based on frequent delivery of value added working software. It can have delivery cycles as short as a few weeks and as long as three or four months, depending upon the complexity of the project, the number of people working on it, and the criticality of the software; how much damage could result if the software didn't work properly. Agile turns the Waterfall world on its head by emphasizing the delivery of useful software over creating documentation. And design is arguably documentation. So are we creating a software without a design step? No. Agile pretty much proves that design is necessary but the documentation of the design often isn't. Writers on Agile processes often invoke two levels of design. What we might call high level architectural design is called coarse grained design. Low level or detailed design is called fine grain design. Coarse grained design is often done in the early delivery cycles of a new project. These may be general screen design principles, so that the rest of the project has a uniform look and feel. They might be hardware specifications or infrastructure specifications on which the rest of the project depends. In order to accomplish these coarse grained designs, however, a knowledge of what the whole project might require is needed. This seems to fly in the face of the piecemeal cyclical development process where there are few new requirements developed into working software. The coarse grained design, however, needs a project long view of what's needed and likely the course grained design will have to be persisted in some kind of written form so that it's available to everyone throughout the length of a project. Course-grained or high level design often takes a lot of effort to create, but relatively little effort to document. For example, it may require many hours to research and decide on the look and feel aspect of the project's user interface, but it may not need more than a few hours to write up. Interestingly, fine grained design, the design of individual modules of code, probably doesn't require as much time to decide on what to do as it does to actually write up. This is why Agile looks at documentation on this level as expendable. Once the Agile project reaches a steady state of delivering software every so often, implementors are assigned user stories or a quanta, a package of functionality to implement. Because this functionality often represents increments above what is currently available, the design set is almost trivial especially for an experienced developer who has been on the project for a while. I worked on one project which was the development of a website for a research project in our university's Department of Psychology. At the beginning of the project there was a fair amount of effort spent in that high level or coarse grained design. Since there was more than one developer working on the project, there were a few documents that were created to describe the overall database and web server architecture. Interestingly, the screen design standards could be expressed in terms of a stylesheet and a general agreement to use the stylesheet. No documentation other than the stylesheet was really required. Requirements for the project were at first recorded on paper-based media, based on interviews with the researchers. Since an Agile approach was being used, a couple of very simple requirements were implemented and the website was put up. Seeing the site and what it looked like made the researchers think more carefully about what they wanted to do. Their ideas were put into a bug tracker and this became our requirements list. It was then a matter of selecting which bugs would be implemented in the next three weeks cycle. The bug tracker was used to record notes developer wanted to preserve about his or her implementation of the bug but that was the only documentation that was produced and it sufficed for the duration of the project. In short, design happened but mostly it wasn't documented. What we have described in this talk is how design can appear in different SDLCs. In the Waterfall SDLC, design can be firmly anchored between two other phases of the SDLC, and this may produce less than desirable results if the rigidity of the design phase doesn't allow for the reality of requirements, errors and changes. We talked about design in a mini Waterfall process used in offshore development. This design process produced a good product but more insight and better decision-making could have reduced the cost of the product. Finally, we looked at design in an Agile environment and how design always occurs but it isn't always documented.
