 |
Feature |
Agile Software Development Streamlines FDA-Regulated Applications
April 14, 2011By: Dan Olivier, Certified Compliance Solutions Inc. and Jeff Dere, Life Technologies
Find more content on:
Login or register to post comments
While the benefits can be many, a firm understanding of the process is required first.
There are many discussions today regarding the application of Agile software development methods for FDA-regulated medical devices and pharmaceutical products. However, before we examine the application of Agile to a regulated industry, it’s appropriate to define what’s meant by Agile methods. It’s also appropriate to discuss whether Agile is just another flavor-of-the- month software-development model that claimed to be a panacea for all historical software ills.
Before Agile can be dismissed as simply the latest fad in software-development models, the methods that comprise the Agile process must first be understood as they may deserve keeping, if only to augment other process models. The first step in understanding Agile is to provide some background information and then analyze it in the light of the regulatory requirements of design control as defined in the FDA’s Quality System Regulation (QSR).1
Processes that align with the “Agile Manifesto” and “Agile Principles” are considered to be Agile processes. Agile consists of a body of practices targeted to make software development more efficient by focusing on people, customer collaboration, being responsive to change, and working software.
The following is an example of some Agile methods:
- Iterative development: A process consisting of time-boxed iterations of fixed length.
- Iteration planning: Detailed planning of tasks that will be accomplished within an iteration.
- Daily stand-up: Also known as scrums, the daily meetings provide updates to team progress.
- User story workshops: Cross functional meetings are used to understand user needs and to refine specifications.
- Iteration demonstration: Represents the end of an iteration and the demonstration of discrete functionality.
- Integration of testing activities: Test cases are developed by testers in conjunction with each iteration.
- Integration retrospective: At the end of each iteration, a review of work completed is performed to evaluate progress and examine improvement opportunities.
- Continuous Integration: Continuous build and release of source code with each iteration.
- Release Planning: Development of a high-level plan that shows project milestones and individual iterations.
Iterative Development
Agile development is successful because the project is broken into smaller, more manageable pieces called iterations. The project is not seen as one long development cycle followed by a testing cycle. Instead, it’s viewed as small sets of incremental deliverables that lead to the project’s completion.
These smaller pieces are critically important to Agile’s success. Each iteration must be of equal length to enable normalizing of metrics and to provide constant sustainable pacing. Most Agile interactions will be three or four weeks longs, but some practitioners will use one- or two-week iterations.
Each iteration is comprised of tasks required to implement the specifications. Each task is estimated to be about one to two days of effort. The estimation of these smaller tasks introduces more management overhead, but that’s more than made up for by having more accurate estimations, more accountability, and more control of the schedule.
 |
| Figure 1: Shown here is an iteration plan and activities that occur within an iteration. |
Daily stand-ups are a management tool to evaluate the progress of each development team on a daily basis. This enables close tracking of project progress and timely corrective actions to be implemented if the team isn’t on track to achieve target goals within the defined timeframe. This also helps keep each of the development teams synchronized. These scrums are usually accompanied by a white board that tracks progress, assigned action items, and project areas of risk.
User Stories
These specification are often described as user stories or use cases, or a piece of functionality described from a user’s prospective. There will be one or many user stories per requirement. This is the true adoption of Agile—user stories may change throughout the product development lifecycle. User story workshops, with active participation of developers, testers, and customer representatives, help to drive out details on how a requirement should be implemented and tested. Each iteration begins by planning the user stories that will be implemented in that iteration and ends with a demonstration of this capability.
The iteration demonstration provides stakeholders the opportunity to provide feedback. Typical questions that may be raised include: Was it implemented as everyone expected? Does the user story still make sense? Are changes needed? This detailed review helps keep everyone on the same page and often results in changes to the implementation, user story, or test cases. This does result in rework but ultimately results in a better, more-useful system. Documentation changes usually aren’t too substantial and are easily managed if addressed immediately; just allocate time for it. The constant refactoring that results from this must be managed well. If done correctly, you’ll end up with a more robust product.
Integration of Test Activities
One of the greatest advantages of Agile is the integration of testing with development throughout the development cycle. The testing teams participate in the user story workshops, architecture reviews, and design reviews. They also ensure that user stories have acceptance criteria that can be tested and can start to create test cases. Testing is performed in conjunction with development. User stories can be tested immediately following implementation. This provides the development team with immediate feedback and uncovers defects early in the product development lifecycle. Most Agile teams use automated test tools to construct test libraries that can perform regression tests for future builds.
The iteration retrospective demonstration capability signifies the end of the iteration and involves the key stakeholders. This provides a good opportunity to review how the team performed during the iteration. Project and quality metrics should be presented and compared to past performance. In addition, there should be a discussion on what went well, what needs to be improved, and what needs to be maintained. In the project’s early phase, when the team is getting into rhythm, these meetings can take a while as people share ideas from other projects they’ve been on. In the later stages of a project, when the team has solidified, these meetings are quick.
Continuous Integration
Integration of developed code through the application build process should be done at least once per day, with most Agile practitioners doing many builds each day. It’s important to maintain builds. Since iterations are fairly short, developers can’t afford to waste three or four days getting a build to work. The automated build system should include automated engineering tests and automated user acceptance tests.
Agile Release plans are comprised of several of these iterations. The release plan also indicates key milestones, and predicts when releases will be available to use by other groups. It serves to link the software development activities to the system, hardware, and other development activities. It’s extremely difficult to have a predictable, well-run product development cycle without having a good release plan.
 |
| Figure 2: The burn chart is used to measure progress throughout the development cycle. |
Burn charts provide information to measure the progress through the release plan. A total number of story points is estimated at the beginning of the project and refined as more details are worked out or feature creep occurs. These story points are allocated over a number of iterations based on the team’s development speed, also known as the team velocity. The actual story points completed by development are calculated and compared to what was planned. The inclusion of the software testing team early in the process enables the capture of the number of story points completed. This is approval by the software testing team signifying that the features have been completed and have adequate quality. By constantly monitoring these elements throughout the product development lifecycle, the project manager can assure that the project is moving forward at the correct pace and quality level.
Adapting Agile for Medical Devices Regulations
Agile methods have proven to be effective for software development; the challenge is how to show how these processes can comply with the regulatory requirements for medical devices. Here’s one approach to integrate medical device compliant activities and specifications into Agile.
A list of candidate specifications that can be used to address the regulatory requirements of medical device design controls as specified in the FDA QSR and ISO 13485:20032 are shown in Table 1. Although FDA guidance documents such as the “General Principles of Software Validation”3 specifically state that the FDA “does not recommend any specific life cycle model,” the required specifications imply a sequence. As stated in IEC 62304 (Medical device software – Software lifecycle processes), “it is easiest to describe the processes in this standard in a sequence, implying a ‘waterfall’ or ‘once-through’ life cycle”4. A summary depiction of such a Waterfall Model is depicted in Fig. 3. The superscript numbers associated with each of the documents/process activities identified in Fig. 3 correspond to the rows of the table that address the key design control elements. The specifications listed in the table and the process sequence shown in Figure 4 depict a traditional design control waterfall-type methodology as is followed by most medical device companies today.
|
|||||||||||||||||||||||||||||||||||
| Shown in the table are the candidate specifications that can be used to address the regulatory requirements of medical device design. |
 |
| Figure 3: The Waterfall software development model is depicted here. |
This traditional methodology, however, is not the only allowable approach for medical device software development. When an FDA reviewer was asked if it was permissible to develop medical device software using an Agile process, the reply was that it’s okay, but suggested that the company not tell the reviewer which lifecycle model was used. This is in line with the earlier reference to the “General Principles of Software Validation” statement that the FDA does not prescribe and “does not recommend any specific life cycle model.”
Next, we’ll map these activities and corresponding specifications to the Agile process described. An example of such a mapping is provided in Figure 5. This shows that the specifications required to support design-control compliance and the sequence of these activities can be overlaid on the Agile process model.
 |
| Figure 4: Agile’s mapping process is outlined in the figure. |
The sequence of the specifications is logical. The order is as follows: requirements before design, design before implementation, and implementation before test. In Agile, this sequence is preserved but is highly repetitive comprising multiple iterations.
In addition to developing specific documents and process activities, the medical device regulations require a safety risk analysis that drives added emphasis on the definition, design, implementation, and testing of safety-related requirements. It’s essential that any medical device software development process provide added emphasis on assuring the correctness and completeness of all requirements identified as related to safety. This emphasis must be added as part of an Agile process as well.
The attraction of an Agile process is that it addresses the weaknesses of the sequential development models. Agile advantages include:
- Applications that are more closely aligned with user needs because of frequent user demonstrations and feedback.
- Improve productivity because of the iterative development methods that focus on the completion of smaller more manageable work units throughout the development process.
- Fewer defects because of continuous build and test processes.
- Facilitates integration of changes because the process is iterative and flexible with respect to previous design decisions.
Even though there are success stories with sequential software development methods, there are also many with a less than stellar record. Many in the software industry are familiar with software projects that have followed a “sequential development” process that has had disastrous results (considerably over budget and over planned schedules) in spite of having followed the logical nature of these processes. Problems experienced by medical device software applications that have followed sequential software development process models include:
- Software defects that take hours to correct but require months of effort to complete the required testing and associated documentation.
- Software with hundreds of pages of review records that don’t find defects, yet include software that has significant quality problems found during formal testing and after release to clients.
- Software projects with significant documentation that may be able to pass compliance audits but have very poor customer satisfaction.
In addition, several studies have shown that sequential software development models in many cases implement functionality that’s not ultimately used by their customers. This is shown in a chart from the “Chaos Report,” prepared by the Standish Group (see Fig. 5) that shows when sequential software processes are used a significant portion of features planned, designed, implemented, and tested are often never used by customers.5 This is a consequence of not receiving frequent user feedback during development as a key element of Agile processes.
 |
| Figure 5. The chart shows the average percent of functionality that’s used when a sequential approach to software development is followed. |
It’s clear that just because a sequential development process closely aligns with design control phases, it doesn’t necessarily guarantee high-quality applications. Alternative process models such as Agile need to be considered. Quality of developed software can’t be measured by the volume of documentation produced but must include measures of customer satisfaction, level of defects, development schedule, and ability to support regulatory requirements for compliance. We believe data in this article supports use of Agile processes as a means to satisfy each of these objectives.
It is clear that software is continually becoming a more significant part of medical devices today and that this trend will continue in the future. As a result, excellence in software development practices can be a key to successful medical device manufacturing.
Agile can’t be considered just a passing fad, as we are now celebrating ten years of Agile and it continues to grow and evolve. The FDA is also participating in an American Association of Medical Instrumentation (AAMI) working group, defining a Technical Information Report (TIR) on the use of Agile methods for medical devices. Tailored versions of Agile are being increasingly embraced in the medical device industry.
Key conclusions from this article include:
- Agile methods provide an iterative development process that offers advantages over sequential development models in that feedback on early releases can be integrated into development iterations resulting in applications that more closely map to user needs.
- Specifications required for design control compliance can be integrated into Agile processes.
- Additional emphasis must be placed on assuring the mitigation of potential safety risks for any medical device development model.
- Agile processes have demonstrated the ability to be more efficient in the development of software applications than sequential software development models.
Many companies are embracing Agile processes to improve the quality of software applications and to help streamline development schedules. Successful companies that have adopted Agile have also dedicated resources to continuously refine their processes to assure that the high quality of software required for medical devices is continuously produced using these methods.
Dan Olivier is the president of Certified Compliance Solutions Inc. (San Diego, CA). Jeff Dere is the director of software technologies for Life Technologies (Arcadia, CA).
References
1. 21 CFR Part 820 - Quality System Regulation, US Federal Register, Food and D General Principles of Software Validation, Final, Food and Drug Administration, Center for Devices and Radiological Health, January 11, 2002, section 2 Scope, page 1.
2. ISO 13485:2003, Medical devices - Quality management systems – Requirements for regulatory purposes, International Organization for Standardization, 2003-07-15.
3. General Principles of Software Validation, Final, Food and Drug Administration, Center for Devices and Radiological Health, January 11, 2002, section 2 Scope, page
4. IEC 62304 Medical device software – Software lifecycle processes, First Edition, International Electrotechnical Commission, 2006-05, page 73.
5. Chaos Report v.3 Standish Group, 2005–2006.
Featured Image:
Login or register to post comments
|
Most Recent |
|
Related articles |
Privacy Policy | Contact | Advertise | Subscribe | Sitemap
© 2012 UBM Canon
Agile is not mature for Medical device software
I agree Agile process reduces software development time and improves some aspects of the software quality attributes. Atleast it has a short term cost benefit advantage. But core Agile principles conflict with medical device software develpment. Medical device software has the following key attributes :
1. Longer product life time (10-15 years)
2. Safety critical( can harm patient or user)
3. Need architectural planning for future enhancement and safety issues
4. Design reviews are crucial for process effectiveness and maturity
5. Input and output of dvelopment activities and tasks shows proper planning, execution, monitoring and control
6. Certication is required both software and tools used for development
Traditional software development processes such as water fall and spiral model provide these indirectly. Agile idea is to think short term, do not invest too much into long term architecture and design, reviews are informal, documentation is to be minimum, no concept of risk management and safety assurance etc.
Without updating the Agile manifesto for these areas and using Agile as it may not be appropriate for any safety critical software development.
Agile, yes, but auditable
Good article Dan. It does contrast the Waterfall with Agile-style product development methodologies, but possibly does not emphasize enough that even though there are no firm methods specified in 62304 / 60601-1-4 / Third Edition, they all insist that the process be auditable and traceable. There is no clause that says, "If you are using Scrum, just tell me that everyone was happy walking out of the daily meeting and there was a periodic demonstration of something."
I was particularly interested in the section that followed "Problems experienced by medical device software applications that have followed sequential software development process" -- yes we can't have these issues hold up deliveries, but they all have to be "written down". We, as medical device development professionals have the legal and moral obligation of preparing these documents in a form that is unabtrusive and efficient. it would be easy if we really just had to follow the original manifesto whose words seem to say "No process, documentation, negotiation or plan", just hump code in an organized, customer-centric manner. Of course, it doesn't rule these out, just says that other qualities like working with customers and software that works is MORE important.
I don't mean to demean and am certain that you know all of this, I just felt that the requirement that the development process be brought up to date before the product ships should get equal billing. But yes, the challenge is getting great, safe, effective products to our medical professionals the quickest regulated way -- and that certainly means adopting the best development and planning techniques.