The Department of Health and Human Services (HHS) Enterprise Performance Life Cycle (EPLC) is a framework to enhance Information Technology (IT) governance through rigorous application of sound investment and project management principles, and industry best practices. The EPLC provides the context for the governance process and describes interdependencies between its project management, investment management, and capital planning components. The EPLC framework establishes an environment in which HHS IT investments and projects consistently achieve successful outcomes that align with Department and Operating Division goals and objectives.

The Enterprise Performance Life Cycle (EPLC) Framework defines IV&V as a rigorous independent process that evaluates the correctness and quality of the project's business product to ensure that it is being developed in accordance with customer requirements and is well-engineered. It recognizes that IV&V partnerships provide high value to many projects and may be introduced at any Phase of a project as determined by the project's sponsorship and/or Operating Division's governance requirements.

EPLC requires the development of an IV&V plan early in the project's life, as a part of the Project Management Plan (PMP), and an IV&V Assessment Project Review at the conclusion of the Development Phase. Depending on project size, risk and other factors, the IT Governance organization should determine the appropriate IV&V activities and may approve tailoring the EPLC IV&V requirements to match the project requirements.
 

IV&V should be performed by parties not directly engaged in the development of the project with the purpose of assessing the correctness and quality of a project's product. Typically IV&V reviews, analyzes, evaluates, inspects, and tests the project's product and processes. This analysis includes the operational environment, hardware, software, interfacing applications, documentation, operators, and users to ensure that the product is well-engineered, and is being developed in accordance with customer requirements.

IV&V provides management with an independent perspective on project activities and promotes early detection of project/product variances. This allows the project to implement corrective actions to bring the project back in-line with agreed-upon expectations. Objectives of performing IV&V include:

• Facilitate early detection and correction of cost and schedule variances
• Enhance management insight into process and product risk
• Support project life cycle processes to ensure compliance with regulatory, performance, schedule, and budget requirements
• Validate the project's product and processes to ensure compliance with defined requirements

IV&V findings and reports provide supporting evidence that the product does satisfy client requirements. IV&V should be performed throughout the project's life and can be executed incrementally at specific points in the life cycle or be performed in a manner that is integrated into all project efforts. Although costs increase, IV&V is most effective when integrated into the entire project life cycle, conducted in parallel with the project and product development activities.

IV&V stands for:

• Independent – assessments are performed by an independent third party
• Verification – verifies that the product is well engineered
• Validation – validates that the product conforms with client requirements

Independent
Maintaining independence of the verification and validation process is an essential element of the IV&V process. The Institute of Electrical and Electronics Engineers standard for Software Verification and Validation (IEEE Std 1012 - 2004) defines independence in IV&V using three main parameters:

• Technical independence is achieved by IV&V practitioners who use their expertise to assess development processes and products independent of those performing the development
• Managerial independence requires responsibility for the IV&V effort to be vested in an organization separate from that responsible for building and/or implementing the project's product. The IV&V effort independently selects the segments of the product to analyze and test, chooses the appropriate technique(s), defines the schedule of activities, and selects any specific issues to act upon
• Financial independence requires that the IV&V effort be funded from a general & administrative expense account in order to insulate the IV&V team from any potential financial pressures

Verification - “Are we building the product right?”
Verification is a quality control technique that is used to evaluate the system or its components to determine whether or not the project's products satisfy defined requirements. During verification, the project's processes are reviewed and examined by members of the IV&V team with the goal of preventing omissions, spotting problems, and ensuring the product is being developed correctly. Some Verification activities may include items such as:

• Verification of requirement against defined specifications
• Verification of design against defined specifications
• Verification of product code against defined standards
• Verification of terms, conditions, payment, etc., against contracts

Some verification techniques may include static testing approaches that check the sanity of code, algorithms used, documentation, etc., of the product with the primary concern of verifying and ensuring proper use of syntax throughout the project's products. Some approaches that may be applied to execute static testing techniques could include:

• Code reviews – Systematic examination of the product's source code
• Inspections – Peer review of work products and documentation
• Walkthroughs – Inspecting source code by following logical paths through the algorithms or code as determined by input conditions and choices made along the way

Validation - “Are we building the right product?”
Validation is the process of establishing documented evidence that provides a high degree of assurance that a product, service, or system accomplishes its intended requirements, and validate that the product being developed does what the user is expecting it to do. This is facilitated by validating that requirements are adequately defined, designs and functionality conform to requirements, data is treated correctly, and that test results are accurate. Some validation techniques may include dynamic testing approaches that test by examining the product's physical response to changing variables. This type of testing helps ensure the product's output is as expected. Some phases of dynamic testing techniques may include:

• Unit testing – Validates that individual units of product are working as designed
• Integration testing – Units of product are combined and tested as a group
• Function testing - Involves validating product functionality against defined requirements
• System testing – Testing of both hardware and software on a completely integrated system
• User acceptance testing – Black-box testing of product functionality to obtain release acceptance

Planning and obtaining IV&V services should begin early in the project's life. The Project Sponsor and governance entities should consider IV&V activities depending on project size, risk and other factors, and select those appropriate to match the project requirements. A list of potential IV&V activities for consideration is available in the IV&V Activities Job Aid. At a high level, IV&V activities include items such as:

• Determining the types and levels of product integrity to be verified and validated
• Developing performance metrics to allow tracking of project completion against defined milestones
• Identifying an integrity schema to measure the project's conformity to requirements
• Planning and scheduling of IV&V activities considering the project management plan and schedule
• Creating a verification and validation plan for the specific IV&V effort
• Consulting with stakeholders to assess their involvement and buy-in regarding system functionality and the system's ability to meet their needs
• Reviewing and providing recommendations to improve both the management and technical aspects of the project including evaluating project progresses, resources, budgets, schedules, and reporting
• Reviewing and analyzing project management and software development activities, performance, and operational policies, processes, documentation, and products for accuracy and completeness
• Reviewing product architecture for feasibility, consistency, and adherence to related industry and HHS standards
• Reviewing traceability of product functions to original requirements
• Documenting IV&V activities and assessment results in the form of task reports, activity summary reports, anomaly reports, test documents, and eventually a final IV&V summary report

When performing IV&V activities the effort and content of any documentation produced from it should answer questions such as:

• What is the current process, product, or technology?
• What is good about the current process, product, or technology?
• What about the current process, product, or technology needs improvement?
• What industry recognized standards is the project following?
• How is progress measured in the areas being evaluated?
• Is the project within the defined scope, schedule, and budget?
• Are project documents and related artifacts accurate and up-to-date?
• Is there adequate stakeholder involvement in the project?
• Are best practices and metrics employed to identify issues, progress, performance, etc?

• Document the IV&V effort and expected levels of commitment in a Statement of Work (SOW)
• Budget accordingly for IV&V activities accounting for project scope, schedule, risk, etc
• Select a qualified independent team to perform IV&V activities
• Identify IV&V acceptance criteria within the SOW
• Ensure that the IV&V team has accessibility to all required data and deliverables necessary to perform effective IV&V services
• Ensure that the IV&V team receives all project artifacts and is invited to all project meetings
• Maintain IV&V team independence
• Make Corrections based upon the recommendations of the IV&V team
• Incorporate IV&V findings into project lessons learned activities and associated documentation

Project & Governance Activities

• Conduct a preliminary assessment to determine the need and initial scope of IV&V services required by the project effort
• Use preliminary assessment results as input into developing the mechanism for obtaining the IV&V team
• Select an IV&V service provider
• Ensure the IV&V provider is familiarized with project processes, products, and environment variables
• Understand and accept the IV&V Management Plan developed by the selected IV&V provider that identifies the scope, depth, schedule, and resource requirements of the IV&V effort and includes the
  • Project processes and products to be included as part of the IV&V activities
  • Performance metrics which allow for the measurement and tracking of project progress against defined deliverables and milestones as they relate to the IV&V items being assessed
• Perform periodic reviews of, and brief stakeholders on, the progress of IV&V efforts
• Evaluate and implement corrective actions based upon IV&V findings/recommendations

IV&V Provider Activities

• Develop and obtain acceptance from the project sponsor, and other appropriate stakeholders, of an IV&V Management Plan for the IV&V project effort
• Conduct a preliminary assessment of project efforts
• Review and make recommendations related to policy and processes in areas such as: o
  • Project management
  • Product development
  • Project and product policies and standards
  • Project and product quality assurance
  • Project and product risk management
  • Project and product configuration management
  • Project and product requirements
  • Project and product security and capacity
• Review and make recommendations related to producing deliverables in areas such as:
  • System engineering assessment of requirements analysis, specification, and interface control
  • Operating environment assessment of system hardware and software
  • Data management assessment of data conversion, software, and database design
  • Development environment assessment of hardware and software used for development
  • Software architecture assessment of design specifications
  • Code and testing assessment of product code, unit test, integration test, system test, acceptance test, pilot test
• Analyze past project performance as an input into identifying and making recommendations as well as providing input into lessons learned for the project
• Provide assessment reports related to both the management and technical aspects of the project as they relate to the above bulleted items
• Provide a final IV&V report summarizing all assessment reports and recommendations prior to concluding IV&V activities
• Evaluate operations and maintenance procedures and any ongoing changes