# Verification & Validation in Scientific Computing

2-Day On-Site Training Course

Engineering systems must increasingly rely on computational simulation for predicted performance, reliability, and safety. Computational analysts, designers, decision makers, and project managers who rely on simulation must have practical techniques and methods for assessing simulation credibility. This short course presents modern terminology and effective procedures for verification of numerical simulations, validation of mathematical models, and uncertainty quantification of nondeterministic simulations. The techniques presented in this course are applicable to a wide range of engineering and science applications, including fluid dynamics, heat transfer, solid mechanics, and structural dynamics. The mathematical models considered are given in terms of partial differential or integral equations, formulated as initial and boundary value problems. The computer codes that implement the mathematical models can use any type of numerical method (e.g., finite volume, finite element) and can be developed by commercial, corporate, government, or research organizations. A framework is provided for incorporating a wide range error and uncertainty sources identified during the modeling, verification, and validation processes with the goal of estimating the total prediction uncertainty of the simulation. While the focus of the course is on modeling and simulation, experimentalists will benefit from a detailed discussion of techniques for designing and conducting high quality validation experiments. Application examples are primarily taken from the fields of fluid dynamics and heat transfer, but the techniques and procedures apply to all application areas in engineering and science. The course closely follows the course instructors’ book, Verification and Validation in Scientific Computing, Cambridge University Press (2010).

Upon completion of this course, attendees will be able to:
• Define the objectives of verification, validation, and uncertainty quantification
• Implement procedures for code verification and software quality assurance
• Implement procedures for solution verification, i.e., numerical error estimation
• Plan and design validation experiments
• Understand procedures for model accuracy assessment
• Comprehend the concepts and procedures for non-deterministic simulation
• Identify sources of uncertainty, such as aleatory and epistemic uncertainties
• Recognize the goals of model parameter calibration/updating
• Interpret local and global sensitivity analyses
• Recognize the practical difficulties in implementing VVUQ technologies

## Who Should Attend

This course benefits model developers, computational analysts, code developers, software engineers, and experimentalists working with computational analysts. Managers directing simulation work and project engineers relying on computational simulations for decision-making will also find this course beneficial. The course will discuss the responsibilities of organizations and individuals serving in various positions where computational simulation software, mathematical models, and simulation results are produced. An undergraduate or advanced degree in engineering or the physical sciences is highly recommended. Training and experience in computational simulation of physical systems is also recommended.

The course is open to both members and non-members of NAFEMS.

## Course Materials Provided

Course attendees will be provided with a copy of the book Verification and Validation in Scientific Computing, Cambridge University Press (2010). The 780-page book provides a comprehensive and systematic development of the basic concepts, principles, and procedures for verification, validation, and uncertainty quantification for models and simulations. The book contains several examples of the most common procedures in VVUQ, including an example of the design and execution of a high quality validation experiment. Attendees will also be provided with an electronic (PDF) file and color print copies of over 270 short course slides presented during the course.

The course is completely code independent, attendees are welcome to bring laptops to take notes, but they are not required.

## Course Program

The contents are presented in eight lectures, tentatively organized as shown. The two-day schedule allows for ample discussion and interaction with the instructors and other attendees. The instructors reserve the right to modify the contents to address the audience’s needs and preferences.

### Day 1

Lecture 1. Introduction, Background, and Motivation

Lecture 2. Terminology and Fundamental Concepts

• Brief history of terminology
• Present definitions and interpretations
• Alternate definitions used by related communities
• Who should conduct verification, validation, and uncertainty quantification?

Lecture 3: Code Verification

• Software engineering
• Criteria and definitions–Order of accuracy
• Method of manufactured solutions

Lecture 4: Solution Verification

• Round-off error
• Iterative convergence
• Iterative error estimation
• Discretization error estimation
• Reliability of discretization error estimators
• Discretization error and uncertainty estimation

### Day 2

Lecture 5: Validation Experiments

• Validation fundamentals
• Validation experiment hierarchy
• Validation experiments vs. traditional experiments
• Six characteristics of validation experiments
• Detailed example of a wind tunnel validation experiment

Lecture 6: Model Accuracy Assessment

• What are validation metrics?
• Various approaches to validation metrics
• Recommended characteristics for validation metrics
• Identification of model discrepancy
• Cumulative distribution function approach

Lecture 7: Predictive Capability of Modeling and Simulation

• Identify all sources of uncertainty
• Characterize each source of uncertainty
• Estimate solution error in system responses of interest
• Estimate total uncertainty in system responses of interest
• Procedures for updating model parameters
• Types of sensitivity analysis

Lecture 8: Final Topics

• Planning and prioritization in modeling and simulation
• Maturity assessment of modeling and simulation
• Practical difficulties in implementing VVUQ technologies

## PSE Competencies addressed by this training

 V&V-SIMMsy9 Design a test for analysis validation purposes. V&V-SIMMsy8 Formulate a series of smaller studies, benchmarks or experimental tests in support of a simulation modelling strategy. V&V-SIMMsy7 Prepare a validation plan in support of a FEA study. V&V-SIMMkn6 State simulation V&V principles V&V-SIMMkn15 List relevant physical tests and their characteristics to calibrate or validate simuation. V&V-SIMMev8 Train engineering staff in validation techniques V&V-SIMMev7 Design appropriate verification and validation procedures in support of simulation. V&V-SIMMev11 Design test/analysis correlation processes, and select analysis validation criteria. V&V-SIMMev10 Assess model/analysis validity from test/analysis correlation studies V&V-SIMMco9 Explain the term model calibration. V&V-SIMMco8 Explain the term code verification. V&V-SIMMco7 Explain the term solution verification. V&V-SIMMco6 Explain the terms Verification and Validation. V&V-SIMMco32 Understand simulation error assessment methodologies and the concept of simulation predictive maturity. V&V-SIMMap6 Perform test /analysis correlation studies V&V-SIMMap5 Perform model calibration from tests V&V-SIMMap4 Perform basic model checks V&V-SIMMap3 Conduct validation studies in support of simulation. V&V-SIMMan7 Analyze test data to support validation activities V&V-SIMMan6 Analyze simulation results to support validation activities. PROBkn8 List types of uncertainty PROBkn5 List typical random sampling techniques. PROBkn3 List the characteristics of a typical probability distribution PROBkn10 List the benefits from probabilistic finite element analysis. PROBkn1 List typical sources of uncertainty in a reliability assessment PROBco9 Explain the relationship between the Normal Probability Density Function and the Cumulative Density Function. PROBco8 Explain how probabilistic sensitivities can be used to  guide product design. PROBco7 Describe how variability in an analysis input quantity  may be characterised. PROBco2 Describe the difference between epistemic and aleatoric uncertainty and how they can be quantified PROBco11 Describe Monte Carlo simulation. PROBco1 Explain the term non-deterministic. MG-SIMMsy15 Implement efficient versioning process for the simulation tools used by your company. MG-SIMMev9 Evaluate and benchmark external supplier validation approach MG-SIMMev14 Assess simulation solution maturity and readiness levels for a new project. MG-SIMMco33 Understand software versioning processes MG-SIMMco1 Understand the need and relevance of analysis specifications. MG-SIMMap25 Apply appropriate procedures for controlling the quality of simulation work MG-SIMMap18 Monitor tool (code) verification for the relevant project and intended use MESMev1 Select appropriate validation measures. MESMco9 Discuss the uncertainties typically present in analyses and explain how these are handled. FEAsy8 Prepare a validation plan in support of a FEA study. FEAkn4 Define the meaning of adaptive mesh refinement FEAkn13 State the word length or arithmetic precision of calculations for any system used. FEAev5 Manage verification and validation procedures in support of FEA. FEAco3 Explain the term solution residual. FEAco12 Outline a common method employed to solve the large sets of sparse symmetric common in FEA. CFDsy3 Formulate a plan to address the uncertainty in input data or modelling when using a CFD code for a design study. CFDkn7 List the main sources of error and uncertainty that may occur in a CFD calculation. CFDco12 Review the issues associated with the estimation of total uncertainty in a flow simulation.

### Interested in this Course?

Please complete this form if you are interested in scheduling an on-site training session, or if you would like to be notified of the next public course session.

# Course Tutors

## Professor Christopher Roy

### Events - Cancellation Policy

Please note NAFEMS cancellation policy for all training courses is as follows:-

• Suitably qualified delegates may be substituted at any time prior to the start of the course

• Bookings are accepted upon condition that either full payment is received before the course commences, or that a valid purchase order is received from a company that has a credit agreement with us

• In the event of a delegate being unable to attend a course that they have booked upon then, NAFEMS will discuss the possibility of transferring to an alternative course. However, a suitable administration charge will be levied.

• A refund of 50% of the course fees will be paid to delegates who cancel their booking and do not re-schedule to an alternative course, provided that the cancellation is received by NAFEMS at least four weeks in advance of the course. No refunds can be given for cancellations made after this time.

• NAFEMS reserves the right to cancel the course, without liability, in which case all training fees will be refunded in full. However, NAFEMS cannot be held liable for any other expenses incurred by participants or their companies due to the cancellation.

NAFEMS will discuss the possibility of transferring to an alternative event/course, however an administration charge will be applicable.

For full terms and conditions, click here. This policy is subject to change.