This Website is not fully compatible with Internet Explorer.
For a more complete and secure browsing experience please consider using Microsoft Edge, Firefox, or Chrome

Simulation Governance: Technical Requirements


Simulation Governance: Technical Requirements


There are substantial economic incentives to reduce reliance on physical testing and increase reliance on numerical simulation. Testing is expensive and time consuming and the results of physical experiments are tied to the specific conditions under which they were performed. Generalization of test results is possible through numerical simulation.

Testing and numerical simulation should be always coordinated under a properly formulated plan on how the results will be interpreted and generalized.

The goal of simulation governance is to ensure the reliability, repeatability and timeliness of computed information generated for supporting engineering decisions. Simulation governance is also concerned with optimal allocation of resources needed for supporting engineering decision-making processes.

The technical requirements of numerical simulation will be discussed from two perspectives: the application of design rules and the creation of design rules. Examples were presented.


  • Mr. Matthew Ladzinski, NAFEMS

Remarks on Simulation as a Strategic Capability

  • Dr. Keith Meintjes, CIMdata

Introductory Remarks on the Goals of Simulation Governance

  • Mr. David Rusk, NAVAIRSYSCOM

Main Presentation:

  • Dr. Barna Szabó, ESRD
  • Dr. Ricardo Actis, ESRD

Q & A Session






Note: The presentation and recording files are only available to NAFEMS members. To access the password for this webinar, login with your member credentials, and then click here.

Questions Submitted (Unanswered)

Q: Are there any softwares the assist in collecting the correlation data and looking for trends? Is this typically, developed internally?

  • Commercial statistical packages such as R, S+, Minitab, Matlab, etc. are typically used for test data reduction and statistical characterization. Other commercial packages are tailored for specific applications, such as ALTA for Accelerated Life Testing. DR
  • This is a process problem, developing confidence in your ability to do simulation and make decisions based on the results. KM

Q: Would development of automated FEA code such as P-Version (Dr. Szabo's Probe program) be really helpful if it is still used by engineers w/o necessary background?

  • Some of us remember when Rasna Mechanica (automated p-analysis) was supposed to solve the problem of use by amateurs. The person asking the question has to understand the engineering problem! That said, the issue of error estimation is like the weather: Everyone talks about it, nobody does anything. Methods are available, but rarely utilized. KM
  • In order to permit utilization of simulation by persons who are not experts in the use of simulation technology, particular applications (Smart Apps) have to be designed by experts with built-in safeguards. It is very good practice to introduce such applications in design workflows because they concurrently increase productivity and reliability. There are commercial implementations which incorporate automated solution verification, such as ESRD’s StressCheck, which is essential for the deployment Smart Apps. The technology base for such implementations has been in existence since the 1980s but has not been adopted by most FEA vendors. One important exercise of SimGov is the adoption of the best available simulation practices, which includes demanding from the CAE S/W tool providers the incorporation of objective measure of quality in their codes. BS/RA

Q: How do you deal with uncertainties in dimensions, material parameters, etc?

  • NAFEMS has a Working Group focused on Stochastic or Robust Design, which is an approach to deal with uncertainty. KM
  • Not all uncertainties are equally important. Uncertainties must be evaluated with respect to the quantities of interest (QoI). Parameterized sensitivity studies provide information on which kind of uncertainty has substantial influence on the QoI. BS/RA

Q: Do you know if there exists some simulation governance for Computational Fluid Dynamics (multiphysics, granular flow)?

  • Academic studies on CFD error estimation and convergence have been published. I believe they are rarely used. KM
  • I do not know of simulation governance implemented for CFD, granular flow and similar applications, however very good work is being done at Sandia in verification and validation through utilization of the SIERRA code, developed under the DOE Advanced Scientific Computing program. For example, a capability exists for the simulation of reacting particulate flows. BS/RA

Q: Presentations were mainly focused on simulation verification. How are you validating simulation without physical tests for those applications where validation is required?

  • Come and listen to the webinar on Verification and Validation. The government and NASA deal all the time with problems they cannot run test for: Weapons, Mars landers, etc. - KM
  • By definition, validation is the testing of the predictive performance of a mathematical model through comparison of predicted and observed data. Therefore one cannot perform a validation experiment without physical testing. Such testing may be done at the coupon level, progressing to subcomponents, components, sub-assemblies and assemblies. Of course, the costs of experimentation rapidly increase with complexity. It is a resource allocation problem for management to find a sensible balance between numerical simulation and experimentation. BS/RA

Q: It is possible in the future to simulate in some conditions where data are not available using simulation governance?

  • I think that proper attention to the formulation of the model and consideration of the uncertainties is a very good way to improve confidence even when no data are available. KM
  • Simulation always involves some measured data (such as, modulus of elasticity, Poisson’s ratio, yield strength, etc.) Although all measured data have some statistical dispersion, in applications of design rules the rules may fix the data (for example allowing to use their average values) and treating uncertainties through factors of safety. BS/RA

Q: Slide 24 is not correct with newer tools of Finite Elements such as COMSOL which do follow your first case "numerical" simulations since COMSOL Multiphysics (r) applies the general differential equations of physics and does not use not "element libraries" as older FEM programme indeed do ! This is important for the classification of the different FEM tools, nevertheless all other pertinent comments on potential errors of insufficient discretisation (mesh density) checks, wrong boundary conditions, etc., fully apply, also to the newer FEM tools. The man difference I see is the V&V part as once the Physics has been verified, it will apply correctly for other models, so long the model implementation remains coherent.

  • Slide 24 distinguishes between the quantities of interest F associated with the exact solution of a mathematical problem and their numerical approximation Fnum. This is essential for any numerical simulation tool independently of whether finite element or some other numerical method is used for obtaining approximate solutions.Mathematical models should be viewed as precisely formulated statements of ideas of physical reality. The same physical phenomena can be represented by different mathematical models. The goal of validation is to test the predictive performance of mathematical models by comparing predicted (F) and observed data (Fexp). The word verification applies to showing that | F ─ Fnum | is not large (solution verification). Therefore Physics cannot be verified, only the numerical solution of a well-posed mathematical problem can be verified. If the exact solution can be computed for the mathematical problem, then solution verification is not needed.

Q: On Slide 26, fourth bullet under Methodology, should it be “Fnum” (instead of just “F”), based on what was presented on slide 24?

  • No. F is correct because the uncertainties in the domain, material properties and boundary conditions affect F. And since Fnum is an approximation to F they also affect Fnum.