forums.silverfrost.com

[JohnCampbell]

John S,

This could be such a broad topic.
I am interested in your description of working in a big stress office, where you may have attempted to define a standard approach to validation.
Where to begin with that ?
In my Structural FE work, most validation attempts I carried out were one-off's that looked as sub-system equilibrium checks or verification of stress fields. These were either using Excel calculations or graphical displays of localised calculated effects. There were rarely "valid results" to compare against. It would have been interesting to have had a defined reporting standard for reporting results.
What would have been the budget for the "defining the standard" project and was that project ever completed ?
In my operational modelling work, a correct answer was even more elusive !

Is that what you are implying ?

John C

[LitusSaxonicum]

This thread is my child, even if it has been adopted and developed by others.
Some definitions for the purpose of this reply are as follows. Validating computer software means checking that it correctly implements the procedure that it claims to implement, whereas Checking means confirming that the input data set properly represents the problem being analysed. Archiving means the retention of data input and matching output together with all checks on it, and if necessary, some record of the software being used and the certification that the software has been validated.
Validation: it appears to me that validation is the job of the originator of the software, but nevertheless it is wise for the user to satisfy himself that for his range of problems the validation is adequate. Mostly, in engineering, validation is done by running through test problems for which the solution is known. The validation stage pretty much tests the hardware and the compiler as well as the program application itself. Inevitably, there will be some cases that are not solved adequately. One can cope with this if the documentation is adequate and warns against the cases that do not run properly.
Checking seems to me to be the responsibility of the user, although this may be delegated to an independent checker. There are multiple stages for this, for example if one is using software in the design of an earth dam, it is not only the computer analysis that needs to be checked - for example the geometry and internals zonation of the dam is correctly represented, but also the material parameters for each zone have been correctly determined by laboratory tests for example. People using standard materials like steel and concrete, or aluminium, can rely on specifying the properties that need to be achieved and indeed they are probably better known than stuff delivered by the lorry load and compacted on site.
The worst ever example of checking that I ever saw was when the checker bought the same model computer (Amstrad, no less), the same software, obtained copies of the input data from the first user on floppy disk, ran the same program on the same model computer and set a junior the job of checking that the outputs were the same!
Archiving is a completely different matter. One may need to archive the outputs, the inputs, or occasionally even the software! I have mostly disappointing experiences in this.
First of all, there is the issue of the media used to the archive. Paper printouts are all very well, but particularly with sizes that don�t go in filing cabinets or which are too voluminous there is always the risk that the paper is �recycled�. At least for a while it is human readable. It has the further advantage that it can be annotated by hand. Digital storage media need devices to read them, and sometimes the drives cease to be available. Floppy disks, tapes, ZIP drives � where are they now?
Secondly, unless the input data is printed out or kept to match the outputs, the results may be largely meaningless. In the beginning this might have meant labelled decks of cards and printout kept in binders, but today it�s likely to require a naming system for files.
Sometimes, one needs to follow a �development of ideas� process, and this needs either a report or many saved analyses. All too often one only sees the final result.
Some of my experience is about tracking down what was done in support of reporting on potential negligence. I have seen information shredded, lost on the hard drives of obsolete computers, stored on media which cannot be read, filed with incomprehensible filenames or only partly preserved.
A particular worry is when the current version of an application won�t read older version files, or when it does, then produces a different result. The former is the case with my drawing application CorelDRAW! and also with Microsoft Word which will no longer read WordPerfect files.
For decades I have believed that there will always be a Fortran* (strike up the music: https://www.youtube.com/watch?v=gWSjNtbT4lg ) compiler ...

[LitusSaxonicum]

... on whatever computer I have, and mainly for engineering applications I only use software I have developed myself, so I could always recompile and redo the analyses, getting the same or similar answer and understanding why. However, a recent post shows that low precision can dog this process. What�s more, making a feature** deleted � even in FTN95 � means that the program won�t be the same, notwithstanding that it isn�t the same once recompiled, even if the source code is.
There are many questions to which the answer is of interest. Sometimes there isn�t an answer, but the question must be asked regardless. I suspect that 'what is a good way of Archiving' falls into the latter category.
Eddie
*At one time I used the rather superior Algol-60 � now where�s that?
**Even ones that might only be used rarely, or consider to be rather stupid.

[LitusSaxonicum]

John_S, You've homed in on what I called 'validation', and the thing I gave least weight to, probably because most software firms have procedures for it. Even so, sometimes the published solutions are wrong, or the published theory is wrong. I've come across both.

What I called 'checking' is being sure that you have fed the right model into the analysis. In your Aerospace analogy, like feeding in Airbus 320 geometry for the wings on a 737, or perhaps fed in more or less the right wing but gave it the wrong angle of sweep. Or maybe have it steel Young's modulus instead of aluminium alloy. Or assumed it was hot when it was cold. Etc. This is far more common than you'd think, although my field isn't (often) aerospace. In my one experience of aero, the object had been 'analysed' in plane strain, and it was axisymmetric so what was 'obviously' compression was, in fact, in tension.

When I proposed the topic, I was thinking mainly about retention of results. I've acted as an Expert in litigation where the analyses were lost, destroyed (sometimes accidentally, but also deliberately), filed incomprehensibly when they could be found, or unreadable on unreadable or corrupted media. In aerospace terms, it may be possible to find the calculations for a Spitfire, or even a Comet, but I'll bet that you can't find them for most of the oldest jet passenger aircraft still in service ( https://www.telegraph.co.uk/travel/travel-truths/oldest-passenger-plane-still-in-service/ ) which do fall into the albeit early computer era.

Eddie