forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Computer Analyses Checking, Verification & Archiving

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Sun Mar 17, 2019 11:44 pm    Post subject: Computer Analyses Checking, Verification & Archiving Reply with quote

.... A Discussion
(as suggested by Eddie in his Fri 15th March 2019 comment here:
http://forums.silverfrost.com/viewtopic.php?t=3961 )

Eddie's suggestion for this discussion pleased me a lot.

I used to work for a bìg UK space company and in the stress office we had a great system which was put into place by the deputy chief of the stress office. I'm talking mid-eighties.

It was christened SOWI (Stress Office Working Instructions) and covered everything, from stress hand calcs and how they should be filed, to the filing of those majestic green fold-out computer outputs (boy did we generate a lot of paper!), through boxes and boxes of punched cards, to 'the Italian Job' magnetic tapes, kept in the computer room of course, but with specific instructions on how they were to be labelled, for reference.

The whole sytem was a pain in the arse at first for anyone arriving in the office, and the frst couple of weeks was spent getting to grips with it.

Move on 35 years and the situation is worse, what with the quantity of data wrecklessly generated, the multitude of storage formats appearing in the intervening period, and of course the arrival on the scene (in my industry in early 90's) of .... the dreaded PC ! Things were so much simpoler before - and time to think rather than headless-chicken-ly churning out results just for the sake of it.

EXCEL is a wonderful tool, probably one of the biggest marketing mistakes M$ ever made making it so good at the beginning (I still use v2003 - no on 'needs' anything more than included therein) ... or maybe it was just a consequence of the effort to rip the crown from the hands of the pioneer, LOTUS 1-2-3 .

Anyway, too much data, generated in a way that follows no standards (internal to whatever office) whatsoever.
Everyone does their own thing.

Everyone struggles when picking up someone else's sheets, the worst being someone who is no longer there !

It deserves a few chapters all of it's own in the 'Great SOWI In The Sky'

I suggest everyone who posts someting heads their posts with a *** TITLE *** at the top saying to what aspect their comment refers to, otherwise, if the post 'explodes' into action we'll be reading loads of comments over and ver again ...... even though that's not a bad consequence actually
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Fri Mar 22, 2019 11:36 am    Post subject: Reply with quote

... as there are no replies ..... I assume everyone goes on the 'I do it my way' highway.

Fair enough ... I wonder if their managers are aware Smile
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Sun Mar 24, 2019 9:01 am    Post subject: Reply with quote

John S,

This could be such a broad topic.
I am interested in your description of working in a big stress office, where you may have attempted to define a standard approach to validation.
Where to begin with that ?
In my Structural FE work, most validation attempts I carried out were one-off's that looked as sub-system equilibrium checks or verification of stress fields. These were either using Excel calculations or graphical displays of localised calculated effects. There were rarely "valid results" to compare against. It would have been interesting to have had a defined reporting standard for reporting results.
What would have been the budget for the "defining the standard" project and was that project ever completed ?
In my operational modelling work, a correct answer was even more elusive !

Is that what you are implying ?

John C
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Tue Mar 26, 2019 10:23 pm    Post subject: Reply with quote

A standardised approach for 'recording of results' and indeed 'validation of results' are 2 100% valid topics. There re many more of course.

imagine someone dug out some of your calculations from say 20 yrs ago and sent them to a 3rd party company ith a view to repeating them for some reason.
It would be useful if there was traceability of some kind to 'pick up' from what was done before as a starting point.
In my experience it seldom occurs.
Particularly, as you mention it, for Excel files !

As for budget, the system I'm talking about that I was thrown into didn't have a budget. It was the cief stressman's right hand man (day-to-day head of stress office who put it into place.
It was the hazy days before the obsession with costing for everything became such a rage (although it was at a time when mainframe jobs had to be costed - big (2000 nodes for a satellite FEM in 1980) models at several 1000's per run !!!.
It was 'the norm back then that engineers proposed certain things to improve the sytstem and were not enforced into being simple number-generating slaves. (I hesitate to use number-cruching because these days there's ot a lot of crunching and real dissecting of the numbers that goes on anymore imo. No thinking. No chcking. No questioning onesself 'are the numbers correct'.

Anyway I digress - th SOWI I referred to were simply a set of lever arch files setting out how things should be documented.
From how to number yur bos of cards to how to layout your stress calcs on standard sheets.

I believe in recent yers there has been an attempt to standardise stess calculations within Airbus (the name of it excapes me). All it actually does is put a front end on a group of 'selected' programs from all the Airbus family of companies and sites. I've heard comments that it's a mess.

I guess they started out with the best of intentions but then hit the obstacles:
a) cost (as you say)
b) techical dispute (who decides which program from a selection of several) to use as 'standard'
c) the problems of 'guiding' the IT company providing the front end program

They probably hit the same problem as a program called 'DOORS', intended for project control of things like non-conformances in the space business, which very few people actually understand how it works.

At the end of the day it depends on how much a company wishes to invest in setting up a doumentation/methods control system.
Usually it's not very much because the people in control (managers) are not in general engineers and because they can't calculate how much money they will save refuse to enter into it with enthousiasm.

My domain is mainly space, although I have worked about 15 years ago on the Airbus A380. What I've seen there beggars belief.
There was NO system at all in place ..... it's no surprise to me that they have just given it the chop. I could tell a tale, and it wouldn't be pretty.

Another thing I saw at another orgnisation was when they brought in a dedicated person as 'Documentalist' I think it was called. She was in charge of sorting out documentation fo the whole dept. . It worrked out great, largely because of her efficiency and hard work. If it still exists today I don't know, not how it might have evolved.
All I know is that everywhere I've worked as a consultant has been shall we say 'less than efficient'.

Another major, I mean MAJOR 'out-of-control' aspect is the size of FEMs.
People are left to get on with it and the models are ALWAYS too big !
I've seen major systems crippled by an innocuous model of a electronic box (500,000 nodes in 1998 !).

There's NO CONTROL whatsoever.

Many aspects, and a very broad scope indeed.

And every 'industry' is different and has different needs.

Another ospect of 'stress office' environment is the 'Design Manual'.
Every aerospace stress office had one, and extracts from many more (Boeing, Lockheed, tc .... which were treated as standard.
I'd be interested to see how aircraft calcs are 'signed off' these days.

Th reason concorde wasn't 'updated' after the acciden
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Tue Mar 26, 2019 10:26 pm    Post subject: Reply with quote

People in different domins (scientific community, universities, etc ....) will of course have different needs and goals, so the subject is wide open as you say.

A combination of gneral 'good practice habits' combind with specific requirements is the order of the day.
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2388
Location: Yateley, Hants, UK

PostPosted: Wed Mar 27, 2019 11:38 am    Post subject: Reply with quote

This thread is my child, even if it has been adopted and developed by others.
Some definitions for the purpose of this reply are as follows. Validating computer software means checking that it correctly implements the procedure that it claims to implement, whereas Checking means confirming that the input data set properly represents the problem being analysed. Archiving means the retention of data input and matching output together with all checks on it, and if necessary, some record of the software being used and the certification that the software has been validated.
Validation: it appears to me that validation is the job of the originator of the software, but nevertheless it is wise for the user to satisfy himself that for his range of problems the validation is adequate. Mostly, in engineering, validation is done by running through test problems for which the solution is known. The validation stage pretty much tests the hardware and the compiler as well as the program application itself. Inevitably, there will be some cases that are not solved adequately. One can cope with this if the documentation is adequate and warns against the cases that do not run properly.
Checking seems to me to be the responsibility of the user, although this may be delegated to an independent checker. There are multiple stages for this, for example if one is using software in the design of an earth dam, it is not only the computer analysis that needs to be checked - for example the geometry and internals zonation of the dam is correctly represented, but also the material parameters for each zone have been correctly determined by laboratory tests for example. People using standard materials like steel and concrete, or aluminium, can rely on specifying the properties that need to be achieved and indeed they are probably better known than stuff delivered by the lorry load and compacted on site.
The worst ever example of checking that I ever saw was when the checker bought the same model computer (Amstrad, no less), the same software, obtained copies of the input data from the first user on floppy disk, ran the same program on the same model computer and set a junior the job of checking that the outputs were the same!
Archiving is a completely different matter. One may need to archive the outputs, the inputs, or occasionally even the software! I have mostly disappointing experiences in this.
First of all, there is the issue of the media used to the archive. Paper printouts are all very well, but particularly with sizes that don’t go in filing cabinets or which are too voluminous there is always the risk that the paper is ‘recycled’. At least for a while it is human readable. It has the further advantage that it can be annotated by hand. Digital storage media need devices to read them, and sometimes the drives cease to be available. Floppy disks, tapes, ZIP drives – where are they now?
Secondly, unless the input data is printed out or kept to match the outputs, the results may be largely meaningless. In the beginning this might have meant labelled decks of cards and printout kept in binders, but today it’s likely to require a naming system for files.
Sometimes, one needs to follow a ‘development of ideas’ process, and this needs either a report or many saved analyses. All too often one only sees the final result.
Some of my experience is about tracking down what was done in support of reporting on potential negligence. I have seen information shredded, lost on the hard drives of obsolete computers, stored on media which cannot be read, filed with incomprehensible filenames or only partly preserved.
A particular worry is when the current version of an application won’t read older version files, or when it does, then produces a different result. The former is the case with my drawing application CorelDRAW! and also with Microsoft Word which will no longer read WordPerfect files.
For decades I have believed that there will always be a Fortran* (strike up the music: https://www.youtube.com/watch?v=gWSjNtbT4lg ) compiler ...


Last edited by LitusSaxonicum on Wed Mar 27, 2019 11:43 am; edited 2 times in total
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2388
Location: Yateley, Hants, UK

PostPosted: Wed Mar 27, 2019 11:40 am    Post subject: Reply with quote

... on whatever computer I have, and mainly for engineering applications I only use software I have developed myself, so I could always recompile and redo the analyses, getting the same or similar answer and understanding why. However, a recent post shows that low precision can dog this process. What’s more, making a feature** deleted – even in FTN95 – means that the program won’t be the same, notwithstanding that it isn’t the same once recompiled, even if the source code is.
There are many questions to which the answer is of interest. Sometimes there isn’t an answer, but the question must be asked regardless. I suspect that 'what is a good way of Archiving' falls into the latter category.
Eddie
*At one time I used the rather superior Algol-60 – now where’s that?
**Even ones that might only be used rarely, or consider to be rather stupid.
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Fri Mar 29, 2019 10:13 pm    Post subject: Reply with quote

John C,

re-reading your post of the other day you talk of validation of FEM results.

Well in the industry Iìm in (Space ... the final fr-....) we have had for many years certain checks that have to be performed on FE models befote they're considered 'validated' - from the mathematical quality (whatever that is) point of view. Strain energy checks, Free-free modes should have 6 rigid body modes 1 etc .... There are formal specs in most big companies for 30 years or more. We had strain energy check via DMAP way before MSc got off their arsenal and stuck one in as standard. We still use the DMAP one because it has great doagnostics for tracing the problems.
One company I worked at then had a standard 'validation checks' results summary sheet. If it wasn't filled in with all checks compliant then the model was rejected.
This is even more important because often your models are sub-system models and have to be supplied further up the chain, eventually getting into the overall spacecraft model.

Anyway that's FEM model quality checiking.
But of course the even bigger challenge is 'model quality.'
Especially for stressing.
For example experience has told me never to use tetras for stressing because Mr. Murphy says if you do ou will get multi-apexes of a group of elements joining together and the stresses will skyrocket ! People often struggle trying to work out design mods for something which doesn't have a negative margin but the FEM says it does !
Much more difficult to create a 'catch-all' validation procedure for models.

I wonder what testing/vvalidation/verification/checking proìcedures were used for the Boing 737 800 MAX fix that's now slowly being confirmed to be the souce of the 2 recent crashes.
I see they're confirming that a fix was introduced after the Lion Air crash but it's been kicking in not only in stall situations but jus when he pilot tries to climb !!!
There'll be a few worridìed people poring over what's gone on in meetings I would guess.
A bit like the challenger disaster - every engineer AND manager should be made to study that to teach them what the consequences of an engineering error could be.

Managers should have a pre-requisite of being an expert engineer before they're let loose on a project and start trying to convince expert stress people for example why they think their calculations are too conservative.
I'm all for lots more testing (the true verification , or is it validation?) just because we'd then get mor failures because of stupid decisions ny managers made on cost and time and above all 'schedule' , oh sorry 'resources' criteria.
It's a log tim since I've seen a schedule that was real, that wasn't built around a need end date and cost. Then chopping essential work willy-nilly with absolutely no regard for th concìsequences and not even a tentative attempt at 'impact assessment'. A real one that sniffs out problems instead of a fabricated one which ignore factors which one knows would sink the ship.

Anyway I digress, but it's important because if anagers continue, there'll be no need for checking/validation/verification of anything because the risk factors of what they do make any impacts of wrong models, wrong computer calculations, whatever, simply obsolete in comparison.

Safety facors are one hige bone of contention because a) they're always far too high b) to reduce them you have to be careful with your calculations (which takes more time and c) you have to do more testing

Both b) and c) are a project managers nightmare because they never take things like that into account anymore in their plannings a the beginning nìbecause they're so keen to win a job that they basially lie about how long it will take and how much it will cost.
It always comes out 3 times mor expensive and doubles the schedule.

I'll give you a concrete example.
You've probably heard a bit of scaremongering going on over the last few weeks/months about the 'danger' of the UK not being involved in the European Galileo satellite system if Brexit happens.
What they don't
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Fri Mar 29, 2019 10:20 pm    Post subject: Reply with quote

Eddie, you wrote:
Quote:
There are many questions to which the answer is of interest. Sometimes there isn’t an answer, but the question must be asked regardless

Hear ! Hear! you hit the nail on the head with that one.
Many problems today exist because people don't ask the questions, the 'what if?' s.
There are of course 'what if's' that are asked , and then the answers ignored because 'they' don't like the answers, or wrì'orse still they don'0t care.
I'm thinking again of a specific example relating to the A380, from personal experience, but it's too late to go into that besides which you might never fly on anothe airplane if I told you !

Lots of other good points you raise in there too.
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Sat Mar 30, 2019 1:09 pm    Post subject: Reply with quote

we've all been here before I'm sure Smile ...

_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2388
Location: Yateley, Hants, UK

PostPosted: Sat Mar 30, 2019 2:04 pm    Post subject: Reply with quote

John_S, You've homed in on what I called 'validation', and the thing I gave least weight to, probably because most software firms have procedures for it. Even so, sometimes the published solutions are wrong, or the published theory is wrong. I've come across both.

What I called 'checking' is being sure that you have fed the right model into the analysis. In your Aerospace analogy, like feeding in Airbus 320 geometry for the wings on a 737, or perhaps fed in more or less the right wing but gave it the wrong angle of sweep. Or maybe have it steel Young's modulus instead of aluminium alloy. Or assumed it was hot when it was cold. Etc. This is far more common than you'd think, although my field isn't (often) aerospace. In my one experience of aero, the object had been 'analysed' in plane strain, and it was axisymmetric so what was 'obviously' compression was, in fact, in tension.

When I proposed the topic, I was thinking mainly about retention of results. I've acted as an Expert in litigation where the analyses were lost, destroyed (sometimes accidentally, but also deliberately), filed incomprehensibly when they could be found, or unreadable on unreadable or corrupted media. In aerospace terms, it may be possible to find the calculations for a Spitfire, or even a Comet, but I'll bet that you can't find them for most of the oldest jet passenger aircraft still in service ( https://www.telegraph.co.uk/travel/travel-truths/oldest-passenger-plane-still-in-service/ ) which do fall into the albeit early computer era.

Eddie
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group