Save attribute for module variables

OK, fair question :lol:

Because - like Klaus, as far as I can tell - I am under the impression that any variable, simply declared in a module, is only required to retain its value, per the standard and without taking further declarative steps, in other program units which USE that module.

In other words, when a module M passes control to another program unit N, X is only required to retain its value if N USEs M. If, therefore, N is the main program, X is required to retain its value throughout execution, wherever control may pass to in the meantime.

(Technical FORTRAN meaning of USE, not shouting, in the above).

I believe that Klaus believes the same thing, and claims on occasion to have observed this requirement being violated by FTN95.

I acknowledge that real compilers may choose not to exercize the freedom to throw variable values away in this fashion (what others are terming 'behaving as one would expect'), but I was under the impression that they were at liberty to. I do not believe, however that this is germane to what is troubling Klaus.

If he is correct in what he claims on occasion to have observed, then either his (and my) understanding of the requirement is incorrect, or FTN95 is not implementing the requirement completely correctly.

I believe he is seeking adjudication one way or the other.

If, therefore, N is the main program, X is required to retain its value throughout execution, wherever control may pass to in the meantime.

I'm not very familiar with the actual Fortran standards publications, and can't find anything in the docs that supports your claim (it sounds fine though, and may be how current compilers work, but if it's not hard-written somewhere you can't rely on it and thus have to use the SAVE attribute to write correct code if the value has to be retained).

In the latest version of the Fortran 95 standard I know of : 'J3/97-007R2 - October 21, 1997 12:28 pm' which I found from jjgermis's link above; in Section 14.7.6 'Events that cause variables to become undefined' on Page 290, point 3 discusses how Modules can remain defined after a return statement. There is always ambiguity about what MUST happen in other cases! We must differentiate between USE and scope of a module in this discussion, as if routines A then B are called from the Main program and the module is only defined in A and B, then the scope of the module ceases on return from A, potentially rendering it undefined on entry to B. Why provide for this potential !! I have many static libraries which do this, and work. My problem with this approach is it denies the basic concept of what a COMMON or MODULE is, which is a storage area for variables which has global access. Causing variables in a Module to become undefined does not appear sensible to me. Indeed no 'compiler' I have used implements this approach, although it is more an attribute of the linker in how it handles the memory allocation for Modules. Making local variables have dynamic memory allocation has some merit, as it minimises the memory usage of many routines and allows for recursion, however making global variable structures dynamic appears to have much less, even no merit. My main problem in this post is I do not agree with the standard where it potentially allows common or modules to become undefined. As I said before, this is managed in the linker and the only place where I saw this implemented was in an overlay linker in a DOS 640k environment. Is Klaus trying to do a standard conforming review of the FTN95 compiler or is the solution simply to put a USE statement in the main program ? My understanding of FTN95/Slink is that COMMON and MODULES do not become undefined, except of for allocatable arrays in modules that are in a DEALLOCATE statement. I have recently used this construct to prepare for 64-bit array allocation and have had no problems of unexpected undefined status. I recall this discussion in the Lahey email forum about 5 years ago. I'm not sure where it ended.

One thing is for sure, we are not going to change the 95 standard. Klaus, do you know if the description of this problem has changed in the 03 or 08 standards ?

John

I have many static libraries which do this, and work.

'It works' and 'It's standard compliant and thus works' are completely different things, the first case may work since a long time but then due to some circumstances (because the standard doesn't require it to be like that) it may change. For example to utilize additional optimizations or whatever.

how Modules can remain defined after a return statement. There is always ambiguity about what MUST happen in other cases!

If it's not explicitly defined, anything can happen. This has some strange effects especially with C/C++ compilers (under/overflowing arithmetics, out of array access, ...) but the most simple case is accessing an uninitialized variable: there is absolutely no guarantee about what value it contains, behaviour is undefined (so any a=b with b undefined may put 0 into a, may put 7 into a or may even execute pacman and the last case is not a joke).

Is Klaus trying to do a standard conforming review of the FTN95 compiler

I think his main point is that in almost all cases the compiler does 'what most people expect' for his problematic code, but some cases turned up where it was different, as he stated the SAVE attribute was required.

but some cases turned up where it was different, as he stated the SAVE attribute was required.

As FTN95 retains Modules and Common as static allocation (except for Allocatable arrays), I would be surprised if applying SAVE to a module would be the reason for the change. Can Klaus confirm if this took place ?

I can report I have not identified the problem that Klaus initially suggested, as in my recent use of modules, I don't use SAVE on any modules and have not identified any loss of variable values. I have converted some of my libraries to use modules with no problems. These would fail the scope test. My Allocatable arrays in modules also retain their value until they are de-allocated, without any impact on other variables in the module, although this module is defined in the main procedure.

John

I restrict myself to important aspects of the discussion only.

I am sorry, that I cannot post an example code that illustrates the problem. The code is much too big for the forum. I have tried to condense it to a suitable size, but failed. In view of the complexity of the code I have never expected that somebody -even if he is author of a book- can tell me from the debugger output where the error is. (I can read the debugger message myself, see also below). Instead I have posted this problem clearly as a question (“My question is whether somebody has made similar experiences?” Nothing more).

The discussion shows that there are quite different views on this subject. I read for instance that there might be some freedom for compiler developers to implement such features with –may be- compiler dependent properties. Hard to believe for a language with the goal to increase portability. However, such implementations are completely unknown to me. I work for good reasons for many, many years with Salford/Silverfrost compilers only.

Putting all this together, I conclude, that Silverfrost developers can best clarify the question and probably nobody else. I understood Paul Laidler that he will have a look to the item and I will wait for his answer.

I have never stated that I can exclude a difficult to find error in the code, which I have not written myself. The code uses 4 huge input data sets from an even bigger code (up to 100 MB direct access data sets) and of course I will check these data sets for consistency as well as the code itself. And I will discuss the problem with one of the authors of the code probably by the end of January.

Therefore I repeat what I have said already: “I cannot contribute anything more to the discussion”. A last remark: This thread is not suffering from insufficient clarity of definition of terms but –much simpler- from a mixture of a general discussion about a Fortran rule, may be its implementation and from a code that is not available to everybody.

                  KL

PS

For AS: I have applied the compiler options /CHECKMATE and /Full_Debug. The error message is the normal error message for undefined variables from sdbg. The definition “undefined not to be relied upon” is unknown to me and I have never seen it in any Fortran book. May be that this concept is discussed in newer books or it refers to specific compiler implementations.

definition “undefined not to be relied upon” is unknown to me

This is a pretty well-defined situation, if something is not explicitly written in the standard, anything can happen. Read for example http://en.wikipedia.org/wiki/Undefined_behavior or an article with really nice examples (especially the security related ones that are the essence of recent GCC complications) at http://blog.regehr.org/archives/213

So if module variable persistence is not guaranteed under the usual use scenario, it's worth knowing that since it is an assumption I'm used to in everyday sources.

Now that I have come to look at this more carefully I find that this thread raises a different question to your earlier thread started on 6 August.

In the first thread you seemed to be saying that the SAVE attribute was added by the compiler when not explicitly requested in your program.

Now you seem to say that the SAVE attribute, which ought to be added by the compiler, is not being added and you have to add it explicitly in your program.

I have tested this using the following code and can find no problem here. Unless you can supply a program that fails, there is little or nothing I can do to investigate the problem.

With kind regards

Paul

module m1
integer,allocatable::k1(:)
end

subroutine s1()
use m1
allocate(k1(10))
end

subroutine s2()
use m1
k1 = 10
end

program p1
use m1
call s1()
call s2()
end

Klaus,

You say

The error message is the normal error message for undefined variables from sdbg.

As you are using SDBG, are you able to look at the value of all variables in the module and see if they are all undefined, or if it is only a few it may be some other problem.

John

Paul,

I would like to thank you for the clarification, which is certainly not only based on the example given.

I have made some further tests myself and it looks as if all variables in a module automatically get the SAVE attribute (or are treated as static variables). The following code gives an example:

winapp

module mod1
 integer , Dimension (:), Allocatable :: intvar
end module mod1
  
program prog1
! use mod1
    call sub1
    call sub2
end program prog1

subroutine sub1 
  use mod1
  Allocate (intvar ( 1:100000))
            intvar = 1 ! define intvar (1:100000)
  write (*,*) sum ( intvar ) * 1
end subroutine sub1

subroutine sub2 
  use mod1
  Implicit None
  write (*,*) sum ( intvar ) * 2
end subroutine sub2

May be that the statement of Metcalf et al., fortran 95/2003 explained (see begin of the thread) is no longer valid or I have misunderstood it.

Klaus Lassmann

As I understand it, the statement in Metcalf and Reid is valid but this does not prevent FTN95 from using SAVE with all module variables if it choses to. It just means that the storage is permanent (whilst the program is running) rather than temporary (whilst the subroutine is executing).

Using 'permanent' store rather than 'temporary' store may be wasteful but will give correct results (unless recursion is possible).

As I said some time back, I assume that it is difficult for the compiler (writer) to work out when only temporary store is required so permanent store has been used in all related contexts.

Having said all this if there is still a problem that can be demonstrated then I am willing to investigate.

Paul,

I fully agree with your position. One suggestion: would it be possible to add 'modules' in the chapter on the SAVE attribute of the handbook.

After this discussion there are two positive aspects: from the discussions I got a much better insight into FTN95 and I am still alive although having been under heavy fire.

Many thanks,

Klaus

I don't think there's a problem here. When a variable is defined in the specification part of a module and does not have the save attribute, it is assumed to be out of scope and 'undefined' as soon as the module is not currently being USEd anywhere.

But the compiler does not have to remove the variable from memory. It can just put the variable in static memory if it wants too (recursion can't occur, so this is the simplest way to implement module variables).

However, people should not write code that relies on static allocation by default but should use the save attribute as necessary.

The situation is rather like local variables in a subroutine Fortran 77. These could be managed on a heap and thrown away when the subroutine exits (variables go out of scope), but a lot of early compilers didn't implement it that way -- they just kept them in memory.

Dear Davidb,

thank you for your explanations. After what has been said in this thread I have decided to follow exactly what you propose: to add SAVE statements where necessary, in case of any doubts to add a blank SAVE statement at the end of the module just to ensure portability. For my programs this does not cause any problems. However, the situation may be different for problems where storage is a limiting factor.

Klaus

Reviving this thread, which was very interesting, in order to report the experiences, similar to those of Klaus, that I made this afternoon.

After a modest reorganization of a real code that I am working on, I found myself in exactly the situation he describes: a variable x, declared in a module m that is USEd by the main program p (albeit indirectly via its USE of other modules that USE m), and initialized to the value 2 in p, acquired the value 0 when control passed to a routine in m. I only know this happened because the value 0 indicates an error condition that is handled by p; I would otherwise have been unaware of it.

So I decided to have a go at reproducing the behaviour in a simple program that aped some of the module USE hierarchy in the real code. To my amazement, after half an hour or so, it appeared that I had done exactly that, and more. Not only did one variable, initialized in the main program, later become undefined, but even more exciting ... another variable, local to the main program, became initialized simultaneously with the first variable!

By this time, I had a main program and (I think) five modules all using each other with various degrees of hierarchy. I decided I would try and simplify the code a little - and the anomalous behaviour went away. Try as I might to undo my simplification, I did not manage to restore it.

Bearing in mind that FTN95 is not perfect in its handling of dependencies, and that I was progressively introducing more complexity in the USE hierarchy as I went along, I think I may at some critical point have relied on a build rather than a rebuild ...

... because I then went away and rebuilt my real code, and the anomalous behaviour went away there too.

So we have two alternative hypotheses to explain the behaviour that Klaus has reported seeing (which I have also on occasion believed I saw):

• a subtle compiler bug
• an 'incorrect build' resulting from compiler miscalculation of dependency implications

In the spirit of Occam's razor, I would like to suggest that the latter should be adopted as the null hypothesis, dull though it is in comparison to the exciting alternative.

This does not make sense to me, as all modules are static in SLINK. Look in the .map file. I don't beleive your claim they become uninitialised. More likely, in the calling path they are being re-initialised somewhere.

While the standard may say you can not rely on their staying initialised, my understanding of FTN95 and Slink is this uninitialising does not happen.

Indeed, I have at least 2 static libraries which I have converted from COMMON to use Modules and they are only referenced in the library routines. If this un-unitialising took place it would happen to them.

John

Well, at the risk of sounding rude, perhaps you should re-read what I wrote with as much care as I took in writing it.

Sparge,

There is a significant risk! but I did re-read what you wrote. Are you suggesting there is a more likely potential problem with compiling a module heirarchy? I have certainly found that problem also. I now start all rebuilds with 'del *.mod' and I am careful with the order of compiling each module. I'm also not sure if when you have multiple modules defined in the same file, they have to be in the correct order, but I always do. The other point I have been making (repeatidly) is that my understanding of FTN95's implementation of the scope of modules (and common) suggests that the problem originally identified is not possible. It's been interesting researching on the approach to Scope. I found a number of papers on 'politically correct' programing which suggest that a global scope is a bad thing. I have been a fortran programmer, continuously, for over 35 years in both numerical methods (finite elements) and simulation of bulk materials handling systems. I have always used the concept of static libraries, rather than the approach of object orientated programming. COMMON and now MODULES with a global scope do not reflect this OOP approach. I start with a heirachy of modules, developing an in-memory database and then program around that. Using FTN95 compiler diagnostics has greatly assisted in a fast implementation of robust code. This has been an approach that works best for me. It amazes me at the number of apparently bug ridden pachages there are out there that use OOP, although fortunately for me I'm typically the only user of my software.

I think the Fortran Standard commitee needed more fortran users!

I managed to cope with the re-read of your post. I hope I got it this time.

John

Hi John,

Thank you for re-reading. I hope it was clear second time around that I was not 'claiming' anything - I was merely narrating actual experience, only a couple of hours after experiencing it.

I have noted your repeated making of the point that the problem originally identified is not possible. Nonetheless, when real people actually observe it happening, then that point becomes rather academic, and we need to seek an explanation instead of how the apparently impossible can actually happen. ('What's the difference beween theory and practice? In theory, there's no difference ...' 😃 ) So I have repackaged my experience yesterday into the form of conclusions:

It is possible to build an exe which exhibits the apparently impossible behaviour that Klaus originally posted about. I did it yesterday - twice.

This behaviour could be observed by others if they ran a copy of such an exe on their own machine.

This is the only reliable way to exhibit such behaviour, because it depends not only on source code but also on an incorrect compilation process.

Complex module hierarchy increases the likelihood of such behaviour being observed in practice, as does being in the early stages of developing new code.

One way to achieve an exe that exhibits such behaviour is to rely on the Plato 'build' command, because Plato does not handle dependencies with complete reliability. Once such an exe has been produced, Plato will be satisfied that the 'target is up to date'. However, 'rebuild' will produce a different exe which does not exhibit the behaviour. Always for me so far, anyway.

Because Plato does not handle dependencies with complete reliability, I presume it is possible in principle that even 'rebuild' might produce an anomalous exe of this kind, but I personally have never found myself in this situation, as far as I know.

I do not understand the detail of what is going on in an anomalous exe of this kind, I am satisfied that its origin is neither a bug in code, nor a bug in the compiler, but a 'bug' in the compilation process. I have been a FORTRAN programmer for nearly as long as you, but not continuously (though I have never gone more than a few months without). When I used FOTRAN 77, I used to have one file, one routine, with the same name for both. Now I am using FORTRAN 95, I have one module, one routine, with the same name for both. Can't see any point in making it any more difficult to get my head round code than it already is!

Andy

Andy,

I probably learnt my programming style from re-writing the tektronix plot10 code in 1970's, which had lots of subroutines or functions. A guiding rule is if a subroutine has more than 60 lines of code it probably is too long, potentially too complex. I think the 60 comes from one printed page of code. Everyone has their own style, and mine has been to simplify what each subroutine does.

Back to the problem, what does SDBG do if the module being used in the .exe you created is not compatible throughout the code ?

John