Size and 64 bits

The release notes say that a 64 bit integer is returned when the SIZE intrinsic is used with the 64 bit compiler. This non-conformance seems to be a design decision. However, it means the following legitimate code fails to compile.

module foo
contains
function find_max(a, n)
integer :: n
real :: a(n)
real :: find_max
find_max = maxval(a)
end function find_max
end module foo

program main
use foo
real a(5)
a = (/1.0,2.0,4.0,3.0,0.0/)
print *, find_max(a, size(a))
end

In Fortran 95, the Size intrinsic has an array argument and an optional DIM argument.

RESULT = SIZE(ARRAY[, DIM])

Fortran 2003 added an optional kind argument so that a result with a particular kind could be returned when necessary:

RESULT = SIZE(ARRAY[, DIM [, KIND]])

It would be wise to implement your size the same way and then it will be standard conforming. I know there is a /SIZE32 command line option to always make SIZE return a 32 bit value, but this isn't as useful as the solution that is in the standard.

Thank you for the feedback. I have made a note of this.

FTN95 has added some of the intrinsics from Fortran 2003 in the latest release. But it isn't claiming to be a Fortran 2003 compiler. It does, however, claim to be a Fortran 95 compiler, and so must strive to fix any non-conformances.

It needs to ensure that SIZE returns a default integer. If it is wanted to return sizes larger than HUGE(1) then changing the intrinsic to behave like the Fortran 2003 function would be a sensible move.

Davidb,

I support the present approach Silverfrost adopted for FTN95 /64 for SIZE being integer*8. It is true you can construct cases where SIZE does not work as defined in the standard, but there are a number of areas where non-standard approaches are required with /64 to improve usability.

The requirement for default integer was not proposed when 64-bit usage was the norm. As LOC is I*8 for /64, SIZE should also be this way. I wonder how often those who object to SIZE not returning default integer do use SIZE with /64.

For F2003, I have to use : 'size(profile,kind=kind(ipos))'. You certainly can't use 'size(profile,kind=8)'. Prior to using a 64 bit environment, I never used kind= with SIZE, so to find it did not default to the expected size of the array.

FTN95 also uses INTEGER*7 for handles, which is an interesting twist. Useful as it highlights handles that differ between 32 and 64 bit usage.

With 32 bit use, LOC annoyingly returns a -ve value for memory above 2gb which has long been a hastle. There has to be a balance between usability and code conformance and I am pleased that usability won in this case. I am sure there are many who would disagree, but I am a FTN95 Fortran user.

John

When you click on the page to buy FTN95 you are told that the compiler is a full ANSI Fortran 95 compiler. But this incompatibility means it isn't for 64 bit so either they will need to remove this claim for this compiler or continue trading on the premise they will just refund when people complain. It would be rather silly to do this when fixing SIZE is so easy.

They could fix this without invoking the use of the KIND parameter (as in F2003). Array dimensions need to be limited by (-maxint to maxint) where maxint is MAXINT(0) the largest default integer.

The current default integer is 32 bits (maxint = 2**31-1) but this can be changed using the option /DEFINT_KIND if one needs larger array dimensions.

I don't quite follow your arguments about usability. Nor do I see why SIZE has to be aligned with LOC. I would be interested to see some short examples where it is deemed necessary for SIZE to return a 64 bit value when the default integer is 32 bit.

David,

'Array dimensions need to be limited by (-maxint to maxint) where maxint is MAXINT(0) the largest default integer. '

What are you saying ?

The whole point of /64, ie 64 bit addressing is that arrays will be larger than 2^31. I have a number of examples where this is the case. I have re-written my equation solver for larger arrays, now using it with a 23 gb skyline matrix by using an integer*8 index. It actually works very well.

The point I was trying to make about LOC as default integer is that even in 32 bit, it did not work for /3GB.

If I code 'SIZE (profile)' I expect to get an answer that is useable. It is a generic intrinsic. I actually need a useable answer and to have an I*8 returned in /64 is a necessary outcome. If at this stage of development FTN95 /64 returned a default integer value, it would be useless.

I wonder how much consideration was given to the useability of SIZE with 64 bit in 1994 when it was defined as default integer ?

There are a number of non-standard features in FTN95, including business formatting, the use of commas in data and transparent I/O. I don't think many would ask for a refund because of the way FTN95 provides stream I/O.

John

Quoted from JohnCampbell

'Array dimensions need to be limited by (-maxint to maxint) where maxint is MAXINT(0) the largest default integer. '

What are you saying ?

The default integer model (32 bit signed integers) has maxint equal to 2**31-1 = 2147483647. In an array or array section expression, the subscripts must be between -2147483647 and 2147483647 (inclusive), so the largest array you can declare is:

BIG_ARRAY(-2147483647:2147483647)

If you don't like non-positive indices and want the first index to be 1, the largest array is

BIG_ARRAY(2147483647)

Quoted from JohnCampbell

The whole point of /64, ie 64 bit addressing is that arrays will be larger than 2^31. I have a number of examples where this is the case.

With FTN95 it isn't possible to create an array in with any one dimension bigger than 2^31-1 unless you use non-default integers for the dimensions. To do this using /64 the default integer model would have to change to allow larger integers (e.g. to 64 bit integers, maxint = 2**63-1). However, currently the compiler doesn't do this.

I would like to see an example, just a code snippet were such arrays get created.

Quoted from JohnCampbell

The point I was trying to make about LOC as default integer is that even in 32 bit, it did not work for /3GB.

Yes, I understand that. LOC is about the concept of address, so it is helpful for it to be coded as a 64 bit integer on 64 bit machines. LOC is an extension and I have no problem with how it works.

Quoted from JohnCampbell

If I code 'SIZE (profile)' I expect to get an answer that is useable. It is a generic intrinsic. I actually need a useable answer and to have an I*8 returned in /64 is a necessary outcome. If at this stage of development FTN95 /64 returned a default integer value, it would be useless.

But SIZE isn't supposed to be generic. The kind of the integer returned by SIZE has nothing to do with the kind of the data in the array.

SIZE is an 'array inquiry function'. It needs to return values which are compatible with the other 'array inquiry functions' UBOUND and LBOUND. However, these don't return a 64 bit value when /64 is used (I assume this is an oversight). It is necessary to maintain the identity SIZE=UBOUND - LBOUND + 1 and to allow UBOUND < LBOUND and SIZE=0.

If the compiler ensured that use of /64 also implied /DEF_INT=4 then SIZE would return a 64 bit value, which would be ANSI compliant and do what you want.

However, I suspect you would not want to change the default integers everywhere, and a much cleaner solution would be to use the Fortran 2003 mechanism of having KIND as an argument.

Here is a simple example of how this works (note the array is just 1 bigger than allowed using default integers, so integers with a larger range need to be used in the declaration of the array). Of course you should use parameters for the kind values, which I haven't done here.

program main
   real a(2147483648_4)
   print *, 'Size = ', size(a, kind=4)
end

Quoted from JohnCampbell

There are a number of non-standard features in FTN95, including business formatting, the use of commas in data and transparent I/O. I don't think many would ask for a refund because of the way FTN95 provides stream I/O

The point I was making is that Silverfrost won't want to make claims that are not true. If they decide not to fix-up this issue in /64 bit mode, I would expect them to say something like 'ANSI compatible 32 bit compiler, but not in 64 bit' somewhere on the web site.

Let's not forget that the 64 bit compiler is very new. Silverfrost deserve much credit for this development, which in time will prove very useful. There are a number

Davidb,

64 bit compilers have been around for a while and I have been using a number of them.

The code that works for me is:

     integer*8 ipos                                  ! profile storage
     real*8,    allocatable, dimension(:) :: Profile
!
      if ( allocated (profile) ) deallocate ( profile )
!
      allocate ( profile(ipos), stat=status )
!
  11  format (a,i0,a,i0,a,i0)
       write (*,11) '  [Profile] allocated; status = ',status,': size = ',size(profile),': loc = ',loc(profile)
!gf    write (*,11) '  [Profile] allocated; status = ',status,': size = ',size(profile,kind=kind(ipos)),': loc = ',loc(profile)
       if (status /= 0) stop

I should also point out that with FTN95, the array PROFILE can also be placed in COMMON if the dimension ipos is known. This makes increasing the size of existing code fairly easy, although all subscripts of PROFILE must be INTEGER*8.

As I indicated, I have been using this approach for a while, as I have 32 gb memory available to run these problems. Without SIZE returning I8, the useability would not be good enough. !gf indicates what is required for F03, although the I4 result was not expected.

64 bit use makes the claim of default integer unusable, while FTN95 kind values are a bit of a problem also.

The real problem is that the Fortran standards were written when 64-bit was not in common use and I am finding supporters of the standard stubborn in their acceptance that some intrinsics need to support 64 bit addressing and array sizes. It might be good for them, but I want to use 64-bit and not be told I have to conform to 32-bit. Time is moving on.

John

But in your code you are using integer*8 (kind=4) values to declare the dimensions in the allocate statement. This is valid, but you would need to re-state this when you invoke SIZE.

Your example shows how to change this at the !gf line (which I assume you turn on for gfortran). I am suggesting that you should be able to use the same syntax for FTN95 too.

The current approach for SIZE in FTN95 doesn't work for many other examples. I will try and post some later.

David,

I know the examples; basically anywhere where size(array) is used as an argument to a routine. I don't use that in any in real programs !

My main point is that these are just one of many areas where /64 has to change to work. You have already mentioned subscripts must be I*8. Any use of memory addresses also has to change, such as LOC and more importantly API handles, which is what is required for clearwin+. This is a much more significant change than SIZE; a potential for conversion errors and you do have to convert programs for /64.

As KIND values are different in FTN95, I try to avoid any use of KIND. Specifying I*8 constants is also cumbersome, as the simple 100_4 is non-portable and easy to misread. You would probably want 100_int64, but requires more discipline.

The alternatives for FTN95 were to change SIZE to I*8, like handles, or introduce more of the F03 syntax, such as what you are describing. My preference would have been to include some of the F08 level of ISO_FORTRAN_ENV module. I also consider F03's SIZE to be awkward and actually prefer FTN95's approach.

However, what we have at the moment is a workable system for /64. My expectation is that if people don't like or understand it then they are probably not fully using /64.

John

PS: I shall post one of my large array test programs to demonstrate use of large arrays. The warning with these is make sure you have the installed memory to run them, or you get a surprise.

I really do understand the issues. I also understand the need to keep things as simple as possible.

For Fortran 95 the developers have only two choices:

(1). The current approach where SIZE returns a 64 bit integer. You then have to add /SIZE32 to the command line to get ANSI compliance (where size returns default integer kind).

(2). Have ANSI compliance as the default (size returns a default integer kind). Then you would have to add /SIZE64 to the command line to get SIZE to return a 64 bit integer.

Both of these do what you want. But (1) is ANSI compliant by default, whereas (2) requires an optional command line parameter. It is generally better to make the compiler compliant by default and then add extensions by the use of command line switches. They have done this the other way around for some reason.

For Fortran 2003 extensions the developers could adopt (2) but also add the optional KIND argument to give additional flexibility.

Thanks again for the feedback. The command line option /ISO is required to ensure strict Fortran 95 compliance. So I guess that we need to review how this interacts with /64 and /SIZE32 (in addition to extending the SIZE intrinsic).

What happens with other compilers when KIND is not used and the result overflows the 32 bit signed integer limit?

What happens with other compilers when KIND is not used and the result overflows the 32 bit signed integer limit?

You get integer overflow and so the wrong result. That is what standard conforming would provide !! Please don't go back to this.

The best solution would be to have the non-standard SIZEOF (x) implemented, as ifort has done. It returns the size in bytes as I4 for /32 and I8 for /64. The other interesting aspect of SIZEOF is that x 'Can be a scalar or array. It may be of any data type. It must not be an assumed-size array.' I assume that includes derived types, which would be a useful addition. At least they have some user experience when defining these functions.

The standard also gave us 'file storage units' for RECL=, so the need for IOLENGTH=. We were not allowed the concept of a byte.

John

John, don't worry the intention is to keep the current behaviour. What we are discussing is how to implement it.

First it is clear that one also needs UBOUND and LBOUND and SIZE to be able to return 64bits. I think this works but isn't mentioned in the release notes.

Technically it is the programmers error when one of these functions is called and gives an out of range result. Remember the programmer is responsible for creating the array and knows the array dimension.

As John says, compilers typically treat this the same way as integer overflow, so they crash. Lahey and NAG can detect the overflow and provide an error.

Paul, for option (1) you could keep the current behaviour, but make /ISO imply the use of /SIZE32. However, can I suggest you use /SIZE_ISO instead of /SIZE32. Then you will be able to make this work whatever the default integer size is - to catch uses of /DEF_INT.

Option (2) just needs /SIZE64.

Then, you can add the F2003 kind argument for additional control on top. John wouldn't have to use this though.

(SIZOF is nothing to do with SIZE. Arrays don't need to be contiguous in memory so you can't convert between the two even if you know the storage unit size. Consider an array of 1000 elements, then SIZE(a(1:999:2)) is 500 )

David,

SIZOF is nothing to do with SIZE. Arrays don't need to be contiguous in memory so you can't convert between the two even if you know the storage unit size. Consider an array of 1000 elements, then SIZE(a(1:999:2)) is 500

This is not ifort. FTN95 would take a copy that is contiguous. I actually wonder if a non-contiguous array complies with the standard. It certainly doesn't conform to my understanding of Fortran and I am sure Eddie would agree on this one. non-contiguous arrays would break a lot of pre F90 codes.

What is the point of making Fortran process SIZE(a(1:999:2)). I suppose you could try ' INQUIRE ( IOLENGTH=len ) a(1:999:2) ' but I would expect FTN95 would crash with that. It will crash with ' INQUIRE ( IOLENGTH=len ) a(:) ' As a work around, you have to use ' INQUIRE ( IOLENGTH=len ) a(1) ; len = len * SIZE (a(:)) '

John

Non-contiguous arrays (also called array sections) have been in the Standard since Fortran 90.

When such an array is passed to a subroutine with an explicit interface and an assumed-shape dummy argument, FTN95 (like other Fortran compilers) passes a 'dope vector' which includes hidden information (start address, stride, size). There is no need for a copy to be made and FTN95 doesn't make one.

When such an array is passed to an external subroutine with an implicit interface (e.g. a F77 subroutine), a copy-in and/or a copy-out must be made. This is handled automatically by FTN95. Existing, F77 codes should therefore not break when a non-contiguous array or array section is passed as an actual argument.

For many F77 subroutines one must pass the array and size information, and it is here where the SIZE intrinsic would be most used. E.g. in subroutine PROCESS below SIZE is used to pass the size of the array to the blas subroutine DSCAL. The compiler needs to do a copy before the call, and a copy after the call.

The compiler doesn't currently support this properly.

module foo
contains
subroutine process(a,x)
double precision, intent(in) :: a
double precision, intent(inout) :: x(:) ! <-- this array may be non-contigous
call dscal(size(x), a, x, 1)
end subroutine process
end module foo

davidb,

I am not aware of FTN95 using a stride. I tested the example below that I adapted from your example in Plato, using FTN95 /64 and gFortran. It appears to show (me) that FTN95 /64 does not provide a stride while gFortran does. Do you agree ?

The use of stride can create problems for pre F90 memory management approaches, especially when contiguous memory is assumed.

I am relieved that FTN95 provides a copy of the array in this case, although it can provide a performance penalty. I have previously posted array section examples where FTN95 provides a copy when it did not need to, incurring a severe performance penalty.

John

module foo 
   contains 
     subroutine process(a,x) 
       double precision, intent(in)    :: a 
       double precision, intent(inout) :: x(:) ! <-- this array may be non-contigous
       
!       call dscal(size(x), a, x, 1)
       write (*,*) 'size   ', size(x)
       write (*,*) 'spacing', loc(x(2)) - loc(x(1))
       write (*,*) ' '
     end subroutine process 
 end module foo 
 
 program alloctest 
   use foo
   implicit none 
  
   real*8 aa(3,7), a
   
   call process ( a, aa(:,1) )
   call process ( a, aa(1,:) )
!
   end 

Hi John,

Yes, in this case FTN95 seems to be making a copy in and copy out. As you say this could be inefficient (with big arrays).

The dope vector must still be being passed, otherwise SIZE, UBOUND, and LBOUND would not work, but the dope vector of the copied array will have a stride of 1.

The following compilers avoid the copy.

gfortran nagfor lahey g95 ifort absoft pvf

I wonder if this is new behaviour for FTN95. I am sure earlier versions avoided the copy.

Perhaps Paul will comment on this.

In the first call to 'process', the array is contiguous and the base address and size of the section are passed. In the second call, the array is not contiguous, and it is copied in and copied out. The base address and size of the copy are passed.

As far as I am aware FTN95 has always worked this way.

Where possible users should avoid passing large non-contiguous array sections.

Also it is worth mentioning in this context that users should always provide an interface (either explicitly or by placing the subprogram in a module) when passing assumed-shape arrays. Otherwise a Fortran compiler may perform a copy-in/copy-out when it is not needed.

Paul,

By your last statement do you mean that you can avoid one of the copy operations when an explicit interface is used. That is:

avoid copy-in when intent(out) is used, and avoid copy-out when intent(in) is used.

If you mean you can avoid copies altogether this would contradict what you said earlier.