FTN95 64-bit beta test

The work on the 64-bit version of FTN95 has moved forward significantly. We are now in a position to offer a beta test version to users who are subscribers to our update program. If you are a member of the upgrade program and would like to try your code on our new compiler send us an email at ftn95@silverfrost.com

That is great. I do not have current upgrade contract which is fixable but, worse, right now have zero free time (better say it's negative) and afraid touching anything potentially time consuming, but i would like to hear the users' comments. If the state of code is really beta not alpha, i would probably jump in.

Key question for me is if debugger can handle 64bit or not like initially was planned. It will be close to impossible for me to scale down the 64bit elephant to 32bit ant to find the errors.

Does this beta diagnose in 64bit mode the error type and the line where it occured at least in the text regime if 64bit SDBG is not ready?

Also, designwise, will 64bit compiler support virtually unlimited amount of 2-4GB static arrays or all static arrays will be limited to 4 GB total size?

The 64 bit development has been added to the existing compiler FTN95.exe so all of the parsing and error reporting will remain essentially unchanged.

A new 'backend' has been to added to FTN95.exe for 'x64' to go alongside the existing backends for 'Win32' and '.NET'. These backends generate the relevant assembly instructions for each platform.

Here is an extraction from the beta release notes...

Arrays that are ALLOCATEd, or which are in COMMON or in MODULEs can exceed the 4GB limit, except that initialised arrays must fit within the .EXE or .DLL file to which they belong, and the the size of these files cannot extend beyond the 4GB limit. This is a Microsoft limit, but is fairly reasonable, since the time needed to load a 4GB file would be excessive!

COMMON blocks and MODULE arrays are allocated dynamically as a program starts in order to enjoy no 4GB restrictions. This is applied to all such storage blocks, because a program may exceed the 4GB limit even though each individual array lies within this limit.

Local arrays (static or dynamic) are restricted as in 32 bits. This is because it is not feasible to extend the hardware stack to sizes > 4GB, and SAVE'd variables must fit within the EXE or DLL file to which they belong. Users who require a very large local array, should put it in a COMMON block or MODULE referenced by only the one routine.

1. Let me be more exact. If i make an error of division by zero for example on array element lying in the 10GB array - does this beta report the offending line and error type? Does /check or /debug or /undef work right now? Does SDGB work?

2. As of 4GB EXE limit hopefully this will be lifted in future since fastest SSDs and RAM drives can load 4GB file in less then 1 second and advanced SSDs are even faster with 20GB/sec speed. But the next question makes this irrelevant because EXE size can be made very small (mine are 10-100 MB), just put all arrays into common blocks or make them allocatable

3. I still did not get it with the no 4GB limit in common blocks. All my arrays are large, total size is close to 3-4 GB and all are in the COMMON blocks or modules. Some are allocatable but the question is not about allocatables but the ones which are good old ones with the standard static dimensions like in FTN77. Can i declare now with 64 bits their dimensions much larger to get each such static array in common block to be 4GB each or larger? Or they have to be total < 4GB in size ?

Dan

The initial release does not include /check etc. when compiling 64 bit executables/DLLs. This will come later. For the moment it is assumed that all testing will be done using 32 bit mode.

I think that your other questions have been answered in the given release notes. If not then I will have to come back to this when I am more awake.

Paul,

1. What is written there is confusing and possibly wrong. Can static arrays in COMMON also have unlimited size like allocatables ? In other words, suppose i currently have an Array1 with one million numbers, or 8 MB total

real*8 Array1 common /commbl1/ Array1(100,100,100)

Now if i take another Array2 which is 1000 times larger, or to 8GB

real*8 Array2 common /commbl2/ Array2(1000,1000,1000)

will your 64bit compiler swallow Array2 with no problem ?

2. I will wait when you at least add /check /debug or /undef to report the offending line number and error type like older compilers have done before. Then this beta will be actually almost usable for me for real job. I need it ASAP. I suppose it does not require development of new SDBG ?

3. How do you plan to sell the 64bit version? As a separate product or as an upgrade or else? As soon as you add any error checking i am ready to pre-order it

I have been doing some beta testing of FTN95 Ver 8.0. The option described of having common larger than 2gb does work, providing there is no BLOCK DATA associated with the common.

This is a significant advance over other 64 bit Fortran compilers that I have tested, which only allow ALLOCATE to provide arrays larger than 2gb. This makes expanding existing 32 bit programs very easy, provided arrays do not exceed 2^31 elements, ie Integer*4 subscripts are suitable.

To show that this works, I wrote this quick example: integer4, parameter :: n1 = 100 integer4, parameter :: n2 = 660 integer4, parameter :: n3 = 850 ! real8 Array1, Array2, Array3 common /commbl1/ Array1(n1,n1,n1) common /commbl2/ Array2(n2,n2,n2) common /commbl3/ Array3(n3,n3,n3) ! ! Now if i increase it 1000 times to 8GB ! integer4 n,i,j,k,ne, ng real4 gb, sec external sec ! n = n1 gb = real(size(array1))8. / (1024.**3) write (,) ' ' write (,) 'Initialising Array 1 gb =',gb, sec(), ' sec' do i = 1,n do j = 1,n do k = 1,n array1(k,j,i) = i+j+k end do end do end do ! write (,) 'Checking Array 1', sec(), ' sec' ne = 0 ng = 0 do i = 1,n do j = 1,n do k = 1,n ng = ng+1 if ( array1(k,j,i) /= i+j+k ) ne = ne+1 end do end do end do write (,) ng,' elements : ',ne,' errors detected', sec(), ' sec' ! n = n2 gb = real(size(array2))8. / (1024.**3) write (,) ' ' write (,) 'Initialising Array 2 gb =',gb, sec(), ' sec' do i = 1,n do j = 1,n do k = 1,n array2(k,j,i) = i+j+k end do end do end do ! write (,) 'Checking Array 2', sec(), ' sec' ne = 0 ng = 0 do i = 1,n do j = 1,n do k = 1,n ng = ng+1 if ( array2(k,j,i) /= i+j+k ) ne = ne+1 end do end do end do write (,) ng,' elements : ',ne,' errors detected', sec(), ' sec' ! n = n3 gb = real(size(array3))8. / (1024.**3) write (,) ' ' write (,) 'Initialising Array 3 gb =',gb, sec() do i = 1,n do j = 1,n do k = 1,n array3(k,j,i) = i+j+k end do end do end do ! write (,) 'Checking Array 3', sec() ne = 0 ng = 0 do i = 1,n do j = 1,n do k = 1,n ng = ng+1 if ( array3(k,j,i) /= i+j+k ) ne = ne+1 end do end do end do write (,*) ng,' elements : ',ne,' errors detected', sec(), ' sec' ! end

   real*4 function sec ()
     integer*4 clock, rate
     integer*4 :: start = -1
!
     call system_clock ( clock, rate )
     if ( start < 0) start = clock
     sec = real(clock-start) / real (rate)
   end function sec     

The run report is

 Initialising Array 1 gb =    7.450581E-03     0.00000     sec

 Checking Array 1    1.130000E-02 sec

      1000000 elements :            0 errors detected    2.020000E-02 sec

  

 Initialising Array 2 gb =     2.14201        2.030000E-02 sec

 Checking Array 2     1.72930     sec

    287496000 elements :            0 errors detected     3.05930     sec

  

 Initialising Array 3 gb =     4.57559         3.05940    

 Checking Array 3     6.69010    

    614125000 elements :            0 errors detected     9.51930     sec

This is very impressive!!

It will make converting existing 32 bit programs much easier than with other compilers, as you just have to change the array sizes. I would still recommend use of ALLOCATE, especially for any large local arrays, as it is the best way to overcome stack overflow errors.

John

I tidied up the test to show arrays larger than 2gb can be transferred through subroutine calls.

 integer*4, parameter :: n1 = 100
 integer*4, parameter :: n2 = 660
 integer*4, parameter :: n3 = 850
!
 real*8 Array1, Array2, Array3 
 common /commbl1/ Array1(n1,n1,n1) 
 common /commbl2/ Array2(n2,n2,n2) 
 common /commbl3/ Array3(n3,n3,n3) 
!
! Now if i increase it 1000 times to 8GB
!
  call test (array1, n1, 1)
  call test (array2, n2, 2)
  call test (array3, n3, 3)
!
  write (*,*) 'end of test'
  end
  
  subroutine test ( array, n, id)
   integer*4 n, id
   real*8    array(n,n,n)  
!
   integer*4 i,j,k,ne, ng
   real*4    gb, sec
   external  sec
!
     gb = real(size(array))*8. / (1024.**3)
     write (*,*) ' '
     write (*,11) 'Initialising Array ',id,' gb =',gb, sec(), ' sec'
11   format (a,i0,a,2f8.3,a)
      do i = 1,n
        do j = 1,n
          do k = 1,n
            array(k,j,i) = i+j+k
          end do
        end do
      end do
!
     write (*,12) 'Checking Array 1', sec(), ' sec'
12   format (a,f8.3,a)     
      ne = 0
      ng = 0
      do i = 1,n
        do j = 1,n
          do k = 1,n
            ng = ng+1
            if ( array(k,j,i) /= i+j+k ) ne = ne+1
          end do
        end do
      end do
     write (*,13) ng,' elements : ',ne,' errors detected', sec(), ' sec'
13   format (i10,a,i10,a,f10.3,a)
!
   end subroutine test

   real*4 function sec ()
     integer*4 clock, rate
     integer*4 :: start = -1
!
     call system_clock ( clock, rate )
     if ( start < 0) start = clock
     sec = real(clock-start) / real (rate)
   end function sec     

The results are:

Initialising Array 1 gb =   0.007   0.000 sec

Checking Array 1   0.009 sec

   1000000 elements :          0 errors detected     0.017 sec

  

Initialising Array 2 gb =   2.142   0.017 sec

Checking Array 1   2.414 sec

 287496000 elements :          0 errors detected     3.767 sec

  

Initialising Array 3 gb =   4.576   3.767 sec

Checking Array 1   8.821 sec

 614125000 elements :          0 errors detected    11.707 sec

 end of test

This is looking very good, although I have only tested programs that are bug free. ( I hope!!)

There are new skills to learn with 64 bit, especially learning to avoid exceeding the physical memory limit.

Dan

1. Hopefully John has provided the answer.

2. /debug with SDBG etc. all require extending to 64 bits and will be released in due course.

3. Purchasing information to follow.

A further test that shows: array larger than 2gb array with more than 2^31 elements use of SIZE and LOC as I*8 .. all working well

 integer*4, parameter :: n1 = 107
 integer*4, parameter :: n2 = 667
 integer*4, parameter :: n3 = 1357
!
 real*8 Array1, Array2         
 integer*2 Array3
 common /commbl1/ Array1(n1,n1,n1)    ! 9.0  mb,  1.2 m elements
 common /commbl2/ Array2(n2,n2,n2)    ! 2.21 gb,  297 m elements : test 2gb
 common /commbl3/ Array3(n3,n3,n3)    ! 4.65 gb,  2.4 g elements : test long addressing
!
! Now if i increase it 1000 times to 8GB
!
  call test8 (array1, n1, 1)
  call test8 (array2, n2, 2)
!
  call test2 (array3, n3, 3)
!
  write (*,*) 'end of test'
  end
  
  subroutine test8 ( array, n, id)
   integer*4 n, id
   real*8    array(n,n,n)  
!
   integer*4 i,j,k,ne
   integer*8 mem,ig, ng
   real*4    gb, sec
   external  sec
!
     ng = n
     ng = ng*ng*ng
     mem = loc(array)
     ig = ((mem/1024)/1024)/1024
     write (*,*) ' '
     gb = real(size(array))*8. / (1024.**3)
     write (*,11) 'Initialising Array ',id,' : gb = ',gb, sec(), ' sec'
11   format (a,i0,a,f0.3,f8.3,a)
     write (*,*) 'n    = ',n           
     write (*,*) 'ng   = ',ng          
     write (*,*) 'size = ',size(array)
     write (*,*) 'loc  = ',loc (array), ig
      do i = 1,n
        do j = 1,n
          do k = 1,n
            array(k,j,i) = i+j+k
          end do
        end do
      end do
!
     write (*,12) 'Checking Array ',id, sec(), ' sec'
12   format (a,i0,f8.3,a)     
      ne = 0
      ng = 0
      do i = 1,n
        do j = 1,n
          do k = 1,n
            ng = ng+1
            if ( array(k,j,i) /= i+j+k ) ne = ne+1
          end do
        end do
      end do
     write (*,13) ng,' elements : ',ne,' errors detected', sec(), ' sec'
13   format (i10,a,i10,a,f10.3,a)
!
   end subroutine test8

  subroutine test2 ( array, n, id)
   integer*4 n, id
   integer*2 array(n,n,n)  
!
   integer*4 i,j,k
   integer*8 ne,ng, mem, ig
   real*4    gb, sec
   external  sec
!
     ng = n
     ng = ng*ng*ng
     mem = loc(array)
     ig = ((mem/1024)/1024)/1024
     write (*,*) ' '
     gb = real(size(array))*2. / (1024.**3)
     write (*,11) 'Initialising Array ',id,' : gb = ',gb, sec(), ' sec'
11   format (a,i0,a,f0.3,f8.3,a)
     write (*,*) 'n    = ',n           
     write (*,*) 'ng   = ',ng          
     write (*,*) 'size = ',size(array)
     write (*,*) 'loc  = ',loc (array), ig
      do i = 1,n
        do j = 1,n
          do k = 1,n
            array(k,j,i) = i+j+k
          end do
        end do
      end do
!
     write (*,12) 'Checking Array ',id, sec(), ' sec'
12   format (a,i0,f8.3,a)     
      ne = 0
      ng = 0
      do i = 1,n
        do j = 1,n
          do k = 1,n
            ng = ng+1
            if ( array(k,j,i) /= i+j+k ) ne = ne+1
          end do
        end do
      end do
     write (*,13) ng,' elements : ',ne,' errors detected', sec(), ' sec'
13   format (i10,a,i10,a,f10.3,a)
!
   end subroutine test2

   real*4 function sec ()
     integer*4 clock, rate
     integer*4 :: start = -1
!
     call system_clock ( clock, rate )
     if ( start < 0) start = clock
     sec = real(clock-start) / real (rate)
   end function sec     

Results are

Initialising Array 1 : gb = 0.009   0.000 sec
 n    =          107
 ng   =               1225043
 size =               1225043
 loc  =              55050304                    0
Checking Array 1   0.014 sec
   1225043 elements :          0 errors detected     0.025 sec
 
Initialising Array 2 : gb = 2.211   0.025 sec
 n    =          667
 ng   =             296740963
 size =             296740963
 loc  =           24324997184                   22
Checking Array 2   2.160 sec
 296740963 elements :          0 errors detected     3.549 sec

Initialising Array 3 : gb = 4.654   3.549 sec
 n    =         1357
 ng   =            2498846293
 size =            2498846293
 loc  =           19327287360                   17
Checking Array 3  15.834 sec
2498846293 elements :          0 errors detected    26.092 sec
 end of test

ctd Results are

Initialising Array 1 : gb = 0.009   0.000 sec
 n    =          107
 ng   =               1225043
 size =               1225043
 loc  =              55050304                    0
Checking Array 1   0.014 sec
   1225043 elements :          0 errors detected     0.025 sec

Initialising Array 2 : gb = 2.211   0.025 sec
 n    =          667
 ng   =             296740963
 size =             296740963
 loc  =           24324997184                   22
Checking Array 2   2.160 sec
 296740963 elements :          0 errors detected     3.549 sec

Initialising Array 3 : gb = 4.654   3.549 sec
 n    =         1357
 ng   =            2498846293
 size =            2498846293
 loc  =           19327287360                   17
Checking Array 3  15.834 sec
2498846293 elements :          0 errors detected    26.092 sec

 end of test

The memory address of common is interesting, as large common start at 22 gb

The 64-bit compiler will be supplied as part of a regular FTN95 release. Customers who have upgrade protection will receive it as part of their service.

Paul,

If you compile the program PRG1 with /debug or /check then two ways exist of error reporting

1. if you just run PRG1 as is, the program will stop and the error will come in a window telling you about offending line number and error message

2. if you run program via debugger: SDBG PRG1 then the Fortran source will be opened at offending line where you can see all other variables too.

I suppose in 64bit mode option 2 with SDBG needs rebuilding of debugger to add functionality but option 1 is the same like it was. Why this functionality currently does not work? The Fortran source sizes are definitely not 4 billion lines long (mine are large but 10000x smaller then that) and the error message text is not related to 64bit

John,

Thanks for the good news. So the older programs can now get up to 32 GB RAM per each single array (4 billion numbers * 8bytes), or 10x more then before with /3GB switch (which also had the limit of 3-4GB total not per each array) and if you have many arrays then the sky is actually unlimited, is this correct ?

Is the moronic word 'stack' burned and spread over the city dumb with this 64bit ?

John C's observation that one has to beware of running out of physical memory probably means using the GlobalMemoryStatusEx function. I think that life would be made easier if Paul could provide a standard Fortran function to do this (or someone else might).

John: is there a performance hit using the 64-bit backend when the program is runnable using 32-bit FTN95?

Dan, The stack is just as big a problem in 64 bit, actually probably more, as it does not take too long to overflow if you are not aware of the causes. That is why I recommend using ALLOCATE for all (large) local arrays. Everything is still finite, just bigger. Actually the first time you run a program that requires more memory than installed, you will be surprised, then very annoyed you did. You will have quite some time to think about this as you will wait a while till the PC returns to normal operation. Eddie's recommendation of Installed_Memory function would be a big help.

Eddie, At present /opt is not supported so there are some performance issues with this beta /64. There is a more general performance issue with /64 as with larger arrays ( lazyness ? ) the run times become more obvious. In general there negligible run time difference between running the same program as /32 or /64, although both have some areas where they perform better (different instruction set). If available I will use 64 bit.

This beta version does not yet have /check or /opt but I am sure it will soon.

I anticipate the big plus with this /64 version is that existing Clearwin programs can be compiled with /64 and get more memory with larger arrays. The main change will be to recognise which (API) handles must be integer*8. Testing clearwin is my next test so I will keep a log of changes required.

John

Things are that some parts of the large codes are written in old style with common blocks and are very reliable, changing them to allicatables may cause hidden roaches to be introduced and cleaned for years. So this transition will take time.

Damn rudiment stack is still there. And is still probably set in bytes so now the compilation will look full line of zeroes...go count them right

SLINK AAA.obj /stack:32000000000

Heck also knows does this mean per array or per total...:-). Kill the stack, Silverfrost!

Quoted from JohnCampbell Dan, The stack is just as big a problem in 64 bit, actually probably more, as it does not take too long to overflow if you are not aware of the causes. That is why I recommend using ALLOCATE for all (large) local arrays. Everything is still finite, just bigger.

I think going from 32-bits to 64-bits don't change the maximum stack size? If I recall correctly the maximum limit for stack size in executable is set by the linker and is the same for 32-bit program and 64-bit program, as it is defined inside the PE format as dword sized 32-bit field.

Over the years the existence of stack caused nothing but the troubles and crashes. Probably it is just poorly implemented or buggy. Plus there is no facilities that tell what is going on with the allocation and how stack is full. OK, how about making at least the option /stack:-1 which will disable all the traces of involvement of stack facilities completely