Largest arrys in the code

Please remind me which compiler switch shows in compilation listing which array is largest in the code? I think it was something like that before. Couldn't find in FTN95/?

By mistake or by other reason I've got some huge array somewhere which does not allow the program to start. Fun is that decreasing some other arrays i get at some point that programs boots OK but decreasing dimensions even more the EXE program again says that it's to large to fit in memory.

Dan,

I do not know of this switch ? I'm not sure I can help with this, but my approach has always been to generate a .map file, import into excel, convert the hex address to decimal and calculate the size of each routine or data element. The problem with this is that there is no report of the stack requirement in each routine. To overcome this I place few variables on the stack, with any large local variable going into common or a module. Both these have a size report in the .map file. Large arrays in a module actually get their own address report. From your description, I would guess that you may have an automatic array with an incorrect dimension. This will not be identified in the .map report FTN95 does not handle large Stack demands well and I have learned to avoid it's use. I am not aware of any switch in FTN95 / Slink to identify stack demand for local variables. An alternative I haven't tried could be to create an /xref and import it into excel, sort out the charf and see if you can parse and calculate the array sizes. After compiling with /xrf, use the following command: find /n /i 'dimension' *.xrf > dimension.tce Import this into excel and try to calculate the sizes.

Hope this might help

John

Holly @$%...thanks ...wow...looks like in the past you've literally came through the hell

John You wrote in the other thread 'With Windows 7_64 ( and XP_64 ) memory between 2gb and 4gb is available using ALLOCATE if compiled as ftn95 program /link. (This is a useful extension to FTN95 when running on 64 bit OS.)'

I understand that's like a straw for a sinking (ideally, compiler allowed us any size arrays with virtual common 64bit Windows permits) but still want to know how exactly you have done that. Can you also make few lines demo code which shows that before you were able to create an array with less than 2 GB only and now you can make either 4GB or two by 2Gb each

Dan,

Here is a simple example, I called big.f95. try running it after using: ftn95 big /link ftn95 big /opt /link ftn95 big /debug /link

The first5 2 options work on my Windows 7-64 OS I am now using.

!  Program to test large arrays
   module big_arrays
      integer*4 n, m
      real*8, dimension(:,:), allocatable :: AA, BB
   end module big_arrays
!
   use big_arrays
!
      integer*4 ii, i, j
      real*8 x
!
!    near 2gb array n = 16000
      n = 16000
      allocate ( aa(n,n), stat = ii)
      x = dble(n)*dble(n) * 8. / 1024./1024.
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating AA(',n,',',n,'); size=',x,'gb : status = ',ii
      write (*,*) 'Address = ', LOC (AA)
!
      do i = 1,n
        do j = 1,n
          aa(i,j) = i+j
        end do
      end do
!
      do i = 1,n
        do j = 1,n
          if (aa(j,i) /= i+j) write (*,*) 'error',j,i
        end do
      end do
!
!    near 1.3gb array m = 13000
      m = 13000
      allocate ( bb(m,m), stat = ii)
      x = dble(m)*dble(m) * 8. / 1024./1024.
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating BB(',m,',',m,'); size=',x,'gb : status = ',ii
!
      end

This will give you 3.3 gb of arrays that can be accessed by using this module big_arrays. You could test further by using the arrays for calculations. You can not allocate a single array of size 2gb or more. The largest one is callocated above 2gb, while the second is below 2gb. You can use LOC to report the address. I also have a program which locates and reports the size of all available memory. I could post it later if required.

John

John, thanks. Do i have to modify something in Windows (boot.ini or better in some another place since my boot.ini is corrupt by installation of XP after Win7_64) because it does not allow me to load it?

P.S. Update: i decreased little bit array size and seems it started to work...investigating now...Wondering if any similar tricks with static arrays like ones in common blocks possible?

Dan,

It is good that you have got it working, but I am surprised that you needed to change the array sizes. I did test big.f95 only on Windows7. As for boot.ini, it appears that both XP-64 and Windows 7_64 automatically have the '3gb' switch available, so no change required. SLINK defaults to /3gb. I have developed a win32 program to scan for all available memory. It uses ALLOCATE and POINTERS, as you can re-allocate a pointer array. I have run it using Salford, Lahey and Intel 32 compilers, on XP-32, XP-64 and Windows 7_64. It produces interesting results, indicating how the linker manages the 'system bits' that upset getting large arrays. As it is a small program, it does not have a big stack demand, so using this large allocated array approach in a large program would have to cope with that. My old Lahey compiler does not provide arrays above 2gb. FTN95 /debug also cancels out access. The program is too big to post so if you send me pm and email, I can send you the program and a dump from the 3 OS I have available. I've made the program a bit non-standard, using B' ' format to make the addresses more readable. Memory address via LOC is also non-standard. The advantage of using this '/4gb' with FTN95 I now get access to a single 2gb array in a module. It does offer some impovement. Windows 7 is also a big improvement, as it's disk cacheing is a big improvement on XP. Strangely, I've never tested Visa !

John

For those interested, I have tested 3 O/S : XP_32, XP_64 and Win7_64, and 2 32 bit compilers : Salford Ver 6.10 and Lahey Ver 5.55 (very old)

The following table lists the size of the largest available allocatable arrays available (in bytes), being: Largest array below 2gb Second largest array below 2gb Largest array above 2gb

Summary of Largest Available Blocks			
O/S and Compiler        Largest block Second largest  Largest block
                            below 2gb   block below       above 2gb
Win 7_64 Salford V6.10  1,615,921,116   182,648,796   2,147,221,468
XP_64 Salford V6.10       995,098,588   528,613,340   2,147,221,468
XP_32 Salford V6.10     1,290,076,124   305,922,012   not available
Win7_64 Lahey V5.55     1,803,550,624   113,110,944   not available
XP_64 Lahey V5.55       1,757,413,280   113,110,944   not available

Note for XP_32 or XP_64, using FTN95 it is also possible to declare a larger common below 2gb of size about 1.6gb, although it is difficult to estimate the size that will not clash with the stack. All thses array sizes are based on the small test program I used. How this interacts with the stack for general use may vary the available size. Win 7_64 and FTN95 offers a significant increase in the size of the largest array, either below or above 2gb, in comparison to XP.

John

John,

While Silverfrost 64bit compiler remains to be a pipe dream for people on all continents, I rewrote my large hydrocode according to your demo example above but leaving some arrays static while converting most larger ones into allocatable. Converting everything into allocatables will be too much work and headache. The hope was to boost array dimensions a bit but i got my code crashing or not loading at almost exactly the same array dimension limits as before with static arrays.

Any ideas what could be wrong? Your code demo above shows i should get almost twice more memory. The code during the run loads not 1-2 large arrays but dozens of smaller from KBs to 10, 100, 300, 500 MB in size with static and dynamic ones filled randomly and intermittently.

Interesting also is that having arrays both static and dynamic you will get around 4GB (more precisely 3.8 GB) which is almost exactly twice the maximum for old static code. For example changing the code above a bit you will get for static and dynamic arrays ranks 15700-15800. Making dynamic a bit larger, say up to 16000 works too but slows dynamic part of the code ~3 times which is specifically well visible in task manager which shows dynamically how arrays are filled. Here is full demo code with the changes, just for simplicity and convenience of demonstration.

! ftn95 big /link 
! ftn95 big /opt /link 
! ftn95 big /debug /link

!  Program to test large arrays 
    module big_arrays 
       integer*4 n, m 
       real*8, dimension(:,:), allocatable :: BB 
    end module big_arrays 
 ! 
    use big_arrays 
 ! 

       integer*4 ii, i, j 
       real*8 x 
       real*8 aa
       parameter (nn=15700) 	
       common /asdasda/aa(nn,nn)
! 
!    near 1.9 GB array n = 15700 
       n = nn  
       
!! not used      allocate ( aa(1:n,1:n), stat = ii) 

       x = dble(n)*dble(n) * 8. / 1024./1024. 
       write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Filling static array AA(',n,',',n,'); size=',x,'MB  ' 
       write (*,*) 'Address = ', LOC (AA) 
! 
       do i = 1,n 
         do j = 1,n 
           aa(i,j) = i+j 
         end do 
       end do 
! 
       do i = 1,n 
         do j = 1,n 
           if (aa(j,i) /= i+j) write (*,*) 'error',j,i 
         end do 
       end do 
! 
!    near 1.9gb array m = 15800 dynamic 
!
	write(*,*) 'press any key to continue with allocatable array BB'
	READ(*,*, err=999)zzz
999     m = 15800 ! 16000 slows the code
       allocate ( bb(m,m), stat = ii) 
       x = dble(m)*dble(m) * 8. / 1024./1024. 
       write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating BB(',m,',',m,'); size=',x,'MB : Err.status = ',ii 


       do i = 1,m 
         do j = 1,m 
           bb(i,j) = i+j 
         end do 
       end do 
 ! 
       do i = 1,m 
         do j = 1,m 
           if (bb(j,i) /= i+j) write (*,*) 'error',j,i 
         end do 
       end do 


	write(*,*) 'press any key to end'
	READ(*,*,err=1000)zzz
1000	continue	
 ! 
       end

,

Dan,

I notice that the large array that is allocated has a starting address below 2gb, so you may be able to allocate only 1 large array beyond 2gb. You would need to make it large so that it went beyond 2gb. I have not tried to split it up, when I call subroutines, giving an address above 2gb. I have only tested the whole array, which has a starting address below 2gb in the subroutine call. You could try allocating 3 arrays of 0.9b and see if any address is above 2gb. My aim was to allocate this large 1.9gb array and split it up using 'fortran 66' memory approach, but have not fully tested this yet. I will see if I can get a small test example from big.f95.

Needs more work to identify the limit.

John

Dan,

Sorry about the last answer, I misread your post. If you allocate a big array in common, this forces SLINK to move all the system bits to the end of the first 2gb of memory, but if you don't then the syetem bits are spread through the first 2gb. This is illustrated in the list of largest available array in 0-2gb.

As for my earlier answer, I modified BIG.f95 on Windows 7 and it shows that yo can allocate large arrays with an address above 2gb. My changed code is listed below (hopefully in full)

John

!  Program to test large arrays 
   module big_arrays 
      integer*4 n, m
      real*8, dimension(:,:), allocatable :: AA, BB, CC
   end module big_arrays 
! 
   use big_arrays 
! 
      integer*4 ii, nn, mm
      real*8 x 
! 
!    near 2gb array n = 16000 
      n = 16000 
      allocate ( aa(n,n), stat = ii) 
      x = dble(n)*dble(n) * 8. / 1024./1024. 
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating AA(',n,',',n,'); size=',x,'gb : status = ',ii 
      write (*,*) 'Address = ', LOC (AA) 
! 
!    near 1.3gb array m = 13000 
      m = 13000 
      allocate ( bb(m,m), stat = ii) 
      x = dble(m)*dble(m) * 8. / 1024./1024. 
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating BB(',m,',',m,'); size=',x,'gb : status = ',ii 
      write (*,*) 'Address = ', LOC (BB) 
! 
!  Test full arrays
      call set_array   (aa(1,1), n,n)
      call set_array   (bb(1,1), m,m)
      call check_array (aa(1,1), n,n)
      call check_array (bb(1,1), m,m)
! 
!  Now test part of arrays
      nn = n/2
      mm = m/2
      call set_array (aa(1,1), n,nn)
      call set_array (aa(1,nn+1),n,nn)     ! has address > 2gb
      call set_array (bb(1,1), m,mm)
      call set_array (bb(1,mm+1),m,mm)
!
      call check_array (aa(1,1), n,nn)
      call check_array (bb(1,1), m,mm)
      call check_array (aa(1,nn+1),n,nn)   ! has address > 2gb
      call check_array (bb(1,mm+1),m,mm)
!
      deallocate (aa)
      allocate ( aa(n,nn), stat = ii) 
      x = dble(n)*dble(nn) * 8. / 1024./1024. 
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating AA(',n,',',nn,'); size=',x,'gb : status = ',ii 
      write (*,*) 'Address = ', LOC (AA) 
!
      allocate ( cc(n,nn), stat = ii) 
      x = dble(n)*dble(nn) * 8. / 1024./1024. 
      write (*,fmt='(a,i0,a,i0,a,f0.2,a,i0)') ' Allocating CC(',n,',',nn,'); size=',x,'gb : status = ',ii 
      write (*,*) 'Address = ', LOC (CC) 
!
      call set_array (aa(1,1), n,nn)
      call set_array (cc(1,1),n,nn)         ! has address > 2gb
      call check_array (aa(1,1), n,nn)
      call check_array (cc(1,1), n,nn)      ! has address > 2gb

      end

overflowed again

      subroutine set_array (aa, n,nn)
      integer*4 n,nn, i, j
      real*8    aa(n,nn)
!
      write (*,*) ' Setting array',n,nn,' at address',loc(aa)
      do i = 1,n 
        do j = 1,nn 
          aa(i,j) = i+j 
        end do 
      end do 
!
      end

      subroutine check_array (aa, n,nn)
      integer*4 n,nn, i, j, e
      real*8    aa(n,nn)
!
      write (*,*) ' Checking array',n,nn,' at address',loc(aa)
      e = 0
      do i = 1,nn 
        do j = 1,n
          if (aa(j,i) == i+j) cycle
!          write (*,*) 'error',j,i 
          e = e+1
        end do 
      end do 
      write (*,*) '  error check for array',n,nn,' errors=',e
!
      end

I see. So placing some static arrays into common and turning others into allocatables should always guarantee us double memory space, am i correct? Then I will play more with different variants to reproduce in small demo code the error i see with large hydrocode.

But may be i do not have any errors and all just works and i get completely non-related error. The beauty is that the hydrocode now loads with no problem with double array sizes but i get some error during the run and i do not know what kind this error is. I afraid that this is the Achill heel of this method: the CHECK mode simply does not work: if error occurs in NOCHECK mode and you try to recompile the code with the CHECK or DEBUG switches then any error get me fatal message 'Salford Run time library. Insufficient memory available for CHECK mode. Fatal run time error'. Will play more with this too but most probably this will be the end of story for this nice trick if compiler developers will not look at that. The compiler without debugger is useless

CHECK mode has additional memory limitations. DEBUG mode does not have additional memory limitations but you must use /3gb on the SLINK command line to get all the memory currently available to FTN95/SLINK. At the moment you cannot use /LINK /DEBUG on the FTN95 command line without limiting the memory but I have changed this for next release.

Wow ! All works! Common+allocatables together get 3.8GB with ease. In addition now my code is ready for future 64bit compiler (hopefully... if we will see one... if not the code still became more universal, will need less or even no recompilations with all these rebuilds i've made incorporating allocatables). Two months of nightmare to squeeze some last final bits of atomic data into existing code or thinking how to rebuild everything ended. John and Paul, PM me how better send you my thanks with a cases of beer.

By the way, John, do you see the slowdown like this? That's what task manager shows at two values of array ranks. It could be in your case at slightly different value (i have RAMDisk also installed)

http://img20.imageshack.us/img20/9832/campbellvar216000.png