Silverfrost Forums

Welcome to our forums

UNDEF causing allocation problems

3 Jan 2023 6:57 #29723

This 64bit program allocates OK till whatever array size you want (0.4, 4, 40GB with nB=107, 108 etc) if compiled without /UNDEF but crashes at miniscule size when compiled with /UNDEF at nB > 9.e6. The limit size is resembling the one of previous generation 32bit compiler ~10^8 elements

! compilation: ftn95 aaa.f90 /link /64 /debug /undef 
!
    real*4, dimension(:,:), allocatable :: Arr4

    nA = 10
    nB = 9.e6

!.. Allocating array
    Print*, 'Trying to allocate GB of RAM :', 1.d-9 * 4. * nA * nB
    allocate ( Arr4 (nA, nB), stat = ierr)

    if (ierr.eq.0) then
       pause 'Allocation success. End Program'
    else
       Pause 'Failed to allocate. End Program'
    endif 

    End
4 Jan 2023 4:47 (Edited: 4 Jan 2023 5:23) #29724

Dan,

Without /UNDEF, this program will allocate successfully until the size of Arr4 exceeds your virtual memory size limit. As you never access / 'touch' Arr4, it will never be allocated physical memory pages (or virtual memory pages).

If you included 'Arr4 = 1' then all of the array will be allocated memory pages, and when the array size exceeds your physical memory size, your PC will look as if it has stopped (while it is shuffling memory pages to your paging disk/file).

If you included 'Arr4(:,1) = 1' then only 1 memory page will be allocated for the array, while the rest of the untouched array will not have allocated memory pages. This will continue until the array size exceeds the virtual array size limit.

The tricky bit is 'what is the virtual array size limit' I think this is the sum of physical memory size plus paging space size, rather than the virtual address limit of Win64 OS.

You could put all this in a do loop, where you increase the size of nB and run it while watching Task Manager, which should show no memory required while Arr4 is not touched. (put a pause in your program at the end to see it in task manager)

You can do further tricks to only touch a subset of the array and see only this subset of required array memory being allocated, eg

   do j = 1,nB,100000
       Arr4(:,j) = 1
   end do

It is rare that this type of array use is available.

The practicality is, if you use 'Arr4 = 1', once you exceed physical memory, your computer will look to stop, as it will start madly copying your array to the pagfile.sys. Remember 40 years ago, when all OS ran on a paging disk and we accepted these 1000 x slower performance.

Try this example, which stops when it exceeds memory + paging ( on my pc of 64 GB mem + 32 GB pagefile.sys goes to 128 GB, rather than 96 GB, so confusing ? )

! compilation: ftn95 aaa.f90 /link /64 /debug
!
    real*4, dimension(:,:), allocatable :: Arr4
    integer, parameter :: million = 1000000
    integer :: nA, nB, ierr,j
    character :: answer

    nA = 1000
    do nB = million, 50*million, million

   !.. Allocating array
       Print*, 'Trying to allocate GB of RAM :', 1.d-9 * 4. * nA * nB
       allocate ( Arr4 (nA, nB), stat = ierr)
   
       if (ierr.eq.0) then
          write (*,*) 'Allocation success. End Program'
       else
          Pause 'Failed to allocate. End Program'
          exit
       endif
       do j = 1,nB, (million/10)
         arr4(:,j) = 1
       end do
   
       write (*,*) ' next size ?'
       read (*,*) answer

       deallocate ( Arr4 )

    end do
    End
4 Jan 2023 5:11 #29725

If running the above program with task manager, show both 'Commit size' and 'Working Set'

4 Jan 2023 10:03 #29726

This failure has been added to the list for investigation.

4 Jan 2023 6:17 #29731

John, But does my code above crash or not?

It seems to me this is very strange crash. Several times lately i also have error reported many thousands of lines off its actual place.

I can not experiment right now, my computer is overloaded with numerous things and some other problems (Virtual Machines also crash). Will do that when the whole comp gets belly up by crash or power outage due to often extreme weather lately 😃

4 Jan 2023 8:38 #29732

It fails for me and I have logged it for investigation.

5 Jan 2023 5:31 #29734

Wow, this looks almost like a devilry ! I am probably the largest fan of UNDEF, this is the most important debug super-tool, i use it for 30+ years, and last several years with 64-bits. And so far UNDEF worked no problem! This crash left me guessing what i could change that triggered these crashing...

9 Jan 2023 7:10 #29780

This bug has now been fixed for the next release of FTN95.

10 Jan 2023 4:58 #29799

Thanks, Paul

Please login to reply.