forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

UNDEF causing allocation problems

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
DanRRight



Joined: 10 Mar 2008
Posts: 2662
Location: South Pole, Antarctica

PostPosted: Tue Jan 03, 2023 7:57 pm    Post subject: UNDEF causing allocation problems Reply with quote

This 64bit program allocates OK till whatever array size you want (0.4, 4, 40GB with nB=10^7, 10^8 etc) if compiled without /UNDEF but crashes at miniscule size when compiled with /UNDEF at nB > 9.e6. The limit size is resembling the one of previous generation 32bit compiler ~10^8 elements

Code:
! compilation: ftn95 aaa.f90 /link /64 /debug /undef
!
    real*4, dimension(:,:), allocatable :: Arr4

    nA = 10
    nB = 9.e6

!.. Allocating array
    Print*, 'Trying to allocate GB of RAM :', 1.d-9 * 4. * nA * nB
    allocate ( Arr4 (nA, nB), stat = ierr)

    if (ierr.eq.0) then
       pause 'Allocation success. End Program'
    else
       Pause 'Failed to allocate. End Program'
    endif

    End
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2450
Location: Sydney

PostPosted: Wed Jan 04, 2023 5:47 am    Post subject: Reply with quote

Dan,

Without /UNDEF, this program will allocate successfully until the size of Arr4 exceeds your virtual memory size limit.
As you never access / "touch" Arr4, it will never be allocated physical memory pages (or virtual memory pages).

If you included "Arr4 = 1" then all of the array will be allocated memory pages, and when the array size exceeds your physical memory size, your PC will look as if it has stopped (while it is shuffling memory pages to your paging disk/file).

If you included "Arr4(:,1) = 1" then only 1 memory page will be allocated for the array, while the rest of the untouched array will not have allocated memory pages. This will continue until the array size exceeds the virtual array size limit.

The tricky bit is "what is the virtual array size limit"
I think this is the sum of physical memory size plus paging space size, rather than the virtual address limit of Win64 OS.

You could put all this in a do loop, where you increase the size of nB and run it while watching Task Manager, which should show no memory required while Arr4 is not touched. (put a pause in your program at the end to see it in task manager)

You can do further tricks to only touch a subset of the array and see only this subset of required array memory being allocated, eg
Code:

   do j = 1,nB,100000
       Arr4(:,j) = 1
   end do

It is rare that this type of array use is available.

The practicality is, if you use "Arr4 = 1", once you exceed physical memory, your computer will look to stop, as it will start madly copying your array to the pagfile.sys. Remember 40 years ago, when all OS ran on a paging disk and we accepted these 1000 x slower performance.

Try this example, which stops when it exceeds memory + paging ( on my pc of 64 GB mem + 32 GB pagefile.sys goes to 128 GB, rather than 96 GB, so confusing ? )

Code:
! compilation: ftn95 aaa.f90 /link /64 /debug
!
    real*4, dimension(:,:), allocatable :: Arr4
    integer, parameter :: million = 1000000
    integer :: nA, nB, ierr,j
    character :: answer

    nA = 1000
    do nB = million, 50*million, million

   !.. Allocating array
       Print*, 'Trying to allocate GB of RAM :', 1.d-9 * 4. * nA * nB
       allocate ( Arr4 (nA, nB), stat = ierr)
   
       if (ierr.eq.0) then
          write (*,*) 'Allocation success. End Program'
       else
          Pause 'Failed to allocate. End Program'
          exit
       endif
       do j = 1,nB, (million/10)
         arr4(:,j) = 1
       end do
   
       write (*,*) ' next size ?'
       read (*,*) answer

       deallocate ( Arr4 )

    end do
    End


Last edited by JohnCampbell on Wed Jan 04, 2023 6:23 am; edited 2 times in total
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2450
Location: Sydney

PostPosted: Wed Jan 04, 2023 6:11 am    Post subject: Reply with quote

If running the above program with task manager, show both "Commit size" and "Working Set"
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7550
Location: Salford, UK

PostPosted: Wed Jan 04, 2023 11:03 am    Post subject: Reply with quote

This failure has been added to the list for investigation.
Back to top
View user's profile Send private message AIM Address
DanRRight



Joined: 10 Mar 2008
Posts: 2662
Location: South Pole, Antarctica

PostPosted: Wed Jan 04, 2023 7:17 pm    Post subject: Reply with quote

John,
But does my code above crash or not?

It seems to me this is very strange crash. Several times lately i also have error reported many thousands of lines off its actual place.

I can not experiment right now, my computer is overloaded with numerous things and some other problems (Virtual Machines also crash). Will do that when the whole comp gets belly up by crash or power outage due to often extreme weather lately Smile
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7550
Location: Salford, UK

PostPosted: Wed Jan 04, 2023 9:38 pm    Post subject: Reply with quote

It fails for me and I have logged it for investigation.
Back to top
View user's profile Send private message AIM Address
DanRRight



Joined: 10 Mar 2008
Posts: 2662
Location: South Pole, Antarctica

PostPosted: Thu Jan 05, 2023 6:31 am    Post subject: Reply with quote

Wow, this looks almost like a devilry !
I am probably the largest fan of UNDEF, this is the most important debug super-tool, i use it for 30+ years, and last several years with 64-bits. And so far UNDEF worked no problem! This crash left me guessing what i could change that triggered these crashing...
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7550
Location: Salford, UK

PostPosted: Mon Jan 09, 2023 8:10 am    Post subject: Reply with quote

This bug has now been fixed for the next release of FTN95.
Back to top
View user's profile Send private message AIM Address
DanRRight



Joined: 10 Mar 2008
Posts: 2662
Location: South Pole, Antarctica

PostPosted: Tue Jan 10, 2023 5:58 am    Post subject: Reply with quote

Thanks, Paul
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group