forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Do we need to zeroise allocated array ?
Goto page 1, 2  Next
 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
DanRRight



Joined: 10 Mar 2008
Posts: 2105
Location: South Pole, Antarctica

PostPosted: Sat Oct 05, 2019 12:53 am    Post subject: Do we need to zeroise allocated array ? Reply with quote

or it is still in arbitrary state?

If not guaranteed being at least zeroised after allocation would be nice to have some kind of new ALLOCATE with initialization at the same time for simplicity.
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6075
Location: Salford, UK

PostPosted: Sat Oct 05, 2019 6:45 am    Post subject: Reply with quote

Calls to ALLOCATE from FTN95 currently automatically zeroise the memory. This may not be portable to other compilers. The feature is not documented, however, there is no reason to think that it might change.

But if you use /UNDEF (or any option that implies /UNDEF) then the memory is initialised for "undef" checking.
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2144
Location: Sydney

PostPosted: Sat Oct 05, 2019 2:43 pm    Post subject: Reply with quote

Paul,

I would question this claim as my understanding is ALLOCATE does not actually allocate the memory pages for the array, but merely reserves memory address range for the array.

It is not until the array is used that the memory is allocated. See the example in the recent "Size of all arrays"

http://forums.silverfrost.com/viewtopic.php?t=4082&postdays=0&postorder=asc&start=0

I could modify this program to test the values of the array before it is initialised. However, I don't think it is standard conforming to use an uninitialised array.
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6075
Location: Salford, UK

PostPosted: Sat Oct 05, 2019 3:12 pm    Post subject: Reply with quote

John

Can you provide an example. When I trace ALLOCATE (without /UNDEF) it leads to a call to HeapAlloc using HEAP_ZERO_MEMORY. This is for 64 bits with a similar call for 32 bits. The documentation for HEAP_ZERO_MEMORY says "The allocated memory will be initialized to zero".

This is not a proof but an illustration...

Code:
integer,allocatable::ix(:)
allocate(ix(10))
print*, ix
end
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1227
Location: Aerospace Valley

PostPosted: Sat Oct 05, 2019 5:59 pm    Post subject: Reply with quote

Paul wrote:

Quote:
Calls to ALLOCATE from FTN95 currently automatically zeroise the memory. This may not be portable to other compilers.

As good a reason as any for ensuring ALL variables are zeroised explicitly within the program !

Quote:
The feature is not documented, however, there is no reason to think that it might change.

Not exactly a robust justification for non-documentation !

Quote:
But if you use /UNDEF (or any option that implies /UNDEF) then the memory is initialised for "undef" checking.

So, are you saying it's only zeroised when /UNDEF is used ? ... which is what the presence of 'But' at the beginning of the sentence implies, even though I read the first sentence as that just by using ALLOCATE the zeroising is automaticaally carried out. And which is clearly as per you your 2nd comment about the MS documentation for HEAP ALLOCATION also.

My Conclusions:

1. Users: just like for IMPLICIT NONE, keep on initialising EVERYTHING yourself whether you decide to use the 'time saving' global features or not, that way you have no one but yourself to blame if anything isn't initialised to your taste ! Wink
2. all this stuff needs to be clearly documented !
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2049
Location: Yateley, Hants, UK

PostPosted: Sat Oct 05, 2019 6:37 pm    Post subject: Reply with quote

John-Silver,

When /UNDEF is used, presumably the memory isn't zeroised, it is all set to a value that signifies 'undefined'.

I agree that the historic requirement that the programmer should explicitly provide an initial value for everything before first use is a good practice, and in particular, aids in transferring things to a different compiler. However, the minute you start using Clearwin+, you can't change.

As well as documentation for FTN95, there is also the matter of documenting one's programs, either with comments, or with paper. Perhaps if one wanted to take advantage of automatic zeroisation, it might be worth doing a check. Just how many locations would one need to test (based on statistics) to be confident that the array was, in fact, zeroed?

Eddie
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2144
Location: Sydney

PostPosted: Sun Oct 06, 2019 5:50 am    Post subject: Reply with quote

Paul,

I adapted the previous allocate arrays test I referred to by:
- after allocating the arrays, counted the non-zero values.
- did the test twice, after changing memory to non-zero in first pass

The results are :
1) either initialising (previous test) or testing values results in the memory pages being allocated.
2) all arrays have been initially zero, as you indicated.

Would this only apply to ALLOCATE (heap arrays) only or also stack arrays (local, automatic) or common also ? ( I guess I should check )

Both FTN95 (/32), FTN95 /64 and gFortran perform in this way.

However, I would never use this assumption

John

The revised program:
Code:
!  program to first allocate big arrays, then initialise big arrays
!   requires FTN95 Ver 8.5 to work with allocatable arguments
!   
module sz
      real*4    :: sz_mb          ! array size in mb
      real*4    :: sz_ni          ! array elements
      integer*4 :: ni             ! dimension of array
end module sz
!
   program use_memory
     use sz
!     
      interface
        subroutine allocate_array ( aa )
         use sz
          integer*4, allocatable, dimension(:,:) :: aa
        end subroutine allocate_array
      end interface
!
      integer*4, allocatable, dimension(:,:) :: a,b,c,d,e,f,g,h
      integer*4 :: i,j, pass
!
      sz_mb = 500                       ! MBytes
!
      call prompt ('What array size to test (in MBytes, default 500) ?', j)
      if ( j > 1 .and. j < 2048 ) sz_mb = j
      sz_ni = sz_mb * 1024.**2 / 4
      ni    = sqrt (sz_ni)
      write (*,*) '2D I*4 array dimension =',ni
!
    do pass = 1,2
!  ALLOCATE all arrays
      do i = 1,9
        select case (i)
          case (1)
            call allocate_array ( a )
          case (2)
            call allocate_array ( b )
          case (3)
            call allocate_array ( c )
          case (4)
            call allocate_array ( d )
          case (5)
            call allocate_array ( e )
          case (6)
            call allocate_array ( f )
          case (7)
            call allocate_array ( g )
          case (8)
            call allocate_array ( h )
          case default
            exit
        end select
        write (*,*) i, ' arrays allocated: check task manager'
        call prompt ('Proceed ?',j)
      end do
!
      write (*,*) 'all allocates done'
      write (*,*) '"Comit Size" should have increased but NOT "Working Set"'
        call prompt ('Proceed ?',j)
      write (*,*) 'now to utilise memory !'
!
!  TEST non-zero arrays
      do i = 1,9
        select case (i)
          case (1)
            call test_non_zero ( a )
          case (2)
            call test_non_zero ( b )
          case (3)
            call test_non_zero ( c )
          case (4)
            call test_non_zero ( d )
          case (5)
            call test_non_zero ( e )
          case (6)
            call test_non_zero ( f )
          case (7)
            call test_non_zero ( g )
          case (8)
            call test_non_zero ( h )
          case default
            exit
        end select
        write (*,*) i, ' arrays tested for non-zero: check task manager'
        call prompt ('Proceed ?',j)
      end do
!
      write (*,*) 'all non-zero tests done : now to initialise'
        call prompt ('Proceed ?',j)
!
!  INITIALISE all arrays
      do i = 1,9
        select case (i)
          case (1)
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2144
Location: Sydney

PostPosted: Sun Oct 06, 2019 5:52 am    Post subject: Reply with quote

program ctd ..
Code:
          case (1)
            call initialise_array ( a, i )
          case (2)
            call initialise_array ( b, i )
          case (3)
            call initialise_array ( c, i )
          case (4)
            call initialise_array ( d, i )
          case (5)
            call initialise_array ( e, i )
          case (6)
            call initialise_array ( f, i )
          case (7)
            call initialise_array ( g, i )
          case (8)
            call initialise_array ( h, i )
          case default
            exit
        end select
        write (*,*) i, ' arrays initialised: check task manager'
        call prompt ('Proceed ?',j)
      end do
!
      write (*,*) 'all arrays initialised : now to test correct setting'
        call prompt ('Proceed ?',j)
!
!  TEST all arrays that they have been initialised correctly
      do i = 1,9
        select case (i)
          case (1)
            call test_array ( a, i )
          case (2)
            call test_array ( b, i )
          case (3)
            call test_array ( c, i )
          case (4)
            call test_array ( d, i )
          case (5)
            call test_array ( e, i )
          case (6)
            call test_array ( f, i )
          case (7)
            call test_array ( g, i )
          case (8)
            call test_array ( h, i )
          case default
            exit
        end select
        if ( i < 9) write (*,*) i, ' arrays tested: check task manager'
        call prompt ('Proceed ?',j)
      end do
!
      if ( pass == 1 ) then
        write (*,*) 'all arrays tested so done for first pass; what is the memory use now'
        call prompt ('Proceed ?',j)
!
        deallocate ( a, b, c, d, e, f, g, h )
        write (*,*) 'all arrays deallocated; check memory then proceed to retest'
        call prompt ('Proceed ?',j)
!
      else
        write (*,*) 'all tests now repeated : How did we go : final ? before exiting'
        call prompt ('Proceed ?',j)
      end if
    end do !  pass
   
   end program use_memory

   subroutine allocate_array ( aa )
     use sz
      integer*4, allocatable, dimension(:,:) :: aa
      integer*4 :: stat
      allocate ( aa(ni,ni), stat=stat )
      write (*,*) 'array allocated : stat=',stat                 
   end subroutine allocate_array

   subroutine initialise_array ( aa, ii )
     use sz
      integer*4 :: aa(ni,ni), ii
      integer*4 :: i,j, n
      n = 0
      do j = 1,ni
        do i = 1,ni
          aa(i,j) = i+j+ii
          n = n+1
        end do
      end do
      write (*,*) 'array initialised',n
   end subroutine initialise_array

   subroutine test_array ( aa, ii )
     use sz
      integer*4 :: aa(ni,ni), ii
      integer*4 :: i,j, n, e
      n = 0
      e = 0
      do j = 1,ni
        do i = 1,ni
          if ( aa(i,j) /= i+j+ii ) e = e+1
          n = n+1
        end do
      end do
      write (*,*) 'array tested',n, ' :',e,' errors'
   end subroutine test_array   
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2144
Location: Sydney

PostPosted: Sun Oct 06, 2019 5:54 am    Post subject: Reply with quote

program ctd ..
Code:
   
   subroutine test_non_zero ( aa )
     use sz
      integer*4 :: aa(ni,ni)
      integer*4 :: i,j, n, e
      n = 0
      e = 0
      do j = 1,ni
        do i = 1,ni
          if ( aa(i,j) /= 0 ) e = e+1
          n = n+1
        end do
      end do
      write (*,*) 'array tested',n, ' :',e,' non-zero'
   end subroutine test_non_zero 
   
   subroutine prompt (string, j)
      character string*(*)
      integer j, iostat
      write (*,fmt='(a)',advance='NO') string
      read  (*,fmt='(bn,i6)', iostat=iostat) j
      if (iostat /= 0) j = -1
   end subroutine prompt
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6075
Location: Salford, UK

PostPosted: Sun Oct 06, 2019 7:57 am    Post subject: Reply with quote

John

Initially you doubted my statement. Is this still the case?
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2144
Location: Sydney

PostPosted: Sun Oct 06, 2019 8:40 am    Post subject: Reply with quote

Paul,

I have changed my tune and no longer doubt you !

Now I understand that FTN95 does initialise allocated arrays.
I also found that with FTN95 /64 that large automatic arrays (that go onto the heap) are also initialised. 64-bit gFortran also appears to have this behaviour.

I tried to test local and automatic arrays for 32-bit, but got a stack overflow (and gave up on this "ugly" approach).
I am not sure what would happen for smaller arrays that do go onto the stack.

However, I have always considered it necessary to initialise an array or variable before it is used, so I will not adopt this "lazy" approach.
I am not sure if first initialising an array is standard conforming, but it should be. I think Eddie agrees with me this time ?

John

PS: the following adaptation appears to show small arrays in the stack are not initialised.
Code:
!  program to test automatic arrays for initialisation
!   
module sz
      real*4    :: sz_mb          ! array size in mb
      real*4    :: sz_ni          ! array elements
      integer*4 :: ni             ! dimension of array
end module sz
!
   program use_memory
     use sz
!     
      integer*4 :: j
!
      sz_mb = 5                       ! MBytes
!
      call prompt ('What array size to test (in MBytes, default 5) ?', j)
      if ( j > 1 .and. j < 2048 ) sz_mb = j
!
      sz_ni = sz_mb * 1024.**2 / 4
      ni    = sqrt (sz_ni)
      write (*,*) '2D I*4 array dimension =',ni
!
      call test_automatic_array (ni)
!
   end program use_memory

   subroutine test_automatic_array (ni)
!
      integer*4 :: ni
!
      integer*4 aa(ni,ni)    ! automatic array
      integer*4 :: pass, i

    do pass = 1,2
!
      i = pass
!
!  Start test for this pass
        write (*,*) 'Automatic array now on stack'
        write (*,*) '"Comit Size" should have increased but NOT "Working Set"'
        call prompt ('Proceed ?',j)
!
        write (*,*) 'now to utilise memory !'
!
!  TEST non-zero arrays
        call test_non_zero ( aa )
!
        write (*,*) 'all non-zero tests done : now to initialise'
        call prompt ('Proceed ?',j)
!
!  INITIALISE array
        call initialise_array ( aa, i )
!
        write (*,*) 'all arrays initialised : now to test correct setting'
        call prompt ('Proceed ?',j)
!
!  TEST array that it has been initialised correctly
        call test_array ( aa, i )
!
!  End this pass
        if ( pass == 1 ) then
          write (*,*) 'all arrays tested so done for first pass; what is the memory use now'
          call prompt ('Proceed ?',j)
!
        else
          write (*,*) 'all tests now repeated : How did we go : final ? before exiting'
          call prompt ('Proceed ?',j)
        end if

    end do !  pass

   end subroutine test_automatic_array   
   
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2049
Location: Yateley, Hants, UK

PostPosted: Sun Oct 06, 2019 9:36 am    Post subject: Reply with quote

John,

As an 'old dog', the idea that anything is initialised without my express intervention is a 'new trick', and it is my habit to initialise everything.

It's brilliant if things are initialised, but think that this ought to be in response to one of the following:

(a) a compiler directive
(b) a command (that doesn't exist) in standard Fortran
or
(c) a programmed test to see if the compiler did or did not have this as a default.

IN any case, the assumption needs to be commented.

On the other hand, since I consider anyone using anything other than FTN95 as somewhat deranged, because CW+ doesn't exist properly elsewhere, then perhaps it is OK to assume that FTN95's behaviour is the norm.

It still needs a comment in the source code!

Eddie
Back to top
View user's profile Send private message
DanRRight



Joined: 10 Mar 2008
Posts: 2105
Location: South Pole, Antarctica

PostPosted: Sun Oct 06, 2019 2:42 pm    Post subject: Reply with quote

Great that allocated arrays are zeroised. Life will be a bit easier with my 140 arrays which need allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise all the time.

Also looks like lazy allocation of sparse arrays in COMMON becomes the necessity of the past. If keeping lazy allocation is the reason of slow performance of compiler with Poly tests, then may be it should be abandoned. Or kept as a non-standard option
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1227
Location: Aerospace Valley

PostPosted: Sun Oct 06, 2019 6:53 pm    Post subject: Reply with quote

I guess whn Paul gets the time to get around to investigating how to 'imp rove' ftn95 performance of the polyhedron tests he'll surely look into the effect of using the 'lazy' method v traditional methods.
There are however surely large number of possible parameters in play there, at least until the one(s) affecing the performance are identified !
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1227
Location: Aerospace Valley

PostPosted: Sun Oct 06, 2019 6:57 pm    Post subject: Reply with quote

Eddie,
Quote:
When /UNDEF is used, presumably the memory isn't zeroised, it is all set to a value that signifies 'undefined'.


that's a question for Paul I think.

JohnC, did you run your test program with/without /UNDEF ?
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group