|
forums.silverfrost.com Welcome to the Silverfrost forums
|
View previous topic :: View next topic |
Author |
Message |
DanRRight
Joined: 10 Mar 2008 Posts: 2826 Location: South Pole, Antarctica
|
Posted: Sat Oct 05, 2019 12:53 am Post subject: Do we need to zeroise allocated array ? |
|
|
or it is still in arbitrary state?
If not guaranteed being at least zeroised after allocation would be nice to have some kind of new ALLOCATE with initialization at the same time for simplicity. |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
Posted: Sat Oct 05, 2019 6:45 am Post subject: |
|
|
Calls to ALLOCATE from FTN95 currently automatically zeroise the memory. This may not be portable to other compilers. The feature is not documented, however, there is no reason to think that it might change.
But if you use /UNDEF (or any option that implies /UNDEF) then the memory is initialised for "undef" checking. |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sat Oct 05, 2019 2:43 pm Post subject: |
|
|
Paul,
I would question this claim as my understanding is ALLOCATE does not actually allocate the memory pages for the array, but merely reserves memory address range for the array.
It is not until the array is used that the memory is allocated. See the example in the recent "Size of all arrays"
http://forums.silverfrost.com/viewtopic.php?t=4082&postdays=0&postorder=asc&start=0
I could modify this program to test the values of the array before it is initialised. However, I don't think it is standard conforming to use an uninitialised array. |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
Posted: Sat Oct 05, 2019 3:12 pm Post subject: |
|
|
John
Can you provide an example. When I trace ALLOCATE (without /UNDEF) it leads to a call to HeapAlloc using HEAP_ZERO_MEMORY. This is for 64 bits with a similar call for 32 bits. The documentation for HEAP_ZERO_MEMORY says "The allocated memory will be initialized to zero".
This is not a proof but an illustration...
Code: | integer,allocatable::ix(:)
allocate(ix(10))
print*, ix
end |
|
|
Back to top |
|
|
John-Silver
Joined: 30 Jul 2013 Posts: 1520 Location: Aerospace Valley
|
Posted: Sat Oct 05, 2019 5:59 pm Post subject: |
|
|
Paul wrote:
Quote: | Calls to ALLOCATE from FTN95 currently automatically zeroise the memory. This may not be portable to other compilers. |
As good a reason as any for ensuring ALL variables are zeroised explicitly within the program !
Quote: | The feature is not documented, however, there is no reason to think that it might change. |
Not exactly a robust justification for non-documentation !
Quote: | But if you use /UNDEF (or any option that implies /UNDEF) then the memory is initialised for "undef" checking. |
So, are you saying it's only zeroised when /UNDEF is used ? ... which is what the presence of 'But' at the beginning of the sentence implies, even though I read the first sentence as that just by using ALLOCATE the zeroising is automaticaally carried out. And which is clearly as per you your 2nd comment about the MS documentation for HEAP ALLOCATION also.
My Conclusions:
1. Users: just like for IMPLICIT NONE, keep on initialising EVERYTHING yourself whether you decide to use the 'time saving' global features or not, that way you have no one but yourself to blame if anything isn't initialised to your taste !
2. all this stuff needs to be clearly documented ! _________________ ''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... " |
|
Back to top |
|
|
LitusSaxonicum
Joined: 23 Aug 2005 Posts: 2390 Location: Yateley, Hants, UK
|
Posted: Sat Oct 05, 2019 6:37 pm Post subject: |
|
|
John-Silver,
When /UNDEF is used, presumably the memory isn't zeroised, it is all set to a value that signifies 'undefined'.
I agree that the historic requirement that the programmer should explicitly provide an initial value for everything before first use is a good practice, and in particular, aids in transferring things to a different compiler. However, the minute you start using Clearwin+, you can't change.
As well as documentation for FTN95, there is also the matter of documenting one's programs, either with comments, or with paper. Perhaps if one wanted to take advantage of automatic zeroisation, it might be worth doing a check. Just how many locations would one need to test (based on statistics) to be confident that the array was, in fact, zeroed?
Eddie |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sun Oct 06, 2019 5:50 am Post subject: |
|
|
Paul,
I adapted the previous allocate arrays test I referred to by:
- after allocating the arrays, counted the non-zero values.
- did the test twice, after changing memory to non-zero in first pass
The results are :
1) either initialising (previous test) or testing values results in the memory pages being allocated.
2) all arrays have been initially zero, as you indicated.
Would this only apply to ALLOCATE (heap arrays) only or also stack arrays (local, automatic) or common also ? ( I guess I should check )
Both FTN95 (/32), FTN95 /64 and gFortran perform in this way.
However, I would never use this assumption
John
The revised program: Code: | ! program to first allocate big arrays, then initialise big arrays
! requires FTN95 Ver 8.5 to work with allocatable arguments
!
module sz
real*4 :: sz_mb ! array size in mb
real*4 :: sz_ni ! array elements
integer*4 :: ni ! dimension of array
end module sz
!
program use_memory
use sz
!
interface
subroutine allocate_array ( aa )
use sz
integer*4, allocatable, dimension(:,:) :: aa
end subroutine allocate_array
end interface
!
integer*4, allocatable, dimension(:,:) :: a,b,c,d,e,f,g,h
integer*4 :: i,j, pass
!
sz_mb = 500 ! MBytes
!
call prompt ('What array size to test (in MBytes, default 500) ?', j)
if ( j > 1 .and. j < 2048 ) sz_mb = j
sz_ni = sz_mb * 1024.**2 / 4
ni = sqrt (sz_ni)
write (*,*) '2D I*4 array dimension =',ni
!
do pass = 1,2
! ALLOCATE all arrays
do i = 1,9
select case (i)
case (1)
call allocate_array ( a )
case (2)
call allocate_array ( b )
case (3)
call allocate_array ( c )
case (4)
call allocate_array ( d )
case (5)
call allocate_array ( e )
case (6)
call allocate_array ( f )
case (7)
call allocate_array ( g )
case (8)
call allocate_array ( h )
case default
exit
end select
write (*,*) i, ' arrays allocated: check task manager'
call prompt ('Proceed ?',j)
end do
!
write (*,*) 'all allocates done'
write (*,*) '"Comit Size" should have increased but NOT "Working Set"'
call prompt ('Proceed ?',j)
write (*,*) 'now to utilise memory !'
!
! TEST non-zero arrays
do i = 1,9
select case (i)
case (1)
call test_non_zero ( a )
case (2)
call test_non_zero ( b )
case (3)
call test_non_zero ( c )
case (4)
call test_non_zero ( d )
case (5)
call test_non_zero ( e )
case (6)
call test_non_zero ( f )
case (7)
call test_non_zero ( g )
case (8)
call test_non_zero ( h )
case default
exit
end select
write (*,*) i, ' arrays tested for non-zero: check task manager'
call prompt ('Proceed ?',j)
end do
!
write (*,*) 'all non-zero tests done : now to initialise'
call prompt ('Proceed ?',j)
!
! INITIALISE all arrays
do i = 1,9
select case (i)
case (1)
|
|
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sun Oct 06, 2019 5:52 am Post subject: |
|
|
program ctd .. Code: | case (1)
call initialise_array ( a, i )
case (2)
call initialise_array ( b, i )
case (3)
call initialise_array ( c, i )
case (4)
call initialise_array ( d, i )
case (5)
call initialise_array ( e, i )
case (6)
call initialise_array ( f, i )
case (7)
call initialise_array ( g, i )
case (8)
call initialise_array ( h, i )
case default
exit
end select
write (*,*) i, ' arrays initialised: check task manager'
call prompt ('Proceed ?',j)
end do
!
write (*,*) 'all arrays initialised : now to test correct setting'
call prompt ('Proceed ?',j)
!
! TEST all arrays that they have been initialised correctly
do i = 1,9
select case (i)
case (1)
call test_array ( a, i )
case (2)
call test_array ( b, i )
case (3)
call test_array ( c, i )
case (4)
call test_array ( d, i )
case (5)
call test_array ( e, i )
case (6)
call test_array ( f, i )
case (7)
call test_array ( g, i )
case (8)
call test_array ( h, i )
case default
exit
end select
if ( i < 9) write (*,*) i, ' arrays tested: check task manager'
call prompt ('Proceed ?',j)
end do
!
if ( pass == 1 ) then
write (*,*) 'all arrays tested so done for first pass; what is the memory use now'
call prompt ('Proceed ?',j)
!
deallocate ( a, b, c, d, e, f, g, h )
write (*,*) 'all arrays deallocated; check memory then proceed to retest'
call prompt ('Proceed ?',j)
!
else
write (*,*) 'all tests now repeated : How did we go : final ? before exiting'
call prompt ('Proceed ?',j)
end if
end do ! pass
end program use_memory
subroutine allocate_array ( aa )
use sz
integer*4, allocatable, dimension(:,:) :: aa
integer*4 :: stat
allocate ( aa(ni,ni), stat=stat )
write (*,*) 'array allocated : stat=',stat
end subroutine allocate_array
subroutine initialise_array ( aa, ii )
use sz
integer*4 :: aa(ni,ni), ii
integer*4 :: i,j, n
n = 0
do j = 1,ni
do i = 1,ni
aa(i,j) = i+j+ii
n = n+1
end do
end do
write (*,*) 'array initialised',n
end subroutine initialise_array
subroutine test_array ( aa, ii )
use sz
integer*4 :: aa(ni,ni), ii
integer*4 :: i,j, n, e
n = 0
e = 0
do j = 1,ni
do i = 1,ni
if ( aa(i,j) /= i+j+ii ) e = e+1
n = n+1
end do
end do
write (*,*) 'array tested',n, ' :',e,' errors'
end subroutine test_array |
|
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sun Oct 06, 2019 5:54 am Post subject: |
|
|
program ctd .. Code: |
subroutine test_non_zero ( aa )
use sz
integer*4 :: aa(ni,ni)
integer*4 :: i,j, n, e
n = 0
e = 0
do j = 1,ni
do i = 1,ni
if ( aa(i,j) /= 0 ) e = e+1
n = n+1
end do
end do
write (*,*) 'array tested',n, ' :',e,' non-zero'
end subroutine test_non_zero
subroutine prompt (string, j)
character string*(*)
integer j, iostat
write (*,fmt='(a)',advance='NO') string
read (*,fmt='(bn,i6)', iostat=iostat) j
if (iostat /= 0) j = -1
end subroutine prompt
|
|
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
Posted: Sun Oct 06, 2019 7:57 am Post subject: |
|
|
John
Initially you doubted my statement. Is this still the case? |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sun Oct 06, 2019 8:40 am Post subject: |
|
|
Paul,
I have changed my tune and no longer doubt you !
Now I understand that FTN95 does initialise allocated arrays.
I also found that with FTN95 /64 that large automatic arrays (that go onto the heap) are also initialised. 64-bit gFortran also appears to have this behaviour.
I tried to test local and automatic arrays for 32-bit, but got a stack overflow (and gave up on this "ugly" approach).
I am not sure what would happen for smaller arrays that do go onto the stack.
However, I have always considered it necessary to initialise an array or variable before it is used, so I will not adopt this "lazy" approach.
I am not sure if first initialising an array is standard conforming, but it should be. I think Eddie agrees with me this time ?
John
PS: the following adaptation appears to show small arrays in the stack are not initialised. Code: | ! program to test automatic arrays for initialisation
!
module sz
real*4 :: sz_mb ! array size in mb
real*4 :: sz_ni ! array elements
integer*4 :: ni ! dimension of array
end module sz
!
program use_memory
use sz
!
integer*4 :: j
!
sz_mb = 5 ! MBytes
!
call prompt ('What array size to test (in MBytes, default 5) ?', j)
if ( j > 1 .and. j < 2048 ) sz_mb = j
!
sz_ni = sz_mb * 1024.**2 / 4
ni = sqrt (sz_ni)
write (*,*) '2D I*4 array dimension =',ni
!
call test_automatic_array (ni)
!
end program use_memory
subroutine test_automatic_array (ni)
!
integer*4 :: ni
!
integer*4 aa(ni,ni) ! automatic array
integer*4 :: pass, i
do pass = 1,2
!
i = pass
!
! Start test for this pass
write (*,*) 'Automatic array now on stack'
write (*,*) '"Comit Size" should have increased but NOT "Working Set"'
call prompt ('Proceed ?',j)
!
write (*,*) 'now to utilise memory !'
!
! TEST non-zero arrays
call test_non_zero ( aa )
!
write (*,*) 'all non-zero tests done : now to initialise'
call prompt ('Proceed ?',j)
!
! INITIALISE array
call initialise_array ( aa, i )
!
write (*,*) 'all arrays initialised : now to test correct setting'
call prompt ('Proceed ?',j)
!
! TEST array that it has been initialised correctly
call test_array ( aa, i )
!
! End this pass
if ( pass == 1 ) then
write (*,*) 'all arrays tested so done for first pass; what is the memory use now'
call prompt ('Proceed ?',j)
!
else
write (*,*) 'all tests now repeated : How did we go : final ? before exiting'
call prompt ('Proceed ?',j)
end if
end do ! pass
end subroutine test_automatic_array
|
|
|
Back to top |
|
|
LitusSaxonicum
Joined: 23 Aug 2005 Posts: 2390 Location: Yateley, Hants, UK
|
Posted: Sun Oct 06, 2019 9:36 am Post subject: |
|
|
John,
As an 'old dog', the idea that anything is initialised without my express intervention is a 'new trick', and it is my habit to initialise everything.
It's brilliant if things are initialised, but think that this ought to be in response to one of the following:
(a) a compiler directive
(b) a command (that doesn't exist) in standard Fortran
or
(c) a programmed test to see if the compiler did or did not have this as a default.
IN any case, the assumption needs to be commented.
On the other hand, since I consider anyone using anything other than FTN95 as somewhat deranged, because CW+ doesn't exist properly elsewhere, then perhaps it is OK to assume that FTN95's behaviour is the norm.
It still needs a comment in the source code!
Eddie |
|
Back to top |
|
|
DanRRight
Joined: 10 Mar 2008 Posts: 2826 Location: South Pole, Antarctica
|
Posted: Sun Oct 06, 2019 2:42 pm Post subject: |
|
|
Great that allocated arrays are zeroised. Life will be a bit easier with my 140 arrays which need allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise-allocate-check-deallocate-zeroise all the time.
Also looks like lazy allocation of sparse arrays in COMMON becomes the necessity of the past. If keeping lazy allocation is the reason of slow performance of compiler with Poly tests, then may be it should be abandoned. Or kept as a non-standard option |
|
Back to top |
|
|
John-Silver
Joined: 30 Jul 2013 Posts: 1520 Location: Aerospace Valley
|
Posted: Sun Oct 06, 2019 6:53 pm Post subject: |
|
|
I guess whn Paul gets the time to get around to investigating how to 'imp rove' ftn95 performance of the polyhedron tests he'll surely look into the effect of using the 'lazy' method v traditional methods.
There are however surely large number of possible parameters in play there, at least until the one(s) affecing the performance are identified ! _________________ ''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... " |
|
Back to top |
|
|
John-Silver
Joined: 30 Jul 2013 Posts: 1520 Location: Aerospace Valley
|
Posted: Sun Oct 06, 2019 6:57 pm Post subject: |
|
|
Eddie,
Quote: | When /UNDEF is used, presumably the memory isn't zeroised, it is all set to a value that signifies 'undefined'. |
that's a question for Paul I think.
JohnC, did you run your test program with/without /UNDEF ? _________________ ''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... " |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|