 |
forums.silverfrost.com Welcome to the Silverfrost forums
|
| View previous topic :: View next topic |
| Author |
Message |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2625 Location: Sydney
|
Posted: Wed Sep 29, 2010 10:32 am Post subject: |
|
|
John,
If I wrote the program as I intended, then your results show you are also addressing 3.6gb of memory.
We now need to understand how to use the 2 or 3 large memory packets.
If we could move the bits that SLINK spread around and also be able to reference a single array bigger than 2gb then there would be potentially nearly 4gb of addressable memory for FTN95 in WOW64.
It is interesting that the largest array "A1" sucessfully crosses the 2gb memory point, although it is less than 2gb in size.
I have tried to use ALLOCATE for arrays larger than 2gb. It gives STAT=0 on XP-32, but I think that is an error. As a result, I'm not sure it is possible to allocate arrays larger than 2gb on XP-64 / WOW64.
This certainly shows potential for some problem size extension using FTN95.
John |
|
| Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2625 Location: Sydney
|
Posted: Thu Sep 30, 2010 1:26 am Post subject: |
|
|
Paul,
The results from these examples appear very promising for FTN95 and Win-64. It would be good if FTN95 and SLINK could be tuned to better utilise the extra available memory space.
There is potential for FTN95 to address near 4gb of memory in a WOW64 environment. What the previous example shows is an array can span the 2gb memory address. I have changed the previous example slightly to demonstrate that I can also allocate an array starting above 3gb.
There are a few problems I am still anticipating:-
1) ALLOCATE can manage to address available memory throughout the 4gb of addressible space, I assume by using a 32-bit unsigned integer.
2) If ALLOCATE is used for an array >2gb, it returns STAT=0, even in Win-32, indicating that ALLOCATE may not be able to handle arrays larger than 2gb.
3) LOC returns a -ve value if the address is > 2gb, although the array appears to be accessed corectly, even if it spans the 2gb address.
4) There is still the problem of SLINK not leaving a large contiguous memory, but placing a large number of memory uses throughout memory, especially above 1.7gb. I assume this, from my previous example of searching for the largest available contiguous space, to the nearest 4 bytes. Now that addressing large amounts of memory is more practical, could the way SLINK uses memory be reviewed. Also (my interpretation of) placing VC at 1gb could be reviewed.
5) Could ALLOCATE and FTN95 support a single array up to 4gb, by using unsigned 32-bit addressing, certainly > 2gb ?
All my examples are for small codes that have negligable use of the stack. I will change the memory search to a subroutine and place it in the middle of a large program to see if it still works.
Please give this some consideration as it might easily provide FTN95 a significant capacity extension in a Win-64 environment.
John
| Code: |
! PROGRAM to test allocating 4 big arrays
integer*4, allocatable, dimension(:) :: a1, a2, a3, a4
integer*4 l1, l2, l3, l4, iostat
!
l1 = 1300 * 1000 * 1000 / 4
l4 = 800 * 1000 * 1000 / 4
l2 = 990 * 1000 * 1000 / 4
l3 = 520 * 1000 * 1000 / 4
!
write (*,*) 'Program to test 4 arrays; one should be located above 3gb'
!
allocate (a1(l1), stat=iostat)
write (*,*) 'A1 allocated',iostat
!
allocate (a2(l2), stat=iostat)
write (*,*) 'A2 allocated',iostat
!
allocate (a4(l4), stat=iostat)
write (*,*) 'A4 allocated',iostat
!
allocate (a3(l3), stat=iostat)
write (*,*) 'A3 allocated',iostat
!
call define_array (a1,l1)
call define_array (a4,l4)
call define_array (a2,l2)
call define_array (a3,l3)
!
call check_array (a1,l1)
call check_array (a4,l4)
call check_array (a2,l2)
call check_array (a3,l3)
END PROGRAM
subroutine define_array (a,l)
integer*4 l, a(l), i, n
real*8 mb
n = l/2
do i = 1,l
a(i) = n
n = n-1
end do
mb = l
mb = mb * 4. / 1024. /1024.
write (*,*) 'Array defined for size =',l, mb
end subroutine
subroutine check_array (a,l)
integer*4 l, a(l), i, n, er
integer*8 mem
mem = loc(a)
n = l/2
er = 0
do i = 1,l
if (a(i) /= n) er = er+1
n = n-1
end do
write (*,2001) 'Array at address ',loc(a),mem,mem,' checked for size =',l,' error = ',er
2001 format (a,b'--,---,---,--#',i12,Z12,a,b'zz,zzz,zzz,zzz',a,i0)
end subroutine
|
|
|
| Back to top |
|
 |
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8283 Location: Salford, UK
|
Posted: Thu Sep 30, 2010 8:05 am Post subject: |
|
|
John
This is all very promising and helpful. I would like to work on this but there are a number of immediate problems that need to be sorted out first.
Please prompt me again in 4 weeks if there are no signs of action.
Paul |
|
| Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2625 Location: Sydney
|
Posted: Thu Sep 30, 2010 9:26 am Post subject: |
|
|
Paul,
I can assure you I will not forget to contact you.
The gap analysis, which shows memory locations that are excluded from being available to ALLOCATE is interesting. I think my routine "get_free_memory_size" sizes them to the nearest 4 bytes. I have increased the DO loop to 30 and these excluded areas keep on coming. It is not what I would have expected from SLINK, or maybe I've got this a bit wrong ? Is there any documention of what these may be ?
The program to test memory to 4gb looks to be working, as Windows Task Manager is now showing 3,711,008k virtual memory in use when I run it for 5 arrays.
It certainly appears very promising. I look forward to your updates.
John |
|
| Back to top |
|
 |
Sebastian
Joined: 20 Feb 2008 Posts: 177
|
Posted: Thu Sep 30, 2010 10:14 am Post subject: |
|
|
| Quote: |
| 2) If ALLOCATE is used for an array >2gb, it returns STAT=0, even in Win-32, indicating that ALLOCATE may not be able to handle arrays larger than 2gb. |
This could be a bug in the ALLOCATE implementation (using a signed 32bit integer instead of an unsigned one) or some limitation of the fortran standard that I don't see.
| Quote: |
| 4) There is still the problem of SLINK not leaving a large contiguous memory |
This may not even be a problem of SLINK but how the virtual address space is divided up (which at some locations may be out of control of the linker), usually the shared libraries are mapped into the end of the first 2GB so for any 32bit application you'll get at maximum a continuous block of 2GB (usually less because the end of the second 2GB block is used by the OS, but this takes up far less space on 64bit windows than on 32bit ones). |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|