|
forums.silverfrost.com Welcome to the Silverfrost forums
|
View previous topic :: View next topic |
Author |
Message |
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Fri Jan 13, 2023 10:23 am Post subject: |
|
|
hi, could you email me that text block? when I copy/paste it, it won't compile (it's got some sort of illegal character (hex"A0") embedded in it)
K |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Fri Jan 13, 2023 12:47 pm Post subject: |
|
|
Hmmm, i don't think that works?
do you get different values returned by alloc if you call it more than once without dealloc-ing?
i'm experimenting with a "hybrid" solution that seems fairly solid but it depends on keeping a record of the start and end addresses allocated in a table.
basically, if the buffer size (to the nearest multiple of 16bytes) is large then i use GET_GSTORAGE@
if it's medium then i use GET_STORAGE@
if it's tiny then i use localalloc (via a STDCALL)
Code: | nbuf = 16
nb7 = (nb-1)/nbuf*nbuf+nbuf
if( mod(nb7,4096).gt.2048) then
CALL GET_GSTORAGE@ (IA, NB7)
else if( nb7.gt.128) then
CALL GET_STORAGE@ (IA, NB7)
else
ia = localalloc(0,nb7)
endif
|
then when freeing the memory, i use the recorded start/end addresses to get the size of the block and free the memory in the appropriate manner:
Code: | ja = ia
isiz=-1
call A_VMmem(ja, isiz) ! returns the size of the block
if( mod(isiz,4096).gt.2048) then
CALL return_GSTORAGE@ (IA)
call GIVEBACK_ALLOCATION_BLOCKS@(kval)
ia = 0
else if( isiz.gt.128) then
CALL return_STORAGE@ (IA)
call GIVEBACK_ALLOCATION_BLOCKS@(kval)
ia = 0
elseif( isiz.gt.0) then
ia = localfree(ia)
if( ia.ne.0) then
write(*,*)'failed to free', ia
endif
endif
|
K
Last edited by KennyT on Fri Jan 13, 2023 2:19 pm; edited 2 times in total |
|
Back to top |
|
|
mecej4
Joined: 31 Oct 2006 Posts: 1898
|
Posted: Fri Jan 13, 2023 12:52 pm Post subject: |
|
|
That is strange, and may be related to the exact details of your copy-paste procedure. I used Firefox on Windows to copy, and pasted by redirection to a file in a command window, and did not find any characters with the high bit set.
I may note that it is not quite correct to use an argument to ALLOCATED that has not been declared to be allocatable. You should probably use ASSOCIATED instead. Similarly, NULLIFY instead of DEALLOCATE? The choice between pointers versus allocatables depends on what you want to do in your application.
Last edited by mecej4 on Fri Jan 13, 2023 1:01 pm; edited 1 time in total |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2593 Location: Sydney
|
Posted: Fri Jan 13, 2023 12:58 pm Post subject: Re: |
|
|
Robert wrote: | It is worth remembering that VirtualAlloc allocates (or uses memory) at page size granularity. So an alloc of a few bytes takes a full 4K page |
Robert, Your earlier statement that you provided interests me, as I am wanting to allocate arrays on the heap, starting on a new memory page.
How do I get access to "VirtualAlloc" ? |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Fri Jan 13, 2023 1:14 pm Post subject: Re: |
|
|
mecej4 wrote: | That is strange, and may be related to the exact details of your copy-paste procedure. I used Firefox on Windows to copy, and pasted by redirection to a file in a command window, and did not find any characters with the high bit set.
|
i also use FF and if i swipe the code block and paste into notepad then i look at it in a hex window:
4C4C204745545F53544F524147454020
2849412C204E4237290D0A202020202020656C7365C2A00D0A20202020202020206961202020203D20206C6F63616C616C6C6F6328302C6E6237290D0A202020
202020656E646966202000
k |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Fri Jan 13, 2023 1:17 pm Post subject: Re: |
|
|
JohnCampbell wrote: | Robert wrote: | It is worth remembering that VirtualAlloc allocates (or uses memory) at page size granularity. So an alloc of a few bytes takes a full 4K page |
Robert, Your earlier statement that you provided interests me, as I am wanting to allocate arrays on the heap, starting on a new memory page.
How do I get access to "VirtualAlloc" ? |
there may be a STDCALL? there is for LocalAlloc for example.
K
PS if you look in WIN32API.INS:
STDCALL VIRTUALFREE 'VirtualFree' (REF,VAL,VAL):LOGICAL*4
STDCALL VIRTUALALLOC 'VirtualAlloc' (REF,VAL,VAL,VAL):INTEGER(7)
but you'll have to google MSDN to find out what the arguments are...
K |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8062 Location: Salford, UK
|
Posted: Fri Jan 13, 2023 2:28 pm Post subject: |
|
|
Kenny:
If you send me a personal message with your email address then I will email you a copy of the code.
mecej4:
Yes ASSOCIATED is better and avoids the warning message. I will edit the post.
John:
For 64 bit applications you can access VirtualAlloc via GET_GSTORAGE@ or GET_GSTORAGE64@ but VirtualAlloc does not get heap memory. For further information Google "VirtualAlloc MSDN". |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Fri Jan 13, 2023 2:57 pm Post subject: Re: |
|
|
PaulLaidler wrote: | Here is some code that illustrates how to get the address of a block of memory from the global heap using a call to ALLOCATE and FTN95.
Code: | subroutine alloc(addr, size)
integer(7)::addr,size
character,pointer::arr(:)
ALLOCATE(arr(size))
if(ALLOCATED(arr))then
addr = loc(arr)
else
addr = 0
print*, "Allocate failed"
end if
end subroutine
subroutine dealloc(addr)
integer(7)::addr
character,pointer::arr(:)
ALLOCATE(arr(1), ABSOLUTE_ADDRESS=addr)
DEALLOCATE(arr) !Note: DEALLOCATE raises an exception on failure.
addr = 0
end subroutine
program main
integer(7) addr
call alloc(addr, 100000_7)
write(*,"(Z8)") addr
call dealloc(addr)
end program
|
p.s. It is better to use ASSOCIATED rather than ALLOCATED. |
Paul, if you change the main routine:
Code: |
program main
integer(7) addr(10)
do i=1,10
call alloc(addr(i), 100000_7)
write(*,"(Z8)") addr(i)
enddo
do i=1,10
call dealloc(addr(i))
ENDDO
end program
|
you see that the call to alloc always returns the same address...
20202000
K |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8062 Location: Salford, UK
|
Posted: Fri Jan 13, 2023 4:22 pm Post subject: |
|
|
I should have also tested the code for a 32 bit application. It works for 64 bits.
There is something wrong with addr = loc(arr). addr = loc(arr(1)) is better but still not right.
I will have to think again for 32 bits. |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2593 Location: Sydney
|
Posted: Sat Jan 14, 2023 2:39 am Post subject: |
|
|
This is my contribution to understanding memory allocation
Code: | module ELEM_DATA_BASE ! (minimal example)
integer*4 :: MXELEM = 0 ! maximum element number expected
integer*4, allocatable :: INDEXE(:) ! (max_elements)
!
TYPE elem_array_record ! set in elcomp
integer*4 :: sd_size
real*8, allocatable :: sd(:) ! S(nd,nd) !symmetric stiffness matrix sd( nt )
END TYPE elem_array_record
!
type (elem_array_record), allocatable :: elem_array_records(:) ! (max_elem_records)
!
end module ELEM_DATA_BASE
Program Report_memory_positions
use ELEM_DATA_BASE
integer :: i, is, stat
is = 1024*1024-8
write (*,fmt='(a,i0,a)') 'allocating arrays of size = ',is*8/4096+1,' pages'
mxelem = 10
allocate ( indexe(mxelem), STAT=stat )
write (*,*) ' indexe(mxelem) allocated : STAT =',stat
call report_location ( LOC(indexe) )
allocate ( elem_array_records(mxelem), STAT=stat )
write (*,*) ' elem_array_records(mxelem) allocated : STAT =',stat
call report_location ( LOC(elem_array_records) )
write (*,fmt='(/a/)') ' now allocating arrays'
do i = 1,mxelem
elem_array_records(i)%sd_size = is
allocate ( elem_array_records(i)%sd(is), STAT=stat )
call report_location ( LOC(elem_array_records(i)%sd(1)) )
end do
end Program Report_memory_positions
subroutine report_location ( addr )
integer*8 :: addr, page, offs, page_size = 4096, last_addr = 0, dpage
page = addr / page_size
offs = addr - page*page_size
dpage = (addr-last_addr)/page_size
write ( *,11) addr, page, offs, addr-last_addr, dpage
11 format (' array allocated at mem= ',i0,' : page ',i0,' offs ',i0,' spacing ',i0,' bytes ',i0,' pages')
last_addr = addr
end subroutine report_location
|
My problem is I don't understand the page spacing of the "sd" arrays, but they all start on a new memory page, which is something I am trying to manage. (but with a 64 byte offset?) |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2593 Location: Sydney
|
Posted: Sat Jan 14, 2023 5:28 am Post subject: |
|
|
Kenny,
I find the combination of Fortran local, automatic and ALLOCATE arrays work well for me for /64. I limit my memory management to call by reference arrays, so do not use pointers or memory address as routine arguments (excuse this example!).
Other Fortran compilers that use non-contiguous arrays or temporary arrays for array sections, this can cause problems for my "F77 wrapper" approach.
https://www.dropbox.com/s/71zgzfenhfjcel7/mem_test2.f90?dl=0
This link to an expanded example shows the location and spacing of ALLOCATE arrays (which go to the heap) and local arrays (which go on the stack)
ALLOCATE is doing what I want, in that large arrays are being located on a new memory page (with a 64 byte offset, which I can cope with), while local arrays appear to be placed on the end of the stack.
Based on the example for allocatable components of a derived type, there appears to be some wastage of memory page addresses, although this is not an issue for /64, as they would not be allocated physical pages. The memory address is basically a virtual address and all that is important is the number of physical memory pages that are allocated to the program.
The use of local or automatic arrays can require defining a larger stack at link time.
Allocatable components of a derived type array are a good way to manage lots of allocatable arrays, although they must each be deallocated to avoid memory leakage. |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8062 Location: Salford, UK
|
Posted: Sat Jan 14, 2023 8:05 am Post subject: |
|
|
Kenny
This works fine for me (for both 32 bits and 64 bits)...
Code: | subroutine alloc(addr, size)
integer(7)::addr,size
character,pointer::arr(:)
ALLOCATE(arr(size))
if(ASSOCIATED(arr))then
addr = loc(arr(1))
else
addr = 0
print*, "Allocate failed"
end if
end subroutine
subroutine dealloc(addr)
integer(7)::addr
character,pointer::arr(:)
ALLOCATE(arr(1), ABSOLUTE_ADDRESS=addr)
DEALLOCATE(arr) !Note: DEALLOCATE raises an exception on failure.
addr = 0
end subroutine
program main
integer(7) addr(10)
do i=1,10
call alloc(addr(i), 1000_7)
write(*,"(Z8)") addr(i)
end do
call dealloc(addr)
end program
|
|
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Tue Jan 17, 2023 10:30 am Post subject: |
|
|
thanks Paul, that appears to work well, although i am still getting some "un-debuggable" run time errors, that i assume are due to errors in our fortran code (or the data being read being too big for the heap - is there a setting in the linker for this?). However, it is very difficult to use UNDEF to isolate these errors due to the persistence of runtime "error 14":
"Attempt to alter an actual argument that is a constant, an expression, an INTENT(IN) argument, or a DO variable"
which is still throwing up "false" errors.
is there a way to turn this error off (or at least let execution continue) so we can get to the true cause of the issue? there appears to be lots of choice about errors that can be ignored in the help but i can't decide which one(s) to use!
K |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8062 Location: Salford, UK
|
Posted: Tue Jan 17, 2023 6:20 pm Post subject: |
|
|
Kenny
You could try using
/inhibit_check 1 2
on the FTN95 command line but that may not be wise.
I am not really in a position to be able to provide advice on how to debug your code. |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 318
|
Posted: Tue Jan 17, 2023 6:30 pm Post subject: |
|
|
i appreciate that, but the runtime error 14 is surely yours to try to address, given that we can clearly demonstrate that, in many circumstances, it's a false positive.
a switch to inhibit this error in the appropriate compilation would be enormously helpful, if you can't isolate what's causing it.
I suppose i could move the routine(s) that exhibit this issue into a source file that gets compiled /DEBUG rather than /UNDEF?
K |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|