forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

GET_GSTORAGE@ doesn't always work?
Goto page Previous  1, 2, 3, 4  Next
 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Fri Jan 13, 2023 10:23 am    Post subject: Reply with quote

hi, could you email me that text block? when I copy/paste it, it won't compile (it's got some sort of illegal character (hex"A0") embedded in it)

K
Back to top
View user's profile Send private message Visit poster's website
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Fri Jan 13, 2023 12:47 pm    Post subject: Reply with quote

Hmmm, i don't think that works?

do you get different values returned by alloc if you call it more than once without dealloc-ing?

i'm experimenting with a "hybrid" solution that seems fairly solid but it depends on keeping a record of the start and end addresses allocated in a table.

basically, if the buffer size (to the nearest multiple of 16bytes) is large then i use GET_GSTORAGE@
if it's medium then i use GET_STORAGE@
if it's tiny then i use localalloc (via a STDCALL)

Code:
      nbuf  =  16
      nb7   =  (nb-1)/nbuf*nbuf+nbuf
      if( mod(nb7,4096).gt.2048) then
        CALL GET_GSTORAGE@ (IA, NB7)
      else if( nb7.gt.128) then
        CALL GET_STORAGE@ (IA, NB7)
      else 
        ia    =  localalloc(0,nb7)
      endif 

then when freeing the memory, i use the recorded start/end addresses to get the size of the block and free the memory in the appropriate manner:

Code:
   ja  = ia
    isiz=-1
    call A_VMmem(ja, isiz) ! returns the size of the block
    if( mod(isiz,4096).gt.2048) then
      CALL return_GSTORAGE@ (IA)
      call GIVEBACK_ALLOCATION_BLOCKS@(kval)
      ia    = 0
    else if( isiz.gt.128) then
      CALL return_STORAGE@ (IA)
      call GIVEBACK_ALLOCATION_BLOCKS@(kval)
      ia    = 0
    elseif( isiz.gt.0) then
      ia  = localfree(ia)
      if( ia.ne.0) then
        write(*,*)'failed to free', ia
      endif 
    endif



K


Last edited by KennyT on Fri Jan 13, 2023 2:19 pm; edited 2 times in total
Back to top
View user's profile Send private message Visit poster's website
mecej4



Joined: 31 Oct 2006
Posts: 1885

PostPosted: Fri Jan 13, 2023 12:52 pm    Post subject: Reply with quote

That is strange, and may be related to the exact details of your copy-paste procedure. I used Firefox on Windows to copy, and pasted by redirection to a file in a command window, and did not find any characters with the high bit set.

I may note that it is not quite correct to use an argument to ALLOCATED that has not been declared to be allocatable. You should probably use ASSOCIATED instead. Similarly, NULLIFY instead of DEALLOCATE? The choice between pointers versus allocatables depends on what you want to do in your application.


Last edited by mecej4 on Fri Jan 13, 2023 1:01 pm; edited 1 time in total
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Fri Jan 13, 2023 12:58 pm    Post subject: Re: Reply with quote

Robert wrote:
It is worth remembering that VirtualAlloc allocates (or uses memory) at page size granularity. So an alloc of a few bytes takes a full 4K page


Robert, Your earlier statement that you provided interests me, as I am wanting to allocate arrays on the heap, starting on a new memory page.
How do I get access to "VirtualAlloc" ?
Back to top
View user's profile Send private message
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Fri Jan 13, 2023 1:14 pm    Post subject: Re: Reply with quote

mecej4 wrote:
That is strange, and may be related to the exact details of your copy-paste procedure. I used Firefox on Windows to copy, and pasted by redirection to a file in a command window, and did not find any characters with the high bit set.


i also use FF and if i swipe the code block and paste into notepad then i look at it in a hex window:

4C4C204745545F53544F524147454020
2849412C204E4237290D0A202020202020656C7365C2A00D0A20202020202020206961202020203D20206C6F63616C616C6C6F6328302C6E6237290D0A202020
202020656E646966202000

k
Back to top
View user's profile Send private message Visit poster's website
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Fri Jan 13, 2023 1:17 pm    Post subject: Re: Reply with quote

JohnCampbell wrote:
Robert wrote:
It is worth remembering that VirtualAlloc allocates (or uses memory) at page size granularity. So an alloc of a few bytes takes a full 4K page


Robert, Your earlier statement that you provided interests me, as I am wanting to allocate arrays on the heap, starting on a new memory page.
How do I get access to "VirtualAlloc" ?

there may be a STDCALL? there is for LocalAlloc for example.

K

PS if you look in WIN32API.INS:

STDCALL VIRTUALFREE 'VirtualFree' (REF,VAL,VAL):LOGICAL*4
STDCALL VIRTUALALLOC 'VirtualAlloc' (REF,VAL,VAL,VAL):INTEGER(7)

but you'll have to google MSDN to find out what the arguments are...


K
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Fri Jan 13, 2023 2:28 pm    Post subject: Reply with quote

Kenny:
If you send me a personal message with your email address then I will email you a copy of the code.

mecej4:
Yes ASSOCIATED is better and avoids the warning message. I will edit the post.

John:
For 64 bit applications you can access VirtualAlloc via GET_GSTORAGE@ or GET_GSTORAGE64@ but VirtualAlloc does not get heap memory. For further information Google "VirtualAlloc MSDN".
Back to top
View user's profile Send private message AIM Address
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Fri Jan 13, 2023 2:57 pm    Post subject: Re: Reply with quote

PaulLaidler wrote:
Here is some code that illustrates how to get the address of a block of memory from the global heap using a call to ALLOCATE and FTN95.

Code:
subroutine alloc(addr, size)
 integer(7)::addr,size
 character,pointer::arr(:)
 ALLOCATE(arr(size))
 if(ALLOCATED(arr))then
   addr = loc(arr)
 else 
   addr = 0
   print*, "Allocate failed"
 end if   
end subroutine

subroutine dealloc(addr)
 integer(7)::addr
 character,pointer::arr(:)
 ALLOCATE(arr(1), ABSOLUTE_ADDRESS=addr)
 DEALLOCATE(arr) !Note: DEALLOCATE raises an exception on failure.
 addr = 0
end subroutine

program main
 integer(7) addr
 call alloc(addr, 100000_7)
 write(*,"(Z8)") addr
 call dealloc(addr)
end program
 


p.s. It is better to use ASSOCIATED rather than ALLOCATED.


Paul, if you change the main routine:
Code:

program main
 integer(7) addr(10)
 do i=1,10
 call alloc(addr(i), 100000_7)
 write(*,"(Z8)") addr(i)
enddo
 do i=1,10
 call dealloc(addr(i))
 ENDDO
end program
 
 


you see that the call to alloc always returns the same address...

20202000


K
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Fri Jan 13, 2023 4:22 pm    Post subject: Reply with quote

I should have also tested the code for a 32 bit application. It works for 64 bits.

There is something wrong with addr = loc(arr). addr = loc(arr(1)) is better but still not right.

I will have to think again for 32 bits.
Back to top
View user's profile Send private message AIM Address
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Sat Jan 14, 2023 2:39 am    Post subject: Reply with quote

This is my contribution to understanding memory allocation
Code:
   module ELEM_DATA_BASE  ! (minimal example)

      integer*4              :: MXELEM = 0         ! maximum element number expected
      integer*4, allocatable :: INDEXE(:)          ! (max_elements)
!
      TYPE elem_array_record                       ! set in elcomp
         integer*4           :: sd_size
         real*8, allocatable :: sd(:)              ! S(nd,nd) !symmetric stiffness matrix  sd( nt )
      END TYPE elem_array_record
!
      type (elem_array_record), allocatable :: elem_array_records(:)        ! (max_elem_records)
!
   end module ELEM_DATA_BASE

Program Report_memory_positions
  use ELEM_DATA_BASE

   integer :: i, is, stat
   
   is = 1024*1024-8
   write (*,fmt='(a,i0,a)') 'allocating arrays of size = ',is*8/4096+1,' pages'

   mxelem = 10
   allocate ( indexe(mxelem), STAT=stat )
     write (*,*) ' indexe(mxelem) allocated : STAT =',stat
     call report_location ( LOC(indexe) )
     
   allocate ( elem_array_records(mxelem), STAT=stat )
     write (*,*) ' elem_array_records(mxelem) allocated : STAT =',stat
     call report_location ( LOC(elem_array_records) )

   write (*,fmt='(/a/)') ' now allocating arrays'
   do i = 1,mxelem

     elem_array_records(i)%sd_size = is
     allocate ( elem_array_records(i)%sd(is), STAT=stat )
      call report_location ( LOC(elem_array_records(i)%sd(1)) )

   end do

  end Program Report_memory_positions

   subroutine report_location ( addr )
     integer*8 :: addr, page, offs, page_size = 4096, last_addr = 0, dpage

     page  = addr / page_size
     offs  = addr - page*page_size
     dpage = (addr-last_addr)/page_size
     write ( *,11) addr, page, offs, addr-last_addr, dpage
  11 format ('  array allocated at mem= ',i0,' : page ',i0,' offs ',i0,' spacing ',i0,' bytes ',i0,' pages')
     last_addr = addr
   end subroutine report_location


My problem is I don't understand the page spacing of the "sd" arrays, but they all start on a new memory page, which is something I am trying to manage. (but with a 64 byte offset?)
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Sat Jan 14, 2023 5:28 am    Post subject: Reply with quote

Kenny,

I find the combination of Fortran local, automatic and ALLOCATE arrays work well for me for /64. I limit my memory management to call by reference arrays, so do not use pointers or memory address as routine arguments (excuse this example!).
Other Fortran compilers that use non-contiguous arrays or temporary arrays for array sections, this can cause problems for my "F77 wrapper" approach.

https://www.dropbox.com/s/71zgzfenhfjcel7/mem_test2.f90?dl=0

This link to an expanded example shows the location and spacing of ALLOCATE arrays (which go to the heap) and local arrays (which go on the stack)

ALLOCATE is doing what I want, in that large arrays are being located on a new memory page (with a 64 byte offset, which I can cope with), while local arrays appear to be placed on the end of the stack.

Based on the example for allocatable components of a derived type, there appears to be some wastage of memory page addresses, although this is not an issue for /64, as they would not be allocated physical pages. The memory address is basically a virtual address and all that is important is the number of physical memory pages that are allocated to the program.

The use of local or automatic arrays can require defining a larger stack at link time.

Allocatable components of a derived type array are a good way to manage lots of allocatable arrays, although they must each be deallocated to avoid memory leakage.
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Sat Jan 14, 2023 8:05 am    Post subject: Reply with quote

Kenny

This works fine for me (for both 32 bits and 64 bits)...

Code:
subroutine alloc(addr, size)
 integer(7)::addr,size
 character,pointer::arr(:)
 ALLOCATE(arr(size))
 if(ASSOCIATED(arr))then
   addr = loc(arr(1))
 else 
   addr = 0
   print*, "Allocate failed"
 end if   
end subroutine

subroutine dealloc(addr)
 integer(7)::addr
 character,pointer::arr(:)
 ALLOCATE(arr(1), ABSOLUTE_ADDRESS=addr)
 DEALLOCATE(arr) !Note: DEALLOCATE raises an exception on failure.
 addr = 0
end subroutine

program main
 integer(7) addr(10)
 do i=1,10
  call alloc(addr(i), 1000_7)
  write(*,"(Z8)") addr(i)
 end do
 call dealloc(addr)
end program
 
Back to top
View user's profile Send private message AIM Address
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Tue Jan 17, 2023 10:30 am    Post subject: Reply with quote

thanks Paul, that appears to work well, although i am still getting some "un-debuggable" run time errors, that i assume are due to errors in our fortran code (or the data being read being too big for the heap - is there a setting in the linker for this?). However, it is very difficult to use UNDEF to isolate these errors due to the persistence of runtime "error 14":

"Attempt to alter an actual argument that is a constant, an expression, an INTENT(IN) argument, or a DO variable"

which is still throwing up "false" errors.

is there a way to turn this error off (or at least let execution continue) so we can get to the true cause of the issue? there appears to be lots of choice about errors that can be ignored in the help but i can't decide which one(s) to use!

K
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Tue Jan 17, 2023 6:20 pm    Post subject: Reply with quote

Kenny

You could try using

/inhibit_check 1 2

on the FTN95 command line but that may not be wise.

I am not really in a position to be able to provide advice on how to debug your code.
Back to top
View user's profile Send private message AIM Address
KennyT



Joined: 02 Aug 2005
Posts: 317

PostPosted: Tue Jan 17, 2023 6:30 pm    Post subject: Reply with quote

i appreciate that, but the runtime error 14 is surely yours to try to address, given that we can clearly demonstrate that, in many circumstances, it's a false positive.

a switch to inhibit this error in the appropriate compilation would be enormously helpful, if you can't isolate what's causing it.

I suppose i could move the routine(s) that exhibit this issue into a source file that gets compiled /DEBUG rather than /UNDEF?

K
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Goto page Previous  1, 2, 3, 4  Next
Page 3 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group