forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Memory allocation fails in /CHECKMATE, works with /RELEASE
Goto page 1, 2  Next
 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sat Dec 07, 2019 3:37 am    Post subject: Memory allocation fails in /CHECKMATE, works with /RELEASE Reply with quote

I have an error that I cannot duplicate with smaller code. It occurs only when the routines are compiled in /CHECKMATE. It works great in /RELEASE or if I reduce the memory allocation for the "offending" ALLOOCATE manually.

The error traceback is:
Quote:
346c9780 __PALLOCATE2 [+0127]
ALLOCATE_EXTRACT_LOGS_HOLEID - in file select_using_holeid.for at line 119 [+0256]
SELECT_LOG_BY_HOLEID_WINDOWS - in file select_using_holeid.for at line 193 [+00b4]
345ede76 call_function(<ptr>(func()ÄreturningÄint),<ptr>char,<ptr>void,enumÄlogical,enum [+0181]
345d7d97 __yield_program_control [+0013]
345fcad0 __temporary_yield [+0016]
34667ce3 __winio [+1f4a]
CATALOG_INTEGRITY - in file catalog_integrity.for at line 230 [+1db8]


The memory allocation attempts to allocate about 157 MB (normal), or half that (when it works).

The variable definition is in a TYPE and is declared and ALLOCATE'd as:
Code:

   type :: extract_log_holeid ! allows the reading of the catalog into a local structure that gets cleared when processing is done
   sequence
   integer,allocatable::   catalog_index_list(:)
   integer,allocatable::   select_catalog_entry(:)
   integer::      num_recs=0
   integer::      num_records=0 ! number of records actually displayed
   character*100,allocatable:: catalog_listview(:)
   TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: extract_logs   ! Pointer to the data array:: catalog data with extended data attached
   end type

   allocate(extracted_logs%extract_logs(max_catalog_records),stat=i)



I tried using the ALLOCATABLE, but same result. Attempting to allocate the memory outside of the TYPE also failed.

I know this is not really fully sufficient for debugging, but wondered if anyone else has had similar issues with ALLOCATE and /CHECKMATE.
Back to top
View user's profile Send private message Visit poster's website
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sat Dec 07, 2019 5:14 am    Post subject: Reply with quote

After some more investigation, it would appear that using the ALLOCATE on a TYPE is the reason for the error. If I declare a variable of the same size as my TYPE and perform the ALLOCATE, it works just fine.

This might be related to a previous posting regarding the use of ABSOLUTE_ADDRESS= in an ALLOCATE; it doesn't work properly for a TYPE but works fine for an intrinsic type.
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6097
Location: Salford, UK

PostPosted: Sat Dec 07, 2019 8:41 am    Post subject: Reply with quote

There is a limited amount of memory available when calling __PALLOCATE2. This call can be avoided by using /INHIBIT_CHECK 4 when /CHECK is used.

This bypasses the Fortran pointer checking code.
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sat Dec 07, 2019 7:18 pm    Post subject: Reply with quote

I have a bit of test code and the compile option did not solve the problem in /CHECK. Again, allocating an intrinsic type versus a user-defined TYPE causes different things to happen.

As an additional test, I defined a simple TYPE that contains another defined TYPE, as well as the more complex TYPE that is causing the "troubles. I cannot get the simple TYPE to fail, yet the complex TYPE does fail.

I have included my test code here, and a DropBox link to the include file.

The only Project option that might be unusual is /SAVE to match my old code assumptions.

Code:

   include 'z:\cmasterf95\parametersandtypes.ins'
   integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: temp_record(:)
    TYPE (MASTER_CATALOG_REC_EXTENDED):: blank
   character*511,pointer,dimension(:):: temp_record1
      inquire(iolength=i)blank
    print *,i
    do j=1,10000
    allocate(temp_record(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    inquire(iolength=k)temp_record(1)
    print *,j,i,k
    end do
    read *,i
    end


https://www.dropbox.com/s/jlybzx24juiithy/ParametersAndTypes.ins?dl=0
Back to top
View user's profile Send private message Visit poster's website
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 5:08 am    Post subject: Reply with quote

This might be a clue. I can now get the program to fail. Just initialize a variable in the TYPE, and when you start allocation, it will program fault.

The 10th line of the program contains an initialization (DEFG=0). Remove the initialization, and the program will end normally.

Code:
   integer,parameter:: max_catalog_records = 65530
   integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg=0
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    character*511,pointer,dimension(:):: temp_record1
    do j=1,10000
    allocate(temp_record2(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    print *,j,i
    end do
    !read *,i ! make the program pause so we can run task manager
    end
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 2147
Location: Sydney

PostPosted: Sun Dec 08, 2019 7:54 am    Post subject: Reply with quote

Bill,

In your latest example, I don't understand why you have introduced the complexity of multiple layers of derived types.
You also use "defg" as integer or character in different types, which could make the code complex to read.
I'd be interested to see how you address the components of my_type and my_type2 in an unambiguous way.

In your first example, I did not understand where you defined "extracted_logs", or why it is needed. It appears to be a collector for a single element of type extract_log_holeid ?

If you have allocated arrays as components of a derived type, all these allocated components must be deallocated before you deallocate the derived type.
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6097
Location: Salford, UK

PostPosted: Sun Dec 08, 2019 9:58 am    Post subject: Reply with quote

Bill

I looked at your program with the Dropbox include file.

My first question relates to the line

Code:
    TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: temp_record(:)


This appears to have the dimension twice. I guess it is a one-dimensional array and one of the (:) should be omitted.

Then the lines

Code:

    do j=1,10000
    allocate(temp_record(max_catalog_records),stat=i)


looks like the allocation should not be within the DO loop.
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 4:12 pm    Post subject: Reply with quote

Hi Paul,

Both of your comments are valid.

The first one was my error, a holdover from using the keyword ALLOCATABLE.

The second was by design, though, as I wanted to try the allocation many many times to use up all available memory to check the return code from the ALLOCATE.

Bill
Back to top
View user's profile Send private message Visit poster's website
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 4:20 pm    Post subject: Reply with quote

John,

Just like many small programs others have used to illustrate an issue, the example here is contrived, but based on my unshared, original code.

And while your comments are, on their own, valid, they miss the entire point of this exercise.

If I can get a failure or undesired behavior to manifest itself in otherwise valid code, then who cares? There appears to be a problem, and the code sample is short enough to be tested and verified on Paul's machine.

Bill
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6097
Location: Salford, UK

PostPosted: Sun Dec 08, 2019 4:29 pm    Post subject: Reply with quote

Bill

Can you post corrected code for me to test for the failure that you describe?
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 4:55 pm    Post subject: Reply with quote

The code post from Sun Dec 08, 2019 5:08 am has the simplified code that causes a failure.
Back to top
View user's profile Send private message Visit poster's website
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 4:56 pm    Post subject: Reply with quote

Sorry, bad paste
Code:

   integer,parameter:: max_catalog_records = 65530
   integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg=0
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    character*511,pointer,dimension(:):: temp_record1
    do j=1,10000
    allocate(temp_record2(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    print *,j,i
    end do
    !read *,i ! make the program pause so we can run task manager
    end
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6097
Location: Salford, UK

PostPosted: Sun Dec 08, 2019 6:03 pm    Post subject: Reply with quote

FTN95 is not coping with the type initialisation...

integer:: defg=0

at the lower TYPE level. This is a known issue.

If you remove the =0 then for me the do loop stops at j = 10 when the memory is exhausted.

I also moved the print statement so that I could see the i value when the ALLOCATE failed.
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Sun Dec 08, 2019 6:30 pm    Post subject: Reply with quote

Paul, thanks for the verification.Working around the issue...

Bill
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 2147
Location: Sydney

PostPosted: Fri Dec 13, 2019 12:02 am    Post subject: Reply with quote

Bill,

I do use allocatable arrays in an allocatable derived type, but not allocatable derived types inside an allocatable derived type.

In my case, I first allocate the allocatable derived type, which provides the structure for defining the allocatable components, then carefully manage the allocate/deallocate of these components.

It is interesting that for all compilers I use, determining the size of this data structure reports the memory allocated for managing the size of the allocatable components, but not the size of the component data after it has been allocated. Not what is wanted.

I am surprised that FTN95 allows your definition approach.
I worry that using derived types inside derived types will one day find an ICE in most compilers. The derived type is only defined once in a module so keep it simple.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group