Memory allocation fails in /CHECKMATE, works with /RELEASE

I have an error that I cannot duplicate with smaller code. It occurs only when the routines are compiled in /CHECKMATE. It works great in /RELEASE or if I reduce the memory allocation for the 'offending' ALLOOCATE manually.

The error traceback is:

346c9780 __PALLOCATE2 [+0127] ALLOCATE_EXTRACT_LOGS_HOLEID - in file select_using_holeid.for at line 119 [+0256] SELECT_LOG_BY_HOLEID_WINDOWS - in file select_using_holeid.for at line 193 [+00b4] 345ede76 call_function(<ptr>(func()ÄreturningÄint),<ptr>char,<ptr>void,enumÄlogical,enum [+0181] 345d7d97 __yield_program_control [+0013] 345fcad0 __temporary_yield [+0016] 34667ce3 __winio [+1f4a] CATALOG_INTEGRITY - in file catalog_integrity.for at line 230 [+1db8]

The memory allocation attempts to allocate about 157 MB (normal), or half that (when it works).

The variable definition is in a TYPE and is declared and ALLOCATE'd as:

	type :: extract_log_holeid ! allows the reading of the catalog into a local structure that gets cleared when processing is done
	sequence
	integer,allocatable::	catalog_index_list(:)
	integer,allocatable::	select_catalog_entry(:)
	integer::		num_recs=0
	integer::		num_records=0 ! number of records actually displayed
	character*100,allocatable:: catalog_listview(:)
	TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: extract_logs	! Pointer to the data array:: catalog data with extended data attached
	end type

	allocate(extracted_logs%extract_logs(max_catalog_records),stat=i)

I tried using the ALLOCATABLE, but same result. Attempting to allocate the memory outside of the TYPE also failed.

I know this is not really fully sufficient for debugging, but wondered if anyone else has had similar issues with ALLOCATE and /CHECKMATE.

After some more investigation, it would appear that using the ALLOCATE on a TYPE is the reason for the error. If I declare a variable of the same size as my TYPE and perform the ALLOCATE, it works just fine.

This might be related to a previous posting regarding the use of ABSOLUTE_ADDRESS= in an ALLOCATE; it doesn't work properly for a TYPE but works fine for an intrinsic type.

There is a limited amount of memory available when calling __PALLOCATE2. This call can be avoided by using /INHIBIT_CHECK 4 when /CHECK is used.

This bypasses the Fortran pointer checking code.

I have a bit of test code and the compile option did not solve the problem in /CHECK. Again, allocating an intrinsic type versus a user-defined TYPE causes different things to happen.

As an additional test, I defined a simple TYPE that contains another defined TYPE, as well as the more complex TYPE that is causing the 'troubles. I cannot get the simple TYPE to fail, yet the complex TYPE does fail.

I have included my test code here, and a DropBox link to the include file.

The only Project option that might be unusual is /SAVE to match my old code assumptions.

	include 'z:\cmasterf95\parametersandtypes.ins'
	integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: temp_record(:)
    TYPE (MASTER_CATALOG_REC_EXTENDED):: blank
	character*511,pointer,dimension(:):: temp_record1
   	inquire(iolength=i)blank
    print *,i
    do j=1,10000
    allocate(temp_record(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    inquire(iolength=k)temp_record(1)
    print *,j,i,k
    end do
    read *,i
    end

https://www.dropbox.com/s/jlybzx24juiithy/ParametersAndTypes.ins?dl=0

This might be a clue. I can now get the program to fail. Just initialize a variable in the TYPE, and when you start allocation, it will program fault.

The 10th line of the program contains an initialization (DEFG=0). Remove the initialization, and the program will end normally.

	integer,parameter:: max_catalog_records = 65530
	integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg=0
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    character*511,pointer,dimension(:):: temp_record1
    do j=1,10000
    allocate(temp_record2(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    print *,j,i
    end do
    !read *,i ! make the program pause so we can run task manager
    end

Bill,

In your latest example, I don't understand why you have introduced the complexity of multiple layers of derived types. You also use 'defg' as integer or character in different types, which could make the code complex to read. I'd be interested to see how you address the components of my_type and my_type2 in an unambiguous way.

In your first example, I did not understand where you defined 'extracted_logs', or why it is needed. It appears to be a collector for a single element of type extract_log_holeid ?

If you have allocated arrays as components of a derived type, all these allocated components must be deallocated before you deallocate the derived type.

Bill

I looked at your program with the Dropbox include file.

My first question relates to the line

    TYPE (MASTER_CATALOG_REC_EXTENDED),pointer,dimension(:):: temp_record(:)

This appears to have the dimension twice. I guess it is a one-dimensional array and one of the (:) should be omitted.

Then the lines

    do j=1,10000
    allocate(temp_record(max_catalog_records),stat=i)

looks like the allocation should not be within the DO loop.

Hi Paul,

Both of your comments are valid.

The first one was my error, a holdover from using the keyword ALLOCATABLE.

The second was by design, though, as I wanted to try the allocation many many times to use up all available memory to check the return code from the ALLOCATE.

Bill

John,

Just like many small programs others have used to illustrate an issue, the example here is contrived, but based on my unshared, original code.

And while your comments are, on their own, valid, they miss the entire point of this exercise.

If I can get a failure or undesired behavior to manifest itself in otherwise valid code, then who cares? There appears to be a problem, and the code sample is short enough to be tested and verified on Paul's machine.

Bill

Bill

Can you post corrected code for me to test for the failure that you describe?

The code post from Sun Dec 08, 2019 5:08 am has the simplified code that causes a failure.

Sorry, bad paste

   integer,parameter:: max_catalog_records = 65530
   integer i,j,k
    type :: my_type2
    sequence
    character*2400 :: defg
    end type
    type :: my_type
    sequence
    type (my_type2):: abcd
    integer:: defg=0
    end type
    type (my_type),pointer,dimension(:):: temp_record2
    character*511,pointer,dimension(:):: temp_record1
    do j=1,10000
    allocate(temp_record2(max_catalog_records),stat=i)
    if(i.ne.0)exit ! get out if we cannot get any more memory allocated
    print *,j,i
    end do
    !read *,i ! make the program pause so we can run task manager
    end

FTN95 is not coping with the type initialisation...

integer:: defg=0

at the lower TYPE level. This is a known issue.

If you remove the =0 then for me the do loop stops at j = 10 when the memory is exhausted.

I also moved the print statement so that I could see the i value when the ALLOCATE failed.

Paul, thanks for the verification.Working around the issue...

Bill

Bill,

I do use allocatable arrays in an allocatable derived type, but not allocatable derived types inside an allocatable derived type.

In my case, I first allocate the allocatable derived type, which provides the structure for defining the allocatable components, then carefully manage the allocate/deallocate of these components.

It is interesting that for all compilers I use, determining the size of this data structure reports the memory allocated for managing the size of the allocatable components, but not the size of the component data after it has been allocated. Not what is wanted.

I am surprised that FTN95 allows your definition approach. I worry that using derived types inside derived types will one day find an ICE in most compilers. The derived type is only defined once in a module so keep it simple.

John, I typically do not allocate memory for a TYPE with more allocations inside. I said typically on purpose, as it does not bother me to do so. That being said, using a derived TYPE inside another TYPE (and perhaps in another TYPE) is actually a good practice. But, it does take some care when designing the nested TYPE's as well as the code that operates on it/them. I have numerous examples of this in my code.

My most complex is a combination of three %ls, one %rd, and one %rs. A selection in the lead %ls rebuilds the next %ls control, which rebuilds the next %ls control. The %rd is only enabled for input when the last %ls is one specific selection, and the %rs is totally optional. Each of the %ls controls has a separate TYPE (because string lengths are different in each one) that maintains itself. Over this structure is a set of data specific routines that allow the controls to interact. These are then coupled (depending on the function desired) to one of 4 additional sets of control to additional data specification.

I also have a TYPE that allows me to handle the same basic data inside a %lv, but display the data in a specific format. So, I came up with a way of passing the address of a processing routine in the TYPE, and make a dynamic call. The handler for the %lv contents is unique for the specific process, while the generic pieces allow me to select items, remove them from the display (and optionally from the raw data), or sort by a column of the %lv. All generically. Saves a LOT of coding time!

I look at a set of controls that performs one specific function to be a great user of TYPE's. When I then couple that set of controls inside another bunch of controls that use their own TYPE's, I can have two or three separate types, sometimes 3 levels deep.

It is regrettable that Fortran chose the % sign as a data separator, versus using a decimal point like in 'C'. It makes reading the code a bit more complex to have a %.

Bill