forums.silverfrost.com

[DanRRight]

Got run-time error ( with 0.5TB RAM )

Insufficient virtual stack (FTN95 /VSTACK <MB-value>)

64bit also needs manual stack control ??? How to use it?

[PaulLaidler]

Dan

Can you send me the code for a program that compiles and runs except for this runtime error.

[DanRRight]

Sending it could be problematic...And making a demo could be almost impossible. That error appeared when i tried to load really large file of 100GB or so which also might demand even more RAM ( i expect with 64bits compiler we now got all limits off and allowed automatic unlimited increases with the code decide by itself. It is not likely the code might demand more than 1TB). During next few days i will investigate what caused this. Before i was able to load ~300GB even having 5x less real+virtual memory

[PaulLaidler]

FTN95 currently has a maximum "virtual" stack size of 8GB. This can be reduced but not increased by using /VSTACK <MB-value> on the FTN95 command line. The maximum value is automatically reduced when there is limited physical memory available.

The FTN95 "virtual" stack is used for so-called automatic arrays and for temporary arrays created by the compiler to handle array sections that are not contiguous or not known to be whole arrays.

With 0.5TB of RAM there is effectively no physical limit for this virtual stack so the limit is currently 8GB.

I have made a note that this needs to be reviewed.

[JohnCampbell]

Dan,

What is Vstack and how does it differ from stack or heap ??

I know about Stack and Heap, but what is Vstack and why is it limited to 8 GBytes, when Heap gets all virtual memory ?

I have recently come across this "VSTACK" limit when trying to write/read 8.5 GByte records to a binary file. ( this is an ongoing issue for solving "Fails to save arrays > 4GB"

I used " read (lu) vector(1:nn)"

The problem is that this array section initiates a temporary copy of vector(1:nn)

Paul indicates this when he stated "The FTN95 "virtual" stack is used for so-called automatic arrays and for temporary arrays created by the compiler to handle array sections that are not contiguous or not known to be whole arrays."

So the solution is to avoid these temproary copies of these very large array sections. Use ALLOCATE wherever possible !!

In the above case my solution was to replace the read statement with a "F77" wrapper :

[PaulLaidler]

John

I think that all of this should work with version 9.02 that can be downloaded from the Support "Sticky Post".

This "virtual" stack is the one that is created and used by FTN95 for 64 bit automatic arrays and compiler generated temporary arrays. It currently has an upper limit size of 8GB but this will be reviewed.

The stack is is generated by a system call to VirtualAlloc and this call is built into the startup code for the user's executable by SLINK64. As a result this call is not currently visible in an /EXPLIST listing.

[DanRRight]

I use 9.02
The place where i got this error is

[JohnCampbell]

Dan,

could you try:
dVolumeCell = XCellsize * dyCellsize * dZCellsize +1.d-30
DensityE3D = DensityE3D / dVolumeCell
ie remove the "array section"

and let me know what happens ?

Also you could try

dVolumeCell = 1.0 / ( XC0ellsize * dyCellsize * dZCellsize +1.d-30 )
DensityE3D = DensityE3D * dVolumeCell
Do you enable avx instructions for this time consuming calculation.

Increasing Vstack size allocation might be a problem for others without with 0.5TB RAM. Increasing the Vstack size reduces the virtual memory pool address size, but perhaps not the virtual memory allocation ?

It would not be an issue for physical memory usage, but might reduce the available virtual memory size ?

I could check, as Gfortran puts bigger memory address strides for stack and heap, compared to FTN95. I have not identified the Vstack address in memory maps.

[DanRRight]

John,
1) What "problems for others" you are scaring here if i will increase stack just for myself (if this limit is ever needed)? Debugger will tell you about your problem place. If there will be a problem it will be your code problem not the compiler problem like now

2) I have also such places: how to eliminate array section here for example?

[JohnCampbell]

Dan,

1) The available virtual memory (not address space) on the physical memory + paging space can sometimes be a limit on x64. This may not be a problem with a larger Vstack, as virtual memory is only allocated if the memory address ( memory pages) are set a value.
Something like "DensitySpecies(:,:,:,1) = 0" could allocate a lot of memory pages.

2) "F77 wrappers" are a great way to avoid temporary arrays.
The following could avoid the problem

[DanRRight]

This is a club of workarounders. Besides mecej4 no one reports any problems and bugs. And suggestions you'll not hear from absolutely anyone. If the company would not move since FTN77 by itself, all would still actively making workarounds in F77. Even AVX is a workaround. Do you know what is needed to change in the code to include AVX with gFortran or Intel? Nothing, just add compilation switch

Do gFortran and Intel also have 8GB limit?

[PaulLaidler]

FTN95 uses AVX in some contexts and no switch is required.

I have already said that this particular limit will be reviewed so I expect that it will at least be increased and possibly made configurable. Your request for no limit is uninformed.

Personally I find your comments disrespectful and hence counter productive because they could discourage others from using FTN95.

[DanRRight]

Sorry, by my words you can feel that the rhetoric between two camps of penguins is heating up in my Antarctica so the ice is melting . By the way, it is considered pro- not counter productive, and usually encourage and not discourage.
Anyway, from my side i apologize if my words sound offending, because, to be 100% objective, the workarounds also could be useful sometimes.

gFortran:
Up to the full RAM memory + swap 933 GB no any limits were found

[JohnCampbell]

[DanRRight]

I am sure you've heard that no one already optimizes codes by hand anymore, compilers do that better than average programmer. Programmer has just to write clear code, not a spaghetti nightmare. And in Soviet Russia already the codes optimize programmers.

gFortran with -O3 -march=native switch optimizes this code to AVX speeds without any workarounds. It even optimizes to the same speeds your Fortran77 workaround above, the FTN95 non-standard AVX@ of course it could not swallow. And yes it goes without crashes, exhausting 0.5 TB RAM it takes Swap. No any stacks. FTN95 stops at 8 GB.