Silverfrost Forums

Welcome to our forums

Voice of doom: Stack overflow in 64bit

19 Jul 2019 4:12 #24043

Looks like I jinxed to myself few days back when wrote in the other thread that i do not like to see the meaningless for 99.99999% users crash reports. Well, programming devilry clearly reads this forum and first time in history it started giving me unbelievable crash reports like that. https://i.postimg.cc/tRZcJkkq/Stack-Overfl.jpg You wanted no binary garbage - you got it. After clicking OK program quietly disappears without a warnings or traces together with the SDBG64. No even info about # of offending line.

How come 64bit program can have stack overflow when it was promised that in 64bits there will be no damn stack anymore (or its limit will automatically adjust without user knowing about stack existence)? Same happens when i debug : i do step by step to the line which should call integer function and at that same moment i get this stack overflow.

This happened when i increased few array variables sizes to 61x61x300 (declared in the source, i.e. non-allocatable). These small arrays are nothing compared to the total memory. There exist other allocatable arrays MUCH larger in size than these, grabbing almost entire memory of PC. Decreasing their size has no effect on stack overflow. Now i can not even load functions containing these smaller arrays -- immediately getting stack overflow and crash.

And later that day true miracles started. I returned back to older arrays but this did not help and other problems started to appear. Inside DO loops execution of code started jumping over multiple lines as if they do not exist. This can be even recorded on movie because is reproducible in debugger...Is today a saturday when this devilry usually danced full force on my dead code ? 😃)) But i have never seen such crazy behavior before. Developers have take a note, that if their compiler will work OK on my codes, it will work on most of others. Have to stop. Will check later if this is the code, or computer or compiler got corrupted. Reboot of computer like in the times of old FTN77 did not help...And always there is some chance that processor hits the memory addressable limit...Fun is that after increasing array sizes code started working ok. Hell knows what's going on with these bewitched computers.

20 Jul 2019 6:47 #24045

Dan

There is nothing for me to work with here.

Who made the promise and where? Where is the means of reproducing your failure?

20 Jul 2019 10:17 #24046

Paul,

It’s a Cri de Coeur, a cry of exasperation or maybe desperation rather than for immediate help. Desperate Dan* knows you have to beware what you wish for, although whether or not he knows where the meme lies in the Aarne-Thompson-Uther Index is another matter. Note that Dan’s recourse to the language of Folklore in his final paragraph could be classified under several headings in the A.T.U index!

If Fairy Tales don’t appeal, then there is always The Monkey’s Paw.

The real point that worries me is the semi-religious belief that 64 bit-ness is the answer to everything, when clearly it is not, in the way that .NET isn’t, and ‘modern Fortran’ isn’t either.

Since a stack overflow occurs when you reach the end (and try to pass it) of a data structure, then you need to be careful as you approach it. It’s like walking towards the edge of a precipice: method 1 is don’t go near, method 2 is to wear a safety harness. In Fortran, method 1 is probably to avoid the use of local variables that take up much stack space. Method 2 might be to have a function that tells the program how much stack space is used or left at any time – perhaps graphed in real time.

I’d be prepared to bet that the problem occurred when Dan edited something with a ’replace all’ and something unintended was replaced. (And it stayed altered when the smaller array extents were re-applied). Such a thing happened to me while experimenting with colours for a logo and changed all RGB@ parameters from (0,255,255) to something else lighter, and all of a sudden my grid lines became less visible – I’d forgotten since 2005 that they were that particular colour. Dan should revert to a backup, and all will be fine and Dandy.

Eddie

When the problem is resolved, I suggest dining on Aunt Aggie’s Cow Pie*! **British meaning.

21 Jul 2019 4:42 (Edited: 21 Jul 2019 3:34) #24049

Eddie, you got me 99.9999% right. I had bad day yesterday, like no other one, no new things worked plus all reliable old things broke. Not beliving my eyes i had to reverse to backups almost one month old. Now i suspect something wrong could be in OpenGL. Because it influenced the appearance of Stack overflow error

The only tiny bit where you were wrong is how to take care of stack. You know well i am furious at its hiccups in previous 32bit compiler and hoped in 64bits it will not be needed. Why the heck this stack still exists in 64bits? This is utter rudiment of poor older times a la '640k will be enough for everyone' but actually this was not enough for anything (today we have 100,000 times more). Default restriction on 100 or 200 MB in 32 bit FTN95 caused a lot of headache. Changing it was not always working reliably. Now with 64bits we can get 10-100 times more RAM why the heck to keep stupid stack low? Is there at least one reason to have word 'stack' not being exterminated from the vocabulary?

21 Jul 2019 9:27 #24054

Dan

64 bit FTN95 is not a new compiler it only has a new 'backend'. If the concept of using a stack is old hat (which I doubt) then we have to live it.

However, it may be possible to get the linker to increase the default stack size depending on the amount of RAM available.

21 Jul 2019 3:49 #24058

Just the way stack is set in 32bit can scare anyone. To setup 12GB stack limit for example you now have to count zeros till you are lost.

/stack:12000000000

I for example ones miscalculated zeros and was out of luck for several weeks pulling my last hairs while trying to find not crashing variants. Please change that for megabytes of least, it was set in bytes a decade before Bill Gates' quote 😃

But better after clicking on OK on that Error Message above would be to receive this, Paul https://i.postimg.cc/R0N79bSB/No-More-Stack.jpg

22 Jul 2019 6:29 #24060

So this is all just a joke. I confess that you had me puzzled by the first dialog which I now assume you created yourself.

22 Jul 2019 2:55 #24061

I'd love to be kidding like this but the original dialog was real. And yes, kidding too, because can not blame compiler for that crash - need to check first if this is not due to hitting 64bit limit of the processor. Damn Intel to sell more chips makes limits on processors gazillion times less than formally 64bit OS allows. To be exact, 64bits allow 4 billion times the 32bit limit which as we well remember was 4GB while Intel gave just ~10x more. So i see sometimes i can not load programs, or browsers, and even sometimes graphics driver aborts to 640x480 (but good it does not crash the computer - they amicably solved the problem of crashes). Now need to buy new processor, new mobo, more RAM...

22 Jul 2019 9:46 #24064

I thought it's FTN95 64bit. I have another screenshot i saved - may be Silerfrost will tell exactly if this is theirs or Microsoft. Will post it later if i have not lost it

23 Jul 2019 4:22 #24071

My best guess is that the first dialog is generated by OpenGL.

There is probably an OpenGL error function that could be used by ClearWin+ to provide a trace back. If anyone has the details then please let me know.

Dan: I know that you don't approve of trace backs but they are a great help to those who can tolerate them. In this context there is certainly nothing else that we can do to help.

23 Jul 2019 8:39 #24072

Here is the screenshot continuation of unusual stack overflow. Could be OpenGL related, it recently behaved unusually few times.

https://i.postimg.cc/8CfJF5Z0/Image126.jpg

24 Jul 2019 6:38 #24080

The extra information makes a big difference. This looks like a call to the API MessageBox function followed by a call to a ClearWin+ function that raises the exception. This is how ClearWin+ reacts to a user or system error reported by OpenGL so it looks like the error report is generated by the OpenGL library.

24 Jul 2019 7:21 #24081

Stack overflows are difficult for the debugger (or any tool). Firstly it means there are megabytes of 'stuff' on the stack. That stuff could be stack frames from recursion gone mad or arrays or anything. The exception will happen mid way through more stack being used so it is very difficult to tell what is valid and what isn't.

25 Jul 2019 7:42 #24086

Here is an quote from the help for SLINK64.

SLINK64 can use 'stack' as an alternative to 'stack_size'. 'stack' takes a decimal value that is the required size in (decimal) megabytes (i.e. millions).

Further information is located at x64 Platform->Further information about 64 bit FTN95.

27 Jul 2019 4:26 #24100

Dan,

I certainly agree that the functionality of the stack is poor and I would suggest that it's design is lazy. The heap has the ability to extend to multiple memory locations.

I don't agree with your post, as the weakness of the stack has been frequently described and should be avoided.

The solution is to avoid the use of the stack, mainly by moving large arrays to the heap. This can be done explicitly using ALLOCATE or can be done by selecting compiler options for large arrays to go to the heap. (Not sure if this is the case with /64 ?) Temporary arrays in array sections and array syntax can be an issue, which should also be avoided with past versions of FTN95.

'Further information about 64 bit FTN95' provides some information about this

John

27 Jul 2019 6:16 #24101

John

When you write that 'the design of the stack' is lazy, is that directed at Microsoft? In what way might it have been designed differently? I wonder how you have formed this opinion.

Paul

27 Jul 2019 8:10 #24102

Quoted from PaulLaidler When you write that 'the design of the stack' is lazy, is that directed at Microsoft?

Yes, definitely, although I think most operating systems have a similar problem.

Quoted from PaulLaidler In what way might it have been designed differently?

The stack is limited to a single memory block, while the heap has multiple memory blocks.

I do not understand why the stack was not designed as extendable, rather than report a stack overflow error. While this error can be caused by the programmer when using more that expected array sizes, this error is more commonly due to temporary memory allocation which is initiated by the system. The memory manager should fix it, especially with 64-bit.

I have forgotten how FTN95 manages large temporary arrays being allocated on the heap. Isn't this by default with /64 ?

I was also trying to point out that this is a well known problem, which should be managed as:

large local arrays should be defined using ALLOCATE

large local or temporary arrays should be allocated on the heap.

'large' needs a definition, which I thought was 20k for FTN95.

John

27 Jul 2019 11:31 #24104

John

The announcement for v8.50 https://forums.silverfrost.com/Forum/Topic/3553 includes

(https://forums.silverfrost.com/Forum/Topic/3553) includes [quote:ecd12ad656]The size of the FTN95 stack is effectively no longer limited.

It would be interesting to test this to see how far you can go.

The instructions in the help file includes information on what to do with very large local arrays but this will need to be revised in the light of the above announcement.

27 Jul 2019 11:36 (Edited: 30 Jul 2019 6:39) #24105

Paul,

Are the size in 32bit and 64bit when you do /stack=size measured differently? For sure the size in 32bits is in KBs but you are saying in 64bits it is in MBs?

Do i correctly understand that ini file addition to SLINK was done for convenience? Say if you do something like quick compilation FTN95 prog.f95 /link you will be able to compile it if large stack is needed? (compilation will not go if do that this way FTN95 prog.f95 /link /stack=1024 ). Otherwise there is no problem to add to BAT file /stack=value and forget about it

John, Doing things by declaring fixed arrays and avoiding allocation/deallocation is just from laziness. Formally with 64bits we have what looks like an infinite ocean of RAM space. And expecting that almost infinity trying to run the code you're abruptly stepping on a rack at the 1/100 of what you have, and even below 32bit limit. Brain just refuses to accept existence of such small stack limit with 64bits and you always waiting that this year or maximum next one the stack concept will be called obsolete. If somebody likes the cars with the manual shift - fine, take and drive them, but most just do not like to deal with something no one knows what it is made for.

27 Jul 2019 12:49 #24106

Dan

'stack_size' takes a hex value for the number of bytes.

'stack' takes a decimal value which SLINK64 multiplies by one million to get the number of bytes.

The default value is 32MB but this can be set to some other value using Slink64.ini. This is for users who don't want to use 'stack_size' or 'size' but would prefer a larger default for the stack size reserve.

I tested it and ran our test suite using a default reserve of 16GB (which is the size of my RAM) and it did not complain but very large local arrays are currently not included in our tests.

It means that having a very large reserve does not appear to cause problems. The operating system will commit memory from this reserve on demand but otherwise the memory is still available for other purposes.

I don't know what will happen when it comes to creating very large local arrays but I recommend having a fire extinguisher to hand.

Please login to reply.