forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

changing the stack at runtime?
Goto page 1, 2  Next
 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
KennyT



Joined: 02 Aug 2005
Posts: 281

PostPosted: Fri May 25, 2018 8:47 am    Post subject: changing the stack at runtime? Reply with quote

i think it's true to say that, in an application that uses DLLs, it's only necessary to change the stack value in the link script for the .exe to overcome a stack overflow.

our application has two processes with conflicting stack needs. one needs a small stack (5mb) to avoid grabbing too much memory (with the default value of 50Mb, this process runs out of memory at about 2Gb, with stack set to 5Mb it runs up to 3.5Gb). The other process requires at least 20Mb stack or it gets a stack overflow.

So the question is, can i override the stack at runtime using a switch or do i have to provide different .exe files and ask the users to run the 'other' one if/when they get an error?

K
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6096
Location: Salford, UK

PostPosted: Fri May 25, 2018 10:25 am    Post subject: Reply with quote

Kenny

The stack size can only be set when linking.
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 666
Location: Morrison, CO, USA

PostPosted: Fri May 25, 2018 5:12 pm    Post subject: Reply with quote

So, this is a 64-bit application, right? Just curious...
Back to top
View user's profile Send private message Visit poster's website
KennyT



Joined: 02 Aug 2005
Posts: 281

PostPosted: Fri May 25, 2018 5:38 pm    Post subject: Reply with quote

no, 32-bit at the moment.

64-bit to come later.

K
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 2147
Location: Sydney

PostPosted: Fri May 25, 2018 11:16 pm    Post subject: Reply with quote

Can we have a new page in ftn95.chm something like "Stack size for dummies", or more "How to manage the stack"

We need actual examples of:
* how to reset the stack when using Slink or Slink64
* how to find out how big the stack has been set
* how to interrogate a .exe as to what the stack size is
* how to read a .map file to see what the stack size is

(can these features be included in Slink and Slink64, if not available)

Using the stack is probably the most uncertain aspect of program development for all but a few FTN95 users. My approach has always be to try and not use the stack, by not using local or automatic arrays. I use ALLOCATE for any large array.

ps: If this works out, we could then consider another helpful page on "how to use common and modules with DLLs"
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6096
Location: Salford, UK

PostPosted: Sat May 26, 2018 7:39 am    Post subject: Reply with quote

Thanks John. I will make a note of your suggestion.
Back to top
View user's profile Send private message
silicondale



Joined: 15 Mar 2007
Posts: 235
Location: Matlock, Derbyshire, UK

PostPosted: Tue Apr 02, 2019 3:36 pm    Post subject: Reply with quote

I just hit a stack size problem (running 32-bit). Allocatable array size 35,000,000 (REAL*Cool blows the stacl. Error message says I need to re-link, but I don't have the SLINK documentation handy (I'm at a project site in deepest rural Portugal). Can i just re-link (what command?) or is there some other solution?
_________________
(Steve)
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 6096
Location: Salford, UK

PostPosted: Tue Apr 02, 2019 5:03 pm    Post subject: Reply with quote

There is an online link here...

https://silverfrost.com/ftn95-help/slink/interactive_mode.aspx
Back to top
View user's profile Send private message
silicondale



Joined: 15 Mar 2007
Posts: 235
Location: Matlock, Derbyshire, UK

PostPosted: Tue Apr 02, 2019 10:14 pm    Post subject: Reply with quote

Many thanks!
_________________
(Steve)
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 2147
Location: Sydney

PostPosted: Wed Apr 03, 2019 5:48 am    Post subject: Re: Reply with quote

silicondale wrote:
I just hit a stack size problem (running 32-bit). Allocatable array size 35,000,000 (REAL*8) blows the stacl.

Allocatable arrays do not go on the stack (they go on the heap), so I am not sure of your diagnosis of the problem.

Your allocatable arrays are 280 Mbyte, which should not be a major issue.
If you are running on a 64-bit OS with high 32-bit memory usage, these could be allocated above 2gb address.
If you do have a stack overflow, check for local or automatic arrays or possible temporary arrays, say due to array sections. This problem can be overcome by making these arrays allocatable.
Back to top
View user's profile Send private message
silicondale



Joined: 15 Mar 2007
Posts: 235
Location: Matlock, Derbyshire, UK

PostPosted: Fri Apr 05, 2019 6:34 pm    Post subject: Reply with quote

Thanks John - I think you're right, and the problem had nothing to do with the allocatable arrays. I found a workaround for that one and will come back to it later.

I now have another problem. Allocatable REAL*8 array, dimension 85 million, and getting message that allocation failed. This is 680 megabytes - is that too big? It could well be too big anyway, as I'm trying to do a sort, and I want it to finish within my lifetime ! It's a set of timestamps from a robot that is returning huge volumes of data - and some of the timestamps are coming back out of sequence. I think the practical solution is probably to filter the data and work on a small subset for now.
_________________
(Steve)
Back to top
View user's profile Send private message Visit poster's website
DanRRight



Joined: 10 Mar 2008
Posts: 2124
Location: South Pole, Antarctica

PostPosted: Sat Apr 06, 2019 12:39 am    Post subject: Reply with quote

Johns suggestion will give you almost 1 GB more with /3GB. But without any changes to the code try first playing with the stack allocation number like in my case 950000000 (here stack is almost 1 GB)
Code:
slink subs0.obj subs1.obj subs2.lib subs3.dll /stack:950000000 /3gb >linkReport__
Think also moving to /64 keeping /32 alive for independent debugging hints
Back to top
View user's profile Send private message
mecej4



Joined: 31 Oct 2006
Posts: 1216

PostPosted: Sat Apr 06, 2019 3:56 am    Post subject: Re: Reply with quote

silicondale wrote:
I now have another problem. Allocatable REAL*8 array, dimension 85 million, and getting message that allocation failed. This is 680 megabytes - is that too big? It could well be too big anyway, as I'm trying to do a sort, and I want it to finish within my lifetime ! It's a set of timestamps from a robot that is returning huge volumes of data - and some of the timestamps are coming back out of sequence. I think the practical solution is probably to filter the data and work on a small subset for now.

"Some of the timestamps are out of sequence" -- that is before sorting, I presume? If not, the sort algorithm/implementation has a bug. If only a small number of timestamps in the input are out of sequence, the first thing to try is to divert those into a second buffer or file, sort those, and merge the result with the bulk of the data that is already in sequence.

If you provide some details and examples of the data, perhaps we could help.
Back to top
View user's profile Send private message
silicondale



Joined: 15 Mar 2007
Posts: 235
Location: Matlock, Derbyshire, UK

PostPosted: Sat Apr 06, 2019 7:35 am    Post subject: Reply with quote

Indeed so - before sorting. That is the reason the sort is necessary. And yes indeed, one option is to divert the out-of-order timestamps into a separate file, then sort that and merge it back. That's effectively what a merge-sort would do, and is the workaround that I could adopt. At the cost of writing another (and fairly simple) program to do this. Certainly would avoid the need for such a huge allocatable array ! Good thinking, mecej !
_________________
(Steve)
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 2147
Location: Sydney

PostPosted: Sat Apr 06, 2019 8:25 am    Post subject: Reply with quote

I have done some testing of 85 million values on 2.8ghz i5.

If the numbers are only slightly out of sequence:
do i = 1,n
call random_number(real_array(i))
real_array(i) = real_array(i) + real(i))*0.125
end do
This results in quick sort of 4 seconds, while bubble sort of 2 seconds. (shell takes 4 seconds also)

If the values are random ( call random_number(real_array) ) my quick sort takes about 50 seconds. My bubble and shell sorts are still running after 2 hours 50 minutes !!

For large N, O(n)^2 is much slower than O(n).log(n), which shows how much better a more suited algorithm can be.

John

I will post a link to the test program when the shell sort finishes or I give up !!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group