View previous topic :: View next topic |
Author |
Message |
KennyT
Joined: 02 Aug 2005 Posts: 317
|
Posted: Fri May 25, 2018 8:47 am Post subject: changing the stack at runtime? |
|
|
i think it's true to say that, in an application that uses DLLs, it's only necessary to change the stack value in the link script for the .exe to overcome a stack overflow.
our application has two processes with conflicting stack needs. one needs a small stack (5mb) to avoid grabbing too much memory (with the default value of 50Mb, this process runs out of memory at about 2Gb, with stack set to 5Mb it runs up to 3.5Gb). The other process requires at least 20Mb stack or it gets a stack overflow.
So the question is, can i override the stack at runtime using a switch or do i have to provide different .exe files and ask the users to run the 'other' one if/when they get an error?
K |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
Posted: Fri May 25, 2018 10:25 am Post subject: |
|
|
Kenny
The stack size can only be set when linking. |
|
Back to top |
|
|
wahorger
Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA
|
Posted: Fri May 25, 2018 5:12 pm Post subject: |
|
|
So, this is a 64-bit application, right? Just curious... |
|
Back to top |
|
|
KennyT
Joined: 02 Aug 2005 Posts: 317
|
Posted: Fri May 25, 2018 5:38 pm Post subject: |
|
|
no, 32-bit at the moment.
64-bit to come later.
K |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Fri May 25, 2018 11:16 pm Post subject: |
|
|
Can we have a new page in ftn95.chm something like "Stack size for dummies", or more "How to manage the stack"
We need actual examples of:
* how to reset the stack when using Slink or Slink64
* how to find out how big the stack has been set
* how to interrogate a .exe as to what the stack size is
* how to read a .map file to see what the stack size is
(can these features be included in Slink and Slink64, if not available)
Using the stack is probably the most uncertain aspect of program development for all but a few FTN95 users. My approach has always be to try and not use the stack, by not using local or automatic arrays. I use ALLOCATE for any large array.
ps: If this works out, we could then consider another helpful page on "how to use common and modules with DLLs" |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
Posted: Sat May 26, 2018 7:39 am Post subject: |
|
|
Thanks John. I will make a note of your suggestion. |
|
Back to top |
|
|
silicondale
Joined: 15 Mar 2007 Posts: 252 Location: Matlock, Derbyshire, UK
|
Posted: Tue Apr 02, 2019 3:36 pm Post subject: |
|
|
I just hit a stack size problem (running 32-bit). Allocatable array size 35,000,000 (REAL* blows the stacl. Error message says I need to re-link, but I don't have the SLINK documentation handy (I'm at a project site in deepest rural Portugal). Can i just re-link (what command?) or is there some other solution? _________________ (Steve Henley) |
|
Back to top |
|
|
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 7933 Location: Salford, UK
|
|
Back to top |
|
|
silicondale
Joined: 15 Mar 2007 Posts: 252 Location: Matlock, Derbyshire, UK
|
Posted: Tue Apr 02, 2019 10:14 pm Post subject: |
|
|
Many thanks! _________________ (Steve Henley) |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Wed Apr 03, 2019 5:48 am Post subject: Re: |
|
|
silicondale wrote: | I just hit a stack size problem (running 32-bit). Allocatable array size 35,000,000 (REAL*8) blows the stacl. |
Allocatable arrays do not go on the stack (they go on the heap), so I am not sure of your diagnosis of the problem.
Your allocatable arrays are 280 Mbyte, which should not be a major issue.
If you are running on a 64-bit OS with high 32-bit memory usage, these could be allocated above 2gb address.
If you do have a stack overflow, check for local or automatic arrays or possible temporary arrays, say due to array sections. This problem can be overcome by making these arrays allocatable. |
|
Back to top |
|
|
silicondale
Joined: 15 Mar 2007 Posts: 252 Location: Matlock, Derbyshire, UK
|
Posted: Fri Apr 05, 2019 6:34 pm Post subject: |
|
|
Thanks John - I think you're right, and the problem had nothing to do with the allocatable arrays. I found a workaround for that one and will come back to it later.
I now have another problem. Allocatable REAL*8 array, dimension 85 million, and getting message that allocation failed. This is 680 megabytes - is that too big? It could well be too big anyway, as I'm trying to do a sort, and I want it to finish within my lifetime ! It's a set of timestamps from a robot that is returning huge volumes of data - and some of the timestamps are coming back out of sequence. I think the practical solution is probably to filter the data and work on a small subset for now. _________________ (Steve Henley) |
|
Back to top |
|
|
DanRRight
Joined: 10 Mar 2008 Posts: 2826 Location: South Pole, Antarctica
|
Posted: Sat Apr 06, 2019 12:39 am Post subject: |
|
|
Johns suggestion will give you almost 1 GB more with /3GB. But without any changes to the code try first playing with the stack allocation number like in my case 950000000 (here stack is almost 1 GB)
Code: | slink subs0.obj subs1.obj subs2.lib subs3.dll /stack:950000000 /3gb >linkReport__ | Think also moving to /64 keeping /32 alive for independent debugging hints |
|
Back to top |
|
|
mecej4
Joined: 31 Oct 2006 Posts: 1891
|
Posted: Sat Apr 06, 2019 3:56 am Post subject: Re: |
|
|
silicondale wrote: | I now have another problem. Allocatable REAL*8 array, dimension 85 million, and getting message that allocation failed. This is 680 megabytes - is that too big? It could well be too big anyway, as I'm trying to do a sort, and I want it to finish within my lifetime ! It's a set of timestamps from a robot that is returning huge volumes of data - and some of the timestamps are coming back out of sequence. I think the practical solution is probably to filter the data and work on a small subset for now. |
"Some of the timestamps are out of sequence" -- that is before sorting, I presume? If not, the sort algorithm/implementation has a bug. If only a small number of timestamps in the input are out of sequence, the first thing to try is to divert those into a second buffer or file, sort those, and merge the result with the bulk of the data that is already in sequence.
If you provide some details and examples of the data, perhaps we could help. |
|
Back to top |
|
|
silicondale
Joined: 15 Mar 2007 Posts: 252 Location: Matlock, Derbyshire, UK
|
Posted: Sat Apr 06, 2019 7:35 am Post subject: |
|
|
Indeed so - before sorting. That is the reason the sort is necessary. And yes indeed, one option is to divert the out-of-order timestamps into a separate file, then sort that and merge it back. That's effectively what a merge-sort would do, and is the workaround that I could adopt. At the cost of writing another (and fairly simple) program to do this. Certainly would avoid the need for such a huge allocatable array ! Good thinking, mecej ! _________________ (Steve Henley) |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2556 Location: Sydney
|
Posted: Sat Apr 06, 2019 8:25 am Post subject: |
|
|
I have done some testing of 85 million values on 2.8ghz i5.
If the numbers are only slightly out of sequence:
do i = 1,n
call random_number(real_array(i))
real_array(i) = real_array(i) + real(i))*0.125
end do
This results in quick sort of 4 seconds, while bubble sort of 2 seconds. (shell takes 4 seconds also)
If the values are random ( call random_number(real_array) ) my quick sort takes about 50 seconds. My bubble and shell sorts are still running after 2 hours 50 minutes !!
For large N, O(n)^2 is much slower than O(n).log(n), which shows how much better a more suited algorithm can be.
John
I will post a link to the test program when the shell sort finishes or I give up !! |
|
Back to top |
|
|
|