forums.silverfrost.com

[JohnCampbell]

Paul,

Have you had any success with improving the use of the /3gb switch.
I think that most of the problems are associated with how SLINK /3gb works.
I have identified a number of problems with SLINK, although some may relate to my lack of knowledge of SLINK.
The problems I have identified, which I can�t solve include:
I do not appear to be able to create a memory SECTION of more than 2gb. If I do, the executable is not recognised by windows in cmd.exe
As a consequence I can not create a single array of more than 2gb, which I would see as a target to be possible. You stated that FTN95 can address arrays larger than 2gb. If so, then the problem is with SLINK.
Does win32 allow segments to be larger than 2gb ?
I can also not create multiple arrays, whose combined size exceed 2gb, as all large arrays are combined into the same .bss SECTION. I have experimented with
� named common
� un-named common
� local arrays and
� module definitions
I have little experience with Virtual Common, but when I tried this as an alternative, all large arrays were again placed in the same Virtual Common section.
The location of Virtual Common (it�s memory address) is not selected to maximise addressable memory available. There needs to be some control on where it can be located. Couldn�t SLINK place it at an appropriate address, rather than defaulting to an optimum address. In the example I tested common started at hex 20000000, or 512mb.
I expect that there needs to be some control on where the STACK and HEAP go, although I am not clear on what the two are. Any attempt I had to change the size of the STACK did not overcome the stack overflow being reported.
Is this a set of functionality that can be addressed to improve how SLINK creates a .exe for use with the /3gb switch, or am I misunderstanding how to manage SLINK ?

Regards John

ps: An alternative interpretation of the program not running may simply be due to where the heap and stack is placed and not the >2gb SECTION symptom, if SECTIONs > 2gb should work ?

[PaulLaidler]

We have made some progress with a new memory model which should allow an increase in the total amount of memory available when using the /3GB switch in the boot.ini file. Further details should be out soon together with a Beta version of salflibc.dll for users to try out.

However, it turns out that you cannot get more than 2GB in one chunk because of the way the OS distributes the memory available.

SLINK is designed to allow a total of 3GB of memory for release mode and a total of 2GB for debug modes (FTN95 /DEBUG etc). These defaults can be over-ridden by using /2GB and /3GB switches on SLINK (e.g. SLINK /3GB for FTN95 /DEBUG etc.)

[JohnCampbell]

Paul,

That is good news.

Three points :
1) As there is a section limit of 2gb, is it possible to have two ".bss" sections. SLINK should be able to manage the allocation of large arrays between the 2 sections. This would still limit the largest array to 2gb, but allow more than 2gb in total array areas. It is unlikely that the "not-array" code would approach 1gb.
2) At present, the solution to "stack overflow" and other stack related errors is very difficult to manage. Could the management of the stack be made easier. Even reporting it's address and size may make the debugging easier.
3) Could the location of VC be more manageable. Could the address be placed at the end of all other code, rather than address at 512mb.

I hope these points are of use.

regards John

[PaulLaidler]

John

I don't have immediate answers to any of your questions.
I will make enquires.

Paul

[JohnCampbell]

Paul,

It has often puzzled me why the management of memory, especially the stack and heap(s ?) has always been so unfriendly.
This is certainly an area where the designers of checkmate should play a part as we need a better interface to how this can be managed. It doesn't have to be this difficult.
I recall a program that could be used to change the stack of a .exe file. Could this concept be expanded to provide a memory layout report and possibly restructure the stack and heap if required.
I am still trying to understand what are the restrictions on getting 3gb of memory, when the /3gb switch is used in boot.ini

I look forward to further updates.

John

[JohnCampbell]

I have just downloaded the 3gb beta version. I look forward to trying it out on my 4gb machine when I return to work.
I am puzzled why the changes are only to the .dll and .lib, as my previous questions assumed that there were problems with SLINK and the stack management.
Was I wrong in my interpretation of the problem and is there and documentation of the management of the heap and stack in SLINK ?

John

[PaulLaidler]

I cannot give you a full answer to your questions without getting expert advise. I will aim to get you more information but it may take a couple of weeks before I can get back to you.

SLINK is fully documented in FTN95.chm. There is no other documentation. This includes information on how to set the program heap and stack size and the base address of "virtual common". Using the SLINK map facility provides data on how the linkage is arranged.

The memory usage that we have been investigating relates to the way in which global memory is allocated (by FTN95) at runtime. This involves runtime calls to Windows API functions like GlobalAlloc and HeapAlloc. By changing the way in way these calls are made (in salflibc.dll) we have seen some improvement in the total amount of global memory that is available to an executable.

[JohnCampbell]

Paul,

Thanks for the advice. It would be good to get some answers on the stack and heap.

I will however test the beta version that you have provided in the next few days with the programs I have and let you know how I go and what array limits I find.

My recent questions to microsoft and HP on /3gb have not been very helpful. I have a new HP workstation, so there should not be any problems with old hardware, which microsoft has suggested.

[JohnCampbell]

Paul,

I have now tested the new compiler you provided and can report that it increased the memory available from about 1.65gb to 1.85gb.

Without the FTN95_NEW_MEMORY switch, I fail at about 1.7gb with a stack overflow error.

With FTN95_NEW_MEMORY selected, I get to 1.85gb, before I get the error "c:\junk\big\big_array.exe is not a valid Win32 application."

I have sent you some results attached to and email to your hotmail account.

I have not tested while running other programs.
I also could not confirm I had the 3gb OS switch on in the operating system. I am pretty sure I did, but could not find the selected OS name. I will update if I find any different.

John.

After a long call to microsoft : you can not confirm that the /3gb switch is selected in boot.ini, so I assume it is.

Is the IMAGE_FILE_LARGE_ADDRESS_AWARE bit set in the .exe files from Slink with the /3gb option ?

[PaulLaidler]

If 1) you set the /3GB switch in your boot.ini file and 2) your are running 32bit Windows XP or similar OS and 3) your SLINK is recent then SLINK will set the IMAGE_FILE_LARGE_ADDRESS_AWARE bit in your exe provided you are not using a debugging FTN95/SCC mode. If you want the bit to be set whilst debugging then you must also apply the /3GB switch to your SLINK linking process.

Actually this is not strictly correct. SLINK will do its stuff regardless, but the IMAGE_FILE_LARGE_ADDRESS_AWARE bit will only be effective for certain operating systems and when the boot.ini is configured correctly.

[JohnCampbell]

Paul,

I have spent some time trying to confirm what I am doing.

For the computer settings in boot.ini and memory available:
I can not see any difference in the O/S settings if I include /3GB in boot.ini.
There appears to be no variable in SYSTEM or CONFIGURATION MANAGER which identifies that 3gb is in use.
Windows Task Manager ( ctl alt del ) only reports 3gb of physical memory, although 4gb is installed, which is a bug for pc's with 4gb installed.

For ftn95 options:
I can not see any difference in the .map file if, in SLINK, I use 3gb or not. If I select 2gb, then there is a change in the .exe, but the run error reported for either .exe file is the same. ( as /2gb does make a difference, perhaps the large addressing bit is being set ? ). Neither option is documented in SLINK commands.

As I can't get a .exe to go beyond 1.9gb, I can't confirm if the problem is the boot.ini /3gb option or slink /3gb option.

The error reported is .exe is not a valid Win32 application.
When is the location of the heap and stack defined ? My on-going suspicions are that they are conflicting with the large arrays, but I can not confirm this.
From the outside, the concept of 3gb of addressible memory does not appear to be too difficult to apply. there are a number of symbol addresses for slink to manage and then slink or cmd.exe place the stack in a safe location, that is compatible with win32.
I spent 45 minutes on the phone to microsoft, trying to get them to confirm if the OS name in boot.ini could be found or to confirm if 3gb was in action. This is a very frustrating problem !

Any ideas ?

My latest batch file to test is:

rem this tests big array for 3gb and standard stack
del large_array.mod
del big_array.obj
del big_3gb.exe
del big_2gb.exe
del big_none.exe
del big_3gb.map
del big_2gb.map
del big_none.map
ftn95 big_array /lis /xref
rem notepad big_array.lis
slink big_array.obj /3gb /map:big_3gb.map /file:big_3gb.exe
slink big_array.obj /2gb /map:big_2gb.map /file:big_2gb.exe
slink big_array.obj /map:big_none.map /file:big_none.exe
fc /b big_3gb.exe big_none.exe
fc /b big_3gb.exe big_2gb.exe
notepad big_3gb.map
big_3gb

Regards

John Campbell

[PaulLaidler]

Here is a copy of my boot.ini which I edited manually in order to include the /3GB switch....

[JohnCampbell]

Paul,

Thanks for that. I have the /3GB at the end. I will try it where you have placed it tomorrow.

Do you get past my apparent 1.85gb limit without getting "is not a valid Win32 application" error ?

John

[PaulLaidler]

The results should be the same provided you have /3GB in roughly the right place.

I do not have any useful information on the maximum amount of memory available. You will not be able to get more than 2GB in one chunk so 1.85 may be the best you can do. Using a number of smaller chunks hopefully will extend the limit.

[JohnCampbell]

Paul,

Do you reproduce the "not a valid Win32 application" error with /3GB in boot.ini ?
All my data is in the same .bss section, so how do I get a second chunk ?
Are there any successful test examples of SLINK and /3GB ?

John