3gb switch update ?

Paul,

Have you had any success with improving the use of the /3gb switch. I think that most of the problems are associated with how SLINK /3gb works. I have identified a number of problems with SLINK, although some may relate to my lack of knowledge of SLINK. The problems I have identified, which I can’t solve include: I do not appear to be able to create a memory SECTION of more than 2gb. If I do, the executable is not recognised by windows in cmd.exe As a consequence I can not create a single array of more than 2gb, which I would see as a target to be possible. You stated that FTN95 can address arrays larger than 2gb. If so, then the problem is with SLINK. Does win32 allow segments to be larger than 2gb ? I can also not create multiple arrays, whose combined size exceed 2gb, as all large arrays are combined into the same .bss SECTION. I have experimented with · named common · un-named common · local arrays and · module definitions I have little experience with Virtual Common, but when I tried this as an alternative, all large arrays were again placed in the same Virtual Common section. The location of Virtual Common (it’s memory address) is not selected to maximise addressable memory available. There needs to be some control on where it can be located. Couldn’t SLINK place it at an appropriate address, rather than defaulting to an optimum address. In the example I tested common started at hex 20000000, or 512mb. I expect that there needs to be some control on where the STACK and HEAP go, although I am not clear on what the two are. Any attempt I had to change the size of the STACK did not overcome the stack overflow being reported. Is this a set of functionality that can be addressed to improve how SLINK creates a .exe for use with the /3gb switch, or am I misunderstanding how to manage SLINK ?

Regards John

ps: An alternative interpretation of the program not running may simply be due to where the heap and stack is placed and not the >2gb SECTION symptom, if SECTIONs > 2gb should work ?

We have made some progress with a new memory model which should allow an increase in the total amount of memory available when using the /3GB switch in the boot.ini file. Further details should be out soon together with a Beta version of salflibc.dll for users to try out.

However, it turns out that you cannot get more than 2GB in one chunk because of the way the OS distributes the memory available.

SLINK is designed to allow a total of 3GB of memory for release mode and a total of 2GB for debug modes (FTN95 /DEBUG etc). These defaults can be over-ridden by using /2GB and /3GB switches on SLINK (e.g. SLINK /3GB for FTN95 /DEBUG etc.)

Paul,

That is good news.

Three points :

1. As there is a section limit of 2gb, is it possible to have two '.bss' sections. SLINK should be able to manage the allocation of large arrays between the 2 sections. This would still limit the largest array to 2gb, but allow more than 2gb in total array areas. It is unlikely that the 'not-array' code would approach 1gb.
2. At present, the solution to 'stack overflow' and other stack related errors is very difficult to manage. Could the management of the stack be made easier. Even reporting it's address and size may make the debugging easier.
3. Could the location of VC be more manageable. Could the address be placed at the end of all other code, rather than address at 512mb.

I hope these points are of use.

regards John

John

I don't have immediate answers to any of your questions. I will make enquires.

Paul

Paul,

It has often puzzled me why the management of memory, especially the stack and heap(s ?) has always been so unfriendly. This is certainly an area where the designers of checkmate should play a part as we need a better interface to how this can be managed. It doesn't have to be this difficult. I recall a program that could be used to change the stack of a .exe file. Could this concept be expanded to provide a memory layout report and possibly restructure the stack and heap if required. I am still trying to understand what are the restrictions on getting 3gb of memory, when the /3gb switch is used in boot.ini

I look forward to further updates.

John

I have just downloaded the 3gb beta version. I look forward to trying it out on my 4gb machine when I return to work. I am puzzled why the changes are only to the .dll and .lib, as my previous questions assumed that there were problems with SLINK and the stack management. Was I wrong in my interpretation of the problem and is there and documentation of the management of the heap and stack in SLINK ?

John

I cannot give you a full answer to your questions without getting expert advise. I will aim to get you more information but it may take a couple of weeks before I can get back to you.

SLINK is fully documented in FTN95.chm. There is no other documentation. This includes information on how to set the program heap and stack size and the base address of 'virtual common'. Using the SLINK map facility provides data on how the linkage is arranged.

The memory usage that we have been investigating relates to the way in which global memory is allocated (by FTN95) at runtime. This involves runtime calls to Windows API functions like GlobalAlloc and HeapAlloc. By changing the way in way these calls are made (in salflibc.dll) we have seen some improvement in the total amount of global memory that is available to an executable.

Paul,

Thanks for the advice. It would be good to get some answers on the stack and heap.

I will however test the beta version that you have provided in the next few days with the programs I have and let you know how I go and what array limits I find.

My recent questions to microsoft and HP on /3gb have not been very helpful. I have a new HP workstation, so there should not be any problems with old hardware, which microsoft has suggested.

Paul,

I have now tested the new compiler you provided and can report that it increased the memory available from about 1.65gb to 1.85gb.

Without the FTN95_NEW_MEMORY switch, I fail at about 1.7gb with a stack overflow error.

With FTN95_NEW_MEMORY selected, I get to 1.85gb, before I get the error 'c:\junk\big\big_array.exe is not a valid Win32 application.'

I have sent you some results attached to and email to your hotmail account.

I have not tested while running other programs. I also could not confirm I had the 3gb OS switch on in the operating system. I am pretty sure I did, but could not find the selected OS name. I will update if I find any different.

John.

After a long call to microsoft : you can not confirm that the /3gb switch is selected in boot.ini, so I assume it is.

Is the IMAGE_FILE_LARGE_ADDRESS_AWARE bit set in the .exe files from Slink with the /3gb option ?

If 1) you set the /3GB switch in your boot.ini file and 2) your are running 32bit Windows XP or similar OS and 3) your SLINK is recent then SLINK will set the IMAGE_FILE_LARGE_ADDRESS_AWARE bit in your exe provided you are not using a debugging FTN95/SCC mode. If you want the bit to be set whilst debugging then you must also apply the /3GB switch to your SLINK linking process.

Actually this is not strictly correct. SLINK will do its stuff regardless, but the IMAGE_FILE_LARGE_ADDRESS_AWARE bit will only be effective for certain operating systems and when the boot.ini is configured correctly.

Paul,

I have spent some time trying to confirm what I am doing.

For the computer settings in boot.ini and memory available: I can not see any difference in the O/S settings if I include /3GB in boot.ini. There appears to be no variable in SYSTEM or CONFIGURATION MANAGER which identifies that 3gb is in use. Windows Task Manager ( ctl alt del ) only reports 3gb of physical memory, although 4gb is installed, which is a bug for pc's with 4gb installed.

For ftn95 options: I can not see any difference in the .map file if, in SLINK, I use 3gb or not. If I select 2gb, then there is a change in the .exe, but the run error reported for either .exe file is the same. ( as /2gb does make a difference, perhaps the large addressing bit is being set ? ). Neither option is documented in SLINK commands.

As I can't get a .exe to go beyond 1.9gb, I can't confirm if the problem is the boot.ini /3gb option or slink /3gb option.

The error reported is .exe is not a valid Win32 application. When is the location of the heap and stack defined ? My on-going suspicions are that they are conflicting with the large arrays, but I can not confirm this. From the outside, the concept of 3gb of addressible memory does not appear to be too difficult to apply. there are a number of symbol addresses for slink to manage and then slink or cmd.exe place the stack in a safe location, that is compatible with win32. I spent 45 minutes on the phone to microsoft, trying to get them to confirm if the OS name in boot.ini could be found or to confirm if 3gb was in action. This is a very frustrating problem !

Any ideas ?

My latest batch file to test is:

rem this tests big array for 3gb and standard stack del large_array.mod del big_array.obj del big_3gb.exe del big_2gb.exe del big_none.exe del big_3gb.map del big_2gb.map del big_none.map ftn95 big_array /lis /xref rem notepad big_array.lis slink big_array.obj /3gb /map:big_3gb.map /file:big_3gb.exe slink big_array.obj /2gb /map:big_2gb.map /file:big_2gb.exe slink big_array.obj /map:big_none.map /file:big_none.exe fc /b big_3gb.exe big_none.exe fc /b big_3gb.exe big_2gb.exe notepad big_3gb.map big_3gb

Regards

John Campbell

Here is a copy of my boot.ini which I edited manually in order to include the /3GB switch....

[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\\WINDOWS='Microsoft Windows XP Professional' /fastdetect /3GB /NoExecute=OptIn

This illustrates where the /3GB switch can go.

You do not need to use the SLINK /3GB switch unless you are also using FTN95/SCC in a debugging mode.

Paul,

Thanks for that. I have the /3GB at the end. I will try it where you have placed it tomorrow.

Do you get past my apparent 1.85gb limit without getting 'is not a valid Win32 application' error ?

John

The results should be the same provided you have /3GB in roughly the right place.

I do not have any useful information on the maximum amount of memory available. You will not be able to get more than 2GB in one chunk so 1.85 may be the best you can do. Using a number of smaller chunks hopefully will extend the limit.

Paul,

Do you reproduce the 'not a valid Win32 application' error with /3GB in boot.ini ? All my data is in the same .bss section, so how do I get a second chunk ? Are there any successful test examples of SLINK and /3GB ?

John

Hi John

We have considered the questions that you raise and suggest that the best way forward would be for you to send us a small program/project that illustrates the problems that you are encountering.

In the mean time here is a sample program that allocates about 2.8 GB on my machine

real(2),pointer::gib(:)
real(1) size
size=0.0
do
  allocate(gib(10000000))
  size = size+80000000
  print*, size
enddo
end

Regards

Paul

Paul,

I emailed you the file big_new_memory.zip on 16 Sep to your hotmail account, which was the account you had previously advised for emailing sample programs. It is a small program with little code, but big arrays. It's a bit of a mess as there is a bit of commented out code, as I tried different ideas. My example used explicit array sizeing with common or modules. After 1.85 gb of arrays, the executable generated gives 'is not a valid Win32 application' error ?

I will try your example, using allocate. My big memory PC is at work. What pool of memory does allocate get the big chunks of memory from ?

John

ALLOCATE ends up using the Windows API GlobalAlloc or HeapAlloc.

We will look at your sample code as soon as we can.

Paul

Paul,

I tried the allocate approach with real*8 arrays and it failed after 1.15gb. the following is the commands I used to run it.

ftn95 big_alloc /lis /xref slink big_alloc.obj /3gb /map:big_alloc.map /file:big_alloc.exe

!     Last change:  JDC  29 Sep 2008   9:00 am
!  Program to test memory sizes
!
      module large_array
!
      real*10,   parameter :: one   = 1
      real*10,   parameter :: two   = 2
      real*10,   parameter :: eight = 8
!
      integer*4, parameter :: megabyte = 1024*1024/8
!
!   Did not work if declared in module
!      integer*4 array_a_size
!      real*8,    dimension(:), allocatable :: a8_array
!
      end module large_array

      use large_array
!
      integer*4 array_a_size
      real*8,    dimension(:), allocatable :: a8_array
!
      integer*4 i, m_100, m, ii
      real*10   s1, s2, sa
      character description*80
!
      open (unit=98, file='big_array.log', position='append')
!
      call date_time_label (98)
!
      do ii = 1000,2500,50
!
         array_a_size = ii * megabyte
         sa = dble (array_a_size) * 8. / 1024./1024./1024.
         write (description,1000) 'This test for 1 allocated array, memory allocated = ',sa,' gb'
         call report (description)
!
         allocate (a8_array(array_a_size)) 
         call report (' array a allocated')
!
         write (description,1001) 'initialising array A of ', array_a_size,' real*8 variables'
         call report (description)
!
         m_100 = (1024*1024/8)*100
         m = m_100
         do i = 1,array_a_size
            m = m-1
            if (m <= 0) then
               s1 = dble(i) * eight / dble(1024)**3 
               write (description, 1004) s1
               call report (description)
               m = m_100
            end if
            a8_array(i) = i
         end do
         call report ('  array a initialised')
!
         sa = array_a_size
         sa = sa*(sa+one)/two
!
         s1 = 0
         do i = 1,array_a_size
            s1 = s1 + a8_array(i)
         end do
         write (description,1002) '  do Sum of array a = ',s1, s1-sa
         call report (description)
!
         s2 = sum (a8_array)
         write (description,1002) '  Sum of array a    = ',s2, s2-sa
         call report (description)
!
         deallocate (a8_array)
         call report ('  array a deallocated')
      end do
!
1000  format (a,f0.3,a)
1001  format (a,b'zzz,zzz,zz#',a)
1002  format (a,es22.15,' (error=',es10.3,')')
1004  format ( f5.1,' gb initialised')
      end

      subroutine report (description)
!
!     returns the time in the form hh:mm pm
!
      character description*(*)
      real*8    seconds
!
      call get_run_time (seconds)
      WRITE ( *,1011) seconds, trim (description)
      WRITE (98,1011) seconds, trim (description)
!
      RETURN
 1011 FORMAT (f10.3,2x,a)
      END

[/code]

the post worked in preview, but clipped when sent. this is the rest of the code.

      subroutine get_run_time (seconds)
!
!     returns the elapsed time, accurate to .001 seconds (?)
!
      real*8 seconds
!
      CHARACTER date*8, time*10, zone*5
      INTEGER   values(8)
      real*8    start_time, x
      data start_time / -1 /
!
      call DATE_AND_TIME (date, time, zone, values)
!
      x = 60.0*60.0*values(5)     &
        +      60.0*values(6)     &
        +           values(7)     &
        +     0.001*values(8)
!   
      if (start_time < 0) start_time = x
      seconds = x - start_time
      RETURN
      END

      SUBROUTINE DATE_TIME_LABEL (LUNIT)
!
!     Displays the date and time on file unit LUNIT
!
      integer*4, intent (in)    :: lunit
!
      CHARACTER LABEL(12)*3, CBUF*22
      STDCALL   GETLOCALTIME 'GetLocalTime' (REF)
      integer*2 ia(8), YEAR,MONTH
      DATA LABEL / 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',            &
     &             'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec' /
!
      call GetLocalTime (ia)
!
      year  = mod (ia(1),100)
      month = ia(2)
!
      WRITE (CBUF,1009) ia(4), LABEL(MONTH), YEAR, ia(5), ia(6), ia(7),ia(8)
 1009 FORMAT (I2,'-',A3,'-',I2.2, i3,':',i2.2,':',i2.2,'.',i3.3)
!
      WRITE (LUNIT,2001) CBUF
 2001 FORMAT (/' ** ( THIS RUN DONE AT ',A,') **')
!
      RETURN
!
      END