Problem with running a large program under win32/vista

Hi I am having trouble running a large FORTRAN program under VISTA that previously ran OK using XP. The program uses large arrays, but now freezes when trying to open up an open file dialog box. This problem has occurred before using windows 2000, but was not as bad. I have experimented with all possible values of stack and heap using SLINK.
Can someone help me? BTW I have the latest version of the compiler. Nick Bowring

This problem has been raised before on this forum but I cannot find the relevants threads. Nor can I remember if there was any clear cut resolution.

If the problem arises from a shortage of memory then under XP you could consider using the 3GB switch in your boot.ini but I do not know if this translates to Vista.

Paul and others,

I am interested in how the memory limits apply as my next hardware upgrade may go beyond 2gb of memory. My present experience is with 1gb, so these larger numbers are quite a change. It wasn't long ago that I first used files more than 2gb for the out of memory solutions !

Could you provide a reference to extending beyond 2gb, especially for a clearwin+ program. Presumably we can't have single arrays larger than 2gb ?

As an alternative, does anyone know of how to create a virtual disk on a 2gb plus machine to get fast I/O that simply uses the fortran read/write routines to shift information in physical memory, at memory transfer rates ?

Regards John

John

You will find some information in ftn95.chm if you search for 3GB.

Paul,

Thanks for the advice. I read the help and have a couple of questions.

Is the addressible length of a single array 1gb, or can this be multiplied by the byte size, ie what is the maximum size of a 1-d real8 array? Can a 2 dimensional array go beyond the 1gb limit ? For example, what is the addressible memory size limit of the following: Integer1 i1array() Integer4 i4array() Integer4 i4array(4,) real8 r8array(*)

Does the reference to 'none of the debugging options' include the /debug option, which mainly provides improved addressing of the source code line location ? I could more understand the resctictions with /check etc.

Any problems with /opt ?

Recent improvement to processor speeds has resulted in an increased relative performance penalty for writing data to disk. To be able to store more information in memory can provide significant improvements. It can take a while to transfer 3gb of info to disk and back again !

I would appreciate your comments,

John

John

The relevant size (of an array in this context) is the byte size, the number of dimensions is not relevant. You need 4 bytes for each INTEGER*4 element etc. As far as FTN95 is concerned the byte size of an array could be up to 4GB (if you had nothing else but naturally you need space for the program and other data, also the operating system addresses the same 4GB!).

The problem is that Windows normally reserves 2GB for the operating system but with XP (and some other operating systems) this can be decreased to 1GB so that you have a virtual address space of 3GB for applications.

For simplicity the phrase 'none of the debugging options' does include /debug because of the way information is passed from the compiler to the linker but you can use 3GB on the SLINK command line to override this. There is no problem with /opt - you automatically get 3GB in this case (provided you have configured your boot.ini for 3GB).

Paul,

In other words, If I go and buy a new PC with 4gb memory, use the /3gb switch, I can write a program with either

real8 r8_array(400000000) or integer4 i4_array(800000000)

and there will be no problem with the addressing. Both arrays are about 2^31.6 bytes, which is less than 3gb My DO loop will use an I*4 variable, which should not overflow. I assume the array should be in COMMON or a MODULE

Any problems with these assumptions ?

I'm wanting to try and reduce the disk I/O and improve performance !

Will this program be paged off to disk at any stage, such as on startup or when the array is first initialised ?

Thanks John

You do not need to buy a new machine to test this out. If you are using XP then set the /3GB switch in your boot.ini and use your existing RAM. Having more RAM will speed things up. Having less RAM will not stop you from increasing the available memory.

Try a simple test program. In my experience you will be lucky to get more than about 1.5GB in a single array. This is because of all of the other demands on memory usage and limitations (by the operating system) on how chunks of memory are made available to applications.

Within FTN95 default integers are signed 32 bit values so the largest array index for a 1-dimensional array might well be 2**32 - 1. I don't think that FTN95 makes use of INTEGER*8 values in this context (at least not at the moment in the sense of extending the index range). I apologise if this contradicts something I wrote earlier.

Paul,

I had a go with the following program. I compiled it on my computer (which has ftn95 installed) then copied the .exe's and .dll files to a machine with 4mb memory and the /3gb switch. It ran for 0.8gb and 1.6gb but crashed for 2.4gb. It actually stoped with no message. I then ran the program on my computer ( with 1gb memory but without the /3gb switch). It ran (slowly !!) for 0.8gb and 1.6gb, but crashed with an access violation for 2.4gb. ( no /3gb switch ?) I was assuming the ftn95 /link option would enable the /3gb link option ?

I would like to know what is stopping the two machines from running. Any ideas on what is wrong with the two configurations? The 4gb machine should have run the program ? Can I use the /3gb switch on the 1gb memory machine to enable more than 1.6gb virtual memory?

I don't understand why you stated 'In my experience you will be lucky to get more than about 1.5GB in a single array'. If 3gb is enabled why can't we approach 3gb? Even at 2gb allowable, loosing 0.5gb is a lot of overhead. Or am I missing some other issue ? I thought we discussed the issue of addressing memory and concluded that this limit is 4gb.

If I can solve this, then when I increase the physical memory, that should improve the run time.

I've never been a fan of microsoft's virtual memory management. These tests may give me a better experience of how well it works. Does anyone regularly use virtual memory and achieve resonable run time performance ?

The test program is :

! Last change: JDC 2 Oct 2007 2:03 pm ! Program to test memory sizes ! module large_array ! real10, parameter :: one = 1 real10, parameter :: two = 2 integer4, parameter :: thousand = 1000 integer4, parameter :: million = thousandthousand integer4, parameter :: billion = millionthousand ! integer4, parameter :: array_size = 200million ! real8 r8_array(array_size) end module large_array

  use large_array
  integer*4 i
  real*10   s1, s2, s3
  character description*80

! 1001 format (a,b'zzz,zzz,zz#',a) 1002 format (a,es22.15,' (error=',es10.3,')') 1003 format (a,es22.15) ! i = ((array_size/1024)8)/1024 write (description,1001) 'initialising array of ', array_size,' real8 variables' call report (description) write (description,1001) 'Array size = ',i,' mb' call report (description) ! do i = 1,array_size r8_array(i) = i end do call report ('array initialised') ! s3 = array_size s3 = s3*(s3+one)/two ! s1 = 0 do i = 1,array_size s1 = s1 + r8_array(i) end do write (description,1002) 'do Sum of array = ',s1, s1-s3 call report (description) ! s2 = sum (r8_array) write (description,1002) 'Sum of array = ',s2, s2-s3 call report (description) ! write (description,1003) 'Answer should be= ',s3 end

  subroutine report (description)

! character description*() real8 seconds ! call get_run_time (seconds) WRITE (*,1011) seconds, trim (description) ! RETURN 1011 FORMAT (f10.3,2x,a) END

  subroutine get_run_time (seconds)

! ! returns the elapsed time, accurate to .001 seconds (?) ! real8 seconds ! CHARACTER date8, time10, zone5 INTEGER values(8) real8 start_time, x data start_time / -1 / ! call DATE_AND_TIME (date, time, zone, values) ! x = 60.060.0values(5) & + 60.0values(6) & + values(7) & + 0.001*values(8) !
if (start_time < 0) start_time = x seconds = x - start_time RETURN END

John

I don't think that I can usefully add anything to what I have already written. Here is a summary.

1. You must use 3GB in your boot.ini. The Microsoft documentation will tell you how to do this.

2. You must use a recent version of SLINK. This has options /3GB and /2GB. 3GB is the default for release mode. 2GB is the default of all debugging modes.

3. The amount of RAM that you are using is not relevant except that it will affect the speed of execution. If there is not enough RAM then memory will be swapped in and out from your hard drive.

4. The 3GB of address space that is released by the operating system is for all of the applications that are running on your machine and probably will not be available in one big chunk. Using the 3GB switch in boot.ini will probably increase the amount of memory available to your program but there are other factors (mentioned above) that can affect the outcome.

Paul,

There are a few points you may be able to clarify.

1. I am generating the test programs on my pc and then transferring them to another pc with 3GB in boot.ini. It uses this for other applications. Do I need 3GB on my machine to generate the test programs ? I only have the compiler installed on my pc. Maybe I should put 3GB into my pc, as I have not heard any adverse comments about performance, due to this change ? I was puzzled by the 2.4gb program crashing on the /3gb pc, with no error message. It gave the expected access violation window when run on my pc. Do you need to reconfigure pagefile.sys when using 3gb ?

2. I am using [SLINK/Win32 v1.42 Copyright (c) Silverfrost Ltd. 1995-2005]. It has no reference to /3GB or /2GB in the /? help but Ver 1.32 is referred to in the associated ftn95 help documentation.

3. I do not understand your point 4. Why do you say 'The 3GB of address space that is released by the operating system is for all of the applications that are running on your machine' ? My understanding is the 3GB allows for each application to have 3gb of virtual address space, and the O/S limited to 1gb of virtual address, common for all applications. That it 'will not be available in one big chunk' is a matter for the O/S's virtual memory management when allocating physical memory. You also state 'Using the 3GB switch in boot.ini will probably increase the amount of memory available to your program but there are other factors (mentioned above) that can affect the outcome.' I do not understand what these factors are. I have many programming examples which don't appear to require significant amounts of my 2gb virtual memory. Why can't I get close to the 3gb limit ? Your earlier reference to 'you will be lucky to get more than about 1.5GB in a single array' implies a loss of 1.5gb for the rest of the program. This does not appear reasonable. There should be ways of improving this result.

Do any other FTN95 users have experience of using 3GB ?

Thanks again Paul, for your assistance in understanding these issues.

regards John

I have now tried to run with the 3gb switch on, with Windows XP and 1gb memory. I get a few problems !!

I have two versions of big_array.f95 a 1.5 gb version with 'array_size = 200million' and a supposed 2.3gb version with 'array_size = 300million'

To generate the program I use the commands ftn95 big_array slink big_array.obj /3gb big_array

( My version of slink is 1.42 and ftn95 is version 4.9.1. I have the ver 5.01 cd but have not yet loaded it up. I'll do that next! )

If I run the 1.5gb version windows task manager shows the virtual memory size as 1,569,039 K but if I run the 2.3gb version, task manager only shows the program as 1,177,048 K and then crashes when the program reaches initiating 1.1 gb

The response is similar if I select the non /3gb statup

Any ideas ?

The latest version of big_array.f95 is:

! Last change: JDC 2 Oct 2007 2:03 pm ! Program to test memory sizes ! module large_array ! real10, parameter :: one = 1 real10, parameter :: two = 2 real10, parameter :: eight = 8 integer4, parameter :: thousand = 1000 integer4, parameter :: million = thousandthousand integer4, parameter :: billion = millionthousand ! ! integer4, parameter :: array_size = 200million ! 1.5 gb works integer4, parameter :: array_size = 300million ! 2.3 gb does not work ! real*8 r8_array(array_size) end module large_array

  use large_array
  integer*4 i, m_100, m
  real*10   s1, s2, s3
  character description*80

! 1001 format (a,b'zzz,zzz,zz#',a) 1002 format (a,es22.15,' (error=',es10.3,')') 1003 format (a,es22.15) 1004 format ( f5.1,' gb initialised') ! i = ((array_size/1024)8)/1024 write (description,1001) 'initialising array of ', array_size,' real8 variables' call report (description) write (description,1001) 'Array size = ',i,' mb' call report (description) ! m_100 = (1024*1024/8)*100 m = m_100 do i = 1,array_size m = m-1 if (m ⇐ 0) then s1 = dble(i) * eight / dble(1024)*3 write (description, 1004) s1 call report (description) m = m_100 end if r8_array(i) = i end do call report ('array initialised') ! s3 = array_size s3 = s3(s3+one)/two ! s1 = 0 do i = 1,array_size s1 = s1 + r8_array(i) end do write (description,1002) 'do Sum of array = ',s1, s1-s3 call report (description) ! s2 = sum (r8_array) write (description,1002) 'Sum of array = ',s2, s2-s3 call report (description) ! write (description,1003) 'Answer should be= ',s3 end

  subroutine report (description)

! ! returns the time in the form hh:mm pm ! character description*() real8 seconds ! call get_run_time (seconds) WRITE (*,1011) seconds, trim (description) ! RETURN 1011 FORMAT (f10.3,2x,a) END

  subroutine get_run_time (seconds)

! ! returns the elapsed time, accurate to .001 seconds (?) ! real8 seconds ! CHARACTER date8, time10, zone5 INTEGER values(8) real8 start_time, x data start_time / -1 / ! call DATE_AND_TIME (date, time, zone, values) ! x = 60.060.0values(5) & + 60.0values(6) & + values(7) & + 0.001*values(8) !
if (start_time < 0) start_time = x seconds = x - start_time RETURN END

Paul,

I have further investigated the problem. I am now using ftn95 ver 5.10, but that hasn't changed the results.

I have taken John Horspool's past advice and transferred the array into unlabeled common and used virtual common in slink '/vc' This has not changed the result, but better reports the memory size.

The changed portion of code looks like: ! real*8 r8_array(array_size) end module large_array

  use large_array

! real*8 r8_array(array_size) common r8_array

the batch file is : ftn95 big_array /lis /xref slink big_array.obj /3gb /map /vc notepad big_array.map

For different array sizes, the .map reports the following sizes: Virtual Common information:

Base Length Commit

array_size = 220*million 20000000 68e7784c (run time library decides) (1678.5Mb)

array_size = 260*million 20000000 7bfa484c (run time library decides) (1983.6Mb)

array_size = 270*million 20000000 405f7e4c (run time library decides) (1030.0Mb)

This appears to place a limit of 2gb on the single array. I don't know if this is also a limit on common, or all common, but this is not achieving more than 2gb, unlike an earlier implied 4gb limit. .. 'As far as FTN95 is concerned the byte size of an array could be up to 4GB'

Where is this limit applied and how can it be overcome ?

Regards John

John

'As far as FTN95 is concerned the byte size of an array could be up to 4GB'

I have corrected this statement. In some, if not all situations, the limit could be 2GB (the upper bound for signed INTEGER*4 values).

The point is that the operating system is usually more restrictive than FTN95 in this context.

The operating system allocates memory (up to 2 or 3GB) for all applications that are currently running on your machine. I cannot think of another way to say this. You can use the Task Manager to see what is currently running.

As far as I know SLINK does not need to have 3GB in the boot.ini when linking.

If SLINK does not complain about the use of /2GB or /3GB then the version number of SLINK will be OK.

When your FTN95 application runs, it asks the operating system for chunks of memory. There will be limitations within FTN95 on the size that can be requested and limitations provided by the operating system on what requests can be granted.

If you use 3GB in your boot.ini then you should see an improvement. I cannot say more than this.

Paul,

I have proceeded to try and get a program to work. I am hitting a limit of 2gb for all variables, either as common, virtual common or a mixture of big arrays in a module and in common. The program will not load, giving either a negative common size or not being a valid win32 program. I tried to split the arrays into two, but was limited in how I could manage them in slink. I will continue to experiment and try to get it working. It may take a while !

Has anyone out there successfully used /3gb and created a program that uses more than 2gb of data ?

Regards John

we have created one on an XP 64 bit operating system

not vista though.

though we had lots (100+)arrays of the order 10MB+ not one single huge array

Carl

Carl,

Thanks for the response. Did you use ftn95 and slink to produce the executable ? What was the 64bit O/S ? I'm using two arrays of 1.3gb. When I use SLINK, it groups all arrays in the same section, either in virtual common or in .bss. I wasn't aware that you could use a win/32 compiler and linker to use the larger memory available on a 64bit O/S

John

Hi All,

I'm disappointed that no one has responded with any experience of the /3gb switch.

Although I am yet to understand how to increase my data area above 2gb, using slink, I have ordered a new pc with 4gb of memory and when it comes will try again, this time without the need to use virtual memory disk transfers as often. (Using the /3gb switch on a pc with 1gb of physical memory did not work well !)

If anyone has tried this before, please let me know

regards

John Campbell

Sorry John I missed you second question

We used

Windows XP Professional x64 Edition Version 2003 service pack 2

there is a file boot.ini that has to have the /3GB line added to it

we were using an old FTN95 code which used to be a FTN77 code which was written to the fortran 66 standard

it was buildt using a set of batch file calling FTN95 and then SLINK also using /3GB so we could debug when required

to create the executable. It has all worked as one would expect though some of the debug information is ocassionally a bit erratic and we have to spit numbers to debug files to ensure the debugger is correct/wrong.

having said that it is old (tested) code that takes 18 hours to run so we don't debug very often.

in terms of the code probably no individual array are likely to be as big as say a 1GB but there are lots of arrays.

All arrays are old style static common blocks the stack size is about 10MB I think.

with regard to win32 on 64 bit ops. there are some hoops (normally to do with drivers for hardware) but 32bit apps work simply because there are so many of them that the vendors have had to come up with a technology to handle them. The visible sign of this is that on XP x64 is that the are two program files folders (a bit 32 one and a 64 bit one)

I don't know the details but I believe there is a conversion process that happens at the operating systemn level to convert from 32 bit to 64 bit

technology is called windows on windows WOW or something like on intel chips.

we were very lucky actually we just tried the x64 thing to see if it would help and then saw some threads here talking about 3GB and tried it out, and it really saved our bacon, since the code needed to double in size unexpectedly and we didn't really have a plan B!

Carl

Carl,

Thanks for the response. Paul, in a previous post said that the memory addressing range is 4gb, using the full unsigned 32bit integer. The /3gb gives the user process 3gb and the system 1gb, rather than the default of 2gb each.

I would be very interested to see the .map for the program you have got to work. The problem I appear to be having is that if the '.bss' data area exceeds 2gb then windows XP does not recognise the .exe as a valid executable. The following is taken from the map summary for my test program.

Executable section map:

Section Base Length Flags

.text 00401000 000013f6 60000020 .bss 00403000 9d5b344c c0000080 .comment 9d9b7000 000000ba 00000a00 .data 9d9b8000 00000450 c0000040 .idata 9d9b9000 000005a1 c0000040 .CRT 9d9ba000 0000000c c0000040 .salfmap 9d9bb000 0000059b 40000040 .salfsys 9d9bc000 00000018 c0000040 .salfvc 9d9bd000 0000002c c0000040 .reloc 9d9be000 00000000 42000040

I can't get slink to produce a valid .exe file which has more than 2gb in the large arrays. With the mixture of large arrays I am using, all are going into the same memory 'section'. Unfortunately my test program is compact, so I can't test if the combination of all sections can exceed 2gb. I have looked at some other programs, where arrays in common are located in the '.bss' and '.data' sections, but I'm yet to understand why or if this could solve my problem.

Again, thanks for the response.

John