Again: 64 bit Compiler

If I could comment, based on my limited experience of 64-bit programming.

1. Eddie is right, in that you typically use only 1 or 2 large arrays.
2. These arrays only work well if they are smaller than the physical memory installed, as exceeding the physical memory implies virtual memory paging, which can be done much more efficiently in the program algorithm than the O/S dumb paging approach. (If you invoke paging then the 32-bit 'out-of-core' algorithm is probably more efficient. )
3. My 64-bit algorithm has seen a significant restructure of my 'FTN77' memory management approach to a more extensive use of ALLOCATE. The 64-bit code assumes the problem can be solved in memory, with no out-of-core capability. If in memory, it is much more efficient, but can only efficiently solve problems smaller than the physical memory available.
4. Removing all the overheads of providing an out-of-core approach has allowed more flexibility in manageing the solution and allowed easier and quicker development of alternative solutions.
5. The 64-bit limitation of requiring ALLOCATE for addresses > 2gb has had some benefits in solution definition, as large fixed size static arrays are no longer useable.
6. The introduction of SSD for I/O has changed the umbalance. With (very) significant increases in disk transfer rates, 32-bit solutions are now less disadvantaged.

The problem with operating in a competitive environment is that other suppliers have the flexibility of 64-bit solutions and we must meet their capability or become an even smaller niche operator.

John

Eddie, with respect to your comment - are you saying that the only thing which needs to be changed in FTN95 to be 64-bit like some other existing compilers are is just making 64bit ALLOCATE/DEALLOCATE which in turn means simply using for them integer8 variables for the array indexes (because the maximum for integer4 numbers is 4GB) plus under-the-hood of course some low level trickery only C/C++/ assembler programmers know?

Isn't this actually how 64bit Intel IVF was made - no changes in the 32bit portion and the only what actually is 64bit is ALLOCATE ?

Well, after playing with 4GB trick in FTN95 i will say that if this is the case, and the only what will be changed to 64bits is ALLOCATE it will fit me for a while even though i have not 1-2 but tons of arrays waiting to be >2GB. I will unload all of them into 64bit ALLOCATABLE space and leave the code part in 32bit one. The trick will work because most probably not me not anyone else will succeed to break 2GB 32bit limit for the code itself any time soon which will probably need 100M lines of Fortran source code. LOL

I'm always pleased if anyone agrees with me, or even if not, then if anyone is provoked to discuss!

PC RAM is cached into a small quantity of faster RAM, now on the cpu itself. Disks are cached into RAM. The mechanism I propose, caching a HUGE array defined as 64bit addressable RAM into a smaller area of RAM that is 32bit addressable has to be a lot faster than even SSDs - but I do accept that virtualising RAM onto a hard drive is going to be slow. I agree, John, that one can sometimes program this faster yourself for particular problems than using a general purpose code.

I think that the hardware to enable the use of lots of really big arrays is some way off, and we'll need faster processing to deal with the volumes of data.

Eddie

Eddie, You said:

I think that the hardware to enable the use of lots of really big arrays is some way off, and we'll need faster processing to deal with the volumes of data.

That time is already here. Others where I work are using a workstation with 96gb of memory for large 3d geotechnical modelling with Abaqus to speed up their non-linear solutions. There are many other commercial analysis packages that can use lots of memory. My packages are nowhere near that size, and the challenge is setting up those large models. Not enough time to develop the range of techniques demanded by these size problems.

John

Just to add my two penneth.

I'm currently trying to integrate DLLs with another package (that comes in 32-bit and 64-bit versions). The other package 'prefers' to be a 64-bit app and would like us to supply 64-bit DLLs to link against (the actual call into our DLL is via LoadLibrary and GetProcAddress calls before calling the process itself).

Is there any way to do this or must they continue to tell their customers to use the 32-bit version if they want to use our 'add in'?

K

Kenny's post made me look at the preceding posts - and John's 96Gb machines got me thinking. I have tried to cost the parts to build one myself (or to think about it) and I can't find a mainboard that supports more than 64 Gb of memory in 8 slots (but I can find the matching quad channel memory).

My statement was about lots of large arrays, and even in 64 Gb one might fit sixteen 4Gb arrays, eight eights, or multiple combinations of various sizes, say a 32 and four eights. If you wanted a 32 and eight fours, you might as well settle for a 32 addressed in 64 bit mode and the eight fours addressed in 32 bit mode.

It does seem to me that the way Windows 64 bit and the Intel compiler work is quite sensible, i.e. most things are really 32 bit, all except for a handful of arrays, and there is a resonable upper bound on the number of those arrays - you are going to get less than 16 arrays needing 64-bit addressing into a 64 Gb memory (or 12 in 96 Gb). I don't consider 12 to be 'lots'. Checking with a number of FE codes written by me or others, it looked to me that they would use the space if it was given in only 1 array (or 2 if the code was re-written a bit).

It seems to me that assuming that one could have (say) 64 arrays addressable in 64-bit mode ought to satisfy the needs of anyone at the present time, and even future-proof things up to a point.

Assuming that Kenny's add-in never wants more than 4 Gb for itself, isn't there a way of 'lying' to the customer's application? After all, if that parent application claims to be 64-bit it has already used a bit of deceit (Windows 64 being what it is) and won't it already have hogged all the 64-bit addressable memory for itself?

Eddie

Eddie,

I checked on the 'pc' with 96gb of memory. It is a blade server, with a Xeon E5630 processor runing Server 2008 OS. It does exist and can run for a week. I just don't understand how many calculations it itterates through. My desktop has only 12gb memory and most of my problems now run in 20 minutes. From my experience of large memory solutions, I have only 1 large array, while the second largest is less than 100mb. You could write your own virtual paging algorithm to use 4gb pages, but those were solution techniques we developed when I started to use computers. There are too many other things to do and most of my paying work can run in 4gb. Given all the work I've done in the past on out-of-core solvers, I still have a keen interest in understanding the benefits and limits of large memory solutions. I find that my pc with 12 gb provides only a small window for solving larger problems, before they too go to out of core. Doubling the mesh refinement in a 3d model is an 8x memory increase.

John

John,

You can't build blade servers yourself (at least, I can't), so not in my league (I never doubted that someone had one!). 64 Gb might be within my capabilities (and only GBP800 for top quality RAM).

I've solved very important problems (my definition) in the past with 60 isoparametric elements running in 96k of memory. I'm using CHM files that are 10x bigger than that now!

Anyway, the important thing as far as I am concerned is that you can fake 64-bittedness by allowing the compiler to allow INTEGER*8 addressing for a very small number of arrays. I did think 8 originally, then thought 64, but probably 4 is enough - or if your experience and mine is a guide, maybe only 1. Absolutely everything else continues as is.

I also looked in an old code at how many lines contained references to the big array, and it was just a handful. One could easily accommodate special syntax if required. (Easily in this case means 'without getting into such a muddle you give up').

Eddie

Searching for advice, how to integrate FTN95 DLLs into 64 bit applications, I became aware of the forum's discussion about the interst in a 64 bit version of FTN95.

I can agree with many of the other members: Yes, there is a demand for a 64 bit version.

My background/characteristics:

• Using Salford/Silverfrost products for a very long time (FTN77, DBOS).
• Large amount of existing Fortran source code.
• Extensive use of ClearWin.
• Integration of DLLs, developed with FTN95, into other applications like MS Office, Delphi, VB.

Recently I had appreciated the development of DBOS to overcome the limitations of a 16 bit operation system and to make more memory available for development and application. I saw - and I am still seeing - Salford/Silverfrost as a front runner to support fortran programmers with application tools which are not directly related to a specific environment reagarding hardware and software and which are designed to fulfill the programmer's needs.

Today we are faced to the problem that the 64 bit development is progressing. More and more of our customers may use 64 bit versions of software products. Although 32 bit seems to be the default for e.g. MS office, 64 bit versions are available and some companies may decide to make use of them.

Therefore a request for 64 bit versions of DLLs will arise. I would prefer to have a solution based on Silverfrost FTN95 or FTN20xx. If there is a prospect of availability within reasonable time, I would appreciate. Otherwise I have to look for alternatives.

I am currently working on this and hope to have something ready for the next release which should come out before the end of 2012.

Quoted from PaulLaidler I am currently working on this and hope to have something ready for the next release which should come out before the end of 2012.

[size=18:8020dbeed0] Is there any news on this yet please?[/size:8020dbeed0]

Yes I have completed my development ready for the next release and will provide details shortly.

Quoted from PaulLaidler Yes I have completed my development ready for the next release and will provide details shortly.

Okay, good news. Do we have a timeframe?

We are aiming for a full release in the near future but if this is not possible before the end of January I will see if I can provide a beta release for '64 bit ClearWin+'.

The basic concept is to provide a 64 bit DLL for ClearWin+ that will provide access to ClearWin+ from third party compilers such as gFortran. Fortran code that is Fortran 95 Standard conforming and calls on the ClearWin+ library, can then be developed using FTN95 and released as 64 bit executable and DLLs.

A Beta release would be appreciated.

Quoted from PaulLaidler We are aiming for a full release in the near future but if this is not possible before the end of January I will see if I can provide a beta release for '64 bit ClearWin+'.

The basic concept is to provide a 64 bit DLL for ClearWin+ that will provide access to ClearWin+ from third party compilers such as gFortran. Fortran code that is Fortran 95 Standard conforming and calls on the ClearWin+ library, can then be developed using FTN95 and released as 64 bit executable and DLLs.

[size=18:3421fac15f] Any news 'he said, expecting the answer No' (for all those Monty Python Cheese Sketch fans out there).

You've got to laugh haven't you? It's not as if we've built our entire business around Clearwin is it? Oh, hang on a minute. Yes we have. Oops.[/size:3421fac15f]

Don't worry. I hope to have something for you very soon.

A beta version of 64 bit ClearWin+ is now available to users who have a Silverfrost support contract.

I will send you details upon receiving a message in my Forum message box.