forums.silverfrost.com

[JohnCampbell]

Dan,

"declare arrays the size you want till these gazillions of heaxabytes"

What you are describing with a very large array which would take too long to analyse.
I have recently solved a surveying problem for a 75km long x 500 metre wide navigation channel which covers an area of 60 km x 50km, with results reduced to 2 metre centres. There are about 60 bytes of information for each point; 750 million virtual points, but only 9 million active points.
By mapping the active points to a virtual 2D page system, I reduce the storage to about 1% and importantly the calculation time by a similar factor.
Sparse matrix approaches still have an important role in large problems, where often the performance savings become even greater.
The luxury you are describing is not what you need !!

John

[DanRRight]

What is too long to analyse? It is computer which analyses Matrices are almost empty by the way. Parallelization exists too. At the end computer have to tell us just one single number or the answer Yes or No

I have to use sparsity or do not even try to solve these tasks i run. The /VC compiler switch of FTN77 allowed me in 1990 run 2GB problem suitable only for supercomputers on Intel 386 PC with 1-4 MB or RAM which was a lot at that time. Having 1000x more RAM on PC was unimaginably at that time. Having billion times more memory is unimaginable right now too. But with the 64bit compiler with virtual common we can repeat similar jump into very distant future TODAY.

So using sparsity is important. My life would be easier if Fortran allowed options to declare the arrays Arr(i,j,k) say like these Arr(10000,10000,100) which contain large 10B number of elements to be of variable bandwidth with all the savings associated with this. That is to accomodate actual number of elements of real task.

For example Array with k=1 is Arr Arr(10000,10000,1)
Array k=2 is (10000,10000,2)
Array k=3 is (10000,10000,3)
Array k=4 is only (10,10,4)
Array k=5 is only (1,1,5)
Array k=6 is larger (1000,1000,6)
Array k=7 is also (1000,1000,7)
and so on.
Actual arrays i use have 4 and ideally for convenience would have 5 dimensions if 32bit allowed me such luxury. The 32 bit limit does not allow me to run comfortably literally any task i tried last decade or two.

Instead i have to declare array of full size Arr(10000,10000,1:3) and declare another Arr2(1000,1000,4:100) to fit all numbers, break the homogeneity of the code, making it way more complex and have tons of error due to that. All these arrays are still probably 99% empty.

[JohnCampbell]

Dan,

You could try something like the following, although I got a stack overflow with copies of test(k)%arr being placed on the stack with calls to report_size. This might be a problem for more general use of these arrays.

[DanRRight]

John, Thanks for the demo. Before, like 10 years back, such Fortran-90 tricks did not work well, so i am very careful to move with them. What do you mean placed in the stack causing overflow? The same code but without allocate? Can you please also check if it overflows on Ifort/gFortran since you have tried many different Fortran compilers recently? I may use this code with the future FTNpro, it may save me on RAM

[JohnCampbell]

The following change reports the memory address of test(k)%arr, indicating that a duplicate is being provided. gFortran reports the same address, indicating that this stack copy is not required.

[DanRRight]

Your intuition was right. That was easy, just remove the workaround condition for k=6 and link (minimum is stack:240000000 which is exactly the size of the largest k=6 array is enough with /nocheck option. With /debug /undef it has to be larger)

ftn95 arr.f95
slink arr.obj /stack:1000000000 /3gb

Why this large array goes to stack with /3gb by the way? Hope in the future 64bit compiler there will be no damn stack at all

[JohnCampbell]

Dan,

I presume the stack value you gave is decimal. I could not find documentation that confirms this, as the example I found in
Win32 platform > Using the linker > Reference > Interactive mode only
gives the examples in hex.
Do you know where hex octal and decimal syntax is described.
I think I once suggested that 240m or 240000k should be supported in stead of 240000000, as I get lost in all the zeros.

Anyway, this does give an example of how to manage variable sizes and moving to 64-bit will not mean that we can ignore the memory usage.
As I have described previously, when you run out of physical memory, everything appears to stop. When it first happens, you think it is like the blue screen system crash. I shut down the PC and then had to check all the disks!

I've found that moving from 2gb to 4gb then 8gb then 16gb does not dramatically change the types of problems you can solve, although it does make it a bit easier and a bit faster.

John

[DanRRight]

And imagine 9 more zeros with 64bit compiler? LOL Does Intel use stack by the way?

There are types of problems which look crippled in 32bit and ones which are not even possible with 32bits as they need minimum 16GB like some our 3D PIC plasma simulations. If you use less than that the numerical fluctuations are so large that they create fake physics reality.
Intel's Knight Landing processor with 72 64bit cores combined with the future Silverfrost 64bit FTNXX supporting OpenMP would be great home supercomputer i think

[John-Silver]

"... this is included in FTN95 today ..."

is this only available to paid-up users or is there a 'trial' personal version ?

[PaulLaidler]

John

I have lost the thread. What is "this"?

[LitusSaxonicum]

"This" is the 64 bit Clearwin+ for other compilers, and to John, yes, it is bundled with the PE.

Eddie