forums.silverfrost.com

PaulLaidler · Posted: Thu Jul 13, 2023 5:55 pm Post subject:

John

Thank you for the information. I hope to get back to this issue soon.

DanRRight · Posted: Thu Jul 13, 2023 10:22 pm Post subject:

The approaches have to be compatible with Intel, gFortran all others, don't you agree, John? This is what for STREAM is good for, otherwise anything proprietary will marginalize this compiler further.

/* Decently i still do not get what you are trying to accomplish and why anyone needs that. Including yourself, as you do now your parallel numbercrunching in gFortran? I have not noticed also that you are a user of Clearwin or SDBG or OpenGL or Plato. Oh, you have ones tried Plato, i am sorry Smile

Do linking of libraries become a problem when the library file is larger than 2GB? No. You can not link Gfortran or Intel library to FTN95? You can. The FTN95 can not read gFortran or Intel stream binary files? It reads them fine, including files made with any other existing software. So what's this confusing fuss for? Or you have found some good solution for something useful here? Intel and gFortran and others read/write large files ok and do that fast. While this thread is already a year old discussing your proprietary headers nobody knows what for.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

DanRRight · Posted: Sat Jul 15, 2023 11:54 pm Post subject:

John,
Al that has zero relationship to the original post.

"FTN95 Stream now supports arrays > 4GB"
It does? Does my test above work?

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

Dan,

"Al that has zero relationship to the original post." I do not agree !

"Does my test above work?" It does?

A modified test program that uses elapsed time and reports the compiler info is generated in Plato and produces :

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

The enhanced code, using FTN95 Ver 8.92 is

DanRRight · Posted: Sun Jul 16, 2023 7:24 pm Post subject:

John,
That was resolved a year ago. And broke again. Read the whole thread with latest posts. Looks like they still in transit to Australia Smile

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

Dan,

Trying to unpack your comments !!

I worked with Paul late last year to identify problems with stream I/O.
Paul provided me with FTN95 updates to test for errors I had found.
These addressed most problems of Stream I/O, but not reading large unformatted sequential file type records.

The latest version of FTN95, which I am using, I have labeled as FTN95_8.92.37b. FTN95.exe is 15-Nov-22, while all .dll and .lib are 24-Dec-22

If the (8.95?) version you are using is later than this 8.92 version, then Paul has not yet incorporated these test changes into the main release.

I had interpreted your description of "Al that has zero relationship to the original post" as referring to the test Posted: Mon Sep 26, 2022 11:13 am
Excuse me if I don't interpret your comments as you intended ! (I am struggling with this point)

I have now modified your latest test Posted: Wed Jul 12, 2023 11:44 am,
which I understand is to also test stream reads of large records.

This does not work with earlier versions of FTN95 Ver 8.92 I have on other pc, but does work with my 8.92_37b version.

I have also written an inefficient file comparison using stream I/O to compare the 2 different files created in this latest test.

There are many aspects to efficient read/write of binary files, to achieve high GByte per second performance you claim and lots of traps if you are not careful.

Trap 1 : don't run this test program on a notebook with only 8 GBytes of memory !
Trap 2 : don't expect all M.2 NVMe SSDs are fast, as it also depends on their internal buffer capacity. Which PCIe version is also a hardware factor.
Trap 3 : then with testing, test performance can be misleading, as OS memory buffering of 64 GBytes of installed memory can hide/obscure a lot of disk performance.

The main outcome of the new tests is FTN95 Ver 8.92_38b passes the write and read tests you were wanting to do in your "original post" or "test above"
I shall now try to post this latest test program and the test logs I got for running on a C: SSD 970 EVO Plus 500GB and an E: 2 TByte WD drive of unknown details!

PaulLaidler · Posted: Mon Jul 17, 2023 7:32 am Post subject:

John

I think that the current release at http://forums.silverfrost.com/viewtopic.php?t=4245 includes the changes that I made for you. I should be grateful if you would check this.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

The below links are to the latest test program and also to the log file that has been generated/appended.

https://www.dropbox.com/s/u16l11lcpaa6v88/read_write.f90?dl=0
https://www.dropbox.com/s/exqqpwzsb0efiwg/stream_tests.log?dl=0

The test program does not report the details of the disk being used, which can be provided at the start of the test, using the details prompt.

Performance is similar between FTN95 and Gfortran.

The test results on my 500GB SSD 970 EVO Plus on my Ryzen 5900x were fairly good and the 2 Tbyte HDD was good (due to 64 GBytes of memory buffers) These are included in the .log file.
However the tests on my I7-6700HQ notebook with a Samsung MZNLF128H (a 128 GB SSD) were much slower.

Basically the FTN95 compiler version I am using supports large record sizes, but the performance difference probably does not justify the large record choice.
Certainly 44 byte records are too small, but you don't need to go to 8 GByte records. If you can get say 64 KByte records that may be a good comprimise, which is what I have been using in my IO library for many years.

Please don't claim this latest post "zero relationship to the original post"

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney

Paul,

I downloaded the maintenance version 8.95
Then installed the http://forums.silverfrost.com/viewtopic.php?t=4245 (Latest executables and DLLs)

This resulted in Ver 8.97.2

I compiled and ran my read_write.f90 I postecd today and it ran to completion.

The results appear to show the program runs without error for stream I/O for small then large records, then confirms both files are the same.

Note : The above report of "Speed of write Method 2= 3.112 1.172 GB/sec" shows 2 different transfer rate estimates. The first is based on CPU time, while the second is based on elapsed time (system_clock). Elapsed time is a better estimate of performance, as CPU time ignores any disk update delays.

Comparing the results between 960 EVO and 970 EVO Plus on the two different PC's I am using I get:

DanRRight · Posted: Tue Jul 18, 2023 5:48 am Post subject:

With my latest demo test ( on page 4, the one which writes large file and then reads it), after successful allocation of 8 GB immediately goes "Access violation". Gfortran works ok

PaulLaidler · Posted: Tue Jul 18, 2023 7:04 am Post subject:

Dan

I have tested the program that you posted on 12 July using the latest downloads. It runs to the end for me giving the output...

Trying to allocate GB of RAM : 8.80000000000
Allocation success
Trying to save the data Method 1
Write OK. Speed of write Method 1 = 1.36699
=====================
================ N O W R E A D ====================
READ OK. Speed of read Method 1 = 3.04432
=============================
Trying to save the data Method 2
Write OK. Speed of write Method 2= 6.54884
=====================
================ N O W R E A D ==================
READ OK. Speed of read Method 2 = 1.23239
======================
**** PAUSE: File LargeFile.dat created OK
Press ENTER to continue:

JohnCampbell · Joined: 16 Feb 2006 Posts: 2580 Location: Sydney