forums.silverfrost.com

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

There appears to be a performance issue associated with UNFORMATTED files, either DIRECT or SEQUENTIAL access.

I had noticed that the work files created as DIRECT, UNFORMATTED seemed very slow to create, and slow to access. So, I ran a set of comparisons.

I wrote (and read) a file of 2000 records, of length=194 and timed the loop. The results are as follows:

For Direct Access (using REC= in the write or read)

mecej4 · Joined: 31 Oct 2006 Posts: 1886

With the files on a ramdisk, on a Windows 8.1 laptop, writing and reading the data file with your test program took less than 0.02 seconds each. Are your files located in a remote file system, perhaps? What is the nature of your hardware, and which version of the compiler did you use?

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

V7.10 system (latest release). HD is 7200 RPM 2 TB, SATA. Not a networked file.

I would expect the RAMDISK to fly on this. As I mentioned in the post, there is a LOT of disk activity while the file is being created DIRECT. It's so slow that I can see it incrementing in 5K chunks every second. Not so for the sequential file. It's running 100K+ per second.

It's the creation time that makes no sense, especially given the access time for the READ operations once it is created is quite snappy.

On my ramdisk, 2000 records took 2 seconds, or .001 seconds/record.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Check your record length in the OPEN statement : RECL=, which should only be used for direct access, unformatted, fixed length records.
REC= can/should only be used for Direct access unformatted files.

I have never identified a performance issue associated with UNFORMATTED files, either DIRECT or SEQUENTIAL access when used as limited by the standard. Your poor elapsed times being reported are probably due to problems with your incompatible use of OPEN and WRITE and are not as expected.

The combination of Fortran unformatted I/O with Windows 7 or 8 gives very good performance for the testing I have done. The file buffering works very well in FTN95.

John

mecej4 · Joined: 31 Oct 2006 Posts: 1886

On an external 7200 RPM hard drive, connected using USB 3.0 to a Windows 8.1 laptop, I get the following results for a data file creation run:

Starting the test at: 20150124 074847.450
For access=direct format=unformatted
writing 2000 records took 1.86719

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

John, thanks for the reply.

Pray tell, what is wrong/incompatible with the code of the timing code that I posted? I start by creating the file using the appropriate OPEN (STATUS='REPLACE') and time this operation, then change the OPEN to reflect its now created status (STATUS='OLD'), and change the WRITE to READ. What could be simpler than this?

How about running this as a WRITE test (not on RAMDISK) and get a timing of it, post the results that you get. And include some pertinent info about your system for the comparison.

I get nearly the same results running on my 3.6 GHz Window 7 system as I do on my 1.9 GHz Windows 2000 system. HD's have the same RPM, interfaces are SATA vs. ATA. RAMDISK shows expected performance, except there is no physical disk access, so seek times are not included in the RAMDISK data (other postings from yesterday).

No, something else is going on.

mecej4 · Joined: 31 Oct 2006 Posts: 1886

Clutching at a straw: Do you have one of these newfangled back-up-to-the-cloud services running?

John's comment about the rule that opening a file for sequential format precludes using a REC= specifier in I/O statements is well founded. This issue is separate from the slowdown that you reported but I have failed to reproduce.

The Fortran 95 standard stipulates as follows:

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

If you look at the results, then you will see it performed both ways. I was surprised as well, it shouldn't have worked, but it does.

There is obviously something interfering with the write operation.

That being said, the Windows 2000 system, with absolutely no anti-virus installed, shows similar times to yours, but only if the status is OLD.

If I run the program on the Network on the primary machine, with STATUS=OLD, then I get times like you show. With STATUS="REPLACE' (creating a new file), and timing the WRITE operation (REC= for direct unformatted), I still get in the order of 42 seconds for 2000 records. And, about the same amount of time on the Windows 2000 system.

The access times for an old file are in line with expectations, but not for NEW (aka REPLACE) files.

I also have a NetBook, no antivirus and Windows XP. It's processing power and hard drive are quite slow, but I'm seeing file creation times of 165 seconds there. Also, not expected.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

I could not see the "write" you are referring to ?
The following is my adaptation of your test code. I did not get the performance times you claim.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

.ctd

IanLambley · Joined: 17 Dec 2006 Posts: 490 Location: Sunderland

Have you tried de-fragmenting the disk? If the file is spread about the disk surface, there will be a lot of time for the read/write heads to respond from a mechanical point of view.
Ian

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

Ian, the disk(s) used are regularly defragged. I've now used a total of 4 different drive/partitions on three machines (poor times), one ramdisk (extraordinary times), and two with network connection to different NAS (great times).

I posted this originally because the slow times for one kind of access on a physical drive puzzled me beyond measure. Having people post actual timing or suggest interactions that I have not considered helps me to narrow the issue, which is an apparent incompatibility between the OS, the media, and the code. Hard data, however it is obtained (and documented) is invaluable in determining root cause of an issue.

John made a comment about 'COMPAT' being not applicable for all disk types. This is an interesting observation, because the manual would suggest that this is the most compatible method. I will investigate this further. Is it folder permissions? Is it some additional interaction with the virus software that manifests itself only when accessing the local hard drives?

All of these questions have (I hope) an answer. And it from folks like you and others offering suggestions and hard data that I hope to find what the truth is!

I thank you all for helping!
Bill

LitusSaxonicum · Posted: Sun Jan 25, 2015 8:15 pm Post subject:

I'm always astonished by radically different hard disk timings, because provided that the file size is less than the disk cache, shouldn't we be measuring the speed of the cache in every case?

Windows is a law unto itself!

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

There's disk cache, and there is disk cache....

Yes, Windows is an animal all to its own.

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

I have completed another set of benchmarks on my primary machine, mapped drive, using all SHARE= options, and writing and reading in both DIRECT and SEQUENTIAL file structures, and the results are seen below. I separated each SHARE= into its own set.

From these data, it is clear that, while some may not see these results, there is something "going on" that is getting in the way of the disk I/O when the SHARE is set to either DENYNONE or COMPAT and ACCESS=DIRECT.

That is where I shall place my attention to uncover the answer.