forums.silverfrost.com

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,
You posted:

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

Here is the test code I used, should you desire to replicate my tests on your machine and share the results.

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

Paul, it would be useful (to me, at least) to know how FTN95 opens a file in each of the available SHARE= modes. Right now, I can only guess. I might be able to find something with this small bit of detailed knowledge, perhaps even able to replicate in another language to identify the root cause. The way I figure it, if I found this, perhaps others have the same problem. And, if there is a root cause, it can be addressed (or not, as the case might be).

Rather than to say "Don't do it this way", perhaps the proper answer is "If you do it this way AND you get these kinds of results...".

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,

I ran your test
on Windows 7 Home Premium, Service Pack 1
i5-2300
921 GB partition of 2Tb HDD
in directory c:\temp\forum
ftn95 bill_file_io /-imp /lgo >zz

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,

I changed randomfile to:
do i=1,length
iorecord(i)=char(int(255.*random@()))
end do
I also tidied up the formats and used SYSTEM_CLOCK for timing. I got the following results.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,

I did another test which is useful to compare with the previous by: - increased the record size to 1940
- introduced a test for no use of share=
The results are:

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

ctd.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

.ctd/

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,

I have produced a test program that tests more alternatives of size and disk buffering.
If you watch the program run, using windows explorer, you will see which share= options are continually updating the disk for access='direct'. Other options show the use of the disk cache with fewer changes to the stored file.
Sequential access does not appear to work this way and might not not support multiple read/write.
I tried to reduce flushing by minimising open/close, but that does not appear to be working as I hoped.

The latest test code is:

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

.ctd/

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

.ctd 2/

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

John,

WOW, you certainly have gone over and above what I could have ever expected, and I am very grateful! You put a lot of effort into this.

If I could state in one sentence what you've found: The use of SHARE has an effect on the performance, and certain SHARE options have a bigger effect than others. There's a lot more you did here, but would that be accurate in your view?

Yes, the original code was inefficient (i.e. use of random()). The original thought there was that if the Antivirus was triggering on some random data, perhaps this would show it. After I was able to identify and workaround the odd IOSTAT values (10005 and 10002) when files were opened, the need for randomized data actually went away. But, left it for consistency.The random() was left in for the reads for timing comparisons.

I only perform one open/close for each block of records, so I am curious about your comment (Mon Jan 26, 2015 3:38 am MST).

It is interesting that the SHARE would cause a difference in performance, and while your numbers are certainly faster than mine it still begs the question: Why?

As far as bigger records, no, not possible. My software has a legacy that dates back to the early 80's, when it was run on machines as simple as S-100 bus, CP/M. File size is set by the variables required. Would that I could change the sizes or formats!! It would make life so much easier. Some of the files created in those early days are still in use today, just much bigger!

When some of the users started using a networking environment in the mid 90's, the question of access restrictions came up. With multiple users on individual machines, sometimes EXCLUSIVE access is required when updating the master files (most of which are binary, unformatted, direct), while in most cases, simple READ access is needed. As well, there are some sequential, formatted files that can be affected. So, yes, I do need to have the SHARE set appropriately, and the setting of this access is dependent on the file usage at the time, and the software asks for that access as needed when opening the file.

I hope to replicate your results on my primary machine later today. It will be an interesting comparison.

I am looking at "perfmon" to do some more detailed looks at the transfers and IO operations counts associated with these various operations. Once I get that up and running (and a little more time), I'll share it here.

JohnCampbell · Joined: 16 Feb 2006 Posts: 2554 Location: Sydney

Bill,

I have now seen your problem !!

I transferred my latest test code to my notebook, which is running Windows 8.1. This OS must rival Vista as I find it an appalling downgrade from Windows 7.

I am getting huge delays on direct access files with some share= options. ( some cases 20sec, 40 sec and even 250 sec ! I'll update when the tests are complete.)
I am trying running in a normal DOS box then running as administrator. Both have big delays for direct access files. Must have something to do with directory access rights. As these delays occur while writing and not just when opening, there could be some incompatibility or tuning required with salflibc.dll and Win 8.1.

Not sure what the solution is, but certainly removing Windows 8.1 would be a good approach.

John

wahorger · Joined: 13 Oct 2014 Posts: 1217 Location: Morrison, CO, USA

John, perhaps removing 8.1 might be a bit premature. It is nice to see that it can happen to others!

I do suspect it has something to do with policies or permissions, but I am at a loss to know. I'm not a SYSADMIN!

Bill

mecej4 · Joined: 31 Oct 2006 Posts: 1886

I have been using W8.0 since May 2013, and W8.1 since Dec 2013. While the tiles interface is correctly worthy of ridicule and condemnation, the underlying OS is solid and competent. I do most of my work at the command line, and have a third party Start program to make the system behave similarly to W7.

I took John's program of Jan 26, 2015 11:19 am, and modified it as follows.

These changes allow the code to be run on a ramdisk using FTN95 and Intel Fortran. I see none of the problems of slowdown that others have mentioned, indicating that "something else" is the culprit. With FTN95 I get speeds of about 30 MB/s, for all except the runs with small files, where the speed can go down to 10 MB/s or less. With Intel Fortran, the corresponding speeds are 45 MB/s and 12 MB/s. (These "speeds" are as reported by the program, on a laptop with an i5-4200U CPU)

Once you set up a comfortable program development environment under it, you will not complain about W8x. In a multiprocessing environment, I/O to files with shared access is subject to many influences in addition to hardware, OS and compiler brand differences.