Question about unformatted output

Dear all,

I hope someone can shed some light on this issue I am having with unformatted binary files.

I want (for various reasons) to write a binary file with C# that can be read using Fortran as if it were an ordinary Fortran unformatted file.

I was expecting the Fortran standard to specify what the output structure is of an unformatted binary file, but I have not been able to find that. Other web pages have stated that a record in an unformatted file is surrounded by a 4byte header and footer which indicate the number of bytes in the record.

However, I have found that when using some test programs with Salford FTN95, I get only a 1 byte header and footer.

Essentially, my problem is this: using this code

program bintest
implicit none

!test the writing of binary data to a Fortran unformatted file
open(10,file='ftnbintest.out',form='unformatted')

write(10) 1
write(10) 16
write(10) 32
write(10) 10281
write(10) 305419896

close(10)

end program bintest

gives different results when compiling with FTN95 and when compiling with GNU fortran. I get a 1 byte header and footer with FTN95 and a 4 byte header and footer with GNU.

Can someone explain? What should be in the output? Can someone point me to some document where the Fortran binary file structure is defined?

Thanks,

Mark.

The single byte header would imply a maximum length of 255 bytes for an unformatted record. The reason for enclosing each record in the length would allow the BACKSPACE command to reverse the sequential access, without the program having to keep a note of all previous record lengths.

To write the file without this record length information, open the file with access='transparent'

Ian

Ian,

Is this the same as opening a file with OPENRW@ and writing to it with WRITEF@ ?

John

Hi,

Perhaps I wasn't clear in my original post...

What it boils down to is a question of exactly what the structure of a Fortran unformatted binary file with sequential (default) access should be.

Is this spelled out in the Fortran standard somewhere? If Salford FTN95 uses a 1 byte header (as it seems to) and GNU fortran uses a 4 byte header (as all webpages I have found about this topic would agree with) then: is one wrong and the other right? Is there actually no specification of what the header size should be? Is it simply that unformatted outputs should not be expected to be readable by the same code compiled with different compilers?

Mark.

Mark,

My understanding is that you can't read an unformatted file created with a program compiled with X with a program compiled with Y. If you treat the file as a binary file, and read it byte by byte, then you can. But you need to know the conventions for each file type. This is no different from reading a Quattro (.WBx) file in Excel (you can't do it by renaming it to .XLS)or vice versa, and Fortran is not therefore 'under specified', and no compiler is 'wrong'.

If it works, they you were lucky, but mostly it doesn't.

There is a good case for saving all data files in plain text, and also transfer between programs in plain text too, as this does open up the possibility of easy data interchange.

Since you could launch a GNU-fortran program from an FTN95 program, you could mix and match what compiler you chose for linked programs in a big system.

Eddie

Seems like it can be defined by the compiler writer.

From the standard: 9.1.2 Unformatted record An unformatted record consists of a sequence of values in a processor-dependent form and may contain data of any type or may contain no data. The length of an unformatted record is measured in file storage units (9.2.4) and depends on the output list (9.5.2) used when it is written, as well as on the processor and the external medium. The length may be zero. Unformatted records may be read or written only by unformatted input/output statements.

9.2.4 File storage units A file storage unit is the basic unit of storage in a stream file or an unformatted record file. It is the unit of file position for stream access, the unit of record length for unformatted files, and the unit of file size for all external files. Every value in a stream file or an unformatted record file shall occupy an integer number of file storage units; if the stream or record file is unformatted, this number shall be the same for all scalar values of the same type and type parameters. The number of file storage units required for an item of a given type and type parameters may be determined using the IOLENGTH= specifier of the INQUIRE statement (9.9.3). For a file connected for unformatted stream access, the processor shall not have alignment restrictions that prevent a value of any type from being stored at any positive integer file position. The number of bits in a file storage unit is given by the constant FILE STORAGE SIZE (13.8.2.3) defined in the intrinsic module ISO FORTRAN ENV. It is recommended that the file storage unit be an 8-bit octet where this choice is practical. NOTE 9.12 The requirement that every data value occupy an integer number of file storage units implies that data items inherently smaller than a file storage unit will require padding. This suggests that the file storage unit be small to avoid wasted space. Ideally, the file storage unit would be chosen such that padding is never required. A file storage unit of one bit would always meet this goal, but would likely be impractical because of the alignment requirements. The prohibition on alignment restrictions prohibits the processor from requiring data alignments larger than the file storage unit. The 8-bit octet is recommended as a good compromise that is small enough to accommodate the requirements of many applications, yet not so small that the data alignment requirements are likely to cause significant performance problems. 9.2.4 File storage units A file storage unit is the basic unit of storage in a stream file or an unformatted record file. It is the unit of file position for stream access, the unit of record length for unformatted files, and the unit of file size for all external files. Every value in a stream file or an unformatted record file shall occupy an integer number of file storage units; if the stream or record file is unformatted, this number shall be the same for all scalar values of the same type and type parameters. The number of file storage units required for an item of a given type and type parameters may be determined using the IOLENGTH= specifier of the INQUIRE statement (9.9.3). For a file connected for unformatted stream access, the processor shall not have alignment restrictions that prevent a value of any type from being stored at any positive integer file position. The number of bits in a file storage unit is given by the constant FILE STORAGE SIZE (13.8.2.3) defined in the intrinsic module ISO FORTRAN ENV. It is recommended that the file storage unit be an 8-bit octet where this choice is practical. NOTE 9.12 The requirement that every data value occupy an integer number of file storage units implies that data items inherently smaller than a file storage unit will require padding. This suggests that the file storage unit be small to avoid wasted space. Ideally, the file storage unit would be chosen such that padding is never required. A file storage unit of one bit would always meet this goal, but would likely be impractical because of the alignment requirements. The prohibition on alignment restrictions prohibits the processor from requiring data alignments larger than the file storage unit. The 8-bit octet is recommended as a good compromise that is small enough to accommodate the requirements of many applications, yet not so small that the data alignment requirements are likely to cause significant performance problems. 9.2.4 File storage units A file storage unit is the basic unit of storage in a stream file or an unformatted record file. It is the unit of file position for stream access, the unit of record length for unformatted files, and the unit of file size for all external files. Every value in a stream file or an unformatted record file shall occupy an integer number of file storage units; if the stream or record file is unformatted, this number shall be the same for all scalar values of the same type and type parameters. The number of file storage units required for an item of a given type and type parameters may be determined using the IOLENGTH= specifier of the INQUIRE statement (9.9.3). For a file connected for unformatted stream access, the processor shall not have alignment restrictions that prevent a value of any type from being stored at any positive integer file position. The number of bits in a file storage unit is given by the constant FILE STORAGE SIZE (13.8.2.3) defined in the intrinsic module ISO FORTRAN ENV. It is recommended that the file storage unit be an 8-bit octet where this choice is practical. 9.2.4 File storage units A file storage unit is the basic unit of storage in a stream file or an unformatted record file. It is the unit of file position for stream access, the unit of record length for unformatted files, and the unit of file size for all external files. Every value in a stream file or an unformatted record file shall occupy an integer number of file storage units; if the stream or record file is unformatted, this number shall be the same for all scalar values of the same type and type parameters. The number of file storage units required for an item of a given type and type parameters may be determined using the IOLENGTH= specifier of the INQUIRE statement (9.9.3). For a file connected for unformatted stream access, the processor shall not have alignment restrictions that prevent a value of any type from being stored at any positive integer file position. The number of bits in a file storage unit is given by the constant FILE STORAGE SIZE (13.8.2.3) defined in the intrinsic module ISO FORTRAN ENV. It is recommended that the file storage unit be an 8-bit octet where this choice is practical. The 8-bit octet is recommended as a good compromise that is small enough to accommodate the requirements of many applications, yet not so small that the data alignment requirements are likely to cause significant performance problems. If the file is connected for unformatted sequential access, the record is created with a length sufficient to hold the values from the output list. This length shall be one of the set of allowed record lengths for the file and shall not exceed the value specified in the RECL= specifier, if any, of the OPEN statement that established the connection.

Have a look at the inquire statement with parameters: IOLENGTH = scalar-int-variable UNFORMATTED = scalar-default-char-variable

Mark,

I tried your program and confirmed a 1-byte header and trailer. This surprised me, as I was expecting a 4-byte header. You could try writing a record of more than 256 bytes to see what happens, as that is certainly legal.

My answer is always to use a text format. I can't count the number of examples I've seen of people wasteing time with binary formats. You havn't yet tried real number formats.

With the speed of computing these days, it is not often you are requiring binary format efficiency. Text format typically does not have a high time or file size penalty, and it's easy to see what should be read.

You have to address the number of characters required to insure the accuracy is what you want.

Alternatively, you can always use transparent format and write what you want. My changes to your program may help you get there.

program bintest implicit none ! integer4 i, iostat, ic character c1 ! !test the writing of binary data to a Fortran unformatted file open(10,file='ftnbintest.out',form='unformatted')

write(10) 1 write(10) 16 write(10) 32 write(10) 10281 write(10) 305419896

close(10)

open(10,file='ftnbintest.out',form='unformatted',access='TRANSPARENT')

do i = 1,99999 read (10,iostat=iostat) c write (,) i,ichar(c),iostat if (iostat/=0) exit end do

end program bintest

I did what I suggested and wrote a record which is 1600 bytes long. Using ftn95 Ver 5.10.0 this produced a 5-byte header and trailer as :-

(255) 1-byte long length indicator (1600) 4-byte record header ... 1600 bytes of info written (1600) 4-byte record header (255) 1-byte long length indicator

I think this is different to the records written with earlier versions of ftn95, which I thought had a 4-byte header and trailer ?

Paul, Is this a recent change ?

My revised program was :-

program bintest implicit none ! integer4 i, iostat, ic, ii(400) character c1 ! !test the writing of binary data to a Fortran unformatted file open (10,file='ftnbintest.out',form='unformatted')

do i = 1,400 ii(i) = i end do ! write (10) 1 write (10) 16 write (10) 32 write (10) 10281 write (10) 305419896 write (10) (ii(i),i=1,400)

close(10)

open(10,file='ftnbintest.out',form='unformatted',access='TRANSPARENT') ! do i = 1,99999 read (10,iostat=iostat) c write (,) i,ichar(c),iostat if (iostat/=0) exit end do ! end program bintest