 |
forums.silverfrost.com Welcome to the Silverfrost forums
|
View previous topic :: View next topic |
Author |
Message |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Thu Jul 28, 2011 9:36 am Post subject: Binary file in FORTRAN |
|
|
I am trying (or need to!) something I have never done in FORTRAN! I'm mainly into coding numerical stuff in Fortran.
Now, I need to write a binary file which is to be generated from another huge file (which contains enormous data).
Can anyone guide me in write direction? I know the info provided is a bit vague.Howver, if you can help or need more clarifications please let me know.
Thanks a lot in advance.
Christy |
|
Back to top |
|
 |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Thu Jul 28, 2011 5:00 pm Post subject: |
|
|
Please can anyone help urgently? |
|
Back to top |
|
 |
PaulLaidler Site Admin
Joined: 21 Feb 2005 Posts: 8210 Location: Salford, UK
|
Posted: Fri Jul 29, 2011 8:00 am Post subject: |
|
|
You may need to use unformatted WRITE statements using standard Fortran. Alternatively FTN95 has its own binary write via WRITEF@ etc. |
|
Back to top |
|
 |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Fri Jul 29, 2011 8:30 am Post subject: |
|
|
Please can you suggest a reference/example concerning this (writing unformatted write statements)??
Please help |
|
Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2615 Location: Sydney
|
Posted: Fri Jul 29, 2011 9:12 am Post subject: |
|
|
I'd recommend that you take the simpler approach and write using formatted write statements to a text file. There are a number of reasons I would recommend this.
1) It is easier to do text. Binary is for when you have sorted it all out and want to retain maximum real precision.
2) It is easier to check as you go. Open the text file with NOTEPAD and you can easily check what you have created.
3) If you are writing real numbers be sure that you retain sufficient precision in the format you use. This can also apply to large integers. You can fix this as you go.
4) Don't expect the first attempt will be the last version, so start with an easy part first, then build on it.
An example of writing information to (say) unit 12
OPEN (unit=12, file='Echo_output.txt', iostat=iostat) ! open the file
if (iostat /= 0) write (*,*) 'error opening file',iostat
do i = 1, many_numbers
write (12,2001) i, (iarray(j,i),j=1,a_few) ! assume integer
write (12,2002) i, (array(j,i),j=1,a_few) ! assume real
end do
2001 format ('Row',i6,' Iarray Values are', 20i8)
2002 format ('Row',i6,' Rarray Values are', 20es13.8 )
Finish with a CLOSE ane exit the program before you look at the results, as the file buffers must be cleared.
Do not write too many values to a single line.
Start with a simple approach and build on it.
You have not stated why you need the output and how you are going to use it. Importing into excel is a common requirement, so have a simple structure to your file. (think of what file layout you want !)
The next stage would be to include key words in the output and then be able to search for these words.
Hope this helps.
John |
|
Back to top |
|
 |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Fri Jul 29, 2011 1:07 pm Post subject: |
|
|
Thanks a lot.
Yes, I know to create a text file.
The idea of binary was as i need to export this file in a software (not Excel) which understands the binary format and carries out a finite element analysis (if you're not familiar with finite element analysis, it is a kind of engineering analysis).
I understand when creating in a binary format will be musch useful as the size will be much smaller than a regular text file in notepad as you (i.e.John) suggests.
Any ideas/examples of creating binary file?
Christy |
|
Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2615 Location: Sydney
|
Posted: Sun Jul 31, 2011 2:07 am Post subject: |
|
|
Binary file formats vary across compilers.
To create a binary file you need to
! open
OPEN (unit=12, file='file_name', iostat=iostat, access='SEQUENTIAL', form='UNFORMATTED', status='UNKNOWN')
! then write
WRITE (12) ((array(i,j),i=1,n),j=1,m)
The write statement creates the "record" on the file, which consists of:
<header> <binary values> <trailer>
The problem is that the format of the <header> and <trailer> vary between compilers. FTN95 and IFORT are different. (FTN95 header format also varies, depending on the length of the record, so be careful. You need to read FTN95 help)
You can overcome this problem, by opening with access='TRANSPARENT' and then writing the <header> and <trailer> in a format expected by the FE program. You can also do this with access='DIRECT'.
Assuming that you are not transfering the files between operating systems, you do not need to consider the binary format. ( see Big-endian and little-endian if you need to go there )
An alternative could be to get FTN95 to provide a compiler option of compatible binary file formats, but this would need to consider which compiler file format to provide.
Years ago a goal of the fortran standard was to improve compatibility between different compilers. There are a few areas, like this, where the standard has gone missing.
You need to read the FTN95 and the FE documentation !
John |
|
Back to top |
|
 |
IanLambley
Joined: 17 Dec 2006 Posts: 506 Location: Sunderland
|
Posted: Sun Jul 31, 2011 12:35 pm Post subject: |
|
|
John et all,
I wrote a bit of example code to convert FTN95 unformatted data to the 4byte header method used by others, and back again.
Here they are:
Code: | winapp
! convert an unformatted binary file from the FTN95 format
integer*1 input(65536),work(65536*2),ibyte_long(4)
INTEGER*2 HANDLE_in,handle_out, ERROR_CODE,itest_ffh
integer*4 i,iwork_pointer,iwork_end,irec_len, ilong, nline,imove, ilen_extra, itransfer_4bytes, nbytes_read
equivalence (ibyte_long,ilong)
iwork_pointer = 1
iwork_end = 0
itest_ffh = -1 !-1 as a single byte is ffh the code for a long record
irec_len = 0
call openr@('ftn95unformattedlong.dat',handle_in,error_code)
call openw@('otherftnformattedlong.dat',handle_out,error_code)
nline = 0
do
call READF@(input, handle_in, 65536L, NBYTES_READ, ERROR_CODE)
if(NBYTES_READ .eq. 0)goto 9999
nline = nline + 1
print *,nline,nbytes_read,error_code
!move previously handled work to the start of the work buffer
imove = 0
do i=iwork_pointer,iwork_end
imove = imove + 1
work(imove) = work(i)
enddo
iwork_pointer = 1
iwork_end = imove
!move the input line to the work
do i=1,nbytes_read
imove = imove + 1
work(imove) = input(i)
enddo
iwork_end = imove
!
! now process the next chunk from iwork_pointer to iwork_end
do while (iwork_pointer .lt. iwork_end)
if(work(iwork_pointer) .eq. itest_ffh)then
! a 255 byte detected, so next four bytes are the record length
iwork_pointer = iwork_pointer + 1
print *,'iwork_pointer',iwork_pointer
irec_len = itransfer_4bytes(work(iwork_pointer))
iwork_pointer = iwork_pointer + 4
ilen_extra = 5
else
! not 255 (FFh or -1 as a single byte) therefore
! as a single byte value greater than 127 is treated as a negative number,
! first clear a 4 byte word and then using equivalence, load up its first byte
! with the integer*1 length
ilong = 0
ibyte_long(1) = work(iwork_pointer)
irec_len = ilong
iwork_pointer = iwork_pointer + 1
ilen_extra = 1
endif
print *,'irec_len',irec_len,ilen_extra
!process record to output data here
! prefix with record length as a 4 byte integer
call WRITEF@(irec_len,handle_out, 4L, ERROR_CODE)
!output the data
call WRITEF@(work(iwork_pointer),handle_out, irec_len, ERROR_CODE)
! postfix with record length as a 4 byte integer
call WRITEF@(irec_len,handle_out, 4L, ERROR_CODE)
! now advance to the next record start
iwork_pointer = iwork_pointer + irec_len + ilen_extra
print*,'next iwork_pointer',iwork_pointer
enddo
enddo
9999 continue
call CLOSEF@(handle_in, error_code)
call CLOSEF@(handle_out, error_code)
print *,nline
end
integer*4 function itransfer_4bytes(idata1)
integer*1 idata1(4),jdata1(4)
integer*4 jdata4,i
equivalence (jdata1,jdata4)
do i=1,4
jdata1(i) = idata1(i)
print '(z2.2)',jdata1(i)
enddo
itransfer_4bytes = jdata4
print '(z8.8)',jdata4
end
|
|
|
Back to top |
|
 |
IanLambley
Joined: 17 Dec 2006 Posts: 506 Location: Sunderland
|
Posted: Sun Jul 31, 2011 12:36 pm Post subject: |
|
|
And in the other direction:
Code: | winapp
! convert an unformatted binary file to the FTN95 format
integer*1 input(65536),work(65536*2),ibyte_long(4)
INTEGER*2 handle_in,handle_out, error_code,itest_ffh
integer*4 i,iwork_pointer,iwork_end,irec_len, ilong, nline,imove, ilen_extra, itransfer_4bytes, nbytes_read
equivalence (ibyte_long,ilong)
iwork_pointer = 1
iwork_end = 0
itest_ffh = -1 !-1 as a single byte is ffh the code for a long record
irec_len = 0
call openr@('otherftnformattedlong.dat',handle_in,error_code)
call openw@('ftn95formattedlong_new.dat',handle_out,error_code)
nline = 0
do
call READF@(input, handle_in, 65536L, nbytes_read, error_code)
if(nbytes_read .eq. 0)goto 9999
nline = nline + 1
print *,nline,nbytes_read,error_code
!move previously handled work to the start of the work buffer
imove = 0
do i=iwork_pointer,iwork_end
imove = imove + 1
work(imove) = work(i)
enddo
iwork_pointer = 1
iwork_end = imove
!move the input line to the work
do i=1,nbytes_read
imove = imove + 1
work(imove) = input(i)
enddo
iwork_end = imove
!
! now process the next chunk from iwork_pointer to iwork_end
do while (iwork_pointer .lt. iwork_end)
!get the 4 byte record length
irec_len = itransfer_4bytes(work(iwork_pointer))
iwork_pointer = iwork_pointer + 4
ilen_extra = 4
print *,'irec_len',irec_len,ilen_extra
!process record to output data here
!if the record length is less than 255 long, then output a single byte
! record length
if(irec_len .lt. 255)then
ilong = irec_len
call WRITEF@(ibyte_long,handle_out, 1L, error_code)
call WRITEF@(work(iwork_pointer),handle_out, irec_len, error_code)
call WRITEF@(ibyte_long,handle_out, 1L, error_code)
else
! prefix with record length as ffh + a 4 byte integer
call WRITEF@(itest_ffh,handle_out, 1L, error_code)
call WRITEF@(irec_len,handle_out, 4L, error_code)
!output the data
call WRITEF@(work(iwork_pointer),handle_out, irec_len, error_code)
! postfix with record length as a 4 byte integer + ffH
call WRITEF@(irec_len,handle_out, 4L, error_code)
call WRITEF@(itest_ffh,handle_out, 1L, error_code)
endif
! now advance to the next record start
iwork_pointer = iwork_pointer + irec_len + ilen_extra
print*,'next iwork_pointer',iwork_pointer
enddo
enddo
9999 continue
call CLOSEF@(handle_in, error_code)
call CLOSEF@(handle_out, error_code)
print *,nline
end
integer*4 function itransfer_4bytes(idata1)
integer*1 idata1(4),jdata1(4)
integer*4 jdata4,i
equivalence (jdata1,jdata4)
do i=1,4
jdata1(i) = idata1(i)
print '(z2.2)',jdata1(i)
enddo
itransfer_4bytes = jdata4
print '(z8.8)',jdata4
end
|
They need tidying and generalising.
Ian |
|
Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2615 Location: Sydney
|
Posted: Mon Aug 01, 2011 1:02 am Post subject: |
|
|
Ian,
Since you have investigated the record structure of the FTN95 sequential binary format, I'd be interested in confirming the format.
What is the <header> format for FTN95 ? Is it a 2 byte / 6 byte mixed length count.
What is the count; bytes or 4-byte words ?
Does FTN95 have a <trailer> to allow backspace ?
I think the idea of a compiler option to standardise on a 4-byte header would be a good option.
I think there still remains the problem of if the length is bytes or words, as I think Lahey and Intel do vary with this. I think Intel provide a compiler switch for changing between bytes and words. The standard refers to "processor-dependent" or "file storage" units, so it's no help in standardising the operation across compilers!
I do successfully transfer binary files between Lahey and FTN95, but use a fixed length direct access file, which does not include the record length header in the file record.
John |
|
Back to top |
|
 |
IanLambley
Joined: 17 Dec 2006 Posts: 506 Location: Sunderland
|
Posted: Mon Aug 01, 2011 7:52 am Post subject: |
|
|
John,
FTN95 header format seems to be
a. For records less than 255 bytes, a one byte length followed by the data terminated by the same one byte length. This allows reading forward and backspacing.
b. For records of length 255 and above, a one byte flag = ffh or -1 as a signed one byte integer, followed by a 4-byte length, the data and then followed by the 4-byte length and a one byte flag = ffh or -1.
So yes records have a trailer to allow backspace.
FTN95 use bytes as the storage size. I seem to remember that VAX F77 used words for record lengths in direct access files. Don't remember whether that was 2-byte or 4-byte - the latter I think. I would have to boot my VaxStation to find out!
Ian |
|
Back to top |
|
 |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Mon Aug 01, 2011 3:26 pm Post subject: |
|
|
Thanks a lot John and Ian,
John as you suggested, I'm reading the docuemntation for the FE part.Please can you suggest any reading refernce for Fortran documentation? Shall be really obliged.
Christy |
|
Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2615 Location: Sydney
|
Posted: Tue Aug 02, 2011 1:13 am Post subject: |
|
|
Why not look at ftn95.chm; especially OPEN and FORM='UNFORMATTED'
lahey.com also has a fortran 95 reference .pdf which is set out in well.
http://www.lahey.com/docs/LangRefEXP72_revG04.pdf
It is a good reference document for the Fortran 95 language.
Using ACCESS='TRANSPARENT' you should be able to write a binary file, plus write the header and trailer in a compatible format to what the FE program expects. For writing a real*8 array A and HEADER is integer*4, the following should work.
HEADER = N*M*8 ! *8 for 1_byte units
WRITE (12) HEADER, ((A(i,j),i=1,n,j=1,m), HEADER
Remember to confirm what "file storage units" are being expected by the FE program. FTN95 and Lahey use bytes, while Intel uses 1-byte or 4-byte units (default), depending on the compiler option. |
|
Back to top |
|
 |
christyleomin
Joined: 08 Apr 2011 Posts: 155
|
Posted: Tue Aug 02, 2011 9:13 am Post subject: |
|
|
John, thanks a lot.Just a very absic question (sorry for that); you said;
To create a binary file you need to
! open
OPEN (unit=12, file='file_name', iostat=iostat, access='SEQUENTIAL', form='UNFORMATTED', status='UNKNOWN')
! then write
WRITE (12) ((array(i,j),i=1,n),j=1,m)
The write statement creates the "record" on the file, which consists of:
<header> <binary values> <trailer>
I don't get what you mean when you said;
The write statement creates the "record" on the file, which consists of:
<header> <binary values> <trailer>
What you mean by: header, trailer and binary values? |
|
Back to top |
|
 |
JohnCampbell
Joined: 16 Feb 2006 Posts: 2615 Location: Sydney
|
Posted: Tue Aug 02, 2011 10:21 am Post subject: |
|
|
When you open a file as
access='SEQUENTIAL', form='UNFORMATTED'
You must use a binary write with a binary (unformatted) file:
write (12) ((array(i,j),i=1,n),j=1,m)
This creates a "record" on the file, which can be read with a :
read (12) ((array(i,j),i=1,n),j=1,m)
The record consists of the binary memory dump of "array". The fortran I/O routines also put a header and a trailer on the record to let the read (and backspace) statement know how long the record is so that the read statements can step from record to record. You don't have to read the full record, although this would be unusual. It is recommended that you use an identical READ to the WRITE
This is the structure of the record in the file.
You should read the the Elements of Fortran > Input/Output section of the Lahey fortran language documentation.
Binary files are easy to use. They are just difficult to look at.
John |
|
Back to top |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|