Welcome to our forums
I have a program distributed by NASA. During debugging I see the mentioned error on reading a binary data file. I'm able to attach the ftn95 project with source files and data file. Or source files can be downloaded from http://nssdcftp.gsfc.nasa.gov/models/atmospheric/hwm07/
I use Win32 build in Plato.
Thanks in advance.
If those files were written by another compiler, then there is no real standard for unformatted binary files and the form of the record length part of each line changes for lines of length less or greater than 256 characters. Try using this program on the .dat files:
winapp
! convert an unformatted binary file to the FTN95 format
integer*1 input(65536),work(65536*2),ibyte_long(4)
INTEGER*2 handle_in,handle_out, error_code,itest_ffh
integer*4 i,iwork_pointer,iwork_end,irec_len, ilong, nline,imove, ilen_extra, itransfer_4bytes, nbytes_read
equivalence (ibyte_long,ilong)
iwork_pointer = 1
iwork_end = 0
itest_ffh = -1 !-1 as a single byte is ffh the code for a long record
irec_len = 0
call openr@('otherftnformattedlong.dat',handle_in,error_code)
call openw@('ftn95formattedlong_new.dat',handle_out,error_code)
nline = 0
do
call READF@(input, handle_in, 65536L, nbytes_read, error_code)
if(nbytes_read .eq. 0)goto 9999
nline = nline + 1
print *,nline,nbytes_read,error_code
!move previously handled work to the start of the work buffer
imove = 0
do i=iwork_pointer,iwork_end
imove = imove + 1
work(imove) = work(i)
enddo
iwork_pointer = 1
iwork_end = imove
!move the input line to the work
do i=1,nbytes_read
imove = imove + 1
work(imove) = input(i)
enddo
iwork_end = imove
!
! now process the next chunk from iwork_pointer to iwork_end
do while (iwork_pointer .lt. iwork_end)
!get the 4 byte record length
irec_len = itransfer_4bytes(work(iwork_pointer))
iwork_pointer = iwork_pointer + 4
ilen_extra = 4
print *,'irec_len',irec_len,ilen_extra
!process record to output data here
!if the record length is less than 255 long, then output a single byte
! record length
if(irec_len .lt. 255)then
ilong = irec_len
call WRITEF@(ibyte_long,handle_out, 1L, error_code)
call WRITEF@(work(iwork_pointer),handle_out, irec_len, error_code)
call WRITEF@(ibyte_long,handle_out, 1L, error_code)
else
! prefix with record length as ffh + a 4 byte integer
call WRITEF@(itest_ffh,handle_out, 1L, error_code)
call WRITEF@(irec_len,handle_out, 4L, error_code)
!output the data
call WRITEF@(work(iwork_pointer),handle_out, irec_len, error_code)
! postfix with record length as a 4 byte integer + ffH
call WRITEF@(irec_len,handle_out, 4L, error_code)
call WRITEF@(itest_ffh,handle_out, 1L, error_code)
endif
! now advance to the next record start
iwork_pointer = iwork_pointer + irec_len + ilen_extra
print*,'next iwork_pointer',iwork_pointer
enddo
enddo
9999 continue
call CLOSEF@(handle_in, error_code)
call CLOSEF@(handle_out, error_code)
print *,nline
end
integer*4 function itransfer_4bytes(idata1)
integer*1 idata1(4),jdata1(4)
integer*4 jdata4,i
equivalence (jdata1,jdata4)
do i=1,4
jdata1(i) = idata1(i)
print '(z2.2)',jdata1(i)
enddo
itransfer_4bytes = jdata4
print '(z8.8)',jdata4
end
I was able to convert the unformatted data files after I enlarged the buffers in IanLambley's converter program to cover the longest of the input data files.
The NASA source code has some bugs that are easy to find and fix, but without those fixes you will not get correct results. Subroutine HWMupdate needs an explicit interface but one is not provided. There are some subroutine arguments that have declared INTENT(IN) but are modified. There are some variables that are not initialized. Here is a brief list of fixes:
File dwm07b.f90:
Change INTENT of GLON to 'IN OUT' in subroutines DWM07B_HWM_INTERFACE and GD2QD
Initialize HR in subroutine GD2QD.
File hwm07e.f90:
Change INTENT of GLON to 'IN OUT' in subroutines HWM07.
Initialize LAST() in subroutine HWMQT.
Change subroutine HWMUPDATE to a contained subroutine or provide an explicit interface to it in its caller(s).
Initialize GZWGHT in subroutine LOADMODEL.
File apexcord.f90: Change INTENT of GLON to 'IN OUT' in subroutine APEX.
If you (original poster) are interested, I can see about posting the modified sources and the FTN95 compatibles *.dat files somewhere from where you can download them. (This forum does not seem to allow attaching files to messages).
It always annoys me to see programs released with binary data files. It would be much better if the .dat files were text files, where the data was written as text numbers. The penalty to be paid for file size or run time is minimal, while the choice of number formats to limit precision errors is ususlly minimal, given the accuracy of the initial data estimates. Using text files as an interface between programs also makes validation of the data much easier.
John
Ian, thanks for your program. It is very useful. It fails where the record size is larger than the input array or the logical record is not fully contained in the work array. I think you need to test if the next logical record is fully contained in WORK, rather than 'do while (iwork_pointer < iwork_end)'. If you change 65536 to be larger than the file ( can change up to about 1gb ) then ther is no problem. If you open the file as access='transparent', you can read the file 1 byte at a time and do away with the input buffer, reading the 4_byte header, then the info, then the 4_byte trailer. You might like to consider the TRANSFER intrinsic to generate the integer*4 value. The other problem is if the source file is from a unix or linux source, where the problem of ??_endian makes it more complex. You then need to know the structure of each record.
Hi all, Thanks for help!
I try to use the posted program but have an error with undefined elements in the input array.
If I correctly understand I need to enlarge dimensions of input and work arrays and also enlarge NBYTES value in the READF@ call.
Is it correct?
Quoted from mecej4 I was able to convert the unformatted data files after I enlarged the buffers in IanLambley's converter program to cover the longest of the input data files.
The NASA source code has some bugs that are easy to find and fix, but without those fixes you will not get correct results. Subroutine HWMupdate needs an explicit interface but one is not provided. There are some subroutine arguments that have declared INTENT(IN) but are modified. There are some variables that are not initialized.
If you (original poster) are interested, I can see about posting the modified sources and the FTN95 compatibles *.dat files somewhere from where you can download them. (This forum does not seem to allow attaching files to messages).
mecej4, It seems it will be really simlier if you are able to update files. If it is possible, of course.
Thanks in advance!
John, Thanks for the info. I could have opened it as a direct access file with a recl=1, but I was trying to avoid too many reads as I thought that could slow down the processing. Yes, the TRANSFER intrinsic would be much better but I'm caught in a Fortran 66 time warp, I must do better, but, heavens, I've just sold my 1961 Riley 1.5 and I'm still in shock! LOL Regards Ian
Konstantin8:
mecej4, It seems it will be really simlier if you are able to update files. If it is possible, of course.
Thanks in advance!
I have combined (i) the modified F90 sources, (ii) the source code of a utility (in C) to convert Intel/GFortran/etc. Fortran unformatted files to FTN95 unformatted files, and (iii) modified FTN95 unformatted data files for the HWM07 package, all into a single Zip file that you may download from https://www.amazon.com/clouddrive/share?s=eA42kBlCS_EvdLiuDMUaUk. [This download link should be valid for at least a few days.]
Please note that there is a bug in the 7.00 FTN95 compiler (and, possibly, earlier versions) that prevents you from compiling with the /check or /undef options. If you use these options, the program will abort with the message 'Attempt to call a subroutine with argument number ten containing too few array elements'. I have prepared a pared-down reproducer and I have posted a bug report for this issue separately.
[UPDATE: Paul Laidler suggests in https://forums.silverfrost.com/Forum/Topic/2410 that you use the option /inhibit_check 5 to work around the bug if you want to be able to use /undef and similar checks.]
Quoted from mecej4 Konstantin8:
mecej4,
It seems it will be really simlier if you are able to update files. If it is possible, of course.
Thanks in advance!
I have combined (i) the modified F90 sources, (ii) the source code of a utility (in C) to convert Intel/GFortran/etc. Fortran unformatted files to FTN95 unformatted files, and (iii) modified FTN95 unformatted data files for the HWM07 package, all into a single Zip file that you may download from https://www.amazon.com/clouddrive/share?s=eA42kBlCS_EvdLiuDMUaUk. [This download link should be valid for at least a few days.]
Please note that there is a bug in the 7.00 FTN95 compiler (and, possibly, earlier versions) that prevents you from compiling with the /check or /undef options. If you use these options, the program will abort with the message 'Attempt to call a subroutine with argument number ten containing too few array elements'. I have prepared a pared-down reproducer and I have posted a bug report for this issue separately.
Thank you a lot! Everything work fine!