forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

FTN95 reads Unformatted files from other sources

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
DanRRight



Joined: 10 Mar 2008
Posts: 2813
Location: South Pole, Antarctica

PostPosted: Thu Sep 26, 2019 4:02 am    Post subject: FTN95 reads Unformatted files from other sources Reply with quote

If you install free app called HDF5 which read/write large files with crazy speed of several GB/second, there exist nice easy to use standalone tool h5dump.exe. It is just one exe file. With it you can extract either directly using FTN95 or manually in DOS prompt all the information from HDF5 files in text format or binary format with little or big endians besides many many other options. Thanks to mecej4 who suggested this thing.

Here is an example of creating text file zzzzText and binary zzzzBinary output from DOS prompt by extracting array called in hdf5 file header as "particles"
Code:
h5dump.exe -d /particles -o zzzzText  Particles.h5

Code:
h5dump.exe -d /particles -o zzzzBinary -b LE -b FILE  Particles.h5

You can see the hdf5 file header which contains variable names, their order and length declarations this way
Code:
h5dump.exe -H Particles.h5 >zzzzHeader'


Same you can program with FTN95 using CALL CISSUE@(text256, ierr) where text 256 are exactly the lines above and your Fortran program will do that extraction for you automatically, CISSUE@ works like charm launching other programs like you'd do from DOS prompt manually.

OK, that was just the lyrics, may be someone will be interested with HDF5 or launching programs from within Fortran but my problem is not exactly with the HDF5. As i still have not figured out how to read HDF5 files directly using FTN95 I try to switch to binary option using this middleman h5dump because its binary creating speed is 10x faster then with text files

With the text format i have no trouble reading h5dump created files using FTN95 but with binary files FTN95 refuses to read anything no matter what binary option in h5dump is used. FTN95 reads its own-created binary files no problem.

Do anyone also had problems reading binary files created with any other non-FTN95 sources like different Fortran compilers or other compilers or apps?


Last edited by DanRRight on Thu Sep 26, 2019 9:04 am; edited 4 times in total
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Thu Sep 26, 2019 6:41 am    Post subject: Reply with quote

Are you using READ or one of the native read routines such as READF@?
Back to top
View user's profile Send private message AIM Address
DanRRight



Joined: 10 Mar 2008
Posts: 2813
Location: South Pole, Antarctica

PostPosted: Thu Sep 26, 2019 8:28 am    Post subject: Reply with quote

Just usual READ

open(unit=18,file=filetoread, STATUS='old', FORM='UNFORMATTED', err=900)
read (18,err=3000, end=3000) x, y, z, px, py, pz...

with each variable in the same order and same length as they were saved and extracted by h5dump into file filetoread
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Thu Sep 26, 2019 8:55 am    Post subject: Reply with quote

READF@ might provide a way forward.
Back to top
View user's profile Send private message AIM Address
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Thu Sep 26, 2019 2:30 pm    Post subject: Reply with quote

Dan,

The problem you are having is possibly that FTN95 uses a different heater/trailer format for Fortran unformatted files.

For most other compilers the header and trailer is 4-bytes, whereas FTN95 uses a 1 or 5-byte header/trailer ( this will work for records up to 2gb in size; I don't know what any compilers do for larger records )

The easiest way to overcome this may be to use ACCESS='Transparent' (Stream I/O).
A good approach would be to write routines READ_Header, READ_Record and READ_Trailer.

This approach works for a single array as the I/O list, ie READ (unit) array(1:n)
If you have a more complex I/O list, then you need to read the record into a buffer and then deconstruct the record, using TRANSFER or other ways.

An alternative would to be to get Paul to provide a compile option to use 4-byte header/trailer formats.

John
Back to top
View user's profile Send private message
DanRRight



Joined: 10 Mar 2008
Posts: 2813
Location: South Pole, Antarctica

PostPosted: Thu Sep 26, 2019 3:10 pm    Post subject: Reply with quote

Thanks Paul, John.
Paul's way works but definitely both ways should work too. If all other compilers use 4bytes or whatever headers then FTN95 has to adopt this as a default.

I will investigate how fast is readf@ in this case, but previously it showed that it is very fast, though it is definitely more unusual that just standard READ binary file. After reading 1 byte data with READF@ i have to convert this to number (i used old EQUIVALENCE)

By some reason devilry kicked me in other place, punishing probably for the "CISSUE@ works like charm" above before all was completed and i lost one day searching for the hidden bug: now after one successful "call cissue@(text, err)" which runs h5dump and extracts data out of HDF5 file, the second call and all calls after that to cissue@ end up h5dump does not extract anything. Doing that manually from DOS prompt works OK infinite amount of times... I had such issue with CISSUE@ when after some time it stopped launching programs. Also bad self-diagnostics of h5dump, it does not tell what's wrong... Difficult life of programmer, eternal bug hunting...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group