forums.silverfrost.com

[silicondale]

I have unformatted direct access files with a wide range of different record lengths, but when reading the files these are not known in advance. However, the record length is always given as an INTEGER*4 in the first word of the first record of each file.

I try to open the files with RECL=4 (or something similarly small: the record length is always much greater than that), so that I can then close the file and reopen it with the correct record length - but I just get the execution error 108 "Direct access record length mismatch".
I have tried opening files without specifying RECL but then get error 106 "A valid record length must be specified if access is direct".

Is there any way to open a direct access file without knowing the correct exact record length in advance?
_________________
(Steve Henley)

[wahorger]

You will get that error if the total file length is not a multiple of 4 with a RECL=4.

I have the same issue, extracting a "type" from the first "record". I have defined this as an INTEGER*2, and open the file with a record length=2. It always works for the reason below.

All my files are created with INTEGER*2 as the minimum size multiple of any data that is present, including characters. The reasons are "lost in antiquity".

You could open the file with a record length of 1, then extract the first 4 records, byte by byte. That should always work.

[silicondale]

Many thanks! Tried a little test program and it works. Simple solution, and probably the last suggestion is the best, because i can't guarantee that all files will have even numbered record lengths. They are exported from a database application.
_________________
(Steve Henley)

[JohnCampbell]

I have received this message "A valid record length must be specified if access is direct", if the file length is not a multiple of the "length=4"

Rather than first opening as direct access, you could try access=stream and then read the first 4 bytes.

Stream access is very useful for integogating unknown files. You can use this approach to read unformatted sequential records and interogate the Header|data|trailer structure of these files. You can even go to the extent of generating a table of records for record length and start address of each record or start address of the data in each record.

This then becomes a variable length random access file, which can be used as a more flexible database. You can then combine the complex I/O lists that generated the data with the file positioning using pos=.

Combining with /64, this works for large data records.

[silicondale]

Wow! Just wow, John! Something I didn't think of, because I have never used access=stream. This seems an extremely flexible option. It's effectively what I am doing in a rather clunky way, because the header information doesn't map to the standard record length but rather is mapped across the first few records of the file, and reassembled as required when reading the file. All the data records following the header are fixed length. However, what you suggest would give an upgrade path that could save a lot of space if I want to include variable-length text fields.

I'm now retired from paid work so (in principle) have plenty of time to experiment with this. Though some people knowing of my retirement have started volunteering me to do other unpaid things. Thus I've been co-opted on to an advisory group for a museum that is being relocated.
_________________
(Steve Henley)

[silicondale]

For some reason access=STREAM didn't work - but access=TRANSPARENT does the same job and I have now used this to recode the whole package. It uses self-describing files which are flat tables with a mix of numeric and text fields that don't need to be defined in advance.

A useful update of something that was originally designed in 1972-73. This was the G-EXEC system for the British Geological Survey, developed by a small team that I was a member of, led by Prof. Keth Jeffery, and based at the Atlas Computer Laboratory (on the Rutherford Lab site, Harwell). It went through a couple of transformations, first to DATAMINE for geological modelling and mine planning, and now for VMINE which I developed and have been using as a platform for development of experimental algorithms to process data from a couple of EU Horizon 2020 projects. When we did this in 1973 we just used REAL arrays to hold numeric or non-numeric data alike, relying on Fortran typing leniency, and IBM360 job control language to run every job as a two-step operation, the first to generate a program automatically, piecing together subroutine calls from a library, and the second to compile and run it!
_________________
(Steve Henley)

[PaulLaidler]

Steve

STREAM access should work if you are using FTN95 v8.80 or after. It is very similar to TRANSPARENT. STREAM is standard conforming, TRANSPARENT is not.

[silicondale]

Hi, Paul -
Thanks - I'll try again. Maybe I mistyped something. But I'm not too concerned about my code not being standard-conforming since it's mainly for my own use. Now 90% retired, and the remaining 10% is generating results from the data rather than supplying the software.
_________________
(Steve Henley)