forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Formatted read with A format converts tabs to spaces

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
mecej4



Joined: 31 Oct 2006
Posts: 1885

PostPosted: Sat Jan 28, 2023 3:50 pm    Post subject: Formatted read with A format converts tabs to spaces Reply with quote

The following program demonstrates a bug in FTN95. When a text data file containing tab characters is read using the A format specifier, each tab is converted and expanded to a number of spaces. This bug is present in the latest FTN95, as well as the old FTN95 7.2 and FTN77 4.03.

Code:
      program afmtbug
      implicit none
      integer i
      character*20 iline
      character*50 jline
c
      iline = 'titleab 1' // char(9) // 'titleabc 2'
      open(10,file='x.txt',status='replace')
      write(10,'(A)')iline
      rewind(10)
      read(10,'(A)')jline  ! <<<=== BUG: input tabs are expanded to spaces
      close(10)
      do i=1,20 ! output enough characters to display expansion of tab to spaces
         print 10,i,ichar(iline(i:i)),iline(i:i),
     +      ichar(jline(i:i)),jline(i:i)
      end do
   10 format(1x,2i5,2x,A1,4x,i5,2x,A1)
      end

The output from Gfortran:
Code:
S:\LANG\FTN95>gfortran afmtbug.f

S:\LANG\FTN95>a
     1  116  t      116  t
     2  105  i      105  i
     3  116  t      116  t
     4  108  l      108  l
     5  101  e      101  e
     6   97  a       97  a
     7   98  b       98  b
     8   32          32
     9   49  1       49  1
    10    9             9
    11  116  t      116  t
    12  105  i      105  i
    13  116  t      116  t
    14  108  l      108  l
    15  101  e      101  e
    16   97  a       97  a
    17   98  b       98  b
    18   99  c       99  c
    19   32          32
    20   50  2       50  2

The output from FTN95:
Code:
     1  116  t      116  t
     2  105  i      105  i
     3  116  t      116  t
     4  108  l      108  l
     5  101  e      101  e
     6   97  a       97  a
     7   98  b       98  b
     8   32          32
     9   49  1       49  1
    10    9            32
    11  116  t       32
    12  105  i       32
    13  116  t       32
    14  108  l       32
    15  101  e       32
    16   97  a       32
    17   98  b      116  t
    18   99  c      105  i
    19   32         116  t
    20   50  2      108  l


A hex dump of the file x.txt written (same with FTN95 and other compilers):
Code:
S:\LANG\FTN95>xxd -g1 x.txt
00000000: 74 69 74 6c 65 61 62 20 31 09 74 69 74 6c 65 61  titleab 1.titlea
00000010: 62 63 20 32 0d 0a                                bc 2..


Note: This bug was discovered when investigating related problems in a recent thread: http://forums.silverfrost.com/viewtopic.php?p=33765 .
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Sun Jan 29, 2023 6:08 am    Post subject: Reply with quote

I don't think this is a bug, but a documented "Feature" !

In FTN95.chm > Fortran Input and Output > OPEN ; this describes what is happening.

Quote:
Reading tab characters
In a Fortran READ statement, by default, tab characters read from a file are converted to spaces. To avoid this conversion you should make a call to the subroutine READ_TABS@(unitno) immediately after the OPEN statement (unitno is the unit number of the stream).


I have no idea how many spaces are provided !
I think this applies to a FORM="FORMATTED" ( ie default )
Something I learnt many years ago when reading text files from other sources!

If you open the files as stream, I think this will not happen.

I think this demonstrates how to control this:
Code:
      program afmtbug
      implicit none
      integer i
      character*20 iline
      character*50 jline
c
      iline = 'titleab 1' // char(9) // 'titleabc 2'
      open(10,file='x.txt',status='replace')
      write(10,'(A)')iline
      rewind(10)
      read(10,'(A)')jline  ! <<<=== BUG: input tabs are expanded to spaces
      close(10)
      do i=1,20 ! output enough characters to display expansion of tab to spaces
         print 10,i,ichar(iline(i:i)),iline(i:i), 
     +      ichar(jline(i:i)),jline(i:i)
      end do

      jline = ' '
      open(10,file='x.txt',status='OLD')
      call READ_TABS@ (10)
      read(10,'(A)')jline  ! <<<=== BUG: input tabs are expanded to spaces
      close(10)
      do i=1,20 ! output enough characters to display expansion of tab to spaces
         print 10,i,ichar(iline(i:i)),iline(i:i),
     +      ichar(jline(i:i)),jline(i:i)
      end do

   10 format(1x,2i5,2x,A1,4x,i5,2x,A1)
      end


( no longer used to fixed format code in Plato !! )
Back to top
View user's profile Send private message
mecej4



Joined: 31 Oct 2006
Posts: 1885

PostPosted: Sun Jan 29, 2023 10:59 am    Post subject: Reply with quote

Thanks, John. The relevant online documentation page is https://silverfrost.com/ftn95-help/inout/open.aspx (scroll to the bottom). I did try to search the documentation with phrases such as "tabs", "convert", etc., but failed; it so happens that if you do not know the name READ_TABS@, you are unlikely to find it using the Search panel in the documentation. Even if you enter READ_TABS@ into the Search panel in the help file ftn95.chm, it returns only OPEN as the sole item to select for display.

Gratuitous conversion of tab characters in input text files to spaces is expected if those text files are Fortran source files. Such substitution is not expected, however, when the input data file is a formatted file (such as a CSV/TSV file) that is not being treated as a file containing Fortran source code. An undesirable consequence is that if I have a file filter program that works with Gfortran, Intel, Oracle Fortran, etc., and tab characters determine the treatment, the may not work with FTN95 unless I modify the source code to add calls to the non-standard subroutine READ_TABS@.


Last edited by mecej4 on Sun Jan 29, 2023 12:08 pm; edited 2 times in total
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Sun Jan 29, 2023 11:58 am    Post subject: Reply with quote

Fortran is very poor at handling delimiter characters.
FTN95 is good at handling comma delimiters for numeric data (by terminating the field length) but it's handling of tabs can insert between 0 to 8 space characters, which does not help at all.
For outbut, FTN95 business format is very useful, especially for large numbers, but this is a UK, but not EU feature.

I don't know what Gfortran and ifort do with comma delimiters for formatted reads, but apparently they let tabs go through when reading into character fields.
I am not aware of any Fortran standard approach, especially for those characters that are not in the Fortran character set.

I don't think there can be a standard conforming "file filter program" as most compilers break the standard differently, while some perform similarly.

Inserting "call READ_TABS@ (lu)" is a useful thing to remember.
Back to top
View user's profile Send private message
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 7916
Location: Salford, UK

PostPosted: Mon Jan 30, 2023 1:10 pm    Post subject: Reply with quote

Thankyou mecej4 for the feedback and John for the information.

I have made changes to the help information to make it easier to get to the relevant topic.
Back to top
View user's profile Send private message AIM Address
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group