False errors reported with Hollerith strings in Format

I ran into a compiler bug when I tried to use FTN95 7.10 on some old Fortran-IV code from Netlib (opt/Varpro). I had intended to use the nice /convert option that is provided in FTN95, but wanted to run the test case first with the original source code and use the output as a base-line.

I received many spurious error messages from Format statements that contain Hollerith string items. Here is an example:

      program xHollerith
      implicit none
      integer N, L, NL, P, IV
      N  = 33
      L  =  3
      NL =  2
      P  =  2
      IV =  1
      WRITE (*, 101) N, L, NL, P, IV
C        1         2         3         4         5         6         7     
C23456789012345678901234567890123456789012345678901234567890123456789012
  101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM //
     1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR PARAMETE
     2RS = , I4  // 33H NUMBER OF NONLINEAR PARAMETERS =, I4,
     3 45H NUMBER OF NONVANISHING PARTIAL DERIVATIVES =, I4 //
     4 34H NUMBER OF INDEPENDENT VARIABLES =, I4)
      END

The source file does not contain any tab (ASCII 9) characters, nor are there any incorrect character counts. Attempting to compile this program gives

[FTN95/Win32 Ver. 7.10.0 Copyright (c) Silverfrost Ltd 1993-2014]
0010)   101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM //
0011)      1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR PARAMETE
0012)      2RS = , I4  // 33H NUMBER OF NONLINEAR PARAMETERS =, I4,
*** Illegal character combination, '=' followed by ','
0010)   101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM //
0011)      1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR PARAMETE
0012)      2RS = , I4  // 33H NUMBER OF NONLINEAR PARAMETERS =, I4,
0013)      3 45H NUMBER OF NONVANISHING PARTIAL DERIVATIVES =, I4 //
0014)      4 34H NUMBER OF INDEPENDENT VARIABLES =, I4)
*** Unknown edit descriptor 'R', or missing comma
    2 ERRORS  [<XHOLLERITH> FTN95/Win32 v7.10.0]
*** Compilation failed

If I use /convert, however, FTN95 produces transformed source code which a subsequent compilation handles without errors.

[P.S., 15 June 2015: This bug is also present in FTN95-7.20]

Quoted from John-Silver ...I'd also suggest that, while not illegal, it's bad coding practice to overlap anything like this onto continuation lines split like this, if nothing else but for readability !

This is code that is nearly 40 years old, and the author is dead, please see http://www.legacy.com/obituaries/sfgate/obituary.aspx?pid=166064331 .

A couple of sentences to provide context. There is a Fortran 90 version at http://wp.csiro.au/alanmiller/varpro/varpro.f90 , and the original program is about 1000 lines. The old program will not run correctly on modern computers unless an option such as /SAVE is used, and I aimed to identify just those variables that need to be saved, being reluctant to use heavy-handed options such as /SAVE. In these circumstances, I want the original program to run with few changes, if any, and I use the results to help with my search for variables that must be SAVEd. For such a search, FTN95 is a good choice because of its excellent diagnostic and debugging capabilities. However, it must also compile and run old code, regardless of style, as long as the code qualifies as valid input to the compiler, because the output is to be used as the reference when running modified versions of the old code.

Given the history outlined, I agree with John S that allowing Hollerith fields to overflow lines has always been a problem area. The example provided does appear to be a bug in FTN95.

With the advantage of a screen editor, the layout can be easily improved. Even in 1975, we did have printouts and I think Format 101 should have been better presented. Cleaning it up should still keep the aim of your project. I frequently apply these changes with historical code, such as replacing 25H NUMBER OF OBSERVATIONS =, I5 with ' NUMBER OF OBSERVATIONS =', I5 Unfortunately Alan Miller's web page address is no longer supported at CSIRO.

      program xHollerith 
      implicit none 
      integer N, L, NL, P, IV 
      N  = 33 
      L  =  3 
      NL =  2 
      P  =  2 
      IV =  1 
      WRITE (*, 101) N, L, NL, P, IV 
C        1         2         3         4         5         6         7      
C23456789012345678901234567890123456789012345678901234567890123456789012 
  102 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM // 
     1 25H NUMBER OF OBSERVATIONS =, I5, 
     2 32H   NUMBER OF LINEAR PARAMETERS = , I4  // 
     * 33H NUMBER OF NONLINEAR PARAMETERS =, I4, 
     3 45H NUMBER OF NONVANISHING PARTIAL DERIVATIVES =, I4 // 
     4 34H NUMBER OF INDEPENDENT VARIABLES =, I4) 
  101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM // 
     1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR PARAMETE 
     2RS = , I4  // 33H NUMBER OF NONLINEAR PARAMETERS =, I4, 
     3 45H NUMBER OF NONVANISHING PARTIAL DERIVATIVES =, I4 // 
     4 34H NUMBER OF INDEPENDENT VARIABLES =, I4) 
      END

Thanks for the feedback. I have logged this for investigation.

Quoted from JohnCampbell ... Unfortunately Alan Miller's web page address is no longer supported at CSIRO.

I forgot to add a space after the URL before the comma. Please try the link again: http://wp.csiro.au/alanmiller/index.html . Miller's pages, hosted at at http://users.bigpond.net.au/amiller/ , became unavailable for a couple of years after he became too sick to maintain the pages. Others hosted copies/snapshots, for example, http://jblevins.org/mirror/amiller/ . CSIRO restored the pages a few years ago, and they continue to be available.

Quoted from PaulLaidler Thanks for the feedback. I have logged this for investigation.

Thanks, Paul. As the other responders hinted, the bug appears to be caused by the presence of a Hollerith string that extends to column-72 and is then continued on the next line. The compiler attempts to interpret the first character after the continuation mark as a format edit descriptor. To demonstrate this, I have abbreviated the reproducer to

      program xHollerith
      implicit none
      integer N, L
      N  = 33
      L  =  3
      WRITE (*, 101) N, L
C        1         2         3         4         5         6         7     
C23456789012345678901234567890123456789012345678901234567890123456789012
  101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM //
     1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR MXYZPTLK
     2UV * , I4)
      END

Given this code, the compiler says

0009)   101 FORMAT (36H1   NON-LINEAR LEAST SQUARES PROBLEM //
0010)      1 25H NUMBER OF OBSERVATIONS =, I5, 32H   NUMBER OF LINEAR MXYZPTLK
0011)      2UV * , I4)
*** Unknown edit descriptor 'U', or missing comma
    1 ERROR  [<XHOLLERITH> FTN95/Win32 v7.10.0]
*** Compilation failed

I remember the dreadful business of writing Holleriths that spanned more than one card, and of using subset Fortrans that only permitted a few continuations.

Laying out the Formats sensibly seems to be something that assists in understanding the code, and does not interfere with the logic of what was written originally - and therefore must be permissible while still keeping 'originality'.

The regrettable demise of the original author does not shield him from the criticism that it wasn't stylish source code to begin with. When I see things like that that offend the eye and one's sense of what is right, my first reaction is to wonder aloud if it ever produced meaningful outputs!

Once again, however, you point out a regression from FTN77, and valid source code should compile correctly, no matter how cranky it looks.

As to /SAVE being 'heavy handed', I think that it is more a question of enforcing static allocation of variables, and thus is 'as it was in the beginning'. Isn't the half-way house of some variables being SAVEd (and therefore, some not) actually the compromise that will lead to problems, say, in another 40 years?

Eddie

Quoted from LitusSaxonicum As to /SAVE being 'heavy handed', I think that it is more a question of enforcing static allocation of variables, and thus is 'as it was in the beginning'. Isn't the half-way house of some variables being SAVEd (and therefore, some not) actually the compromise that will lead to problems, say, in another 40 years?

Eddie, on most computers of the 1960s and 1970s, SAVE applied, and the user had no other choice. Often, the variables were initialized to zero, in addition to being saved. Most algorithms, however, require only a small subset of the variables to be saved. Therefore, uninitialized variables simply remained hidden until their existence was suspected because of incorrect results being noticed, and the errors were then laboriously tracked down.

In VARPRO.f, only twelve variables need to be saved, and they all occur in just one of the eleven subprograms that the package contains. Declaring those variables with the attribute SAVE is exactly what I would do were I to rewrite the subroutine from scratch, assuming that I knew every step of the algorithm. All other variables are either subprogram arguments, or else local (automatic) variables whose values need not be retained at subprogram exit. I do not see why there should be any problems with future extensions of the code because some variables were declared with the SAVE attribute.

There is a penalty for using SAVE when it is not required. The memory footprint of the program increases. The program code size increases, since all addresses need to be full addresses, instead of smaller offsets relative to ESP or EBP (or another permitted register). And, as I noted earlier, lapses in variable initialization are harder to detect and rectify.

The memory footprint of 40 year old code increasing even rather dramatically ought not to bring shivers of fear to anyone's spine, should it? I remember being (almost) perfectly happy back in 1975 with 32k 24-bit words (96k bytes) and deliriously happy with 64k 60-bit words - around half a megabyte! So assuming that I used the maximum memory available, I might start to get worried if it got several thousand times bigger ...

I went back to FTN95.CHM to remind myself what /SAVE does (and indeed, what is and isn't done without it). It struck me that /SAVE does things that we were warned against doing by McCracken not just 40 years ago - but 50! I'll have to mull over whether or not I like the idea of all local variables being on the stack.

I remember SAVE fixing a problem with Clearwin+, but I can't remember what ... perhaps it was a stack issue.

Eddie

If the variable attached to %lw is local to a subprogram then it should be SAVEd.

Paul,

My memory is poor on this, but is slightly jogged by reading my source code.

I have divided some of my WINIO@ calls over several subroutines, partly to limit the length of some of them, but also in particular to allow the user to select alternate toolbar layouts - the ones using %tb being a lot nicer than those using %ib, but giving me problems with spacing for non-standard dpi settings. I suspect that the problem lies in the loss of the variable used to store the WINIO@ return code when the subroutine exits, and this is why they needed SAVE. Perhaps in my style of programming the return code should be in a COMMON block - I've always put handles that I needed in COMMON blocks.

Elsewhere in my program(s) I have divided WINIO@ calls across subroutines the windows/dialogs don't have toolbars - they certainly have buttons - but have not needed SAVE because what I have put in a subroutine call is simply the stylistic information such as no_border, %bg colour etc, for which the WINIO@ return code is presumably immaterial.

Your observations on WINIO@ return codes - not what they are, but how they are transferred down the chain as it were, would be valuable.

Eddie

The value returned by a call to winio@ should not cause a problem. It is just stored in a register and then copied to the related variable which can not be out of scope.

If there is a small program that demonstrates the problem then I can take a look at it.

Hi Paul,

The problem is one of understanding, and that is improved by your 'which can not be out of scope', thus explaining why SAVE helped in the particular context.

Eddie

This Hollerith string problem has been investigated but would require major recoding. At the moment we will have to accept this as a limitation.

The original Hollerith string failure was eventullly fixed for v8.30.