Loading/linking unnecessary code

This is an outgrowth of my previous post 'SLIM vs SLINK'.

I was going through the MAP file for one of the executables and came across a LOT of symbols that should not have been included in the executable. I've been poking around for several days trying to find out why, if I am using a library, so much extra code was loaded when it was never called for. I now have a clue.

If I happen to declare a named COMMON block that is also called out in other objects in the library, then those objects also appear to get loaded, then objects that are called by these objects, etc., etc.

I verified that by looking at the MAP file for a smaller piece of code. Compiles without errors, links fine, creates a very tiny executable that consumes about 1.8 MB at run time. Link Map: https://drive.google.com/file/d/13AdQS-x5i0qaBGYgAAHPVzAaKsndCD3h/view?usp=sharing

I then added the name of a COMMON block (CLOGCM) used by other modules in my library, and included one data item of 8 bytes in this common block. Other than this, there was no change to the code, and I never referenced the data item in the code. I then compiled and linked (got one warning about differences in COMMON block sizes, which was expected) and, since I did not change the SLINK step, I got undefined symbols. I now have en executable that, when run, now consumes 509 MB. Link Map: https://drive.google.com/file/d/1QiojnSa8HOx3PENklb7Qyztn4Ng_to4j/view?usp=sharing

Stated another way, the final link processing not only includes the linked objects that one requires, but also every module that those modules have in COMMON with all other modules, and as these modules have other COMMON, then those other objects get loaded, etc.

I don't think this is the way it should work.

V8.20.0

Bill

This post appears to be truncated.

Bad sentence. Should have been removed before submitting.

One is reminded of The Siphonaptera':

Big fleas have little fleas, Upon their backs to bite 'em, And little fleas have lesser fleas, and so, ad infinitum.

Noting that parodying this ditty has a long and distinguished pedigree, I offer you by way of consolation:

Big codes have little codes, that link to take up RAM, And little codes have lesser codes, and thus enlarge your program.

I am a devotee of COMMON, but perhaps in your case, passing everything via subroutine arguments might perhaps work better.

Eddie

Eddie,

I like the poem.

Trouble is, as far as I can tell, if you even declare a COMMON block, you can get this to occur. Even if you mostly pass parameters by arguments. Try passing arguments to a WINIO@ callback? Can't do it...... Have to use some other mechanism.

You have a MAIN that calls a few routines. In your library are ALL of the routines you have written. Some of them have data in COMMON blocks. Really BIG COMMON blocks in some cases.

MAIN calls A. A is in a library that contains routines A, B, C, D, ... Some of those routines have references to COMMON blocks C1, C2, C3, C4, ... 'A' references COMMON blocks C1 and C2, while 'B: references C2 and C3, 'C' references 'C3 and C4, ...

In this example, A does all the processing required by MAIN. A contains COMMON blocks C1 and C2. When the library is searched, not only is 'A' loaded, but 'B' is loaded because it references C2, which loads 'C', which loads 'D', ...

So, all the library routines get loaded, and all the COMMON blocks get allocated. When all you needed was 'A'.

Not good.

In my case, COMMON is the only way to pass along data to callback functions that then deal with those data. These routines themselves need access to other data also in COMMON. Which brings me back to the cheesy example above. Everything can get loaded and allocated regardless of what actually is required to get the job done.

Again, not good, especially when one of the programs allocates data areas dynamically to do its job. There is over 500 MB of data area allocated what cannot be used by the program because of this particular issue.

I tried to build a SLINK step manually for one program that could most benefit from the space savings, but with over 500 routines, it becomes more than tedious to overcome. And, if something changes, or a different routine is needed, I have to manually update the SLINK process again. And not just for one program, but for several.

Isn't this a problem with SLINK treating the COMMON symbol the same as a routine. In Fortran, if the routine is not called, the common reference will not be acted on, so the COMMON reference should not imply loading the routine ? Is this different with C.. ?

Bill, I am still surprised by the size of your .exe. You must be using the equivalent of block data, which you should replace by an initialisation routine at run time. I thought removing block data was a necessity to use FTN95 /64. FTN95 /64's approach for defining huge /64 COMMON at run time is a very good development.

John

John, it is some kind of issue, just not sure what it might be.

No one has ever said I would just let a problem fester....

BLOCK DATA

That seems to be the key. I took the BLOCK DATA sub-programs out of the library, did the one line change that allocate half a gig of memory, and, lo and behold, no problems.

Since BLOCK DATA is a special kind of program unit, any reference to a BLOCK DATA that contains other COMMON blocks loads the ENTIRE sub-program.

What doesn't make sense (still) is the inclusion of ALL the executable sub-programs that reference those various COMMON blocks.

John, the common blocks contain data that is quite large. I use a lot of LISTVIEW controls to allow the user to select/sort on columns of data, etc. The raw data for one of these controls is 75 MB, and the LISTVIEW character array for this control is 50% larger. This is only one of several. I expect those programs to have a large static allocation. One of the support programs, however, does not reference those large common blocks, thus should be much smaller executables. Yet, it is not. Hence, the issue.

I'll continue to work on my solution. I need to separate BLOCK DATA into smaller chunks, for sure. It's a lot of work, but worth it.

And, John, to address the issue of run-time init versus BLOCK DATA. If one has a run-time init, then there are multiple INIT's to be created, depending on which COMMON's need to be initialized.

Some serious restructuring of the code is going to happen over the next few weeks!

BLOCK DATA is a troublemaker. Once, I broke up a large Fortran 77 program so that each source file had a single program unit. The BLOCK DATA was in its own file, but I forgot to add the corresponding OBJ file to the link line in the batch file used for building the program.

With or without that OBJ file used by the linker, an EXE was produced with no error messages. The EXE that was produced without the BLOCK DATA .OBJ file, of course, gave wrong results. Running it in a symbolic debugger was a bewildering and frustrating experience.

Fortunately for me, the BLOCK DATA provided the initialization for \pi, and I noticed that \pi did not have the value 3.14159... in the debugger's variables pane.

Mecej4,

My former colleague Les Hatton who at one time wrote extensively on Fortran suggested using named BLOCK DATA routines, and listing them in EXTERNAL statements to give the linker something to bite on. This was a misuse of EXTERNAL that nevertheless seemed to work.

Like Bill, I recognise that to communicate with a callback function one needs to do it (old style) with COMMON, which suits me fine. As COMMON is initialised to zero by FTN95, it is pointless doing explicit initialisation to this value in a Clearwin+ program, as that pretty much excludes using any other compiler (if one still uses 32 bit), and that must save on the EXE size.

Eddie