forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Loading/linking unnecessary code

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support
View previous topic :: View next topic  
Author Message
wahorger



Joined: 13 Oct 2014
Posts: 480
Location: Morrison, CO, USA

PostPosted: Wed Nov 15, 2017 10:39 pm    Post subject: Loading/linking unnecessary code Reply with quote

This is an outgrowth of my previous post "SLIM vs SLINK".

I was going through the MAP file for one of the executables and came across a *LOT* of symbols that should not have been included in the executable. I've been poking around for several days trying to find out why, if I am using a library, so much extra code was loaded when it was never called for. I now have a clue.

If I happen to declare a named COMMON block that is also called out in other objects in the library, then those objects also appear to get loaded, then objects that are called by these objects, etc., etc.

I verified that by looking at the MAP file for a smaller piece of code. Compiles without errors, links fine, creates a very tiny executable that consumes about 1.8 MB at run time. Link Map: https://drive.google.com/file/d/13AdQS-x5i0qaBGYgAAHPVzAaKsndCD3h/view?usp=sharing

I then added the name of a COMMON block (CLOGCM) used by other modules in my library, and included one data item of 8 bytes in this common block. Other than this, there was no change to the code, and I never referenced the data item in the code. I then compiled and linked (got one warning about differences in COMMON block sizes, which was expected) and, since I did not change the SLINK step, I got undefined symbols. I now have en executable that, when run, now consumes 509 MB. Link Map: https://drive.google.com/file/d/1QiojnSa8HOx3PENklb7Qyztn4Ng_to4j/view?usp=sharing

Stated another way, the final link processing not only includes the linked objects that one requires, but also every module that those modules have in COMMON with all other modules, and as these modules have other COMMON, then those other objects get loaded, etc.

I don't think this is the way it should work.

V8.20.0


Last edited by wahorger on Thu Nov 16, 2017 2:59 pm; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
PaulLaidler
Site Admin


Joined: 21 Feb 2005
Posts: 5584
Location: Salford, UK

PostPosted: Thu Nov 16, 2017 9:07 am    Post subject: Reply with quote

Bill

This post appears to be truncated.
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 480
Location: Morrison, CO, USA

PostPosted: Thu Nov 16, 2017 3:00 pm    Post subject: Reply with quote

Bad sentence. Should have been removed before submitting.
Back to top
View user's profile Send private message Visit poster's website
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 1873
Location: Yateley, Hants, UK

PostPosted: Thu Nov 16, 2017 5:18 pm    Post subject: Reply with quote

One is reminded of The Siphonaptera":

Big fleas have little fleas,
Upon their backs to bite 'em,
And little fleas have lesser fleas,
and so, ad infinitum.

Noting that parodying this ditty has a long and distinguished pedigree, I offer you by way of consolation:

Big codes have little codes,
that link to take up RAM,
And little codes have lesser codes,
and thus enlarge your program.

I am a devotee of COMMON, but perhaps in your case, passing everything via subroutine arguments might perhaps work better.

Eddie
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 480
Location: Morrison, CO, USA

PostPosted: Thu Nov 16, 2017 11:33 pm    Post subject: Reply with quote

Eddie,

I like the poem.

Trouble is, as far as I can tell, if you even declare a COMMON block, you can get this to occur. Even if you mostly pass parameters by arguments. Try passing arguments to a WINIO@ callback? Can't do it...... Have to use some other mechanism.

You have a MAIN that calls a few routines. In your library are ALL of the routines you have written. Some of them have data in COMMON blocks. Really BIG COMMON blocks in some cases.

MAIN calls A. A is in a library that contains routines A, B, C, D, ... Some of those routines have references to COMMON blocks C1, C2, C3, C4, ... "A" references COMMON blocks C1 and C2, while "B: references C2 and C3, "C" references "C3 and C4, ...

In this example, A does all the processing required by MAIN. A contains COMMON blocks C1 and C2. When the library is searched, not only is "A" loaded, but "B" is loaded because it references C2, which loads "C", which loads "D", ...

So, all the library routines get loaded, and all the COMMON blocks get allocated. When all you needed was "A".

Not good.

In my case, COMMON is the only way to pass along data to callback functions that then deal with those data. These routines themselves need access to other data also in COMMON. Which brings me back to the cheesy example above. Everything can get loaded and allocated regardless of what actually is required to get the job done.

Again, not good, especially when one of the programs allocates data areas dynamically to do its job. There is over 500 MB of data area allocated what cannot be used by the program because of this particular issue.

I tried to build a SLINK step manually for one program that could most benefit from the space savings, but with over 500 routines, it becomes more than tedious to overcome. And, if something changes, or a different routine is needed, I have to manually update the SLINK process again. And not just for one program, but for several.
Back to top
View user's profile Send private message Visit poster's website
JohnCampbell



Joined: 16 Feb 2006
Posts: 1997
Location: Sydney

PostPosted: Fri Nov 17, 2017 2:46 am    Post subject: Reply with quote

Isn't this a problem with SLINK treating the COMMON symbol the same as a routine. In Fortran, if the routine is not called, the common reference will not be acted on, so the COMMON reference should not imply loading the routine ?
Is this different with C.. ?

Bill, I am still surprised by the size of your .exe. You must be using the equivalent of block data, which you should replace by an initialisation routine at run time. I thought removing block data was a necessity to use FTN95 /64.
FTN95 /64's approach for defining huge /64 COMMON at run time is a very good development.

John
Back to top
View user's profile Send private message
wahorger



Joined: 13 Oct 2014
Posts: 480
Location: Morrison, CO, USA

PostPosted: Fri Nov 17, 2017 3:11 am    Post subject: Reply with quote

John, it is some kind of issue, just not sure what it might be.
Back to top
View user's profile Send private message Visit poster's website
wahorger



Joined: 13 Oct 2014
Posts: 480
Location: Morrison, CO, USA

PostPosted: Fri Nov 17, 2017 4:47 am    Post subject: Reply with quote

No one has ever said I would just let a problem fester....

BLOCK DATA

That seems to be the key. I took the BLOCK DATA sub-programs out of the library, did the one line change that allocate half a gig of memory, and, lo and behold, no problems.

Since BLOCK DATA is a special kind of program unit, any reference to a BLOCK DATA that contains other COMMON blocks loads the ENTIRE sub-program.

What doesn't make sense (still) is the inclusion of ALL the executable sub-programs that reference those various COMMON blocks.

John, the common blocks contain data that is quite large. I use a lot of LISTVIEW controls to allow the user to select/sort on columns of data, etc. The raw data for one of these controls is 75 MB, and the LISTVIEW character array for this control is 50% larger. This is only one of several. I expect those programs to have a large static allocation. One of the support programs, however, does not reference those large common blocks, thus should be much smaller executables. Yet, it is not. Hence, the issue.

I'll continue to work on my solution. I need to separate BLOCK DATA into smaller chunks, for sure. It's a lot of work, but worth it.

And, John, to address the issue of run-time init versus BLOCK DATA. If one has a run-time init, then there are multiple INIT's to be created, depending on which COMMON's need to be initialized.

Some serious restructuring of the code is going to happen over the next few weeks!
Back to top
View user's profile Send private message Visit poster's website
mecej4



Joined: 31 Oct 2006
Posts: 1024

PostPosted: Fri Nov 17, 2017 6:45 am    Post subject: Reply with quote

BLOCK DATA is a troublemaker. Once, I broke up a large Fortran 77 program so that each source file had a single program unit. The BLOCK DATA was in its own file, but I forgot to add the corresponding OBJ file to the link line in the batch file used for building the program.

With or without that OBJ file used by the linker, an EXE was produced with no error messages. The EXE that was produced without the BLOCK DATA .OBJ file, of course, gave wrong results. Running it in a symbolic debugger was a bewildering and frustrating experience.

Fortunately for me, the BLOCK DATA provided the initialization for \pi, and I noticed that \pi did not have the value 3.14159... in the debugger's variables pane.
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 1873
Location: Yateley, Hants, UK

PostPosted: Fri Nov 17, 2017 4:33 pm    Post subject: Reply with quote

Mecej4,

My former colleague Les Hatton who at one time wrote extensively on Fortran suggested using named BLOCK DATA routines, and listing them in EXTERNAL statements to give the linker something to bite on. This was a misuse of EXTERNAL that nevertheless seemed to work.

Like Bill, I recognise that to communicate with a callback function one needs to do it (old style) with COMMON, which suits me fine. As COMMON is initialised to zero by FTN95, it is pointless doing explicit initialisation to this value in a Clearwin+ program, as that pretty much excludes using any other compiler (if one still uses 32 bit), and that must save on the EXE size.

Eddie
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> Support All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group