forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Poly .... Wake Up Poly .... is this an Ex-Benchmark ?

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
John-Silver



Joined: 30 Jul 2013
Posts: 1226
Location: Aerospace Valley

PostPosted: Thu Sep 26, 2019 9:28 am    Post subject: Poly .... Wake Up Poly .... is this an Ex-Benchmark ? Reply with quote

Poly .... Wake Up Poly .... is this an Ex-Benchmark ?
... bereft of Life ?
... gone to meet it's maker ?

The polyhedron publishedbenchmark results are getting quite an airing again on several posts recently, for obvious reasons.

After having a quick shufty at them , apart from the obvious derogatory impression givn to ftn95 because of the timings (which if I would also get sorted pdq - most people will just look at those and dump even the thought of using ftn95 without a second thought as to if it matters to them) I had a quick look at the older results and I see that the results for the latest runs are in general for ALL compilers WORSE than previous results with a newer 'more powerful' (?) (5 core) machine !!!!
Conclusion - don't buy a newer more 'powerful' machine ? !



The specs ofthe 2 machines are isted below, note also the newer macho'ine has 4 times as much memory !!!

I believe that while it's obviously desíreable to have these benchmarks run with the best machines/configurations out there so as to get the best 'absolute speed' results, it's not the be all and end all.
They should also be run with a wide ranging set of 'real life, rel user' machines too.
How many pople out there have 16Gb memory let alone 64Gb !!!???
There should be a bog-standard 'commercial' config machine (at least 3-5 years old too!) at the bottom of the scale , and several inbetweenies too.
Otherwise the list ans tests become somewhat 'elitist'.

At the end of the day most (90%) of fortran users don't give a flying ferret for the results of benchmarks, don't even look at them.
But if they do it's cursory when they are new to Fortran and will just glance at that table and immediately rule out using ftn95 without another thought !!
I think this is what Dan's trying to say.

I'm surprised that SF have not got seriously involved in creating their own benchmarks to show off the best of the compiler's wares.


* STOP PRESS * -
So, hands up if you bought a 'newer' & 'better' Xeon processor !!!!
https://cpu.userbenchmark.com/Compare/Intel-Xeon-E5-2643-0-vs-Intel-Core-i5-2500K/m3389vs619
This is obviously why the benchmarks used x2 of them !!! and 4 times as much memory !!!

It also drags another parameter into any multi-user benchmarking attempt.

Maybe Intel/AMD ought to get dragged into this conundrum !?
____________________________

'Latest' Polyhedron results are with:-

2 x Xeon E5-2643 3.30GHz quad core processors, running at stock speed with hyperthreading disabled, 64 GBytes memory, and running 64-bit Windows 7 64-bit.

2014 Polyhedron results are with:-

Core i5 2500k 3.30GHz processor, running at stock speed, with 16 GBytes memory, and running Windows 7 64-bit
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1226
Location: Aerospace Valley

PostPosted: Thu Sep 26, 2019 9:31 am    Post subject: Reply with quote

Here's a comparison table I knocked together ..... some food for thought

Maybe not unsurprisingly the 'top f́do' in the speed stakes (Intel) show some improvements between latest and previous tests, but their 'evolution' is somewhat haphazard with other test cases they have greater losses in speed.
Is this because they're able to tune their compiler behaviour better to their hardware performance in certain domains of application?

The haphazardness is illustrated by the fact that even though it gains speed on some, the Intel compiler loses out on several tests by between 10 & 20% between the latest results and the previous 2015 results !!!

What this table demonstrates though is what I expresed above in general erms, that it's not worth paying out the 800 € spondoolicks, TWICE remember !, to get worse performance, whatever compiler you're using !


_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2049
Location: Yateley, Hants, UK

PostPosted: Thu Sep 26, 2019 5:26 pm    Post subject: Reply with quote

There's something wrong with the Polly Benchmarks, that's for sure. When I looked at the source codes some years ago I decided that the effort to understand them was a waste of time as they were not what I would hold up as good examples of coding style.

I have a feeling that the Intel compiler makes a better job of using multiple cores than others, like FTN95, do, and a more realistic (for most users) measure would be the performance on a less highly-specified machine (as J-S suggests). Also, the Polly Benchmarks don't use the latest FTN95, nor do they use 64-bit FTN95, which might have a bearing. Frankly, I'd like to see a comparison of 32-bit, 64-bit and .NET flavours of FTN95, and also with and without /OPT and so on, so I know which options to choose rather than comparisons with other compilers.

Mind you, I'm impressed by Polly, because they can get the Absoft compiler to run, which I couldn't.

The execution speed of the Intel compiler on the worst benchmarks where it is around 10x faster than FTN95 is immaterial, as it doesn't have Clearwin+, and is therefore rather useless! Also, it wouldn't make any perceptible difference to my users if the software were 10 or a 100 times faster.

Eddie
Back to top
View user's profile Send private message
DanRRight



Joined: 10 Mar 2008
Posts: 2102
Location: South Pole, Antarctica

PostPosted: Fri Sep 27, 2019 2:33 am    Post subject: Reply with quote

In 90th this compiler was light years ahead of everyone. Then dark times and recessions came when no one interested in engineering, science and technology. Now healing of the king definitely needed.
https://www.youtube.com/watch?v=Y6wE2W3ag1g
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2143
Location: Sydney

PostPosted: Fri Sep 27, 2019 3:15 am    Post subject: Reply with quote

There are 4 main areas where FTN95 is behind iFort ( I started at 2!)
1) optimisation of inner loops
2) implementation of vector instructions
3) recognition of bad coding
4) using array sections, especially as arguments
5) OpenMP support

It is the 3rd that is interesting, as there are a few examples in the Polyhedron benchmarks where bad coding is common. Some compilers have been tuned to recognising these poor constructs, also called optimisation. ( eg real**2.0 )
( my posts from a few years ago have more details )

There are 2 main areas where FTN95 is ahead of iFort
1) very good diagnostics.
2) clearwin

What this says is that if you can eliminate some bad coding constructs, the relative performance of FTN95 vs iFort (or gFortran) is not as bad as reported.

mecej4 should be thanked for helping to make FTN95 more reliable for /opt which helps FTN95 to address most of the points above.
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2049
Location: Yateley, Hants, UK

PostPosted: Fri Sep 27, 2019 10:53 am    Post subject: Reply with quote

John,

Was the miscounting a reference to further Monty Python sketches - ('No one expects the Spanish Inquisition if you need reminding)?

The use of vector instructions comes are a risk of increased roundoff, and at a guess, worse performance with transcendental functions
.
I think it's your 5. that gives the huge differences in the run times, and that would be far less apparent with fewer cores. I hesitate to adduce the appeal of filthy lucre, but perhaps Polly makes more from a sale of iFort than FTN95, and hence their choice of machine to run the benchmarks on? It would, I think, be a closer run thing on a single or dual core machine.

Eddie
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2143
Location: Sydney

PostPosted: Fri Sep 27, 2019 12:03 pm    Post subject: Reply with quote

Eddie,

My counting 4 and listing 5 was my sense of humour ! There are always more for the list.

64-bit does not support 80-bit reals, so that "increased roundoff" has been with us for a while.

Regarding the choice of processor, the Polyhedron benchmark being quoted is a single thread, 32-bit test, so I am not sure why they are now using 2 x Xeon E5-2643 and 64GB memory. I am definitely "not a fan" of this class of processor. I find the i7 to be much better for computation and my i7-8700K definitely gives good performance for the analysis I am now performing.
Probably most of the referring to slower recent performance could be due to the 3.3 GHz Xeon change. Based on my coding approaches, they really are not suited to a single thread, 32-bit test, or to numerical computation in general. (perhaps I don't know how to use them effectively, but I was forced to use Xeon for a number of years, but no more.)
Back to top
View user's profile Send private message
LitusSaxonicum



Joined: 23 Aug 2005
Posts: 2049
Location: Yateley, Hants, UK

PostPosted: Fri Sep 27, 2019 4:22 pm    Post subject: Reply with quote

Then perhaps you don't know the Spanish Inquisition sketch: https://www.dailymotion.com/video/xsefeg

Eddie
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1226
Location: Aerospace Valley

PostPosted: Fri Oct 04, 2019 1:34 am    Post subject: Reply with quote

... or ... Antioch style
_________________
''Computers (HAL and MARVIN excepted) are incredibly rigid. They question nothing. Especially input data.Human beings are incredibly trusting of computers and don't check input data. Together cocking up even the simplest calculation ... Smile "
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group