forums.silverfrost.com

[DanRRight]

To run the programs in parallel this compiler has one neat trick related to NET which i'd like to use. Here are two small programs which illustrate the approach. One is straightforward, it uses separate subroutines for each thread and works fine, it gives you speedup almost proportional to amount of processors. And another, of more general form, simpler and smaller and where you can divide the workload on arbitrary amount of threads, and hence much more practical, by unknown reason is unstable.

What is in the program - just the DO loops doing some fake simulation 200M times. First do loop works on single processor and gives you estimation of usual single-threaded CPU time. Then in the first program, which works fine, i explicitly start 4 threads (my PC has 4-core processor, and though it can run 8 independent threads the speedup is not 8 times of course because of only 4 floating point units per CPU) each thread takes 1/4 of the DO loop cycles. In the second program, which is unstable, I use another DO loop which creates and starts 4 threads using one single subroutine where the workload is automatically divided on needed amount of threads (i took 4 by mentioned above reason).
While first program sustains any torture, but is not how parallel programs have to be written with 10, 100 or more processors, with the second I can not even get notification that first thread is started...

Please run both programs and tell me what you get. Any hints of making the second example stable are appreciated.

/* Code is almost 100% standard Fortran, except just one new command LOCK to avoid conflicts of threads which does not need explanation and calls to the couple FTN95 library functions. Also, after playing a bit with the code you can decrease first cycle 10 times so it will not annoy you.
By the way the example for just two threads is here http://forums.silverfrost.com/viewtopic.php?t=2176&postdays=0&postorder=asc&highlight=f2003&start=30

[DanRRight]

...continue...END OF THE FIRST PROGRAM

[DanRRight]

HERE IS SECOND PROGRAM

[davidb]

In your second program, you should pass the thread number as an argument to the subroutine runN? This is probably what is wrong.

If I can, I will post a working example of multi-threading with FTN95 in a few days; just a little busy at the moment.
_________________
Programmer in: Fortran 77/95/2003/2008, C, C++ (& OpenMP), java, Python, Perl

[DanRRight]

Thanks for the critical point. Yes, adding integer argument to RunN (m) can reveal the thread handle (the one which is = 20+iCurrThread), which could be used for thread counting.

When I added argument m to RunN and changed one line
ithr = iCurrThread

to
ithr = m-20

all started working OK. So any guess why iCurrThread passed via common was wrong? Is there a chance that anything passed via common could be wrong ? When we clear that and formulate some simple rules what's to do and what's not i think anyone will be able safely write parallel programs using FTN95

[davidb]

Data in common will probably be Shared amongst the threads, so it isn't possible for each thread to have different sets of common variables.

On the other hand, arguments passed as arguments, and local variables in the subroutine, will be stored on the stack. There should be a different stack for each thread.

However, this isn't the point. The subroutine runN requires an integer argument is passed to it. If you leave this out you will be potentially corrupting the call stack frame at each call and the behaviour you wil get will be unpredictable.

You may find it helpful to read up on the difference between shared and private variables in OpenMP as the distinction used there is what you need to be aiming for with your own codes using FTN95. For example, you will learn it is OK for each thread to read data in shared variables, but to write requires the use of locks.
_________________
Programmer in: Fortran 77/95/2003/2008, C, C++ (& OpenMP), java, Python, Perl

[DanRRight]

[davidb]

You cannot pass the thread number in COMMON. When you run subroutines using different threads, they run "concurrently" in PARALLEL, or at least overlap with each other in time. This is the whole point of using multiple threads on different processors or cores - to get some parts of your code running in parallel.

In your second code, each iteration of the loop sets up one thread and sets up iCurrThread (the thread number) in COMMON. The first iteration of your loop calls runN which enters an expensive DO Loop (i=1, 200000000). At the start of this loop, the value of iCurrThread in memory is correct. However, before this loop can complete, a second call to runN is made with a different thread. This second call changes the value of iCurrThread whilst the first thread is still running runN which leads to incorrect results. The problem is made worse when the third and fourth threads start up.

The only way to fix this is to pass the thread number as an argument to the subroutine you want to call. Each thread will then pass a different value on the stack and each call of runN can safely run in parallel.

Your code is a little bit unusual in the way you have partitioned the work between threads. I have made another Post showing how to do this in a more general way, which people may find useful.

I have run your modified code 2 and it does give me an improvement, but I only have 2 cores here. I would be interested to see what performance you are getting with 1 thread, 4 threads and 8 threads, where the last is using hyperthreading.
_________________
Programmer in: Fortran 77/95/2003/2008, C, C++ (& OpenMP), java, Python, Perl

[jalih]

[DanRRight]

Well, here is the same code #2 after simplification, beautification and syntactic sugar together with the runtime results. Total CPU time is elapsed time * amount of threads. Computer - i7 laptop, 2.2GHz, 4 cores, 8 threads set in BIOS, Windows7.

It can not be simpler and more self-explanatory than that. Code creates independent threads running subroutine runN (iThrHandle) using CALL CREATE_THREAD@(runN,20+i) where iThrHandle = 20+i is thread handle and follows the moment they finish when all flags kThreadEnded are =1. Exactly the same has been generally done by any other parallelization methods like OpenMP, CUDA etc when the threads could be independent. Add into subroutine RunN any specific task you want, Monte-Carlo, matrix operations, parallelizing DO loops etc partitioning them the way you want. Set when start the thread kThreadEnded(ithr) = 0 and make it =1 when finished. That's all. I set parameter nThrMax=8, use your own CPU cores/threads count. You may also consider adding or removing tiny delay when launching the threads (see the line call sleep1@(0.02) ) if "threads jam" (?) conflict takes place when amount of threads is very large (why? this is the question to FTN95 developers. This is second time i find necessity for tiny delay with NET, first time it was for Clearwin+ GUI Run-Pause-Stop code, it was posted here 3-4 years ago).

And the results as a function of # of threads are great, the speedup on 4 cores is like on almost 8-core CPU. Isn't this amazing?

1 Elapsed time, total CPU time= 15.6210 15.6210
2 Elapsed time, total CPU time= 7.94200 15.8840
4 Elapsed time, total CPU time= 4.07800 16.3120
5 Elapsed time, total CPU time= 3.39700 16.9850
6 Elapsed time, total CPU time= 2.98200 17.8920
8 Elapsed time, total CPU time= 2.49800 19.9840

[DanRRight]

Wondering why this functionality is available only in NET...Would be great if Silverfrost made it for regular x86 mode.

[PaulLaidler]

I have this as a high priority on my list of things to do.

[DanRRight]

THAT deserves BIG THANKS!

[PaulLaidler]

I have uploaded a new beta version of salflibc.dll to http://www.silverfrost.com/beta/salflibc.exe

This contains new Win32 thread functions illustrated in the following sample.
The new routines are described in the file cwplus.enh that is included in the download.

[PaulLaidler]

I have edited the above sample program and removed a call to ExitThread that was not needed because the wrapper function in the library makes this call.