Memory mapped files

Please provide an example on following:

1. how to open a previously defined memory mapped file to read from it using READ statement

2. how to open a new memory mapped file and write to it using the WRITE statement

Thanks Abhishek

Have you looked at https://forums.silverfrost.com/Forum/Topic/1063

Sharing inter-process memory with FTN95 under Win32

Yes I have seen that article.

Only problem is that the article shows how to create a variable called msg from shared memory. I have multiple READ statements for sequential formatted reading and internal READ is not suitable for the same as it is not record oriented and is always positioned in the beginning.

I was hoping that if I could get an IO unit for memory mapped file like a normal file open unit, then the READ/WRITE statements can be used 'normally'

Is there a good reason for doing normal READ and WRITE on a memory mapped file?

Anyway there are routines like OpenFileMapping and MapViewOFFile that might possibly be used in this context but I have not thought out or explored the concept.

Thats because I am trying to make a dll out of existing code which reads from disk based files. The task can be accomplished with very little change in the original code if it is done this way.

Can you please provide a short example / pointer to such a use of OpenFileMapping and MapViewOFFile ?

Sorry but I do not have an example to hand. Given time I could explore and test the concept but it would be necessary to charge you for this service.

OK Please send me a proposal for the same.

Problem definition is as follows:

'Find the solution in a FORTRAN dll such that sequential formatted READ / WRITE statements can be performed over an IO unit which is based in RAM and not based on disk. '

Possible solutions -

1. string variable passed from VB .NET
2. Memory mapped file

Thanks

OK I will give that some thought.

Am I right in thinking that you will use formatted input and output. If not then the 'files' will have to be the result of FTN95 output.

The origin of problem is as follows:

I have some FORTRAN programs which are independent exes and they take their input via disk files and use formatted READ statements and they give their output to disk files via formatted WRITE statements.

I wish to make a dll out of them and call them from visual basic. Using disk files for I/O is making the whole operation slow; so i need to find a way to communicate via memory instead of disk.

So i have the input contents in visual basic string variables [over which sequential formatted READ statements need to be executed] and i need to obtain output finally as visual basic string variables [which is formed using sequential formatted WRITE statements].

I hope the problem and background are clear. Shall be happy to answer any further questions.

Abhishek,

How big are the disk files ? Unless these files are extremely large, I'd be looking for other causes of the slow performance. In a typical PC, disk I/O is at a rate of about 20mb to 30mb per second, less if there is sufficient memory for disk buffering. Alternatively, if the READ and WRITE statements are not too complex, you could try replacing them with a suite of memory buffer management routines and hold the data in memory. A WRITE_MB could be given a packet of information and length, then return a packet ID which is used to retrieve with a READ_MB routine. With minimal modification to the original code, and having a list of ID's in a MB management MODULE, this would work quite effectively ( up to the 1.7gb of available memory ) Subroutine Init_MB ! to initialise the memory buffer Subroutine Write_MB (Array, size_bytes, ID) ! returns ID Subroutine Read_MB (Array, size_bytes, ID) ! returns Array, using ID ID is typically the address in the memory buffer. I use Array as Integer or Real, as it is the address in memory of the array of informantion being stored. Not standard conforming but works very well.

John

The files are small and the performance drop is seen because the expected time < 1s. In such a short time, any disk I/O activity will hamper performance.

Writing and reading memory buffers is fine but how to take care of the formatted sequential READ and WRITE statements ?

Thanks Abhishek

Abhishek,

You should be able to estimate the elapsed time of the read statements, based on the size of the files. You have implied this is very short. However there may be other system delays in opening the files and initiating I/O. You should look to see the cause of these delays. It may be that these are due to using I/O and moving to memory buffers may remove this problem. Alternatively, the delays may be due to other factors, so changing the I/O may be a lot of unnecessary work.

As for formatted sequential READ and WRITE, I have effectively suggested unformatted Read/Write of a single array. You can emulate the read/write list, by first transfering the values to a single vector. You can handle mixed mode through a subroutine call.

It all depends on how important it is to not change the original programs vs providing a shared module of key array declarations then simplifying the information transfer via the module arrays. This often requires re-writing (or removal of) the data input phase of some of the programs.

Best of luck,

John

Now that I have time to think about this in some detail I am not clear how to proceed.

I can show you how to map an external file to memory and I can then attempt to READ from this file. But you say that the file is already in memory.

Given that I need to reproduce this on my machine,

1. How do you create an predefined image of a file in memory so that I can do the same. Does it have the same name in memory as the external file name?

2. What would your READ statement look like if it worked in the way you wanted. What 'file' name do you use in this context?

Well the 'file' in memory is actually a named share without a disk file backing. Such a file is created by calling Win32 function CreateFileMapping with INVALID_HANDLE_VALUE as the first parameter.

So I do this inside VB, copy the contents to be read by FORTRAN to this map and then call fortran function with the name of the Memory Mapped File.

The FORTRAN function should open this file and assign it a I/O Unit. (lets say 10)

The READ statement remains unaffected as it would have been for a 'normal' file. For example, READ(10,130) IDEN where 10 is the I/O unit assigned earlier 130 is a label identifier for FORMAT statement and IDEN is a variable.

This allows all the READ statements to remain as-it-is while the reading happens from memory instead of disk.

Can you post the VB code that creates the file mapping and also the details of the Fortran OPEN statement that you would like to work.

Well I guess I have found an alternative way to the problem.

In effect, what I was trying to do was the following:

1. VB creates memory mapped file (lets say named 'MYMAP')
2. VB calls FORTRAN dll
3. FORTRAN dll 'OPENS' that memory mapped file and somehow assigns it an I/O unit --> In Intel Visual Fortran, you would do that by using OPEN and USEROPEN functions. i dont know how it would be done in FTN95
4. You then use this I/O unit to execute READ statements as usual.

However, as it turns out, handle returned by OPENFILEMAPPING or CREATEFILEMAPPING is not the same kind of handle that can be used by OPEN method for subsequent READS

So I have instead passed file contents as string and I use a pointer to the current location in the string to emulate records. this pointer is increased after every subsequent READ and allows internal READ to read sequentially.

This Problem solved. for now.

Thanks