|
forums.silverfrost.com Welcome to the Silverfrost forums
|
View previous topic :: View next topic |
Author |
Message |
Notquitenewton
Joined: 25 May 2021 Posts: 20 Location: England, UK
|
Posted: Wed May 26, 2021 8:01 am Post subject: Text files and clipboard |
|
|
I wonder if anyone can help...
The problem...I have a large text file which I would like to place in the clipboard (to use elsewhere) using FTN95. The file can contain up to 1,000,000 lines and up to 100 columns (can be as big as 60 MB). I've knocked up a piece of code to do this but it is slow. Methinks it's due to all the Character string manipulations and concatenation! As an example, a text file containing 24,000 rows and 12 columns (all numbers) took about 40 seconds on my PC (i7 processor). I've concluded there must be a quicker way of doing this. The code (a bit on the rough side) is shown and it works, albeit take a coffee break waiting for it to complete!! The text file numbers are TAB separated. Any help most welcome. Thanks.
From Notquitenewton.
IMPLICIT NONE
INCLUDE <windows.ins> , nolist
CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: str1, buff
INTEGER :: j, i, k
C Open text file (tab separated numbers) with up to 100 cols.
C and 1,000,000 rows. The file tested has 12 columns and 24,000
C rows. This routine took about 40 seconds (i7 processor!!) to
C concatenate all rows into 1 string (this took the time) before
C copying to the clipboard. Compiler Siverfrost FTN95 V8.70.
OPEN(FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)
str1 = " "
50 READ(45,'(A)',END=100)str
j = len_trim(str)
i = len_trim(str1)
buff = str1(1:i)//str(1:j)
j = j + i
C Add CR/LF to end of each line
str1 = buff(1:j)//char(13)//char(10)
GOTO 50
100 j = len_trim(str1)
buff = str1(1:j)//char(0)
j = j+1
CLOSE(UNIT=45)
C Send string to clipboard and get size.
i = copy_to_clipboard@(buff(1:j),j,cf_text)
j = sizeof_clipboard_text@()
k = WINIO@('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2560 Location: Sydney
|
Posted: Wed May 26, 2021 10:57 am Post subject: |
|
|
I suspect most of the time is taken in assembling the buffer.
This could be easily be verified by reporting the size of the file read at statement 100
I have not tested your program, but the following changes minimise refreshing very long character buffers str1 and buff.
Perhaps you could test this adaptation. Code: | IMPLICIT NONE
INCLUDE <windows.ins> , nolist
CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: buff
INTEGER :: j, i, k, next, nl
! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.
OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)
! buff = " "
!
next = 1
nl = 0
do
READ (45,'(A)',END=100) str
j = len_trim (str)
! Add CR/LF to end of each line
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nl = nl+1
end do
100 buff(next:next) = char(0)
CLOSE (UNIT=45)
write (*,*) nl,' line recovered;',next,' characters'
! Send string to clipboard and get size.
i = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@ ('%ca[Run complete]Finished! Size %rd%nl%nl%cn%5BT[OK]',j)
END |
|
|
Back to top |
|
|
Notquitenewton
Joined: 25 May 2021 Posts: 20 Location: England, UK
|
Posted: Wed May 26, 2021 11:28 am Post subject: |
|
|
John,
I'll take a close look at your mods for the code. Except for one omission it worked. Thank you so much. The time went from 45 seconds to less than 0.5 second. Cor blimey!!!
Neat piece of coding now thanks to you. The size of the buffer by the way was 4,536,189
A rather large string methinks. This routine can now transfer a text file straight into the clipboard from FTN95 Fortran if it contains numbers and/or indeed normal text. Hope other users find it useful. After running, I simply pasted the clipboard into various other programs and even Notepad...worked a treat!
I've attached the final code below for information...
The few changes that you made were perfect and the speed is now outrageous!
Best wishes, Notquitenewton (Graham).
IMPLICIT NONE
INCLUDE <windows.ins> , nolist
CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: buff, str1
INTEGER :: j, i, k, next, nl
! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.
OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)
! buff = " "
!
next = 1
nl = 0
do
READ (45,'(A)',END=100) str
j = len_trim (str)
! Add CR/LF to end of each line
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nl = nl+1
end do
100 j = len_trim(str1)
buff(next:next) = char(0)
CLOSE (UNIT=45)
! Send string to clipboard and get size.
i = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2560 Location: Sydney
|
Posted: Wed May 26, 2021 11:33 am Post subject: |
|
|
Note I changed the line "100 j = len_trim(str1)" which should have been deleted, as str1 is no longer used. |
|
Back to top |
|
|
Notquitenewton
Joined: 25 May 2021 Posts: 20 Location: England, UK
|
Posted: Wed May 26, 2021 11:42 am Post subject: |
|
|
Final below...thanks again. The location of the text file can be changed depending on where the user has it stored!
Brilliant 10/10.
Best
Notquitenewton
Graham
INCLUDE <windows.ins> , nolist
CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: buff
INTEGER :: j, i, k, next, nl
! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.
OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)
! buff = " "
!
next = 1
nl = 0
do
READ (45,'(A)',END=100) str
j = len_trim (str)
! Add CR/LF to end of each line
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nl = nl+1
end do
100 buff(next:next) = char(0)
CLOSE (UNIT=45)
! Send string to clipboard and get size.
i = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END |
|
Back to top |
|
|
JohnCampbell
Joined: 16 Feb 2006 Posts: 2560 Location: Sydney
|
Posted: Wed May 26, 2021 1:15 pm Post subject: |
|
|
I would retain the report of lines read and also a report of maximum line length read, to know what you have recovered.
Reducing str len= would improve performance.
Line lengths of 5000 characters can be difficult to deal with. (not sure if there is a limit in clipboard) |
|
Back to top |
|
|
Notquitenewton
Joined: 25 May 2021 Posts: 20 Location: England, UK
|
Posted: Wed May 26, 2021 4:07 pm Post subject: |
|
|
John,
I tried out a few text files and just to let you know that my last test was using a text file with the longest line length of 325, with 1.1 million lines and 402 million numbers/characters. I ran the below code in FTN95 and it copied the whole lot to the clipboard in about 2 seconds. I am very pleased indeed with your suggestions about tweaking the strings. Now I can add this to my programs!! Also, note the increase in size of Character buff. Some of my text files are monsters!
Best
Notquitenewton
Graham
IMPLICIT NONE
INCLUDE <windows.ins> , nolist
CHARACTER (len=4000) :: str
CHARACTER (len=1000000050) :: buff
INTEGER :: j, k, next, nline
! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. Compiler Siverfrost FTN95 V8.70. Change the text file location from
! XXXXXXX to your own choice.
OPEN (FILE='C:\XXXXXXX\supout.txt',UNIT=45)
next = 1
nline = 0
! Add CR/LF to end of each line
DO
READ (45,'(A)',END=100) str
j = len_trim (str)
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nline = nline + 1
ENDDO
100 buff(next:next) = char(0)
CLOSE (UNIT=45)
! Send string to clipboard and get size.
j = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]No. Lines = %wd'//
& ' No. Characters = %wd%nl%nl%cn%5BT[OK]',nline,next)
END |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|