forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Text files and clipboard

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
Notquitenewton



Joined: 25 May 2021
Posts: 9
Location: England, UK

PostPosted: Wed May 26, 2021 8:01 am    Post subject: Text files and clipboard Reply with quote

I wonder if anyone can help...
The problem...I have a large text file which I would like to place in the clipboard (to use elsewhere) using FTN95. The file can contain up to 1,000,000 lines and up to 100 columns (can be as big as 60 MB). I've knocked up a piece of code to do this but it is slow. Methinks it's due to all the Character string manipulations and concatenation! As an example, a text file containing 24,000 rows and 12 columns (all numbers) took about 40 seconds on my PC (i7 processor). I've concluded there must be a quicker way of doing this. The code (a bit on the rough side) is shown and it works, albeit take a coffee break waiting for it to complete!! The text file numbers are TAB separated. Any help most welcome. Thanks.
From Notquitenewton.

IMPLICIT NONE

INCLUDE <windows.ins> , nolist

CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: str1, buff
INTEGER :: j, i, k

C Open text file (tab separated numbers) with up to 100 cols.
C and 1,000,000 rows. The file tested has 12 columns and 24,000
C rows. This routine took about 40 seconds (i7 processor!!) to
C concatenate all rows into 1 string (this took the time) before
C copying to the clipboard. Compiler Siverfrost FTN95 V8.70.

OPEN(FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)

str1 = " "
50 READ(45,'(A)',END=100)str
j = len_trim(str)
i = len_trim(str1)
buff = str1(1:i)//str(1:j)
j = j + i
C Add CR/LF to end of each line
str1 = buff(1:j)//char(13)//char(10)
GOTO 50

100 j = len_trim(str1)
buff = str1(1:j)//char(0)
j = j+1
CLOSE(UNIT=45)

C Send string to clipboard and get size.
i = copy_to_clipboard@(buff(1:j),j,cf_text)
j = sizeof_clipboard_text@()
k = WINIO@('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2341
Location: Sydney

PostPosted: Wed May 26, 2021 10:57 am    Post subject: Reply with quote

I suspect most of the time is taken in assembling the buffer.
This could be easily be verified by reporting the size of the file read at statement 100
I have not tested your program, but the following changes minimise refreshing very long character buffers str1 and buff.
Perhaps you could test this adaptation.
Code:
IMPLICIT NONE

  INCLUDE <windows.ins> , nolist

  CHARACTER (len=5000)    :: str
  CHARACTER (len=5000050) :: buff
  INTEGER :: j, i, k, next, nl

! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.

    OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)

!   buff = " "
!
    next = 1
    nl   = 0

    do
      READ (45,'(A)',END=100) str
      j = len_trim (str)
 !    Add CR/LF to end of each line
      buff(next:next+j+1) = str(1:j)//char(13)//char(10)
      next = next + j+2
      nl = nl+1
    end do

100  buff(next:next) = char(0)
    CLOSE (UNIT=45)
    write (*,*) nl,' line recovered;',next,' characters'

!  Send string to clipboard and get size.
    i = copy_to_clipboard@ (buff(1:next), next, cf_text)
    j = sizeof_clipboard_text@ ()
    k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
    k = WINIO@ ('%ca[Run complete]Finished! Size %rd%nl%nl%cn%5BT[OK]',j)
    END
Back to top
View user's profile Send private message
Notquitenewton



Joined: 25 May 2021
Posts: 9
Location: England, UK

PostPosted: Wed May 26, 2021 11:28 am    Post subject: Reply with quote

John,
I'll take a close look at your mods for the code. Except for one omission it worked. Thank you so much. The time went from 45 seconds to less than 0.5 second. Cor blimey!!!
Neat piece of coding now thanks to you. The size of the buffer by the way was 4,536,189
A rather large string methinks. This routine can now transfer a text file straight into the clipboard from FTN95 Fortran if it contains numbers and/or indeed normal text. Hope other users find it useful. After running, I simply pasted the clipboard into various other programs and even Notepad...worked a treat!
I've attached the final code below for information...
The few changes that you made were perfect and the speed is now outrageous!
Best wishes, Notquitenewton (Graham).

IMPLICIT NONE

INCLUDE <windows.ins> , nolist

CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: buff, str1
INTEGER :: j, i, k, next, nl

! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.

OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)

! buff = " "
!
next = 1
nl = 0

do
READ (45,'(A)',END=100) str
j = len_trim (str)
! Add CR/LF to end of each line
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nl = nl+1
end do

100 j = len_trim(str1)
buff(next:next) = char(0)
CLOSE (UNIT=45)

! Send string to clipboard and get size.
i = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2341
Location: Sydney

PostPosted: Wed May 26, 2021 11:33 am    Post subject: Reply with quote

Note I changed the line "100 j = len_trim(str1)" which should have been deleted, as str1 is no longer used.
Back to top
View user's profile Send private message
Notquitenewton



Joined: 25 May 2021
Posts: 9
Location: England, UK

PostPosted: Wed May 26, 2021 11:42 am    Post subject: Reply with quote

Final below...thanks again. The location of the text file can be changed depending on where the user has it stored!
Brilliant 10/10.
Best
Notquitenewton
Graham

INCLUDE <windows.ins> , nolist

CHARACTER (len=5000) :: str
CHARACTER (len=5000050) :: buff
INTEGER :: j, i, k, next, nl

! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. This routine took about 40 seconds (i7 processor!!) to
! concatenate all rows into 1 string (this took the time) before
! copying to the clipboard. Compiler Siverfrost FTN95 V8.70.

OPEN (FILE='C:\PCModfit V7.1\Results\supout.txt',UNIT=45)

! buff = " "
!
next = 1
nl = 0

do
READ (45,'(A)',END=100) str
j = len_trim (str)
! Add CR/LF to end of each line
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nl = nl+1
end do

100 buff(next:next) = char(0)
CLOSE (UNIT=45)

! Send string to clipboard and get size.
i = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]Finished! Size %rd'//
& '%nl%nl%cn%5BT[OK]',j)
END
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2341
Location: Sydney

PostPosted: Wed May 26, 2021 1:15 pm    Post subject: Reply with quote

I would retain the report of lines read and also a report of maximum line length read, to know what you have recovered.
Reducing str len= would improve performance.
Line lengths of 5000 characters can be difficult to deal with. (not sure if there is a limit in clipboard)
Back to top
View user's profile Send private message
Notquitenewton



Joined: 25 May 2021
Posts: 9
Location: England, UK

PostPosted: Wed May 26, 2021 4:07 pm    Post subject: Reply with quote

John,
I tried out a few text files and just to let you know that my last test was using a text file with the longest line length of 325, with 1.1 million lines and 402 million numbers/characters. I ran the below code in FTN95 and it copied the whole lot to the clipboard in about 2 seconds. I am very pleased indeed with your suggestions about tweaking the strings. Now I can add this to my programs!! Also, note the increase in size of Character buff. Some of my text files are monsters!
Best
Notquitenewton
Graham

IMPLICIT NONE

INCLUDE <windows.ins> , nolist

CHARACTER (len=4000) :: str
CHARACTER (len=1000000050) :: buff
INTEGER :: j, k, next, nline

! Open text file (tab separated numbers) with up to 100 cols.
! and 1,000,000 rows. The file tested has 12 columns and 24,000
! rows. Compiler Siverfrost FTN95 V8.70. Change the text file location from
! XXXXXXX to your own choice.

OPEN (FILE='C:\XXXXXXX\supout.txt',UNIT=45)

next = 1
nline = 0
! Add CR/LF to end of each line
DO
READ (45,'(A)',END=100) str
j = len_trim (str)
buff(next:next+j+1) = str(1:j)//char(13)//char(10)
next = next + j+2
nline = nline + 1
ENDDO

100 buff(next:next) = char(0)
CLOSE (UNIT=45)

! Send string to clipboard and get size.
j = copy_to_clipboard@ (buff(1:next), next, cf_text)
j = sizeof_clipboard_text@ ()
k = WINIO@ ('%fn[Courier New]%bf%ts&',1.5D0)
k = WINIO@('%ca[Run complete]No. Lines = %wd'//
& ' No. Characters = %wd%nl%nl%cn%5BT[OK]',nline,next)
END
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group