forums.silverfrost.com Forum Index forums.silverfrost.com
Welcome to the Silverfrost forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Where is error?

 
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General
View previous topic :: View next topic  
Author Message
DanRRight



Joined: 10 Mar 2008
Posts: 2813
Location: South Pole, Antarctica

PostPosted: Tue Jun 07, 2016 8:44 am    Post subject: Where is error? Reply with quote

Do i have to stop programming? Damn, i can not find the error for hours. The data is gotten by scanning large database, it has produced a lot of errors due to that, but i seems found all of them. I spotted the error making smallest snippet from the data. Some error is still there


Code:
   
        real*4 EZ0(100)
        character*1 aaaa


        OPEN (UNIT=3,FILE='data',STATUS='OLD')

   READ (3,*,ERR=1003,END=1000) IZNatmd
   READ (3,'(A)') AAAA
   READ(3,*,ERR=1004,END=1000) (EZ0(i),i=1,IZNatmd)

   close(3)
   goto 1000

1003    print*, 'Error reading data 1 '
   goto 1000
1004    print*, 'Error reading data 2'

1000   end


The file called "data" is here:

10
!...IONIZATION  POTENTIALS (ALL DATA IN FORMAT = *)
6.815E0,1.447E1,2.349E1,3.632E1,4.9l4E1,9.266E1,1.187E2,1.30lE2,1.595E2,1.797E2,
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Tue Jun 07, 2016 8:54 am    Post subject: Reply with quote

Code:
   real*4 EZ0(100)
   character*1 aaaa
   integer*4 IZNatmd ,i

    OPEN (UNIT=3,FILE='data',STATUS='OLD')

    READ (3,*,ERR=1003,END=1000) IZNatmd
    READ (3,'(A)') AAAA
    READ(3,*,ERR=1004,END=1000) (EZ0(i),i=1,IZNatmd)

    close(3)
    goto 1000

 1003    print*, 'Error reading data 1 '
    goto 1000
 1004    print*, 'Error reading data 2'
         write (*,*) (EZ0(i),i=1,IZNatmd)

 1000   end


Dan,

Try this as it shows the error is in the 4th number. You typed "9L" instead of "91" in the 5th number : 4.9l4E1
Back to top
View user's profile Send private message
mecej4



Joined: 31 Oct 2006
Posts: 1885

PostPosted: Tue Jun 07, 2016 12:27 pm    Post subject: Reply with quote

If the "large database" file is suspected to contain more such errors, it would be helpful in spotting the bad input line to keep track of the number of lines read from the data file and, when the ERR=... clause is triggered, print that line number.

In addition, you could also print out the offending line of data to facilitate further manual inspection and correction. Here is your program, modified to do these additional checks:
Code:

        Program XDRR
        real*4 EZ0(100),EZi
        character*1 aaaa
        character*132 line     
        OPEN (UNIT=3,FILE='data',STATUS='OLD')

        READ (3,*,ERR=1003,END=1000) IZNatmd
        READ (3,'(A)') AAAA
        READ(3,'(A)',END=1000) line
        READ(line,*,ERR=1004)(EZ0(i),i=1,IZNatmd)

        close(3)
        goto 1000

1003    print*, 'Error reading data 1 '
        goto 1000
1004    print*, 'Error reading data 2'
        print*, 'Line containing error: ',trim(line)
        k=1
        l=len_trim(line)
        do i=1,IZNatmd
           read(line(k:),*,ERR=1010)Ezi
           do while(line(k:k).ne.',')
              k=k+1
              if(k.gt.l)then
                 write(*,*)'Input line exhausted after reading ',
     1               i-1,' items'
                 exit
              end if
           end do
           if(k.gt.l)exit
         k=k+1           !skip ','
        end do   
 1010 write(*,'(A,I1,2x,A)')'Data item causing error is #',
     1                      i,' contents: ',trim(line(k:))
1000   end

The modified program outputs:
Code:

 Error reading data 2
 Line containing error: 6.815E0,1.447E1,2.349E1,3.632E1,4.9l4E1,9.266E1,1.187E2,1.30lE2,1.595E2,1.797E2,
Data item causing error is #5   contents:  4.9l4E1,9.266E1,1.187E2,1.30lE2,1.595E2,1.797E2,
Back to top
View user's profile Send private message
DanRRight



Joined: 10 Mar 2008
Posts: 2813
Location: South Pole, Antarctica

PostPosted: Tue Jun 07, 2016 6:58 pm    Post subject: Reply with quote

Ufff, thanks John and Mecej4. And looks like not one but two such errors are here...Do not remember when last time I confused l and 1...

Mecej4. Lately I do things like you have demonstrated, but a bit differently. But this was older code touching which was as dangerous as touching old landmine. Will use your method too, thanks, still have a lot of data to load.

Or may be someone has good quality database of ionization potentials of all ion stages of all atomic elements of periodic table? Mine old one has errors in every second number
Back to top
View user's profile Send private message
mecej4



Joined: 31 Oct 2006
Posts: 1885

PostPosted: Tue Jun 07, 2016 7:49 pm    Post subject: Reply with quote

Dan,

You can spot the offending 'l' (l.c. L) characters more easily if you make an 'upper case' copy of your data file and compare the two versions with a file difference utility such as Windiff or Kdiff3

6.815E0,1.447E1,2.349E1,3.632E1,4.9l4E1,9.266E1,1.187E2,1.30lE2,1.595E2,1.797E2
6.815E0,1.447E1,2.349E1,3.632E1,4.9L4E1,9.266E1,1.187E2,1.30LE2,1.595E2,1.797E2

A variable-width font in your browser or a text-editor will show the second line to be a bit longer than the first, alerting you to the presence of the 'l' or a similar character with different widths for lower and upper case.

Even though these lower case ELL-s may have seemed troublesome, the reality is that there are data errors that are more harmful and harder to catch. Consider this: neither John nor I needed to know anything about ionization to spot the problem -- the compiler and the RTL did that, and John spotted the exact location for you. Take that number: 4.914E1. Is it correct? What are the rules for deciding? Suppose that the physics of the problem imposes a threshold value of 4.974E1, and that the '7' was typed in as '1' by mistake. How will you ever catch this error?

In general, gross errors tend to be spotted soon. The tough-to-catch errors are those that give reasonable-looking but incorrect results. Such errors can remain unfixed for decades.
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Wed Jun 08, 2016 12:40 am    Post subject: Reply with quote

Dan,

A different approach: I have written a text file scanner that counts all characters (0:255) in the file.
I actually wrote it to test stream I/O and also find dos vs unix file formats.
Effectively, this reports all numeric and non numeric characters.
It would be very easy to also create a report that lists all non-numeric characters in the file, or those lines only, by replacing all numeric characters with a blank.
It would be a quick data verifier, that would work for text files.
I might see if I can patch a "non-numeric" report up.

John
Back to top
View user's profile Send private message
John-Silver



Joined: 30 Jul 2013
Posts: 1520
Location: Aerospace Valley

PostPosted: Thu Jun 09, 2016 12:31 am    Post subject: Reply with quote

mecej4 makes a good point about the general difficulty of finding errors in data imput.

It remindsme of a saying I once saw which goes ... "... when it coms to input data, don't trust anyone ... not even your grandma ... ". Wise words indeed, and which would be further enhamced by the addition " ..... and especially yourself !"
We've all been there I'm sure.

As for the explicit example given by mecej4, that should be easy to track since 'threshold value checks' could be built in to the program immediately after the read, although as we all know those type of 'additions' are time consuming .... until we count the time lost trying to track silly errors in input data.
Back to top
View user's profile Send private message
JohnCampbell



Joined: 16 Feb 2006
Posts: 2554
Location: Sydney

PostPosted: Thu Jun 09, 2016 1:06 am    Post subject: Reply with quote

Dan,

This is a link to the scan program that can report all non-numeric characters.
It provides statistics of what characters are used in the file, which can be good to understand the type of data in the file or testing for DOS or UNIX format text files.
It also gives an example of using FTN95's stream I/O, opening with form = 'UNFORMATTED', access = 'TRANSPARENT',

https://www.dropbox.com/s/trk4k3acmj4gz9e/scan_file.f95?dl=0

To compile use : ftn95 scan_file /link
To report all non-numeric characters use : scan_file text_file_name /numeric /stats

I hope it can provide some ideas and let me know if there are any major errors in the program.

John
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    forums.silverfrost.com Forum Index -> General All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group