forums.silverfrost.com

[JUSTIN]

I am trying to pull data out of a .CSV file, however, i am running into a few problems.

Is there a way to read from a comma delimited format?
I can read the entries of the 43 segments of input in the first record but The complier is not recognizing the end of the record hence it will not read the data in the second through last records. I am quite lost at this point.

[JohnCampbell]

There are two suggestions I would make:

1) I successfully read .csv files by reading each record into a character buffer then using my own parsing routine to find each field (segment). the 2 rules I have are:
each field is terminated by a comma ,
text fields, enclosed in "text" can enclose a comma

This has worked well for the .csv files I use
My character buffer is line*1024 , which is the largest record length I require.
You can then read each field, using the appropriate format

2) depending on where you got it, the file you have may not have the normal text file record ending, which is compatible with a fortran read statement.
read (unit,fmt='(a)') line

If this is the case, you can open the file as "transparent" or 1 character fixed length records and search for an end of line terminator, by reading 1 character at a time.

Both these approaches work well for me. Good luck

John

[JUSTIN]

John,
Thank You for your help.
Sorry to say but I am a beginner at programming and I don't understand what you mean by a character buffer or even what a parsing routine does.
I will study up on it a bit and have a go at it.

I have been reading my textbook for the last two hours and I finaly figured out what is truly going on.

All of the data is enclosed in quotation marks except the numeric date and time. I can give you the first bit of info to illustrate what my hang up is. The data looks like this when I open it up in note pad.

"Order ID","Date","Numeric Time",
9000,6/17/2003 19:52:47,1055879567,
9001,6/17/2003 19:57:17,1055879837,

There are 43 data fields per record and I am reading each record into a DIM=34 array. The first record or line reads in just fine The problem is when the compiler sees the / sign in the date. The / sign tells the compiler to advance to the next record so what is happening is that it is only reading new information into the first two cells of the array each time it reads inside the loop. and I am not getting the information I need out of the file.

[IanLambley]

Justin,

What John means is that you should read each line into a character variable and take them to bits "by hand". I've being doing it that way since the early 1980s, and I really needed to update some of my routines, so I've just spent the lunch hour writing the following:

[IanLambley]

[IanLambley]

[IanLambley]

and finally

[JohnCampbell]

Justin,

To read only the example csv file you gave, the following may help.
If you need to convert the date into days, the function JDATE will also help.

[code:1:f7032696f5]
! Last change: JDC 30 May 2008 1:28 pm
! Program to read a number and date .csv file
!
! "Order ID","Date","Numeric Time",
! 9000,6/17/2003 19:52:47,1055879567,
!
character line*100, fields(5)*40, string*12
integer*4 num_line, iostat, field_val(5), date_time(6), i,j, mc, nf, k, nd, num
!
open (unit=11, file='csv_file.csv')
!
num_line = 0
do
num_line = num_line+1
read (11, fmt='(a)', iostat=iostat) line
if (iostat /= 0) exit
if (num_line == 1) cycle ! ignore headers
!
! Get the fields seperated by a comma
mc = len_trim (line)+1 ! length of active line
nf = 0
fields = ' '
i = 1
do j = 1, mc
if (line(j:j) == ',' .or. j==mc) then
nf = nf+1
fields(nf) = line(i:j-1)
i = j+1
end if
end do
!
! Now get each field
do k = 1,nf
select case (k)
!
case (2) ! date and time field
nd = 0
i = 1
do j = 1,40
if (fields(k)(j:j) == '/' .OR. &
fields(k)(j:j) == ' ' .OR. &
fields(k)(j:j) == ':' ) then
nd = nd+1
string = fields(k)(i:j-1)
read (string, 1001) num
1001 format (bn,i12)
if (iostat/=0) then
write (*,*) 'Unable to read date field',nd,' in line',num_line,' : ',fields(k)
end if
date_time(nd) = num
if (nd == 6) exit
i = j+1
end if
end do

case default ! numeric field
string = fields(k)
read (string, 1001,iostat=iostat) num
if (iostat/=0) then
write (*,*) 'Unable to read field',k,' in line',num_line,' : ',fields(k)
else
field_val(k) = num
end if
end select
end do
!
! Now store the field values recovered
write (*,2001) num_line, field_val(1), date_time(1:6), field_val(3)
2001 format (i6, i8, 2i3,i5,3i3, i12)
end do
!
end

integer*4 FUNCTION jdate (year,month,day)
!
! Converts DAY/MONTH/YEAR to a Julian date transformed to 2000
!
INTEGER*4 day, month, year, yyyy
!
! Test for valid year
if (year > 1800) then
yyyy = year
else if (year > 100) then
yyyy = -1
else if (year > 79) then
yyyy = year + 1900
else if (year > 0 ) then
yyyy = year + 2000
else
yyyy = -1
end if
!
! "Tony T. Warnock" <u091889@lanl.gov>
! 31/10/2002 02:04 AM
!
! jdate = 367*year
! 1 - 7*(year+(month+9)/12)/4
! 1 - 3*((year+(month-9)/7)/100+1)/4
! 1 + 275*month/9+day
! 1 - 730516
!
jdate = 367*yyyy &
- 7*(yyyy+(month+9)/12)/4 &
- 3*((yyyy+(month-9)/7)/

[IanLambley]

Oops!

Deliberate mistakes in "Subroutine string_extract(...)

before do loop insert:

istat=0

DO loop requires

do i=ipos,ilen+1

Ian
PS - Have made the updates