Large Packets, Multiple Delimiters

The next example is perhaps more typical, taken from a real-world data stream arriving over a TCP socket. The data represents the answer to an auto parts query, and is made up of a list of auto makes. Within each make is a list of models. And within each model is a list of years. The model records are terminated with a "|", the make records are terminated with ~, and the individual fields are terminated with semicolons. So a block of data may look something like this:

Ford; Mustang; 1975-79; 1985-88; 1999-2004|Pinto; 1968,1971,1980-85,1992|Explorer; 1992,1994-96; 1998-2004|Expedition; 2001; 2003~Chevrolet; Impala; 1957-61; 1965-70; 1976,1979|Camaro; 1962; 1964; 1966; 1968; 1975~Mazda; RX7; 1985-91; 1994|Miata; 2004~

Our program to parse the data looks like this:

MAP1 PARAMS

MAP2 MAKES(20)              ! Up to 20 makes

MAP3 MKNAME,S,10            ! name of make

MAP3 MODELS(20)             ! up to 20 models per make

MAP4 MODNAME,S,10       ! model name

MAP4 YEARS(30),S,10     ! up to 30 years per model

MAP1 PARAMS$,S,124200,@PARAMS   ! for easy init of all

 

MAP1 INFO,S,10000           ! block of data

MAP1 RDELIM,S,2,""

MAP1 FDELIM,S,1,";"         ! field delimiter

MAP1 OPCODE,B,1

MAP1 MKX,F,6                ! index to MAKES()

MAP1 MDX,F,6                ! index to MODELS()

MAP1 YX,F,6                 ! index to YEARS()

 

!   <receive data from socket into INFO>

INFO = "Ford;Mustang;1975-79;1985-88;1999-2004|" &

+ "Pinto;1968;1971;1980-85;1992|Explorer;1992;" &

+ "1994-96;1998-2004|Expedition;"2001;2003~Chevrolet;"

+ "Impala;1957-61;1965-70;1976,1979|Camaro;"

+ "1962;1964;1966; 1968;1975;1976; 1977;1978;"

+ "1980;1982;1984;1986; 1988;1990;1992;" &

+ "1994;1996;1998;1999;2000;2002;2004~" &

+ "Mazda;RX7;1985-91;1994|Miata;2004~"

 

PARAMS$ = ""   ! Clear entire param array

MAKE'LOOP:      ! input one 'make' record per loop

MKX = MKX + 1

RDELIM = ""

MAKE$ = ""   ! clear entire MAKES(MKX)

 

xcall STRTOK,OPCODE,INFO,FDELIM,MKNAME(MKX)

IF MKNAME(MKX)="" GOTO DONE

MDX = 0

 

MODEL'LOOP:     ! input models, one per loop

MDX = MDX + 1

RDELIM = "~|"   ! could end on either delimiter

 

xcall STRTOK,OPCODE,INFO,FDELIM,MODNAME(MKX,MDX),RDELIM, &

YEARS(MKX,MDX,1),YEARS(MKX,MDX,2),YEARS(MKX,MDX,3), &

YEARS(MKX,MDX,4),YEARS(MKX,MDX,5),YEARS(MKX,MDX,6), &

YEARS(MKX,MDX,7),YEARS(MKX,MDX,8),YEARS(MKX,MDX,9), &

YEARS(MKX,MDX,10),YEARS(MKX,MDX,11),YEARS(MKX,MDX,12), &

YEARS(MKX,MDX,13),YEARS(MKX,MDX,14),YEARS(MKX,MDX,15)

 

! check what delimiter we ended with...

IF RDELIM="~" GOTO MAKE'LOOP   ! get next make

IF RDELIM="|" GOTO MODEL'LOOP  ! get next model

IF RDELIM<>";" GOTO ERROR      ! unexpected delimiter

 

! if last delimiter ; get more fields – note we have

! a limit of 30 params so we could not do them all in

! one xcall

RDELIM = "~|"   ! could end on either delimiter

xcall STRTOK,OPCODE,INFO,FDELIM,YEARS(MKX,MDX,16),RDELIM, &

YEARS(MKX,MDX,17),YEARS(MKX,MDX,18),YEARS(MKX,MDX,19), &

YEARS(MKX,MDX,20),YEARS(MKX,MDX,21),YEARS(MKX,MDX,22), &

YEARS(MKX,MDX,23),YEARS(MKX,MDX,24),YEARS(MKX,MDX,25), &

YEARS(MKX,MDX,26),YEARS(MKX,MDX,27),YEARS(MKX,MDX,28), &

YEARS(MKX,MDX,29),YEARS(MKX,MDX,30)GOTO MAKE'LOOP

GOTO MAKE'LOOP

 

Note that STRTOK does not clear the unused fields, so it is up to the program to reset them in advance. In the program above, although this is not a problem since we only use the arrays once, we use a string overlay to clear the entire 3 level array in one step just to illustrate the general case.