Packet Fragmentation and Consolidation
The TCP transport protocol guarantees that all bytes sent are delivered in the order sent. However, packets may be fragmented and/or consolidated at any point along the process. For example, if you send out several packets of 100 bytes in rapid succession, they may be consolidated and delivered as a single packet of several hundred bytes. Or conversely, a large packet of 4000 bytes may be broken up and delivered as two or more smaller packets. In fact, the very idea of a "packet" doesn’t really exist in TCP, which is a byte-stream protocol. "Packets" are used at the network and datalink layers to implement the TCP byte-stream, but are essentially invisible to us as the TCP level.
This is an important consideration when using TCPX (or TCPCLI / TCPSRV), since the fact that they can send or receive a logical packet of data at a time, may falsely give the impression of these logical packets being the same as physical packets, i.e. more like "messages" or "data records", when they are not. They are just groups of bytes, which may well be regrouped (but never reordered) during the transport process. It is up to the application to design a protocol that works with this paradigm. There are several possible approaches to doing this.
One is to build your own two-level protocol on top of TCPX (TCPCLI / TCPSRV), with the lower level simply being responsible for accumulating (or transmitting) bytes, and the upper level mapping the buffered bytes into logical "packets" or "records". This is a bit awkward to do in BASIC, although it is certainly possible using unformatted variables with substring addressing, or arrays, along with the ability to map overlay structures in BASIC.
Another is to make extensive use of Opcode 8 to always check that the number of bytes in your "packet" is available before reading, and to always specify the "packet" size in the FLAGS argument when reading, so that you don’t accidentally read more than one "packet" at a time. This, of course, works in conjunction with a protocol involving fixed length logical "packets".
As discussed in the preceding topic, TCPX added the ability to combine a time-out with a blocking read operation. This simplifies the task of reading complete packets, but doesn’t completely eliminate the possibility that fragmentation and network delays will result in the timer expiring after receiving only a part of the desired logical packet. (This, however, would be pretty rare, and gets rarer still as the time out value is increased and the packet size is decreased. In fact, with packets less than about 1000 bytes, the chance of them being fragmented is practically nil, and even if there is fragmentation, the chance of there being a significant delay in arrival of the packets is another order of magnitude smaller. So it wouldn’t be unreasonable to forego the effort of reassembling physical packets into logical ones and instead just treat such a condition as a general protocol error, and reset the connection.)
A third approach, which might be combined with either of the others, is to include mandatory "ACK/NAK" messages in your protocol. These allow you to synchronize the two processes, as well as providing a way to abort the connection at convenient points where errors might be anticipated. For example, consider a simple file transfer protocol. The client sends a (fixed length) "request packet" to the server asking to retrieve a certain file, and the server responds first with a "header packet" giving details about the file (most importantly its size) and then sends a series of "data packets" containing the file. If the sender immediately followed the "header packet" with "data packets", they might arrive merged, requiring the receiver to separate them. But if the sender waits for the client to "ACK" the "header packet" before proceeding, the client can avoid excess manipulation of the incoming data, and if necessary, it can abort the transmission right here before it starts by sending a "NAK". The following code excerpt illustrates such a client:
! Simple file transfer client
map1 CONTROL'PACKET ! arbitrary packet layout (must match server!)
map1 CTL'OPCODE,B,1 ! 0=end, 1=request file, 2=ACK, 3=NAK
map1 CTL'SIZE,B,4 ! # bytes in file (0=NA)
map1 CTL'DATA,S,251 ! requested file name
map1 CTL'PAK'SIZ,F,6,256
map1 DATA'PACKET,X,1024
FLAGS = 0 ! (blocking connection)
call CONNECT ! connect to server
if STATUS <= 0 goto ERROR
CTL'OPCODE = 1
CTL'DATA = "MYDATA:SAMPLE.DAT"
FLAGS = CTL'PAK'SIZ ! send fixed length request packet
call SEND'CONTROL'PACKET
if STATUS < CTL'PAK'SIZ goto ERROR
call READ'CONTROL'PACKET ! wait for response
if STATUS < 1 goto ERROR ! premature close (0) or other error (<0)
if STATUS < CTL'PAK'SIZ then ! (partial packet received – loop and get more)
< get rest of control packet >
endif
open #CH, CTL'DATA, OUTPUT ! open file ch to write received file
! (if error during open, our error trap
! should send a NAK to server, possibly
! with a reason in CTL'DATA)
CTL'OPCODE = 2 : FLAGS = CTL'PAK'SIZ
call SEND'CONTROL'PACKET ! send ACK to server
if STATUS < CTL'PAK'SIZ goto ERROR
RCV'LOOP:
FLAGS = 0 ! receive up to size of DATA'PACKET
call READ'DATA'PACKET
if STATUS <= 0 goto ERROR ! premature close (0) or error (<0)
PRINT #CH, DATA'PACKET[1,STATUS]; ! write received bytes to file
TOT'RCVD = TOT'RCVD + STATUS
if (TOT'RCVD < CTL'SIZE) goto RCV'LOOP
close #CH
call CLOSE'CONNECTION
