Bug fix/refinement: When STRTOK stops on a field delimiter, there may be an expectation that it looks forward to the next character in the input to see if that character is a record delimiter, and if so, to consume the record delimiter and return that character in the RDELIM return parameter. Otherwise, if processing data in which the last field of a record is terminated with both a field delimiter and a record delimiter, your next XCALL STRTOK is going to return an empty record, possibly causing you to think that you've hit the end of the input data.
Unfortunately, this behavior has been inconsistent over the years. Prior to 6.1.1398.0, the look-ahead only occurred if the field delimiter was NOT specified in the list of record delimiters supplied in the RDELIM parameter. This handled the case where the record and field terminators were different, the last field of the record was followed by both types of delimiters, and the FDELIM/RDELIM parameters did not share any characters. Unfortunately, due to natural confusion about how to handle this situation, many applications specify the field delimiter(s) in both the FDELIM and RDELIM parameters, which was causing this look-ahead behavior to not always work. In 6.1.1398.0, the logic was revised so that the look-ahead occurred only when the field delimiter was also listed in the RDELIM parameter. This was perhaps worse, as it caused some applications that previously worked to now work incorrectly.
6.2.1426.0 revises the logic again, hopefully this time handling all of the most likely cases as expected. Now, after tokenizing and returning the number of fields requested in the XCALL, it always looks ahead to the next character to see if it is a record delimiter, and if so, it consumes that character too and returns it in the RDELIM parameter. This way, it matters not if you mix up the field delimiters and record delimiters in the FDELIM and RDELIM parameters. Just remember that the RDELIM parameter is updated by each call, so you most likely need to reset it prior to each XCALL STRTOK.
