B64ENC Limit
#37289
19 Apr 24 04:13 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
Hi,
I seem to be running into an issue with using B64ENC to decode a very long base64 encoded string. The string I am specifying is 309392 bytes long, but the status value returned from B64ENC is only 35436. I did check the limits and capacities page and did not see a mention of a limit for this routine. Is it something that could be easily expanded?
Thanks, John Andreasen
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37291
19 Apr 24 05:07 PM
|
Joined: Jun 2001
Posts: 11,794
Jack McGregor
Member
|
Member
Joined: Jun 2001
Posts: 11,794 |
Hi John - Offhand I don't see any obvious limitation in the length of the data strings supported by B64ENC. So perhaps there is some other issue -- status variable overflow? dynamic variable complication? To help pin that down, I resurrected an old test program, adding a "Big Test", and uploaded it here: b64enc.bp(See the notes in the source about configuring the length of the data to encode - currently it's at 3,000,000 bytes (much larger than your example). Here's what my output looks like:
.run b64enc
Base 64 encoding...
1)ENC file, 2)ENC string, 3)DEC string, 4)Sample test, 5)Big test, 0)End: 5
Generating random data to encode ( 3000000 bytes )...
50 , 49 , 49 , 49 , 49 , 50 , 49 , 50 , 49 ,...
Encoding...
Status = 4000000 (bytes in encoded string)
Decoding...
Status = 3000000 (bytes in decoded string)
Decoded data matches original
Program stop at location counter &h6E0 of B64ENC.RUN
Enter CR to continue:
Re-encoding...
Status = 4000000 (bytes in encoded string)
Success
Encoded and re-encoded data matches
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37292
19 Apr 24 05:17 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
Hi Jack, thanks for the quick response! I will study the difference between your example and my program and see if I can find the difference.
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37303
19 Apr 24 07:35 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
Hi Jack,
I wanted to let you know that I located the problem. Our function wrapper that does base64 decoding was using SPACE() to expand a dynamic X variable to receive the decoded data in. SPACE() must have some type of internal limit. Changing to FILL$() fixed the issue. Sorry to have troubled you with this!
Thanks, John Andreasen
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37305
19 Apr 24 08:12 PM
|
Joined: Jun 2001
Posts: 11,794
Jack McGregor
Member
|
Member
Joined: Jun 2001
Posts: 11,794 |
John - thanks for the detective work! And for bringing up the issue in the first place. It's not yet clear to me why SPACE() should have any length limitation different than FILL$() -- both are constrained by the memory available on the string stack and the size of the destination, but that should be the same in both cases. And deserves further investigation.
On the bright side, the documentation did lead you to the solution. But this is a good example of what I was mentioning earlier in regard to the Stephen's question about the "stable 7.0" -- the more you turn over stones in the code, the more things you find that probably should be cleaned up. In this case, it would make more sense (and be more consistent with DRY principles) if SPACE() internally used the FILL$() logic, since after all, SPACE(n) is equivalent to FILL$(chr(32),N). I'll put that on the to-do list...
Thanks again, Jack
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37306
19 Apr 24 10:34 PM
|
Joined: Aug 2016
Posts: 371
John Andreasen
OP
Member
|
OP
Member
Joined: Aug 2016
Posts: 371 |
Hi Jack,
Well, I am glad some good came out of this! Thanks again for the help.
John
|
|
|
Re: B64ENC Limit
[Re: John Andreasen]
#37307
20 Apr 24 12:33 AM
|
Joined: Jun 2001
Posts: 11,794
Jack McGregor
Member
|
Member
Joined: Jun 2001
Posts: 11,794 |
To put a cap on this, I did discover that length argument passed to SPACE(N) was getting truncated at 64K, which explains the problem. But I ended up mapping it to the implementation of FILL$(chr(32),N), whose maximum theoretical length is 2GB. (The practical limit remains bound by the available size of the string stack.)
As a bonus it's also about 10X faster when filling with single-character pattern (not that the difference will be meaningful to anyone!). So yes, some good did come out of it.
The update will 1757.3, although I'm going to wait to try to get a couple more fixes taken care of first.
|
|
|
|
|