What is the maximum Random File size that ashell supports on x86_64 RHEL?

Hi Dominic--Jack is traveling today and probably won't see your message until this evening, our time. He'll give the authoritative answer when he sees it. In the mean time, I'm pretty sure the answer is "no limit." There used to be a max file size of 2GB, but that limit was eliminated quite some time ago.

Thanks Ty, no hurry as we are now all off for the xmas/new year break til the 3rd, so I guess only 'belly size' is important between now and then.

Have a great xmas/new year.

I don't know about you guys, but I would say my belly capacity is about 2GB. A-Shell, however, as Ty suggests, has essentially no limit. (Actually it's a 64 bit limit; it uses the "LARGE FILE SYSTEM", aka LFS in the ABOUT.LIT display, regardless of whether the OS is 32 or 64 bit.)

When doing an ABOUT on our in-house AIX development system running 6.1.1364 we show LFS (Large File System) in License Options, while on a user's AIX system running 6.0.1240 they do not show LFS. Both systems are running AIX 7.1.

Is LFS now just automatically included and for whatever reason our system shows it but the other does not?

LFS isn't really a licensing option (although it shows in that list for convenience). It's actually compilation option. But according to my notes, it wasn't showing up in the non-Windows list until 6.1.1308, so that would probably explain it.

(I don't think there have been any non-LFS releases created since A-Shell 4.x)

But if you don't believe me and want to actually test it, I guess the procedure would be to create a file that as a little over 2GB in length and then confirm that you can read/write to a record beyond that boundary.

Probably later this summer we will be updating our programs to allow (selected) files >2GB and have absolutely no doubt that Ashell will work fine - but if it doesn't you will be the first (actually second) to know!

Thanks.

I don't doubt any of that!

But just "because", it might be a good idea to update those 6.0.1240 machines to at least the latest 6.0 (.1271).

Note that in scanning the ash60notes.txt for "GB", there are a few patches related to the 2GB limit after 6.0.1240. Most have to do with ISAM which probably doesn't affect you, but 1268.1 relates to parameter passing, which might conceivably affect you.

A customer is closing in on exceeding a 2GB "random-file" size, I thought it be a good idea to double check to see the latest file size limit (if any at all in Ashell) running 6.5.1643.0? Customer is running RedHat what we believe the file system has a limit of about 2 terrabytes. Thanks.

PS. a test in Training company expanded to 4750001 blocks ok... (2.26gb) ...

Going back at least as far as the beginning of 6.4, A-Shell has used 64 bit offsets to reference locations within both ISAM{-A} and random files. Although the File Size Limits topic mentions only ISAM/ISAM-A, it applies to random as well. Which is to say that the 2GB limit applies to records, or in the case of ISAM IDX files, blocks, not bytes. So, for example, in a file with 128 byte records, the limit would be more like 256GB.

It is common though for Linux to be configured to set per-user limits much lower than that. Typically that shows up in self-expanding files, where the program aborts with some kind of file-related error, after which you notice the file in question has a suspicious size, like exactly 1GB.

OK, Thanks for confirming... and let them expand it further. .

Good day -

Just want to wake this thread back up, as we also have a large customer in need of file expansion.

The largest file in this database is 2GB. (with 4.4 million records)

Ashell: A-Shell Version 6.4.1547.0
Linux: CentOS 7.1

The Docs say:
• The 2GB OS file size limit: In order for any file to exceed 2GB, the file system must support it, and you must be running an "LFS" (Large File Support) version of A-Shell. All versions of A-Shell after 4.9.948 (December 2005) contain LFS, except the Windows versions which contain a "c" in the version string. So, other than those limitations, A-Shell does not impose any inherent limit on overall file size.

It's a bit ambiguous that >4.9 it "contains LFS" it as in its "automatic"? Or does LFS have to be specified in the License/About? In this case it is not:
License Options: POLY ATE ATS APEX

So do i need to activate LFS or is it built-in?

If the natural size limit of ashell based on the linux flavor we are running? I think 1/2 TB is what we can expect.

Is there an internal address limit as to the max record number ashell can address (RANDOM)? Our record pointers are all mapped as F,6

TIA - i know this is redundant/review just triple checking... measure twice, cut once

Sorry for the confusion -- the documentation needs to be updated to clarify this. "LFS" (Large File Support) is compiler/linker option that substitutes 64 bit versions of some type definitions and functions into the otherwise 32 bit executable so that we can handle file positioning beyond 2GB. It was implemented in the A-Shell source code somewhere in the 4.x era. For a good while, we listed it in the license options, but at some point I think we (or Ty) decided that since it wasn't really a license option, it didn't belong there. And since it is now effectively always on (except for the Windows "c" version) it didn't seem necessary to identify it separately.

So, in your case, there is nothing to activate and (almost) nothing to worry about as far as A-Shell is concerned. (You still have to make sure your current file system can handle such large files, but in the Linux world, every file system since at least ext2 has easily supported a terabyte, provided there is actually space available.)

The one A-Shell limit is on the record numbers, which are effectively limited to 32 bit by legacy RUN structures. So if your file has 1-byte records, you'll still be limited to 2GB. But if the record size 256, you can reach 256 x 2GB. 1/2 TB might potentially be an issue if the record size is less than 256. In that case, we may need to do some deeper analysis.

Thanks for the reply. Sounds good, Ashell is good to go on the LFS.

Not sure I understand the record number limit. As far as the 32 Bit record number limitiation, isnt that 4 Bytes which is +/- 4,294,967,295

We would never be anywhere close to that number, but not sure i am interpreting it correctly.

FWIW these large files have a rsize of 256,512,1024

Frank, for what it's worth, we have one file that is a little over 13GB with 3 million records but this is windows. We've had no trouble at all. This is a national file of doctors and a text file is downloaded to the user each night and a new file built from that. It is incredibly fast. This is ISAM-A I'm using. I should say, this is a requirement of eScripts.

32 bits is actually +/- 2,147,483,648, although you're right that we can probably treat it as unsigned and squeeze the full 4,294,967,295 out of it.

You'll be underneath that, but not by a huge amount, since 1/2 TB is 512 GB, which would be 2,147,483,648 records of 256 bytes each.

If that's too close for comfort, we could almost certainly do some internal refinements to A-Shell to increase the maximum record beyond 32 bits. Might be a good excuse to charge extra for the "Terabyte Edition"!

Herman - that's quite the upload! I am impressed that it does the ISAM update that quickly... especially in windows. We have had some performance issues on our windows servers which caused users to update to a virtual linux back end.

Jack - thanks for the info. In our case, we have our own internal limit of 99,999,999 records... before we would force some sort of archive process. Well below the ashell 2 billion record threshold. So we should be all set unless you still see some sort of issue.