(File this with "640K should be enough for any foreseeable need.")...

At some point in the distant past we decided to extend the traditional AMOS-style (floppy disk inspired, RAD50-enabled) 6.3 filename limit to whopping 10.3. It was a big help, but it is becoming increasingly clear that we should have gone farther. The 3 character extension limit is particularly constricting in AXL / CSV2XL environment where the .XLSX format is increasingly supplanting the old .XLS. Not to mention web programming (.HTML), imaging (.JPEG), etc.

As it stands, any file that doesn't fit within the 10.3 structure is considered irredeemably "native", which sounds like some kind of imperialist/chauvinistic comparison to the "civilized" strictures of the refined AMOS filesystem. But it really is just an acknowledgment of the fact that a number of aspects of emulating AMOS (and applications originally developed for it) depend on certain assumptions about the syntax of filenames. (Note that A-ShellBASIC has no particular problem with opening a file using a native fspec; it's just that the AMOS-compatible paradigm seeps into a lot of areas that might be affected by relaxing the 10.3 rule.)

One example is that it's hard to parse a command line like DIR FOO[]/W when it's unclear if [], and for that matter /W might be part of the filename, as opposed to references to peculiar AMOS wildcarding features and/or switches.

I'm inclined not to mess with the set of allowed characters in A-Shell filenames, for that reason. But it's less clear what dangers we face if we allow filenames to extend to, say, 16.4, or 32.5.

One issue is the format/spacing used by DIR. For example, under the 6.3 regime DIR/W would use 4 columns; under 10.3 (SET LONGDIR) uses 3 columns. If we allow 32.5, we really we'd need to stick with just 1 (provided we want to fit in an 80 column display). But that problem is mainly localized to DIR (and a few other commands that display filenames), plus any applications that use DIR to generate listing which are then read back in by the program.

Another class of potential problems relates to the variable sizes used in existing applications to store filenames. That may also be a fringe problem, except in applications which expect to be able to work with files that administrative operators might create at the dot prompt with COPY and its ilk. (To be precise, that problem already exists for any application that is limited to 6.3 format, not to mention the fact that COPY and other commands do allow you to create and access non-conforming native filenames provided that you quote them. But that's a trick not likely to be known by users who are otherwise familiar with the dot prompt environment of yore.)

A third class of potential problems relates to backward compatibility. For example, currently the MX_FSPEC routine (and functions built on it, like Fn\'FQFS$() will truncate beyond 3 character extensions due to limitations of the emulated implementation of the AMOS DDB structure. If we extend that, a program that runs under the new version will run differently (possibly not in an obvious way) under an older version. (That sounds like another possible argument for the updated RUN header described the thread Compiler/Runtime Version mismatch, except that in this case, the compiler couldn't really identify cases where the expanded internal structures might result in different behavior under earlier versions, because the problem isn't related to any language feature in the source code.)

Anyway, I'm mostly thinking out loud here before taking any action. If anyone has any thoughts or concerns relating to such a change, feel free to share them.

Ahoy -

I am all for longer file names, as i am sure we all are. I appreciate you pre-thinking the pitfalls, and there are probably more. This seems to me like more of a minor rev ashell update as it effects not only the OS but also the libraries and application software. This way it could be easy to determine if the platform is long file name compatible, i.e. if it was say >= Ashell 6.6 then we would know that it's all been fully vetted for long file names and its safe to use them. Just a thought...

I agree with that point. But should we squeeze it into the 6.5 development cycle, wait until we stabilize it (at 6.6 or maybe even 7.0?) and then put it into the next development cycle?

And how long do we want to go? Does it even matter?

I think it would be good to get a few more ideas about what issues people are having, and what workarounds they use.

One workaround we have is a function to convert ashell paths with long native filenames. It just parses the filename.ext and converts the AMOS path to native. This is handy when there's not enough filler available in a RECORD to handle a full native spec, but enough to allow for longer then 10.3 files.

We just have to be careful to make sure we're using the native path when opening, copying, etc. these files.

I would vote for 7.x.

As for potential issues, it would just be on our application code side to accommodate the longer filenames. That isn't really an ashell problem, we just have to be careful before using them and expecting the software to see the longer name.

As for name size, i will let Herman decide...

Herman is fine just as it is. I vote for I,8 and B,8 :rolleyes:

Ok then never mind Herman's opinion...

What is the current windows / linux standard? Perhaps that would be a good start.

From ashinc:types.def ...

I think it's 255 for Linux and 260 for Windows, but hopefully you aren't tempted to get anywhere close to that! Note that in both cases, there is no official breakdown between the name and the extension, nor any limit to the number of dots. To the extent that it makes sense to identify the file's extension, the standard it to just look at the characters following the last dot.

gotit. 255 or 260 should be plenty

Ping!!

Is there any traction on this request?

ongoing issues persist with trying to squeeze all these logs and files and forms into 10.3

I mean, Herman is on board, doesn't that count for something? We need to hurry before he retires... again

PS: As for ashell versioning.. maybe dropping it into the current 6.5 beta and creating a 6.6 stable from that? I'm afraid if it makes it into the 7.0 dev cycle, it will be in perpetual test mode. Yeah yeah i know above i was thinking 7.0 but i might be dead by then... :rolleyes:

Actually, I have spent a fair amount of time on this, but since I thought you wanted to postpone any action until 7.0, there hasn't been much urgency to get the changes in shape for testing.

Unfortunately, the more I dig into it, the murkier it becomes. But here's a brief summary of the issues and situation:

1. Filenames appearing on the command line of LIT commands (DIR, COPY, etc.) -- As far as A-Shell internals go, even 6.4 theoretically can handle up to 32.8; it's just the LITs themselves which are setting the 10.3 limit. I have a new beta set of LITs that recognize up to the internal limit (32.8, due to increase to 72.8 in 6.5.1662). DIR also now supports a /L:##{.#} switch to allow you to specify the display width.

I went with 72.8 instead of 250.8 or something like that because:

a) there are a lot of internal structures which allow for full AMOS-style specs, and it seemed ridiculous to waste so much space for the extremely rare situation.

b) the 72.8 only refers to the file name and extension, not to the full path, which can be up to the full 256 characters allowed by the operating system. So chances are that the 72.8 limit for the file.ext is more than equivalent to the typical 256 limit in Linux and Windows.

2) Filenames opened within programs. Native-style specs are already supported here.

The main complications for both of the above have to do with special characters (like spaces) that are allowed in native specs but not in AMOS-style specs. And Mixed CAse.

Mixed case is not an issue for Windows, but it is for Linux. If we see:

OPEN #1, "MYDIR:SALES projections.HTML", input

we would want to convert that to all lower case for uniformity, since AMOS-style commands are meant not to be case sensitive. But if the file was already created outside of your application, that's not going to work. It may be possible to add some fancy logic to try multiple versions, but this is where it starts getting very murky, and is one of the reasons I've set it aside for the moment.

I think maybe we need to revisit EXACTLY what it is you're trying to do that the current limitations get in the way of.

I can only speak for myself here:

Just looking for ASHELL itself to go beyond 10.3.

I don't need any special handling for special characters or case sensitivity, that's not to say others would want that, however, that could let pandora out of the box.

72.8 is a vast improvement over 10.3, it should be fair to conform to that "limitation".

As you may note, i did think 7.0 but i am not lying here, it could be years before a "7.x" stable is available, i would love for this to be available in the earliest stable release possible. We are saving all sorts of files locally now from credit card receipts, statements, claims files, EDI files, and trying to squeeze meaningful data into 10 digits (assuming leaving the .3 for a meaningful ext) is near impossible.

It's easy to get "anylysis paralysis", it seems that this request goes back many years - if you have already researched it and done 99% of the work let's rip the band-aid off and get it going.

Agreed, but I'm still not clear on what it is you want to do with these non-conforming files...

If you just want to be able to open them from within a program, you can already to that, provided you use native syntax. You can also scan them using MX_FINDFIRST and MX_FINDNEXT, or using the functions in the SOSLIB module fndirscan.bsi

So is the motivation the ability to use AMOS-style filespec notation (device,unit, [p,pn], maybe ersatzes)? Or to be able to access them with LIT commands like DIR?

My goal is to access them as native amos style filespec as you stated.

If possible, and you are the only one that has the ability to say yay or nay...

I want to do everything with them as the current 10.3.

My desire is not to play with native filespecs.. especially when support teams are looking for files and triaging at the amos prompt.

So can we move to 72.8 or can we not?

I can probably make that happen in 6.5.1662 by next week. Is that going to be useful to you? Keeping in mind...

1) Requires updating to 6.5 (although perhaps only for your support team).
2) Assumes that for Linux, all files are actually stored with lower case name.
3) No non-AMOS-compatible filename characters.

Yes, would be hyper-useful thanks... As i would assume other vendors but crickets here makes me wonder?

See replies:

1) Understood thanks... and since we are pushing 6.4 out may be a good time to wait til 6.6?

2) Wouldn't this already be true? Does ashell convert them differently now then when you install this 72.8 update? Or is this just a "warning"?

3) Again, wouldn't this already be true? Does the current 10.3 support non-amos filename characters?

We've ran into issues with this limitation. The middle ground I came up with was to store non-comforming filenames with an erstaz, and a user defined function to convert these to native paths for actually working with them.


Something like the following.

Would you not prefer this be a native feature of the OS?

And what about file maintenance at the (.)? How does your support team handle?

To answer Frank's questions: yes, limitations 2 (lower case) and 3 (special characters) are already true, so no change there. I only mentioned them because I think that they may be significant issues when trying to accommodate modern, real-world filenames within the device/ppn AMOS-style framework.

For example, I'm dealing with a site now that receives files with names like:

ShippingInformation20190620131559001.SHR.psv

Everyone would like to be able to use DIR and COPY and RENAME on them, but the combination of the UPPER case and double extension gets in the way, even with the longer limits. (People are getting good with HOST followed by Linux commands.) So I guess I'm just a little worried about over-selling this.

That also reminds me of another limitation that is not currently in line to be lifted: the maximum filename lengths for programs, subroutines, and ersatzes, all of which are (for the moment) staying at 10.3.

On the other hand, to lighten the negativity factor somewhat, there is one area where adding just one character to the maximum length will make a huge difference: in AXL where the increasingly-most-common file extension is XLSX. (Same goes for HTML and JSON, although those are commonly shortened to HTM and JSO, whereas XLSX cannot be shortened to XLS without changing the meaning.)

As for producing a stable 6.6, perhaps we are coming to a good juncture for that. Let's start with introducing this into 6.5 and allowing it a little time to shake out, and then figure on turning the screw once again to create a 'stable' 6.6 and a development 6.7. (Or maybe 7.0 and 7.1 - I'll leave that to the marketing department.)

Sorry, crossing posts, although I did manage to anticipate your second question (i.e. by accessing Linux shell commands via HOST. Or in Windows, by using EXPLORE.LIT to launch the Explorer and do your dirs, copies, etc. there.)

I'm not sure what you mean by preferring it to be a native feature of the OS. Those kinds of filenames ARE a native feature of the OS. The issue we're struggling with is how to include them within our alternate dev#:file.ext[p,pn] framework.

Sorry Jack those were directed at Stephen's post...

Perhaps we need to update the BBS to introduce threads... :p

As for your post, sounds good... really excited to get this and start playing around with it. I understand the odd filename issues, we deal with that as well.. mostly we try to rename them to valid ashell-recongnizable names, but the 10.3 makes that a challenge too.. so the new extended names will give some nice upgrade here as well...

i.e, for your update .bin and .tz names, we tend to move around inside of ashell for support purposes, and we shorten the names to a1546.bin for example, now we can incorporate the version, edit and even the build, i.e. el4, el5... etc.

Ok, I'll try to get this packaged up for testing by Monday.

Hi Frank,

We use the Linux shell for most of this stuff. We're trying to automate as much as we can; for both consistency and and much less hands on support.

What exaclty are you using .bin/.tz files for from dot prompt?

Hey Stephen -

We store the files on our server but like to see them from an ashell dir... we then send them from our server to the customer server using ztxfer which is limited to the 10.3 syntax. We do very little maintenance from the linux prompt unless Mo or I are working on something... Our level 2 support staff doesn't work from the linux prompt so we like to have as much native to amos as possible.

Thanks Jack sounds good. As we are working on the automated email credit card receipt this will help when saving the log and matching it up to the outbound file... trying to store the acct #, practice, time, date, etc in 10 digits was making me

I can appreciate the problem of trying to jamb all those details into 10.3. (It seems increasingly common, at least in my experience, for external data providers to send or expect to receive files with all of those attributes in the file name. Just the full timestamp CCYYMMDDhhmmss (not even counting milliseconds) is too much!

But as for ZTXFER being limited to 10.3, I don't think that's correct. You just need to quote the local filespec, e.g.

It makes sense not to let them use the Linux shell for security resasons alone.

Agree to both ^^

I've posted a couple of packages in the beta test area for those interested in testing the new filename length limits:

ash65notes.txt
ash-6.5.1662.0-web.exe
ash-6.5.1662.0-w32c-upd.zip
ash-6.5.1662.0-el7.bin

Installed (in-house)

Firstly, thank you.

We are running RHEL5 inhouse. Is it possible to create a build for it?

Thanks.

Working on that. But I discovered a problem with DIR in the meantime; once I resolve that I'll post an RHEL5 build.

I thought I was hallucinating about DIR and seeing double so said nothing ...

Ok, here's an update for RHEL5. The issue with DIR also affected a couple of other LITs (new versions noted in the updated ash65notes). The other packages need to be rebuilt to include the updated LITs, but in the meantime anyone affected can just use UPDCUR to update them.

ash65notes.txt
ash-6.5.1662.0-el5.bin

Thanks Cap!