Hi,

We are using xcall HTTP to download files using a call similar to this:

The destination directory has an ACL defined so that new files should get read/write permisions for owner, group, and other.
getfacl -d ddstransfer
# file: ddstransfer
# owner: root
# group: root
user::rwx
group::rwx
mask::rwx
other::rwx

With libashnet 1.14.178.el7, this yields files as you might expect (with the expected permissions).
With libashnet 1.14.194.el7, the files get created with 600 permissions. So, it would appear the ACL is not honored.

So, I am not sure if this is an actual issue in libashnet or if there is some other technique we should be using.

Thanks,
John Andreasen
Diversified Data Software

Is this problem limited to HTTP, or does it happen with any file created under A-Shell?
I don't yet see any obvious explanation but am digging...

I just reproduced the behavior, at least under Debian 12 with the libashnet.1.14.196.d12. It seems to affect not just HTTP but also {S}FTP.. Which does make it seem specific to that library. But I'm at a loss to explain how the library is able to bypass the ACL, even if it wanted to. (Seems like it should be implemented at a lower level so that it was beyond reach of application code.)
I'll have to take it up with library supplier and get back to you...

Actually, for the record, I don't see any difference in the behavior with the 1.14.178.el7 library vs the 1.14.196.el7.

Hi Jack,

To answer your question, it does not appear to happen with regular files created under A-Shell. They are getting the rw permission for user, group, and other. Reverting to 1.14.178 seems to resolve the issue in the production environment.

That's what I'm seeing here, except that 178 has the same problem. Falling back is one workaround but obviously not ideal. Another workaround would be to make a copy of the file after the download (although that seems silly). Hopefully I'll have a better answer tomorrow.

OK, sounds good.

According to the library developer, and my own tests, there hasn't been any obvious change in the way the downloaded file is created over the span of library versions in question. Both A-Shell and the library end up using the same standard C open() call to create a file, with the difference between the two being that the HTTP download is not requesting anything other than rw access for the owner (not the group or other), whereas A-Shell defers to the user's UMASK, typically resulting in rw-rw-rw-.

The ACL connection seems to be a red herring, as there is nothing in either the library or the ashell executable that fiddles with the ACL directly. I think much of the problem is that the library's filemode specification of rw------- for the file somehow conflicts with or undermines the ACL? I'm not sure I understand the ACL ins and outs well enough to explain that.

But a simple-enough workaround would be to just add an XCALL HOSTEX,"chmod g+rw file.ext" after the XCALL HTTP. Perhaps that should be built in to the XCALL HTTP so that its behavior matches rest of A-Shell's file creation behavior. Let me think about that...

We would certainly be willing to perform a chmod on the file after downloading it, but since we set the 's' permission on the ashell executable, the file ends up being owned by the same user that owns the ashell executable (in our case ashusr.) When we try to perform an operation such as chmod via XCALL HOSTEX, it is unsuccessful and outputs an error of "Operation not permitted". I am not sure what the origin or necessity of setting the 's' permission is. I may have to get with Stephen on that.

The +s privilege on the ashell executable isn't absolutely necessary, but it's handy in most cases to ensure that all files created by A-Shell act as if they were created by the owner of the ashell executable, rather than the actual user. That way you can create an ashell group which all the ashell users share access to, while excluding not ashell users. Another benefit is that it allows signals to be sent between jobs (used by JSTAT, KILL, SEND, SYSTAT, and perhaps a few other utilities of that type). Other than KILL, which absolutely needs signals, the others may work without them, but be sluggish to respond. (A signal is used to tell the target that a message has been sent; otherwise jobs check their 'mailbox' on their own sweet time, although typically that's only a matter of seconds.)

I've have further conversation with the supplier of the HTTP download library component and confirmed that it's just a matter of the code requesting only rw------- instead of rw-rw-rw- like A-Shell normally does. Neither of us can see any special connection with the ACL, although I have to admit that I'm still not sure I understand all the ins and outs of ACL configuration.

Maybe we should to back to the beginning and review exactly what you are trying to accomplish. What I'm picturing is that you have a directory named ddstransfer where you want to drop files that are accessible by a wider range of users than would otherwise be able to access files created by A-Shell? But in that case, it would seem that the something like the chmod g+w approach should work, even without the ACL. But the devil's in the details, so we may need to flesh them all out.

In this particular use case, we are downloading a file and intend to process it in A-Shell. Looking closer at the existing code, it does a HOSTEX to give user, group, and other rw permission (which fails now.) Then, it does a HOSTEX using cp to move the file into a temp directory for processing (which fails related to the permission issue as well.) Since the goal is to process the file in A-Shell anyway and the owner is ashusr (the same as the A-Shell executable), it might be possible to download the file directly into the temp directory (skipping the cp operation altogether.) Perhaps the original intent was to keep a copy of the downloaded file around. After testing a bit more (in /tmp instead of ddstransfer), while HOST using cp fails, using XCALL MIAMEX, MX_COPYFILE works and yields a new file with 660 permissions. It seems a little surprising to me that this works as the does seem to indicate that it uses cp internally to complete the copy. Perhaps it is something to do with how it is invoked.

If you SET TRACE FOPENS ON, it will trace the COPY operations, which you see get handled via cp -p. This was a workaround to preserve the original file mode, privileges and ownership.

But setting that aside, I'm still not quite sure why you can't just access the file in A-Shell immediately after the HTTP operation. Even if the privileges have been reduced to rw-------, doesn't it have the same owner as A-Shell? One complication I could imagine is if you wanted to process the file outside of A-Shell by some other user, but that doesn't sound like the case here.

Yes, the intent is to process the file in A-Shell. So, the permissions should not be an issue for accomplishing that if we remove the HOSTEX call that was manually copying the file.