I just became aware that Microsoft is terminating support of unsecure transactions with Exchange. My app has used EMAILX successfully for years to send email from Linux A-Shell to an Exchange, or more recently to an Office 365 server. However, the app is configured to pass a username and password.

Last night, EMAILX started failing with error 535 (invalid credentials). This has happened before (recently) and without making any changes, the error stopped occurring the next morning. Apparently, MS is terminating support of the old unsecure connection protocol as of Dec 31, 2022, at which point it will require OAUTH2 authentication tokens. In the meantime, perhaps as a warning reminder, MS has been temporarily disabling the old protocol and then restoring it.

Now it's my problem. I need to update my app to use OAUTH2 within EMAILX to send out emails.

HELP!

----

I researched this a bit and have a rudimentary understanding of what's needed. I understand that EMAILX supports AUTH=XOAUTH2 where in lieu of an encrypted user password, I would supply an OAUTH2 token in XCALL EMAILX. I also know that there is an OAUTH2.SBR that can obtain and refresh such tokens.

My situation is as follows:
1. Most of our email is sent in batches, not individually. This happens in real-time and the user has to wait for EMAILX to finish. Then I tell them if the process was successful or encountered an error.
2. Instead of passing a username/pswd in the EMAILX call, I use a hard-coded username/pswd for each type of document that gets sent. This combination is specified in the relevant EMAILX CFG file.
3. These CFG files specify a specific SMTP Server, Host, and Domain and use STARTTLS security. AUTH is set to "*".
4. I first need to get this functioning on my in-house server, then I need to configure it on my customer's system where I can coordinate the effort with their IT people.

I sent Jack an email to this effect last night and his reply follows in the next post.

(Excerpted from Jack's reply to my email regarding the above:)

We actually did add support for OAUTH2 authentication back in mid 2018, and I think I did a demonstration of it at the Conference in Newtown that year. But I have to admit there hasn't been a lot of use of it in EMAILX, probably because it's a bit of a pain and no one wants to deal with it unless they have to. Which is starting to be the case, first with Gmail, and apparently now with Microsoft.

In reviewing it now, I see that although it is mentioned in the revision history for EMAILX ...

... there isn't any other explanation in the documentation for how to use it. I had to look at the source code to refresh my memory.

Basically, the only changes to the EMAILX config are to set the AUTH = XOAUTH2 and then to specify the OAUTH2 access token in place of the password parameter.

But that leaves it to the application to obtain the access token by other means, namely xcall OAUTH2

There are some notes at the top of the sample program referenced in that topic which may help explain it. It has been awhile since I've played with this, but this Google API doc link may help get you started.

Typically you have to create an API account which will then give you your credentials, which you then need to save into a file formatted something this...

{
"client_id":"42d335511501-18fhuvs6eASDFasdcu5obtc3bsasl6at0p1eda0.apps.googleuserc...
"project_id":"a-shell-analytics",
"auth_uri":"https://accounts.google.com/o/oauth2/auth",
"token_uri":"https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs",
"client_secret":"8CaBzzzA3S431Vck4V5_DXvp-LyQRDJZ7M",
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]
"challenge": "S256",
"scope" : "https://www.googleapis.com/auth/analytics",
"port" : 61111,
"api_key" : "AIzaSyCEeT6zDXlQAIhVDA2zUShVLJ3O5GIZdeM"
}

The above credentials were from a Google Analytics API project; the GMAIL credentials may be simpler but I'll have to try working through the process myself to know for sure.

As I recall, it's easy to get credentials for lightweight testing purposes, but for a real app that will be used at customer sites, it's a bit involved.
Once you have your credentials, the OAUTH2.SBR can then be used to obtain the access tokens, which will look something like this:

{
"access_token": "ya29.GlvRBaLXVQ19Pu5nBpfBd9azPIiLFMara2X-FHAc42B4...
"token_type": "Bearer",
"expires_in": 3600,
"refresh_token": "1/RE4wWFDb1FO4Yw-P6uIraHefETMeZX4DzQRVzF0DyBw"
}

It's that access_token that you'll be using in place of the password in EMAILX.

Another complication is that the token is only good for a limited time, typically one hour, after which you have use the refresh_token to request a new one. That can be repeated for a much longer time, but initially, and at some intervals, you'll have to actually go through a browser-based UI song-and-dance to indicate that you want to allow the app to use this service. If you want the whole thing to appear professional to your user, you will want to customize the web page that informs the user about the process.

I suggest we move further discussion on this topic to the forum, since it's bound to be of interest to other developers. (If you search for OAUTH2, you'll see there are already a few threads on various OAUTH2-related subjects although I don't see one specifically addressing how to deal with email.)

I am under a significant time-constraint. My app's ability to send email the "old" way will stop functioning in less than 2 months. In the meantime, due to Microsoft, intentionally or due to their own development and testing, there are times when that ability stops working now.

One work-around would be to direct my unsecured emails to another server, which IT would set up and from which they would do the OAUTH2 handling. I have no idea how complex that would be, nor do I know whether even that solution could continue working after the 12/31/22 deadline.

The better solution would be for me to change my A-Shell app to utilize OAUTH2. But, I have no idea how complex or time-consuming a task that would be. To that end, does anyone have a template that I could follow to simplify the process?

1) Have other people already solved this problem for Exchange/Office 365? If so, could you contact me or share your solution?

2) Jack indicated in his reply that I would need to "create an API account which will then give you your credentials". Does this mean I first have to register as a Microsoft developer? If so, how involved is that and how long would that take?

3) He also stated "it's easy to get credentials for lightweight testing purposes, but for a real app that will be used at customer sites, it's a bit involved." What does that imply? Could I just ask my customer's IT dept to get those credentials for me?

4) Then there's this: "you'll have to actually go through a browser-based UI song-and-dance to indicate that you want to allow the app to use this service." What does that mean?!

5) Also this: "If you want the whole thing to appear professional to your user, you will want to customize the web page that informs the user about the process." Isn't there a way to do this entirely in the background without a web-page? I don't want the user to have to enter credentials at the start or in the middle of an email operation.

I hope someone out there can answer these questions. If it turns out this is a truly complicated, time-consuming process, I would be willing (eager!) to pay someone to do the tough work for me and my customer.

Thank you.

Also:

6) My in-house development version of A-Shell is v6.4.1556.8 and my EMAILX.SBX is v6.4(137).
My in-house Linux is Centos 6.10

7) I also have a VMWare Workstation test system set up running Centos 7.9.2009.
That system's version of A-Shell is v6.5.1711.0 with EMAILX.SBX v6.4(137)

8) My customer's version of A-Shell is v6.4.1557.5 and their EMAILX.SBX is only v5.1(124)!
My customers Linux is Centos 7.5.1804

Do I need to upgrade to A-Shell v6.5 to be able to get EMAILX v6.4(138) or higher? Or to use XOAUTH2.SBR?

Are any Linux updates or packages needed to use OAUTH2?

Just a quick / preliminary correction to the comments I made earlier about getting credentials: Although you clearly indicated Microsoft Office365, I was referring to Google API credentials (probably because virtually all of my OAUTH2 testing has been with Google APIs). The OAUTH2 basic protocol is the same in either case, but the mechanism for getting your initial credentials will be different. (Google at least makes it fairly easy for developers to set up API accounts without committing to paid service subscriptions; it's not yet clear to me if that applies to Microsoft as well.)

I haven't yet gone through the Microsoft process myself, but here's a technical note / recipe that might help, assuming you have an Azure login with admin privileges: create-azure-app-registration-for-use-with-imap-pop3-and-smtp. To register your app, this may help: Quickstart: Register an application with Microsoft Identity Platform

As for your timeline, I think you have plenty of time to work this out, but it may take a week or two to familiarize yourself with the details of OAUTH2, the ins and outs of the Azure credentials, to add the necessary logic in front of your EMAILX calls to obtain and refresh your access token, do some testing, etc.

Unfortunately, unlike most A-Shell development challenges, the demo program (OAUTH2.BP) is only able to illustrate the A-Shell coding part of the process. But a big part of the challenge is in figuring out the third-party API registration process, which, for the big providers like Google and Microsoft, can be daunting due to their overall size and complexity. There are however lots of online resources to help you get started, and it will just take some time to work through the various web setup screens and processes.

I'll start working through it on my end in parallel with you and will report progress back here.

Here's a progress report. After following the above steps and creating my credentials file ...


(Note: the xxxxx in the endpoint URIs is to be replaced by the directory aka tenant id)

And a few very minor updates to the OAUTH2.BP sample program (mainly to extend the size of a few fields to accommodate longer response strings in the Azure API), I'm able to request an access token successfully.

That might be enough to get you there as long as you can put up with the need for an interactive consent at the start of an email run. (The consent is good for an hour or so, and may possibly be configurable for much larger time spans in the Azure portal.)

The one detail which I haven't been successful with is requesting a refresh of the access token. That would normally eliminate the need for the interactive consent operation, allowing continuous operation over a long period of time. But I'm been stymied by some detail in the Azure portal thinking that I haven't granted that kind of access. It may be because I have only a free Azure account, or because I don't have an Office 365 subscription tied to it. But I don't think it's an issue with the A-Shell / OAUTH2.SBR code.

Since your client presumably has Azure expertise, they can probably resolve that issue. But unless they grant you the necessary access to their account, you may run into the same issue I did when trying to set up a proof-of-concept on your development machine.

Thank you, Jack. My client said users would not see any "consent" prompts because they would grant EMAILX "Admin Consent". They said they have already done this for other software. So I believe I can take it from here. Thanks again for going through all the trouble to test this with Azure.

Beautiful. I was hoping for an outcome like that!

Question: I have libashnet.so.1 (etc.) in .../miame/bin and /usr/lib. How do I extract OAUTH2.SBR from it?

I noticed that your OAUTH2.BP test program lists among its requirements, A-Shell v6.5. My client is running v6.4. Does that mean that OAUTH2.SBR won't work for them?

While most of the underlying code for OAUTH2 is in the libashnet.so library module, the XCALL OAUTH2 interface code is part of the main A-Shell core, and A-Shell 6.4 doesn't contain that interface. (For what it's worth, OAUTH2 was added in June 2018.)

I'm fairly confident that it would be safe to update (after all, 6.5 has been around for along time and the vast majority of active sites are probably running it). But if a full update is too scary right now, one possible shortcut would be to install the ashell 6.5 executable in the miame/bin directory as ashell65 to keep it separate from your regular ashell executable. You'll also need the latest libashnet.so.1, but I suspect it will be backwards compatible (assuming you're using any of the other routines like FTP2, HTTP2 or CRYPTO). The you should be able to use HOSTEX to launch the ashell65 version just for this operation.

Another possibility would be to recreate the equivalent of the XCALL OAUTH2 interface using DYNLIB. That would require considerably more work though, starting with the need to document the actual libashnet.so entry points.

I think it would be best to update my client to the latest v6.5. In the meantime, I can test on my v6.5 development system.

OK. Testing under A-Shell v6.5.1711.0 with libashnet.so.1.14.184.el7 -- calling OAUTH2.SBR with opcode 1 (and 2) -- I got this:

[img]https://www.dropbox.com/s/bi924bnlr5xetzj/AESOPS%20OAUTH2%20test%20results.JPG?dl=0[/img]

How do I find what Status Code -41 means?

BTW: How do I insert an image in a post instead of just the link to that image?

The OAUTH2 status codes come from the ASHNET Status Codes. But in this case they're not specific enough to be of much help. Does the opcode 1 launch the permission page in the browser as in the image below? Do you get back a tokens file? I have an updated version of the ashnet library that I haven't posted yet that generates some additional debug info that might be helpful. I'll see if I can't pull that together shortly.

As for attaching images or other files, you can use the "Attachment Manager" (see the paperclip icon and link down below the "Post Options" below the editing window.)

I now see more info about why xcall OAUTH2 with status code 1 is failing:

.compil oauth2aes /P
.complp oauth2aes /m /a /av /s /n /i /x:2 /P
End of compilation

.run oauth2aes
OAUTH2AES version 1.0(100) - testing OAUTH2.SBR...
Enter fspec to load client/api params from [client.jso] :
Loading client parameters from client.jso
Successfully loaded client parameters needed for OAUTH2
Enter fspec for access tokens [tokens.jso] :
0) end 1) Request new access token, 3) refresh access token,
8) Raw variation of 1, 9) test emailx: 1

Initiating OAUTH2 request....
STATUS = -41
STSMSG = ChilkatLog:
StartAuth:
DllDate: Nov 28 2021
ChilkatVersion: 9.5.0.89
UnlockPrefix: MCRSAB.CB1062024
Architecture: Little Endian; 32-bit
Language: Linux C/C++
VerboseLogging: 0
initialAuthFlowState: 0
bindAndListen:
port: 3017
backlog: 5
listenOnPort:
socketErrno: 98
socketError: Address already in use
Socket bind failed.
--listenOnPort
listenPort: 3017
Failed.
--bindAndListen
Failed.
--StartAuth
--ChilkatLog

RESP =

So I changed the JSON file to specify port 8080. It got past this step once. Subsequent runs get the above error.


1) How do I know which port to specify in the Client JSON file?

2) Are these references to Centos 7 or Windows 10 ports? Or both?

3) Do I need to free that port after each call to OAUTH2.SBR? Is there a MIAMEX call or Linux command that does that?
.

The port in the client JSON file has to match the port specified in the Azure portal (see image below) ...

You may have to make sure the port isn't blocked by the firewall. It is opened for listening by OAUTH2.SBR, so it will be on the machine that is running the XCALL. It doesn't need to be freed afterwards. (It will be closed by the op 2 operation after receiving the response.)

HOWEVER, I must admit that I'm having trouble with this step myself when trying it on my LInux virtual machine. I'm not sure why it should be any different between platforms (UDP ports are very platform independent!), but it's probably a complication related to the network configuration of my virtual machine and its relation to the host.

It might be time for a slightly different kind of end run: moving the code to Windows. Presumably you can at least test the program there without much difficulty, and assuming you can get that to work, then we can set it up to run as an ATE-side subroutine.

After further naval-gazing, I think it may be difficult to get this version working in the Linux environment. The problem is that that the OAUTH2 client wants to listen on the port on the machine that is running the app, but the request involves launching a browser, which almost certainly running on the Windows client. The browser is then going to get redirected to http://localhost:#### but that would be relative to the browser rather than the Linux machine. It would make sense if the browser was running on Linux but that's another can of worms since we don't have a direct way to launch that and pass the massive/cryptic URI to start the process.

So I think we have to go with the plan of executing the OAUTH request under ATE. It's not too hard to wrap it into a SBX and invoke it from the server, but first you should verify that you can make it work in A-Shell/Windows, as it will be easier to tinker with.

OK. I'm making progress. I added the permissions you've show above to my Azure App. I also set up a single page redirect URI like yours, but with the port number I'm using in my client JSON file. When I test, OAUTH2 opcode 1 is fine, but the program aborts with opcode 2 and throws me out of A-Shell.

Increasing the wait time from 30 to 60 seconds doesn't make a difference.

I get that same abort when I try it under Linux, I believe for the reasons mentioned in my previous post.
Let's verify that you can get it to work under Windows, and then I'll show you how to wrap it into an SBX that can be called from Linux but executes under Windows.

One other thing -- I tried to grant admin consent to myself and got this disturbing message (after logging into Office 365):

I'm finished for the day. I will try to set up a Windows version of this test, but I will first have to install A-Shell v6.5 for Windows. I will let you know when that's done.

I suspect that's a matter of needing to update the API permissions in the Azure Portal. I think it should look something like this...

Unfortunately my permissions look like this. You will note that Status in all cases is blank and I can't grant any
Admin permissions because I am not considered an Admin by Microsoft.

I also don't know how/where you found "Office 365 Exchange Online" permissions.

I'm not sure I can really help with either of these issues. The first one is confusing though, as I thought this was your personal Azure account. In that case, you should have all the privileges. But if it's really an account assigned you from the end user, then they would be in charge of that.

On the second question, I have the same problem with my account, i.e., I don't have an Office 365 subscription tied to my Azure subscription, and thus there are no Office 365 options in the "+ Add a Permission" section. The image that I posted was actually from the supplier of the library component used in the ashnet module on which OAUTH2.SBR is built. I spoke to them about it and was told that we had to add or link our Office365 subscription to the Azure portal, but never got that far. (I don't even have an Office 365 subscription so would have to start there.)

(I'm going to be out for the next 2-3 hours, so hopefully you didn't have your heart set on a series of rapid fire exchanges right away!)

I need some guidance for installing A-Shell v6.5 for Windows 10. I downloaded and invoked ash-6.5.1721.7-web.exe from the 65dev/bin-win folder.

1) Did the above exe file install all the DLLs I need or are there some I need to download separately. If so, which ones and where do I place them? (Or is there a special tool for installing DLLs?) For example, ashnet.dll, libashtls, et. al.

2) I need to configure the Windows version of A-Shell. Is there a simple tool to replicate (at least part of) my Linux setup or do I have to manually configure everything?

3) I need to add EMAILX modules - can I just copy my Linux EMAILX: ppn to the corresponding Windows ppn? Or is EMAILX installed some other way? (Does it use the same emailx140.zip file for both Linux and Windows?)

4) What other modules do I need in order to do the kind of OAUTH2 testing that you (Jack) suggested above?

It's my personal Azure account. I think I must have setup something wrong or omitted a step. Or maybe I had to launch WIndows as an Admin or run my browser as an Admin before logging into Azure and I just launched Windows normally and ran my browser normally. I have no Azure experience and little Windows admin expertise so I'm floundering in the dark here, following your guidance, but making a lot of guesses and doing intensive Google searches. Should I delete the entire App and create a new one, this time trying to run everything as Admin? (BTW: My Windows login is defined as an Admin, so I don't understand what more I need to do.)

I have an Office 365 subscription, but how do I connect that to my Azure subscription?

I realize I am asking a lot of questions. I can wait for answers. In the meantime, I will try to find a way to link my Azure and Office 365 subscriptions. I appreciate all of your help.

To answer your questions about A-Shell Windows installation...

1) The installer should have installed all the necessary DLLs (they go in the bin directory along with the ashw32.exe)

2) You can copy your entire vm\miame directory tree from Linux on top of the Windows version. (You don't need anything from the bin directory, but the executables and libraries all have different names so they won't overwrite the Linux versions.) And files in the dsk directories are all compatible (except that maybe your Linux versions were older). So maybe you should rerun the A-Shell Windows install after doing the copy, to make sure you get the latest versions. Or, you can probably just use the UPDCUR.LIT utility (while logged into SYS:, and then again in BAS:) to get the latest versions of the LITs and SBXs. The one thing you'll need to manually change will be the MIAME.INI, but mostly that will just be a matter of editing the statements that have directory or filespec values (e.g. , DEVICE, CONAME, ERSATZ, LANGUAGE, QUEUE, etc.) Although if you used the %MIAME% environment variable in those definitions, you might be able to use them as is, since Windows will accept forward or backward slashes. Note that the QUEUE specification for Windows though must use DISK: rather than MEM:

3) The EMAILX files are 100% the same across platforms, so just copy them over.

4. The test programs in 908,77 are also the same across platforms, so if you copied all your dsk directories, you should be set.

On the Azure setup, I'm afraid it's the blind leading the blind here. Did you more or less follow the steps I outlined in the Google doc I recorded my notes in? After lunch I'll try retracing my steps with a new application definition to see if I left anything out.

After reading a bunch of MS Azure docs I’ve come to these conclusions:

1. For connecting a background service app (like OAUTH2) to an API (like Outlook 365/Exchange Online), you need to register both the background app and the API. I have not yet figured out how to do the latter.

2. When authenticating an app (as opposed to a user), you can’t use “delegated permissions”. You must use “application permissions” granted by an admin. I’m aware that all my permissions are of the “Delegated” type. I noticed that in your last screenshot, the two “Office 365 Exchange Online” permissions are of the “Application” type.

3. I need to figure out how to grant “Admin Consent” in the “Status” column to all of my permissions.

4. What’s the difference between Application Permissions and App Roles? Do I need to assign any of the latter?

5. When requesting a token, one of the passed fields (Grant Type) must be set to client_credentials. Is OAUTH2.SBR doing that?

6. When a Token Request (OAUTH2 opcode 2) fails, more info is returned than just the Error Code and Description. Apparently, there is also a list of STS-specific error codes (huh?), a trace_id, and a correlation_id, all of which are supposed to be helpful for debugging. Does OAUTH2.SBR receive any of these and is there a way to see them?

7. For background service connections, “Refresh” tokens will never be granted. If a token expires, we must request another “Access” token. Thus, there’s no point testing that particular OAUTH2 opcode (should I ever get that far!)

8. Also, if you don’t convert your “Free” subscription to “Pay-As-You-Go” before the 30-day trial ends, you will lose everything you’ve set up when the trial ends.


These are some of the docs I found useful:

A) OAuth 2.0 Client Credentials Flow on the MS Identity Platform

B) Overview of Permissions and Consent in the MS Identity Platform

C) Add App Roles and get them from a Token

D) Register the Azure App for Exchange Online in the Azure Portal

(I’m not sure whether this one is relevant, but it was the closest thing I could find to help me solve problem 1, above.)

And thank you for the A-Shell Windows recommendations. I will work on getting that set up Monday.

Also: Yes, I used your Google Doc as my guide. That's probably why we're both stuck at the same point.

I went went through the process of registering a new app again, and updated the [url=https://docs.google.com/document/d/17-0qcZPgBtj2ySDrPHG7lGZW69ibbKB3Bk_ph5l-puU/edit?usp=sharing]Google doc (see the section with today's date on it.) As you'll see, this time it worked even better than before (even OAUTH2 refresh now works). I'm not sure why, other than my guess is that it is related to me removing the Office365 scope entries and just using the https://graph.microsoft.com/mail.send scope. Unfortunately, that's a bit short of what you really want to do, and we can't really verify that the access token can be used with EMAILX since I don't have Office365 set up.

As for your numbered notes, I'm about as confused as you are. OAUTH2 is complex enough, but it's just a tiny part of the vast Azure landscape. I'm not even sure that these alternate non-interactive authentication schemes even use OAUTH2. On the other hand, I don't see why Office 365 SMTP authentication with OAUTH2 should be fundamentally different than Google SMTP authentication with OAUTH2. In the latter case, you should be able to interactively get the initial access and refresh tokens just once, and then refresh the token for months without further UI intervention. All that stuff about background apps, or apps with no keyboard or browser, I don't think really applies to the A-Shell situation. Even the initial user interaction for the initial access token can probably be eliminated with the proper delegation options, if we could only figure out what they are.

At this point I think it would be helpful to seek advice from the IT experts at your end user who presumably administer their Azure setup. I don't quite see why they shouldn't or wouldn't provide you with the client tokens (client_id, client_secret and the endpoints) and whatever delegated rights you need in order to obtain the OAUTH2 access tokens for their Office365 SMTP service. I think you're going to have to do that anyway, even if you do first manage to simulate the entire setup in your personal trial account.

I'm trying to emulate your Google Docs procedure, but I'm still:
a) not getting an approval popup dialog
b) crashing out of A-Shell when opcode 2 times out

I am testing from AlphaLAN (with TERM=am62cz). From there, the MIAMEX MX_SHELLEX consistently fails with status -99.

So, I tried re-running the test under ATE (with TERM=am62cg) and finally got the approval popup. I approved, but OAUTH2 with opcode 2 still failed. This time it looked like so:
FATAL: exception not thrown
Then in an ATE Connection popup:
SSH Rcv Status (-24) -1
When I click "OK", ATE closes.

This morning I emailed my customer's IT staff asking for Azure help, but so far I've received no response.

I get basically the same thing you do when I run it under Linux via ATE, which I believe to be because of the conflict between the browser running on the PC workstation where it expects to be able to redirect to the specified port, but the OAUTH2 call is running on the Linux server where the port is actually being made available. As I think I may have mentioned previously, I think the solution to that is to repackage the OAUTH application code as an SBX that can be run entirely on the ATE client side. That isn't very complicated, but before doing that, I wanted to make sure you could run the process in A-Shell/Windows, taking Linux and ATE out of the picture. Were you able to do that?

Question: Does the failure of MX_SHELLEX mean that there's no way to get an OAUTH2 token under AlphaLAN? Does the token request have to come from a browser? (AlphaLAN has a built-in browser, but I'd have to research how to implement it.)

RE: The Windows test - not yet attempted, but I will get to it soon.

I'm not 100% sure of anything. Logically it seems (to me) that the browser part of the operation should be able to run separately from the part where we get back the tokens via the specified http://localhost:<port>, but since neither of us can get that to work, while I can make it work under Windows, that seems like the least obstructed path forward. But in order to run the OAUTH2 operation under Windows while driving it all from your application under Linux, you're going to need ATE. Not so much because of the browser capability, but in order to have the OAUTH.SBR implementing running under Windows.

Regarding the MX_SHELLEX failure, you could probably replace that with whatever ESC sequences are supported by AlphaLAN to tell it to launch the browser. But that isn't going to address the problem of OAUTH2.SBR needing to run under Windows.

I finished setting up A-Shell Windows v6.5 on my in-house system. I followed your directions and reinstalled A-Shell after copying over all of my directories and installing SOSLIB and EXLIB. I also reconfigured my miame ini (as NIF.INI). I then recompiled and ran my customized version of your OAUTH2 test program. But the opcode 2 call is still returning status -41.

Opcode 1 opens a browser window that simply displays: "OAuth2 access is granted." (and remains open - I have to manually go back to the A-Shell window)

Here's my A-Shell output:

.run oauth2aes
OAUTH2AES version 1.0(100) - testing OAUTH2.SBR...

Enter fspec to load client/api params from [client.jso] :
Loading client parameters from client.jso
Successfully loaded client parameters needed for OAUTH2

Enter fspec for access tokens [tokens.jso] :

0) end 1) Request new access token, 3) refresh access token,
8) Raw variation of 1, 9) test emailx: 1

Initiating OAUTH2 request....
STATUS = 0
STSMSG =
RESP = https://login.microsoftonline.com/a68fdccd-d735-439b-a1f7-40641e1daa5f/
oauth2/v2.0/authorize?response_type=code&scope=openid%20email%20profile%20offlin
e_access%20https%3A%2F%2Fgraph.microsoft.com%2Fmail.send&redirect_uri=http%3A%2F
%2Flocalhost%3A8080%2F&client_id=906b10b5-3976-492b-b1a4-6eca211bc691&state=ktWl
hScuG5VUmXeUVqlW6Fe9QonAKBUYwmNPykH6cbc&code_challenge=5GmN3rbgW5pBYCQntpcMrOPRg
0B5XRVAuV-PKYyCemU&code_challenge_method=S256
Browser launch status: 0

Requesting the OAUTH2 token...
After xcall OAUTH2 with opcode 2...
STATUS = -41
STSMSG = query param: code: 0.AVIAzdyPpjXX ... (a very long string of characters)
Failed to obtain access token: status = -41

I have good news and bad news.
The good news is that I can reproduce your results.
The bad news is that I'm not sure what the problem is. There certainly does seem to be a disconnect between the "OAuth2 access is granted" result page and the error -41. I activated some debugging logic on my end and the operation seems to proceed as expected

Let me back up to how I got it to work in the first place. I started with this scope...

... and did not grant admin access to the graph email api. And when the permission request web page came up, I did not check the box to grant permission on behalf of my organization. And everything works as it's supposed to, giving me back an tokens.jso file with the access token.

If I then repeat the process, it doesn't even ask for permission and goes straight to the "OAuth2 access is granted" page, but it still returns me STATUS 0 and a new set of tokens...

The fact that it works differently the second time suggests that Azure has some kind of timer/memory and decides that it isn't necessary to go through the full authorization consent process a second time. Which may be reasonable, but makes it more difficult to do repeated testing since we can't always determine whether the behavior is changing because of something we changed or because the server's mood changed.

If I add the graph email API to my portal and grant it admin privileges through the portal or by checking the box on the initial authorization request, or if I add the https://outlook.office365.com/SMTP.Send to the scope in the client parameters, then I start getting the behavior you're getting, i.e. status -41. Unfortunately, as previously noted, that error isn't very specific (which might explain why it is appearing in these two seemingly different situations).

Also, the first time I tried this with the https://outlook.office365.com/SMTP.Send scope, it all worked. So it does appear as if the behavior evolves. But the fact that I don't have Office365 attached to my Azure further complicates things.

Two questions for you: 1) What do you have in your scope parameter? 2) Did you manage to get Office365 connected to Azure?

1) My scope is the same as yours (i.e., the one that worked):

{
"client_id":"906b10b5-3976-492b-b1a4-6eca211bc691",
"client_secret":"8iw8Q~z6a5KmEHTc.oXHM3aqv24uVQ32jDWQ8c66",
"auth_endpoint":"https://login.microsoftonline.com/a68fdccd-d735-439b-a1f7-40641e1daa5f/oauth2/v2.0/authorize",
"token_endpoint":"https://login.microsoftonline.com/a68fdccd-d735-439b-a1f7-40641e1daa5f/oauth2/v2.0/token",
"scope":"openid email profile offline_access https://graph.microsoft.com/mail.send",
"challenge":"S256",
"port":8080
}

2) I kind of/sort of got Office 365 connected to Azure:

NOTE: I already have two Office 365 accounts, one for AESOPS and one for Jencap. I am concerned that fooling around in Azure will mess up one or both of these other accounts. I have still not heard back from IT and doubt they are going to be very helpful.

1. I went back to "App Registrations"
2. Clicked "New Registration"
3. At the bottom of the screen it said:
"Register an app you’re working on here. Integrate gallery apps and other apps from outside your organization by adding from Enterprise applications."
4. So, I clicked the "Enterprise applications" link in that message.
5. This took me to an "Enterprise Applications" screen.
6. I scrolled down and selected “Office 365 Exchange Online”
7. On the Overview Screen, a popup indicated: "1. Provision User Accounts -- “You’ll need to create user accounts in the application”
8 I clicked the “Learn More” link after that message. It displayed:
Provision users
In order for single sign on to work correctly, users must be created both in Azure AD and the target application.
Open the application's admin console and follow the directions for adding users, if you haven't done so already.
9. The last line confused me since I don't see how I can add a user to EMAILX or OAUTH2.SBR
10. Anyway, I next clicked “Users & Groups” in the side menu -- none were listed
11. I selected “+ Add user/group” – got msg: I can’t add a group under the Free plan
12. I was then on "Add Assignment" screen. Under "Users" it showed "None Selected"
13. I clicked on the "None Selected" link
13. That presented me with a list of 4 email addresses: 2 for friends whom I'd defined as an Outlook Group (related to my AESOPS account), 1 for myself (as AESOPS), and an email for LMS, the organization that used to manage my Exchange accounts (but hasn't done so for years. They defined themselves as Admin and I cant undo that)
14. I clicked on my email address and hit the "Select" button
15. This took me back to the “Add Assignment” screen - under "Users:" it now showed "1 user selected"
16. I clicked the “Assign” button at the bottom of that screen – Application Assignment successful
17. I was back on the "Users and Groups" screen. It now showed 1 user: me
18. I didn't know what to do next, so I clicked “Permissions” in the side menu
19. The "Permissions" screen showed: “The ability to consent to this application is disabled as the app does not require consent. Granting consent only applies to applications requiring permissions to access your resources.”

After all that, when I went back to the "App Registrations" screen, nothing had changed. I don’t see “Office 365 anywhere.

Now what?

Is there a way your OAUTH2.SBR can return a more detailed error message? When I search online, if an OAUTH2 authorization attempt fails, an error description is returned with the error code.

The first part mystifies me, since in theory that should be equivalent to what I'm doing successfully here, unless there's some disconnect between our API permissions. Here's what mine look like (below).

As for the Outlook configuration stuff, I'm not sure how much help I can provide there. The reference to adding the user to the application I don't think is relevant here, other than that your application is going to have to provide a user login to access Outlook, but that was always the case. The fact that it tells you that you don't need permission for the app does seem relevant and suggests that whether or not they are planning to require OAUTH2, it seems as if you were allowed to somehow bypass that requirement, at least for now. Or worse, you aren't even being allowed to use OAUTH2, which makes it kind of hard to test. Perhaps it has something to do with configuring this under the Enterprise framework rather than the individual one. (As an aside, the one Office365 enterprise client I know is not aware of this pending OAUTH2 requirement, and uses a private front-end URL to access the Office365 SMTP service without requiring any login at all from EMAILX. But no one seems to know exactly how that was set up.)

Regarding more detailed error status information, I have actually done some work on that this morning and will release an updated DLL later today after more testing. But, for the error -41 situation (which I can reproduce), the details are very disappointing, i.e. the status message from the server that it supposed to tell us something, instead looks like an actual refresh token. I've sent a tech support request to the library developer to see if maybe that is some kind of glitch in the library, i.e. the failure code is some kind of mistake. In your Office365 test though, I suspect you're going to get an error status message that looks similar to to what you're seeing in the browser, i.e. “The ability to consent to this application is disabled as the app does not require consent. Granting consent only applies to applications requiring permissions to access your resources.”

One other think we could try just for kicks: can you add jack@microsabio.com as an external user to your Aesops tenant? If so, then I should be able to tap into the same server environment as you have, which may or may not reveal something useful.