Did you convert your subscription to Pay-As-You-Go or is it still the basic Free plan? Because I'm still on the latter. I wonder if I converted to the former, if I would then be able to grant Admin Permission?
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3573422 Nov 2201:36 PM
Ok. Now, for what it's worth (probably not much), I can reproduce exactly the status -41 that you get. But I can't see the API permissions so don't have much to go on. (It might even be that although I can now go through the motions of consent, my authority is still being overridden at a higher level.)
Unfortunately the debug information doesn't give any obvious reasons. The "failure message" is just the STSMSG you see on the screen (which looks more like an access token than an error message), and the new detailed low-level log shows what appears to be a normal operation...
Code
StartAuth:
DllDate: May 28 2021
ChilkatVersion: 9.5.0.87
UnlockPrefix: MCRSAB.CB1062024
Architecture: Little Endian; 32-bit
Language: Visual C++ 2017 (32-bit)
VerboseLogging: 1
initialAuthFlowState: 0
bindAndListen:
port: 8080
backlog: 5
listenOnPort:
createForListening:
createSocket_ipv4:
Setting SO_SNDBUF size
sendBufSize: 262144
Setting SO_RCVBUF size
recvBufSize: 4194304
(leaveContext)
(leaveContext)
bindSockAddr: 02001F90000000000000000000000000
Socket bind successful.
(leaveContext)
listenPort: 8080
Success.
(leaveContext 16ms)
listenPort: 8080
appCallbackUrl:
m_redirectUri: http://localhost:8080/
scope: openid email profile offline_access https://graph.microsoft.com/mail.send
.. start of the OAuth2 background thread to wait for the redirect request from the browser on this localhost ..
clientId: 906b10b5-3976-492b-b1a4-6eca211bc691
codeChallenge: 1
codeChallengeMethod: S256
url: https://login.microsoftonline.com/a68fdccd-d735-439b-a1f7-40641e1daa5f/oauth2/v2.0/authorize?response_type=code&
scope=openid%20email%20profile%20offline_access%20https%3A%2F%2Fgraph.microsoft.com%2Fmail.send&
redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F&client_id=906b10b5-3976-492b-b1a4-6eca211bc691&
state=5Ld4sr0zRvKTZ6n461Ia8k0HbEwNCZkfrTbMHr6uRIc&code_challenge=CnOCIlJEsviW6J8NXpOKv-SKvNiU4f2_EdcRxGycJJ4&code_challenge_method=S256
Success.
(leaveContext 16ms)
.. received the connection from the localhost browser ..
receivedStartLine: GET /?code=0.AVIAzdyPpjXXm0Oh90BkHh2qX7UQa5B2OStJsaRuyiEbxpFSABQ.AgABAAIAAAD--DLA3VO7QrddgJg7WevrAgDs_
wQA9P-kpmqGnBokmuRLS2rQfoSn4xQjx_XujAvj-82t_xmBnBS-NGH27OTF7ABHUfu9u01nDllYiKxbfaZvhqaCEWfwnRE2OIvlA80nmZb0s3HnUGiXgZ5N
cAF32FqvKAFlNgKbCYGCo0buAp5pC5bpzLtMlJnCS0VE4IIZ8323kdiuOwMuEVigtIsdFyW1W5ox7rn7N7QpDkM43kyWe565MzZjZTsZ5I_5duXu1TjJhD
4OM3XHEKIcey_G3nBDS_wYlbglBOxrtcMU2OZHMHtHZv3E3rsgUUcrj917qVOXecoULAxAiMzvgj3h9Ln7IceVkq7pRCbBzpDWAQX5risRS1k6IAOdX60ia
QYC3hJbgeuReoTKBEtMibx7aSE9IlwIfZzJozEQomPmXZALVz6-JvVjAxvPHgQvfIH60kwY_AWHFWHE33TVK4852TOp6GpmhJK6yIskOAyZrP0a9cF_cSx
uNSBqA9GklSYeGly7rEjO1nHjYw7zTiSLdV8EhraeDoPtKKEe8Ytq1jcm15_V_P2ey7CoFMHxQ9n7Jzw62AcQSgwbL8iAh7h_67v3A-n2OVFLPbosjSxks4
AaAeW4AndInL5DWv-v_T02WSgHT-cLPbwt8ueqJQ02IDQhbvVFXu4Z3sgdpc2yslBhoW_tVIoG9H74O9fsWgMSiMV_7TNRXYSqzF1xUJqHRfS3hrQdB6
TMRiOFiIzCXAcOXwLFhGApsp8-NPkCljF1yXJ07wUwr-lE2vBhWEGAqjKSPYZXtTB5FHhffp-GtdrgkBwhmNEDYw6LyJ-BClZ1u58dpOkFohgKuXD3f5aHy
1XyJvBHhdQc53lszKGU096EANua1KEYNcM8MFBF8R4FDqj6auNl5CIO_iG7ZadMhqU-lxEa3XuVISMtZZz3PNG60YeqoR9fZWq4Hu1BZe09MraKqejKG1
7GEKRW_YVZ-MCZ_jOO3Z8_WOQUOBElS-9UkxRbWZf0W2Mnr73bpBbA_L-IHpqTwJdf2Xg5DXNkrzrTKHnExuVIKBs2T8VAyFazngOAZO89j7U4m4DZ
QxCkN5XB4_YadTkPFD5p3uDuDsvhVJnavC03N1S3OgQuQR8RutNECqghKFcLHwdgGAlGZWlMlkAEogYorMmwaNXqUo__lH48K40EVP49HDl0dorqm
DMKDBgYXwhDLLkdSjNJJVJlgkuXd-hhigBwwbdVA3tXoCPl-mhlIvvMENZ67vpB8ySVn56dVDyi0JpydCpDrRStA7PcAT4AMiaD1m3UOU-YLq51ijwvrxWm2
thAsmg&state=5Ld4sr0zRvKTZ6n461Ia8k0HbEwNCZkfrTbMHr6uRIc&session_state=11f7bc1d-ac8b-4fbf-a0ed-24a5a865fa9d HTTP/1.1
receivedRequestHeader: Host: localhost:8080
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Sec-GPC: 1
sendingResponseHeader: HTTP/1.1 200 OK
Content-Type: text/html
Connection: close
Content-Length: 160
<html><head><meta http-equiv='refresh' content='0;url=https://www.chilkatsoft.com/oauth2_allowed.html'></head><body>Thank you for allowing access.</body></html>
.. the user granted authorization, exchanging the "code" for an access token ..
.. exiting the OAuth2 background thread that waits for the redirect request from the browser on this localhost ..
We may need to wait for more information from the library developer as to whether there are any clues here to explain the failure status.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3573622 Nov 2202:14 PM
HOLD EVERYTHING: After adding the new app, I reran the test program and this time it worked!
I got this token: {"token_type":"Bearer","scope":"email openid profile https://graph.microsoft.com/Mail.Send","expires_in":5392,"ext_expires_in":5392, "access_token":"eyJ0eXAiOiJKV1Qi ...
I then tried to Test Emailx (option 9), but so far it isn't working. I keep getting A-Shell popups like the ones below. They're obscuring the authentication problem. How do I fix them?
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3573722 Nov 2202:30 PM
I think the EMAILX problem is that I am testing with SMTP and I did not include that in the OAUTH2 app's API permissions. I will go back and add those and then retest.
No go. Even after adding back the IMAP and SMTP permissions (and updating the scope in the JSON file), EMAILX times out instantaneously. (The config file has WAIT=60)
QUESTION: The access token I received is 2203 characters long. Can EMAILX handle such a large password?
Last edited by Steven Shatz; 22 Nov 2202:43 PM.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3573822 Nov 2203:04 PM
Good point. The prior limit was 1024 (more than enough for GMAIL SMTP OAUTH2) but here's a link to an updated version that removes the limit entirely: emailx142.zip
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3573922 Nov 2203:22 PM
On the LIBASHTLS problem, please upgrade to this version: libashtls-216.zip (unzip it on top of the old one in the bin directory).
The other problem with the EMAILX test routine is that it assumes the existence of an EMAILX.CFG file in the current directory. Mine looks like this:
Code
WAIT = 10 ; max wait (secs) for SMTP response
WAIT2 = 500 ; secondary wait (milliseconds)
SERVER = smtp.gmail.com:587 ; SMTP server
SECURE = STARTTLS
SECUREHELLO = 1
HOST = microsabio.com ; ID for HELO (identifies sending machine)
DOMAIN = microsabio.com ; default domain to add to unqualified add
AUTH = XOAUTH2
USERNAME = microsabio@gmail.com
RTNADDR = return@microsabio.com ; return address (default FROM address)
REPLYTO = replyto@microsabio.com ; default Reply-To address
LOGFILE = EMAILX.LOG ; log file name (amos or native spec)
LOGLVL = 5 ; 0=off, 1=errors only,
Of course that is for GMAIL's SMTP server, but in theory you would substitute in your office365 SMTP address and your USERNAME. The password will be plugged in from the access token. Your first error message seems to suggest that you're requesting a printout of the emailx.log file, which apparently doesn't exist (because you didn't get that far with EMAILX.SBX). But I'm not sure why or what was initiating that print request - my version of the test program only offers to view it using XCALL EZTYP. If you don't like the APEX stuff getting in the way, you may want to go to Settings > Preview Preferences > Default=Off
Last edited by Jack McGregor; 22 Nov 2206:08 PM. Reason: fix link
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574022 Nov 2203:27 PM
FYI I have to pivot to something else for the next couple of hours but if it seems like approach is going to be viable, I'll create an SBX version of the test routines that can be called from Linux (either tonight or tomorrow).
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574122 Nov 2204:42 PM
Before testing EMAILX I replaced your EMAILX.CFG with the one I normally use. So that isn't an issue.
Your link (above) for libashtls-251 is invalid, but I retrieved and installed libashtls-215 from the development site.
I don't know why I'm getting these popups, but my Preview Preference setting is already "Default=Off". I stopped getting the Print File error popup, but when I test EMAILX (option 9) I get four TCPX.SBR ("unable to load library libashtls.dll) popups in a row, while the screen shows: "Sending message...". Thus, it looks like these are coming from EMAILX.SBX.
I also made sure that the version of EMAILX.SBX in BAS: is v6.5(142).
The big problem is this: If I include IMAP and SMTP permissions in my app and scope, I get error -42 when I try to get an access_token. If I omit those permissions, I get the token, but then EMAILX immediately aborts with status 72. If I reinstate those permissions and call EMAILX with the token (that I obtained without those permissions) I get the same abort with status 72.
I think I've done enough today. I'll pick this up again tomorrow.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574222 Nov 2205:05 PM
Once installed, the properties should show version 1.3.216 and 1.14.187 respectively. Note that if you already have an instance of A-Shell running, you'll need to close it and then unzip the file to make sure that the dlls weren't previously in use.
As for the rest of the details, let's go through it again. Status -42 is "Denied" which is different that we were getting before. And EMAILX status 72 is "timed out", which is somewhat unexpected (looking at the EMAILX.LOG file might clarify if that was due to it rejecting the first attempt and waiting for another, or something else.)
As for the unexpected APEX operation, the best way to debug that would be to activate the XCALL and/or LP traces so we can see what print-related operation is being called.
And thanks for the document link!
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574422 Nov 2206:37 PM
Reading that Deprecation... document, it's not completely clear that you really need to do this...
Quote
We're also disabling SMTP AUTH in all tenants in which it's not being used.
(What if it is being used?). And this...
Quote
SMTP AUTH will still be available when Basic authentication is permanently disabled on October 1, 2022. The reason SMTP will still be available is that many multi-function devices such as printers and scanners can't be updated to use modern authentication. However, we strongly encourage customers to move away from using Basic authentication with SMTP AUTH when possible. Other options for sending authenticated mail include using alternative protocols, such as the Microsoft Graph API.
... appears to indicate that they may never be able to require OAUTH2 for all clients. Not to suggest that there's any fundamental reason why we can't make it work, but it raises the question of what kind of authentication you were using that is at risk of being discontinued. To add to the confusion, they are using "SMTP AUTH" in a sense slightly different from the "AUTH" parameter in EMAILX (which we continue to use even for XOAUTH2). I think they are referring to the standard AUTH options such as PLAIN and LOGIN. Is that what you were using?
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574522 Nov 2207:01 PM
As far as I know, "SMTP AUTH" simply refers to SMTP authentication. (The original SMTP protocol didn't have any authentication, presumably having used the US Postal curbside mailboxes as the model). I think that the use of "AUTH" all in caps rather than "authentication" is a nod to the SMTP message that the server sends to the client to let it know what kind of authentication is supported. You can see this if you active LOGLVL=5 in your EMAILX.CFG file ...
The above sequence shows the start of the STARTTLS sequence, which causes the SMTP server to start over again in TLS mode, after which you can see that it reports 250-AUTH listing several authentication modes (LOGIN PLAIN XOAUTH2...)
Microsoft's literature referring to the deprecation of SMTP AUTH seems confusing, given that the recommendation is to use OAUTH2, which is in fact one of the protocols supported by SMTP AUTH. The Wikipedia article on SMTP Authentication only seems to confirm that. On the other hand, the reference to "Basic Auth" in the title of The Official Basic Auth Deprecation doc does perhaps help clarify that what they really mean is the basic variations of authentication that involve the simple use of a static username and password. In any case, the doc further reiterates that they are not attempting to apply this new deprecation regime to SMTP, so ....?
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574723 Nov 2202:12 PM
As you can see, our 250-AUTH line shows only LOGIN and OAUTH2. That's a lot fewer protocols than in your example. Why should that be the case? And does it matter?
My in-house EMAILX log is almost identical with the same 2 protocols. Is that a function of Office 365?
I emailed the IT group again. If they schedule a Zoom call, would you want to be included?
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3574823 Nov 2202:37 PM
The list of AUTH protocols declared by the SMTP server is presumably configurable on the SMTP side. So it's not surprising that your Office 365 server is limiting the protocols (PLAIN uses plaintext passwords so isn't secure, and XOAUTH has been mostly replaced by XOUATH2; I'm not even sure what PLAIN-CLIENTTOKEN OAUTHBEARER are but they're not supported by EMAILX.)
It's also not surprising that the log looks nearly the same for LOGIN and XOAUTH2, since it doesn't show the password, and other than the preliminary machinations to obtain the token which is then used as a password, the handshake with SMTP is essentially the same.
Regarding this question...
Quote
How can I find out if EMAILX's username/pswd authorization technique is SMTP AUTH and that it doesn't need IMAP (which is being disabled)?
... EMAILX doesn't have anything to do with IMAP, which is a protocol for managing your message in and out boxes. (It differs from POP in that all the messages are maintained on the server, and simply download as needed to the client agent, whereas with POP everything is maintained by the client agent, i.e. Outlook or Thunderbird.) As for whether EMAILX is using AUTH LOGIN or AUTH XOAUTH2, it comes down to what you specify in the AUTH directive of the EMAILX configuration file. If you specify something it doesn't support, you'll get error status 78 (syntax error in config file). You could also confirm that it is doing something different by using SET DEBUG (which will spit out a lot of traces).
I'd be happy to sit in on the Zoom call, assuming the scheduling works.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3575525 Nov 2202:18 PM
I just retried emailing from my version of your OAUTH2 test program (i.e., option 9) after getting a token (without SMTP or IMAP scope). I keep getting four popup errors in a row (The first time: 1 "Load library" error and 3 "TCPX.SBR" errors; subsequent times: 4 "TCPX.SBR" errors)
Before testing, I re-downloaded the latest libashtls module from libashtls-216.zip and after unzipping it, I copied the resulting libashtls.dll into %MIAME%\bin (replacing an identical copy). I exited A-Shell Windows and restarted it. Every test I did, I got four darn popups.
Suggestions?
Last edited by Steven Shatz; 25 Nov 2202:21 PM.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3576830 Nov 2207:08 PM
Sorry, I seem to have overlooked this latest question. So clearly it's not able to load the libashtls module. Assuming you have the libashtls.dll in the bin directory as you say you do, then my next best guess is that it is missing one of its GNU dependencies. Try unzipping the contents of libgnutls-30.zip into that same bin directory.
Re: How To Implement EMAILX & OAUTH2 from Linux to Exchange
[Re: Steven Shatz]
#3576901 Dec 2208:49 AM
I copied in the 5 files, but still get the same error.
It looks like my %miame% environment variable isn't set correctly in A-Shell Windows. I created one under Windows itself and if I type "dir %miame%" in a Command prompt window I see a list of files. But when I type the same command in A-Shell Windows I get: "?Device not found or not mounted - C:".
Here is the command line I am using to invoke A-Shell windows:
DIR.LIT has never supported listing native directories, so "DIR %MIAME%" isn't going to work. So that's not the problem. (Now that you mention it, perhaps that would be a nice enhancement. Although DIR is really designed for AMOS-like directory listings; if you really want to see a Windows directory, probably "EXPLORE %MIAME%" would be what you want.)
Note that VUE supports native specs, so "VUE %MIAME%\miame.ini" should work, as should "VUE %MIAMEFILE%",
As for the original problem, if you have the dll in the bin directory but it can't be loaded, it's most likely because there is a missing dependency. I good tool that I've used to identify such missing dependencies is called, well, dependencies. I suggest downloading it, then in its Dependencies_x64_Release folder launch the DependenciesGui.exe and use it to open (or drag drop) the libashtls.dll file. You should get something like the image below, although probably with some warning indications (red or orange?) of missing components.
