This is long overdue, but in the course of working on the (probably infamous by now) IFE(expr1,expr2) feature documented in the thread IFELSE, IFELSE$ functions improvement suggestion, I've decided it's time to implement a means for the compiler to embed into the RUN file the minimum acceptable A-Shell version needed at runtime. Otherwise, it's just too easy to use a new feature and then send it to someone running an older version that won't recognize the run code, resulting in a cryptic runtime error (e.g. "illegal user code", "illegal function value").

We could insert this information into an appendix at the end of the RUN, but that seems like overkill (overly complex) just to embed a single value (the minimum runtime edit number). (The appendix mechanism is more suited for significant appendices, like the dynamic function table.)

So my thinking is to recapture some unused space in the existing RUN header that is now used for storing the total size of the embedded DATA statements. Currently it allows 32 bits, but my thinking is that:

a) DATA statements are hardly used at all, and

b) Even when they are used, it's doubtful the total amount exceeds say, 32K? 64K? 128K?

Since the we've only burned through 1657 edit numbers (latest about-to-be-released version is 6.5.1657.0) in 20+ years, 13 bits (up to 8192) should be sufficient until we reach the Singularity, or go back to paper and pencils, whichever comes first. That leaves 19 bits to handle up to 512K of embedded DATA.

Before I act on that plan, does anyone have any concerns about setting a 512K limit on embedded DATA statements?

To further clarify how this will work, the plan is to introduce a new header program format signature (0xF2--, up from the current 0xF1--), which will signal the new interpretation of the DATA size longword. Older versions of A-Shell will simply refuse to run such a program, aborting with either a "Bad program header" or "RUN file incompatible" error. (Note that this signature is reported by VERSYS.LIT; it will also need to be updated though to report what the new signature means and what the minimum version required is. We might also consider adding a new /MINVER switch to DIR to report this information for RUN programs, along the lines of the /MEMREQ switch.)

But, in order not to completely break backward compatibility for programs not using any newer features, the new compiler will only use the new header format/signature when the program contains a relatively new feature requiring runtime support. I'm not yet sure where to draw the line, but probably it makes sense to start with any feature added in 6.5.

For example, if you use the new compiler to compile a program that contains the feature added in 6.5.1629 to INPUT CSV directly into a dynamic array, then the compiler would emit the new header and plug in (at least) 1629 as the minimum version, which means that existing A-Shell versions, even if beyond 6.5.1629 and therefore theoretically capable of running that code, will now refuse to run it due to the new signature which no existing A-Shell version will recognize. That will force you to update to 6.5.1657.0+. But for programs which do not use any new features added in 6.5, they'll continue to compile with the old header and be compatible with existing versions.

Going forward (i.e. 6.5.1657+), the runtime will be able to report exactly the minimum version needed in the case of an attempt to run a program containing a feature known to require a version beyond the current version (e.g. "Program requires
minimum A-Shell level 1671")

Sounds OK in theory, if you can keep backward compatibility best you can for the majority of features new and old.

I am pondering if some where I've used something like "INPUT CSV/dynamic array" in a program where such a new feature in part of the program is enable and runs on newer Ashell versions but the same RUN file is also is sent to older AIX/Ashell customers but such a feature that uses this is disabled (program checks the Ashell version) - I do this a lot for dynamic array's etc (but not necessary that often along with INPUT CSV)

That's a good point, i.e. it's entirely possible to compile into a RUN a feature that isn't compatible with the run time version, but which doesn't cause any problem as long as you aren't actually using that bit of code in that environment.

The /RC switch had a bit of that logic behind it, i.e. it allowed you to create supposedly AMOS-compatible RUNs which nevertheless may have contained features that really weren't AMOS-compatible (on the theory that the developer was smart enough to disable the use of those features under AMOS).

I hate to keep throwing more switches at the problem, but maybe that's the only way. For example, a /BC compiler switch for Backwards Compatibility which disabled this new header signature feature?

Whats one more switch

I may be a lousy negotiator, but you convinced me.

A little late to the party here but to Steve's point, just to f/up... if we are running 6.5+ on our development system and the customer has 6.4 runtime, would it work out of the box? Or would we be required to add the new /BC switch?

I like the idea, but before as was mentioned as long as we werent using an incompatible feature the older runtime was oblivious and was fine. On the other hand, i like the idea of making sure that all sides are on the same sheet of music.

I had second thoughts about the choice of /BC, partly because of the suggestion of ancient compatibility (which grossly exaggerates the truth), and partly because it of the possible misunderstanding of what kind of "compatibility" was on offer. So I decided to go with the more cryptic /F1, as explained here in the (as yet unreleased) release notes...

For example, here's my test program for the brand new IFE(a,b) function, compiled with and without /F1 ...

(Note the 0xF176 vs 0xF276 "Program Format", aka header signature.)

So yes, you would be required to add the /F1 switch IF you are developing under 6.5.1657+ (compiler 986+) AND using any new features introduced in 6.5, if you want to be able to run them under 6.4.

Considering the risk you're taking (of accidentally trying to execute code that won't make sense under 6.4), it seems appropriate to make you figuratively sign the disclaimer by adding the /F1.

Thanks, understood.

So perhaps a dumb question -

If you are embedding this new data in the header of the run module, and we send it to a 6.4 system, isn't going to just ignore this data anyway? Even w/o the /F1?

That would have been the ideal. The problem is that the original RUN header format didn't provide any unused space for things like this to be added. So my options were:

a) Add it as an appendix to the end of the RUN (ruled out as too complex or at least overkill for just a single value)

b) Increase the size of the header to make room for this (and perhaps a few extra bytes for future uses). Doing this will render any such RUN unintelligible to earlier versions, since they expect the object code to start at a specific offset. So it would require the new signature (0xF2??) so that older A-Shell runtimes would realize that they don't understand that RUN format. And after some exploration of this approach, I realized that there were a number of hidden areas sprinkled throughout the compiler and runtime which were based on the initial offset being a fixed value. That's when I decided on ...

c) Re-purpose part of the header that didn't seem to be currently used to its potential, i.e. the 32 bits reserved for the total size of the DATA area. As mentioned above, I doubt that anyone uses DATA for more than very simple purposes (many of the XTREE test programs, for example use this approach to combine sample data with the program itself). So I figured that limiting the DATA size to 512K wouldn't be an issue for anyone, thus freeing up 13 bits (8K) for the minimum version. This approach still requires the special header signature, because otherwise the program loader will detect that there is a mismatch between the header and the actual RUN module. But at least it doesn't change the starting point, which seemed like an attractive simplification.

Here's what you'll get if you try to run such a program under an older A-Shell...

That's actually not the error I was hoping for ("unsupported program format") but it turns out that the analysis of the validity of the size fields in the header preempts the complaint about an unrecognized signature. The wacky objsiz in the message comes from the fact that it combines both the true object code size with the DATA area size, and the DATA area size now contains the minimum version in the upper 13 bits, making it seem like a huge DATA size to the unsuspecting older runtime interpreter.

That could possibly be cleaned up via a 6.4 update, but I'm not sure it really matters in the application environment exactly what the error says, since the user isn't going to be debugging the problem anyway. In any case, I think this is much better than having programs stumble into code that the interpreter doesn't understand, deep into some process where the error is harder to diagnose and/or recover from.

If you want to quickly scan all of your programs for the new header before sending them out to customers, we could add a /LIST switch to VERSYS, which you could then easily scan for the 0xF2 program format.

FWIW, here's a test release containing the above-mentioned compiler update (compiler edit 896). Note that you'll need to execute UPDCUR to update the COMPIL.LIT, OCMPIL.LIT, COMPLP.LIT and VERSYS.LIT programs also.

ash65notes.txt
ash-6.5.1657.0-w32-upd.zip
ash-6.5.1657.0-w32c-upd.zip
compil-6.5.896-w32.zip

ash-6.5.1657.0-el7-upd.tz
compil-6.5.896-el7.tz
ash-6.5.1657.0-el5-upd.tz
compil-6.5.896-el5.tz

Hi,

While searching for the version where the XTREE self service cell problem started, I found that programs compiled with the 6.5.896 compiler don't run before 6.5.1657.0 A-Shell release.

This was easy to find because I was working in my "login" module and received the message below on my first test using a prior A-Shell version.

I realized what the problem was but didn't mixed it with the other subject before confirm the details.

My question is, wasn't this supposed to happen only when the program makes use of something not compatible with the A-Shell version?
That's not the case here because "login" is very basic and definitely don't use any super new function.

Hi Jorge,

I believe if you want a version that doesn't have the new program header (which isn't known to older A-Shell versions) you have to compile with the /F1 switch.

Hi Stephen,

But, what to do if I want the new header and still be able to run my programs on previous versions?
Anyway, maybe I didn't have done all the investigation because, I believe to have compiled a program this morning and delivered to a customer that is not using the newest version and he didn't complained.

Maybe another thing to entertain me in the flight tonight.

I'm not sure how well this can compete in the entertainment category vs. the in-flight movies, but here goes...

There's definitely a gray area for programs compiled since 1657 and run on earlier versions of 6.5, which sounds like the scenario you (and many others) either will have or already do have.

In that case, the compiler is likely to flag the RUN as requiring some 6.5.xxxx edit level, where xxxx might actually be less than the runtime level (and therefore theoretically compatible), but still won't run because the older version doesn't understand the header.

The workaround is to use the /F1 (or -f1 for APN) switch as Stephen suggests. But to answer the question of why the compiler thinks the program requires the new header, you have to use VERSYS on the RUN to see what the minimum edit level is. There are the main triggers:

1624 - .CLEAR on a non-collection array
1638 - INPUT CSV into an auto_extend array
1647 - DYNSTRUCT, DYNAMIC functions
1650 - addition/subtraction on entire $map()
1657 - IFE()

Bon voyage!

I forgot to address your question about how to get the new header and still maintain backward compatibility. Unfortunately there is no way to do that, because the new header renders the program unintelligible to older runtimes resulting in the error you got.

(It's this exact problem which caused me to delay so long doing anything about it, i.e. couldn't come up with any way to make older versions of the runtime "understand" what was being foisted on them. But I finally decided better to start somewhere. I think you're going to have to use the /F1 switch, at least until you get all of your customers updated to 1657+, and then you can remove that switch. Or, stick with compiler 894 until then.)

Are these all or just a few things that active the new header?
1624 - .CLEAR on a non-collection array
1638 - INPUT CSV into an auto_extend array
1647 - DYNSTRUCT, DYNAMIC functions
1650 - addition/subtraction on entire $map()
1657 - IFE()

oh! definitely was the IFE() that triggered the error.
Ok, no problem, if they are not all updated it's just because I didn't do it already but, that I can't do during my flight or, at least, can't do it w/o get bankrupted :p
thanks

I think that is the complete list, at least so far. But it's possible I overlooked something. Note the VERSYS will indicate the minimum level, so if you see something outside of the numbers listed, that would be for another feature.

Thanks