In order to help you with your New Year's resolutions to upgrade the sophistication of your code, I've been working on some enhancements to the DYNSTRUCT mechanism that I could use some feedback on. The enhancements can be summarized as:
Allow DEFSTRUCT information to be embedded in RUN files, allowing DYNSTRUCTs to be used without having to distribute source code.
Support auto-binding of a DYNSTRUCT when passing a DEFSTRUCT object to a function receiving it as DYNSTRUCT
The motivation for the first one is the easiest to understand. The RUN file doesn't currently contain the names of variables or structures or even provide any way to identify which memory locations or variables are part of a group. Which is why the MX_DYNSTRUCT DYNOP_DEF function needs access to the source code of the DEFSTRUCT in order to gather up the information needed to support dynamic binding of a structure definition to a variable at runtime.
In order to eliminate the need for the source code at runtime, we need for the compiler to include additional information (essentially the same information visually represented by the DEFSTRUCT source code). But that takes up considerable space, especially if you like long field and structure names, and/or have common include files with large numbers of DEFSTRUCTs. It seems overly wasteful to include all that stuff in the RUN if you're not going to use it.
So the question becomes how to identify which DEFSTRUCTs to embed. Obviously, if there are no DYNSTRUCT variables defined in the entire program, there's no need at all for this. But the existence of a few DYNSTRUCTs doesn't necessarily mean you want to use this new method of binding with all of the DEFSTRUCTs. One possibility would be to add a tag to the DEFSTRUCT statement, allowing you to specify individual DEFSTRUCTs that you want to embed. Something like ...
Code
defstruct ST_FOO, EMBED
The problem here though is that if using common include files for your DEFSTRUCTs, you would have to decide whether any of your programs were going to use that DEFSTRUCT for dynamic binding, and if so, then all of your RUNs would have to embed it.
The opposite extreme would be to add a compiler switch, so that you could decide at the individual program level. But that's not especially convenient (having to change your compiler switches for individual programs) or precise (having to either embed all or none of the DEFSTRUCTs in a program.)
I'm leaning towards a pragma, something like ...
Code
++pragma EMBED_DEFSTRUCTS_BEGIN ! begin embedded DEFSTRUCTs into RUN...
defstruct ST_FOO
map2 name,s,50
map2 weight,f,6
map2 age,b,1
endstruct
++include morestructs.def
++pragma EMBED_DEFSTRUCTS_END ! end embedded DEFSTRUCTs into RUN...
...But even that doesn't solve all the shortcomings of the other methods, especially if you tend to put all of your DEFSTRUCTs in one common module.
Another possibility would be a pragma that allowed you to specify DEFSTRUCTs for embedding by name, something like...
Yet another possibility would be to just rely on the compiler to see which of your DEFSTRUCTs are being bound to DYNSTRUCTs and automatically embed just those. That could never work for programs that drive such operations from external data, such as a generic file viewer where you enter the name of the fie to view and the program looks up the corresponding DEFSTRUCT from a file. Or even for existing MX_DYNSTRUCT operations where you specify the DEFSTRUCT name in a string variable. But it could be useful in conjunction with the second envisioned enhancement: auto-binding.
The idea behind auto-binding can be split into two parts. The first is just to make it easier to bind a DEFSTRUCT to a DYNSTRUCT. Currently you have to make at least two calls to MX_DYNSTRUCT, one to define (compile) the DEFSTRUCT, and the other to bind it. There are already combined functions for this (see fndynst.bsi), and the the main source of complexity and potential problems involves compiling the source (which is nearly eliminated by having the compiler embed the information into the RUN), so I'm not sure too much to be gained by some kind of new language statement, e.g.
But one place where it's easy to see how auto-binding would be handy would be in a function that receives a DYNSTRUCT parameter...
Code
map1 mystruct, ST_FOO
...
call fn'foo(ds=mystruct)
...
function fn'foo(ds as DYNSTRUCT) as i2
<code that works on a variety of structures by focusing on certain common fields>
ds.amount = fn'xxx()
xputarg @ds
endfunction
So in other words, we pass an ordinary DEFSTRUCT variable (mystruct) to a function expecting a DYNSTRUCT, and somehow the compiler and runtime together manage to bind the one to the other so that the function can then operate on an already-bound dynamic structure.
The idea of a function that can handle a variety of different kinds of input structures (or objects) may seem a bit exotic, but I'm guessing that some of the clever programmers out there could find uses for it. The main requirements would be either:
The source structures all have a set of common fields that PartNo, Address, that the function operates on, possibly without even caring what kind of structure it is,
Or the function retrieves the layout of the bound structure (DYNOP_INFO) and uses that to either perform a generic operation (such as viewing any kind of structure data) or tailors the operation to the type of structure.
You could still accomplish this with the existing MX_DYNSTRUCT operations, but it would be a lot more cumbersome. Besides simplifying the code, auto-binding would facilitate a more object-oriented approach by facilitating both encapsulation and poly-morphism.
That definitely is the plan for those DEFSTRUCTs that appear in function calls where the receiving argument is a DYNSTRUCT (i.e. the last code example in my previous post). But the question remains whether that covers all the situations where you might want them embedded. For example if you wanted to distribute a generic file viewer (like ADDSVUE) but didn't want to distribute the necessary source, you'd want to embed all of your file-related DEFSTRUCTs in the RUN for that viewer. But it wouldn't necessary contain any code that would allow the compiler to recognize from the source which DEFSTRUCTs you were binding to.
Although I guess you could use a dummy function and calls to it as an alternative to some other method of tagging the DEFSTRUCTs of interest, i.e.
Code
map1 st1,ST_STRUCT1
map1 st2,ST_STRUCT2
...
call fn'embed'defstruct(ds=st1) ! force compiler to embed ST_STRUCT1
call fn'embed'defstruct(ds=st2) ! force compiler to embed ST_STRUCT2
...
function fn'embed'defstruct(ds as DYNSTRUCT) ! dummy function
endfunction
But that seems a bit like going the long way around the block.
I could see advantages to both of the ++PRAGMA approaches listed above. Are you leaning toward one or the other ++PRAGMA approaches, or would it be reasonably easy to support both?
i guess we could do both, although if I had to choose, I'd probably go with the ++pragma EMBED_DEFSTRUCT <structdefname> version -- it's a little simpler to code (no _BEGIN and _END issues) and also a bit more human-friendly when reading the code, even though it almost certainly requires more typing and a few more CPU cycles to compile.
Clearly it wouldn't make sense for ST_FOOBAR be embedded based on the above. (It might still be embedded due to its usage in a function call elsewhere though.)
But as to whether to embed DEFSTRUCTs whether they are defined, or instantiated, within the _BEGIN/_END, that would could work either way. (Probably safest to include both, as you seem to prefer, since the harm of embedding a DEFSTRUCT unnecessarily is a lot less than the harm of thinking you embedded it when you didn't.)
The main downside is just the possible confusion. I could imagine people having to look up the doc over and over again to be sure which of ST_FOO and/or ST_BAR would be embedded in the above scenario, whereas with the ++pragma EMBED_DEFSTRUCT <structname> syntax, it's pretty self-explanatory.
In order to use a generic file viewer like addsue with embedded DEFSTRUCTs it would seem to be much simplier to be able to use the _BEGIN/_END pragmas around all the DEFSTRUCT include statements.
I also don't feel too strongly about it. Whatever you think is best.
Sorry it's taking so long... I've been making incremental progress but have been reluctant to release partial updates. Basically the state of affairs right now is:
1) ++EMBED_DEFSTRUCTS_BEGIN and _END have been implemented; any DEFSTRUCTs between them are embedded into the RUN. I haven't decided yet whether to implement the ++pragma EMBED_DEFSTRUCT <structdefname> variation discussed above.
2) In addition to the explicit directives to embed DEFSTRUCTs, certain references to them will result in implicit embedding. This includes a reference to a DEFSTRUCT in a function call parameter list where the receiving argument is a DYNSTRUCT, as well as any DEFSTRUCT mentioned in the new .BINDSTRUCT statement (see next).
3) A new statement, .BINDSTRUCT ds, st has been implemented as a shortcut to MX_DYNSTRUCT calls to define the DEFSTRUCT of which st is an instance and bind the DYNSTRUCT variable ds to it.
4) The MX_DYNSTRUCT DYNOP_DEF call now supports setting the src$ parameter to "" to reference the embedded DEFSTRUCT information, eliminating the need for a copy of the source for the DEFSTRUCT at runtime. (This technique may still be useful as a transitional option, but is essentially superseded by the new .BINDSTRUCT statement which effectively combines the DYNOP_DEF and DYNOP_BIND operations into one.)
Yet to be implemented is the automatic binding of a DEFSTRUCT variable passed in a function call to the DYNSTRUCT variable received by the function in the following scenario...
Code
defstruct ST_FOO
map2 bar,s,10
...
endstruct
map1 grok, ST_FOO
call fn'foo(ds=grok) ! pass DEFSTRUCT instance grok to function
function fn'foo(ds as DYNSTRUCT)
ds.bar = "ice cream" ! (this requires auto-binding of grok to ds)
xputarg @ds
endfunction
To be honest, I'm still not quite sure of the best way to do it. It can almost be accomplished by inserting the equivalent of an automatic .BINDSTRUCT ds,grok statement into start of the function, but the function doesn't know anything about that grok parameter; it only knows about the ds parameter that received it. Furthermore the whole point of this construct is to allow a single function to support multiple structure types being passed to it, so we can't possibly embed any explicit binding instructions into the function. Instead, we need to add something to the calling parameter mechanism to perform the binding, or to the calling parameter structure to pass the necessary information to the function.
At this point the skeptical reader will be wonder how the function is supposed to do anything useful with the passed in argument without knowing its structure. To take the example above, how would it know that there was a member named bar? The answer, I guess, is one of two ways: One would be to assume that the programmer is careful enough to only pass structures that contain such a member, which might make sense if the application deals with a variety of structures each containing a common field, perhaps customer number. Although in this case the new mechanism would be vastly overkill if the function only dealt with one member, since it would be easier to just pass the member as an ordinary simple variable. So it would only make sense if several structures contained multiple common fields. The other possibility would be for the function to start out with a DYNOP_INFO call to find out what kind of structure it was dealing with, and then use appropriate conditional logic. But in that case the skeptic might still wonder how that would be a more useful programming technique than just creating multiple functions and having the caller decide which function to call based on which kind of structure it was dealing with.
So perhaps some further analysis of the objective here might be warranted before diving into this last step. In contrast, the value of being able to embed DEFSTRUCTs into the RUN is fairly obvious in that it eliminates the need to distribute the source and also simplifies the code.
There are a lot of cases where we have records that share the exact same behavior using the exact same field names. Yes, it would be possible to do this with function parameters, but a lot of times the number of members on the structure involved in the function is high enough that it is simpler to be able to pass the whole structure to work on those fields.
Another big use for DYNSTRUCT for us would be in our dialog generated XTREE code. I'd like to convert the templated code to use DYNSTRUCT for the XTREE array elements. Currently, there is a DEFSTRUCT templated based on the XTREE config in the generator UI, and this makes it unflexible in adding/removing columns at run-time. Converting the generated code to use DYNSTRUCT would allow for much greater potential flexibility for adding/removing columns at run-time. There are hundreds of generated functions per XTREE that receive an instance of an array element DEFSTRUCT. Changing these to DYNSTRUCT in the generated code wouldn't be too difficult. The issue is the hand-written application code that uses the generated code. Most of them don't need the flexibility that using DYNSTRUCT provides and don't need to change. As it stands we'd have to add a map of a dynstruct variable, def/bind it, and set the variable from the defstruct variable for calling a function with a DYNSTRUCT parameter, remember to update the defstruct variable on return.
This has been too daunting a task to tackle to this point, and having the run-time/compiler handle this seems like a big boon for making DYNSTRUCT almost equivalent in normal use to DEFSTRUCTS.
Although it is currently possible to base an XTREE entirely on DYNSTRUCTS (see the EXLIB generic file viewer utility ADDSVUE as an example), the new auto-binding mechanism should offer opportunities to simplify the source code related to building and validating an XTREE. But the details may remain murky, depending on the application program style.
One challenge related to XTREE is how to cross-reference the column numbers (critical to validation of editable trees) with the logic associated with the column. One way to do that, as you suggest, is to build a custom DEFSTRUCT for each tree. Having the compiler be able to auto-bind to DEFSTRUCTs that are included in the RUN will certainly be a simplification when dealing with structures that can be fixed at compile time, but if you want to be able to dynamically create structures at runtime, you'll still need to use MX_DYNSTRUCT to define them. And to really get the maximum flexibility, it seems like you'd want to create the array of field definitions (requiring a DYNOP_INFO call) so that you can then automatically link XTREE column numbers to field names and the logic that goes with them.
In any case, I have the alpha version working, but still need to do a lot of quality control before it will be safe to release as a beta. But we're getting close...
There should have been a new libashmysql.so.1.4.144 in the install package which I think solves that problem. (Same goes for the libashodbc.so.1.4.114 driver.)
I'm not really sure what broke the compatibility -- seems like some upstream component update. Beware that the newer connectors are equally incompatible with the older A-Shell.
As a reminder, the fastest way to check if the connector is available and compatible is to use the SQL:SQLTEST1 utility.
If you somehow didn't get them in the install bundle, you can also download the updated connectors them from here:
P.S. I tried to attach directly to the forum, but it doesn't allow .gpg files. Not sure if you have access to change the allowed extensions. If so, could you add .gpg thanks?
Last edited by Stephen Funkhouser; 15 Feb 2209:30 PM.
Ok, the file transfer/decode is working now. And I tracked down and fixed the cause of the segfault, but haven't yet figured out the underlying cause of the errors. It's related to the use of PRIVATE variables with the same names across several modules (e.g. ch'variables). Somehow the ++include stack is getting off by one, causing it lose track of the module that it returns to after un-nesting ++includes, leading to a bogus duplicate variable error.
The fact that the consolidated program over 200 thousand lines of code, with hundreds of ++includes nested 15-20 levels deep doesn't make it any easier to study! I'll figure it out eventually but it may take another day or so...
I haven't had a chance to study it further to determine if the error I'm getting is specific to the LSX or occurs equally in the the actual source code. (Feel free to test that yourself if you want.) Either way, I'll keep plugging away at the issue.
Apparently there is a bug in the LSX compile relating to the simulated ++include operation stack getting out of sync (which is still happening). I suspect there may still be an issue with the automatic embedding of the DEFSTRUCT/DYNSTRUCT definitions, but that would probably only become apparent if trying to use automatic binding on one of the affected structures. I'll keep investigating...
