I saw in the release notes for 6.1.1418.4 that compil.exe now does the tagger outputting. I was going to test this, but didn't see that compil.exe available for download. Is there an APN changeset coming, or can you post compile.exe? Are there any changes to pn.exe for it to just use compile.exe?

I saw you pushed an update, and it is a lot faster loading the main .bas files. I'm not seeing any tags inside include files anymore.

Just curious if this post has been seen?

Seen by me

With APN 1.23 and greater there are no longer tags when editing include files. This also results in the new auto-completion not working when editing include files because they don't have any tagger related data.

Is this a settings issue, or something that's not working because the compiler is outputting the tagging directly?

Jack, it appears your notification system hasn't been working. Posting this to resurface this inquiry.

Indeed, it seems that I'm only getting about 60% of the notifications. Not sure why or what broke, but I need to do a better job of checking manually until I resolve it.

And in the case of the tagging, you're right there too. I think we have gone back and forth on this issue, and I apparently forgot that we had last left it in the mode where the individual include files were tagged as if they were program files. This approximately doubles the amount of tagging work, because each tag shows up associated with the include file, and also with the parent program. Depending on how much of your code is in include files, this is obviously going to be a significant factor in the overall tagging spend (probably about 2X).

Perhaps before trying to put it back the way it was, we can review the motivation and for tagging the individual include files? Since the main program entry in the tag panel lists all of the tags, and you can jump from any of them directly to the corresponding source file, what exactly is the advantage of listing the tags separately for each include file? Do you use that as a way of browsing the contents of an individual include file? Or does it just reduce the number of tags to a more reasonable amount?

Also, am in correct in remembering that when tagging the individual include files, we don't want to nest them? i.e., show only the tags physically in the include file, rather than all the tags in all the nested files (as we do with main programs)?

Unfortunately, putting it back the way it was isn't going to be trivial, because of two reasons:

1) Improvements in the tagging selectivity, such as eliminating local variables, requires using the compiler, rather than the original tagger (which only understands line-oriented REGEX patterns and thus can't distinguish local from global MAP statements). Merely adding a switch to use the new compiler-based tagging for main ASB programs and the old generic tagger for others will cause us to lose that improvement. It would be better to modifier the compiler tagging logic to output both sets of tags (main program and include file) during the same pass.

2) The autocomplete and call tips features are based on the tags, so there will need to be some additional logic there to filter out the include file tags so the duplicates don't mess up the autocomplete/call tip database.

The compiler based tagger is much faster for main programs than the ctags.exe is, so that's a huge win. In some cases of old large programs ctags.exe takes 10-plus seconds versus 1-2 for the compiler, so yeah huge win. Also, I find with the new version that if I save an include file it gets tagged, and auto-complete works in the include file. When I edit a different file and go back to the one that was tagged; it's tags disappear, and auto-complete doesn't work anymore.

Yes this is what we originally settled on. It's very handy to be able to browse the tags for just the file your currently editing to find a particular function, or find that it doesn't exist.

I'm fine with whichever approach you think best, but I think performance is probably a huge key in trying to successfully convert old VUE users to use an IDE.

I'm going to experiment with adding include file tagging to the compiler. In theory, it should just be a matter of essentially outputting each tag line twice - once for the main file and once for the local include file (assuming the line came from an include file). That will obviously increase the amount of tag file output (with 2X being the worst case), but should have a much smaller effect on the overall time for the compiler-based tagging operation (since it won't increase the time involved in parsing the source).

When loading files into APN, I think the include file tags can be ignored until such time that you actually open an include file for editing, so hopefully that will have little or no effect on the program load time. (All bets are off though if you save a workspace containing many include files and then reload the workspace later.)

I'll look into the problem of the tags disappearing if it still exists after the update. (But the update might take a few days - obviously any changes to the compiler require special care; this is the main downside to using the compiler for tagging, rather than a separate utility.)

As for what effect performance has on converting old VUE users, it's hard to say because we don't get much feedback from those who haven't converted. In fact, as far as we can tell, once people try out APN, they're converted. The main problem is with people who don't want to try something new, either being convinced (probably mistakenly) that nothing could be better than VUE, or fearing that learning APN will take more time than they can afford (equally mistaken).

Anyone in that category should take a look at the introductory video on the APN videos page . And on a related aside, although not mentioned in that video, APN now supports a VUE compatibility mode, so you can keep most of your VUE control commands if you really want them.

Sounds good.

As always reading with interest (and can't touch VUE since the Ashell editor and then APN came along turbo charging programming and output and money..)
Anyway, I may of miss understood but will the new fantastic auto-complete not work in the main program if the function is in an include and you do this:
"When loading files into APN, I think the include file tags can be ignored until such time that you actually open an include file for editing"

If you're confused, join the party. The more I look at this, the less sure I am of what really makes the most sense.

As it stands now, only main programs are tagged, but that includes the entire contents (i.e. all of the include files that make up the main program.) And I don't think anyone wants that to change. So there should be no problem with auto-complete working for any function included anywhere in the program, whether or not that include file is currently open for editing.

The feature that's missing (that we used to have) is the ability to see include files in the tag panel and browse just the tags for an individual include file. (Right now, all the tags are consolidated under the main program; the only way to tell which file they came from would be to double-click on one.)

But I'm trying to figure out how to implement this without losing all of the performance gains we got when the the compiler started doing the tagging.

Yesterday, I was thinking that when you opened the main file, the tag panel immediately showed sections for all the include files, even if those include files were not actually opened, i.e.



But after reviewing an older version, I see now that I was mistaken, and it only showed the tags for files that were actually opened. For example:



In the above image, aside from the main program plxsv1, we see the tags only for fnfqfs.bsi, because that is the only include file open. So in other words, the separate tagging of the include files is driven by opening the include file for editing/viewing, not by opening the main program that includes the file.

This is probably a good thing for performance, since we don't have to generate all those separate sets of tags all at once like I was previously thinking. We only need to do it when the include file is opened. But there are still a number of complications that I'm struggling with.

I posted a 1.0.25 update to the APN repository which I think restores the old tagging behavior for individual include files, while keeping the new behavior for the consolidated tags associated with the main program.

OK, Thanks for the non-confusing explanation Must admit I seldom look at the Tags Dialog and open an include this way.

Works great now. Thank you.

One minor issue. I open main program and then open an included module. At this point I can auto-complete everything from the main program in the include file. When I make a change to the include file and save the auto-complete now only works for whats in the include module, even when editing the main program. I can then change and save the main program and auto-complete goes back to being based on it. It seems that the auto-complete is based on whatever file was saved last.

Hi guys, apologize for my lack of participation on this very interesting discussion but, I don't want to participate w/o have played enough with this and add more confusion but, as soon as I finish a few things here, I'll catch you.

That problem has more to do with how APN rebuilds the auto-complete/call-tip database when opening/closing/switching files, than with the tagging itself.

The raw tagging (what you see in the tag auxiliary panel) is relatively straightforward and static. The tags are loaded when a file is loaded, and discarded when the file is discarded. When switching focus between files, the tag panel automatically opens up the tag section for the current file. This latest change merely added separate tagging for include files (which previously were only tagged indirectly by their inclusion in a main file). So they didn't show up separately in the tag panel; now they do.

The auto-complete/call-tip database is built from, but otherwise independent of the tag database, and it is anything but straightforward. It's not even clear what the ideal objective is.

Consider the following files open in the editor:

A.BP (includes A.BSI, C.BSI)
B.BP (includes B.BSI, C.BSI)
A.BSI
C.BSI
X.BSI (not included by either)

With the focus on A.BP, do we want the auto-complete/call-tip database to include tags from all of the open files and all of the associated includes? (Call that the GLOBAL option.) Or just the tags associated with A.BP and its includes? (The CURRENT CONTEXT option.)

In the former (GLOBAL) case, a switch of focus to any of the other files would have no effect on the database. But each time we open or close a a main program file, or an independent include file, or perhaps a non-ASB file, we would probably have to rebuild the database for all of the open files and all of the include files for each main program. (We'll ignore for a moment the question of how to distinguish whether a non-main program is associated with or independent of any of the other files.)

But in the CURRENT CONTEXT case, switching the focus to another file would require clearing and reloading the database relative to the current context. (More work when switching focus, but less work when opening/closing files.)

And within the CURRENT CONTEXT case, it still isn't quite clear what we want when the focus is on an include file. Since C.BSI is included in both the main programs A.BP and B.BP, does that suggest that we'd like to have all the labels and functions of both those programs available to us when working within C.BSI? Or what if we are working in A.BP, and we use the Goto Definition command on a function call to jump to (and open) A.BSI? Should we immediately lose auto-complete visibility of functions in A.BP or C.BSI which were just available an instant before? And then we hit the Go Back command and immediately regain them?

Both scenarios have the pros and cons. Offering auto-complete suggestions that are not really part of the current program context seems less-than-useful. But since one of the benefits of APN is easy and fast switching between file contexts, I'm not sure we want to slow that down by forcing a rebuild of a database potentially containing thousands of items on every context switch.

It might also be the case that what makes sense for auto-complete doesn't necessary apply to call-tips, even though they use the same database. (Offering call tips even for functions not currently reachable in the current program context seems more benign than offering auto-complete for those unreachable functions.)

Currently APN doesn't do anything about the database when switching focus. Whatever it does, it does when opening or closing files. But the current end result is some kind of mash-up of GLOBAL and CURRENT CONTEXT modes. It needs a thorough reworking, but I need to get it clear in my own head what we really want before diving into it. (The code is complicated enough by itself without adding confusion over the objective.) Suggestions welcome.

I posted a 1.0.26 update which swings the pendulum the other direction. Now it accumulates new auto-complete/call-tip database items as files are opened, but doesn't remove them until all the files are closed. So now you might end up being offered an autocomplete suggestion for a file that was previously closed, but perhaps that isn't much of a negative (especially compared to how it was before).

In a perfect world we'd have auto-complete where an include module would be completely aware of what functions/procedures available at compile time, and only those. But, we don't live in a perfect world, and we have hard problems to solve about knowing what is actually available to a given module at compile time. So, I personally prefer to have too much auto-complete data rather than none.

Thanks for the fast response.

Couldn't have said it better myself.

But while on the subject, let me point out an additional factor which contributes to the possibility of "too much auto-complete data" - the asb.api file(s).

schemes/asb.api contains a fixed set of function definitions for the most common (in my opinion) SOSFUNC: routines, and these are added to the auto-complete/call-tip database without regard to whether the corresponding include files are referenced. (settings/asb.api contains an additional set intended for customization by each developer, i.e. your personal set of common library functions.)

The theory behind adding these to the database even before they are ++included into the program is that a common programming sequence of events works like this:

a) While writing some code, you determine a need for some routine which is probably common, say, to test if a bit in a bit field is set. But you're not sure of the exact function name, much less the include file name (and thus obviously haven't already included it).

b) So you take a guess, figuring it contains "bit" and call up the auto-complete helper (Control+Space) ...



c) Seeing the routine you want, you select it (TAB), and then type "(" to bring up the call-tips:



d) Because this tip is from the predefined asb.api, it is able to contain additional hand-coded information that the automatically-added call tips cannot; in this case, the name of the include file.

e) Now you can jump to the top of the program to add the ++include fnisbit.bsi, then use the Go Back (Control+Alt+B) to jump back to where you were and continue on. Of course it might be even nicer if the call tip display automatically put the "++include fnisbit.bsi" into the clipboard so you could just paste it - maybe something to add in a future update? Automatically adding the include statement to the proper place in the program, if needed, might be even better, but that would be quite tricky, especially if the current file was not the main program.

Anyway, the point is that "too much auto-complete data" is often just what you needed. The scenario just described could apply to the case where the auto-complete data came from an include module that was previously opened and then closed and is not (yet) associated with the current file. Knowing that it exists allows you to use it in the current context, even if you have to figure out how to ++include it later (perhaps when the compiler points out that it isn't defined). At that point, even if you don't have a clue, you can use the File-In-Files feature to scan the project or directory.

At least for me, this is how much of my programming energy is spent - trying to find existing reusable routines so as to avoid recoding the same logic multiple times (not only wasteful but often disastrous when you end up with slight variations scattered around an application). Anything the editing system can do to help with this kind of this is, well, very helpful.