It's probably not news to anyone that The Editor has been unsupportable for quite some time. It's a sad state of affairs, because it is such a magnificent tool, particularly compared to the existing alternatives (i.e. VUE, or some generic editor). But we have to move on, because:

a) Existing users are having increasing difficulty getting it to run on newer PCs.

b) We can't in good conscience recommend it to anyone new.

c) VUE, as beloved and entrenched as it may be, is a non-starter for any programmer new to the AMOS or A-Shell environment.

d) Even for those of us with hard-wired VUE circuitry in our brains and fingers, the loss of programmer productivity due to its many limitations is costing us dearly.

The idea of a replacement may have bubbled up periodically over the last year or so without any real action, we are finally serious about finding/implementing a suitable replacement (for both The Editor and VUE). Of course we can't force every die-hard to switch to the new editor (whatever it turns out to be), but our objective is for it to be essentially a no-brainer for the vast majority of A-Shell programmers to switch.

It's tempting to approach that objective by adding every feature that every single person asks for, but that's simply not realistic. Not only because of limitations on time and resources, but because there's an inherent trade-off between the number of features and the ease of learning/adopting/using it. As an example, Eclipse has a fantastic set of features and is free, but it's so big and complex that for most of us it would seem like driving an aircraft carrier to get coffee. (We suspect this is one of the reasons why more people didn't adopt The Editor - that its very comprehensiveness made it seem too big/complex/slow for the people who prefer small/simple/quick (i.e. VUE). So we think the success of this project is going to depend on our ability to make the right compromises.

The first compromise is that we're going to start with an existing editor/IDE rather than building one from scratch. On the upside, since editors are very complex, starting with an existing one is a huge head start, like starting a trip to the Moon from the Space Station rather than Earth. On the downside, we'll inherit all kinds of baggage and have to struggle to get it to understand the peculiarities of A-Shell and to adapt to its peculiarities. It's a compromise that could be debated but the overriding factor is that we don't have the time and resources to do it from scratch. Fortunately there are lots of editors/IDEs available to choose from, and we're currently evaluating a number of them, trying to determine how easily we can customize/extend them to get to our sweet spot/compromise zone.

Although we are certainly open to suggestions of which ones we should consider, what we think will be more helpful is to focus on identifying the core/minimum set of features and priorities needed to get you to switch from what you're using now (whether The Editor, VUE, or something else). Note that we expect the tool to evolve and improve over time, so it's not essential that version 1.0 be able to win a feature-comparison showdown with every other editor/IDE out there. It only has to be good enough that most of us can start adopting it with pleasure.

So this is an official call for feedback on what features, capabilities, characteristics, requirements, attributes, etc. that you need in an Editor (or VUE) replacement. Please resist the temptation to get bogged down in aesthetics, minutiae and exotica. (There will be plenty of opportunity for that after we've gotten the basic implementation up and running.)

To get the ball rolling, here are my own thoughts about the most important or typical features likely to be under consideration (starting from the larger/more important features and working down).

1) Standard Windows MDI editing: Nearly every editor (except VUE) has a basic set of capabilities that are essentially standard (multiple tabbed windows, typical Windows editing commands and menus, etc.) so this is a given. A few deserve special mention (for those stuck with VUE):

1a) undo/redo
1b) ability to mark a block and indent/unindent or comment/uncomment
1c) quick access to most recently used (MRU) files
1d) powerful search options (direction, regex, search-in-files, etc.)
1e) bookmarks

2) Integrated compiling: The ability to compile directly from the editor, see the list of errors in an output window and click directly to the file/location (including opening the include file if necessary). This probably requires some tinkering with the editor in question, and possibly modifications to our compiler to interface as required, but it is such a useful feature that it's a must-have.

3) Basic syntax highlighting to identify comments, strings, keywords. This is not only aesthetically pleasing to the eye but helps you maintain mental focus when looking at vast amounts of code.

4) Project Management: Some kind of basic capability with a "Project Explorer" to allow grouping all the files that go together in "projects" and more easily browse them. Project management features can get increasingly complex, but my sense is that the most important part of it is just the simplest part.

5) Workspaces: the ability to save your current environment (mainly the list of files/windows that are open) and recover it later. This makes it much easier to interrupt an editing session (for lunch, reboot, to work on a different project, etc.) and return to where you were later.

6) ++Include file handling: Now we're getting into an area where any existing editor will probably require signficant enhancement to recognize our ++include file specs (AMOS-style and ERSATZ) and search paths. So while the Includes Explorer in The Editor is a nice feature, it's a relatively expensive one to obtain/develop. It's notable that extremely high-end IDEs like Visual Studio don't even offer such a feature, although for another reason: to really do it right, the editor has to understand all of the compiler's conditional logic to determine whether a ++include was really in effect or not, plus the potentially enormous CPU as you add/delete ++include statements in programs with complex hierarchies of nested includes. The most essential feature is that the compiler error processor can locate an error in an include file and automatically open it and jump to it in the editor when compiling. (For the most part, the very idea of ++include modules is to hide details and thereby simplify the main program.) When I want to look at the contents of an include file, I typically don't mind just locating it in the Project Explorer, or even the overall Computer Explorer, although it might be nice to have an on-demand feature whereby you highlighted the ++include and it located and opened the file. It would also be nice to have a search program-and-includes option (like The Editor has), although the difficulty of knowing exactly what the compiler will do remains. (Making the alternative of having the compiler generate an LSX file and searching it seem like a reasonable workaround.)

7) Object/Tags Explorer (i.e. an auto-generated list of things like labels, functions, procedures, structures, etc.) that can be browsed and clicked on to jump the associated definition. Definitely a nice feature, but I don't use it that much, and I don't want to pay very much CPU overhead for it. My sense is that we can more-or-less get this capability as a standard-extension to most editors, at least on a file-by-file basis. (Organizing it on a program-plus-includes basis is better, but more costly.)

8) Text Clips (Code Blocks): a handy way to define, organize, and retrieve useful snippets of code that are used over and over again, such as function definition skeletons, program headers, XCALL examples, etc. (In VUE, I maintain a collection of such snippets in a special ersatz and just use the >Yank command to bring them in.) Modern editors usually make this slightly easier by allowing you to see a explorer-style list of available snippets and just click on them, or type a few characters and hit TAB or some other key to trigger it. Plus they may have special variables embedded.

9) Autocomplete: I have mixed feelings about this. I use it in C programming, but I turned it off in the Editor because it often was either just a bit too slow or otherwise got in the way. Often I find that when coding a complicated XCALL or function where autocomplete is handy, I also need to jump around in the code to decide on what to enter for a parameter, and doing that breaks or interferes with the autocomplete. So instead, I tend to prefer text clips to insert dummy versions of XCALLs and functions with many parameters and then I just replace the dummy parameters with real ones.

10) Keyword hyper-linking to docs: The ability to highlight a keyword in a program and hit or click something to bring up related documentation. Handy, but easily worked around (by manually switching to your browser/help window and looking up the keyword there).

11) Fast: One can argue that programmers may leave their editor/IDE open all day, so who cares if it takes 30 seconds to load. The counter-argument is that most of us have numerous occasions every day where we need to quickly look something up in a file. If it takes 30 seconds to launch the editor, or even 5 seconds to open a new window in the current editor instance to load the file, we're probably going to resist (and revert back to VUE instead). Another aspect of fast is the ability to open up a huge file without delays. Ideally the new editor should be fast enough to launch and load, even from a dot prompt command, that we use it for everything.

12) Keyboard mapping: In terms of editing commands, I don't think this is important at all, since it seems almost crazy to try to hang on to, say, VUE editing command keystrokes in a Windows editor environment. But I know this is something we did press Jorge to add to The Editor, so apparently some people think it is important. If this is the issue that keeps people from letting go of VUE, then let's hear about it and discuss it now.

Ok, that's my list. Just having the first 2 would probably be enough for me to switch from VUE, and by the time we get half way down the list, I'm ready to switch from The Editor. I don't expect everyone to submit a similarly wordy and comprehensive list, and I'm not interested in your pet exotic nice-but-not-essential features (at least yet), but some well-considered feedback on what you feel is most important will be essential for success here. (Where success might be defined as: nearly everyone adopts the new editor, becomes more productive, and we can move on to other issues.)

I don't really think I can add much of anything to all of that. I'm with you that I would switch for the first two.

Under 1d, I would like whole/partial word matches. I have carry over from trying to save space 30 years ago with such variable as Q. Would really be nice to be able to search for something like that.

Also, #4 would never be of interest to me.

I promise that whatever we come up with, you'll have the option of whole/partial word matches, as well as regular expression searches. For example [b]\bQ\b[b] will find occurrences of [b]Q[b] with some kind of boundary character on either side (equivalent here to the whole word match).

To start, let me comment your list.
Items #1 to #6 are vital to move from The Editor to the New Editor.
Item #7 is easily replaceable by a good search engine; at first was one of the musts of The Editor but, with the improves on the easy usage of the search engine, it became somehow obsolete. It's also one of the processes that slows down the Editor's launch.
Items #8, #9, #11 and #12; will not miss them if not available.
Item #11, really important to be faster than The Editor.

Now, here is my list of realistic wishes:
13a) Recognize functions, procedures and routines
Right-click and go to the corresponding positions, even if in an include file
13b) Recognize variables, defines, structures
Right-click and go to the corresponding mapping position
NOTE: These a) and b) requests seems to be incoherent with my comment above regarding item #7 because, The Editor, to achieve this, takes advantage of that startup procedure addressing all these items but, maybe the New Editor can be more efficient.

13c) view map of variable structures
very useful for building coldef xtree; righ-click in a variable that belongs to a map structure and display a window with the start/end position of each variable that belongs to the structure

14) insert TRACE/DEBUG
automatic build of trace statements with all the variables of the right-clicked line or, selected block of code

And,being realistic, that's it.

Thanks for the excellent feedback.

I took the liberty of assigning numbers to your points to allow them to be more easily cross-referenced with my numbers. (And I grouped your "view map of variable structures" with 13 since although the output is different, the most difficult aspect of the implementation is basically the same as for 13a and 13b, i.e. the ability search for a token across all of the related ++include files.)

Assuming we stick with the base editor that I'm currently leaning towards (PN2), 14 and anything similar can be handled with a Python script. It may be that at least initially, 13 will also have to rely on such a script. The advantage of the scripting approach is that it avoids have to modify the internals of the existing editor environment. The main downside may be that we can't integrate into a context menu, but instead would require that you click on the line or token and then double-click on the name of the script (in a side-panel listing the available functions) as in the picture below. How horrible would that be?

Not horrible and, a very good starting point.
Better than reach them using context menu, would it be possible to assign keyboard shortcuts?

This could probably be assigned number 43, but debugging with stack inspection would be really nice to have.

I must admit it be near impossible to work with out the current Ashell Editor and find the most important/useful features being:

1) Standard Windows MDI editing
3) Basic syntax highlighting

4) Project Management
or more the ability to enter a program name to search/edit and it searches a set path and brings up all the locations its found asking what one your like to edit.

6) ++Include file handling
(or in our case $COPY what works very much like the ++INCLUDE command)

7) Object/Tags Explorer
8) Text Clips
11) Fast

13a) Recognize functions, procedures and routines
13b) Recognize variables, defines, structures
13c) view map of variable structures I use this a lot.

also:

The ability to launch a macro (eg when compile or something is pressed) passing the program name and path as a parameter. (as the current Ashell Editor does) we use this to launch a small EXE I wrote that does the actual compile on the UNIX server with in an ATE session.

Forgot the creation of the .LCK lock file saying its being edited (so compatibly with VUE users)

Good point Steve, I also need that to copy the result file to another folder for Updatecenter purposes.

I'm trying to keep cool and not list all the dozens of features embedded in The Editor because, what we are trying here is to get a compromise to embrace a new project and have as many people as possible to get in; Jack, correct me if I'm wrong.
No new project comes to life with all the bells and whistles possible.
My question is, considering that you're using an open source base (PN2) and scripts, I guess, it will be possible, to everyone, to contribute with scripts or modules or whatever, to implement new features, correct?
If so and, if you agree and, if this is the correct time to do it; can you point the directions about, where to start and how to do it?
Obviously that, if this goes ahead, we should define some plan to avoid different people do the same.

P.S. Steve, how about those (one more) paradisiac vacations, fully recharged?

paradisiac vacation was excellent Jorge, lots of sun, rest and rum punch so yes fully recharged and it's back to work...til the next.

...til the next... let's see, would that be next month?

To answer Jorge's questions:

- Assuming we go with PN2 as the base (which is seeming more likely but is not cast in stone), it already has support for executing arbitrary exe commands (this is how the compiler is launched). So I don't see any problem replacing the call to compil.exe with one to some other compiler wrapper, as long as it is able to send the error list to the stdout (so it can be captured and parsed to jump to the errors). For non-compiler executables, as long as it can output to stdout, you can display the output in the output window.

- It also supports the ability to write scripts in Python. In addition to all the standard Python libraries, there is an API for interacting with the editor and also with the PN2 environment. I'm really just starting to explore that, so I'm not yet sure what the limits are, but certainly it will be easy for individuals to write their own scripts (and to share them, since installation is just a matter of dropping the script into the right directory). So that seems very promising. Just to give an example of what such a script looks like, here's my first attempt at creating a script to simulate The Editor's block comment/uncomment feature (using "!>!" instead of just "!"):

The Python scripting seems pretty fast and powerful, but they can also get quite complex in a hurry. (I started working on the TRACE.PRINT-generator script, and it is considerably more complex than the above because it has to parse the language to identify variables that can be output.) So I'm not yet sure at what point it becomes impossibly unwieldy. And I'm not yet sure how or if we can activate a script by a hot key or context menu. (The standard method is to just double click on the script name in the script sidebar window, as shown in a previous post.) It also isn't clear whether scripts can be left running in the background to capture events.

- For lower level interaction, one can write DLLs in C++ that are auto-linked, but I haven't gotten that working yet so it's still an unknown.

As far as letting you loose to start creating your own, it's still a bit premature, but my plan is to work towards assembling the installation instructions and examples over the next few days. Maybe by the end of this coming week we'll be ready to take that step.

Of the various wish-list items mentioned so far, the most obvious challenge is to combine all the ++includes into a single unit for the purposes of locating variable definitions, structures, functions, etc. Probably those kinds of things can be accomplished with Python scripting, but I'm not yet sure if it will be fast enough for very large programs, or whether it will require building a whole-program index in C++ and storing/updating it persistently, rather than repeating the scan each time. I'm also unsure how easy it will be to modify the user interface.

My general sense is that we can probably accomplish most of the basic requests without too much difficulty, and as Jorge suggests, it can evolve from there. (Whether we ever get to #47 is an open question!) But I don't want to drag others down this path before having a bit more familiarity with the issues. So keep the comments flowing, and I'll keep investigating.

A lot of the requirements need to have an understanding of how to translate a A-Shell file path to a native one, so that seems like it needs to be done early on. That requires being able to specify a miame.ini file, so should one of the first issues to tackle be a start on handling projects? At least, a root directory and miame.ini for the project.

I have some ideas about how to deal with some of the potential performance issues with the Python scripts, but that's probably better left until we make an official decision on using PN2 as the base.

I found the PN2 ScriptShare .

As a learning exercise I thought I would try to write a Python script to translate A-Shell to native paths (by scanning the miame.ini and ersatz.ini files).

But as for adding the miame.ini and related attributes to the project file, I'm yet sure of the best way to approach that. PN2 already has "Project Group" and "Project" definition files (XML format) that would be good places to store such additional attributes, but the problem is that the files get recreated whenever you save the project/group, so adding support for additional fields would involve modifying that code, which another level of investigation/activity. (Currently I'm trying to focus on ways to extend it via the exiting APIs rather than by direct modification.) So I'm inclined to think that just creating another sidecar project file to contain the the miame.ini (and perhaps compiler options) for a project would be a reasonable first step. (Even if it was just edited by hand, it's not something one has to do much of, so getting fancy with the UI and integration doesn't have a major pay-off in terms of making the developer's life easier.)

It might, however, make sense to add a new optional switch to compil.exe to specify the name of the project file, from which it can retrieve the miame.ini and other switches, which would then permit a single standard compiler tool definition to be used across all projects and installations.

The existing project file concept already allows for one (or more) root directory(ies), so I don't know that anything more is needed there.

As for the ScriptShare, unfortunately there aren't a lot of scripts there to choose from, but I did find some of them useful as a jump-start. (My short-term goal is to create a small collection of ASB-related utility scripts that are sufficiently well documented as to provide a better jump-start for others.)

As for performance, I agree that it's not yet a consideration we should worry about until we know we have a problem and that we are going to go this route. There is always the possibility of building a Python interface to some A-Shell API routines, which would presumably eliminate most or all of the potential performance issues.

I think it would be best to create the necessary ASB python library modules and share them via a mercurial repo on github (readonly for most users). All scripts should have access to the same A-Shell specific modules. i.e. Amos to native translation, map/defstruct parser, etc..

I agree about the sidecar file for A-Shell specific project settings, that's the same thing I was thinking.

If a developer doesn't want to use mercurial, it easy to create a zip of the repo at any changeset. Keeping them up to date via a zip wouldn't be difficult if they're uncomfortable with using mercurial, or if it's too large a barrier for entry.

FYI, using parenthesis in conditionals isn't "pythonic".

I'll try to toe the pythonic line in the future! :rolleyes:

(Python is admirably minimalistic in its syntax, although I must admit to having difficulty unlearning some old habits.)

Agreed on all your points though.

I would also like to add that for anyone browsing this thread and wondering how or if any of this might apply to them, please focus on these points:

1) While the ability to write scripts in Python is a useful feature (both in terms of quickly implementing the core utility functions that most users want, as well as allowing those who are so inclined to create customized features for themselves), most users of the new editor (i.e. A-Shell programmers) will have no need for any of this fancy stuff. So don't be put off by it.

2) We are consciously trying to keep it simple to facilitate an easy conversion from VUE. So while the input from the hard-core Editor users is very helpful and appreciated, we'd also like to hear from the KISS (Keep It Simple Stupid) crowd about what it will take to pry your cold arthritic fingers loose from the VUE keyboard.

My idea for dealing with that last problem (cold arthritic fingers and all) is simply to launch the new Editor when the user types "VUE." Everybody will learn quickly that way!

Perhaps this into 2 threads. This thread should remain as for a high level overview/feedback. A second technical thread for those who'd plan on contributing technically.

As someone who never got the hang of the original Editor, I look forward to learning the new one. Lord knows, while I'm comfortable with VUE, it's long past time to set foot in the 21st century. A clear set of installation instructions with specifics for a Linux-based server and geared towards those less familiar with A-Shell for Windows would be my only starting requirement. All of the features described above sound great. I leave it to others to choose the most important ones.

Just a quick comment on 6) ++Include file handling. For me, just an "Includes Explorer" view like the one in the current editor adds a tremendous amount of value. Being able to see the all included files and open them from one location is a huge time saver. I don't think it's necessary to try and determine if the compiler will include this file from this particular location while editing. In fact, I think it would be counter productive. I open files from the "Includes explorer" constant, so not including files because the compiler wouldn't have included it there would require locating the directory the files at and then find the file, or open another file to see if the one your looking for is in it's "Include Explorer".

Just my 2 cents.

Good point. I haven't yet decided how that can be added cleanly into the existing structure. I'm hoping I can extend the Tags concept to handle it. Currently it can identify the ++include statements (grouped under the heading "module" in the image below), and you can double-click it to go to the associated ++include statement:



What we need is to extend the Tags collection mechanism to combine all the tags from all the associated ++include modules (ignoring conditions, as you suggest), and also to modify the double-click behavior so that it opens the module instead of jumping to the ++include statement for the module.

I, not only, agree with the simple solution to populate the Tags>module header with every single include found in the program but, must confess that even didn't remember about that compile conditional options

Jack, that looks pretty good and promising

Any progress updates?

Sorry, like every other development project, it's taking longer than expected. But I do expect to have a package available by the end of the week for a couple of rugged pioneers (like yourself) to play with. I'm mainly working on documentation for it right now. (And don't expect too much in this 0.1 release -- it's still way short of the target goal, but almost surely far better than VUE, and probably better than any other option aside from The Editor.)

I'd like to discuss the design of the python miame/ersatz parser, and AMOS to native path converter when you have a minute.

Can't do it this afternoon, tomorrow looks better. But I should let you know that I have an alternate partial solution. The editor calls the ctags utility to generate the "tags explorer" information (list of functions, variables, include modules, etc.) for each file. Ctags is a standard utility; I created a wrapper for it that uses a new extension of compil.exe to generate a list of all the ++includes that are referenced by a main source file. So instead of generating the tags for just the main file, it generates them for the combined main file and all its includes. And the tags file format already contains the full filespec for each tag. Unfortunately PN2 doesn't currently process the filespec when building the tags index (it assumes they are all from the current file). So it can't find the tag if you click on one that wasn't in the current main file. (Fixing this will require a relatively minor but internal change to pn.exe which I'm not prepared to get into right now.)

But, I wrote a script allowing you to click on the ++include line and it will locate the include from the output generated by the compiler wrapper for the ctags utility. It's a little sloppy right now in that it will be fooled if you have multiple ++include files with the same name.ext (referenced by one program) but in different directories (hopefully that's not common practice!). The upside is that the logic requires nothing more than regex matching - no need to scan the ersatz(es) and miame.ini files and actually parse out the ++include spec.

A minor variation of the logic will allow you to click on a tag (such as a variable or function reference) and locate its definition, or to do a regex search across the entire program with includes.

So while it's not quite the perfect object explorer mechanism, it's maybe 75% of it.

What time would be best?

Note that most activity relating to APN has been moved, at least for the time being, to a Bitbucket repository. The sections of most interest are:

Wiki (documentation)

Issues Tracker

Currently it is just a free repository under Stephen Funkhouser's BitBucket account, which limits real users to just a few people who are actively contributing/developing, but anyone can browse.

The general plan is to try to somewhat stabilize and round-out the package thru September, then introduce and discuss it in depth at the Conference in October, after which we'll decide on a more permanent model for distributing and supporting it and will start actively trying to convince everyone to switch to it.