This is a question that comes up repeatedly. The A-Shell reference does include a couple of topics ( Launch Background Process on Boot and Background Jobs ) but people are always looking for more explanation. (Increasingly it is common for A-Shell applications to be running in environments where the application developer/vendor doesn't actually have much knowledge of the OS nor sufficient privileges, which are instead jealously guarded by the end-user's "IT department".) So here's another approach that you can implement entirely from within A-Shell.

The idea is to use SUBMIT.LIT to launch your process. To do this, create a CMD file that will launch the program (typically with a CTL extension for "control file", but if you want to also be able to run it like a command file in the foreground, CMD will work for both cases). In this example the background program will be called BG1, and our startup command will be BG1.CMD.

The RUN CHKDUP BG1 line is a refinement to prevent more than one instance of BG1 from running at a time (which may or may not be applicable to your background process). You can retrieve it from the EXLIB 908078 directory. You can omit it if your background process can be run multiple times simultaneously, or if it has it's own way of determining if there is another instance already running (perhaps by a file lock).

You can test your command file by just executing it from the dot, but in that case it will be running in the foreground. To get it to run in the background, use SUBMIT to launch it as follows:

The /NEXT:0-0-0 switch is critical here in that it forces the process to run under control of the background task manager, and thus independent, rather than a child of your current process.

Tip: your background job should maintain a log, allowing you to easily check on its activities. But don't name the log xxx.LOG where xxx matches the name of the control file submitted to SUBMIT, since SUBMIT will also create the same LOG file by default.

What's missing? Some way to launch it automatically when the system boots, and ideally whenever it should stop running on its own. The simplest way to handle both situations would be to add some code to your startup CMD to submit the control file to launch the background job. Assuming it has the CHKDUP test as in the example above, each such submission will detect if it is redundant and just shut itself down. It does mean that you would end up with a superfluous A-Shell launch (followed by an immediate shutdown) every time someone executes the main startup CMD. But that's not the end of the world, especially if the startup CMD was only launched once per real A-Shell session (as opposed to every time a user returns to your main menu).

The main gaps in that strategy are:

1) The background job won't get started after a reboot until the first normal A-Shell session is launched. (That might be perfectly acceptable or not, depending on the nature of the background job, i.e. whether it is there to service foreground jobs, or is needed to service some automated process or external events.) The only way to avoid this gap would be to edit the system startup scripts to launch the background job on reboot, or perhaps to create a crontab entry that ran every couple of minutes to launch the job (if it isn't already running).

2) If for some reason the background job dies, it wouldn't get restarted until the next A-Shell session (or the next execution of the main startup CMD file). That also might be perfectly reasonable, or not. The crontab approach mentioned above is probably the only way to eliminate this gap. (You'd have to decide how long you were willing to wait for the restart, in the case of such an unexpected termination, and set your crontab launch frequency accordingly.)

If it seems like overkill to launch a new A-Shell session just to check if the background job is running and then (assuming it is running) immediately shutdown, another approach would be to add some logic to various places in your application that checked to see if it was running, and if not, SUBMIT the control file again (using XCALL AMOS, SUBMIT... or XCALL SUBMIT). The CHKDUP utility is not good for this purpose, since it terminates your session if the program is already running. Instead, you'll need to write some code to check if the process is running. You can do this easily with the SOSFUNC: function Fn'Is'Running'Program(prog$) (available in the SOSLIB repository in 907010). (This is actually the function that CHKDUP.RUN uses.)

Or, another idea would be to create a second background process that acts as a "watchdog" on the first. The watchdog process could just sit in a loop, something like:

Note that the CHKDUP and Fn'Is'Running'Program() approach to verifying the operation of the background job is less than perfect, since it only tells you that the job is "running" (exists and not just sitting at the dot prompt). But it doesn't verify that it is really "functioning". A more sophisticated approach would rely on the communication method already being used by the background job to service requests, to support an "are you ok?" request/acknowledgment.

For example, if the interface is based on client jobs dropping files into a directory which the background server then processes and renames/deletes, you could drop a dummy request file there to see if it gets renamed/deleted. (Unfortunately, that might take some waiting if the background job is busy.) A refinement would be for the background job to output some kid of "heartbeat", such as as a small control file. The watchdog could then check for that heartbeat, possibly reading a timestamp from the control file, or maybe just checking the timestamp on the file itself. (The SOSLIB function Fn'FileAge(file$) will tell you how many seconds since the file was last updated.)

For a sockets-based background process, you could just make a dummy request to the designated socket to see if the server was functioning.

All of the above techniques will work similarly under Windows. SUBMIT.LIT merely launches another session on your workstation, rather than in the "background", but the new session can be minimized (-min), hidden (-z), or tucked into the system tray (-zi), and it will run independently of the parent. (You could also use HOSTEX rather than SUBMIT to achieve the same effect.)

To deal with "gap #1" under Windows, the simplest approach is to just create a shortcut in the Startup folder.

Yet another approach under Windows would be to run it as an actual Windows service, taking advantage of the service option to automatically restart the service at launch and if it should die. (That would deal with both of the gaps described above.) To learn more about how to launch an A-Shell application as a Windows service, see Installing ATSD as a Service in the ATSD manual.

Did not know about Fn'Is'Running'Program that come in handy for a background routine I’ve recently written, I currently update an INI on launch say it’s in use/running and then again on exit.

If the INI is yes then I know a) it’s still running or b) something not so good occurred and program crashed with out cleaning exiting and a user is emailed to take a look.

Another background server case that wasn't discussed above is that where the background process is a one-off process that runs only at specific times. Or maybe once per day (as in a nightly recap/cleanup/backup process). In this case, most of the above techniques are overkill and/or otherwise unhelpful, and you might as well just use the operating system's background scheduler.

For Linux, that would be crontab. Here's the procedure for setting that up from within A-Shell...

1) Figure out the command line you need to launch the process. For an A-Shell program, it should be something like this:

Breaking that down, we have:

a) /vm/miame/bin/ashell (the ashell executable)

b) -i /vm/miame/miame.ini (the miame.ini file)

c) -td dumb (this selects a teltyp-style terminal driver, which is appropriate for background tasks and will keep the captured log clean of escape sequences)

d) -j bgjob (optional to force the jobname to "BGJOB".) Note that if such a job already exists, the new job will try to kill/remove the existing one, which may or may not be appropriate.

e) bg1 (the command file which launches your program.) It should be in the CMD: diretory, e.g. CMD:BG1.CMD, and handle both logging in and perhaps even checking if the job is already running (as in the example given at the top of this thread).

f) > /vm/miame/dsk0/001002/bg1log.log (redirect the screen output, i.e. stdout, to the specified file). That might be useful for debugging. If you don't need it, use > /dev/null instead.

g) 2>&1 (redirect the stderr stream to the same place as the stdout). This would typically only consist of OS-level error messages that occur during startup, but covers all bases.

2) You can test your command line from the A-Shell prompt by preceding it with host and adding a & at the end so that it runs in background. Otherwise the command will hang your job as it waits, perhaps endlessly, for the background job to run. Note that you can use SYSTAT to see if the job is running, and perhaps look at the file where you redirected the output to see any messages that the process output to the screen, not to mention any log files explicitly created by the process.

3) Once you're happy with the way the process launches, set it up to run at the appointed time by creating a crontab entry as follows:

The above will launch a vi (editor) session allowing you to edit the crontab for the current user, i.e. you. The crontab file consists of individual lines with 5 numeric fields at the start, followed by the command line you want to launch. (There may also be comment lines starting with the # character, and vi uses the ~ character to indicate EOF lines.)

The 5 numeric fields at the start of the command line control when the command line is executed:

Field 1: minute within hour, 0-59
Field 2: hour within day: 0-23
Field 3: day of month, 1-31
Field 4: month, 1-12
Field 5: day of week, 0-6 (0=Sunday)

Each of the fields can be a single number, or a comma-delimited list of numbers, or a range (e.g. 1-5) or * (meaning not applicable). So for example, to run a command every day at 11:30 pm, you would use:

30 23 * * * command

(i.e. 23:30 every day of the month, every month, every day of the week)

To run a command every minutes every day except Sunday:

0,10,20,30,40,50 * * * 1-6 command

(There are lots of Internet references to this for more detail).

The vi editor has essentially 3 modes:
- editing
- command (cursor movement and editing commands)
- command prompt (save, search, etc.)

The commands you need to know are (everything is case sensitive!) :

i - insert (and start editing)
ESC - cancel editing or command prompt (back to the command mode)
x - deletes character at cursor
dd - deletes the entire line
: (colon) - brings up command prompt at bottom of screen

Otherwise use the arrow keys to move around, BACK for rubout. To insert a new line, position the cursor at the end of an existing one, hit i and then ENTER and then start typing.

To save your file, hit ESC to cancel editing mode, then : to get the command prompt, then wq! to save (write) and quit. To quit without saving, do the same but use the q! command (quit without first writing).

Cron typically wraps at column 80, so after entering the command line above, and setting it up to run at 10:45 PM each night, it would look like this:

When you save the file, crontab will verify that you haven't made any egregious errors, and if not, it will say something like:

crontab: installing new crontab

You can check the existing crontab with:

If you don't like the setup, you can just repeat the crontab -e command, using the dd command to delete entire lines, or insert # in front to comment them out.

Once you're satisfied, sit back and let the scheduler take care of launching your program at the appointed time. You can check later to see that it ran at the right time by looking in the ashlog.log file (assuming you have TRACE=INOUT in the miame.ini, as you should).

For Windows, the concept is similar but the details are all different because you would be using the Task Scheduler, which, in inimitable Windows fashion, is slightly different in every version of Windows. (So best to just look it up online.) The A-Shell command line would be essentially the same as the UNIX case, except no need for the > redirection, and you probably want to use the -z or -zi Command Line Switches

The INI file approach described by Steve is a perfectly sensible alternative. Although if you really want to be thorough and detect the case where something bad happened (the INI says "yes" but the program crashed without cleaning up), you'd presumably have to use something like the Fn'Is'Running'Program() function anyway. Which might beg the question of whether the INI test is helping any?

The question of how to make sure that a background task is always running (restarting it if necessary) has been posed once again so here's another update on the subject...

For Windows, one idea would be to run your background process as a Windows Service. You can use the ATSDSRV.EXE utility that comes with ATSD to launch any A-Shell program as a service. That not only takes care of starting it up when the system is rebooted, but the service manager will also detect when the process dies and restart it.

For UNIX systems, adding your background process command line to /etc/inittab might be the best way to go. The inittab format supports an option to respawn the process if it dies, and is automatically triggered on bootup when the system reaches a specified run level.

The main gap for both of the above would be if the process becomes non-functional without actually crashing and closing. To detect that kind of problem requires some higher level method of probing the background program than is likely to be possible with a generic tool. Which leads to the question of whether it is worth creating an A-Shell specific watchdog utility that would be able to support a set of configurable higher-level methods of verifying that a background service is running properly. (Possible methods include checking if the service accepts a connection on a TCP port, or if it consumes a file put into it's inbox directory, or if is updating the timestamp on some designated status file.) One downside of such a watchdog is that it would probably have to run as an A-Shell job itself, but that might not be much of an issue. It's just not clear to me how much demand there would be for such a tool, and what kinds of configuration options it would require in order to be useful for a variety of applications.

^^^ Hmmm wondering if such a tool was ever developed??!

This is related to above... but just wanted some clarification... getting ready to embark on setting up an autonomous background process on a large customer with many databases (running in their own ppn). For the sake of this project i decided to create a separate task for each independent practice, with a filename related to the practice number.

Clarification: In the manual it states that if I submit the same task name it will be ignored. Consequently that is not exactly what's happening. It actually kills the running task, then spawns a new one. This creates an error condition reported to the end user because, well, we want to be proactive in reporting errors so that we can diagnose them. So is this correct behaviour? Or is the manual wrong on this count?

Another thought repeated above is how to Know if its running.. i cant check if the program is running because it could have multiple instances running. I know i can run .SUBMIT but i don't want that to blow out my screen.. is there a way to pipe results to a file? Or is there another xcall to poll the submit results? In this case i can at least check if running and if not launch accordingly. Also want to present the end user of the bridge status when needed.

TIA

No, I don't think such a tool was ever developed.
It's not clear to me whether it makes sense for a duplicate SUBMIT to be ignored or replaced; probably need more details to think that one through. Not sure where in the manual it says that it will be ignored either.

But as for outputting the SUBMIT display to a file, doesn't the SUBMIT/LIST{:fspec} work for you? (I see now that it isn't shown as a command line option in the manual - Ty? - but it does show up in the SUBMIT/? display.) But I'm not sure that's really the best way to determine if these jobs are running. I would probably opt for scanning the job table to check the program and PPN, and then implement some kind of communication protocol (as suggested in my previous post) to make it easy to verify that the job is running properly. Having it update some kind of status file whose timestamp you can then check would probably be the least obtrusive.

There is a handy SOSLIB function Fn'FileAge() you can use to get the time (in seconds) since the last update to a specified file which would help.

Ah good suggestion thanks... i can spin the jobtbl and look for what i need. I also output a status file.log which i can poll...

Yes, the /LIST works, sorry i was unawares of that feature.

For the manual, i believe its under the discussion of background tasks... let me find it...

http://www.microsabio.net/dist/doc/conref22/00ashref.htm#!backgroundjobs.htm


Job dies and needs to be re-launched

The first thing your background application should do is check to see if it is already running. One simple way to do this is to use the -j <jobname> switch to force it to use a specific job name. If the job name already exists, the A-Shell launch will fail, so there would be no harm in just re-launching it as often as you like, either manually or automatically. (This assumes that if the job dies, it will close itself down entirely. You can help guarantee that by using the -e startup switch, adding HOST to the end of the command file which starts the app, adding XCALL MIAMEX, MX_EXIT to your error trap, etc.)

Hey Cap i know you're busy so no hurry just wanted to clarify above behavior is correct. I don't really mind the kill/relaunch if that is what it's doing here.. i just need to track it.

TIA

Ok, I see the confusion. The discussion of background tasks is talking about the -j jobname command line switch for A-Shell. In the case of an interactive session, it will ask you whether you want to abort the pre-existing job of the same name. But when launched as a background process, since there is no keyboard interface, the answer is taken to be null, i.e. no, in which case the attempt to launch the new session fails.

But if you use the SUBMIT /J jobnam switch, the logic is reversed. Offhand, I'm not really sure why that is, i.e. whether it was a random decision, or was carefully thought out. I could easily come up with an argument for why the request to SUBMIT is different than the more general ashell command line switch, since the latter is likely to occur within some batch context, whereas the former is presumably under either user control or program control, either of which could check to see whether the job was already running.

For permanent background tasks, I normally set them up in crontab. That's typically more common for jobs that just need to run at specific files, but you can also set jobs up to run every hour, or even every minute. Here's an example of a crontab entry for a job that runs once a day, Monday-Saturday at 11 PM ...



My ashterm script looks like this:



If I added the -j jobnam switch to the ashell command line and the job was already running, the new launch would abort. So in your case, one option might be to have crontab restart your job every few minutes, relying on the auto-abort if it's already running. But obviously this is a still a crude mechanism. If you're going to implement a watchdog that can not only detect if the job is running, but that it is working correctly, then you would probably want to use SUBMIT with the /NEXT:0-0-0 switch to relaunch the task under the control of the at daemon, when needed. And in that case, having it abort the existing job to replace if with the new one would be exactly what you wanted.

I learn something new everyday....(Could be very useful)

(We use cron / scripts to run selected background tasks every X hours etc)

Great. Thanks for the feedback.