SysPage -- HTML Server Tool
SysPage is a daemon process for OS/2 which can
- write HTML page(s) containing intermediate and long term
performance/usage information for an OS/2 system,
with as low an overhead as possible.
- log process start, stop, and user-defined events to HTML or disk.
- optionally accept authorized commands from the WEB and display output
Legal Issues
* THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
* INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
* AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
* THE CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS
* OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
* WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
* OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
How to Get Started
- Installation and initial Configuration:
Put the program anywhere. Put the config/template file anywhere.
Use an ASCII editor to modify the sample configuration file test.spg.
Replace any references to my system with yours.
Your Company Name Here. Yadda-yadda.
For a quick install, change the output path file name references in each of the
segment headers (bracketed heading lines for HTML) to places in
your WEB server document path. Then (carefully) choose a value for
Peek_Interval.
Take care in assigning Peek_Interval.
A value of 7 or 8 should be OK for a 500MHz PC.
If you wish to use any of the command/logging
functions see below for Command_Port and
Command_Pfx.
- Invocation:
x:/fullpath/SYSPAGE.EXE y:/fullpath/your.spg
The program can be put in your path, but that's not necessary.
Once you are satisfied with the setup, run the program as early
as possible after boot. On our system it's necessary to wait until NFS is initialized and
mounts accomplished for the Linux directory which will be the target
of the HTML.
- Termination:
Ctrl-C to the Attention thread then enter 'q', 's', etc.
Or send XCPT_KILL_PROCESS (Warp4 termination button).
See the Attention Routine section for more details.
Some general statistics will be presented at termination, including
the amount of storage in bytes used for module history and for storing the
text portion of your HTML (does not include control structures).
There is a single argument to the program: the path to
a file which contains configuration information for SysPage and
the templates for the desired HTML.
The file consists of sections headed by the brackets "[<" in the first
position of a new line, enclosing the section name with ">]".
The names of these sections are pre-defined by SysPage, although their relationship
to displayed variables and row definitions may be re-defined by the user.
The currently supported section names are: Config, Commands, IPL, Process, Module, Log,
Aux1, and Aux2. The first two sections contain parametric information and the
others contain HTML source interspersed with symbolic variables.
Configuration Section
[<Config>]
The configuration section will accept lines beginning with a '#' sign
as comments. Blank and empty lines are ignored. This section is NOT
reloaded during execution; the program must be cycled to effect any changes.
Items in the configuration section are of the form
'Keyword = value'. The following are the recognized keywords:
-
Peek_Interval = nn
the interval for collection of process and module information.
This is in 10ths of a second. Other functions are timed in terms of
this interval. Getting this correct is important. The assumption is
that a 300Hz PII will use about 1% of its resources if Peek_Interval
is set to 10 (1 second). The faster the CPU, the smaller the tolerable number. Your pain may vary. The process and logging
information will be lost for programs which begin and end within
a Peek_Interval; they are below the event horizon. Once this value has been
set correctly, the other values of [<Config>] in the distributed sample
should function reasonably.
-
Module_Names = nnn
The amount of storage in 'k' (1024 byte units) to be allocated for retaining
the module path strings of executed programs. SysPage with allocate a default
amount, probably much larger than required, if this is not specified.
The actual amount in bytes will be displayed at program termination.
This number should allow for growth during execution.
-
HTML_Text = nnn
The amount of storage in 'k' (1024 byte units) to be allocated for retaining
the text strings assocaited with the template HTML in the control file.
The actual amount in bytes will be displayed at program termiantion.
This number should allow for growth during execution via the 'Reload'
Attention Routine command.
-
Proc_Thread = nn
the number of Peek_Intervals to wait before writing the
count of processes and threads to pipe /PIPE/SYSDISP1.SB2.
Zero (the default) means do not write this information. This is for use as
a SysBar/2 data source.
-
Unused_Swap = nn
the number of Peek_Intervals to wait before writing the
amount of unused space in the current SWAPPER.DAT allocation.
Zero (the default) means do not write this information. This is written to
the same pipe (/PIPE/SYSDISP1.SB2) as Proc_Thread for SysBar/2 purposes.
Note:
Take care in choosing the preceding two values, since using the same interval
for both will produce predictably poor results. A couple of reasonably
separated prime numbers might be a good choice (i.e. 5 and 9).
-
Max_Buffers = nn
The maximum number of buffers to allocate for Dos32QuerySysState
output data. When this threshold is reached, a message is issued
and the oldest buffer is deleted, instead of processed. Too high
a number might cause paging activity, too low, loss of interval data.
The Attention Routine statistic, Buffer Waits
will track the number of these potentially lost events. The default is 8.
-
Save_Path = x:\full\pathto\save.file
The full path to a file used to save accumulated data between
executions of SysPage. The file will be written into at each termination
of SysPage, and data areas will be re-loaded from it at each initialization.
This parameter is not mandatory, but must be defined to use the 'S' command
from the Attention Routine.
-
Log_Depth = nn
Number of log entries to maintain in memory. Should be larger than the
Log_Display value.
-
Log_Display = nn
Number of log entries to output in the Log HTML file.
Should be greater than or equal to LogDepth. This value may be adjusted
via the LogDisp remote command from the CGI client sysphmsg.
-
Log_Delay = nn
Minimum number of Peek_Intervals to allow before rewriting the LogHTML.
This is the 'delay' interval for the logging thread. If program start/stop
activity is low on your machine, a value of 1 would be OK; if the start/stop
activity is very high, raisingf this value will cause the Log page to be refreshed
only once per 'LogDelay' number of log events.
-
Log_Console = nn
Whether to display the process start/stop information in the
SysPage text (VIO) window. This should be set to 0 if you intend to 'detach' SysPage,
or only desire critical messages to appear in the test window.
-
Exclude_Dev = ABxyz
A list of logical devices (mode letters) to exclude from the Logical Device
report rows (%&ldevices). "AB" would be the general usage. This will prevent SysPage
from (very slowly) probing the floppy drive(s) or other non-ready device.
-
Command_Port = nnnn
The UDP port number on which to accept connections for issuing a command
to SysPage. The minimum is 1000.
Input to this port prefixed with the Command_Prefix will be
treated as a command. Currently, only the LogDisp command is
supported.
-
Command_IPs = auth-group a.b.c.d,e.f.g.h
One or more pairs of tokens: the name of an authorization group, such as 'admin',
followed by
a comma-delimited list of IP addresses in the fully-qualified dotted notation.
These are the addresses from which Command_Port commands are acceptable.
This is NOT
a security feature; it is only to avoid potential hassles.
-
Command_Pfx = 01434D4402
A string of hexadecimal characters depicting the binary string to be
expected as a lead-in or prefix to an operator command for SYSPAGE.
This is NOT a security feature; it is only to avoid potential hassles.
The matching string in sysphmsg.c must be modified by the user, and
sysphmsg rebuilt. It is the user's responsibility to construct the
necessary security arrangements in sysphmsg. The distributed sysphmsg.c
is not configured. The code compiles on my old RedHat 5.2 system.
It should not be too stressful to compile it for an OS/2 client with
either GCC or the IBM (VA)C. The inboard code for SysPage currently only supports
the LogDisp command. Please feedback on any desired changes and additions.
-
Log_Port = nnnn
The UDP port number on which to accept connections for logging to the SysPage Log
feature. Messages introduced via this port are displayed in the text (VIO) window
if Log_console is enabled, in the log disk file(s) if Log_File is defined,
and in the Log HTML page, if that section is defined. If the port is defined
as 514, then SYSLOGD messages will be displayed by it. Included in the product.zip
file is the executable sysplog.exe. This program accepts one or two strings
as arguments. If two strings are given, the first is assumed to be a list
of keywords, equal signs, and values separated by blanks and/or commas.
The second is the message itself. Both parameters must be enclosed in
quotes or apostrophes (in the REXX style, if one contains the other).
The parameters are:
- PORT=
- The UDP port for the message transmission.
Default is 514.
- HOST=
- The target host for the message.
Default is 'localhost'.
- PRI=
- The UNIX-style SYSLOG priority field, currently numeric in the
range 0.0 to 31.7. Default is 1.5 (user.notice). OR specify
the facility-name '.' level-name. Facilities are:
- kern
- user
- mail
- daemon
- security
- syslog
- lpr
- news
- uucp
- cron
- local0 through local7
Levels are:
- emerg(ency)
- alert
- crit(ical)
- err
- warning
- notice
- info
- debug
- TAG=
- A one to 15 character tag string associated with the message.
Default is a null string.
- PID=
- 'Yes' or 'No' to include the parent PID of sysplog in the message.
Default is NO.
-
Log_File = x:\full\pathto\syspage.log.templ??.ext
The SysPage disk log template. This is a file path template which must have
a suffix consisting of "??.ext". The "ext" portion
can contain any 3 admissible file name characters.
This describes the file name group
and directory to contain the disk version of the SysPage log.
-
Num_Logs = nn
The number of log files in the log file rotation. From 1 to 99.
Log_File must be defined.
-
Log_Switch = 1 | 2 | nnnn
When to switch log files. The value '1' is interpreted as HOURLY;
the value '2' is interpreted as DAILY. All other values are considered
the 'rollover' file size in 1000 byte units, so that a value of 100 means
that the log will switch when the current log file is larger than 100,000 bytes.
The largest value is 2000.
-
Log_Flush = nnn
How many log entries to accept before checkpointing the log file to the backend.
A value of '1' will have every line cause a flush (DosResetBuffer) on the log file.
The limit value is 500.
Commands Section
Commands which may be executed through the defined command-port by
a CGI program (the source for sysphmsg.c is in the disribution).
This section allows comment lines with a leading '#' sign.
Each line in this section defines a program or internal command.
There are four tokens to a line when defining an external command,
and only two tokens for a Syspage internal command.
- Command Verb
- The string used to identify this function in the VALUE clause
of an HTML INPUT statement with NAME 'CMDVERB'.
- Authorization Group
- The auth-group where this command belongs. These names are
assigned with the Command_IPs parameter.
- Command Path
- The full file path to the executable, including the .EXE suffix.
- Log attribute
- Required but currently ignored; must be LOG or NOLOG.
You will need to compile (and modify for your site requirements)
the CGI program sysphmsg.c included in the distribution.
HTML Sections
Other sections are basically HTML page text containing symbolic
variables to be substituted. These variables are prefixed with the string
"%&" or "%%".
"%%" is used to denote an OS/2 environment variable.
Variables which are part of table rows
are pre-defined for those rows. These variables are
case-insensitive. Output Path is the full path specification
for HTML output from this section. These keywords are case insensitive.
The realationship of HTML sections and table rows is basically at the discretion
of the user. Most of the sections have names which suggest the table row
which they might contain. This is not hard and fast.
The Report_Interval parameters designated indicate an integer
value specifying the number of Peek_Intervals
between generations of that report. For the IPL report, the value is required
but ignored.
The Output_Path strings designated below should probably point into the
same directory. This directory would greatly benefit from being on a
RAM (VDISK.SYS) or Virtual Disk (Albert Shan's SVDISK.SYS) for low overhead. I've
been running mine out via NFS to a Linux directory soft-linked into
the Apache Document path.
-
[<Module> Report_Interval Output_Path]
Page intended to be emitted containing a Modules row, at the interval
specified by Module_Interval.
-
[<Process> Report_Interval Output_Path]
Page intended to be emitted containing a Processes row at the interval
specified by Process_Interval.
-
[<Log> Report_Interval Output_Path]
Page intended to be emitted containing a Log row.
-
[<Aux1> Report_Interval Output_Path]
Extra page for any combination of variables and rows.
-
[<Aux2> Report_Interval Output_Path]
Extra page for any combination of variables and rows.
-
[<IPL> 00 Output_Path]
This page is emitted only at SysPage startup and after an HTML reload.
The '00' is a dummy placeholder for Report_Interval.
This section contains a page describing system configuration information
which is basically unchanged during a boot. The variables are:
- "INITDATE" -- starting date of SysPage.
- "INITTIME" -- starting time of SysPage.
- "BOOTTIME" -- starting date/time of system.
- "PHYSMEMORY" -- number of meg installed memory.
- "NUMPRINTERS" -- number of installed printers.
- "NUMFLOPPIES" -- number of installed floppy drives.
- "NUMCOMPORTS" -- number of functional COM serial ports.
- "NUMPARTDISK" -- number of fixed hard drives.
Other variables are available for any one of the pages, but are
recomputed and reformatted for each request:
- "CURRDATE" -- the date at page-generation time.
- "CURRTIME" -- the clock time at page generation time.
- "AVAILMEMORY" -- aggregate reamining memory (Dos16 call)
- "SWAPREMAIN" -- disk space remaining in swap file.
- "CPULOAD" -- percent CPU usage over last interval.
- "UPTIME" -- amount fo time system has been running.
The variable '%&reload' is evaluated depending on the HTML file that is
being processed. It returns the millisecond value of the corresponding
configuration variable. For the Module section, it returns
Module_Interval * ( Peek_Interval * 100 ); for the Process section, it
uses Process_Interval * ( Peek_Interval * 100 ).
Environment variables are substituted at program start and identified
with a '%%' prefix, i.e., %%hostname, %%tz.
Table Definitions
Table definitions are 'special' symbolic variables for which an entire
set of rows is substituted. The symbol must begin in the first column of an
HTML template line.
Available tables are Partitions, Modules, Processes, Ldevices, LogTable,
and Units.
The table row data will ordinarily be refreshed during the corresponding
report generation cycle.
Units
"%&Units" represents the physical hard drive units present
at boot time. The sub fields are:
- "PHYSICALDRIVE" -- the fixed drive number (from 1).
- "CYLINDERS" -- the number of cylinders on the drive.
- "HEADS" -- the number of heads per cylinder.
- "SECTORS" -- the number of sectors per track.
Partitions
"%&Partitions" represents the entire table row, and the subfields
"%&PhysDisk", etc. the request for and order of the subfields.
So, the entire "Partitions" entry must be given, with as many
subfields as desired, in their desired order. Here are the members
of the Partitions group.
- "PHYSICALDISK" -- the fixed drive number (from 1) containing the partition.
- "PARTTYPE" -- partition type, if known.
- "MODE" -- mode letter as OS/2 would assign it.
- "STARTSECT" -- starting sector number
- "SIZESECT" -- size of partiton in sectors
- "ENDSECT" -- ending sector number
- "STARTMEG" -- (TBD) starting address in megabytes.
- "SIZEMEG" -- (TBD) size in megabhytes.
- "ENDMEG" -- (TBD) ending address in megabytes.
The "%&Processes" table has the following entries:
- "PID" -- Process ID.
- "PMODHANDLE" -- module handle of primary executable for the process.
- "PMODNAME" -- the path to the primary executable.
- "PUSERCPU" -- cumulative CPU consumed (ms.) in user mode.
- "PSYSCPU" -- cumulative CPU consumed (ms.) in kernel mode.
- "STARTTIME" -- starting time of day for process.
- "STARTDATE" -- starting date for process.
The "%&Ldevices" table has the following entries:
- "SMODE" -- Device Mode Letter.
- "STYPE" -- type of device.
- "MEDIA" -- media type code.
- "FREESPACE" -- free space on device.
- "DEVSTATE" -- state of device (Ready, NotReady, etc.).
- "DEVATTRIB" -- device attribute code.
- "DEVFSDNAME" -- File System Name.
- "DEVFSADATA" -- File System Attach Data.
- "VOLUME" -- the volume label.
- "VOLSER" -- the volume binary serial number.
The "%&Modules" table has the following entries:
- "MODNAME" -- full path to module.
- "TOTUSERCPU" -- total user mode cpu time for module.
- "TOTSYSCPU" -- total system mode cpu time for module.
- "TOTELAPSED" -- total elapsed time used by a module.
All these entries may represent multiple invocations of the
module.
TotElapsed has the granularity of the Peek_Interval.
The AUX1 and AUX2 page definitions may contain any symbolic, environment,
or table row variables that the user requires to be in a separate page.
The "%&LogTable" table row has the following entries:
- "TIMESTAMP" - event time stamp.
- "LOGPID" - process ID involved.
- "LOGMODNAME" - module name.
- "LOGELAPSED" - elapsed time for ending process
- "LOGSCPU" - system (kernel) CPU time for ending process
- "LOGUCPU" - user CPU time for ending process
- "LOGEVENT" - string "Starting ", or "Ending "
- "LOGMODHANDLE" - module handle involved.
The Logtable set of rows has some special features. If you are using the
Log_Port option and sysplogp to supply SYSLOG entries to the log,
and especially if you have defined Log_Port as 514 and are collecting
all SYSLOG output via SysPage/2, the field "LOGEVENT" will contain
the SYSLOG tag field; the field "LOGMODHANDLE" will contain the
SYSLOG priority value. If 'TAG=' was specified on the 'sysplog' invocation,
the issuing PID will be available in "LOGPID" otherwise blanks
will be supplied.
The "%&FileDir" table row has the following entries:
- "FILENAME" - file name.
- "FILESIZE" - size of file in bytes.
- "FILECRDATE" - Creation Date of file.
- "FILECRTIME" - Creation Time of file.
- "FILEWRDATE" - Last Write Date of file.
- "FILEWRTIME" - Last Write Time of file.
- "FILEACDATE" - Last Access Date of file.
- "FILEACTIME" - Last Access Time of file.
The FileDir row requires additional syntax. The template line should
start with %&FileDir[x:\complete\path\to\dir\*] if all the files
of a given directory are desired, or with an exact path if only one
file is desired. Wildcards are permitted as with the DosFindFirst API.
Don't forget the open and close brackets for this file specification.
If SysPage is started in a text window, entering a Ctrl-C into this window
will obtain the attention of a routine which accepts command lines.
SysPage will continue to collect data and emit HTML while the user interacts
with it in this way. When the Attention Routine is activated, it will display
some of its single-character options and their meanings.
The Attention Routine functions are:
- Q Quit SysPage without saving Module Data.
- S Save Module Data and quit SysPage.
- R Reload the HTML and Commands portions of the configuration file.
- I Exit the Attention Routine.
- M Display thread status and exit Attention Routine.
- G Display memory usage and exit Attention Routine.
- W Switch the Log file now.
The 'R'eload function should also refresh the entries in the Command section.
The 'M' request will display some internal information about the threads
controlled by the Schedule subsystem. The thread ID is given as 'TID',
followed by state which should be 'Running' or 'Waiting'. This is from
Schedule's point of view; i.e., 'Running' means that some piece of
work is currently scheduled to run on this thread. The thread itself
may be waiting for a system event. So the Attention and UDP Ports tasks
will generally appear to be 'Running'. The 'Attention' task actually is
running since it is producing the task list.
To obtain the ID string for the version that you have, execute SYSPAGE.EXE
without any argument. This will display usage and version.
SysPage is linked with some routines which attempt to write a post-mortem
dump file containing context information, stack, and heap. The debug version
of the program corresponds to the large shipped SYSPAGE.MAP file.
Should a program exception occur, these routines will attempt to write
a file named like 'M2DUMP.nnn' in your SPOOL directory; i.e., the directory defined in
System-Setup->Spooler under the tab Spool Path. If at all possible,
obtain this file as produced by the debug version. The production version
will produce a dump, but the SYSPAGE.MAP shipped in the product.zip package
will only give general offsets within object segments.
Please ZIP the dump file and corresponding map file for whatever version was
in use at the time of the error, and email it to
sbsbugs@dashs.denver.co.us
Using the 'G' Attention Routine command, you can peek at the memory usage of
the program. This should tend to level off at some point. There probably exists
some leakage in the area of HTML-reload which has not yet been addressed.