[history]
[concepts]
My Exosat software : history
archeo-Exo
- I started planning for a complete X-ray astronomy analysis system to be used
for Exosat (note the wording !) as early as 1982 when I was planning my return
to Milan. These plans exist in form of paper notes only.
paleo-EXO
- However then I moved to the Exosat Observatory at ESOC, where I found I had
lot of things to do (see references
22,
35-41 and
48) and my official duties did not
include participation to the development of the Automatic Analysis and
Interactive Analysis (which was however planned as a set of monolitic
[although perforce segmented] instrument-specific environments to be run
on the Observatory HP-1000 computers under the control of an OPCO scheduler
(programs were named like MXAAn, GXCAn or LXIAn i.e. "n-th Automatic, Calibration
or Interactive program for instrument ME, GSPC or LE".
I exploited this time to gain familiarity with the instruments and with the
programming environment.
meso-EXO
- During a later time (but not so late, I stayed at ESOC only for 2 and half
years) there were two concurrent developments :
- With the departure of Rodney Blissett, former responsible of LXIA2, the
Exosat LE Interactive Analysis, I became in charge of this, and, together
with Paolo Giommi, revamped it in a modular way creating LXIA7,
formerly known as the Pirate Interactive Scientific System built
mostly as creatures of the night (i.e. unofficially and
during night shifts at the Observatory). The help file of LXIA7 is
report S10 (entry 52)
- At the same time in Milan Bianca Garilli was developing an independent
system in the context of her thesis (see talk T2 ;
entry 24), part on the IBM
mainframe, part integrated with IHAP on our HP 1000 (coincidentally,
this was fully hardware compatible with the system at ESOC).
ceno-EXO
- With my return at IFCTR in 1985, all developments merged in the Milan Exosat
Analysis System which is documented in
report R2 ((entry 47)and
paper P27 (entry 66). This included
both LE systems (since I was entitled to export LXIA7 to which I
continued collaborating for a while, plus newer programs written from scratch
for the ME instrument (on our IBM mainframe).
In the latter context were included also programs for spectral fitting
(presented separately) and later for timing analysis (described in report
R12 (entry 104 ; these timing
programs were developed in two parallel, fully compatible, versions on HP and IBM).
Programs were also integrated with the PABLO graphics
program and related utilities.
Because of the forced splitting of programs between the IBM mainframe and
the HP 1000, when it was no longer feasible to exchange data bringing tape
reels down a flight of stairs, a file transfer link was established
(see report R3; entry 75).
To speed up things some of the LXIA7 accumulation programs were
ported to the IBM operating system.
Further technical details are documented by an extensive set of private
software notes and data sheets, and by reports
report R6 (entry 84)and
R7 (entry 85).
- See also a separate history of fitting programs
and graphic programs
My Exosat software : concepts
The de-facto concepts of this first complete package I wrote
(whose history is reported above) resulted to be :
- each command was to be handled by a separate program, invoked with an
English verb (with the limitation of 8 characters on IBM and 5 characters on
HP systems).
While this was a natural evolution of my IUE system
on IBM, it was in contrast with the eXIAn programs at ESOC, and this
modular approach (carefully hidden) was one of the key differences between
LXIA2 and LXIA7, using native HP EXEC calls.
- each command supported also a number of arguments on the command string. The
implementation of these was native on HP RTE-6, while it required a workaround using
the console stack on the IBM VM/CMS operating system.
A noteworthy improvement (originally mutuated from the parameter file
concept of the PABLO graphic
program ) was the definition of a flexible user interface which
allowed arguments to be supplied either on the command string, or prompted
interactively, or supplied in an external command file, or whatever
combination thereof (including terminal echo of parameters supplied outside
of the interactive dialogue). On IBM some
parameters were compulsory on the command string (typically file names, since
opening files by name inside a program was not possible on VS Fortran, but
required a preconnection in the calling EXEC2 procedure). The implementation
of the second, final version (unified in its look between HP and IBM) of this
interface is described in report R6
(entry 84).
This interface also contained improvements on and standardization of
the behaviour of the standard Fortran list directed (free format) input.
This interface was later ported to VAX VMS and Sun Unix before being revamped
within the XAS user interface (document revision of 1991).
- the concept of data accumulation from the instrument dependent telemetry
files into almost instrument independent data files
- the definition of logical data grouping and association of the physical disk
where a given class of logical data resides done virtually (on IBM this
used the filemode)
- sticking to the observation concept for telemetry data storage
(this was an ESOC concept, defining an observation as the sub-interval of an
observing period in which an instrument was in a fixed configuration and
operating mode : therefore there was a set of data files for each numbered
observation).
- the definition of standard file formats, usually arranged with a header and
a data portion, and generally using the native binary representation of the
machine. These formats were, when possible, defined to be as natural and
instrument independent as possible. They included :
- images in two format, the original LXIA2 fixed size Exosat
specific one, and the new, natural and general LXIA7 format.
- sum signal histograms specific of Exosat LE
- rate files, i.e. light curves, in a format originally defined at
ESOC, and later extended at IFCTR (to support also things as HK time profiles),
but preserving the arrangement of bins in frames.
- AOCS profiles, specific of Exosat
- spectra, defined at IFCTR (later also as pseudo-spectra to support
LE count rates for fitting) in a reasonably natural way, except that they were
defined to support also sequences of multiple spectra for time-resolved
spectral analysis
- response matrices for spectral fitting, also defined at IFCTR
- ancillary files like grouping masks and time windows were instead plain
"ASCII" files, as well as the PABLO files (generic tabular files suitable
for plotting)
- calibration files were instead kept in the original dreadful CCF format
defined at ESOC (all data as records in a single binary file) and
accessed in a neater way through a suitable library
- commands may operate on more files (e.g. algebra) or on one (e.g. header or
data modification)
ad hoc program transfored multiple spectral files (sequences) to the rate or
image format for particular purposes or for display.
timing programs were desigend to work on both the more compact rate files as well as
on the simpler but bulkier PABLO files (the former for Exosat data, the latter for
generic time profiles, like optical photometry)
- interactive graphics (using cursor) was done on HP graphics terminals or on
the Ramtek TV. In the latter case mixing of image and vector graphics was
fully supported at library level.
some adhoc programs (sum signal or AOCS profiles) used the HP autoplot
facility of the HP2648 terminals
plotting of all other stuff occurred (mainly via PABLO) on the HP inclusive of
contouring for images and chi-square grids (no IBM graphics at this stage).
One can see that these concepts build willingly upon those defined in nuce in the former
IUE era. Whenever external conditions allowed, things were
defined with portability and mission independence in mind.
This package was quite satisfactory as a first prototype of a complete system for
X-ray spectral analysis (despite of the limitations given by external conditions in
defining file formats, or given by limitations of the then current operating systems
and hardware) : the planned streamlining and rationalization of these concepts in
the XAS era was rather successful (as it was the optimization in a better programming
style), but the degree of completeness of this Exosat system was far from achieved,
at least in the public version, and some programs are still awaiting conversion.
Lucio Chiappetti 04 Sep 02 14:21
sax.iasf-milano.inaf.it/~lucio/WWW/SW/exo.html
:: original creation 2002 set 04 14:21:12 CEST ::
last edit 2002 Sep 04 14:21:12 CEST