History of my fitting software

Not all that is *SPEC does glitter
nor all who do not use XSPEC are lost

archeo-fitting

• My very first fitting program was written at MSSL in 1980, adapting CURFIT-based code supplied by R.J.Blisset, and was used to fit Ariel 5 X-ray data (it ran on an IBM 360)
  The competing program suite was represented by the standard SPECFIT and SPECGRID suite in the Ariel 5 standard package developed at MSSL by many persons (actually it was more complementary than in competition)

• The same program was adapted to run in Milan on an IBM 370 with VM/CMS (using front end EXECs) in 1981 (it was still a personal program used to fit Ariel 5 data I brought with me and the associated documentation still) exists but is handwritten on paper !)

• More or less at the same time I adapted it to a "version zero" of a general purpose spectral fitting program for UV spectra (hence no convolution, support of tabular data files in ASCII format, but had support of multiple spectral models. This program was named UVFIT

  This program was a late addition to the "version zero" of the IUE data analysis system which had its own ASCII format for spectra, and originally did not include any fitting program, but just a way of creating or plotting analytic model files. The original system was documented in my report R1 (entry number 15 in my publication list), inclusive of the limited number of spectral forms supported.

• While I was away it was handed over to A.L.Ciapi which did some development under the name of BIFIT et al. in 1983-84

  paleo-fitting

• When I came back I refurbished the UVFIT "version 2" (still under VM/CMS) cleaning it out a lot using :
  • INCLUDE files (as supported by VS Fortran MACLIBs) to keep standard setting for parameters
  • the READIN interface to get runtime parameters from the user (via EXEC2 console stack; the same interface did exist also for HP systems; both are documented in my report R6 (entry number 84 in my publication list),
  • using a standard library of spectral forms and associated routines (called as "revised" or "version 3" spectral form package) which is documented in my report R7 (entry number 85 in my publication list) whose main characteristics was to support spectral forms independent of the user choice for physical unix for the energy and flux axes.
  This refurbishment occurred in 1987.

• At the same time a variant UVGRID was developed (instead of fitting on all parameters did a chi-square grid stepping two interesting parameters on a 16x16 grid, producing ASCII output

• In 1992 the variant UVGRID1D was added (still on VM/CMS) based on the latest UVGRID (1989) to make a 1-d grid over a single interesting parameter.

• All the above families of UV programs (with the exception of UVGRID1D which was added later semi-privately (in collaboration with E.Pian which also exported it privately) were supported and documented care of T.Belloni together with the complete refurbished IUE data analysis package in my report R4 (entry number 82 in my publication list).
  See however report R19 (entry 120) for UVGRID1D.

  There was no actual competing program for UV spectral fitting (the fitting commands within IHAP were geared to fitting Gaussian lines, not continua)

• Let us now make one step back. The version zero of UVFIT and BIFIT were used in 1985 for the first version of the X-ray spectral fitting program called trivially FIT for use with EXOSAT data. The convolution with a response matrix was handled, together with a library of spectral forms in dN/dE, and the support of the sum of several components. This is properly a "version 1.0". Input data were represented by an ad-hoc kind of binary files.
  A "version 1.1" in 1986 added an output interface file in ASCII form to be interfaced with the PABLO plotting program (by Giommi & Chiappetti, after earlier work by P.Giommi at CfA and ESOC; the Milan version of PABLO is documented in my report R8 (entry number 92 in my publication list)).
  A "version 2" in 1987 was aligned with the corresponding version of UVFIT using the "revised spectral form package" described above.

• Similarly a GRID program undertook parallel developnment to produce chi-square grids for X-ray spectra. A difference with the UV program was that the chi-square grids were produced as binary images (in the EXOSAT ad-hoc format.
  Actually later binary images were added to the UV programs as well, and both X-ray and UV programs shared a common set of utilities to compute and plot confidence levels (also for uninteresting parameters).

• all this is documented in the handbook of the Milan EXOSAT data analysis system (also with extensive participation of B.Garilli) which is my report R2 (entry number 47 in my publication list)). See also talk T2 (entry 24 and proceeding paper P27 (entry 66 ). This package also included a spectral convolution program based on the "revised spectral form package".

  The fitting programs in use while I was at ESOC ( I would not call them competing since I started developing my FIT suite) were the *XAA* programs in the Exosat Automatic Analysis (there were never full fitting programs in the IA or Interactive Analysis) by G.Pollard et al and later a program with an awkward user interface called ESPEC which I presume was brought there by N.E.White.

• During all same years, other minor "dedicated" fitting programs were used for specific calibration tasks (i.e. programs fitting a specific hardcoded function like those used for EXOSAT CMA calibrations (see my report S1 (entry 22 in my publication list)), or Zebra/GAMTEL calibration lines (see my report S17, entry 74 in my publication list), or the GEX gaussian extraction for IUE spectra (see my report R9, entry 98 in my publication list), or finally some light-curve fitting programs like those in the add-on package for EXOSAT timing data analysis described in my report R12 (entry 104 in my publication list).

  meso-fitting

• The meso epoch is the VAX VMS and early Sun Unix era in contrast with the former IBM CMS and HP RTE paleo era. In practice there was an overlap in usage of the IBM-based fitting programs with the meso-era.
  Hence there was no formally documented porting of the above fitting programs except for some IUE related programs (but not the fitting package) as described in report R19 (entry 120)
  A further exception is represented by a stripped down version of the X-ray fitting program with an hardcoded sample dataset, which was (and still is) used a prototype for benchmarking (see report R15 (entry 113 in my publication list) on all kind of systems including those supported in the ceno era.

• The main competing program for X-ray spectral fit, XSPEC also arose at the Exosat Observatory at this time (on VAX VMS based on work by R.Schafer at Cambridge) and was later ported to other systems care of GSFC HEASARC and Exosat Observatory becoming widespread in the ceno era.

  ceno-fitting

• The ceno epoch is the (mainly Unix) era of the XAS software which was using system independence (or hidden system dependency) : all programs written in this era (in Fortran 77 or allowed extensions according to the rules devised for XAS in reports S14 and S15 (entries 69 and 70) use the standard XAS libraries and can run on any (VMS or Unix) system having a XAS VOS library. References for XAS are the design document S22 (entry 94 aka the dream book), the online manual S52 (entry 164; see also further technical details) and the proceeding P43 (entry 154).

• The "version 3" of the UV suite of programs (UVFIT, UVGRID and UVGRID1D) was ported to Unix using XAS libraries in 1993 and supported until at least 1995, with minimal changes to the previous philosophy.

  The same program has been privately supplied to B.Garilli to form the basis of a galaxy profile fitting program.

• Also the "version 3" of the X suite of programs (FIT and GRID) were ported to Unix using XAS libraries in 1993. While the basic philosophy was unchanged, the data file format was changed from the EXOSAT specific one to the instrument independent XAS data format (for a formal reference see report S35, entry 126).
  The program was actively supported adding handling preanalysis of response matrices to "compress" null elements, channel selection in energy, support of cross-normalization for multi-instrument spectra in native way unlike XSPEC which uses constant scaling multiple models. Active support lasted until 1998, when also a GRID1D program was added, while usage continues also presently in 2001.

• ad hoc fitting programs (including 2-d cases) were developed also for specific (typically calibration work) like for SAX calibrations described in report S59 (and references therein, entry 199; the main results being reported in paper J39 (entry 153)).

• ad hoc fitting routines are also imbedded in XAS (referred above) typically for gaussian fitting of instrument calibration sources.

  neo-fitting

• planned activities for the future include (not necessarily in this order) :
  • conversion to f90 or f95 (but see my worries)
  • streamlining and adding new models (particularly to the X-ray suite)
  • ideally adding new models without recompilation : a solution based on external programs connected by named pipes is possible, similar to what already done for my randomizer 1993-95 program (documented in a private report) but I doubt it will be too slow
  • improved graphics support to display fitted function and/or parameters during the fit via direct connection with the XAS graphic servers
  • possibly adding a faster confidence limit determination program which does a sort of binary search of limits exceeding the delta-chisquare without stepping.
  • development of a "universal" fitting program for any adhoc function or adaption/merging of the UV and X-ray suite into a non-convolution general purpose fitting program
  • I will definitely NOT add GUIs instead !