From bschlesinger@nssdca.gsfc.nasa.gov Sat Oct 22 22:31:59 1994 From: bschlesinger@nssdca.gsfc.nasa.gov (Barry M. Schlesinger) Newsgroups: sci.astro.fits,sci.data.formats Subject: FITS basics and information (periodic posting) - 1/2 Followup-To: poster Date: 18 Oct 1994 17:18 EDT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Distribution: world Reply-To: fits@nssdca.gsfc.nasa.gov NNTP-Posting-Host: nssdca.gsfc.nasa.gov Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Summary: This section of the posting contains the table of contents, a brief introduction to FITS, and a discussion of FITS documentation. Last-modified: 1994/10/18 News-Software: VAX/VMS VNEWS 1.41 FITS basics and information, Part 1 Preface This basic Flexible Image Transport System (FITS) information is posted and updated periodically by the NASA/Science Office of Standards and Technology (NOST) FITS Support Office, under the overall supervision of Donald M. Sawyer (GSFC/NSSDC). It provides a brief description of FITS and information on software and documentation, discusses some topics that have appeared on sci.astro.fits, and answers some questions on FITS frequently received by the FITS Support Office. Changes from Previous Month o Description of FITS software available for xv 3.0 applications o Information on World Coordinates software available from NRAO o Reorganized description of the NRAO FITS archive. o New NOST library hours Table of Contents Part 1 Preface Changes from Previous Month Table of Contents 1 Introduction 1.1 What FITS Is 1.2 How FITS Evolves 1.3 What FITS Is Not 2 FITS Documents 2.1 The FITS Papers 2.2 Binary Tables 2.3 User's Guide 2.4 NOST Definition of FITS 2.5 Floating Point Agreement 2.6 World Coordinates Part 2 3 Software and Sample Data 3.1 NOST 3.1.1 FITS Product Conformance Tester with Instructions 3.1.2 Header Lister 3.1.3 Error Test Files 3.2 HEASARC 3.2.1 FITSIO 3.2.2 FTOOLS 3.3 ADC FITS Tables Browser 3.4 Converting FITS Files to Image Format 3.4.1 Major Astronomical Imaging Packages 3.4.2 pbm+ 3.4.3 IMDISP (IBM/PC) 3.4.4 Applications with xv 3.0 3.4.5 ViewFITS (OS/2) 3.4.6 FITS and the Macintosh 3.5 World Coordinates 4 On-line Information Sources 4.1 NOST 4.2 HEASARC 4.3 NRAO 4.4 HEAFITS exploder 5. Contributors (non-exhaustive list) 6. NOST services 1 Introduction 1.1 What FITS Is FITS (Flexible Image Transport System) is a data format designed to provide a means for convenient exchange of astronomical data between installations whose standard internal formats and hardware differ. A FITS file is composed of a sequence of Header Data Units (HDUs). The header consists of keyword=value statements, which describe the format and organization of the data in the HDU and may also provide additional information, for example, about instrument status or the history of the data. The data follow, structured as the header specifies. The data section of the HDU may contain a digital image, but, except for the first, *it doesn't have to*. Other possible formats include tables and multidimensional matrices that are not images. The first HDU must contain a multidimensional matrix or no data at all; the data in subsequent HDUs, called extensions, may be of any type, consistent with certain rules. The "Image" in the name comes from the original use of the format to transport digital images, but it's not just for images any more. FITS supports 5 data types in the multidimensional array of the first HDU: 8-bit unsigned binary integers, 16-bit twos-complement signed binary integers, 32-bit twos-complement signed binary integers, 32-bit IEEE-754 standard floating point numbers, and 64-bit IEEE-754 floating point numbers. For signed integers, the byte that includes the sign bit is first and the byte that has the 1-bit as its least significant bit is last. FITS does not support the 16-bit unsigned integer data type generated by many analog/digital converters. Conforming FITS files can be produced from such data by subtracting 32768_decimal from the converter output before writing to the file, while setting the BZERO keyword in the FITS header equal to 32768 and the BSCALE keyword equal to 1. A FITS reader will then add 32768 to the value in the file, restoring the original value, before interpreting it. Whether a 16-bit unsigned data type should be added, and if so, how, is controversial and under discussion, especially in sci.astro.fits. 1.2 How FITS Evolves The international authority for FITS is the International Astronomical Union (IAU) FITS Working Group (IAUFWG), which was given authority over FITS matters by the 1988 IAU General Assembly. This Group is associated with the Working Group on Astronomical Data. The current chair is D. Wells (NRAO) and the vice-chair is E. Raimond (NFRA). When the developer of a data structure finds that it does not fit well into an existing standard FITS format, a new design may be developed. No change can be made that would cause existing FITS files to be out of conformance -- the "once FITS, always FITS" rule. A unique name for any new extension type must be registered with the IAU FITS Working Group, optionally through the NOST FITS Support Office. After community discussion, most of which will be electronic, a formal proposal is distributed. This proposal is discussed by the community and may be further modified. Tests are run using the new format to confirm that it can be practically used for data transport. If the astronomical community reaches a consensus that the proposal should be adopted as standard FITS, and if successful data transfer using the proposed extension can be demonstrated, it is submitted for ratification to the regional committees--the European FITS Committee, the Japanese FITS Committee, and the American Astronomical Society Working Group on Astronomical Software (WGAS) FITS Committee. Following approval by the regional committees, it is submitted to the IAU FITS Working Group. Approval by the Working Group establishes it as a standard extension. 1.3 What FITS Is Not FITS is not principally a graphics format designed for the transfer of pictures; it does not incorporate "FITS viewers", packages for decoding the data into an image. Users must develop or obtain separate software to convert the data from the FITS file into a form that can be readily displayed. There is no standard package for all applications; section 3.3 discusses some possibilities. 2 FITS Documents 2.1 Published Papers The fundamental references on FITS are the following five papers. The first four have often been referred to collectively as the "Four FITS Papers". These papers are the formal standard for FITS, endorsed by the IAU. Wells, D. C., Greisen, E. W., and Harten, R. H., "FITS: a flexible image transport system," Astronomy and Astrophysics Supplement Series, 44, 363-370, 1981. Greisen, E. W. and Harten, R. H., "An extension of FITS for small arrays of data," Astronomy and Astrophysics Supplement Series, 44, 371-374, 1981. (NOTE: The format described in this paper has been used almost exclusively to transport radio interferometry and is likely to be replaced by other formats in the future. Writing data other than radio interferometry data using this format is not recommended.) Grosbol, P., Harten, R. H., Greisen, E. W., and Wells, D. C., "Generalized extensions and blocking factors for FITS," Astronomy and Astrophysics Supplement Series, 73, 359-364, 1988. Harten, R. H., Grosbol. P., Greisen, E. W., and Wells, D. C., "The FITS tables extension, Astronomy and Astrophysics Supplement Series, 73, 365-372, 1988. Ponz, J. D., Thompson, R. W., and Munoz, J. R., "The FITS Image Extension," Astronomy and Astrophysics Supplement Series, 105, 53-55, 1994. 2.2 Binary Tables On June 15, 1994, the IAU FITS Working Group announced the acceptance of BINTABLE, the binary table extension, as a standard extension. Copies of the final proposal are available from the NRAO site (see section 4.3, in part 2). 2.3 User's Guide A User's Guide for FITS, commissioned by NASA Headquarters, is maintained by the NOST FITS Support Office. This Guide is intended to be a tutorial for new FITS users. In addition to presenting the rules of FITS, it provides some of the history and reasoning behind the choice of the rules, adds recommendations on good practices, and discusses current developments in FITS. The current version, 3.1, was issued in May 1994. 2.4 NOST Definition of FITS The NOST has codified FITS as endorsed by the IAU into a formal standard, eliminating some contradictions and ambiguities in the original FITS papers. This Definition of FITS, version 1.0, was developed by a Technical Panel chaired by Dr. Robert J. Hanisch (STSci), with review by the astronomical community. On June 18, 1993, it was approved as a NOST Standard by an Accreditation Panel consisting of the NOST Executive Board and an astronomical community representative; this review was to confirm that the community had been given a satisfactory opportunity to review the standard and that the Technical Panel had properly considered and responded to all comments. The NOST standard has been submitted to the IAUFWG for endorsement as the international FITS standard, to replace the endorsed standard -- originally the four FITS papers, to which the Floating Point Agreement (section 2.4), and now the IMAGE and binary table extensions and the physical blocking conventions have been added. While oversights in non-controversial areas may be rectified as a result of the review by the IAUFWG, significant changes are unlikely, because members of this committee were active in the process of reviewing the standard and their comments were given significant weight in the deliberations of the Technical Panel. A new Technical Panel has been formed to draft a revised NOST definition of FITS, incorporating IMAGE, BINTABLE, and the blocking agreements. 2.5 Floating Point Agreement Originally, FITS permitted only integers in the data array following the first, or primary header. The IAU has since endorsed the Floating Point Agreement, which specifies the use of IEEE-754 floating point and describes its use in FITS. The basic agreement appears verbatim in the User's Guide, and the substance is incorporated in the NOST standard. 2.6 World Coordinates A draft text of conventions for World Coordinates is currently under community review. It proposes rules for relating a FITS data array to the physical quantities the numbers represent, with detailed discussion of projections from the celestial sphere to the array plane. It is available electronically from the NRAO site. From bschlesinger@nssdca.gsfc.nasa.gov Sat Oct 22 22:32:02 1994 From: bschlesinger@nssdca.gsfc.nasa.gov (Barry M. Schlesinger) Newsgroups: sci.astro.fits,sci.data.formats Subject: FITS basics and information (periodic posting) - 2/2 Followup-To: poster Date: 18 Oct 1994 17:27 EDT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Distribution: world Reply-To: fits@nssdca.gsfc.nasa.gov NNTP-Posting-Host: nssdca.gsfc.nasa.gov Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Summary: This part discusses FITS software, network resources, and NOST services. News-Software: VAX/VMS VNEWS 1.41 Last-modified: 1994/10/18 FITS basics and information, Part 2 3 Software and Sample Data 3.1 NOST All NOST software is available from the NOST ftp site (4.1). 3.1.1 FITS Product Conformance Tester with Instructions The FITS Product Conformance Tester is a software package in C being developed by the NOST for validating FITS files. The available prototype validates required keywords in the primary header and, at the user's option, prints selected values from the primary data array. Because this software is still under development, it should not be run before reading the separate instructions file. 3.1.2 Header Lister This program prints out all the headers in a FITS file, including the primary header and all extension headers. It does not evaluate them for errors. It is a useful tool for obtaining a quick summary of the contents of a FITS file. 3.1.3 Error Test Files These files consist of several versions of the same FITS file, a valid one and several with different kinds of header errors. They are useful for testing the ability of software designed to read FITS files to cope with files that are not standard FITS or with errors in the file. The file includes only a primary header and data array. Be sure to use binary transfer for ftp access of FITS test files. 3.2 HEASARC The NASA/Goddard High Energy Astrophysics Science Archive Research Center (HEASARC) has developed and maintains FITS software and utilities, including the FITSIO package and FTOOLS utilities. See 4.2 for information on electronic access to FITSIO and supporting documentation. 3.2.1 FITSIO The FITSIO package, maintained by Bill Pence, provides FORTRAN software for easily reading and writing FITS format files. There is an accompanying set of C wrapper routines called CFITSIO that provides a convenient interface for use by C programmers. FITSIO runs on most types of machines. It supports all the currently defined standard FITS extensions including IMAGE and BINTABLE. 3.2.2 FTOOLS James Kent Blackburn and Bill Pence document the FTOOLS collection of utility programs to create, examine, or modify FITS data files. These programs are useful for examining the contents of FITS files and modifying them for input to more involved analysis tasks; they cannot generally be used for detailed data analysis or model fitting. The current version 2.9 was released in May 1994; new versions are released about every 3 months. The FTOOLS tasks can be run as an IRAF package, which has been tested on SUNs and DECstations, or as a set of stand alone executable tasks, a version that also runs on VAX/VMS and on Alpha OSF/1 operating systems. 3.3 ADC FITS Tables Browser The Astrophysics Data Center has released the ADC FITS Table Browser, which has been tailored specifically for use with the ADC CD-ROMs but may be used with other FITS ASCII Tables. It reads standard FITS ASCII tables and allows the user to interactively browse through them and selectively display any field or record in a table. File extraction facilities allow the writing of all or part of the input table to disk in FITS or text file format. Copies of the program for MS-DOS and Unix are available by anonymous FTP on hypatia.gsfc.nasa.gov, in the directory /pub/software/ftb. See the file README.FTB for instructions on downloading, installation, and use. 3.4 Converting FITS Files to Image Format Disclaimer: The mention of particular software packages is not necessarily intended as an endorsement of those packages to the exclusion of others. Information about publicly available nonproprietary packages is welcome, and, if the package appears relevant and useful, will be added to this posting. Information about such packages should include how they can be obtained and whom to contact with questions. It should also describe any limits on the FITS files that the package can handle (e. g., NAXIS must be 2; data array members must be integers). 3.4.1 Major Astronomical Imaging Packages The three major astronomical imaging packages, the Astronomical Image Processing System (AIPS), the European Southern Observatory Munich Image and Data Analysis System (ESO-MIDAS), and the Image Reduction and Analysis Facility (IRAF) provide facilities for displaying images stored in FITS files. These packages are large and probably best installed on major systems. AIPS was developed by the National Radio Astronomy Observatory (NRAO), and IRAF by the National Optical Astronomy Observatories; the origins provide an indication of the best applications. Europeans might find ESO-MIDAS more convenient. 3.4.2 pbm+ The Extended Portable Bitmap Toolkit (pbm+) can be used for converting many FITS files to image format. However, support is not guaranteed for all FITS files where the data are in the form of an image. In particular, there may be problems when the data array members are in IEEE floating point format (BITPIX<0) or the array has more than two dimensions (NAXIS>2). 3.4.3 IMDISP (IBM/PC) Archie Warnock and Ron Baalke have announced release of version 7.9 of the IMDISP program. IMDISP is an interactive image processing program that runs on an IBM PC computer and supports FITS input. IMDISP 7.9 is available via anonymous ftp at explorer.arc.nasa.gov [128.102.32.10] in a file called imdisp79.zip in the pub/SPACE/SOFTWARE subdirectory and at hypatia.gsfc.nasa.gov in the pub/software/imdisp subdirectory. It may also be available (at least for the time being) through Simtel-20 [192.88.110.20] 3.4.4 Applications with xv 3.0 For a system that has xv 3.0, some unsupported modifications to apply it to FITS files exist. Adam Bernstein (adam@dragon.jpl.nasa.gov) has made some additional modifications to the original set, to overcome some implementation problems. They are available by ftp to csi.jpl.nasa.gov in the /pub/users/adam directory. They can handle files with any of the valid FITS values of BITPIX, and can handle NAXIS>2 provided NAXISn=1 for n>2 (a convention often used in radio astronomy). The file "convert.tar.Z" contains these changes, as well as a new program called "FITS2GIF" created to allow non-interactive conversion of a large number of FITS files to GIF format. Use of xv 3.0 at a site must be licensed. 3.4.5 ViewFITS (OS/2) Dominique Beauchamp, Universite Laval, Quebec City, has released version 0.4, rel. 1 of ViewFITS, a FITS reader for OS/2 2.1 presentation manager. It is on ftp.cdrom.com in /pub/os2/incoming, file name vf04r1.zip, although it will probably be moved to /pub/os2/2_1/graphics/vf04r1.zip. This program gives the user the opportunity to display FITS images (8,16 or 32 bits, integer or float), modifying the gray shade palette, doing negation and zooming out. 3.4.6 FITS and the Macintosh Thorsten Lemke, at Brunswick in Germany has added FITS support to GraphicConverter for the Macintosh and provided the following information. GraphicConverter 1.7.7 or later can import FITS array files with all five permitted data types (8, 16, 32 bit integer and 32, 64 bit real). Every FITS file will be converted to 8 bit gray scale on opening because this is the maximum number of grays on a Macintosh. GraphicConverter can convert FITS and other files to a lot of different graphic file formats i.e. PICT, TIFF, GIF, PCX, IFF, PPM... GraphicConverter is available via ftp info-mac archives. A mirror site is amug.org 165.247.10.2 pub/ftp1/info-mac 1/1 ALL ftp email Arizona Macintosh Users Group, Phoenix, Arizona, USA GraphicConverter is in the folder grf/util/graphicconverter* The site at NRAO described in 4.3 contains a collection of Usenet postings and electronic mail messages on the subject of use of FITS on Macintosh hardware, in fits/os-support/mac-os/. 3.5 World Coordinates Two ANSI C functions, worldpos() and xypix(), convert (RA, dec) <--> pixel location for 8 common types of projective geometries where "(RA,Dec)" are more generically (long,lat). These functions are based on the World Coordinates implementation of Classic AIPS. The software is in src/wcs/worldpos.tar.gz at the NRAO site (section 4.3) and supporting documentation can be found in documents/wcs/. 4 On-line Information Sources 4.1 NOST The NOST maintains a file of FITS information available by anonymous ftp from nssdca.gsfc.nasa.gov/fits or by DECnet copy from NSSDCA, in the directory FITS. A proposed reorganization could move the FITS material to a FITS subdirectory under a STANDARDS directory. The FITS files include copies of the current NOST standard, the Definition of FITS, in flat ASCII, PostScript, and LaTeX. Style and index files are provided for the LaTeX form. The only difference among the three formats is that the ASCII form lacks typeface formatting; therefore, only one need be retrieved. A current list of the extension type (structure) names registered with the IAU FITS Working Group is maintained. Also available, in LaTeX form, is a copy of the proposal for the IMAGE extension, an earlier version of the paper cited in 2.1. The current version of the User's Guide is available in LaTeX, Unix(.Z)-compressed PostScript and uncompressed PostScript, along with a style file for the LaTeX version. A copy of the conventions for physical blocking on various media, as adopted by the IAU FITS Working Group, is available in flat ASCII text form. The directory also contains a modified version of this posting. An AAREADME.DOC file describes the contents of the directory. A SOFTWARE subdirectory contains the FITS Product Conformance Tester prototype, along with instructions. The instructions should be retrieved along with the software. The C software to read and list the headers of a FITS file is also in this subdirectory. Another file provides information on publicly available FITS software packages. The ERRTEST subdirectory contains the error test files described in section 3.1.1. Remember that binary transfer should always be used for FITS files. Both the SOFTWARE and ERRTEST subdirectories include AAAREADME.DOC files describing their content. 4.2 HEASARC The FITSIO package, FTOOLS utilities, supporting documentation, and some other software resources are available from ftp://legacy.gsfc.nasa.gov/software. The FTOOLS software is being distributed as compressed tar files. The fitsio and ftools subdirectories both have descriptive README files; in addition, the fitsio subdirectory contains documentation files. 4.3 NRAO A library of FITS material can be found at ftp://fits.cv.nrao.edu/fits, located at NRAO. This machine supports a WAIS server named nrao-fits which has an index of all of the FITS-related text files in the archive; the file nrao-fits.src is available at think.com and at ftp://fits.cv.nrao.edu/fits/wais-sources/nrao-fits.src. There is a World Wide Web server for the FITS Archive as well, at the URL http://fits.cv.nrao.edu. The documents subdirectory of the fits directory contains a number of subdirectories. A proposals subdirectory is reserved for detailed proposals currently being considered by the FITS committees. There are none at present, IMAGE and BINTABLE having now been accepted by the IAU FITS Working Group. A drafts subdirectory contains drafts of designs not yet submitted, for example, a proposed method for incorporating data compression under FITS. The wcs subdirectory contains a draft of the current proposal for world coordinate system conventions now under community review and earlier documents and presentations on world coordinates. Other subdirectories of the fits directory include sample FITS files -- both actual data files and files specially constructed to test the ability of software to read all kinds of FITS structures, some code for particular environments, pointers to other code, and an archive of Usenet postings related to FITS. 4.4 HEAFITS Exploder An electronic mail listserver called HEAFITS has been set up as a specialized group for discussion of High Energy Astrophysics- specific FITS issues that would not necessarily be of interest to the majority of subscribers to the existing sci.astro.fits newsgroup and fitsbits mailing list. To *subscribe* to the HEAFITS group, send the following one-line e-mail message to listserv@legacy.gsfc.nasa.gov: subscribe heafits Your Name where 'Your Name' is your actual First and Last names. Messages to the actual mailing list are sent to heafits@legacy.gsfc.nasa.gov. 5. Some contributors (non-exhaustive list) A. Bernstein (JPL) D. Beauchamp (U. Laval) T. Lemke (U. of Brunswick) W. Pence (GSFC/HEASARC) D. Wells (NRAO) and the participants in sci.astro.fits and the fitsbits mailing list 6. NOST Services The NOST library can provide printed copies of many of the documents listed above. Printed or electronic copies of the User's Guide and the NOST Standard, for those without ftp access, are available. Because of restrictions set by the copyright holder, NOST can send copies of the four FITS papers only to non-profit organizations. The NOST cannot provide copies of the IEEE floating point standard, which is copyright to IEEE. The NOST can be reached as follows: (Postal) NASA/Science Office of Standards and Technology Code 633.2 Goddard Space Flight Center Greenbelt MD 20771 USA (Internet) nost@nssdca.gsfc.nasa.gov (DECnet) NCF::NOST Telephone: +1-301-286-3575 9:00 a. m. - 6:00 p. m., U. S. Eastern Time (-0500 from the last Sunday in October through the first Saturday in April; -0400 the remainder of the year) If the Librarian is unavailable, a phone mail system takes the call after four rings. Please mention this posting in your request. Use the FITS office electronic mail address below for replies or questions. It is monitored by other NOST staff members when I am away from the office and provides a greater certainty of rapid response. When communicating with the NOST library or FITS Support Office, please provide your name, affiliation, and location. Barry M. Schlesinger Coordinator, NASA/NSSDC NOST FITS Support Office +1-301-441-4205 fits@nssdca.gsfc.nasa.gov NCF::FITS