Reducing VLT/FORS2 data

File conversion

I converted .fits files to .sdf files, then renamed them and put them into directories (CHIP1 and CHIP2, depending on the CCD) using this script:

#!/bin/tcsh
starlink >& /dev/null
kappa >& /dev/null
figaro >& /dev/null
convert >& /dev/null
if ( -e ~/adam/fits2ndf.sdf ) rm ~/adam/fits2ndf.sdf
foreach infile (*.fits)
  set outfile = `echo $infile | sed s/:/-/g | sed s/T/_/g | sed s/FORS2.//g`
  set outfile = $outfile:r:r".fits"
  set ccd = `fitshead $infile | grep EXTNAME`
  set ccd = `echo $ccd[3] | sed s/"'"//`
  if ( -e $ccd ) then
    if ( ! -d $ccd ) then
      echo '### Problem: cannot create directory $ccd'
      end
    endif
  else
    mkdir $ccd
  endif
  fits2ndf $infile $ccd/$outfile:r
  echo $infile moved to $ccd/$outfile:r.sdf
  echo $infile $outfile >> rename.lis
end
exit

Observing log

I got rid of the CHIP2 images (because they hold no data useful to me) and then made an observing log from the headers of the CHIP1 images using this script:

#/bin/tcsh
starlink >& /dev/null
if ( -e obslog.txt ) then
  echo '### File obslog.txt already exists. Stopped.'
else
  foreach file ($argv)
    hdstrace $file:r".more.fits" nlines=all eachline > zzz_obslog.tmp
    set mjdobs  = `grep "'MJD-OBS" zzz_obslog.tmp | cut -c 22-35`
    set target  = `grep "TARG"     zzz_obslog.tmp | cut -c 38-47`
    set grism   = `grep "GRIS1 NA" zzz_obslog.tmp | cut -c 38-48 | sed s/"'"//g`
    set exptime = `grep "'EXPTIME" zzz_obslog.tmp | cut -c 27-35`
    set obscatg = `grep "DPR CATG" zzz_obslog.tmp | cut -c 38-45`
    set obstype = `grep "DPR TYPE" zzz_obslog.tmp | cut -c 38-45`
    set ra      = `grep "'RA     " zzz_obslog.tmp | cut -c 26-35`
    set dec     = `grep "'DEC    " zzz_obslog.tmp | cut -c 26-35`
    set airmass = `grep "AIRM STA" zzz_obslog.tmp | cut -c 44-48`
    echo $file:r $mjdobs $target $grism $exptime $obscatg $obstype $ra $dec $airmass >> obslog.txt
  end
endif
rm zzz_obslog.tmp
exit

Debiassing

I found that the bias images for CCD1 were very flat and well-behaved. I had some problems median-stacking them with medsky, which I traced to some bias images being larger than they should be (I am not sure why). Once the problem images had been removed I took the median of the remaining images to make a master bias.

grep BIAS obslog.txt > bias.lis
medsky bias.lis FALSE masterbias
delobj masterbias.axis
delobj masterbias.wcs

I have found that there is no region on the science images which gives a reliable bias level: what I initially thought was an overscan region is actually illuminated in those observations which used the 1400V grism. Therefore I will debias the other images simply by subtracting the master bias image them using the figaro command isub.


Preparing images for reduction

Preparing images for reduction: I ran this script on all the images from CHIP1 after creating the master bias image. It deletes some unwanted header items (which would mess up plotting later on), fixes the headers, rotates and debiases:

#!/bin/tcsh
# Shell script to prepare FORS2 data for pamela
# Written by T.R.Marsh 18/03/2004
# Modified by JKT 07/02/2006
if($#argv < 2) then
  echo "usage: night_number file1 file2 ..."
  exit
endif
set night = $argv[1]          # Night number needed for fixhead command
shift
starlink >& /dev/null
convert >& /dev/null
kappa >& /dev/null
figaro >& /dev/null
pamela >& /dev/null
      # Get the grism name from the fits header (files will be put into directories according to their grism)
      # delete from the header some unwanted stuff using the Starlink/figaro/delobj command
      # Rotate the image by 90 degrees so it satisfies pamela's assumed orientation
      # Run the pamela/fixhead command to sort out some header information for later
      # Debias the image by subtracting the master bias image
foreach file ($argv)
  if ( -e zzz_prepare.sdf ) rm zzz_prepare.sdf
  if ( -e zzz_prepare2.sdf ) rm zzz_prepare2.sdf
  cp $file zzz_prepare.sdf
  delobj zzz_prepare.wcs
  delobj zzz_prepare.axis
  isub zzz_prepare masterbias zzz_prepare2
  rotate zzz_prepare2
  fixhead $night 9 zzz_prepare2
  mv -f zzz_prepare2.sdf $file
end
exit

Separating out different types of images

Now there are several different types of image here: biases, flat-fields (useful ones with grisms, and useless ones without), acquisition observations (both images of the field of the target and images of the target in the spectrograph slit without a grism in place), science (containing spectra of the target stars), arc-lamp spectra (for wavelength calibration, which go in the same place as the science spectra) and various other crap. I used this script to split the observations up and group similar types of images into particular directories:

#!/bin/tcsh
starlink >& /dev/null
convert >& /dev/null
kappa >& /dev/null
figaro >& /dev/null
pamela >& /dev/null

foreach file ($argv)
  hdstrace $file:r".more.fits" nlines=all eachline > zzz_split.tmp
  set grism = `grep "GRIS1 NAME" zzz_split.tmp`
  set grismsize = `echo $grism | wc -w`
  if ( $grismsize >= 7 ) then
    set grism = `echo $grism[7] | sed s/"'"//g`
  else
    set grism = 'none'
  endif
  set obscatg = `grep "DPR CATG" zzz_split.tmp | cut -c 38-45`
  set obstype = `grep "DPR TYPE" zzz_split.tmp | cut -c 38-45`

  if ( $obscatg == 'SCIENCE' && $obstype == 'SKY' ) then
    if ( ! -e $grism ) mkdir $grism
    cp -f $file $grism/$file
    echo $file found to be science: moved to $grism/$file
  else if ( $obscatg == 'ACQUISIT' && $obstype == 'SKY' ) then
    if ( ! -e acquisition ) mkdir acquisition
    cp -f $file acquisition/$file
    echo $file has been found to be an acquisition and moved to acquisition/$file
  else if ( $obscatg == 'ACQUISIT' && $obstype == 'SLIT' ) then
    if ( ! -e onslit ) mkdir onslit
    cp -f $file onslit/$file
    echo $file has been found to be an acquisition on slit and moved to onslit/$file
  else if ( $obscatg == 'CALIB' && $obstype == 'BIAS' ) then
    if ( ! -e bias ) mkdir bias
    cp -f $file bias/$file
    echo $file found to be a bias: moved to bias/$file
  else if ( $obscatg == 'CALIB' && $obstype == 'FLAT,LAM' ) then
    if ( $grism != 'none') then
      if ( ! -e flat ) mkdir flat
      if ( ! -e flat/$grism ) mkdir flat/$grism
      cp -f $file flat/$grism/$file
      echo $file found to be a flat for grism {$grism}: moved to flat/$grism/$file
    else
      echo $file found to be a flat without a grism: not moved
    endif
  else if ( $obscatg == 'CALIB' && $obstype == 'WAVE,LAM' ) then
    if ( ! -e $grism ) mkdir $grism
    cp -f $file $grism/$file
    echo $file found to be an arc: moved to $grism/$file
  else
    echo $file has obscatg=$obscatg and obstype=$obstype and has not been moved
  endif
end
rm zzz_split.tmp
exit

Creating flat-field images

Flat-field images. For each wavelength range (in this case I used grisms 1400V and 1200R) I combined all decent flat fields (those with plenty of counts but which were not saturated) into a master flat by median stacking. I fitted the log of the spectral response of the lamp with polynomials of high order (9 for the 1400V grism and 11 for the 1200R grism) because these provided clearly better fits than lower-order polynomials but were still robust.


Setting up the reduce scripts

I made a separate reduction datafile for the two grisms and ran reduce on all (science and arc) images from all nights for each grism. For each grism, I ran track then regpic then skyfit then skymov the profit on the first image of the set, adopting two sky regions of about 100 pixels each and carefully recording the value of the spectrum position from skymov. In the datafiles I adopted block sizes of 10 for track_nblock and 5 for profit_nblock. Readout noise was set to 5.0 and photon gain to 0.7 from information supplied at the telescope by the night astronomer. The datafiles can be found here for the 1400V and 1200R grisms.

I ran the reduce script using the commands

\ls 2*sdf > science.lis
reduce datafile @science.lis

Problems encountered:

  1. I initially got an incorrect value for the reference position of the spectrum in the datafile, resulting in large and incorrect sky estimation and extremely nasty spectra.
  2. skymov found a cosmic ray instead of one of the spectra, resulting in the wrong regions being used to extract the spectrum and the sky. I solved this by decreasing the limits over which skymov looks for spectra, so the problematic cosmic ray region was not included.
  3. I got mixed up with the sky files generated by regpic, skyfit and skymov. As a guideline, call the output file from regpic mastersky.sdf and don't allow it to be overwritten. This goes in the reduction datafile for parameter master sky.

Wavelength calibration

Arc line maps

Creating the arc line map for the 1200R grism was very simple due to the profusion of strong lines and the decent arc line lists provided by ESO for FORS2. I used a polynomial of order 4, giving an r.m.s. of the fit of about 0.01Å (about 0.5 km/s at Halpha). My masterarc.lis master arc map file for 1200R contains:

  5944.830   104.056     0.005 User          20931.5   0
  6074.338   291.628     0.005 NeI           22383.4   0
  6096.160   322.875     0.004 User          37882.1   0
  6143.063   389.669     0.003 User          70602.0   0
  6266.495   563.468     0.003 NeI           45191.4   0
  6334.428   658.001     0.003 User          66359.5   0
  6382.991   725.160     0.002 User          87876.0   0
  6402.246   751.681     0.002 User         215974.3   0
  6506.528   894.480     0.002 User         105443.3   0
  6532.880   930.346     0.004 User          33346.5   0
  6598.953  1019.921     0.003 NeI           43572.7   0
  6678.149  1126.724     0.001 User         375972.3   0
  6717.040  1178.903     0.003 User          52516.3   0
  6929.468  1462.013     0.003 User          70805.4   0
  6965.431  1509.680     0.001 User         913651.6   0
  7032.413  1598.200     0.002 User         218959.3   0
  7147.041  1749.416     0.004 User          37132.0   0
  7245.167  1878.546     0.003 NeI           73365.5   0
  7272.930  1915.086     0.002 User         194542.2   0
  7281.349  1926.124     0.005 User          23760.3   0

However, creating masterarc.lis for the 1400V grism was more complicated because there are very few decent lines. I eventually managed to measure quite small lines and fit a fourth-order polynomial, giving an r.m.s. of the fit of about 0.02Å (about 1.2 km/s at Hbeta). My masterarc.lis for 1400V contains:

  4713.200   141.122     0.016 User           2475.7   0
  4799.920   284.048     0.025 User           1111.9   0
  4921.929   481.699     0.008 User           7842.0   0
  5015.675   631.362     0.004 User          35563.1   0
  5085.824   742.252     0.015 User           2569.6   0
  5400.562  1231.186     0.011 User           5017.6   0
  5460.742  1323.415     0.001 User         618145.8   0
  5769.598  1793.420     0.003 User          70059.1   0
  5790.656  1825.329     0.003 User          68833.9   0
  5852.488  1919.102     0.001 User         354046.6   0

Note the much smaller strengths (second column) of the lines which I was forced to use.

Applying the arc-line calibration to the spectra

This was accomplished by using this shell script:

#!/bin/csh

\ls *arco.sdf > arco.lis
\ls *opt.sdf  > opt.lis
\ls *skyo.sdf > skyo.lis
\ls *arcn.sdf > arcn.lis
\ls *nor.sdf  > nor.lis
\ls *skyn.sdf > skyn.lis

molly << EOF
mxpix 4000 SURE
confirm

lndf arco.lis 1
arc 1
abort
load
masterarc.lis
f
c
4
1000
master
confirm
c
2
10000

tweak
0.2
1 100
3.5
0
co
1 1000
0.2
q
write arco 1 1000 n

lndf opt.lis 1001
hfix 1001 2000 VLT
acal 1001 2000 1 1000 n 1.0d8 1.0d8 0.0001
write opt 1001 2000 n

lndf skyo.lis 1001
acal 1001 2000 1 1000 n 1.0d8 1.0d8 0.0001
write skyo 1001 2000 n

lndf arcn.lis 1
arc 1
co
1 10000
0.2
q
write arcn 1 1000 n

lndf nor.lis 1001
hfix 1001 2000 VLT
acal 1001 2000 1 1000 n 1.0d8 1.0d8 0.0001
write nor 1001 2000 n

lndf skyn.lis 1001
acal 1001 2000 1 1000 n 1.0d8 1.0d8 0.0001
write skyn 1001 2000 n
q
y
EOF

exit

Sky-line wavelength calibration

This was needed here because there are no arc lamp exposures during the night. Whilst ESO reckons that this should not be a problem because FORS2 is very rigid, I found shifts of up to 0.6 pixel overall (about 20 km/s) in some night sky emission lines. The sizes of the shifts depend very clearly on airmass (i.e. the pointing altitude of the telescope) and definitely need to be compensated for. The 5577Å sky line was useful for the data taken with the 1400V grism, and the 6300Å line for the 1200R grism data.


Separation of spectra of different objects

Once I had adjusted the wavelength calibration usingthe sky emission lines, I separated out the resulting opt-fix.mol file into individual files for each target by using the select.csh shell script explained here.