Mattia's Slides, 25 September 2003

Last Updated on 25 September 2003

Padova

Mattia Vaccari

Department of Astronomy and CISAS

University of Padova

vaccari@pd.astro.it

http://dipastro.pd.astro.it/~mattia http://web.pd.astro.it/poe

Signals: their Theory and Processing

By a loose ''definition'', any kind of ''random'' quantity (i.e. a quantity whose variations are ''a priori'' unpredictable in rate and amount) is a SIGNAL

SIGNAL THEORY is then concerned with clarifying the statistical properties of signal variations and how they depend on the surrounding ''environment''

SIGNAL PROCESSING is the ensemble of analytical and numerical techniques developed with the purpose of interpreting, manipulating, reconstructing... signals

Signals: Examples

Time-Varying Signals: e.g. Vibrations and Sound-EM Waves

Space-Varying Signals: e.g. Images

Signal T&P: Related Disciplines

In studying signals, one thus basically deals with unpredictable quantities (formally, random variables) and phenomena (formally, random processes)

Therefore, Signal T&P are deeply connected with more fundamental theories such as probability and statistics

Besides, digital signals are presently ubiquitous in all technical areas as they arise from (ir)regular sampling of time/space-varying "phenomena", and related Digital Signal Processing thus greatly benefits from refined computing means and techniques

Basics: Probability and Statistics

Probability Density Function

Statistical Expectation

Variance

Probability Distribution Function

Stationary and Ergodic Random Processes

Random Variable ⇔ Random Process

Set of samples x ⇔ Set of sample functions x(t)

A random process is said to be STATIONARY if its statistical properties do not depend on absolute time

A random process is said to be ERGODIC if its statistical properties do not depend on chosen sample function

In practice, one always assumes that the random process of interest satisfies these requirements, at least over the time range of interest (statistically time-varying processes are better described by WAVELET ANALYSIS)

Correlation

Different definitions of CORRELATION formalize the intuitive concept that ''to an increase of a variable X corresponds an increase/decrease of the variable Y''

Correlation Coefficient of two Random Processes

Auto-Correlation of a Random Process

Cross-Correlation of two Random Processes

Fourier Analysis

An essential tool in almost every scientific field (e.g. Optics, Image Restoration, Quantum Physics...), Fourier Analysis relies on the possibility of decomposing any periodic function x(t) into a trigonometric series

where

i.e. into a superposition of trigonometric functions whose periods and angular frequencies are

Fourier Analysis

Generalizing this result to a generic function x(t) by heuristically calculating the limit of previous relations for

one obtains that x(t) can be decomposed into the superposition of a continuous set of trigonometric functions, i.e. into a trigonometric integral

so that one can work in the ''time'' or ''frequency'' domain, from time to time exploiting the properties of both

Spectral Analysis

Since the auto-correlation function contains information on the frequencies ''making up'' a signal, one can investigate the spectral properties of signals through the Fourier Transform of their auto-correlation functions, or

SPECTRAL DENSITY

''breaking down'' the signal into its spectral components

and thus allowing the distinction of narrow-band, broad-band and white-noise processes

Spectral Density

Spectral density is related to the underlying signal by

resembling the relation between the electric field associated with an electro-magnetic wave and its intensity

The spectral density of a derived random process dx/dt is

so that

Examples

The DIFFRACTION pattern produced by an aperture is given by its Fourier Transform in the space domain

The VIBRATION pattern produced on a linear system can be described in frequency space

Digital Signal Processing (DSP)

Digital Signal Processing deals with the properties, analysis and treatment of digitized signals

The most common practical need for DSP arises when trying to determine the spectral components and in reconstructing the ''real'' signal on the basis of a discrete sequence of measurements

The Sampling Theorem

A band-limited signal can be perfectly reconstructed from samples taken uniformly at a rate > 2 max(supp(S())).

Optimal sampling is of order of this latter upper limit!

Sampling, Aliasing and Folding

Under-sampling generates spurious spectral components

Over-sampling generates a lot of useless information

Discrete Fourier Transform (DFT)

If one is to recover the spectral components of a discrete time series {x(r t)} with r=0,..., N-1 following the recipe

signal⇨auto-correlation function ⇨spectral density

the integrals defining the Fourier coefficient can be ''trapezoidally'' approximated by

which are estimates of the spectral density for frequencies {r } with  = 2p N/T so that the original discrete time series (but not its continuous ''parent'') is exactly given by



Fast Fourier Transform (FFT)

Direct calculation of DFTs requires ~ N^2 operations

FFT algorithms, starting from Cooley and Tukey's one (1965) reduces such value to ~ N log_2 N, i.e.

N log_2 N N^2 / ( N log_2 N)

4

16

64

256

1024

4096

16384

65536

262144

2

4

6

8

10

12

14

16

18

2

4

11

32

102

341

1170

4096

14564

The Large Scale Structure of the Universe

Observations of the LSS of the Universe are mostly carried out by determining 2 or 3-D galaxy positions

Clustering

''Clustering'' is usually quantified by the two-point angular/spatial correlation function defined by

and gives the excess probability of finding pairs of sources at a given distance with respect to a uniform distribution. However, the spectral density of galaxy distribution, or Fourier Transform of correlation function, gives an estimate of the spatial frequencies making up the galaxy distribution on the sky, and is extensively used whenever computational or noise-fighting related reasons make its use more effective

References

Printed Material

Newland D.E. 1993, An Introduction to Random Vibrations, Spectral and Wavelet Analysis, Third Edition, Longman

Papoulis A. And Pillai S.U. 2002, Probability, Random Variables and Stochastic Processes, McGraw-Hill

On-Line Material

On-Line Digital Signal Processing Class, Bores Signal Processing, http://www.bores.com/index_online.htm

An Introduction to Fourier Theory, Forrest Hoffman, http://aurora.phys.utk.edu/~forrest/papers/fourier/

MathWorld, Eric Weisstein's World of Mathematics, http://mathworld.wolfram.com/

The LARI Method

The LARI method was developed to overcome the main difficulties affecting ISO-CAM/PHOT data:

- Poor spatial coverage, sampling and redundancy

- Transient behaviour after a flux change

- Cosmic ray hits and ''glitches''

These severe problems call for the individual treatment of the ''time history'' of each pixel, in order to model the effects of transients and glitches and to determine the point-per-point flux excesses ascribable to potential sources and to project them onto the sky as a map

S = Signal I = Incident Flux Q = Accumulated Charge

Completely conservative model

Charge is released with two different time scales

Glitches as discontinuities in charge release

Mathematical Model for Charge Release

Pixel Time History Fitting: Glitches

Pixel Time History Fitting: Sources

IDL Developed Programs

Raw data export into IDL environment

Pipeline reduction:

- Stabilization background determination

- Strong sources' and glitches' identification

- Fitting parameters' ''first guess''

Pixel time history fitting

Interactive analysis (graphical user interface)

Mapping, mosaicing and source extraction

Flux Estimation through Autosimulation

Astrometric calibration (vs. stellar positions)

Photometric calibration (vs. stellar fluxes)

Data products formatting/release

Simulations:

- Astrometric and Photometric Accuracy

- Reliability and Completeness

Data Reduction: An Overview

The Fitting Procedure

Widget Graphical User Interface

Widget Graphical User Interface

Widget Graphical User Interface

Widget Graphical User Interface

Break: Interactive Analysis Session...

Data Products

Text Files

ISO Source Lists

Optical/NIR IDS

TYCHO2, USNO, GSC2

APM, SDSS

2MASS

Multi- Follow-Up Catalogues

Flags + Notes

Images (FITS, EPS and PNG)

Flux and S/N Sky Maps

Flux and S/N Contour Maps

Optical Finding Charts

Break: Browse Catalogues...

Documentation

NAME

PURPOSE

CATEGORY

INPUTS

OPTIONAL INPUTS

KEYED INPUTS

OUTPUTS

OPTIONAL OUTPUTS

KEYED OUTPUTS

WRITTEN FILES

DEPENDENCIES

COMMON BLOCKS

EXAMPLES

SIDE EFFECTS

RESTRICTIONS

SEE ALSO

NOTES

MODIFICATION HISTORY

IDL-like documentation within source files for each program

Overall HTML documentation hyperlinked to source files

Break: Browse Documentation...

Software Test Plan

Pipeline Testing:

- Run on different ''kinds'' of observational datasets

- Visual inspection and statistical tests

- Fine tuning of relevant parameters!

Overall Data Reduction Testing:

- Generate a catalogue of additional sources

- Simulate their effects on ''true'' pixel time histories

- Reduce such simulated data as done with real data

- Compare ''measured'' and ''injected'' quantities

- Estimate Performance of Data Reduction Technique

Observational Parameters:

- Filter

- Gain

- Pixel Field Of View

- Raster Step Size

- # of Pointings per Raster

- # of Readouts per Pointing

- Exposure Time per Readout

Pipeline Testing

Completeness: Non-Repeated Regions

Completeness: Repeated Regions

Astrometric Accuracy

Photometric Accuracy: Linearity (1)

Photometric Accuracy: Linearity (2)

Photometry: Internal Calibration and Accuracy

Photometry: External Calibration and Accuracy

Conclusions

Software Development:

- Complete

- Some add-ons desirable

Software Documentation:

- Essentially complete

- A ''proper'' User's Manual and ''recipes'' desirable

Software Testing:

- Pipeline: extensive but superficial

- Overall Data Reduction: limited but thorough

Critical Steps

Raw data export into IDL environment

Mapping/Mosaicing and source extraction

Pipeline reduction (first guess of fitting parameters)

Single pixels' time history fitting

Interactive analysis ⇒ graphical user interface

External calibration with stars

Simulations:

Astrometric accuracy

Photometric accuracy

Completeness

Possible Applications/Contributions to PACS

Experience with ISO data suggests that:

- Pre-flight detectors' characterization is most desirable

- Mission success will critically depend on timely software development

- Effective observing and data exploitation can only be guaranteed through accurate knowledge of instrumental performance

- One must be ready to promptly face unexpected problems

Willingness to provide expertise during mission planning!

Things one would like to know about PACS

Are there (planned) realistic Num. Simulations and/or

Lab. Tests reproducing the satellite operational conditions?

and in particular

How do PACS Photoconductors and Bolometers

respond to Cosmic Ray Impacts???

Some work on available material is desirable for

LARI Method Testing / Dedicated Method Development

Application of the LARI Method

Instrument:

ISOCAM LW detector : matrix 32 x 32 (pixel size  6'') f.o.v  3 ' x 3 '

ISOPHOT C100 detector : matrix 3 x 3 (pixel size  45'')

f.o.v  1.5 ' x 1.5 '

Projects being carried out:

ELAIS ( ~ 12 sq. deg.)

- CAM-LW3 filter : =15 m ( S1: Lari et al. 2001; S2: Pozzi et al. 2003, sub.)

( N1, N2, N3: Vaccari et al. 2003, in prep.)

- PHOT-C90 filter : =90 m (in progress)

- CAM-LW3 Ultra-Deep 15 ' x 15 ' N1 Field (in progress)

Lockman Hole ( ~ 40 ' x 40 ')

- CAM-LW3 filter : =15 m ( Fadda et al. 2003, in prep.)

- PHOT-C90 filter : =90 m (Rodighiero et al. 2003, sub.)

- Deeper LW3 integration on 20 ' x 20 ' central region (in progress)

The ELAIS fields as seen by CAM (@ 15 m)

ELAIS 15 m Final Analysis

Catalogue : 2000 sources

Flux range : 0.4 - 150 mJy

S2 field (21' x 21')

Simulations

- Completeness : 90% at 2mJy

- Positional accuracy : 1-2''

- Photometric accuracy : 10-20 %

A spiral @ z=0.02, S=5.6 mJy

Two interacting galaxies @ z=0.22, S=1.5 mJy

An AGN I @ z=1.56, S=6 mJy

The LOCKMAN HOLE as seen by PHOT (@ 90 m)

LOCKMAN HOLE at 90 m:

Catalogue : 36 sources

Flux range : 20 - 500 mJy

44 arcmin

N

E

Simulations

- Completeness : 80 % at 100 mJy

- Positional accuracy : 20 ''

- Photometric accuracy : 20 %

The LOCKMAN HOLE as seen by CAM (@ 15 m)

Number of sources detected:  350

Minimum flux level: 0.25 mJy

Far-IR contours on Mid-IR map

Herschel for cosmology

A review of recent facts:

The Background Radiation: new discoveries

Observations with millimetric telescopes

IR observations with space observatories

Main open problems to be addressed by Herschel cosmological surveys:

Formation of galaxies

Formation of quasars and AGNs

Relevance of long- observations: are they needed ?

Herschel cosmological surveys in the context: what is unique compared to the variety of planned space and ground experiments

COBE

Discovery of the Cosmic Infrared Background (CIRB)

Puget et al. 1996

Hauser et al. 1998

=100 m

The Infrared and Optical Background Radiations

COBE

HST

(IR)=40 nW/msr

(opt)=17 nW/msr

JCMT 15 m telescope

The SCUBA

37 element

bolometer array

operating

at =850 m

SCUBA 850 m survey

of CFRS 1415h field

Eales et al. 2000

Noise Map

6.9 x 6.4 arcmin

19 sources above 3.5 mJy

Noise Map

Signal Map

Map at 250 GHz

(1.2 mm) in

A2125 field

Bertoldi et al 2000

Scale in arcsec

rms = 0.5 mJy

IRAM

30 m

bolometer

array

SCUBA sources in A1835 field

Optical Image:

Palomar 200 UBI

Sub-mm contours:

SCUBA smoothed to

20'' resolution

Note: the brightest

SCUBA source has

no optical counterpart

Spectral Energy Distributions of sub-mm selected sources are similar to those of local UltraLuminous IR Galaxies

Arp 220

NICMOS

false-color image

(1.1-1.6-2.2 m)

Integral counts

at 850 m

Franceschini et al. 2001

170 m

Puget et al. 1999

ISOPHOT

THE INFRARED SPACE OBSERVATORY

ISOPHOT

170 m

Puget et al. 1999

Fadda et al. 2002

ISOCAM 15 m differential counts (Elbaz et al. 1999)

HDF-North image

overlayed by the

ISOCAM 15 m

contours by

Aussel et al. 1999

2 arcmin

Comoving star-formation rate density

Optical &

X-ray AGNs

UV-optical galaxies

IR-starbursts

Franceschini et al. 2001

Minimum detectable mass for

dust-enshrouded protostars

The Formation of Stars

Rich information available from observations at long  only

All these pioneering results indicate that a new important branch of astrophysics, with especially relevant implications to understanding key issues about the formation of galaxies and QSOs, and the complex physics ruling the generation of stars, has been born

Herschel

The Herschel Space Observatory (formerly known as FIRST) is the fourth cornerstone mission in ESA's Horizon 2000 programme, scheduled for launch in 2007.

A 3.5 m diameter telescope, passively cooled to around 70 K, will feed a suite of three cold focal plane instruments providing high-throughput photometry and spectroscopy in the 60 - 670 µm range.

The instruments will be contained in a liquid helium cryostat with an operational lifetime of at least three years.

Imaging Photometry

two bands simultaneously (60-90 µm or 90-130 µm and 130-210 µm) with dichroic beam splitter

two filled bolometer arrays (32x16 and 64x32 pixels)

point source detection limit ~ 3 mJy (5 sigma, 1h)

PHOTODETECTOR ARRAY CAMERA

AND SPECTROMETER (PACS)

Integral Field Line Spectroscopy

wavelength range : 57-210 µm

optical image slicer rearranges 2-D field of view (5x5 pixels) along 1-D slit (1x25 pixels)

long-slit grating spectrograph (R ~ 1500) disperses light

dispersed slit image is projected on 16x25 pixel Ge:Ga Photoconductor arrays (stressed/unstressed)

16 spectral channels recorded simultaneously for each spatial element

point source detection limit is 2.5 - 8 x 10-18 W/m2 (5 sigma, 1h)

PACS sensitivity (5  in 1h) in photometry and spectroscopy modes with (dashed lines) and without (solid lines) on-array chopping

SPIRE Instrument Features

3-band Imaging Photometer

- Wavelengths (m): 250, 350, 500

- Simultaneous observation in 3 bands

- Beam FWHM (arcsec): 17, 24, 35

- Field of view (arcmin): 4 x 8

Imaging Fourier Transform Spectrometer (FTS)

- Wavelength Range (m): 200-400 (req.) 200-670 (goal)

- Simultaneous imaging of the whole spectral band

- Field of view (arcmin): 2.0 (req.) 2.6 (goal)

- Max. spectral resolution: /=100 (req.) 1000 (goal)

- Min. spectral resolution: /=20

Spectral and Photometric Imaging Receiver (SPIRE)

The bolometer arrays (spider-web bol. with Ge-microthermometers)

Photometer observing modes

SPIRE

Imager

SPIRE Imaging Spectrometer (highly-optimized 2-band FTS)

300-670 m 200-300 m

SPIRE Performance

Photometer (mJy, 5, 1h)

Spectrometer (5, 1h)

Herschel Extragalactic Surveys

General motivation for Herschel was identified long ago in the framework of the worldwide systematic effort undertaken to Search for the Origins, involving most major future space and ground projects ( see NASA dedicated program + next-generation observatories such as ALMA, OWL, SKA )

ORIGINS OF

Large Scale Structure

Galaxies & Quasars

Stellar Populations

Planetary Systems (Life!)

ALMA

NGST

SIRTF

SCUBA 2

HERSCHEL

Can select cosmic sources from their bolometric emission over an extremely wide redshift interval

(overcoming the limitations of optical cosmological surveys such as

K-correction and limited spectral coverage, sampling the ISM)

The most efficient and unbiased way to select the active phases in galaxy evolution : surveys at 100 - 300 m (PACS + SPIRE)

Although conceived long ago, Herschel keeps unique features among existing projects to sample the peak of dust emission with enough spatial resolution

Deep ''cosmological'' imaging surveys ( + spectroscopic follow-up of significant sub-samples of detected sources) over large sky areas are among the main motivations for PACS & SPIRE and for the mission itself

Herschel as a critical element of this effort

The Herschel Window

Herschel included

confusion

SIRTF

NGST

Herschel spectroscopy of long- ionic lines

Fischer 2000

OPEN ISSUES TO BE ADDRESSED

BY HERSCHEL EXTRAGALACTIC SURVEYS

BASICALLY, THEY WILL ALLOW TO MEASURE THE

BOLOMETRIC EMISSION BY DISTANT GALAXIES

IN A COMPLETELY UNBIASED WAY

NATURE OF IR EMISSION BY FAINT IR SOURCES AND ORIGIN OF CIRB: STARBURSTS OR AGNS ?

ORIGIN OF SPHEROIDAL GALAXIES

ONSET OF QUASARS

LARGE SCALE STRUCTURE

HOW MUCH FIR OBSERVATIONS ARE NEEDED TO TRACE THE ACTIVE PHASE ?

ISO SURVEYS

0.0047, 0.0047

> 6400, > 6400

7, 15

ISOHDF South

0.0025

34000

7

Deep SSA13

0.0025, 1.2, 1

44640, 48, 128

7, 90, 175

Deep Lockman

0.067, 0.067, 0.067, 0.067

720, 1000, 3000, 3000

7, 15, 60, 90

CFRS

0.013

3520

7

CAM Ultra Deep

0.11

302

12

Comet Fields

0.0014, 0.0042

12800, 6400

7, 15

ISOHDF North

0.28, 0.28, 0.28

800, 990, 144

7, 15, 90

CAM Deep

0.42, 0.42

150, 50

60, 90

SA 57

1, 1, 1

23, 27, 27

90, 135, 175

IR Back

1, 3

256, 128

175

FIRBACK

1.3

180

15

CAM Shallow

6, 11, 12, 1

40, 40, 24, 128

7, 15, 90, 175

ELAIS

6, 11, 12, 1

40, 40, 24, 128

7, 15, 90, 175

CAM Parallel

7000

0.5

175

PHT Serendipity

Area (deg²)

Integration (s)

 (m)

Name

SIRTF LEGACY SCIENCE PROGRAM

Program Name Time PI

GLIMPSE (Galactic Plane) 400h Churchwell

GOODS (Ultra Deep EG Survey) 647h Dickinson

Molecular Clouds & Planetary Disks 400h Evans

SINGS (Nearby Galaxies ISM) 512h Kennicutt

SWIRE (Deep EG Survey) 851h Lonsdale

Planetary Systems 350h Meyer

About 50 % of the allocated observing time will be devoted

to GOODS and SWIRE cosmological surveys

GOODS vs. SWIRE

3.6, 4.5, 5.8, 8.0 (+ 24 ?) m

on 300 arcmin² / 2 fields

3.6, 4.5, 5.8, 8.0,

24, 70, 160 m on

70 deg² / 7 fields

ISO+SIRTF-RELATED ITALIAN CONTRIBUTION/EXPERTISE

Post-Mission Data Reduction

- LARI Method

- ELAIS Shallow Fields at 15 and 90 m

- Deep Fields (LH, HDFs, CAM ELAIS N1 UltraDeep)

Multi-Wavelength Follow-Up

- Radio (ELAIS + LH + Marano)

- NIR (JHK in ELAIS)

- Optical (BVRIZ in ELAIS)

- X-rays (XMM in ELAIS)

- Spectroscopy of IR-selected sub-samples

HERSCHEL SURVEYS

Dedicated panel discussion at Toledo ESA Symposium in Dec. 2000 (Proceedings available at Herschel web page)

Baseline: Herschel will be the ONLY space mission observing large areas at FIR s with ground-like imaging quality in the next 15-20 years. This leads to:

- Detection of the bulk of dust emission from high-z sources

- Resolution of most long- CIRB before confusion sets in

- Bolometric luminosities and thus evolutionary properties

- Active phases of galaxy evolution along cosmic history

PACS and SPIRE are both suited to cover large areas, albeit with a different (PACS's < SPIRE's, both TBD) mapping speed ⇨PACS/SPIRE multi- mapping

The issue of large-area survey (Key vs. Legacy) programs data policy is closely related to Science Management Plan

SURVEY TYPES

Unbiased ''proper'' surveys of ''blank'' fields

&

Deep integrations on pre-selected high-z targets

IN BOTH CASES

PACS + SPIRE coverage is highly desirable

BUT

PACS higher resolution will bring along deeper observations and smaller error boxes but lower mapping speed



PACS survey areas as subsets of SPIRE's?

UNBIASED SURVEYS: OPTIONS

Confusion will severely limit the Depth-Area plane coverage (e.g. no GOODS-like survey)

Built-in simultaneous observations at different s

Low cirrus emission, no known bright sources...

Observing areas not covered at some (many) other wavelength(s) would be of little use

SWIRE fields are most natural and straightforward (thus highly competitive!) choice

PACS + SPIRE SURVEYS

Two SPIRE survey strategies employing ~ 2 months each were identified in Toledo:

A deep survey reaching down to SPIRE confusion limit (~ 10 mJy) over a total area of ~ 50 deg²

A shallower survey (~ 50 mJy) over ~ 500 deg², thus looking for intrinsically rare and bright objects (for e.g. ALMA follow-up)

The prospect of deep surveys by PLANCK with an optimistic sensitivity limit of ~100 mJy at 350 and 550 μm may indicate the second as a lower priority task?

PACS could reach ~ 10 mJy over ~ 20 deg² in ~ 2 months, i.e. with a 4-month two-consortia PACS+SPIRE program!

Smaller (possibly ''national'') programs must be carefully devised and planned in order to be competitive!

At the almost-common wavelength of 200 m

PACS Mapping Speed: 1/12 deg²/day at 5 mJy (5)

1/3 deg²/day at 10 mJy (5)

4/3 deg²/day at 20 mJy (5)

SPIRE Mapping Speed: ~ 1 deg²/day at 10-20 mJ y (5)

10 deg²/day at 50 mJy (5)

Comparable mapping speeds at about the same !

??? Uncertainties in the estimation ???

Mattia's Slides, 25 September 2003

2003-06-10T11:49:32