22 Sensor innovations
Suggestions of all kinds for this book draft are welcome — whether it’s fixing small errors, raising bigger questions, or offering new perspectives. Please share comments through GitHub Issues. To make feedback easier to address, please point to the section you have in mind — by section number or a short snippet of text.
22.1 Sensor innovations overview
Over time, as technology has scaled, more complex electrical circuitry has been placed on the sensor. Modern image sensors frequently include circuitry that performs local processing to increase the dynamic range of the sensor (well recycling), or to reduce the intrinsic noise (correlated double sampling). Some of this processing is adaptive, that is the circuit actions depend on the property of the input image. Consequently, the sensor output can depend upon both the control parameters set by the user and the image content.
22.2 Well recycling?
22.3 Split Pixel and HDR Sensors
Autonomous driving. HDR.
22.4 Foveon and Stacked Color Sensors
Link back to physics.
22.5 Spectral Imaging Sensors
Spectricity. IMEC.
22.6 Event sensors
Event sensor Bo’s talk at SCIEN is good content.
22.7 Charge-Coupled Devices (CCD) in Astronomy
As we saw in the discussion of readout paradigms (Section 17.2), CMOS active-pixel sensors completely displaced CCDs in consumer cameras, smartphones, and industrial machine vision. CMOS sensors run on standard low voltages, consume orders of magnitude less power, and integrate timing and digital conversion on a single chip.
Yet in high-precision scientific imaging—most notably in deep-sky astronomy—the charge-coupled device remains vibrant. A prime modern example is the world’s largest digital camera: the 3.2-gigapixel Legacy Survey of Space and Time Camera (LSSTCam) at the Vera C. Rubin Observatory in Chile.
Why would a 21st-century, multi-hundred-million-dollar astronomical telescope choose a detector technology invented in 1970 over modern CMOS? The answer lies in the fundamental trade-offs between charge transport and in-pixel amplification that we introduced in Section 17.2: photometric linearity, noise stability, and the calibration of Fixed Pattern Noise (FPN).
22.7.1 The Calibration Challenge: Billions of CMOS Amplifiers
In a CMOS sensor, every individual pixel has its own in-pixel amplifier (source follower), and each column has its own readout amplifier and ADC:
- Massive Circuit Diversity: In a 3.2-gigapixel focal plane, a CMOS design would require over three billion separate analog amplifiers and tens of thousands of column ADCs.
- Fixed Pattern Noise and Drift: Microscopic variations in silicon fabrication cause small differences in gain, threshold voltage, and bias offset across these billions of amplifiers. While these variations can be calibrated for consumer photography, deep-sky cosmology demands exquisite photometric precision—measuring subtle variations in brightness across billions of galaxies over a 10-year survey to map dark matter and dark energy. Calibrating billions of independent signal chains, and monitoring their drift over a decade, is an astronomical calibration challenge.
22.7.2 The CCD Advantage: Charge Transport to Few Channels
The CCD architecture avoids this challenge by keeping the pixel passive:
- Bucket-Brigade Charge Transport: Pixels in a CCD have no individual amplifiers. The photoelectrons generated in each pixel remain in potential wells and are physically shifted across the silicon array row-by-row into a serial readout register, finally reaching a dedicated output charge-to-voltage amplifier at the corner.
- Uniformity Through Shared Electronics: Because thousands of pixels share the exact same physical output amplifier and analog signal chain, pixel-to-pixel gain variations are virtually eliminated.
- Fewer Channels to Calibrate: The LSSTCam focal plane uses an array of 189 large, custom-built scientific CCDs (each \(4\text{K} \times 4\text{K}\)). To speed up readout, each CCD is segmented into 16 output segments, yielding a total of 3,024 readout channels for the entire 3.2-gigapixel camera.
Instead of calibrating billions of independent on-chip CMOS amplifiers, the observatory engineers need only calibrate 3,024 high-precision, external 18-bit ADCs and associated video electronics. These channels are housed in a temperature-stabilized cryostat and can be characterized and monitored with extreme precision.
The LSST Camera, designed and assembled at the SLAC National Accelerator Laboratory at Stanford (under the leadership of Stanford faculty including Dr. Steven Kahn and Dr. Aaron Roodman), is the largest digital camera ever constructed for optical astronomy. Its 3.2-gigapixel focal plane is cooled to \(-100^\circ\text{C}\) to suppress dark current to negligible levels.
Over its 10-year Legacy Survey of Space and Time, the Rubin Observatory will image the entire visible southern sky every few nights. The choice of CCD detectors provides the exceptional photometric linearity, high dynamic range, and ultra-stable spatial uniformity required to detect the faint gravitational lensing signals of dark matter.
In short, the CCD provided the unmatched linearity, low noise, and stable readout uniformity that deep-sky science demands. CMOS won the mobile revolution where power, integration, and form factor rule; but where absolute calibration and photometric precision are paramount, Boyle and Smith’s bucket-brigade architecture remains unmatched.
https://diffractionlimited.com/calibrating-cmos-images/#:~:text=CMOS%20APS%20sensors%20are%20quite,switch%20photoelectrons%20onto%20internal%20wires.
https://rubinobservatory.org/gallery/collections/main-gallery/07gi6gchk16918o21l49n0mu3f
https://www6.slac.stanford.edu/lsst#:~:text=The%20U.S.%20Department%20of%20Energy’s,Survey%20Telescope%20at%20the%20observatory.
https://www.energy.gov/science/articles/nsf-doe-vera-c-rubin-observatory-installs-lsst-camera-telescope#:~:text=%E2%80%9CThis%20is%20a%20pivotal%20moment,National%20Accelerator%20Laboratory%20(SLAC).
22.8 TOF
Time of Flight sensor
22.9 SPAD
Single photon avalanche detector