20 Sensor innovations
The book is still taking shape, and your feedback is an important part of the process. Suggestions of all kinds are welcome—whether it’s fixing small errors, raising bigger questions, or offering new perspectives. I’ll do my best to respond, but please keep in mind that the text will continue to change significantly over the next two years.
You can share comments through GitHub Issues.
Feel free to open a new issue or join an existing discussion. To make feedback easier to address, please point to the section you have in mind—by section number or a short snippet of text. Adding a label characterizing your issue would also be helpful.
Last updated: October 15, 2025
20.1 Sensor innovations overview
Over time, as technology has scaled, more complex electrical circuitry has been placed on the sensor. Modern image sensors frequently include circuitry that performs local processing to increase the dynamic range of the sensor (well recycling), or to reduce the intrinsic noise (correlated double sampling). Some of this processing is adaptive, that is the circuit actions depend on the property of the input image. Consequently, the sensor output can depend upon both the control parameters set by the user and the image content.
20.2 Global Shutter Technology
Instead of using a floating diffusion as a memory element, Aptina has utilized a surface-pinned storage node in the pixel to address dark current challenges. Available in its newest global shutter sensor, the MT9M031, the storage node also enables using a true correlated double sampling technique to reduce readout noise to four electrons, resulting in excellent low-light performance. The combination of the effective use of an anti-reflective metal light shield in close proximity to the memory node and careful doping and potential profile design results in a high GSE.
Illustrate rolling shuttter artifact.
Then say something about global shutter technology.
Global Shutter Pixel Technologies and CMOS Image Sensors – A Powerful Combination – (Aptina white paper)
Well recycling?
20.3 Light Field Cameras
Refocus. Depth estimation
20.4 Split Pixel and HDR Sensors
Autonomous driving. HDR.
20.5 Foveon and Stacked Color Sensors
Link back to physics.
20.6 Spectral Imaging Sensors
Spectricity. IMEC.
20.7 Event sensors
Event sensor Bo’s talk at SCIEN is good content.
20.8 CCD
Charge coupled device
Tell the Vera Rubin choice of CCD.
Yes, your belief regarding the rationale is well-supported by the history and technical documentation of the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) Camera (LSSTCam).
The choice of CCD (Charge-Coupled Device) over CMOS (Complementary Metal-Oxide-Semiconductor) imagers was largely driven by the absolute necessity for high photometric precision and exquisite linearity across the camera’s massive 3.2-gigapixel focal plane.
Here is the rationale described in technical documents from the time of the camera’s design:
20.8.1 1. The Calibration Challenge: Fixed Pattern Noise (FPN)
The core issue was exactly what you described: the sheer number of independent signal chains in a CMOS array makes calibration exponentially more difficult compared to a CCD.
- CMOS Architecture: Each pixel (or small group of pixels/columns) on a CMOS sensor typically has its own internal amplifier and often its own Analog-to-Digital Converter (ADC). For a \(4\text{K} \times 4\text{K}\) sensor, this results in potentially millions of individual readout circuits.
- The Calibration Problem: Variations in the manufacturing process mean that the gain, offset (bias), and noise characteristics of these millions of on-chip amplifiers and ADCs are not perfectly uniform. This non-uniformity manifests as Fixed Pattern Noise (FPN) in the image. To achieve the LSST’s demanding photometric accuracy (required to measure dark energy and dark matter effects), every one of these independent signal chains would need to be precisely mapped, calibrated, and monitored for drift over the 10-year survey. At the time of the LSSTCam’s design (before the widespread availability of modern scientific CMOS sensors), the calibration stability and uniformity were considered an unacceptable risk for the scale of the project.
20.8.2 2. The CCD Advantage: Fewer, Higher-Precision Channels
Describe CCD technology issues. Then use this background information for a modern usage. They have lots of power/volts. They are fanatical about calibration.
The chosen CCD solution, while still massive, dramatically reduced the number of high-precision electronic chains that needed calibration.
CCD Architecture: The LSSTCam focal plane uses 189 large CCDs. While each CCD is segmented for fast readout, the signal from thousands of pixels is serialized and routed to a dedicated output amplifier and off-chip ADC.
The Calibration Solution: Each of the 189 CCDs is divided into segments, resulting in a total of 3,024 readout channels (video channels) for the entire camera. Instead of millions of independent ADCs, the system has 3,024 high-precision (18-bit) external ADCs and associated electronics. Calibrating the gain and offset of 3,024 highly stable, purpose-built channels is far more manageable and reliable than calibrating millions of on-chip circuits in a CMOS array.
In short, the CCD provided the high linearity, low noise, and stable readout uniformity that the LSST’s deep-sky science requires, especially at the scale needed for the world’s largest digital camera.
https://diffractionlimited.com/calibrating-cmos-images/#:~:text=CMOS%20APS%20sensors%20are%20quite,switch%20photoelectrons%20onto%20internal%20wires.
https://rubinobservatory.org/gallery/collections/main-gallery/07gi6gchk16918o21l49n0mu3f
https://www6.slac.stanford.edu/lsst#:~:text=The%20U.S.%20Department%20of%20Energy’s,Survey%20Telescope%20at%20the%20observatory.
https://www.energy.gov/science/articles/nsf-doe-vera-c-rubin-observatory-installs-lsst-camera-telescope#:~:text=%E2%80%9CThis%20is%20a%20pivotal%20moment,National%20Accelerator%20Laboratory%20(SLAC).
20.9 TOF
Time of Flight sensor
20.10 SPAD
Single photon avalanche detector