[math-fun] Structured light (was Single pixel cameras)
Henry Baker <hbaker1@pipeline.com> wrote:
A classical (1930's-1990's) TV camera *raster scans* an image (which hopefully remains still for the duration of the scan) with a single point spot, so one could -- at least in theory -- dispense with all of the lenses, stage lights, etc., and simply have a single-pixel sensor which receives light that was produced by a laser beam producing a single point scanned over the scene whose image is being captured.
I came up with almost the same idea decades ago. My idea was a "poor man's video camera." A black and white TV would be adjusted to display a bright blank raster. A lens would be used to project this raster onto the scene you want to get a video image of. The signal would be picked up by a photocell. I realized that the image would look as if the camera was where the projecting lens was and as if the light source was where the photocell was. You could aim the photocell at the ceiling to get more uniform illumination rather than a spotlight effect. You could have multiple photocells with color filters in front of them to get a color image. You'd need to exclude light from all other sources, as it would tend to wash out the image, reducing the contrast. I later realized that the image quality would be poor due to the slow phosphor on the TV. But I'm sure it would have worked. I wonder if anyone ever did it. I'm curious how you'd scan the laser over the scene. Mechanically? For a color image you'd need at least three lasers, one for each primary color. I've given a lot of thought to what would be possible with what I call structured light, light whose properties (wavelength, polarization, duration, direction, etc.) are carefully controlled, together with detectors that are sensitive to those properties. It might be possible to replace x-ray machines for subjects that aren't totally opaque to visible light. Generate a very brief (picosecond) flash of light on one side of the subject, and watch for just the *first* light to pass through the subject. That would be light that took a direct path rather than being scattered. A variation of that is to take the difference between the brightness from each direction light that's reflected after slightly different periods of time. For instance if someone is buried in an avalanche, it should be possible to look successively deeper through the snow, and notice a difference when the time allowed is just barely long enough for the pulse of light to reach the subject before being reflected back. Since the pulse would be very brief and a very narrow wavelength, background light wouldn't interfere significantly, unless you were looking through something that was almost opaque. Another idea that I had years ago was that it would be possible to reconstruct what's on a video screen if you can see a surface that's illuminated by the screen. Someone eventually did it; see https://dl.acm.org/citation.cfm?id=830537 Then there's the laser microphone, a way of listening to whatever you can illuminate with a laser. And you can read all the data being transmitted over a modem if you have a good view of the modem lights -- or even of a surface that's illuminated by them. And there's a way of seeing around corners using lasers: https://www.youtube.com/watch?v=JWDocXPy-iQ With some effort, it ought to be possible to play a CD or DVD without taking it out of its box, even without taking its box down from the shelf. Or to read closed books on a shelf. Party trick: Consider a set of six wavelengths, red wavelengths R1 and R2 which are indistinguishable to the eye, green wavelengths G1 and G2 which are indistinguishable to the eye, and blue wavelengths B1 and B2 which are indistinguishable to the eye. Consider a pigment or dye which reflects R1, G1, and B1, but not R2, G2, or B2. By switching the LEDs illuminating a room between R1 and R2, between G1 and G2, and between B1 and B2, you can make a surface covered with that pigment or dye any color you like. Wearing what looks like an ordinary cotton t-shirt which keeps changing color will make you the hit of the party. Ultimately, if you can capture and reproduce the instantaneous amplitude of light with a space resolution small compared to the wavelength of light and a time resolution short compared to the frequency of light, it should be possible to re-create any visual experience whatsoever, including a "TV screen" you can use a telescope, spectroscope, or microscope on, or an invisibility cloak. Of course this would require processors millions of times faster than today's. Good luck hiding the waste heat.
Of course, there's nothing special about the raster scanning pattern; indeed, *any* pattern which covers the image will work, so long as the reconstruction laser follows the same pattern. I believe that some of the alternatives to 1920's and 1930's TV systems used other nonraster scanning schemes.
Yes. Mechanical systems work better with smooth curves than with the sawtooth waveform used with electronic TVs.
But wait -- there's more!
There's also nothing special about using a *point* of light! One could illuminate the picture to be scanned with a long sequence of random 2D patterns of light, with the single-pixel sensor converting its *average* taken over the entire scene into a time-varying signal whose next value would be the *average over the entire scene* of the light pattern reflected from the next random pattern, and so on.
For greatest efficiency, the 2D patterns should be Costas arrays. If I recall correctly, an image from a multiple-pinhole camera can be disambiguated only if the pinholes form a Costas array.
At 09:21 PM 3/29/2018, Keith F. Lynch wrote:
With some effort, it ought to be possible to play a CD or DVD without taking it out of its box, even without taking its box down from the shelf. Or to read closed books on a shelf.
Your idea is so good, I believe it's currently being used to read ancient scrolls w/o unrolling them! This is particularly important for the scrolls found in Herculaneum -- right next to Pompeii -- which was incinerated by the ashes & debris from Mt. Vesuvius in 79AD.
At 09:21 PM 3/29/2018, Keith F. Lynch wrote:
Henry Baker <hbaker1@pipeline.com> wrote:
There's also nothing special about using a *point* of light! One could illuminate the picture to be scanned with a long sequence of random 2D patterns of light, with the single-pixel sensor converting its *average* taken over the entire scene into a time-varying signal whose next value would be the *average over the entire scene* of the light pattern reflected from the next random pattern, and so on.
For greatest efficiency, the 2D patterns should be Costas arrays.
If I recall correctly, an image from a multiple-pinhole camera can be disambiguated only if the pinholes form a Costas array.
Actually, simple classical orthogonality works just fine. If you have a rectangular array of pixels, the various arrays created by putting exactly one "1" into an array with all 0's, form a basis for the vector space. So a raster scan simply enumerates the basis vectors in a convenient ordering. But there are lots of other bases, including those generated by the inverse Fourier transform of these raster scan bases. So any set of patterns which are linearly independent can be utilized, hence the interest in random arrays, which are with very high probability independent. You could do even better with Singular Value Decomposition of the image (as an approximation to the pixel array), but that would require a priori knowledge of the image(s) to be scanned. BTW, standard MPEG encoding breaks an image into little 8x8 blocks, and MPEG approximates the DCT of these blocks, so if we were concerned only with an 8x8 image we could approximate it directly (and optically) by quickly running through ~64=8*8 special DCT-type patterns.
Dual photography: https://www.youtube.com/watch?v=D4p4XUZYfp4 https://www.youtube.com/watch?v=mgJlwdTVGL0 On Thu, Mar 29, 2018 at 11:55 PM, Henry Baker <hbaker1@pipeline.com> wrote:
At 09:21 PM 3/29/2018, Keith F. Lynch wrote:
Henry Baker <hbaker1@pipeline.com> wrote:
There's also nothing special about using a *point* of light! One could illuminate the picture to be scanned with a long sequence of random 2D patterns of light, with the single-pixel sensor converting its *average* taken over the entire scene into a time-varying signal whose next value would be the *average over the entire scene* of the light pattern reflected from the next random pattern, and so on.
For greatest efficiency, the 2D patterns should be Costas arrays.
If I recall correctly, an image from a multiple-pinhole camera can be disambiguated only if the pinholes form a Costas array.
Actually, simple classical orthogonality works just fine.
If you have a rectangular array of pixels, the various arrays created by putting exactly one "1" into an array with all 0's, form a basis for the vector space. So a raster scan simply enumerates the basis vectors in a convenient ordering.
But there are lots of other bases, including those generated by the inverse Fourier transform of these raster scan bases.
So any set of patterns which are linearly independent can be utilized, hence the interest in random arrays, which are with very high probability independent.
You could do even better with Singular Value Decomposition of the image (as an approximation to the pixel array), but that would require a priori knowledge of the image(s) to be scanned.
BTW, standard MPEG encoding breaks an image into little 8x8 blocks, and MPEG approximates the DCT of these blocks, so if we were concerned only with an 8x8 image we could approximate it directly (and optically) by quickly running through ~64=8*8 special DCT-type patterns.
_______________________________________________ math-fun mailing list math-fun@mailman.xmission.com https://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun
-- Mike Stay - metaweta@gmail.com http://www.math.ucr.edu/~mike http://reperiendi.wordpress.com
One could utilize the random 2D patterns as a form of encryption: Both the sender and receiver have the same complete set of patterns. In fact, since we're simply *accumulating* the results of all of the patterns for a single *still* image, the *sequence* of patterns *doesn't matter* ! So, in effect, we're computing our image as a *linear combination* of the random 2D patterns, and transmitting only the coefficients of that linear combination. Suppose we have a hidef BW image of 1920x1080x8 bits, then we have 1920*1080 2D patterns, each of 8 bits, or (1920*1080)^2 ~ 4TBytes of shared secret. I'd call this encryption scheme "Plato's Cave", after his discussion of people who live in a cave and try to determine what's going on in the world based solely upon the changing shadows that they see on the walls of the cave. If the outside scene is illuminated by these random 2D light patterns, and if the people inside the cave are also privy to these 2D light patterns, then they should be able to reconstruct the outside scenes from a single pixel camera inside the cave. === Mary Lou Jepsen's company Openwater (www.openwater.cc) is developing an MRI-equivalent device made up of near-infrared displays, sensors and computers that can "see inside" the human body to exquisite resolutions -- i.e., individual neurons. Since infrared light can travel several inches into the body and scatter back out, she can capture a hologram of the scattered image, and then invert it computationally. She claims that Openwater's device will be much cheaper/smaller/ faster than those hospital MRI machines that cost $millions and kill people who walk by them carrying metals that can be attracted to magnets. At 10:55 PM 3/29/2018, Henry Baker wrote:
At 09:21 PM 3/29/2018, Keith F. Lynch wrote:
Henry Baker <hbaker1@pipeline.com> wrote:
There's also nothing special about using a *point* of light! One could illuminate the picture to be scanned with a long sequence of random 2D patterns of light, with the single-pixel sensor converting its *average* taken over the entire scene into a time-varying signal whose next value would be the *average over the entire scene* of the light pattern reflected from the next random pattern, and so on.
For greatest efficiency, the 2D patterns should be Costas arrays.
If I recall correctly, an image from a multiple-pinhole camera can be disambiguated only if the pinholes form a Costas array.
Actually, simple classical orthogonality works just fine.
If you have a rectangular array of pixels, the various arrays created by putting exactly one "1" into an array with all 0's, form a basis for the vector space. So a raster scan simply enumerates the basis vectors in a convenient ordering.
But there are lots of other bases, including those generated by the inverse Fourier transform of these raster scan bases.
So any set of patterns which are linearly independent can be utilized, hence the interest in random arrays, which are with very high probability independent.
You could do even better with Singular Value Decomposition of the image (as an approximation to the pixel array), but that would require a priori knowledge of the image(s) to be scanned.
BTW, standard MPEG encoding breaks an image into little 8x8 blocks, and MPEG approximates the DCT of these blocks, so if we were concerned only with an 8x8 image we could approximate it directly (and optically) by quickly running through ~64=8*8 special DCT-type patterns.
participants (3)
-
Henry Baker -
Keith F. Lynch -
Mike Stay