by Anthony Mack
The idea of virtual windows is not new. They have been built and sold by many companies for many years. My wife gave one to her sister and she loves it. They all serve the same purpose, to give the illusion of a window to the outside world on an inside wall.
Windows make a room. They are usually the most important part of any interior space. Physical windows are so important, they often dictate the entire shape of the building. Most hotels are about 60 to 80 feet thick so that every room has a window.
Windows have a psychological effect on people. A window that looks onto a pretty field of flowers with blue skies can have an uplifting, emotional effect. Seeing nature has a calming effect, and an ocean view is simply impressive. A lack of windows has an inverse effect. Some people feel claustrophobic in small, windowless rooms and offices. A closed in room can feel like a trap or even a prison. This is why humankind has been hanging tapestries, paintings and other artwork on interior walls since the dawn of man.
If windows are so important, why don’t all rooms have windows? They can’t. There are many reasons. For one, it is physically impossible for the small rooms in a large building to have windows. It can also save on AC costs and increases security. Most large stores (department stores, grocery stores, malls, etc), warehouses and factories do not have outside windows for multiple unavoidable reasons.
The market for a convincing interior window is potentially huge. It needs to be done right though. Here are some examples of existing attempts to improve interior space with virtual windows:
“The Virtual Window Project” by Hoady 2004
(www.hoagy.org/virtualwindow/index.html)
“Winscape” by Rational Craft
(rationalcraft.com/Winscape.html)
SkyV by Sky Factory
(http://www.theskyfactory.com/products/SkyV/)
TESS USA, Inc.
“eScape” by Sky Factory’s
(http://www.theskyfactory.com/products/eScape/)
Every virtual or artificial window/skylight I have found on the net has at least one of the following problems:
· The window frames do not look realistic
· The images are not animated (clouds, trees and birds do not move)
· The scene does not change over time (day/night, Summer/Winter)
· There are no curtains, blinds or drapes
· Insufficient light (a window should light up a room)
· Insufficient pixel resolution
· High power consumption
· The perspective does not change as the observer moves around the room
Any one of the above defects can completely kill the illusion. If a window does not look real, it will not have the positive effect that windows can have on people.
The idea is to solve the problem all current virtual windows have; they don’t look real. I feel I can design a virtual window/skylight that will fool non-observant people. A patient walking down a hospital hallway, worried about his problems, should not realize the windows are synthetic.
Building nice casements, providing spectacular scenes, using large bright LED displays and offering a plethora of options all go without saying. The hardest part of this illusion is solving the perspective problem.
Look out any window and move your head up, down, right and left. The image you see at the glass “pans” with you. Walk towards and away from the window, and the image “zooms”, showing you more or less of the view. Panning and zooming is all it has to do to pull off the illusion. No rotation or other image changes are needed (to start with). This is because our outside views will not have anything close enough to require a perspective change.
If the image does not pan and zoom to the observer, your mind will tell you the image is at the glass, like a picture, not far away.
Here is an example of how the image changes from two different points of view.
As your point of view moves, so does the image in the window
This effect instantly tells the brain that the mountains are far away, giving the illusion of distance. Even if the observer knows the window is fake, their subconscious is still fooled into thinking there is space around them.
The best example of what this illusion can be found on YouTube. If you haven't already, watch this video of Rational Craft’s Winscape window:
In the video you can see how the window pans and zooms with the observer. However, their design is fundamentally flawed. The observer has to carry a transmitter that tells the computer where they are in the room. A commercial or public building could never require this of the people that walk by. This brings us to the crux of the idea.
Observer Location System
The Observer Location System (OLS) is the heart of these virtual windows. It consists of four small cameras in the four corners of the window. Like the camera in your cell phone, these are tiny and cheap. The video from these cameras is fed into face detection software. Thirty times a second, the software finds any and all open eyes. In normal operation, two or more cameras identify a single observer’s eye(s). Using simple triangulation, the computer can easily calculate the 3D position of the observer. When the position moves, the pictures are panned and zoomed to match.
If there are multiple observers, the illusion may disappear. If the observers are moving together the illusion can be maintained, however if they are not, all panning and zooming must stop. If the display tracks for one observer, the other observer would get the feeling the room is moving, which is unacceptable. It is much better to become a flat TV screen when there are multiple observers.
There are some hardware and software tricks that can lessen the problem, but the fact remains, the view flattens out when multiple observers are moving relative to each other.
The term “observer” is not the same as “person”. An observer is a person who is facing the window and has his eyes open. People can see 180º in front of them, but not behind. On average, half the people in the room are not observers, so their points of view do not matter. Also, in many situations (such as a hallway) the observers all move in a group. As long as the distances between the observers do not change much, the system can pan and zoom for the whole group.
The reason we have not seen this kind of product before is because the underlying technology has not been available in the past.
A truly realistic virtual window requirements:
· Flat Screen Technology (a wall hanging can not be done with a CRT)
· High Contrast. Extremely bright highlights and very dark shadows
· High resolution. Better than the eye when viewed at normal distances
· Low power consumption. Plasma runs too hot and is too wasteful
· Large (54" or bigger). People do not like small windows
· Inexpensive
· Software Technology
· Very good facial detection software
· High resolution video standards
· Real-time cropping and scaling routines
· Computer Hardware Technology
· Fast.
a. Must find faces at 120 frames per second
b. While doing 120 or 240 hi-res image crops & scales per second
c. Probably needs a quad-core with multiple video cards
· 4 gigs to hold the whole video loop in memory
· Sold state disk drive (does not wear out and is quiet)
· Inexpensive
All these requirements are available now, but only recently. The dynamic LED backlit LCD displays are the first technology that achieves all the requirements. They can light up a room without getting hot or using much power. The same display can also show a dark nighttime scene with subtle differences in the very dark areas. They are also getting big enough to be a whole window. Samsung now has a 65", 1920x1080 LCD with LED backlight. Organic LED displays are even better but are not currently available in large sizes.
This product should take a few engineers, and a few non-technical people, around 8 months to design and implement. This is only a guess, but there is nothing fundamentally hard to solve.
As for the carpentry
work, a normal window installer can be easily shown how to install any of our
products.
Approximate schedule:
Sales projections are the hardest part to predict. This is a high-end product for corporations around the world. It has a huge target market and lot of per unit profit. There is a possibility of creating a $50m business if it turns out to be a smash success. At this time I have no market survey or market research information, but I can guess at a few numbers:
· Average per window cost for a mid-range unit should be about $5,000
· A similar product (SkyV by Sky Factory) is selling for about $10,000
· Sky Factory is a $3.9 million, 34-employee company in Fairfield, Iowa
According to this article Bill Witherspoon, who couldn’t afford braces for his children, built up Sky Factory from nothing to 34-employees in 8 years. I figure, with good resources, this one should go much faster.
The first product is a nicely framed large, hi-res (1920x1080 or more) LED backlit LCD flat screen. See software section for details on how user photos are displayed. Hardware options include 3 sizes (L, XL and XXL), hundreds of frame types, photocell and/or IR detector.
There is no intention of competing with the small digital frame market you see today in stores. It is not clear, but L might be 40", XL=54" and XXL=65". Built to the highest possible standards, the digital picture frames are intended to be hung on the wall by the user.
The computer that drives the picture frame can be anywhere in the build, such as a drop ceiling, attic, basement, under a desk, in a closet or anywhere within 50 feet of the display.
Suspended virtual windows do no damage to the wall they hang on. Except for the small hole for the wire and the screws it hangs on, the wall is unchanged. These are similar to digital picture frames with three main differences:
1. The frame looks like a window
2. Optional curtains, drapes and blinds
3. Observer Location System (OLS)
This product simulates a window to the outside world. It can display art, but usually doesn’t. It gives the feeling of openness to a room. To do that, it tracks the observer’s position and repaints the scene accordingly.
Embedded virtual windows are installed into an interior wall. Most walls are at least 4" thick and this space can be used to enhance the illusion.
Real window frames and glass are used in front of the displays. In the example above, six displays show a real-time image produced from a hi-res camera mounted outside. This allows people in basements and other windowless parts of the building to see what is going on outside.
Note: If the wall is load bearing, a header will have to be incorporated into the framing, just like a real window. Any in-wall pipes or electric wires will have to be moved. After installation, the user must be careful not to hammer or screw any fasteners into the wall opposite the window.
With “new wall” installations the interior wall can be made thicker. This allows for the displays to be farther from the window. With only a few inches separation, the perspective illusion is partly done automatically. That is, the image will appear to move as you the observer move. Not as much as with a real view, but a little helps a lot. This allows multiple observers to get the feeling of openness even while they are moving in different directions. The more the setback, the better the illusion. However, the gap is limited by the maximum size of the display. For every inch the display is set back, it has to be 4 inches wider and 4 inches taller. This puts a practical limit of 2" to 3" on the setback. If someday, displays can be bent into a curve, a larger setback could be employed.
Another advantage to building a new wall is that it can be any shape. A normal straight wall can be transformed into two back-to-back Bay Windows by building a zigzagging the wall.
Virtual skylights work the same way as windows with the four cameras looking down instead of out. Skylights can be linked with any virtual windows in the room to give one coherent outdoor scene. Like the windows, they should be recessed into the space above, such as into the void above a drop ceiling, to help with the illusion of distance. This is important because multiple observers are more likely.
Since LEDs are so bright and efficient, the skylight can replace the existing lighting providing a more natural and varying light while saving energy.
Software
So far this document has concentrated on the hardware aspects of the different products, however, the really exciting part is the software. The possibilities are endless. The hardware described above can support many different types of software. The displaying of photos and even the basic scene viewer is just the beginning. There are many other possibilities including (but not limited to): fish tanks, remote monitoring, video conferencing, a security system that keeps pictures of people’s faces, HDTV viewer, etc. Even odd things like an eBook reader, web browser or computer monitor are possible.
As exciting as future software ideas are, this section will mostly cover the software that the first version will support; the “Photo Viewer”, the “Scene Viewer” and the web interface that controls them.
The software runs on a small, but fast processor that comes with the picture frame, window or skylight. It will probably be a one to four core, 4GB PC running Linux. It will most likely have multiple video cards and a solid-state disk drive. It does not have to have Internet connectivity (Ethernet or WiFi), but it is strongly recommended. Without connectivity, the user cannot change the scene and options.
Photo Viewer
All products will come with the “photo viewer” software. The digital picture frame supports nothing else. It allows the user to display any picture or photograph they want on the display. Using a browser, the user uploads pictures to the picture frame/window/skylight directly or via a dedicated server. After selecting some options, the display does the rest. It fades from one picture to the next according to the users choices. Possible choices might be:
· Transition Type – fade, cut, pan, wipe, etc.
· Transition Speed – Slow, medium, fast or an amount of time
· Presentation Duration – Long, normal, quick or an amount of time
· Dim with Ambient Light – (Photocell option required)
· Power Saver – Turn off when no one is present. (IR detector required)
· Professional Art – Include photos and painting from a library.
· Remote Access – Allow family and friends to upload pictures.
The default choices should be very slow fades so as not to draw anyone’s attention to the display. If an inferred detector or camera is available, change the picture when there are no observers present. Overall, the photo viewer software is fairly straightforward and does not require much description.
Scene Viewer
All windows and skylights come with the “scene viewer” software, but digital picture frames do not. The scene viewer is what distinguishes this product from everyone else’s. As far as I know, it is unique and not being done by any company today. That being said, I fear it is obvious and it’s only a matter of time before another company will do the same. Because it is obvious, I feel it is not patentable, but I strongly recommend a patent lawyer first do a patent search and if none is found, try to patent this. It will probably lead to nothing, but it would be good insurance.
The end effect of the scene viewer is best demonstrated by Winscape’s video (rationalcraft.com/Winscape.html). If you have not seen it yet, you should watch it now. The images on the displays are cropped and scaled according to the observer’s location in the room (see OLS below). A “scene” is a very high-resolution video (4096x4096 or more) at 30 frames per second digital video about 3 minutes long. The duration is limited by the available memory. Thirty times a second, the scene viewer software calculates which parts of the frame are needed by each display. It sends only the needed data to each video card which then scales and displays the image. Winscape uses OpenGL to do this, but we might write it ourselves for performance reasons. Calculating the view and displaying of the images should be relatively simple compared to finding the observer’s location.
Observer Location System (OLS)
The OLS is responsible for finding a 3D location, thirty times a second, to be used by the rest of the scene viewer. It does this by analyzing frames from multiple video cameras placed at known locations compared to the display(s). Using existing face detection software, the observer’s eye or eyes are found in each 2D image from two or more cameras. Using facial features (like the distance between the eyes) our software will try to find matches. That is, the same face from two different angles. Existing facial recognition software may be used to accomplish this. Once a match is found, it is a simple calculation to determine the 3D position of each eye in the room.
Consider the room layout to the right:
In this example, the same face and eye are found in two pictures taken at the same time from two fixed locations. The pixel in the center of the eye must lie along a line extended out from the camera in a known direction. The intersection of these two lines tells the rest of the system precisely where the eye is located.
User Interface
The user will interact with their virtual window through a web page. The computer will have an HTTP server running on it that the user can connect to. Also, I feel it would be useful if the user could also connect to a dedicated server. This would allow anyone to be able to upload pictures or change scenes, not just the people on the same LAN. The web interface will be simple and easy to use.
Conclusion
I feel this product line is doable by a few people, yet has the potential of making millions of dollars a year. There is no part of this project that I could not do personally including putting the hardware together, shooting the scenic videos, designing and building the physical windows and doing installations. I have no expertise in marketing and sales, but could even give that a try. However, to succeed, it would need a strong commitment and tons of enthusiasm.