Filling in the Gaps: The Status and Possible Applications of Volumetric Holograms in Historic Landscape Reconstruction

When interpreting historic sites and ruins, a lot of trust is placed in the imagination of the visitor. A docent may describe the rolling fields of a plantation with communities of slave houses dotting the landscape, or the remainder of a Greek temple which has long since been destroyed; but ultimately their description only provides an outline, and they can only hope they have provided enough detail that the visitor can mentally paint in the rest of the picture. Reconstructions can eliminate this subjectivity, but are costly, and run the risk of damaging the historic resources they are trying to aid. Artistic depictions provide detail, but cannot be overlain on the landscape, resulting in visitors having to look back and forth between location and signage, breaking their immersion in the space. But picture, if with the press of a button a structure could appear, the gaps on a temple would be filled in, and a full landscape could be seen in context. That without relying on disparate imaginations a museum could temporarily display a reconstruction, without jeopardizing the existing features above or below ground. And being able to show all of this not to one person at a time with the use of goggles or screens, but to a crowd, who can examine and analyze together. The possibilities for interpretation, collaboration, and understanding could grow exponentially with the use of one developing technology. I am speaking, of course, of holograms.

Misconceptions, Preconceptions, and Definitions

To start, we should define our subject. Far from being a technological marvel, humanity has had “holograms” for nearly eight decades now. While science fiction calls to mind images of Princess Leia being projected by R2-D2 in 1977’s Star Wars, (Lucas 1977) the reality is much more mundane. In 1971, Dennis Gabor won a Nobel Peace prize for his 1947 work utilizing the interference of light waves to print a three-dimensional image on a two dimensional surface,(Nobel Prize 1971) earning him the nickname of “father of holography.”(Beléndez 2015) While we would not look at his depictions with any great awe today, his hologram images birthed the technical definition of “a three-dimensional image formed by interference of light beams from a laser or other coherent light source.”(Oxford Languages) In 1967 Gabor’s work was replicated by two students at the University of Michigan, Emmet Leith and Juris Upatnieks, who were able to produce a three-dimensional image of a model train on a glass plate. (Electical and Computer Engineering) Most readers will recognize the iterative effect of their work on their passports and credit cards in the holographic windows used as a security measure. By all definitions, these are holograms,(Encyclopedia Britannica 1972) but it’s not what this research aims to explore. As such, a more definite description is needed.

The International Hologram Manufacturers Association lamented this definition issue at their annual forum in 2019, commenting that the advent of augmented reality (AR), virtual reality (VR), and mixed reality (MR) have further complicated hologram terminology, with all four terms being used interchangeably by different researchers for increasingly similar technologies. (Deakes 2019) The conference dubbed these as “faux” holograms, which display a three-dimensional image to the viewer, but do not meet the threshold of “a three-dimensional image formed by interference of light beams from a laser or other coherent light source.”(Oxford Languages) Other definitions have expanded on this idea, based on Gabor, Leith, and Upatnieks’ work, adding that the resulting image must be able to be seen “without any special equipment, such as cameras or glasses.”(Byers 2018) While this clarification does eliminate augmented reality, virtual reality, and mixed reality viewers it still covers a huge amount of two-dimensional images, and does not necessarily describe our ideal set out in science fiction.

That image of Carrie Fisher giving her impassioned plea for the fate of the galaxy (Lucas 1977) is most accurately described as a Volumetric Display, or Volumetric Hologram.(Blundell 2011) The qualifier is critical here, as it sets apart the inherent three-dimensional presentation of all holograms, into something with the actual presence of height, width, and depth, and not just the illusory appearance of it. Furthermore, this volumetric display, to match our science fiction ideal, should be able to be projected into free space, rather than onto a screen or an enclosure. As such, throughout this research the term “volumetric hologram” will be utilized, as it appears to be the semantic middle ground between popular understanding and scientific research.

Overcoming the Laws of Light

At present volumetric hologram technology is still science fiction, foiled by real-world physics and the scientific properties of light, however, ingenious solutions are being developed by artists and engineers worldwide.

The greatest obstacle to a truly freestanding volumetric hologram lies in an issue with the most basic principle of how light works. For most wavelengths of light to be seen by the naked eye, they have to bounce off something. Open air provides little for light to bounce off of. This inherent fault has led to the development of the “faux” holograms discussed earlier, either through augmented reality, virtual reality, or through more traditional methods such as projection onto a screen. However, these are merely illusions of three-dimensional space, with no actual volume. As such, three potential solutions have been developed to overcome this fundamental law of light: water, gas, and plasma.

The water and gas solutions are operationally different, but create similar results. In these methods, a fine mist of water or concentration of gas is released into an area where the volumetric hologram is to be displayed, providing an invisible, multidimensional surface for light to bounce off at different points and vectors. French artist Joanie Lemercier has been pioneering the water method, creating interactive projections in mist.(Mashable Deals 2018) However, these are still very much in their experimental phases, and work better as a proof of concept than as actionable technology. Gas has been utilized as a medium primarily in scientific laboratories, but has been successful in producing a small, but fully volumetric, hologram.(Gu 2004) Unfortunately, as one adds variables to their mechanism, they must also manage the natural state of those variables. Gas, by its nature, matches the shape and volume of whatever container it is in. As such, this method is only possible for enclosed spaces, and with such concentrations required for the display that it is unlikely humans would be able to enter them. It’s perfectly functional for small scale volumetric holograms, but an impossible option when discussing large or outdoor displays. Furthermore, in the context of historic sites and ruins, fine mists of water or particulates begin to risk preservation concerns to the exposed historic features, and should be approached with care.

So, how do you project without a screen or particulate to project onto? The plasma method from Japanese firm Aerial Burton offers another solution to this issue, developing a system to project images directly into the air.(Edwards 2014) (Figure 1) No screen, no surface, and no gas or water utilized. This is accomplished via an ultra-precise pulse laser ionizing specific air particles to create small plasma bursts. These bursts only last a nanosecond, but the resulting points of light form the boundaries of a shape, freely floating in midair. While this technology is by no means perfect, it is a consequential first step towards the development of volumetric displays.

Applications, and Where the Museum Comes In

Still then, lies the question of the potential applications of these volumetric holograms. Two-dimensional projection holograms have already been established in museums around the world, almost exclusively in small, indoor exhibits. The 19th century Pepper’s Ghost Effect (Figure 2), popularized by theme park attractions such as Disneyland’s Haunted Mansion, has been increasingly utilized in museums to display animals or people inside of exhibit cases, interacting with collections objects without actually touching them (Figure 3).(HOLOCO 2022) Art collectives such as teamLab in Japan have utilized projection technology to make the viewer feel as though they are flying with birds (teamLab 2017) or dancing with ancient warriors,(teamLab 2018) bringing a level of immersion and interactivity rarely experienced in learning environments.

In a showing of the possibilities of the technology, the Illinois Holocaust Museum has integrated projection holograms as critical to the learning experience; the museum has recorded hundreds of hours of stories of Holocaust survivors as digital holograms, allowing visitors to interact, listen, and ask questions of them as though they were present in the room.(Wolff 2023) Each of the eight survivors interviewed has recorded answers to between 20,000 to 30,000 questions, ranging from everything to what their life was like before the Holocaust to their favorite color. The impact of this on museum visitors has been strong, and the interactions are seen as invaluable to the experience.(Wolff 2023) But each of these examples are limited in what they can present to the visitor within the walls and boundaries of the museum.

What becomes possible with volumetric holograms is the large-scale interpretation of historic landscapes. Landscape interpretation always takes a measure of creative interpretation for museums and historic sites, especially in cases where the historic features are no longer standing. Ruins themselves tell a story, but interpretive narratives are limited when the full scale of a historic scene is absent from view. Descriptions or photographs encourage visitors to use their imaginations to envision long absent vistas or the former grandeur of an estate, but a barren landscape stands antithetical to this view. Reconstructions are costly, run the risk of damaging the remaining ruins or subsurface features of a site, and they cannot be changed after being built, meaning the later discovery of any inaccuracies cannot be addressed. While virtual reality provides some answers to these issues, the virtual reconstruction of the world the viewer enters into inherently separates them from the historical landscape and features they are standing amidst. Augmented reality works to bridge this gap, but still separates the viewer from the subject through goggles or a handheld screen. What volumetric holograms might provide , that these other technologies do not, is the ability to project into space the form of a structure, or missing parts of a ruin, and have that visible in real time as an extension of the landscape around the viewer, rather than replacing it.

So far, we’ve mainly seen the replication of small objects inside of exhibits, as in the examples discussed above, but imagine how much more could be done with a change in scale. With volumetric holograms, ruined structures, buildings, churches, and so on could be displayed at their full form, not just as the remains currently extant. Historic houses, befallen to fire or neglect, could have gaps filled in to give an impression of the peak of their use. Ancient ruins of structures or cities could be visible in their original context, with spatial relationships and construction able to be analyzed and interpreted relative to not only each other but the landscape around it. Further extrapolating, entire landscapes could benefit, visualizing slave structures on historic plantations or fortification locations on battlefields. Locations where archaeology, oral history, or archival records identify that cultural resources lie could be rebuilt without ever impacting the subsurface, or even making a permanent change.

The recently added UNESCO World Heritage Site of the Hopewell Ceremonial Earthworks in Ohio is one such site where the possibilities are immediately apparent. The site, built by the Hopewell Culture between 1600 and 2000 years ago,(NPS Hopewell Ceremonial Earthworks) once featured geometric mounds over 12 feet tall, but now, after hundreds of years of plowing and modern construction, they appear as little more than bumps on the landscape. The National Park Service, which manages the site, has taken to “interpretive mowing”, allowing the tall grasses to define the slight remnants of some mounds.(NPS Hopeton Earthworks) (Figure 4) Visitors appear to appreciate, but find trouble being impressed by, the scope of the work the NPS is attempting to impart.(Kois 2023) Now, imagine the same site with the possibilities of volumetric holograms. Towering mounds of light, in the locations where earth once lay. A drastic visual change to the landscape to show the true scale and scope of the Hopewell Culture’s work, without any danger to the site underneath. This visual impact, pointing the attention of the viewer above their heads and across the mounds on the landscape, does more for interpretation than looking down towards a sign or a two-dimensional artistic rendering ever could.

Challenges to Overcome

Large-scale holograms for landscape interpretation have a variety of inherent drawbacks, even in their conceptual state. To be worthwhile for museums, the ideal technology would involve what the American Alliance of Museums has dubbed “frictionless” technology:(Davis 2020) fully rendered, three-dimensional holograms which a viewer can see without the use of assistive devices, and which they can move around to examine at different angles. The technology should be simple for museum staff to operate and maintain with basic training, and without the help of an outside specialist, ideally with something as simple as pressing a button or running a software program. The machine hardware itself should be built to survive outdoor environments, or have critical components easily removable to protect from weather and theft. For the purposes of this research, this will be the established ideal, so as to limit the infinite possibilities of technological advancement.

As the foundation of this entire concept revolves around manipulation of light, immediately we must discuss what we’re competing with; namely, the sun. Because research on volumetric holograms is still ongoing, their ability to project during daytime is largely speculation. The heated plasma technology from Aerial Burton is shown working in daylight,(Aerial Burton 2014) though the water and gas particulate methods discussed earlier have almost universally been shown in darkened rooms or after sunset. The limitations of the technology are largely going to be based on which system is adopted.

For museums, volumetric holograms which can be displayed in the daylight are ideal, as it allows connection with the largest number of visitors with the smallest inconvenience to current operations. Those that can only be displayed at night are limiting. In many public museums, this is outside of normal opening hours, and would restrict viewing of any reconstructions to special events or selected evenings. This results in a familiar cost-benefit analysis for the museum; is the cost of the technology offset by the public interest and benefit to the museum’s mission? Luckily, there is precedent for exactly this scenario.

The Forum of Augustus and the Forum of Caesar in Rome, Italy both stand as partial ruins, long missing the paint and sculpture they exhibited thousands of years ago. In lieu of reconstruction, the teams at both sites have digitally mapped the face, curve, and flaws in every wall and column, utilizing it as a single giant screen in a technology known as projection mapping.(Viaggio nei fori 2022) The bare stone ruins, weathered and damaged as they stand, have been fully projection mapped to exhibit the rich painted colors, elaborate sculptures, and intricate tiled floors which would have been familiar to a citizen of Ancient Rome. (Figure 5) This is viewed by the audience in real time without the use of assistive technology other than headphones for the docent’s audio tour. The tour, “Viaggio Nei Fori” (Journey through the Fora) was offered nightly throughout the summers of 2015-2022, and, while attendance numbers are not publicly available, the online reviews and seven-year-long run suggest it was popular.(Trip Advisor) While the projection mapping is not quite to the level of a volumetric hologram, the intentions and functionality are similar. And, despite the required night hours, visitors appear to have enjoyed the uniqueness of the display, and many recommended it as a “must see” when visiting Rome.(Trip Advisor) Clearly, the interpretive benefits were worth the drawbacks the nightly show required.

Now, it stands to reason that in light of this success, projection mapping should be considered as the next great frontier for museums. This is already well underway. Many exhibits are already implementing projection mapping as an integral part of their displays, including teamLab in Japan (teamLab) and traveling exhibits such as the Van Gogh Experience. (Van Gogh Exhibition) However, there is a fundamental difference between this and a volumetric hologram. If there are ruins or walls to project onto projection mapping is a perfectly viable interpretive tool, taking what we have historically relegated to the mind’s eye and combining it with analytical archaeology or artistic interpretation for a more complete picture. The light projections utilized have something to bounce off to create an image. But in the absence of such a backdrop, or any backdrop at all, projection mapping falters. Bridging this gap is where volumetric holograms become valuable for large scale interpretation, where the absence of structures impacts the efficacy of the story the museum is trying to relay to visitors.

It is critical however that these volumetric holograms assist in interpretive efforts, not conflict with the existing narrative of a site. The digital models must be as accurate and well researched as any other part of museum interpretation, working in tandem with existing exhibits and tours to supplement the site’s story rather than replace it. Dissonance between the two will only serve to confuse or distract visitors. It is likely that three dimensional models will need to be approached the same way as artistic drawings are now; utilizing a combination of comparable historic sites which are still standing, documentary research, and excavation. In this case, the tools in the artist’s toolbox are the things which change, exchanging pencil and paper for programs like CAD and Blender. The freedom of interpretation and artistic expression do not. From there, integrating the resulting model is familiar ground for the museum.

Once this initial hurdle is crossed, the next primary concern is spatial accuracy. Any structures, landscape features, or architectural details restored with light should ideally line up with their historic counterparts, or else risk the same dissonance in the understanding of the viewer as with an inaccurate model. Again, however, this is not necessarily new ground. Advancements with Global Positioning Systems (GPS), Geographic Information Systems (GIS), and Light Detection and Ranging (LiDAR) have allowed for increasingly precise surveys of features on the ground, which, when combined with traditional research methods like Sanborn Fire Insurance Maps and probate inventories, provide a picture of a historic landscape which was unthinkable only a few decades ago. It is likely that a good volumetric hologram, which is both historically accurate and visually comprehensible, will need to utilize both the broad accuracy measurements these provide with the fine detail work of the dimensional modeling. As such, it should come as a comfort that the technology for these steps is already so well established.

Also alluded to but left undiscussed in this research has been the matter of cost. At present, the speculative technologies discussed in this paper are prototypes and proofs of concepts, with no associated costs publicly known. However, it can be expected that whatever initial technology makes it to the consumer market will almost certainly not be cheap. Though, as with all technology, this will decrease with time. While our stated ideal involves a projection easily run by museum staff, outside contractors may be considered on a temporary basis to mitigate the costs of the machine and computer modeling for what is likely to be a trial run of such events.

The proposed benefits to interpretation and the understanding of visitors may well outweigh the costs and challenges volumetric holograms bring with them. Museums are no stranger to these kinds of calculations, and the emergence of a new technology should excite as much interest as it does wariness. These issues are delicate, but not insurmountable, and the resulting creation has the potential to add an unmatched new tool to assist our storytelling.

Where we go From Here

Technology is never stagnant. People look at what they have and use on a day-to-day basis, and dream of something better. Creative endeavors like Star Wars show us a future which, while fraught with war and struggle, is filled with technology that we can only imagine. And then, we try to make it a reality. We start experimenting, and wondering if we really could have what we saw on the screen: a cell phone, an intelligent robot, or even a hologram. Incremental improvements are made, refined, and released as each moves us one step closer to the machinery we imagine. Volumetric holograms have captured the collective human imagination, and designers, programmers, and artists are racing to catch up. Their utilization in the museum, at least in their ideal form, poses many potential benefits. This is especially true in large scale landscape interpretation which at present is so limited by the risks and costs associated with reconstruction, but is so critical for the interpretation of context of a now ruined structure or estate. Despite their likely costs, the elimination of reliance on the imagination of a visitor may allow for more accurate interpretation and information to be imparted by the museum.

No interpretation method can be completely perfect, but the possibilities which volumetric holograms provide may allow for a new horizon for museums to utilize. In the next few decades, we could see slave quarters projected atop their foundations showing the spread of an estate far past the main house. We could see the engineering feats of indigenous groups like the Hopewell recreated to tower over us. We may see Roman ruins and temples, once lost to the impacts of time, restored to completeness, if only for a moment. Maybe, in the corner, we’ll add a small Carrie Fisher, pleading for the fate of the galaxy, just to remind us how far we’ve come.