Novel Televisual Environments: Immersive Spectatorship and the Future of Stereoscopic 3DTV

: This article focuses on one of the most ground-breaking technological attempts to create a novel immersive media environment for a heightened televisual user experience: 3DTV, a Network of Excellence project that was funded by the European Commission 6th Framework Information Society Technologies Programme. Based on the theoretical framework mainly outlined in the works of Jonathan Crary and Brian Winston, and on empirical data obtained from the author ’ s laboratory visit notes and discussions with 3DTV practitioners, this article explores the claimed novelty of 3DTV through a focus on the history of stereoscopic vision and addresses the inconsistency between the research project ’ s expected and actual results


I n t r o d u c t i o n
Two-dimensional representations of images, as seen by the human eye within the course of everyday life, have been with us almost since the beginning of known human history in the form of drawings, sketches and paintings.With the rise of modernity, which has been "normally considered resolutely ocularcentric," and beginning especially with the scientific progress of the Renaissance, the hegemony of two-dimensional images established itself as the dominant factor of the modern culture, almost forcing us to constantly confront the "ubiquity of vision as the master sense of the modern era." 1 Today, especially with the increase in academic and corporate research on three-dimensional interactive and immersive media environments, we are witnessing new technological developments within the various areas of data visualization and user interaction that promote immersion into media ecologies.It is not uncommon that such initiatives are accompanied by a plethora of unrealistic expectations raised by the popular press, and not only by the most sensationalist tabloid science and technology magazines but sometimes by the most respected and serious newspapers and academic journals as well.These technical innovations thus gain from being critically analysed from the perspective of media archaeology.As an approach that questions linear historical narratives, media archaeology invites us to examine "new media cultures through insights from past new media" and to focus on the "forgotten, the quirky, the non-obvious apparatuses, practices and inventions."2A number of brilliantly conducted media archaeology studies have already clearly demonstrated that we are not the "first generation to wonder at the rapid and extraordinary shifts in the dimension of the world and the human relationships it contains as a result of new forms of communication."3Thus, just as with the ability of an academic study on the 19 th century early telegraph to provide a "fascinating perspective on the challenges, opportunities, and pitfalls of the Internet,"4 a media archaeological exploration of past communication techniques can guide scholars towards a better treatise on today's technologies.
This article focuses on an interesting example of a 'new' communication medium: 3DTV.Envisioned as the one of the most ground-breaking technological attempts to create novel immersive media environments for heightened televisual and cinematic user experiences, and based on the human physiological foundation of stereoscopic vision, 3DTV was an academic Network of Excellence research consortium funded by the European Commission 6th Framework Information Society Technologies Programme.From the beginning of the consortium's research, its attempts were enveloped with an aura of arguments claiming that 3DTV was destined to become television's next revolution.Some of the claims went so far as to argue that this research would change our lives and habits forever because the "development will revolutionise the TV viewing experience because people will feel that they are not just watching the action but are actually in it."5Yet the actual project deliverables by the end of the funding period were much more modest in comparison with the mass-media-fuelled fantasies.
Based on the theoretical framework that was mainly outlined in the works of Jonathan Crary and Brian Winston, the first part of this article explores the history of stereoscopic vision and related technological progress from the perspective of media archaeology.This historical introduction is included in order to better address the 3DTV project's acclaimed revolutionary novelties.Enriched with the empirical data obtained from the author's visit notes to Middle East Technical University's (METU) 3DTV research laboratory and discussions with Turkish 3DTV researchers, the article's second part addresses the discrepancies between the consortium's expected and actual results.

H i s t o r y o f S t e r e o s c o p i c V i s i o n a n d E a r l y 3 D
Human eyes use stereoscopic vision to produce images.Each eye sees differently, but the brain takes the two distinct images and synthesizes the retinal disparity into a single unitary image, at the same time adding depth perception and producing human vision as the end result.Scholarly research exploring how human vision in general and visual perception in particular work has been conducted for centuries: 6 Leonardo da Vinci realized that to truly capture reality in a painting, we should have two paintings, the views seen by each of the artist's eyes.Giovanni Battista della Porta practiced this technique in his stereoscopic drawings at the turn of the 17th century, but it was Euclid, the philosopher and father of classical geometry, who laid the foundations of stereoscopic "art" almost 2000 years earlier when he wrote, "To see in relief is to receive by means of each eye the simultaneous impression of two dissimilar images of the same object." 7s soon as the principles of human vision were established, experiments with different technical devices that were capable of simulating and even reproducing human visual perception were intensified as well because for centuries, the ultimate goal of the viewing experience was imagined as the ability to "create the illusion of a real environment in its absence." 8 1838, Sir Charles Wheatstone invented a mirror device to deliver stereoscopic 3D images, which he called a stereoscope.This was one year before Daguerre announced that he had successfully reproduced scenes by a photographic process.In this way, the stereoscope even preceded photographic development.9Jonathan Crary notes that the stereoscope is the "most significant form of visual imagery in the nineteenth century, with the exception of photographs," and it is usually forgotten "how pervasive was the experience of the stereoscope and how for decades it defined a major mode of experiencing photographically produced images." 11However, because the Wheatstone stereoscope was cumbersome and photography was still in its infancy, the initial optical device was not particularly successful commercially.The success and immense popularity of stereoscopes erupted after 1849, when, originally "urging for the stereoscope to be an enlightenment device for the teaching of geography, architecture and natural history," 12 David Brewster introduced a better elaborated device that in the end, "following refinement by Oliver Wendell Holmes went on to become the ubiquitous Victorian 'parlor stereoscope'." 13The technical and aesthetic refinements that Brewster applied to the optical device encouraged its widespread adoption, in the end ensuring that "no other form of representation in the nineteenth century had so conflated the real with the optical" 14 as the stereoscope.
The stereoscopic three-dimensional counterparts of film and television quickly became a reality after the invention of their two-dimensional cousins.For example, the concept of stereoscopic cinema appeared way back in the early 1900s; early stereoscopic television was proposed as a technical idea during the 1920s; and by the 1950s, three-dimensional movies were already quite popular, and "3D movie theaters spread all over the world with their high resolution format, and gave audiences a highly satisfactory stereoscopic 3D experience." 15All of these stereoscopic visual motion image experiments required sophisticated 3D glasses in order to actually experience the sense of depth while viewing threedimensional movement, and it was this obligation to wear special glasses while viewing 3D films that "has often been considered as a major obstacle in front of wide acceptance of 3DTV technology." 16According to Brian Winston, [t]he familiar red and green coloured glasses, for use with projected still images, date back to the 1850s.These were initially used for movies in 1909, when the first of what has been a number of attempts to market stereoscopic films took place.The earliest real fad for 3-D was in 1922.It flared again in 1936.For the World Fair in 1939, Land's light polarisation principle was used in the glasses allowing for colour 3-D for the first time.There was another fad in 1953/4 when a number of Hollywood features, including Dial M for Murder and Kiss Me Kate, were released. 17In the 1990s, the large-screen IMAX system revived the glasses once again. 18 can easily be seen from this historical overview, the principles of stereoscopic vision have been known and utilized for a very long time; actually, it is very "instructive to note that 3-D counterparts of photography, cinema, and TV are not new: indeed, stereoscopic 3-D versions of these common visual media are almost as old as their 2-D counterparts." 19In this sense, it would not be wrong to claim that the "2D and 3D technologies have been developed in parallel." 20dditionally, it was stereoscopy that laid the foundation for most of these technological developmentsthe heart and the basis of technical imagination which has always been closely related to the fantasy of being able to travel to remote locations and exotic places without the need to leave one's home and to explore these distant grounds in "3D through their stereoscopic recording-an obvious precursor to later virtual reality technologies." 21

T h e D T V C o n s o r t i u m ' s A c a d e m i c R e s e a r c h
As the historical introduction above shows, attempts to create devices and technologies that an observer can no longer "distinguish whether or not what he sees is real or an optical illusion," 22 have been going on for the better part of the last two hundred years.These attempts continued in the 21 st century as well, as the case of 3DTV explored in this article clearly indicates.
3DTV, or, in full, 3DTV: Integrated Three-Dimensional Television: Capture, Transmission, and Display, was a consortium of 19 entities that was formed as a Network of Excellence and was funded by the European Commission 6th Framework Information Society Technologies Programme.The group was composed of academic institutes, laboratories, universities and research companies from Turkey, Germany, Greece, Czech Republic, Bulgaria, Finland and the UK, 23 and led by Bilkent University in Ankara, Turkey.The 48-month project began on 1 September 2004 and ended on 31 August 2008, and it received 6.15 million Euros from the European Commission.On its website, http:// www.3dtv-research.org,the consortium's goals and research mission were described as The primary objective of this project is to align European researchers with diverse experience and activity in distinct, yet complementary, areas so that an effective network for achieving full scale 3D video capabilities integrated seamlessly to a more general information technology base (like internet) is established and kept functional for a long time.The project will create a highly needed synergy among the European partners, at a critical time since 3DTV related research has been significantly accelerating throughout the world, and therefore will boost the European competitiveness. 24Potential application areas and social impact of 3DTV will also be investigated. 25

. 1 D o i n g O r a l H i s t o r y a t t h e L a b
During the spring of 2009, as part of my graduate course-related research about the university laboratories and how science is produced within them, I visited Dr. Aydin Alatan, associate professor in the Department of Electrical and Electronics Engineering of the Middle East Technical University in Ankara, Turkey.Dr. Alatan and his laboratory team were one of the 19 entities that comprised the 3DTV consortium, and they were extensively involved in 3DTV research.Their main aim was to create a computer system that could add the sense of depth to ordinary two-dimensional videos without the need to wear special 3D glasses, to be later used in the "off-line conversion of existing 2-D media into 3-D, which will help to overcome the current 3-D content shortage." 26For this purpose, the team had developed sophisticated software that through the automated determination of camera calibration parameters was able to segment the footage of ordinary 2D video sequences into the background (still imagery) and the foreground (moving objects).By adjusting the distance between these two segments, the program added an artificial sense of depth, thus creating an illusion of 3D.To emphasize the effect even more, these manipulated videos were displayed on a Philips 3D series LCD monitor, then newly introduced to the commercial market.The METU project's research assistants were kind enough to conduct numerous demonstrations with the system they had developed.I have to admit that despite my early scepticism, the result was quite fascinating because they were able to completely achieve their initial aim: when an observer looked steadily at the display's screen, the threedimensional sense of depth almost immediately showed itself, but even a slightest movement or the merest tilt of the head broke the feeling until the observer's eyes readjusted themselves in the new viewing position.Thus, even if during a period of some moments the achieved effect of three-dimensionality looked amazing, after only a few minutes it began to be rather discomforting, creating eyestrain and even nausea at some point.This feeling of uneasiness, experienced while looking at the artificially created 3D effect, goes hand in hand with the observations of Levent Onural and Haldun M. Ozaktas that "within minutes after the onset of viewing, stereoscopy frequently causes eye fatigue and feelings similar to that experienced during motion sickness, caused by a mismatch of perceptory cues received by the brain from different sensory sources." 27TU researchers explained that they were attempting to address the problem, yet even they were not optimistic about the issue.According to their evaluation, the main problem was caused by the lack of dramatic improvements in the hardware section of 3D systems.If the computer processors and graphics cards were accelerated quickly enough to more realistically trick the human vision system, the continuous 3D motion effect could be flawlessly achieved without visual disturbances when the spectator's head moved.
Yet it appears that this problem is at the physiological level given that all the way back in the 1860s, the early stereoscope observers also complained about "weariness of the eye." 28The issue is related to the biological fact that whenever we look at a real-life three-dimensional object, our eyes simultaneously perform two tasks.First, they verge, that is, rotate "slightly inward or outward so that the projection of an image is always in the centre of both retinas."Secondly, they accommodate; that is, they change the "shape of each lens to focus the image on the retinas," 29 with the end result being that without appropriate convergence, we would see double, and without appropriate accommodation, our vision would be blurry.Furthermore, as contemporary researchers themselves observe, although with the "adoption of digital technologies in all aspects of motion picture production, it has become possible to eliminate some of the factors which result in eye fatigue", some "intrinsic causes of fatigue may still remain as long as stereoscopy remains the underlying 3D technology." 30This claim is also supported by a number of perception analysis studies in which some of the findings indicate that "watching films in stereoscope increased threefold the risk of eyestrain, headache or trouble with vision" 31 and that "dizziness, headaches and nausea happen while watching 3-D or IMAX movies" 32all because of the fact that in artificial 3D settings, the "natural linkage between convergence and accommodation is broken." 33

. 2 A n a l y s i n g C o n s u m e r S u r v e y s
All of these factors were also reflected in consumer surveys in which "79 per cent of 3D TV owners surveyed said they regret the purchase, unhappy at the lack of 3D stuff to watch and the need to keep shelling out for more kit," with an additional emphasis that "nearly a third complained of nausea, headaches and dizziness while watching 3D." 34 It is likely because of such low customer satisfaction rates that the 2013 Consumer Electronics Show, for the first time in many years, did not display any new 3D technologies, 35 and the fair's main floor space that was otherwise dedicated to "threedimensional imagery has been decimated, relegating a dubious technology to its proper position as a sideshow rather than a leading cause to upgrade your TV." 36 Thus, in a way, the entire aim of the carefully designed 'trickery' of the human binocular system through an artificially added sense of depth to the cinema screen or the traditional television setapparatuses that by their own nature are two-dimensionalmay have been misguided.This point is raised even by the 3DTV researchers themselves in that they highlight that "stereoscopic 3-D, whether more conventional as in stereoscopic cinema, or more modern as in autostereoscopic monitor viewing, will never yield the ultimate experience."37But if stereoscopy-based 3DTV was never even believed to be the future of three-dimensional TV viewing, which technology was actually intended instead?
The opening page of the 3DTV consortium's website, provided in Figure 2, featured a graphic image created by visual artist Erdem Yu ¨cel that depicts a television viewer leaning over his table-top holographic three-dimensional TV set while watching a football match.Designed in obvious resemblance to the projection of Princess Leia by R2D2 in Star Wars, this picture envisioned the group's real main goal: creating holographic 3D television sets.These future sets were described as involving a "ghost-like, yet high quality optical replica of an object that is visually indistinguishable from the original (except perhaps in size)," where "these moving video images would be floating in space or standing on a Fig. 2 The index page of the 3DTV consortium's official website.
tabletop-like display, and viewers would be able to peek or walk around the images to see them from different angles or maybe even from behind." 38Clearly, this was the actual aim of the 3DTV research network from the beginning given that both in interviews and in numerous publications, Levent Onural and a number of consortium scientists constantly referred to the holographic 3D television set as the "ultimate 3-D viewing" experience, emphasizing that although the "[h] olographic cinema, digital holography, and holographic 3DTV are still in their infancy…there is no doubt that the obstacles will be eliminated…as the underlying optics and electronics techniques mature." 39Interestingly enough, both the image designed by Erdem Yu ¨cel and the assertions of Levent Onural contradicted the warning issued by the 3DTV network on their own website that [i]t is believed that the 3DTV concept is well understood by a large majority of the public.However, it is perceived as a highly futuristic mode of real-time visual communications.There are many examples of it in science-fiction type movies.Therefore, it might not be wrong to assume that the current image of 3DTV in the public perception is a "highly desirable" but "probably impossible to achieve in the foreseeable future" technological wonder. 40d yet, on numerous occasions, the ongoing research within the 3DTV network was promoted on various media platforms with obvious hints (as Figure 3  3DTV consortium's researchers themselves.How, then, can we understand the discrepancy between the intended and the actually conducted research?Where should we place the paradox that the same website that was issuing public warnings to lower the futuristic expectations from the project was at the same time constantly using images that boosted these sci-fi expectations manifold?

. A d d r e s s i n g t h e P a r a d o x
The answer to the questions above might be related to the fact that the history of the stereoscope's initial scientific usage and the later technical modifications that were applied to the deviceleading to the stereoscope's widespread appropriation as the everyday cheap mass entertainment mediumis truly unique.This progress clearly demonstrates the discrepancy between the device's early scholarly intentions and its commercial implementations.
According to Laura Burd Schiavo, what we observe is an obvious paradox in that the same apparatus that in "its original laboratory setting…raised fundamental questions about the status and reliability of vision" was rapidly transformed into a commercial "instrument that delivered perfected, truthful representations." 41Thus, it is important to note that the capitalist mass market and the social psyche of the period refused even the slightest hint of scientific doubt that our eyes can 'lie' and disallowed the uncertainty about the 'truth' that human vision reproduces.Instead, in a brilliant maneuver, the consumer market re-appropriated for its own purposes the same device that had planted those seeds of uncertainty in the first place.This was likely also related to the general imperative of the era, in which to "resemble was long taken to be the peculiarity" 42 of that specific zeitgeist.Philip Hayward and Tana Wollen make an interesting contribution at this point; according to their analysis, [t]he development of audiovisual technologies has been driven not so much by a realist project as by an illusionary one.That is to say, the illusion of the real has had to be made more convincing and the spectacular has had to be made more "realistic".The second-hand has had to become first-hand, the vicarious has had to be made vivid. 43is assertion underscores the clear discrepancy between the fact that whereas Wheatstone's initial stereoscope "had raised questions about the relationships between objects and viewers, and about the truth of appearances," the subsequent commercial parlour stereoscope of David Brewster, which was "incessantly defined within the world of photography and popular amusement…stabilized and solidified those very relationships." 44Thus, both marketers and consumers simply chose to completely ignore the origins of the device and instead embraced the gadget as an instrument of turning the vicarious into the vivid.In her intriguing essay on the striking parallels between medieval visual culture and cinematic special effects, Alison Griffiths makes an interesting contribution to this discussionaccording to her claim, it really does not matter if "it's a representation of God in his kingdom surrounded by angels or a meteor spinning headlong toward earth, there is an implied invitation to believe in this representation, to accept its conditions of possibility." 45That is, consumers of the visual never demanded a truly real image; the only thing they asked for was a believable enough representation.This is precisely the point at which holographic 3D radically differs from the millennialong traditions of two-dimensional representation.As described above, future holographic 3DTV sets are imagined as table-top devices that will project exact replicas of recorded video footage, and the viewer will be able to walk freely around the taped scene and look at the stage from different angles.But how then will such an (almost) primitive experience be different from observing an ordinary statue, or from watching a theatre play, especially street performances, where there are no specified seating places and one can freely walk in circles around the performers?How will viewers be able to retain their belief in the possibilities of watched representation?In this sense, Alison Griffiths' claim immediately evokes the question raised by Brian Winston two decades ago, when he wondered if the holographic techniques would be the "ultimate redundancy, an elaborate system for recreating the experience of the theatre and concert hall, thereby doing nothing more than mechanically returning us to the pre-electric experience of performance?" 46

C o n c l u d i n g R e m a r k s
Were any of the factors discussed above influential in discouraging scientists from conducting research on 3D holography?Or was it just the cautiousnessnot even mentioning holography 3D in bureaucratic documents and dossiersin order to not lose competitive research funding?Was it a mistaken scientific judgment about the maturity of televisual technologies in general, and the perceptual and physiological demands of various stereoscopic 3DTV devices in particular?Was it related to a misjudgement of consumer demand and program material for future 3DTV broadcasts, or was it just a symptom of the unrealistic expectations about the technological novelties of the researchers themselves?The real reason for the discrepancy between the research project's expected and actual results will likely never be known. 47us, we never might know if it was the knowledge of the stereoscope's development history or the uncertainties about the commercial outcomes and usages of 3D holography that made 3DTV scientists abstain from research on the subject they were clearly passionate about given that according to the Community Research and Development Information Service (CORDIS) 48 page of the 3DTV network, when the EC project funding ended in 2008, the results delivered by the consortium were as follows: Established its governance structure and bodies; held many meetings; aligned its researchers' interests and partners' research facilities; completed many researcher exchanges; completed extensive surveys and technical reports on all aspects of 3DTV; published more than 160 papers related to its technical scope; prepared a graduate 3DTV course; conducted research on all technical aspects of 3DTV; organised the first 3DTV-CON-a conference focusing on all aspects of 3DTV; prepared an end-to-end multi-view 3DTV video streamer demo. 49re detailed information is provided in the network's final annual report (2008), in which it is stated that many smaller and more focused projects were incubated from the 3DTV network, resulting in the "establishment of a 3D Media Cluster consisting of EC funded 3D-related FP6 and FP7 projects on 16 April 2008" and new research initiatives such as "3DPHONE, MOBILE3DTV, Real 3D, Helium 3D, MUTED, 3D4YOU, Victory, 2020 3D Media and i3DPost." 50ne of the final deliverables of the 3DTV network, nor the follow-up projects that the consortium incubated, touch even slightly upon the subject of holography 3D, despite the fact that on numerous occasions, researchers clearly hinted that a 3D holography television would be the only viable option to overcome the eye fatigue and nausea that were frequently encountered in stereoscopy-based 3D devices, the main reasons for the devices' commercial failure in the first place.
46 Brian Winston, Media Technology and Society: A History From the Telegraph to the Internet, Routledge, 1998, p. 341.
47 Even in my own case, my initial laboratory visits at METU were never intended to be based on participatory observation, or even be focused specifically on 3DTV.It is these deficiencies that prevent this study from being a 'true' example of media archaeological exploration of stereoscopic 3DTV.This said, I strongly believe that this study can still be placed under the rubric of media archaeology, that is with a "spirit of thinking the new and the old in parallel lines" excavating the "past in order to understand the present and the future"

22
Levent Onural, Thomas Sikora, Jo ¨rn Ostermann, Aljoscha Smolic, M. Reha Civanlar and John Watson, 'An Assessment of 3DTV Technologies,' National Association of Broadcasters (NAB) Broadcast Engineering Conference Proceedings 2006, Las Vegas, USA, April 2006, 456.23 The complete member list of 3DTV research group was as following: Bilkent University (Turkey); Bremer Institut fu ¨r angewandte Strahltechnik GmbH (Germany); Central Laboratory of Optical Storage and Processing of Information to Bulgarian Academy of Sciences (Bulgaria); De Montfort University (United Kingdom); Fraunhofer Gesellschaft zur Fo ¨rderung der angewandten Forschung e.V. (Germany); FogScreen Oy (Finland); Technische Universitaet Ilmenau (Germany); Tampere University of Technology (Finland); Centre for Research and Technology Hellas (Greece); Koc ¸University (Turkey); Middle East Technical University (Turkey); Momentum Bilgisayar Yazilim, Danişmanlik, Ticaret A.Ş. (Turkey); Max-Planck-Institut fu ¨r Informatik (Germany); University of West Bohemia in Plzen (Czech Republic); Universitaet Hannover (Germany); Technische Universitaet Berlin (Germany); Eberhard-Karls-Universitaet Tu ¨bingen (Germany); University of Aberdeen (United Kingdom); Yoğurt Bilgisayar Teknolojileri Tic. Ltd.Şti.(Turkey).24 The emphasis on European competitiveness was related with the fact that besides the European 3D-oriented research initiatives a lot of state-ofthe-art experiments on various three-dimensional media applications were conducted by different universities, companies and research centers all around the world.Some notable examples here are the academic-corporate research collaboration programs implemented by Mitsubishi Electric Research Laboratories (MERL) and Korea Electronics and Telecommunications Research Institute (ETRI).25 http://www.3dtv-research.org/introduction.php.26 Evren Imre, Sebastian Knorr, Burak O ¨zkalayci, Ugur Topay, A. Aydin Alatan and Thomas Sikora, 'Towards 3-D scene reconstruction from broadcast video,' Signal Processing: Image Communication, 22, 2, February 2007, 109.
Fig. 3 Representational image of holographic 3DTV, as presented by the 3DTV network researchers in a Futuris/Euronews video.Watch the full video here.
which is exactly what this article tried to do.Jussi Parikka, What is Media Archaeology?, Polity Press, 2012, p. 2.48 CORDIS is the European Commission's primary public repository and portal to disseminate information on all EU-funded research projects and their results in the broadest sense.