This forum contains results from the eponymous seminar from WS2012/2013. Each work from the students contains a summary of their results and the presentations attached (partly in German Language).
Post Reply
nils
Posts: 1
Joined: 10.02.2015, 07:53

Analysis of brain activity and response [ENGLISH]

Post by nils » 26.02.2015, 07:31

Stereoscopic 3D Visualization WS2012/13 - Paper Review - Nils Rothe
Analysis of brain activity and response during monoscopic and stereoscopic visualization

Introduction
The final goal, and really, the whole selling point of stereoscopic visualization is, of course, providing a sense of immersion in the viewer by imitating the depth perception the human visual system (HVS) is used to in the real world. Whereas the usage of stereoscopy in virtual reality visualizations has the added bonus of enabling interaction via a third dimension of freedom, and studies on the performance of users in these environments have already been made, the effect of stereoscopic movies on their viewers are much more subtle, as they don't include direct interaction, and also has been less thoroughly studied yet.
In 2012, Calore et al. from the University of Milan, sought to rectify this by presenting their first results of their research into the level of emotional involvement of viewers of stereoscopic movies, examined via electroencephalography (EEG), during the 23rd conference on Stereoscopic Displays and Applications in Burlingame, California.

As they explain in their paper, "Analysis of brain activity and response during monoscopic and stereoscopic visualization", the premise is a daunting one, considering that the concept of emotional involvement is rather difficult to measure, especially since movies, even short ones, provide a multitude of sensory stimuli. While they do admit that the presented findings are only the beginning of ongoing research, their methodology and presented findings nevertheless leave something to be desired, which will be analyzed in this review later on.

Stereoscopic visualization
Just to recap, the basic principle of stereoscopy is the creation of the illusion of depth in the viewer by presenting two bidimensional images corresponding to the different views of a scene as it would be perceived by the HVS in the real world. Common early techniques for achieving this were dichromatic images and corresponding color filters placed over each eye, then leading to rather unwieldy goggle-sets with two separate monitors placed directly in front of the viewers eyes.
Nowadays, technological advances have enabled more comfortable and loss-free viewing of content via active (like shutter glasses), passive (via interference or polarization filtering) or autostereoscopic techniques.

The technique used during the studies is a commercially available one distributed by INFITEC® GmbH, Ulm, Germany (http://www.infitec-global-sales.com/), which uses an interference filter system to split the basic colors making up an image into differing wavelengths that are nevertheless perceived as the same by the human eye. It is further advertised as delivering crisp, homogenous pictures without any artifacts, a very high light efficiency for a passive system, and great comfort by allowing a wide viewing angle. Which appears to be a valid choice for reducing distractions for the test subjects, which might otherwise influence the EEG measurements.

Electroencephalography
Electroencephalography, at its most basic, is the recording of electrical activity along the scalp via electrodes placed on the test subject's head. It measures voltage fluctuations caused by neuronal activity of the brain. It is one of the first brain imaging techniques developed and still widely and routinely used for diagnosing epilepsy and other brain disorders, as well as a valuable tool in neuro- and cognitive science studies.
However, it offers a rather low spatial resolution, can only pick up surface activity of the brain and is prone to a multitude of noise sources in the produced signals, requiring sophisticated data analysis, experience in interpreting the results, and a large number of test subjects to isolate significant information.
The graphic information provided by EEG is then divided into multiple bands, characterized by their frequency and associated with certain activities, e.g. beta waves generally being attributed to active thought processes and problem solving.
However, for ease of setup and in order to reduce subjects stress, Calore et al. had chosen to forgo the rather complex use of a full-scalp electrode array, instead turning to a commercially available Brain-Computer Interface (BCI) called the "Neurosky MindSet".
While it does feature a rather advanced dry sensor and automated artifact filtering, and certainly is a great deal more comfortable to wear than gel-based systems, Neurosky specifically advertises their product as "[meant to] power mass adoption in toys, mobile devices, educational gear and other, popular EEG technology".
Even the research team admits that the usage of only a single electrode limits observations to very general phenomena and would not allow the association of the collected data with any specific area of the brain, therefore only giving a very general idea of overall brain activity, somewhat focused on cognitive processes due to the placement of the electrode on the forehead.

Experimental setup
Equipped with both the aforementioned devices, test subjects were placed inside the Virtual Theater of the University of Milan, "a multi-user VR installation".
For the test, three different short stereoscopic movies had been chosen from those freely available on the internet: "Stereoscopic Skydiving Footage", "Do 27 Anniversary Meeting" and "The Eye 3D (Trailer)", courtesy of http://www.3dtv.at. They were chosen for their relatively short length, lack of dialogue and differing visual content.
Furthermore, background music was cut from all three movies to reduce the influence of audio cues and focus on the reaction to changes between mono- and stereoscopic viewing.
Participants were not asked to perform any specific task, just to avoid moving or talking while simple watching and concentrating on the movies. They were wearing the stereoscopic goggles even during monoscopic sessions, simply to keep the level of user comfort constant for all the experiments.
Two experiments were performed under these circumstances. The first consisted of two sessions, with at least a 24 hour pause in between them, first watching the monoscopic versions, then the stereoscopic versions of the movies.
In order to reduce inter-subject variability due to the aforementioned pause and to better compare EEG signal during mono- and stereoscopic viewing, a second experiment was conducted which consisted of two sessions of watching the skydiving footage, the first half monoscopic, the other stereoscopic, back to back in one session, then after another pause, watching it again with the combination of visualizations switched around.

Results and discussion

In both experiments, sample size was limited by the number of voluntary participants (10 and 9, respectively), and further reduced by technical issues with the BCI, reducing the usable data sets by about half each (to 4 of 10 and 6 of 9, respectively).
For the collected data, average Power Spectral Distribution in alpha, beta and gamma waves had been calculated, and from that, the beta/alpha ratio had been chosen as an index of attention level and emotional involvement from neurophysiological literature, where it is "often used as a feature in software for the automatic classification of emotions based on EEG signals".
Changes in gamma band power had been noted as well and tentatively linked to different narrative rhythms between the movies, but Calore et al. also conceded that the changes might be due to the lack of a clear task for participants to perform and not be significant at all. In any case, gamma band activity had been noted as a subject of study for further research because of its possible significance regarding complex cognitive processes.
The collected data from the first experiment seemed to confirm the initial hypothesis that stereoscopic visualization does indeed increase viewer attention and involvement. However, due to the low amount of usable data and the high variances of the average ratios, the researchers have refrained from jumping to a definitive conclusion.
The second experiment's data is in line with the same trend, although variances again forbid formulating a definitive conclusion. Also, Calore et al. fail to explain the apparent differences in beta/alpha ratio change between the two different sessions, and how they reflect on changes of brain activity regarding the order in which mono- and stereoscopic sequences are viewed.
They do note however, that these differences, as well as the variance values, are mostly due to the data collected from a single participant, who had been present for all four sessions of the whole research project and reported boredom from watching the test movies, suggesting an influence of task repetition on brain activity. Including this participant's specific data sets, combined with the generally low sample size, is admitted to be a leading cause on why the presented data is not more definitive.

Conclusion
In conclusion, the research team has established baseline methods for measuring the impact of stereoscopic visualization on brain activity, have tentatively linked an increase in the beta/alpha ratio of brain wave power distribution to emotional involvement and subject arousal, and have found peculiarities both in the behavior of gamma band activity and on the influence of stimulus saturation.
They close their presentation with future plans of increasing the number of available data sets as well as producing a more robust experimental setup, to "eventually validate the trends noticed in the presented experiments."

Source
Analysis of brain activity and response during monoscopic and stereoscopic visualization
Enrico Calore ; Raffaella Folgieri ; Davide Gadia ; Daniele Marini
Proc. SPIE 8288, Stereoscopic Displays and Applications XXIII, 82880M (February 9, 2012)

Presentation and PDF
Please find the presentation and the PDF of this text with addtional images in the area with limited access:
http://www.cellmicrocosmos.org/Cmforum/ ... f=55&t=775

Post Reply