Synesthesia
What is Synesthesia, and name some types:
Extreme form of cross-modal association where stimulus in one modality is experienced concurrently in another modality.
Grapheme-Color
Number-Form
Sound-Touch
Sensory deprivation can induce hallucinations because
Our perception is giving us no input, so our brain tries to make up for the lack of information/sensation.
In multimodal perception, vision typically wins during ambiguity. How has this factor driven the form VR has taken so far?
VR, typically and commercially, is available as a Head-Mounted Visual Display with some small headphones added on. It focuses on visual immersion, as that alone can provide strong immersion.
Predictive ______ is to Feedforward as Predictive _____ is to Feedback.
if we cut back on what needs to be coded, it takes less processing power to perceive. we save storage and power by only coding deviations from normal.
Barratt et al. (2017)
1) To establish multisensory factors that contribute to ASMR
2) Surveys
3) "Lower-pitched, complex sounds were found to be especially effective triggers, as were slow-paced, detail-focused videos. Conversely, background music inhibited the sensation for many respondents."
Synesthesia is reportedly very rare, what are some phenomena described in class that argue for relatively common forms of Synesthesia?
Misophonia - possibly sound & touch synesthesia
ASMR - possibly sound & touch synesthesia
Trypophobia - possibly visual texture & touch synesthesia
Describe three theories of why psychedelics induce hallucinations.
Psychotomimetic: psychedelics induce psychotic state
Huxley: reduce attentional filter
newer findings: facilitate communication between separate areas of the brain
Explain a way to induce an out-of-body experience with VR. Generally or specifically.
1. Wear a HMD with the POV of someone standing behind you, and have them poke your back with a stick. Perception is as if you are standing in front of yourself.
2. In the HMD, have the visual information of your body travel away from the proprioceptive and tactile information your actual body is providing you. Example: The study of the people having balls bouncing off of them while the vision POV rises.
What is the role of play in developing predictive models?
Play allows for refining predictive models in safe and consequence-free environments. Play allows for many repeated opportunities to resolve predictive error and update models.
Instead of looking at just the p value, you consider the likelihood of this finding given all other scientific or worldly knowledge you have.
Bayesian Statistics
Pink-Hashkes (2017)
Based on the predictive coding account model, they make the claim that when on psychedelics these predictions become more decomposed, leading the brain to make overly detailed predictions, which leads to increased prediction error. (Entropic brain hypothesis of predictive coding)
Describe the 3 "rules" of Synesthesia:
1. Associations must be fixed over time
2. Associations must be automatic
3. Associations must be stable and specific
Psychedelics increase the detail and granularity of predictions.
More detail = more predictions will be wrong = hallucinations.
More granularity = attending to everything = everything is interesting.
Describe the Proteus Effect and how it is relevant to VR.
The Proteus effect describes the change in behavior and cognition due to embodying virtual avatars.
VR allows you to be fully immersed in the game and character, arguably giving a stronger Proteus effect.
You're driving along a freeway that you have driven on many times. Imagine the 5 or the 1. You have been driving for a few hours along the freeway, staring out at a consistent scene. Suddenly, a deer pops out in front of your car, you see it, and you must act.
Prior to seeing the deer, would you be experiencing high or low predictive error? Once you see the deer, do you have high or low predictive error? Is the experienced predictive error precise or imprecise?
Prior to seeing the deer, you would be experiencing low predictive error.
Once you see the deer, you will experience high predictive error.
The experienced predictive error would be precise.
Bootstrapping Prediction (broadly)
going from basically knowing nothing about the world to the rich full perception of the world we have now. via: innateness, training, and self-organization
Percie du Sert et al. (2017)
Treatment of auditory verbal hallucinations (AVH) in patients with schizophrenia.
Engaging in dialogue with computerized representation of their voice has therapeutic effects.
Tested immersive VRT or TAU (treatment as usual). VRT helped more with severity of AVH, and led to lasting benefits to depression and quality of life.
Describe the two competing theories for Synesthesia in the brain:
DuMaurier - More connections between areas of the brain. Accounts for automatic nature of Synesthetic experiences.
Eagleman - Less inhibition between different areas oft he brain, less inhibition on spreading activation. Supported by perception during psychedelic use.
Psychedelics can be useful in therapy because
Support neurogenesis, allow for greater plasticity
Greater connectivity wakes up dormant brain areas
Envoke spiritual/mystical experiences and help with interpersonal connection (empathy)
Provide insight from new perspectives
Renewed appreciation for life and natural world
What is Simulator Sickness? Explain some ways to fight it:
Motion sickness associated with visual and vestibular input mismatch.
Don't allow for free-walking.
Fade peripheries.
Put a nose on it.
Power through it -- just get used to it.
Describe some differences and similarities between signed and unsigned error.
Similarities:
Signed and unsigned error describe reward prediction error signaling, and thus mediate learning.
Signed and unsigned errors both represent a difference in model's prediction and the true state of the world.
Differences:
Signed Prediction Error: Reflects the valence of the unexpected outcome. Better or worse than what was predicted. Mediates approach and avoid response. Describes which way to adjust the model.
Unsigned Prediction Error: Reflects degree of surprise or unexpectedness. Indicates amount of error between prediction and observed state of the world. Mediates motivation or arousal. If very high unsigned prediction error, then we must adjust our models.
How do most neural networks learn?
Provide input and desired output
Use a learning algorithm
Provide input without desired output and evaluate the accuracy of the model
Kilteni et al. (2015)
1) How and why body ownership illusions are induced.
2) BOI induction depends on spatiotemporal congruence between the seen and felt stimulation. However, visuotactile correlations are not always necessary to induce BOI (asynchrony in touch/vision can also sometimes induce it).
Can Synesthesia be learned? Yes, no, or kinda - explain the caveats.
Kinda.
Witthoft & Winawer connected the Fisher Price Alphabet toy to adults' Grapheme-Color perception (2013).
Bor et al. found it is possible to induce synesthesia but it takes hours of training, it didn't happen with everyone, the strength of the experience varied, and most lost their associations after 3 months.
The difference between hallucinations and delusions
Hallucinations are generally attributed to inside forces (i.e., meditation, drugs, etc.). whereas delusions are attributed to outside forces (cannot distinguish hallucination from reality).
Describe how VR might help:
Treat mental illness
Increase empathy for other's with differences in identity
Impact education
1. For anxiety disorders: safe exposure therapy.
2. Allow users to embody an avatar of a different race, provided lower scores on implicit bias tests.
3. Online classes in VR, impacts accessibility of education in positive and negative ways.
Explain how Hogendoorn's (2022) description of "restructuring the past" or the idea of "Perception as a Timeline" from lecture can solve the Reverse Causality Problem.
Our perception presides within a window of time, not an instant. Events that occur within the temporal window can alter other events within the temporal window even if the events occurred earlier.
For example, Masked Priming. Some stimulus was truly sensed and processed. A mask then appears immediately after, within the same window. Due to a mask occurring afterward so soon within the same window, the original stimulus is subsequently inaccessible.
The mask is presented so soon after the stimulus, it is as if your prediction that you saw the original stimulus was deemed wrong, and the mask is deemed the correct interpretation. "There couldn't have been a stimulus there, given that we saw the mask was presented there, so it must be that the mask was always there and the stimulus wasn't there." The prediction of the letter is then is then overwritten by the prediction of the mask in order to resolve the error.
Give an example of a self-organizing system
ants, birds
Van du Cruys et al. (2011)
1) Predictive model for aesthetics, art appreciation
2) Argue: art appreciation associated with harmony, gestalt. Great artists play with this and violate our predictions. The process of overcoming the prediction error leads to a (new) understanding of art, leading to greater appreciation. Basically a reward for overcoming a state of uncertainty.