27
The Machines Are Taking Over
I'm not even gonna pretend to be able to summarize how this was done (read about it here or in the video summary), but basically now you can look at something and an MRI and computer can sort of jumble together a pretty good representation of what you're seeing just by reading your brain activity. And "sort of" is close enough to INSANE since it's literally a machine scanning your brain and it might work with dreams and did you even watch that video? Boom! The future. We're all gonna die.
Oh shit.
@thebigcheese SkyNet is here.
This brings up several questions, like why is the video on the right so blatantly borrowing footage from The Ring? Also, I don't understand how you can have sex with this technology? And in conclusion, why were they using clips from what appear to be one of the two Pink Panther remakes with Steve Martin? Was one of the researchers a huge remade-Inspector-Clouseau fan? "At last! My opportunity to forcibly show more people Steve Martin's greatest work!"
Change is scary!
But I really, really want to watch my own dream movies.
@NeenerNeener dream movies? do we really need to see our dreams projected out from our own heads? um, yikes. they may not let me out in public anymore.
@CrescentMelissa I didn't say anyone else was allowed to watch.
Unfortunately this result is not nearly as interesting as it seems… here's a step by step way of recreating what they did.
1. Pick 5 youtube videos
2. Record MRI data of the brain looking at those 5 videos
3. Take a break, have a coffee or something
4. Show the same subject one of the 5 videos, at random
5. Calculate the similarity between the current MRI data and the MRI data from step 2
6. Overlay all 5 youtube videos, weighted by the similarity measure from step 5
There's nothing interesting here. We will never get a good idea of what the brain is actually doing using MRI images — gotta get our hands dirty and record neurons directly!
@Trevor Bekolay@twitter GET OUR HANDS DIRTY?! I am suddenly very protective of the back of my head. Someone might sneak up behind me, attempting to get their hands dirty on my neurons!…I need a hat.
@sharkey Haha your neurons are safe! Unless you're a mouse or rat. Then don't trust anybody in a lab coat.
@Trevor Bekolay@twitter Not quite, it's more like:
1. Pick 500 youtube videos
2. Record MRI data of the brain looking at those 500 videos
3. Take a break, have a coffee or something
4. Show the same subject a video not included in those 500 videos
5. Calculate the similarity between the current MRI data and the MRI data from step 2
6. Overlay all the ten most similar youtube videos, weighted by the similarity measure from step 5
Which is a bit more impressive, although the bank of videos they were using (18 billion one second clips) was so large that it is still pretty much the same as what you said. I agree that you can't learn that much about what the brain is doing with these images without a more detailed approach, but it is still interesting that such a crude approach can give such impressive results.
@BEARpoint more like FAIRpoint
But yeah, the results are, if nothing else, nice to look at
@Trevor Bekolay@twitter
Actually, you both are missing a step here (step 3 below), which is what actually makes the results cool.
1. Subjects watch 7,200 seconds of movies in the scanner. These are the "training" stimuli.
2. From this, the unique brain activity elicited by each clip is recorded and fed into the model (paired with the clip that elicited it). The model is being trained to associate specific visual patterns/motion in each movie with specific patterns of brain activity.
3. 18 million seconds of random YouTube videos that the subjects have not previously watched are fed into the model from step 2. The model analyzes the visual patterns/motion in these videos and produces the brain activity that would be predicted if the subjects were actually watching the videos.
4. Subjects also watch 540 seconds of different movies in the scanner. These are the "test" stimuli. They watch each movie 10 times to get enough data on these movies.
5. The brain activity elicited by the test stimuli are compared to the predicted brain activities produced in step 3. The model picks the 100 clips with predicted brain activity most similar to the brain activity elicited by the test stimuli. These are merged together to form the blurry videos seen above.
@drlolo So to sum it up: the composite movies were composed of clips never actually seen by the subjects. There was no "actual" MRI data to compare the subject's "test" MRI data to, only "predicted" MRI data from the model.
@Trevor Bekolay@twitter
P.S. "We will never get a good idea of what the brain is actually doing using MRI images — gotta get our hands dirty and record neurons directly!"
I could just as easily say "We'll never get a good idea of what the brain is doing recording from a single or even hundreds of neurons — gotta get our hands dirty and look at networks across the brain!"
If it makes you feel better, this is one more step towards Animorphs. Without the creepy alien parasite infestations.
is anyone familiar with Jason Salavon's art? a) he's awesome, & b) some of his composite paintings–like the ones linked below (averages of Playboy centerfolds from 1960-1999)–look weirdly similar to this.
http://salavon.com/work/EveryPlayboyCenterfoldDecades/grid/11/
Am I the only person who thinks the reconstructed clips are shit?
Don't understand how this works but I find it interesting that the human faces seem a lot clearer. Even newborns will stare longer at inanimate objects that imitate a human face, like two raisins above smiling banana. Fascinating!
@merritt@twitter It's probably because the youtube videos included lots of human faces to use to reconstruct (and, according to a different interview with the study author, no videos of elephants)
What "YOUR" seeing? Hairpin?!
Also, just to nit-pick a lot of speculation I see flying around the internet about this:
The model as-is could not be used to reconstruct what video a person is watching naturalistically. This is because the eyes move around when you watch a video. In the experiment, the subject's gaze was fixated on a certain point for the whole movie. This ensured that the video stayed centered in the visual field and thus the same voxels processed the same parts of the image. If your eyes are moving around, the brain activity in V1 (primary visual cortex–the first cortical area to process visual information) will shift with gaze and the model would not work (it processes voxel-by-voxel).
For this reason, the model as-is would probably also not work with imagined scenes (can you imagine sticking a "fixation point" into an imagined scene?). In addition, although V1 is activated in imagining scenes, I don't know how precise the information is. Detail might be added by higher-level visual centers, and thus you'd be unable to reconstruct the imagined scene through the activation of V1 alone.
Finally, dreaming is completely and utterly far-off into the future. I sincerely doubt that V1 is activated precisely in dreaming. It is more likely much more spread-out activation in various areas of cortex, something this model in its current form can't even hope of decoding.
@drlolo once again science has crushed the dreams of thousands
What else is science for???? …jk, love you, Science!!!!!!!!!
My mind, it is blown. HOLY FUCK I LOVE SCIENCE.
Aahh I love this crap. I'm taking a class on fMRI with this guy, who uses it to communicate with people in vegetative states: http://www.psych.ucla.edu/faculty/faculty_page?id=187&area=3 I will be nerding out weekly.
Also, once you have been studying long enough, these flawed, incrementally-improving attempts to see inside the brain seem earth-shattering. Nothing like grad school to make you appreciate tiny effect sizes and un-generalizable findings.
Am I the only person who finds this visually really scary?
Oh man I had this theory during the OJ Simpson fiasco kinda. I thought they could just reconstruct what Nicole Brown saw by looking at her brain.
Didn't they do something like this on an episode of House? I remember seeing this deus ex machina with the subtlety of an anvil being used to solve the mystery of the week with just 5 minutes to spare. "Oh, let's just scan her brain with this experimental technology and we'll be able to see her thoughts!"
And I was all,"Anyone else see that shark in the rearview mirror?"
But it's for real? Really?