View source for Brain-reading

{{Short description|Use of fMRI to decode brain stimuli}}
'''Brain-reading''' or '''thought identification''' uses the responses of multiple [[Voxel|voxel]]s in the [[Biology:Brain|brain]] evoked by [[Biology:Stimulus (physiology)|stimulus]] then detected by fMRI in order to decode the original stimulus. Advances in research have made this possible by using [[Biology:Neuroimaging|human neuroimaging]] to decode a person's conscious experience based on non-invasive measurements of an individual's brain activity.<ref name="Decoding mental states">{{cite journal|last1=Haynes|first1=John-Dylan|last2=Geraint|first2=Rees|title=Decoding mental states from brain activity in humans|url=http://www.nature.com/nrn/journal/v7/n7/full/nrn1931.html|journal=Nature Reviews Neuroscience |date=1 July 2006 |volume=7 |issue=7 |pages=523–534 |doi=10.1038/nrn1931 |pmid=16791142 |s2cid=16025026 |access-date=8 December 2014}}</ref> Brain reading studies differ in the type of decoding (i.e.&nbsp;classification, identification and reconstruction) employed, the target (i.e.&nbsp;decoding visual patterns, auditory patterns, cognitive states), and the decoding algorithms (linear classification, nonlinear classification, direct reconstruction, Bayesian reconstruction,&nbsp;etc.) employed.

In 2007, Professor of [[Social:Neuropsychology|neuropsychology]] Barbara Sahakian qualified, "A lot of neuroscientists in the field are very cautious and say we can't talk about reading individuals' minds, and right now that is very true, but we're moving ahead so rapidly, it's not going to be that long before we will be able to tell whether someone's making up a story, or whether someone intended to do a crime with a certain degree of certainty."<ref name=Guardian>{{Cite news |work=The Guardian |url=https://www.theguardian.com/science/2007/feb/09/neuroscience.ethicsofscience |title=The brain scan that can read people's intentions |date=9 February 2007 |first=Ian |last=Sample}}</ref>
__FORCETOC__

==Applications==
===Natural images===
Identification of complex natural images is possible using voxels from early and anterior visual cortex areas forward of them (visual areas V3A, V3B, V4, and the lateral occipital) together with [[Bayesian inference]]. This brain reading approach uses three components:<ref name="Naselaris">{{cite journal |last1=Naselaris |first1=Thomas |last2=Prenger |first2=Ryan J. |last3=Kay |first3=Kendrick N. |last4=Oliver |first4=Michael |last5=Gallant |first5=Jack L. |title=Bayesian Reconstruction of Natural Images from Human Brain Activity |journal=Neuron |volume=63 |issue=6 |pages=902–15 |date=2009 |pmid=19778517 |pmc=5553889 |doi=10.1016/j.neuron.2009.09.006}}</ref> a structural encoding model that characterizes responses in early visual areas; a semantic encoding model that characterizes responses in anterior visual areas; and a Bayesian prior that describes the distribution of structural and semantic [[Scene statistics|scene statistics]].<ref name="Naselaris"/>

Experimentally the procedure is for subjects to view 1750 black and white natural images that are correlated with voxel activation in their brains. Then  subjects viewed another 120 novel target images, and information from the earlier scans is used reconstruct them. Natural images used include pictures of a seaside cafe and harbor, performers on a stage, and dense foliage.<ref name="Naselaris"/>

In 2008 [[Company:IBM|IBM]] applied for a patent on how to extract mental images of human faces from the human brain. It uses a feedback loop based on brain measurements of the fusiform gyrus area in the brain which activates proportionate with degree of facial recognition.<ref>{{Cite web |url=http://www.freepatentsonline.com/20100049076.pdf |title=IBM Patent Application US 2010/0049076: Retrieving mental images of faces from the human brain |date=February 25, 2010}}</ref>

In 2011, a team led by Shinji Nishimoto used only brain recordings to partially reconstruct what volunteers were seeing. The researchers applied a new model, about how moving object information is processed in human brains, while volunteers watched clips from several videos. An algorithm searched through thousands of hours of external YouTube video footage (none of the videos were the same as the ones the volunteers watched) to select the clips that were most similar.<ref name="Nishimoto2011">{{citation|first1=Shinji|last1=Nishimoto|first2=An T.|last2=Vu|first3=Thomas|last3=Naselaris|first4=Yuval|last4=Benjamini|first5=Bin|last5=Yu|author5-link=Bin Yu|first6=Jack L.|last6=Gallant|title=Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies|journal=[[Biology:Current Biology|Current Biology]]|doi=10.1016/j.cub.2011.08.031|volume=21|issue=19|year=2011|pages=1641–1646|pmid=21945275|pmc=3326357}}</ref><ref name="Yam2011">{{Cite web |url=https://blogs.scientificamerican.com/observations/breakthrough-could-enable-others-to-watch-your-dreams-and-memories-video/ |title=Breakthrough Could Enable Others to Watch Your Dreams and Memories &#91;Video&#93;, Philip Yam |access-date=2021-05-06 |archive-url=https://web.archive.org/web/20170506165430/https://blogs.scientificamerican.com/observations/breakthrough-could-enable-others-to-watch-your-dreams-and-memories-video/ |archive-date=2017-05-06}}</ref> The authors have uploaded demos comparing the watched and the computer-estimated videos.<ref name="Nishimoto2011Vid1">Nishimoto et al. 2011 uploaded video 1 ''[https://www.youtube.com/watch?v=nsjDnYxJ0bo Movie reconstruction from human brain activity]'' on Youtube</ref><ref name="Nishimoto2011Vid2">Nishimoto et al. 2011 uploaded video 2 ''[https://www.youtube.com/watch?v=KMA23JJ1M1o Movie reconstructions from human brain activity: 3 subjects]'', "Nishimoto.etal.2011.3Subjects.mpeg" on Youtube</ref>

In 2017 a [[Face perception|face perception]] study in monkeys reported the reconstruction of human faces by analyzing electrical activity from 205 neurons.<ref>{{Cite journal |last1=Chang |first1=Le |last2=Tsao |first2=Doris Y. |date=2017-06-01 |title=The Code for Facial Identity in the Primate Brain |journal=Cell |language=English |volume=169 |issue=6 |pages=1013–1028.e14 |doi=10.1016/j.cell.2017.05.011 |issn=0092-8674 |pmid=28575666|pmc=8088389 |s2cid=32432231 }}</ref><ref>{{Cite news |last1=Devlin |first1=Hannah |last2=Correspondent |first2=Science |date=2017-06-01 |title=Scientists discover how the brain recognises faces – by reading monkeys' minds |language=en-GB |work=The Guardian |url=https://www.theguardian.com/science/2017/jun/01/scientists-discover-how-the-brain-recognises-faces-by-reading-monkeys-minds |access-date=2023-03-26 |issn=0261-3077}}</ref>

In 2023 image reconstruction was reported utilizing [[Software:Stable Diffusion|Stable Diffusion]] on human brain activity obtained via fMRI.<ref>{{Cite journal |last1=Takagi |first1=Yu |last2=Nishimoto |first2=Shinji |date=2022-12-01 |title=High-resolution image reconstruction with latent diffusion models from human brain activity |url=https://www.biorxiv.org/content/10.1101/2022.11.18.517004v2 |language=en |pages=2022.11.18.517004 |doi=10.1101/2022.11.18.517004|s2cid=253762952 }}</ref><ref>{{Cite web |title=AI re-creates what people see by reading their brain scans |url=https://www.science.org/content/article/ai-re-creates-what-people-see-reading-their-brain-scans |access-date=2023-03-26 |website=www.science.org |language=en}}</ref>

===Lie detector===
Brain-reading has been suggested as an alternative to polygraph machines as a form of [[Social:Lie detection|lie detection]].<ref name=Wolpe2005>{{cite journal|author1=Wolpe, P. R.|author2=Foster, K. R.|author3=Langleben, D. D.|name-list-style=amp|year=2005|title=Emerging neurotechnologies for lie-detection: promises and perils|journal=The American Journal of Bioethics|volume=5|issue=2|pages=39–49|pmid=16036700|doi=10.1080/15265160590923367|citeseerx=10.1.1.728.9280|s2cid=219640810}}</ref> Another alternative to polygraph machines is blood oxygenated level dependent functional MRI technology (BOLD fMRI). This technique involves the interpretation of the local change in the concentration of oxygenated hemoglobin in the brain, although the relationship between this blood flow and neural activity is not yet completely understood.<ref name=Wolpe2005 />  Another technique to find concealed information is [[Social:Brain fingerprinting|brain fingerprinting]], which uses EEG to ascertain if a person has a specific memory or information by identifying [[Philosophy:P300 (neuroscience)|P300]] event related potentials.<ref>{{cite journal |last1=Farwell |first1=Lawrence A. |last2=Richardson |first2=Drew C. |last3=Richardson |first3=Graham M. |title=Brain fingerprinting field studies comparing P300-MERMER and P300 brainwave responses in the detection of concealed information |journal=Cognitive Neurodynamics |date=5 December 2012 |volume=7 |issue=4 |pages=263–299 |doi=10.1007/s11571-012-9230-0 |pmid=23869200 |pmc=3713201 }}</ref>

A number of concerns have been raised about the accuracy and ethical implications of brain-reading for this purpose. Laboratory studies have found rates of accuracy of up to 85%; however, there are concerns about what this means for false positive results among non-criminal populations: "If the prevalence of "prevaricators" in the group being examined is low, the test will yield far more false-positive than true-positive results; about one person in five will be incorrectly identified by the test."<ref name=Wolpe2005 />  Ethical problems involved in the use of brain-reading as lie detection include misapplications due to adoption of the technology before its reliability and validity can be properly assessed and due to misunderstanding of the technology, and privacy concerns due to unprecedented access to individual's private thoughts.<ref name=Wolpe2005 /> However, it has been noted that the use of polygraph lie detection carries similar concerns about the reliability of the results<ref name=Wolpe2005 />  and violation of privacy.<ref>{{cite journal|author1=Arstila, V.|author2=Scott, F.|name-list-style=amp|year=2011|title=Brain Reading and Mental Privacy|journal=TRAMES: A Journal of the Humanities & Social Sciences|volume=15|issue=2|pages=204–212|doi=10.3176/tr.2011.2.08|url=http://www.kirj.ee/public/trames_pdf/2011/issue_2/Trames-2011-2-204-212.pdf}}</ref>

===Human–machine interfaces===
[[File:EPOC IGN.jpg|thumb|right|140px|The Emotiv Epoc is one way that users can give commands to devices using only thoughts.]]
Brain-reading has also been proposed as a method of improving human–machine interfaces, by the use of EEG to detect relevant brain states of a human.<ref>{{cite journal|author1=Kirchner, E. A.|author2=Kim, S. K.|author3=Straube, S.|author4=Seeland, A.|author5=Wöhrle, H.|author6=Krell, M. M.|author7=Tabie, M.|author8=Fahle, M.|year=2013|title=On the Applicability of Brain Reading for Predictive Human–Machine Interfaces in Robotics|journal=PLOS ONE|volume=8|issue=12|page=e81732|doi= 10.1371/journal.pone.0081732|pmid=24358125|pmc=3864841|doi-access=free}}</ref> In recent years, there has been a rapid increase in patents for technology involved in reading brainwaves, rising from fewer than 400 from 2009–2012 to 1600 in 2014.<ref>{{cite web|title=Surge in U.S. 'brain-reading' patents|date=May 7, 2015|work=BBC News |url=https://www.bbc.com/news/technology-32623063}}</ref> These include proposed ways to control video games via brain waves and "[[Finance:Neuromarketing|neuro-marketing]]" to determine someone's thoughts about a new product or advertisement.

[[Company:Emotiv Systems|Emotiv Systems]], an Australian electronics company, has demonstrated [[Company:Emotiv Systems#Emotiv Epoc|a headset]] that can be trained to recognize a user's thought patterns for different commands. Tan Le demonstrated the headset's ability to manipulate virtual objects on screen, and discussed various future applications for such brain-computer interface devices, from  powering wheel chairs to replacing the mouse and keyboard.<ref name=TedWaves>{{Cite web |first=Tan |last=Le |url=https://www.ted.com/talks/tan_le_a_headset_that_reads_your_brainwaves?language=en |title=A headset that reads your brainwaves |work=TEDGlobal |date=2010}}</ref>

===Detecting attention===
It is possible to track which of two forms of rivalrous binocular illusions a person was subjectively experiencing from fMRI signals.<ref>{{cite journal |last1=Haynes |first1=J |last2=Rees |first2=G |title=Predicting the Stream of Consciousness from Activity in Human Visual Cortex |journal=Current Biology |volume=15 |issue=14 |pages=1301–7 |date=2005 |pmid=16051174 |doi=10.1016/j.cub.2005.06.026|s2cid=6456352 |doi-access=free }}</ref>

When humans think of an object, such as a screwdriver, many different areas of the brain activate. Marcel Just and his colleague, Tom Mitchell, have used fMRI brain scans to teach a computer to identify the various parts of the brain associated with specific thoughts.<ref name=SixtyMinutes>{{Cite news |work=CNET News |url=http://news.cnet.com/8301-11386_3-10131643-76.html |title='60 Minutes' video: Tech that reads your mind |date=January 5, 2009}}</ref> This technology also yielded a discovery: similar thoughts in different human brains are surprisingly similar neurologically. To illustrate this, Just and Mitchell used their computer to predict, based on nothing but fMRI data, which of several images a volunteer was thinking about. The computer was 100% accurate, but so far the machine is only distinguishing between 10 images.<ref name=SixtyMinutes/>

===Detecting thoughts===
The category of event which a person freely recalls can be identified from fMRI before they say what they remembered.<ref>{{cite journal |last1=Polyn |first1=S. M. |last2=Natu |first2=VS |last3=Cohen |first3=JD |last4=Norman |first4=KA |title=Category-Specific Cortical Activity Precedes Retrieval During Memory Search |journal=Science |volume=310 |issue=5756 |pages=1963–6 |date=2005 |pmid=16373577 |doi=10.1126/science.1117645|doi-access=free }}</ref>

December 16, 2015, a study conducted by Toshimasa Yamazaki at Kyushu Institute of Technology found that during a rock-paper-scissors game a computer was able to determine the choice made by the subjects before they moved their hand. An [[Biology:Electroencephalography|EEG]] was used to measure activity in the [[Biology:Broca's area|Broca's area]] to ''see'' the words two seconds before the words were uttered.<ref name="IEICE-Yamazaki1">{{Cite journal|url=https://www.ieice.org/ken/paper/20150317SBYj/eng/|title=Silent Speech BCI - An investigation for practical problems|journal=IEICE Technical Report |date = 2015-12-16|volume=114 |issue=514 |pages=81–84 |access-date=2016-01-17|publisher=IEICE Technical Committee |last1=Ito |first1=Takashi |last2=Yamaguchi |first2=Hiromi |last3=Yamaguchi |first3=Ayaka |last4=Yamazaki |first4=Toshimasa |last5=Fukuzumi |first5=Shin'Ichi |last6=Yamanoi |first6=Takahiro }}</ref><ref name="discovery-Yamazaki1">{{Cite web|url=http://news.discovery.com/tech/gear-and-gadgets/mind-reading-computer-knows-what-youre-about-to-say-160107.htm|first1=Alyssa|last1= Danigelis|title=Mind-Reading Computer Knows What You're About to Say|date = 2016-01-07|access-date=2016-01-17|publisher=Discovery News}}</ref><ref name="Nishinippon-Yamazaki1">{{Cite web|url=http://www.nishinippon.co.jp/nnp/national/article/216215|title=頭の中の言葉 解読 障害者と意思疎通、ロボット操作も 九工大・山崎教授ら|language=ja|date=2016-01-04|access-date=2016-01-17|publisher=Nishinippon Shimbun|archive-url=https://web.archive.org/web/20160117074934/http://www.nishinippon.co.jp/nnp/national/article/216215|archive-date=17 January 2016|url-status=dead}}</ref>

In 2023, the University of Texas in Austin trained a non-invasive brain decoder to translate volunteers' brainwaves into the GPT-1 [[Language model|language model]]. After lengthy training on each individual volunteer, the decoder usually failed to reconstruct the exact words, but could nevertheless reconstruct meanings close enough that the decoder could, most of the time, identify what timestamp of a given book the subject was listening to.<ref>{{cite news |last1=O'Sullivan |first1=Donie |title=How the technology behind ChatGPT could make mind-reading a reality |url=https://www.cnn.com/2023/05/23/tech/chatgpt-mind-reading/index.html |access-date=17 June 2023 |work=CNN |date=23 May 2023 |language=en}}</ref><ref>{{cite journal |last1=Tang |first1=Jerry |last2=LeBel |first2=Amanda |last3=Jain |first3=Shailee |last4=Huth |first4=Alexander G. |title=Semantic reconstruction of continuous language from non-invasive brain recordings |journal=Nature Neuroscience |date=May 2023 |volume=26 |issue=5 |pages=858–866 |doi=10.1038/s41593-023-01304-9 |pmid=37127759 |s2cid=252684880 |url=https://www.nature.com/articles/s41593-023-01304-9}}</ref>

===Detecting language===
Statistical analysis of EEG brainwaves has been claimed to allow the recognition of [[Social:Phoneme|phoneme]]s,<ref>{{cite journal |last1=Suppes |first1=Patrick |last2=Perreau-Guimaraes |first2=Marcos |last3=Wong |first3=Dik Kin |title=Partial Orders of Similarity Differences Invariant Between EEG-Recorded Brain and Perceptual Representations of Language |journal=Neural Computation |volume=21 |issue=11 |pages=3228–69 |date=2009 |pmid=19686069 |doi=10.1162/neco.2009.04-08-764|s2cid=18097705 }}</ref> and (in 1999) at a 60% to 75% level color and visual shape words.<ref>{{cite journal |doi=10.1073/pnas.96.25.14658 |last1=Suppes |first1=Patrick |last2=Han |first2=Bing |last3=Epelboim |first3=Julie |last4=Lu |first4=Zhong-Lin |title=Invariance of brain-wave representations of simple visual images and their names |journal=Proceedings of the National Academy of Sciences of the United States of America |volume=96 |issue=25 |pages=14658–63 |date=1999 |pmid=10588761 |pmc=24492|doi-access=free }}</ref>

On 31 January 2012 Brian Pasley and colleagues of University of California Berkeley published their paper in PLoS Biology wherein subjects' internal neural processing of auditory information was decoded and reconstructed as sound on computer by gathering and analyzing electrical signals directly from subjects' brains.<ref name= "Pasley">{{cite journal | last1 = Pasley | first1 = BN | last2 = David | first2 = SV | last3 = Mesgarani | first3 = N | last4 = Flinker | first4 = A | last5 = Shamma | first5 = SA |display-authors=0 | year = 2012 | title = Reconstructing Speech from Human Auditory Cortex | journal = PLOS Biol | volume = 10 | issue = 1| page = e1001251 | doi = 10.1371/journal.pbio.1001251 | pmid = 22303281 | pmc = 3269422 | doi-access = free }}</ref> The research team conducted their studies on the superior temporal gyrus, a region of the brain that is involved in higher order neural processing to make semantic sense from auditory information.<ref name= "BBC auditorydecoding">{{Cite news |url=https://www.bbc.co.uk/news/science-environment-16811042 |title=Science decodes 'internal voices' |work=BBC News |date=31 January 2012 |first=Jason |last=Palmer}}</ref> The research team used a computer model to analyze various parts of the brain that might be involved in neural firing while processing auditory signals. Using the computational model, scientists were able to identify the brain activity involved in processing auditory information when subjects were presented with recording of individual words.<ref name= "NHS_inner_voice_unlocked">{{Cite news |url=http://www.nhs.uk/news/2012/02February/Pages/mind-reading-telephathy-inner-voice.aspx |title=Secrets of the inner voice unlocked |publisher=NHS |date=1 Feb 2012 |url-status=dead |archiveurl=https://web.archive.org/web/20120203022143/http://www.nhs.uk/news/2012/02February/Pages/mind-reading-telephathy-inner-voice.aspx |archivedate=2012-02-03}}</ref> Later, the computer model of auditory information processing was used to reconstruct some of the words back into sound based on the neural processing of the subjects. However the reconstructed sounds were not of good quality and could be recognized only when the audio wave patterns of the reconstructed sound were visually matched with the audio wave patterns of the original sound that was presented to the subjects.<ref name="NHS_inner_voice_unlocked"/> However this research marks a direction towards more precise identification of neural activity in cognition.

===Predicting intentions===
Some researchers in 2008 were able to predict, with 60% accuracy, whether a subject was going to push a button with their left or right hand. This is notable, not just because the accuracy is better than chance, but also because the scientists were able to make these predictions up to 10 seconds before the subject acted – well before the subject felt they had decided.<ref>{{Cite journal| doi = 10.1038/nn.2112| pmid = 18408715| year = 2008| last1 = Soon | first1 = C.| last2 = Brass | first2 = M.| last3 = Heinze | first3 = H.| last4 = Haynes | first4 = J.| title = Unconscious determinants of free decisions in the human brain| volume = 11| issue = 5| pages = 543–545| journal = Nature Neuroscience| citeseerx = 10.1.1.520.2204| s2cid = 2652613}}</ref> This data is even more striking in light of other research suggesting that the decision to move, and possibly the ability to cancel that movement at the last second,<ref name="Kuhn Brass 2009">{{cite journal | last1 = Kühn | first1 = S. | last2 = Brass | first2 = M. | year = 2009 | title = Retrospective construction of the judgment of free choice | journal = Consciousness and Cognition | volume = 18 | issue = 1| pages = 12–21 | doi=10.1016/j.concog.2008.09.007 | pmid=18952468| s2cid = 9086887 }}</ref> may be the results of unconscious processing.<ref name="Matsuhashi, M. 2008">{{cite journal | last1 = Matsuhashi | first1 = M. | last2 = Hallett | first2 = M. | year = 2008 | title = The timing of the conscious intention to move | journal = European Journal of Neuroscience | volume = 28 | issue = 11| pages = 2344–2351 | doi=10.1111/j.1460-9568.2008.06525.x | pmid=19046374 | pmc=4747633}}</ref>

John Dylan-Haynes has also demonstrated that fMRI can be used to identify whether a volunteer is about to add or subtract two numbers in their head.<ref name=SixtyMinutes/>

=== Predictive processing in the brain ===
Neural decoding techniques have been used to test theories about the predictive brain, and to investigate how top-down predictions affect brain areas such as the [[Biology:Visual cortex|visual cortex]]. Studies using fMRI decoding techniques have found that predictable sensory events<ref>{{Cite journal|last1=Kok|first1=Peter|last2=Jehee|first2=Janneke|last3=de Lange|first3=Floris|date=2012|title=Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex|journal=Neuron|volume=75|issue=2|pages=265–270|doi=10.1016/j.neuron.2012.04.034|pmid=22841311|issn=0896-6273|doi-access=free}}</ref> and the expected consequences of our actions<ref>{{Cite journal|last1=Yon|first1=Daniel|last2=Gilbert|first2=Sam J.|last3=de Lange|first3=Floris P.|last4=Press|first4=Clare|date=2018|title=Action sharpens sensory representations of expected outcomes|journal=Nature Communications|language=en|volume=9|issue=1|pages=4288|doi=10.1038/s41467-018-06752-7|issn=2041-1723|pmc=6191413|pmid=30327503}}</ref> are better decoded in visual brain areas, suggesting that prediction 'sharpens' representations in line with expectations.

===Virtual environments===
It has also been shown that brain-reading can be achieved in a complex [[Virtual environment|virtual environment]].<ref>{{cite journal |last1=Chu |first1=Carlton |last2=Ni |first2=Yizhao |last3=Tan |first3=Geoffrey |last4=Saunders |first4=Craig J. |last5=Ashburner |first5=John |title=Kernel regression for fMRI pattern prediction |journal=NeuroImage |date=2010 |doi=10.1016/j.neuroimage.2010.03.058 |volume=56 |issue=2 |pages=662–673 |pmid=20348000 |pmc=3084459}}</ref>

===Emotions===
Just and Mitchell also claim they are beginning to be able to identify kindness, hypocrisy, and love in the brain.<ref name=SixtyMinutes/>

===Security===
In 2013 a project led by University of California Berkeley professor John Chuang published findings on the feasibility of brainwave-based computer authentication as a substitute for passwords. Improvements in the use of biometrics for computer authentication has continually improved since the 1980s, but this research team was looking for a method faster and less intrusive than today's retina scans, fingerprinting, and voice recognition. The technology chosen to improve security measures is an electroencephalogram (EEG), or brainwave measurer, to improve passwords into "pass thoughts." Using this method Chuang and his team were able to customize tasks and their authentication thresholds to the point where they were able to reduce error rates under 1%, significantly better than other recent methods. In order to better attract users to this new form of security the team is still researching mental tasks that are enjoyable for the user to perform while having their brainwaves identified. In the future this method could be as cheap, accessible, and straightforward as thought itself.<ref>{{cite web|title=New Research: Computers That Can Identify You by Your Thoughts |url=http://www.ischool.berkeley.edu/newsandevents/news/20130403brainwaveauthentication|website=UC Berkeley School of Information |date=April 3, 2013 |publisher=UC Berkeley|access-date=8 December 2014}}</ref>

John-Dylan Haynes states that fMRI can also be used to identify recognition in the brain. He provides the example of a criminal being interrogated about whether he recognizes the scene of the crime or murder weapons.<ref name=SixtyMinutes/>

==Methods of analysis==
===Classification===
In classification, a pattern of activity across multiple voxels is used to determine the particular class from which the stimulus was drawn.<ref>{{cite journal |last1=Kamitani |first1=Yukiyasu |last2=Tong |first2=Frank |title=Decoding the visual and subjective contents of the human brain |journal=Nature Neuroscience |volume=8 |issue=5 |pages=679–85 |date=2005 |pmid=15852014 |pmc=1808230 |doi=10.1038/nn1444|author1-link=Yukiyasu Kamitani }}</ref> Many studies have classified visual stimuli, but this approach has also been used to classify cognitive states.{{citation needed|date=July 2019}}

===Reconstruction===
In reconstruction brain reading the aim is to create a literal picture of the image that was presented. Early studies used voxels from early [[Biology:Visual cortex|visual cortex]] areas (V1, V2, and V3) to reconstruct geometric stimuli made up of flickering checkerboard patterns.<ref>{{cite journal |last1=Miyawaki |first1=Y |last2=Uchida |first2=H |last3=Yamashita |first3=O |last4=Sato |first4=M |last5=Morito |first5=Y |last6=Tanabe |first6=H |last7=Sadato |first7=N |last8=Kamitani |first8=Y |title=Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders |journal=Neuron |volume=60 |issue=5 |pages=915–29 |date=2008 |pmid=19081384 |doi=10.1016/j.neuron.2008.11.004|s2cid=17327816 |doi-access=free }}</ref><ref>{{cite journal |last1=Thirion |first1=Bertrand |last2=Duchesnay |first2=Edouard |last3=Hubbard |first3=Edward |last4=Dubois |first4=Jessica |last5=Poline |first5=Jean-Baptiste |last6=Lebihan |first6=Denis |last7=Dehaene |first7=Stanislas |title=Inverse retinotopy: Inferring the visual content of images from brain activation patterns |journal=NeuroImage |volume=33 |issue=4 |pages=1104–16 |date=2006 |pmid=17029988 |doi=10.1016/j.neuroimage.2006.06.062|s2cid=13361917 }}</ref>

===EEG===
EEG has also been used to identify recognition of specific information or memories by the [[Philosophy:P300 (neuroscience)|P300]] event related potential, which has been dubbed '[[Social:Brain fingerprinting|brain fingerprinting]]'.<ref>{{cite journal | last1=Farwell | first1=Lawrence A. | last2=Richardson | first2=Drew C. | last3=Richardson | first3=Graham M. | title=Brain fingerprinting field studies comparing P300-MERMER and P300 brainwave responses in the detection of concealed information | journal=Cognitive Neurodynamics | publisher=Springer | volume=7 | issue=4 | date=2012-12-05 | issn=1871-4080 | doi=10.1007/s11571-012-9230-0 | pages=263–299| pmid=23869200 | pmc=3713201 }}</ref>

==Accuracy==
Brain-reading accuracy is increasing steadily as the quality of the data and the complexity of the decoding algorithms improve. In one recent experiment it was possible to identify which single image was being seen from a set of 120.<ref>{{cite journal |last1=Kay |first1=Kendrick N. |last2=Naselaris |first2=Thomas |last3=Prenger |first3=Ryan J. |last4=Gallant |first4=Jack L. |title=Identifying natural images from human brain activity |journal=Nature |volume=452 |issue=7185 |pages=352–5 |date=2008 |pmid=18322462 |doi=10.1038/nature06713 |pmc=3556484|bibcode=2008Natur.452..352K }}</ref> In another it was possible to correctly identify  90% of the time which of two categories the stimulus came and the specific semantic category (out of 23) of the target image 40% of the time.<ref name="Naselaris"/>

==Limitations==
It has been noted that so far brain-reading is limited. "In practice, exact reconstructions are impossible to achieve by any reconstruction algorithm on the basis of brain activity signals acquired by fMRI. This is because all reconstructions will inevitably be limited by inaccuracies in the encoding models and noise in the measured signals. Our results {{who|date=June 2013}} demonstrate that the natural image prior is a powerful (if unconventional) tool for mitigating the effects of these fundamental limitations. A natural image prior with only six million images is sufficient to produce reconstructions that are structurally and semantically similar to a target image."<ref name="Naselaris"/>

==Ethical issues==
With [[Biology:Neuroimaging|brain scanning]] technology becoming increasingly accurate, experts predict important debates over how and when it should be used. One potential area of application is criminal law. Haynes states that simply refusing to use brain scans on suspects also prevents the wrongly accused from proving their innocence.<ref name=Guardian/> US scholars generally believe that involuntary brain reading, and involuntary polygraph tests, would violate the 5th Amendment's right to not self incriminate.<ref>{{cite journal | last1=Allen | first1=Ronald J. | last2=Mace | first2=M. Kristin | title=The Self-Incrimination Clause Explained and Its Future Predicted | publisher=Elsevier |volume=94 |issue=2 |date=Winter 2004 | journal=The Journal of Criminal Law and Criminology | doi=10.2139/ssrn.480143| url=https://scholarlycommons.law.northwestern.edu/jclc/vol94/iss2/1 }}</ref><ref>{{Cite web |last=Brennan-Marquez |first=Kiel |url=https://yjolt.org/modest-defense-mind-reading |title=A modest defense of mind reading |work=Yale Journal of Law & Technology |publisher=Yale University |volume=15 |year=2012 |page=214 |quote=Ronald Allen and Kristen Mace discern 'universal agreement' that the (Mind Reader Machine) is unacceptable.}}</ref> One perspective is to consider whether brain imaging is like testimony, or instead like DNA, blood, or semen. Paul Root Wolpe, director of the Center for Ethics at Emory University in Atlanta predicts that this question will be decided by a Supreme Court case.<ref name="CBS 60 Minutes">{{cite web|title=How Technology May Soon "Read" Your Mind|url=http://www.cbsnews.com/news/how-technology-may-soon-read-your-mind/|website=CBS News|date=December 31, 2008|access-date=8 December 2014}}</ref>

In other countries outside the United States, thought identification has already been used in criminal law. In 2008 an Indian woman was convicted of murder after an EEG of her brain allegedly revealed that she was familiar with the circumstances surrounding the poisoning of her ex-fiancé.<ref name="CBS 60 Minutes"/> Some neuroscientists and legal scholars doubt the validity of using thought identification as a whole for anything past research on the nature of deception and the brain.<ref name=Lying>{{cite web|last1=Stix|first1=Gary|title=Can fMRI Really Tell If You're Lying?|url=http://www.scientificamerican.com/article/new-lie-detector/|website=Scientific American |date=August 1, 2008|access-date=8 December 2014}}</ref>

''[[Finance:The Economist|The Economist]]'' cautioned people to be "afraid" of the future impact, and some ethicists argue that privacy laws should protect private thoughts. Legal scholar [[Biography:Henry Greely|Hank Greely]] argues that the court systems could benefit from such technology, and neuroethicist [[Biography:Julian Savulescu|Julian Savulescu]] states that brain data is not fundamentally different from other types of evidence.<ref>{{cite journal |last1=Smith |first1=Kerri |title=Brain decoding: Reading minds |journal=Nature News |date=24 October 2013 |volume=502 |issue=7472 |pages=428–430 |doi=10.1038/502428a |pmid=24153277 |language=en|doi-access=free }}</ref> In ''Nature'', journalist Liam Drew writes about emerging projects to attach brain-reading devices to speech synthesizers or other output devices for the benefit of [[Medicine:Tetraplegia|tetraplegics]]. Such devices could create concerns of accidentally broadcasting the patient's "inner thoughts" rather than merely conscious speech.<ref>{{cite journal |last1=Drew |first1=Liam |title=The ethics of brain–computer interfaces |journal=Nature |date=24 July 2019 |volume=571 |issue=7766 |pages=S19–S21 |doi=10.1038/d41586-019-02214-2 |pmid=31341310 |language=en|doi-access=free }}</ref>

== History ==
[[File:MRI-Philips.JPG|thumb|MRI scanner that could be used for Thought Identification]]
Psychologist John-Dylan Haynes experienced breakthroughs in brain imaging research in 2006 by using fMRI. This research included new findings on visual object recognition, tracking dynamic mental processes, [[Social:Lie detection|lie detecting]], and decoding unconscious processing. The combination of these four discoveries revealed such a significant amount of information about an individual's thoughts that Haynes termed it "brain reading".<ref name="Decoding mental states" />

The fMRI has allowed research to expand by significant amounts because it can track the activity in an individual's brain by measuring the brain's blood flow. It is currently thought to be the best method for measuring brain activity, which is why it has been used in multiple research experiments in order to improve the understanding of how doctors and psychologists can identify thoughts.<ref name="fMRI">{{cite web|last1=Saenz|first1=Aaron|title=fMRI Reads the Images in Your Brain – We Know What You're Looking At|url=http://singularityhub.com/2010/03/17/fmri-reads-the-images-in-your-brain-we-know-what-youre-looking-at-video/|website=SingularityHUB|date=17 March 2010|publisher=Singularity University|access-date=8 December 2014}}</ref>

In a 2020 study, AI using implanted electrodes could correctly transcribe a sentence read aloud from a fifty-sentence test set 97% of the time, given 40 minutes of training data per participant.<ref>{{cite news |title=Scientists develop AI that can turn brain activity into text |url=https://www.theguardian.com/science/2020/mar/30/scientists-develop-ai-that-can-turn-brain-activity-into-text |access-date=31 March 2020 |work=The Guardian |date=30 March 2020 |first=Nicola |last=Davis |language=en}}</ref>

== Future research ==
Experts are unsure of how far thought identification can expand, but Marcel Just believed in 2014 that in 3–5 years there will be a machine that is able to read complex thoughts such as 'I hate so-and-so'.<ref name="CBS 60 Minutes"/>

Donald Marks, founder and chief science officer of MMT, is working on playing back thoughts individuals have after they have already been recorded.<ref name="Mind Reader">{{cite web|last1=Cuthbertson|first1=Anthony|title=Mind Reader: Meet The Man Who Records and Stores Your Thoughts, Dreams and Memories|url=http://www.ibtimes.co.uk/mind-reader-meet-man-who-records-stores-your-thoughts-dreams-memories-1463094|website=International Business Times|access-date=8 December 2014|date=2014-08-29}}</ref>

Researchers at the University of California Berkeley have already been successful in forming, erasing, and reactivating memories in rats. Marks says they are working on applying the same techniques to humans. This discovery could be monumental for war veterans who suffer from [[Medicine:Posttraumatic stress disorder|PTSD]].<ref name="Mind Reader" />

Further research is also being done in analyzing brain activity during video games to detect criminals, [[Finance:Neuromarketing|neuromarketing]], and using brain scans in government security checks.<ref name="CBS 60 Minutes" /><ref name="fMRI" />

== In popular culture ==
Episode ''Black Hole'' of American medical drama ''House'', which aired on March 15, 2010, featured an experimental "cognitive imaging" device that supposedly allowed seeing into a patient's subconscious mind. The patient was first put in a preparation phase of six hours while watching video clips, attached to a neuroimaging device looking like [[Biology:Electroencephalography|electroencephalography]] or [[Biology:Functional near-infrared spectroscopy|Functional near-infrared spectroscopy]], to train the neuroimaging classifier. Then the patient was put under [[Medicine:Twilight anesthesia|twilight anesthesia]], and the same device was used to try to infer what was going through the patient's mind. The fictional episode somewhat anticipated the study by Nishimoto et al. published the following year, in which fMRI was used instead.<ref name="Nishimoto2011"/><ref name="Yam2011"/><ref name="Nishimoto2011Vid1"/><ref name="Nishimoto2011Vid2"/> In the movie Dumb and Dumber To, one scene shows a brain reader. In the Henry Danger episode, "Dream Busters," a machine showed Henry's [[Biology:Dream|dream]].

==See also==
*[[Bayesian approaches to brain function]]
*[[Cyberware]]
*[[Philosophy:Mind uploading|Mind uploading]]
*''Minority Report'' (film)
*[[Neural decoding]]
*[[Neuroinformatics]]
*[[Social:Thoughtcrime|Thoughtcrime]]
*[[Thought recording and reproduction device]]

==References==
{{Reflist|30em}}

==External links==
*[https://www.newscientist.com/article/mg20427323.500-brain-scanners-can-tell-what-youre-thinking-about.html?full=true Brain scanners can tell what you're thinking about] New Scientist article on brain-reading 28 October 2009
*[https://web.archive.org/web/20110227150548/http://pbc.lrdc.pitt.edu/?q=2007-home 2007 Pittsburgh Brain Activity Interpretation Competition]:Interpreting subject-driven actions and sensory experience in a rigorously characterized virtual world
*[https://www.youtube.com/watch?v=qt21L7tCHYU Mind-reading tech is closer than you think], 2022 BBC video
{{Neuroscience}}
{{emerging technologies|topics=yes|neuro=yes|infocom=yes}}

[[Category:Visual perception]]
[[Category:Neurotechnology]]
[[Category:Computational neuroscience]]

{{Sourceattribution|Brain-reading}}