What are we doing?
Here are brief descriptions of some of the areas we have been working on in recent years. See also Publications
The Nature of Compositionality: Investigating Semantic 'gaps' in Sentence Comprehension
How are sentences understood? And, more specifically, how do we attain the meaning of a sentence from its parts? At first, one might think that this simply requires understanding the sentence's individual words and how they combine. But oftentimes sentences have semantic 'gaps'. For instance, a sentence such as 'the boy finished the letter' might be understood as 'the boy finished [reading] the letter', while in reality the sentence could be about anything (even 'eating' the letter). Usually we understand these sentences in context, but some believe that regardless of context, there is a 'default' interpretation. We are investigating the default hypothesis (called 'coercion') using a variety of techniques and also advancing theoretical work aiming to understand the mechanisms responsible for filling the gap in--whether by contextual information (e.g., pragmatics) or whether by default semantic change ('coercion'). The illustration depicts partial results from our fMRI experiment showing right hemisphere involvement in sentences such as 'the boy finished the letter', suggesting that pragmatic processes are recruited in the interpretation of these sentences (see de Almeida, 2004), beyond their recruitment in other, fully specified sentences (such as 'the boy read the letter'). (See de Almeida & Riven, 2012, for discussion, and de Almeida, Riven, Manouilidou, Lungu, Dwivedi, Jarema, & Gillon, 2016, for a full set of results [link to article]). A recent study using a recognition memory paradigm investigates how these sentences are interpreted in context [link to article]. Two recent articles, one with Ernie Lepore (Rutgers University), and one with Caitlyn Antal, discuss theoretical issues related to this topic. [see Publications]
How are sentences understood? And, more specifically, how do we attain the meaning of a sentence from its parts? At first, one might think that this simply requires understanding the sentence's individual words and how they combine. But oftentimes sentences have semantic 'gaps'. For instance, a sentence such as 'the boy finished the letter' might be understood as 'the boy finished [reading] the letter', while in reality the sentence could be about anything (even 'eating' the letter). Usually we understand these sentences in context, but some believe that regardless of context, there is a 'default' interpretation. We are investigating the default hypothesis (called 'coercion') using a variety of techniques and also advancing theoretical work aiming to understand the mechanisms responsible for filling the gap in--whether by contextual information (e.g., pragmatics) or whether by default semantic change ('coercion'). The illustration depicts partial results from our fMRI experiment showing right hemisphere involvement in sentences such as 'the boy finished the letter', suggesting that pragmatic processes are recruited in the interpretation of these sentences (see de Almeida, 2004), beyond their recruitment in other, fully specified sentences (such as 'the boy read the letter'). (See de Almeida & Riven, 2012, for discussion, and de Almeida, Riven, Manouilidou, Lungu, Dwivedi, Jarema, & Gillon, 2016, for a full set of results [link to article]). A recent study using a recognition memory paradigm investigates how these sentences are interpreted in context [link to article]. Two recent articles, one with Ernie Lepore (Rutgers University), and one with Caitlyn Antal, discuss theoretical issues related to this topic. [see Publications]
The Nature of Concepts
Concepts are the most elementary mental representations bearing meaning. They enter into other higher cognitive processes such as thoughts (in fact, thoughts have concepts as their constituents just like sentences have words as their constituents). But how are concepts represented in the mind/brain? How do you represent, say, the concept DOG? Is it represented as a set of features (for instance, +barks, +animal, +domesticated, +furry)? How are concepts tied into categories? One line of investigation is on the nature of concepts that denote causation—such as the meaning of the verb to KILL. Some people think that verbs are represented in the brain by a "conceptual template" such as [[x ACT] CAUSE [y BECOME [dead]]] which contains primitive predicates (e.g., ACT, CAUSE, BECOME) and argument positions, x and y, specifying the kinds of elements the verb requires to form a grammatical sentence. We are investigating this hypothesis with causative verbs, employing different types of experimental techniques (memory for sentences, eye tracking) and populations (see our paper in Cognitive Neuropsychology). Another line of investigation on the nature of concepts involves a rapid presentation of words and pictures simultaneously (and, in one variation, separated by hemisphere) attempting to uncover how features contribute to the processing of object concepts. The figure shows part of the technique we developed to understand how and when object concepts are accessed in the brain. Preliminary results from this study can be seen here. A current discussion on the theory of conceptual atomism appears in several chapters of a recent book (de Almeida & Gleitman, 2018 [link to Oxford]) and in a recent chapter ( [.pdf] [book]).
Concepts are the most elementary mental representations bearing meaning. They enter into other higher cognitive processes such as thoughts (in fact, thoughts have concepts as their constituents just like sentences have words as their constituents). But how are concepts represented in the mind/brain? How do you represent, say, the concept DOG? Is it represented as a set of features (for instance, +barks, +animal, +domesticated, +furry)? How are concepts tied into categories? One line of investigation is on the nature of concepts that denote causation—such as the meaning of the verb to KILL. Some people think that verbs are represented in the brain by a "conceptual template" such as [[x ACT] CAUSE [y BECOME [dead]]] which contains primitive predicates (e.g., ACT, CAUSE, BECOME) and argument positions, x and y, specifying the kinds of elements the verb requires to form a grammatical sentence. We are investigating this hypothesis with causative verbs, employing different types of experimental techniques (memory for sentences, eye tracking) and populations (see our paper in Cognitive Neuropsychology). Another line of investigation on the nature of concepts involves a rapid presentation of words and pictures simultaneously (and, in one variation, separated by hemisphere) attempting to uncover how features contribute to the processing of object concepts. The figure shows part of the technique we developed to understand how and when object concepts are accessed in the brain. Preliminary results from this study can be seen here. A current discussion on the theory of conceptual atomism appears in several chapters of a recent book (de Almeida & Gleitman, 2018 [link to Oxford]) and in a recent chapter ( [.pdf] [book]).
The Language-Vision Interface
During the past couple of years we've been developing a series of studies on the interaction between linguistic and visual representations during language comprehension. These studies investigate, more specifically, how verb-semantic information may constrain the domain of visual attention to referents in the scene. Two theses (Di Nardo and Van de Velde) have explored this interface, both employing dynamic scenes and different verb classes. See our recent paper in Frontiers in Psychology (de Almeida, Di Nardo, Antal & von Grünau, 2019). Results from this study show that causative verbs yield faster—but not anticipatory—saccades (eye movements) to semantically related objects in the scene compared to verbs of perception. This suggests that information from vision and language interact quickly during naturalistic linguistic and dynamic scene processing, at a central conceptual system. The figure is a simplified version of the model we propose on how language and vision interact—via conceptual system. Current studies using a similar technique involve different types of linguistic and visual materials. [see our Materials page for samples]
During the past couple of years we've been developing a series of studies on the interaction between linguistic and visual representations during language comprehension. These studies investigate, more specifically, how verb-semantic information may constrain the domain of visual attention to referents in the scene. Two theses (Di Nardo and Van de Velde) have explored this interface, both employing dynamic scenes and different verb classes. See our recent paper in Frontiers in Psychology (de Almeida, Di Nardo, Antal & von Grünau, 2019). Results from this study show that causative verbs yield faster—but not anticipatory—saccades (eye movements) to semantically related objects in the scene compared to verbs of perception. This suggests that information from vision and language interact quickly during naturalistic linguistic and dynamic scene processing, at a central conceptual system. The figure is a simplified version of the model we propose on how language and vision interact—via conceptual system. Current studies using a similar technique involve different types of linguistic and visual materials. [see our Materials page for samples]
The Nature of Metaphors
Several studies with former graduate student Carlos Roncero focus on the representation and processing of metaphorical expressions (e.g, My lawyer is a shark). The studies employ different techniques (e.g., eye-tracking, priming, judgments, corpus data) and populations (healthy young adults, Alzheimer's patients). Metaphors are said to involve an "alternative meaning", one that goes beyond what is literally said. The current thinking is that because, say, lawyers cannot really be sharks, the interpretation of a word such as 'shark' involves the access to another meaning—sometimes even directly, without first accessing what it really means (viz., some kind of fish). In a recent eye-tracking study, we have shown that a metaphorical expression takes longer to process than an expression with the same basic constituents but conveying a (literal) similarity (such as My lawyer is like a shark). Our suggestion is that metaphors are not accessed directly qua metaphors but require the computation of the actual (literal) meaning. The graph shows partial results from one of our eye-tracking studies (Ashby, Roncero, de Almeida, & Agauas, 2018 [link to article]), with total reading time on the vehicle words ('shark'). A set of norms for metaphor and simile properties has also been published recently (Roncero & de Almeida, 2015 [link to article]). A recent study has investigated the interpretation of metaphors by Alzheimer's patients (Roncero & de Almeida, 2014 [link to article]). Current related studies, with graduate students Iola Patalas and Laura Pissani, involve eye-tracking and priming techniques.
Several studies with former graduate student Carlos Roncero focus on the representation and processing of metaphorical expressions (e.g, My lawyer is a shark). The studies employ different techniques (e.g., eye-tracking, priming, judgments, corpus data) and populations (healthy young adults, Alzheimer's patients). Metaphors are said to involve an "alternative meaning", one that goes beyond what is literally said. The current thinking is that because, say, lawyers cannot really be sharks, the interpretation of a word such as 'shark' involves the access to another meaning—sometimes even directly, without first accessing what it really means (viz., some kind of fish). In a recent eye-tracking study, we have shown that a metaphorical expression takes longer to process than an expression with the same basic constituents but conveying a (literal) similarity (such as My lawyer is like a shark). Our suggestion is that metaphors are not accessed directly qua metaphors but require the computation of the actual (literal) meaning. The graph shows partial results from one of our eye-tracking studies (Ashby, Roncero, de Almeida, & Agauas, 2018 [link to article]), with total reading time on the vehicle words ('shark'). A set of norms for metaphor and simile properties has also been published recently (Roncero & de Almeida, 2015 [link to article]). A recent study has investigated the interpretation of metaphors by Alzheimer's patients (Roncero & de Almeida, 2014 [link to article]). Current related studies, with graduate students Iola Patalas and Laura Pissani, involve eye-tracking and priming techniques.
Structure and Meaning in Word Recognition
How are words processed and represented in the brain? Regarding the structure of words, some people believe that we recognize and process words in full form while others believe that we recognize words by first breaking down their constituent morphemes (e.g., when you see a word such as remaking, you first decompose it into re, make, and ing). Together with Gary Libben (Brock University), we have been working on the nature of the lexical recognition and access codes employing different types of morphologically complex words. Our projects include experiments with inflected (e.g., barking) and compound (e.g., keyboard) words, as well as morphologically complex ambiguous words (e.g., undoable), and employ several experimental techniques (eye-tracking, priming, RSVP, etc.). The figure depicts a recent technique we developed employing anaglyph glasses to understand the early moments of word recognition (LH: left hemisphere; RH: right hemisphere) via different visual pathways. In this technique, words are split into legal (FOOT+BALL — as shown) and illegal (FOO+TBALL) constituents to understand how the different hemispheres brain decodes early word constituents as well as how the word constituents are integrated [link to article ].
How are words processed and represented in the brain? Regarding the structure of words, some people believe that we recognize and process words in full form while others believe that we recognize words by first breaking down their constituent morphemes (e.g., when you see a word such as remaking, you first decompose it into re, make, and ing). Together with Gary Libben (Brock University), we have been working on the nature of the lexical recognition and access codes employing different types of morphologically complex words. Our projects include experiments with inflected (e.g., barking) and compound (e.g., keyboard) words, as well as morphologically complex ambiguous words (e.g., undoable), and employ several experimental techniques (eye-tracking, priming, RSVP, etc.). The figure depicts a recent technique we developed employing anaglyph glasses to understand the early moments of word recognition (LH: left hemisphere; RH: right hemisphere) via different visual pathways. In this technique, words are split into legal (FOOT+BALL — as shown) and illegal (FOO+TBALL) constituents to understand how the different hemispheres brain decodes early word constituents as well as how the word constituents are integrated [link to article ].
Semantic Deficits in Alzheimer's Disease
Cases of category-specfic deficits (i.e., conceptual or semantic deficits caused by brain injuries or diseases) have been taken as evidence for the way concepts are represented and organized in the brain. Research on this topic has involved empirical investigation of verb-semantic deficits in patients with Alzheimer's disease (Manouilidou, et al., 2009; Manouilidou & de Almeida, 2009); dissociation between verb classes employing dynamic scenes depicting different events and states (de Almeida, Mobayyen, Antal, Kehayia, Nair, & Schwartz, 2021 [proofs | .pdf]); and deficits in metaphor interpretation (Roncero & de Almeida, 2015; see above). The figure illustrates results from Manouilidou et al. (2009), showing that Alzheimer's patients have difficulty differentiating between two sub-classes of psychological verbs (e.g., fear, frighten), which don't have agents but experiencers (download .pdf ). Current related studies also investigate deficits in patients with aphasia.
Cases of category-specfic deficits (i.e., conceptual or semantic deficits caused by brain injuries or diseases) have been taken as evidence for the way concepts are represented and organized in the brain. Research on this topic has involved empirical investigation of verb-semantic deficits in patients with Alzheimer's disease (Manouilidou, et al., 2009; Manouilidou & de Almeida, 2009); dissociation between verb classes employing dynamic scenes depicting different events and states (de Almeida, Mobayyen, Antal, Kehayia, Nair, & Schwartz, 2021 [proofs | .pdf]); and deficits in metaphor interpretation (Roncero & de Almeida, 2015; see above). The figure illustrates results from Manouilidou et al. (2009), showing that Alzheimer's patients have difficulty differentiating between two sub-classes of psychological verbs (e.g., fear, frighten), which don't have agents but experiencers (download .pdf ). Current related studies also investigate deficits in patients with aphasia.