Research Article: The dominance of introspective measures and what this implies: The example of environmental attitude

Date Published: February 15, 2018

Publisher: Public Library of Science

Author(s): Siegmar Otto, Ulf Kröhne, David Richter, Mohammad Shahid.


The behavioral sciences, including most of psychology, seek to explain and predict behavior with the help of theories and models that involve concepts (e.g., attitudes) that are subsequently translated into measures. Currently, some subdisciplines such as social psychology focus almost exclusively on measures that demand reflection or even introspection when administered to persons. We argue that such a focus hinders progress in explaining behavior. One major reason is that such an exclusive focus on reflections results in common method bias, which then produces spurious relations, or in other words, low discriminant validity. Without the valid measurement of theoretical concepts, theoretical assumptions cannot be tested, and hence, theory development will be hampered. We argue that the use of a greater variety of methods would reduce these problems and would in turn foster theory building. Using a representative sample of N = 472 participants (age: M = 51.0, SD = 17.7; 54% female), we compared the validity of a classical introspective attitude measure (i.e., the New Ecological Paradigm) with that of an alternative attitude measure (i.e., the General Ecological Behavior scale). The latter measure, which was based on self-reported behavior, showed substantially better validity that we argue could aid theory development.

Partial Text

The central aim of behavioral sciences such as social psychology is to explain and predict behavior [1]. Thus, behavioral sciences have to produce knowledge (i.e., develop and test theories and models) that can be applied to explain and predict behavior. The next step in this quest is to develop consistent and parsimonious theories with testable models that consist of measurable concepts (e.g., attitudes, intentions, control beliefs), and finally, to explain or predict behavior. In order to increase explanatory power, researchers develop theories with the help of additional constructs that explain how attitudes influence behavior (for a broader overview on attitude research see [2]). However, the increases in the amount of variance that can be explained with each additional concept are often modest, and theory building suffers from high interdependencies between explanatory concepts, or in other words, low discriminant validity (e.g., [3]). These interdependencies could, of course, result from true correlations between the concepts. But what if they are nothing more than methodological artefacts that obstruct a clear view of the true interdependencies of the underlying theoretical constructs?

Ethical permission is provided by the Scientific Advisory Board of DIW Berlin.

The mean for the NEP score was M = 3.81 (scale range 1–5; SD = 0.51), which replicated the common finding of a strong endorsement of the NEP [10]. The mean for the GEB was M = 0.07, SD = 0.80 (Logits), and the NEP was positively correlated with the GEB (r = 0.19, p < 0.001). The NEP was positively correlated with only two and even negatively correlated with one of the five ecological behaviors and intentions (see Table 1). The GEB was correlated with four of the five behaviors and intentions in the expected way. Four of the five convergent cognitive statements about life aspects were positively related to the NEP, whereas all five of them were positively related to the GEB. Please note that correlations below .3 are usually interpreted as small effects, whereas correlations of .3 or higher are considered good [33]. Table 1 shows that only two of the convergent validation items had correlations with the GEB above .3. However, the magnitudes of the correlations between the predictors of ecological behavior such as the NEP and GEB with actual behavior were usually in the lower ranges. One reason is the broad and heterogeneous class of behaviors that are considered ecological behaviors. Thus, within the domain of ecological behavior, the correlations of the GEB as a general measure of environmental attitude with the convergent validation behaviors were relatively substantial [13]. Our exemplary comparison between an introspective attitude measure (i.e., the NEP) and a behavioral attitude measure (i.e., the GEB) revealed two major weaknesses of the introspective attitude measure. First, our data showed low convergent validity for the NEP, thus replicating the commonly identified, often discussed, and never solved attitude-behavior gap that has been linked to classical introspective attitude measures. Second, but most important for this study, our results indicated low discriminant validity for the NEP, thus referring to a common method bias that has been shown in particular for introspective attitude measures [4]. Even though our competitive behavior-based attitude measure (i.e., the GEB) also showed some weaknesses because it did not show a perfect validation pattern, it nevertheless showed significantly better convergent and discriminant validity than the introspective attitude measure. By employing a widely used introspective attitude measure (i.e., the NEP), we were able to provide a solid replication of the problems (attitude-behavior gap and common method variance) involved in the use of introspective measures in general. Beyond the methodologically undesirable focus on just one measurement approach (e.g., [4, 8]), the corroborated gap between introspective attitude measures and behavior remains an unresolved problem for introspective attitude measures [34].   Source:


0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments