Disclaimer: There is a ridiculous, David Foster Wallace-esque number of hyperlinks and footnotes in this blog entry. The main text is meant to be a standalone piece while the links and footnotes are accessory bits of related information that readers can check out at their own pace if they so choose (I would highly recommend it). This blog entry is basically a brain dump of everything I’ve always wanted to talk about but never had the proper forum to do so (this may not even be the right forum…). So without further ado, enjoy!
Exponential growth seems to be the rule when it comes to all things human-related, and the intellectual field of paleoanthropology is no exception. The number of PhDs awarded each year is increasing nonlinearly and to make room in our relatively small discipline, new researchers are becoming more specialized, discovering new topics on the outskirts, or squeezing themselves in between fields under the label “interdisciplinary.” This is a BIG problem for many reasons[1], but I will focus on a particular one for this blog. In addition to having a cornucopia of researchers, academia incentivizes high publication output at all career stages, so there are a LOT of diverse topics, findings, and literature (see graph) to digest, and even more comes out every year. It is overwhelming and impossible to keep up with.
Bar plot produced from Web of Science showing the exponential growth of yearly publications from 1950-2015 relating to the keywords “hominin” or “paleoanthropology.”
This is not an insignificant issue because as Nate Silver has said, “We face danger whenever information growth outpaces our understanding of how to process it.” More specifically, in order to process enormous amounts of information, we as humans cherry-pick data and “simplify the world in ways that confirm our biases”[2,3]. Perhaps it is no wonder then that in the age of online news and 24-hour news networks, U.S. citizens are more politically hyper-partisan than ever. Therefore, we need to be actively conscious of our own and others’ cognitive biases[4] when sifting through all the data, analyses, and publications that exist in the digital ether.
But, what can we do about the noisy information overload itself? I present two different philosophies on how to mitigate the noise. For starters, we as a field probably have too many ideas which we keep around and are unwilling (or unable) to decisively reject[5]. Instead, we should be practicing strong inference (sensu Platt 1964) or at least something approaching that idea. Strong inference, which is similar to Karl Popper’s hypothetico-deductive method, advocates developing mutually exclusive working hypotheses to explain an observation. The multiple hypotheses are then iteratively falsified and pared down until the “correct” one remains[6,7]. Of course, this may be too idealized since biological patterns are complex and may have countless causes which are non-independent (Quinn & Dunham 1983)[8], but I think this is a nice ideal to strive towards. This means we should be competing multiple hypotheses[9] against each other instead of testing trivial null hypotheses one by one[10] and accumulating their respective alternative hypotheses[11].
Regarding the second philosophy, we must rely more heavily on theory and practice more prediction. Paleoanthropologists, by their nature, are empiricists, so we always approach problems and questions from the perspective of our study objects and data. This means when it comes time to discuss our analyses, a common practice is to look at the results first and then post hoc fit an explanation to them[12]. The problem here is that paleoanthropological data by their nature are very noisy with lots of wiggle room and large error bars. Combine that with the human mind’s predilection to detect patterns in anything, and the result is a litany of superficially sensible explanations for an observed phenomenon[13]. Instead, our workflow should proceed from the opposite direction: theory deductively produces predictions, which are then independently tested with empirical data[14]. As philosopher of science Imre Lakatos has said: novel, risky predictions that are precise enough to be wrong are perhaps the sine qua non for making sense of noise and advancing a scientific field[15]. Discard the predictions that don’t pass muster (thereby reducing noise), and keep the ones that do. What we are left with is a framework of well-tested ideas that independently explain our data. As we continue to build on our body of theory, the ultimate goal is to generate as many successful predictions from as few theories as possible. These unified theories represent a common framework for making sense of all the noise and uniting seemingly disparate subfields, creating a robust, synthetic view of human origins[16].
The two philosophies I have presented are not mutually exclusive and can even be thought of as complementary or a continuum. There will always be some difficulties in applying these two ideas verbatim to one’s study system, but I think they are excellent ideals to strive towards in order to make our science more efficient, i.e., focusing on the big, risky questions rather than the atomized, easy-to-answer ones. I would be curious to hear what others have to say on this issue. I borrow a lot of my thinking from macroecology and quantitative paleobiology, where data are abundant and questions are asked at large enough scales that emergent, less contingent properties become apparent. Perhaps paleoanthropology is just too data-poor[17], and the taxonomic[18], temporal[19], and spatial[20] scales are just too small for robust, predictable patterns to emerge. Perhaps noise is just a way of life in paleoanthropology, and all we can do is describe and piece the evidence together from scratch. I honestly don’t think this is true, especially as more data and centralized databases begin to become more common, but only time will tell. I think as long as we remember the teachings of Platt and Lakatos, paleoanthropology will one day understand the complete picture of how our forebears lived and evolved instead of just random snippets.
Acknowledgments: A lot of the ideas presented in this blog entry are not original and are heavily influenced by ecologist, Brian McGill. I encourage everyone to read his papers, or at least his blog over at Dynamic Ecology. And thanks to Jordan Miller for her insightful input and comments.
1 One of the more obvious issues that haunts graduate students and post-docs is what to do with all these new researchers? Unless faculty members retire or new academic positions are created at the same rate that PhDs are anointed, there will be a lot of jobless doctors in the future. This issue is exacerbated in paleoanthropology because there are not many non-academic options for someone holding a degree in “human origins studies.”
2 Quote from same Nate Silver book.
3 This is also known as confirmation bias. This definitely happens in paleoanthropology research, especially with researchers who have pet hypotheses (as we all know, there are many). In fact, one should increase the burden of proof benchmark as a more stringent test when reading papers by confirmation-biased researchers. Perhaps the rule of thumb should be a researcher only gets to publish a maximum of three (or some other arbitrary number) papers on his/her pet topic. Let other more independent-minded researchers offer more objective tests. And if you think confirmation bias doesn’t apply to you or many of the saintly researchers in our field, read this article on “researcher degrees of freedom,” and remember that biases can be (and most likely are) subconscious.
4 Our way of thinking is the machinery by which we do science. If that machinery is faulty in any way, our science suffers. Therefore, always be mindful and cautious of the way your mind works, especially when it comes to fast thinking and heuristics (although Daniel Kahneman, who specializes in cognitive bias research, has said that simple self-awareness is not enough to overcome innate biases, but I suppose being self-aware is better than nothing). But, don’t do it to the point of debilitating solipsism. It’s a tough balance to strike and one I certainly haven’t successfully navigated yet. But don’t just take it from me; listen to Bruce Springsteen.
5 Hypotheses for what drove bipedalism are a good example. Such hypotheses include more energy-efficient locomotion over long distances, less overall surface area being exposed to the sun, the ability to see over tall grasses, freeing up the hands for carrying and transporting things, freeing up the hands for tool-making, reaching up to grab fruit from trees, sexual signaling, etc. Or, hypotheses on what Acheulean handaxes were used for: butchery of large animals, cutting wood, digging in soil, using them as discus-like projectile weapons, using them as a source of flakes, symbolic burial practices, sexual selection (paleoanthropologists sure love this one), etc. Long-lived ideas that are no longer correct and have outlived their usefulness have been appropriately called “zombie ideas” in economics and the ecology blogosphere.
6 This means the “correct” hypothesis must be included in your original pool of multiple hypotheses, which may not necessarily be true.
7 This means being “wrong” is important for advancing science, so negative results should be published! Not to mention, not doing so results in publication bias, which can lead to validation issues and acceptance of faulty ideas.
8 Platt advocates decisive rejection of one of more competing, mutually exclusive hypotheses. This can be a problem if the pattern in question is caused by multiple processes that interact, as is usually the case in biology. An alternative approach, and one that Quinn & Dunham advocate, is to weigh the relative importance or contribution of multiple processes on an observed outcome. Either way, it is important to properly frame your research question (a working knowledge of philosophy of science helps here) which then suggests the most appropriate statistical test(s). This is why in my mind, research, philosophy of science, history of science (for understanding theory, researcher biases, scientific paradigms, etc.), and statistics are all inextricably linked and cannot survive without the other. In other words, it is important to be knowledgeable in all of these!
9 This also means there is no one pet hypothesis for researchers to glom onto.
10 “The hallmark of empirical progress is not trivial verifications: Popper is right that there are millions of them. It is no success for Newtonian theory that stones, when dropped, fall towards the earth, no matter how often this is repeated.” From here.
11 Perhaps we can be considered “scientific hoarders.”
12 As ecologist, Brian McGill, has said in his blog, “Ultimately, if all we’re doing is post hoc explanation, it is at best a deeply diminished form of science.”
13 There are many hypotheses for why bipedalism is adaptive in open grassland environments. I bet if the data showed bipedalism arose in a closed environment (some would actually argue this is the case), the paleoanthropological community would be able to come up with just as many hypotheses stating why bipedalism is adaptive in woodland environments.
14 The only researchers in paleoanthropology that I know who have done this are Blumenschine and Peters (Peters & Blumenschine 1995; Blumenschine & Peters 1998), where they a priori developed a model for how hominins would have used the landscape based on behavioral ecological theory. From this followed predictions as to what one would expect in the fossil/archaeological record based on different lines of evidence, which were then tested empirically (Blumenschine et al. 2012; Blumenschine et al. 2012). This is a fine example of what I’m talking about, although they rely primarily on verbal models which provide “soft,” non-numerical predictions with wiggle room (e.g., “X is greater than Y”). Any time there’s wiggle room, there’s flexibility in which data and analyses can be fiddled with to match one’s predictions or confirm one’s bias (remember researcher degrees of freedom?). More precise numerical predictions based on mathematical or simulation models would be preferable (e.g., “X is at least twice as big as Y.” Or even better, “X is 3.75 times bigger than Y”). Or if precise numerical predictions aren’t possible for whatever reason, one can practice “dipswitch theory testing” (FYI: this is a dipswitch), where an easy-to-support qualitative prediction is supplemented with additional independent side predictions (in this case, more is better!). The idea here is that one may find support for a qualitative prediction simply by chance (or data futzing), but it is much harder to do so when there are many simultaneous qualitative predictions.
15 Though predictions are an important part of Popper’s philosophy, they take center stage in the philosophy of Lakatos, who was actually Popper’s student. Lakatos in a way combined the philosophies of Popper and Thomas Kuhn. He developed the idea of “research programmes” (analogous to Kuhn’s scientific paradigms) which are surrounded by a “belt” of falsifiable hypotheses which can be continually ad hoc adjusted to protect the research programme (contra Popper, research programmes aren’t discarded at the first sign of falsification). This explains why it can take quite some time for certain scientific ideas to be overturned even in the face of contradictory evidence, especially when social power dynamics are involved (for a paleoanthropological example, read Bones of Contention). However, if a research programme cannot make novel predictions which are later confirmed and must keep relying on ad hoc explanations, the research programme can be superseded by a more effective, “correct” one. So, “dramatic, unexpected, stunning predictions” are the key method by which research programmes are advanced and iteratively replaced.
16 This will require a LOT of data. Therefore, we need to continue producing new data but organize them into centralized, public databases, so they’re not just random noise but something that can be called upon to test our risky predictions. If we’re looking to unify paleoanthropology with a small but robust body of theory, we need standardized data from multiple locales and time periods to test the generality of our theory and predictions.
17 After all, hominins are really rare.
18 Hominins belong to the Tribe Hominini, a small, species-poor clade. Because of this, we don’t have a well agreed upon phylogeny, which makes inferring evolutionary process and pattern tough.
19 The hominin clade is only 6-7 million years old, and data are not uniformly distributed across that time period (not a lot of hominin specimens early on but many more later).
20 Though hominins occupy a relatively large geographic expanse even at 1.8 million years ago (Eastern & Southern Africa, the Caucasus, and East & Southeast Asia), sites are sparsely and unevenly distributed within that expanse.