Since Messerli (2012) first documented this correlation between chocolate consumption and Nobel laureates in different countries in 2012, others have discovered other, similar, correlations, such as milk consumption or GDP per capita. In truth, eating chocolate is almost certainly not going to make you a Nobel laureate. This nugget does, however, reveal several important insights into how to think about causation.

First, the presence of a correlation is not sufficient evidence to suggest causation (the old adage that "correlation is not causation").

Second, that by thinking about potential explanations for a correlation, we can land on causal questions to investigate.

1.3 A neurophysiology example

Let's take a closer look at a neurophysiological example. Roitman & Shadlen (2002) sought to investigate the neural regions responsible for decision-making. They hypothesized that neural activity in the Lateral Intraparietal (LIP) area was involved, and so conducted the following experiment.

 Monkeys were trained to observe moving dots on a screen that moved to the left or right, and to respond by moving their eyes to the left or right depending on the direction of motion. At the same time, neural activity in the LIP area was recorded.

The results indicated the following:

• Neurons in the LIP area fire approximately 100 msec before eye movement.
• Activity in the LIP area is sensitive to the stimulus strength. In trials where the motion of the dots was strong and easily visible, there was more firing compared to trials with weaker motion.
• Different neurons fire if the subsequent eye movement was to the left vs. to the right.

Consequently, Roitman & Shadlen reported

"“Because the time course and level of activity depend on the strength of random-dot motion, it is likely that LIP neurons not only represent the planned eye movement response but also the visual information on which the developing decision is based—in other words, a decision variable.”

The phrase "decision variable" suggests that LIP activity is causal to the eye movement, but is this actually the case?

Let's reframe this in the form of causal questions we used earlier:

Now that we've specified the causal question more explicitly, we can identify that in the study, the manipulation was the direction and strength of the moving dot stimulus. Both LIP activity and eye movement were observed, not manipulated.

In other words, this was an indirect experiment. There was no direct manipulation of the independent variable (LIP activity) of interest.

1.4 Manipulating neural activity

What would it look like to do a direct manipulation of neural activity? Luckily, that study has already been done!

Katz et al. (2016) trained monkeys on the same task as Roitman & Shadlen, but their study contained one key difference—they infused specific brain areas with muscimol to cause (temporary) neural inactivation.

This manipulation enables a direct test of the causal hypothesis:

• If LIP activity is necessary for eye movement…
• then inactivating the LIP area should stop eye movement.

But Katz et al. find that eye movement still occurs as normal even without activity in the LIP area. This suggests that eye movement is not caused by the LIP area.

As an additional check, Katz et al. also inactivated the MT (middle temporal) area, a region known to be important for visual processing of motion. When doing so, the monkeys show impaired performance on the task, indicating that muscimol is effective at inactivating neural regions in a way that affects task performance.

2.1 Causality is a spectrum

There isn't a single factor that can separate research into "causal" and "non-causal". Instead, we need to evaluate causality in terms of our confidence about the available evidence, which could arise from a single study or a collection of studies.

In other words, we should reframe our investigation into causality as:

Based on the evidence, how confident are we that X causes Y?

This lesson is about 2 attributes of studies that affect the strength of causal evidence:

• directness
• randomization.

The best way to test a causal question is with a direct experiment.

2.2 What makes an experiment direct?

When you get ready to carry out your study that tests the relationship between an independent variable and a dependent variable , you will make choices such as:

• how to implement your intervention
• how to measure outcomes

These choices are often influenced by feasibility and ease of access (what variables can be measured or manipulated and the difficulty involved), but can have major impacts on the rigor of your experiment and the resulting evidence.

2.3 How direct is the intervention?

The directness of the intervention captures the extent to which your method of implementing the intervention is aligned with the independent variable of your causal question. 

For example, in the Roitman et al. study, the causal question was:

"how does neural activity in the LIP area affect eye movement decisions?"

However, the manipulation in the study was to change the type of visual stimuli shown to the subjects, and both the activation of LIP neurons and behavior were observed as outcomes.

This is an example of an indirect manipulation of the independent variable.

When designing the experiment to test this causal question, you will need to make choices about how to manipulate the neural activity in the LIP area.

• For example, you might choose to manipulate neural activity indirectly, by introducing a stimulus or engaging the subject in behavior that results in LIP neural activity.
• You might also choose to manipulate neural activity directly, such as by using transcranial magnetic stimulation (TMS) to induce neural activity directly.

Any of these choices has the potential to introduce complications to your experiment that can negatively impact your ability to draw causal conclusions!

The indirect stimulation approach adds additional elements to the intervention beyond LIP neural activity, and if any of these cause or impair the observed outcome of eye movements, that will muddy the results.

Stimulating neural activity by TMS may be imprecise and affect activity in other areas. It may also cause a pattern of neural activity that is different from what would happen in natural settings; such that a realistic pattern of activity leads to eye movement, but TMS-stimulated activity does not.

2.4 How direct is the outcome measure?

The directness of the outcome measure captures the extent to which what you measure is aligned with the dependent variable of your causal question.

For example, suppose that your hypothesis involves improvements to memory. What measures will you use to measure memory? If you use a memory test, there may be ways that your subjects can perform well without recalling the actual content you desire (e.g. multiple choice questions that are easy to guess on). On the other hand there may also be factors that negatively influence performance and are unrelated to memory (e.g. stress in a test-taking environment).

If your research interest is in the neurobiological aspects of memory formation, then you may consider imaging or chemical methods to measure the biological processes the produce recall.

Although it can be tempting to measure everything that is relevant, that could add unwanted cost and expense to the experiment; and introduce complexity in data analysis later—what if one measure shows a change but another measure does not? (We won't get into the details here, but do check out the unit on multiplicity and statistical analysis planning for more guidance on data analyses.)

These aspects of experimental design are all examples of construct validity. 

Construct validity refers to how well what you have implemented or measured represent variables or concepts that are difficult to ascertain directly.

Some aspects of construct validity can be verified through manipulation checks or positive controls. See lesson 5 of our unit on Controls for more on this subject!

2.5 Pathway directness

In addition to how you operationalize the independent and dependent variables in your study, you may also want to check that your study does a thorough job of testing that the relationship between the variables is occurring through the causal pathway that you theorize.

In other words, in addition to the hypothesis that the independent variable influences the dependent variable, you probably have a hypothesis for HOW the independent variable influences the dependent variable.

In other words, in addition to the hypothesis that the independent variable influences the dependent variable, 

For example, suppose that you are conducting an experiment to test the hypothesis that creating flash cards improves exam scores. You might theorize the following mediator variables that connect the independent variable (creating flash cards) and the dependent variable (memory):

• To create flash cards, ravens identify key concepts,
• creating memories of these concepts, 
• which are recalled at exam time.

How can we verify if these mediators are actually operating? 

A manipulation check will test whether an intervention operated as intended.

• Did a raven actually create the flash cards?
• Did they print them out from an online source?

Depending on your theorized causal pathway, you may want one or more manipulation checks as part of the experiment. Including manipulation checks can distinguish between different pathways through which the independent variable may be influencing the dependent variable. 

For example, perhaps the act of making flash cards is more active than rewatching lectures, and so merely doing an active form of studying improves exam scores. We may also want to watch out for other processes that interfere with our outcomes—what if the exam is so easy that all ravens ace it; such that no difference is detected even though one study method did actually improve memory.