In the last post, I discussed the trade-offs between false negatives (guilty people who are treated as innocent) and false positives (innocent people who are treated as guilty). By changing where the line that represents the standard of evidence required for pulling someone over, executing a search, making an arrest or using force is drawn, you can shift the balance between false negatives and false positives. I also touched on a recent study that looked at where police departments actually draw that line to see if it is uniform across racial and ethnic groups. In this post, I will take a closer look at that study and do a bit of data exploration of my own to see if there is evidence that African Americans are treated differently.
What do equality and bias look like?
To set the stage, let’s talk about equality. What does equality look like in the context of policing? The first things you might examine are the raw numbers. How many people of each race are stopped or arrested? You would immediately notice that a lot more white people get stopped than people of any other group. The same applies to arrests and police shootings. Now some people might be tempted to stop there and say, “See, if anything, white people are the ones being discriminated against.” However, that misses an important fact: there are a lot more white people than black people (over 4 times as many) in the US! If the number of people stopped were the same, that would actually mean that each individual black person was far more likely to be stopped than each white person.
To illustrate why this is the case, suppose your friend has a (huge!) bag with 10000 skittles, 8000 of which are red and 2000 which are green. Suppose your friend were to mix up the bag and pick out a random candy to give to you, the odds that it would be green are 20%. Now suppose that your friend does this 100 times and you end up with 50 greens and 50 reds. What would you conclude? The probability of getting that many greens by random chance is less than 0.000001. (Note: even if 40% of the skittles were green, you would get 50 or more greens out of 100 less than 3% of the time). So, either you have just witnessed an incredibly unlikely event, or your friend is favoring greens in some way. Maybe the greens are larger or stickier and this was unintentional, or maybe your friend doesn’t like the green ones and he is actively trying to get rid of them. We cannot use statistics to prove intent, but we can use it to provide evidence that some sort of bias was present. On the other hand, if your friend gave you 22 greens and 78 reds, that would be entirely plausible to have occurred by random chance (you get 22 or more greens around 35% of the time by random chance), so that would not be evidence of bias. The takeaway message is that, in the absence of some hidden bias, the proportions within the sample should be close to (but not necessarily equal to) the proportion within the larger population (i.e. the bag of skittles).
In the context of policing, we might expect that given that there are over 4 times as white people as black people in the population, then there should be a similar proportion when we look at stops, arrests, etc.
Are Michigan state troopers biased?
I decided to look at my home state of Michigan and, thanks to the Stanford Open Policing Project, I was able to download records from over 800,000 stops by state troopers between 2001 and 2016. Of these, the race of the person stopped was recorded for 347,328. It is not entirely clear why the other 452,974 races were missing (perhaps the officers could not tell), but I will assume that they were missing at random and look for patterns in the rest of the data.
The first thing I looked at was the number of records of each type. In Michigan, around 74.8% of the population is white and 13.6% is black. This means that we would expect around 5.5 times as many stops of white people as black people. However, in the data the ratio of white stops to black stops was 4.7. The odds of this happening by chance are astronomically small (p<0.00001) which suggests that black people were more likely to be stopped by police than white people. The next thing I looked at was arrests. When white people got stopped, they were arrested 0.9% of the time. Black people on the other hand were arrested 1.4% of the time which means that they were 56% more likely to be arrested when they got stopped. Again, the probability of this happening by chance is tiny (p<0.00001).
At this point we might be tempted to stop and conclude that policing in Michigan is biased against African Americans, but before we jump to that conclusion, there are two things worth noting. First, this sort of bias was not evident in every category I looked at: white people got tickets 76% of the time, but black people got tickets only 74.2% of the time. So, perhaps not everything favors white people. The second (and more important) thing to recognize is that this tells us that black people were more likely to be stopped and arrested, but it does not tell us why. Perhaps they were more likely to have broken the law? Unfortunately, the data doesn’t give a ton of information about why they were stopped and the information it has is unwieldy.
If we want to determine whether this is evidence of unfairness, then we need to be able to focus in on situations where we can actually tell what led to the interaction in the first place. Thankfully, the dataset contained information about the drivers clocked speed, the posted speed limit, and the speed for which they were ticketed. I don’t know if this is a nationwide phenomenon, but police officers in Michigan will often give you a ticket for a lower speed than you were actually going which results in a lighter fine. As far as I know, there are no guidelines for how to do this. It is simply a matter of the officer’s judgment. The nice thing about this data is that it allows us to tell how serious the infraction was and therefore, how lenient the punishment was.
So, did cops go easier on white people or black people? White people were stopped for speeding 79728 times and they were cited for a speed that was on average 9.72 mph lower than their actual speed which amounts to a reduction of 12.5%. Black people were stopped for speeding 10465 times and they were cited for a speed that was on average 8.89 mph lower than their actual speed which amounts to a reduction of 10.7%. As before, these differences are statistically significant (p<.000001) meaning that they cannot be explained by random chance. Having an extra 1 mph taken off your ticketed speed may not sound like a big deal, but that is equivalent to one in five drivers having their ticketed speed reduced by an extra five miles per hour. Under Michigan’s tiered ticketing system, which uses 5 mph increments, those types of reductions could mean lower fines (by $10 to $25) as well as fewer points on the driver’s record (which affect insurance rates and which if accumulated can lead to a suspension of driving privileges). So, this suggests that Michigan state troopers tended to go easier on white drivers than black drivers.
Findings from the Stanford Open Policing Project
This analysis raises a number of questions. Is this phenomenon unique to Michigan? Does it vary from one department to the next? Enter: The Stanford Open Policing Project. They collected data from around the country encompassing around 100 million police stops and analyzed it to see if there was evidence of bias in policing. They specifically tried to infer the standard of evidence that police officers used when determining whether to stop and search drivers.
They first focused on searches. They found that black drivers were more likely to be stopped than white drivers. Next they looked at search rates and they found that black drivers were more likely to be searched than white drivers. This could be evidence of bias or it could be that they were more likely to commit violations or to carry contraband. Then they looked at how often those searches turned up contraband. They found that despite being searched more often, police officers found contraband on black drivers slightly less often (29% of the time) than their white peers (32% of the time). The fact that these two quantities were relatively close makes the evidence of bias inconclusive. It also suggests that to some extent the higher search rates may have been warranted.
To address this, the research group at Stanford developed a test called the threshold test (which you can read more about in the publications posted on the group’s website). They suggested that officers would use a threshold on the standard of evidence when determining whether to search a given driver. These thresholds might be different for different races and police departments and they cannot be observed directly. However, we might be able to infer them from the data.
[Warning: Mathematical Digression!] To do this they treated each stop as an independent event with some random signal, p, representing the suspiciousness of the subject. To account for the fact that crime rates could vary across races and cities they let the average suspiciousness (and the variability of the suspiciousness) depend on both the subject’s race and the police department involved. They then compared this signal to the threshold for that particular race and police department. If the suspiciousness signal exceeded the threshold, then they assumed that a search would take place.
This probabilistic description of searches (which I find to be quite reasonable) includes a number of unknown parameters describing:
1. The average suspiciousness of each race
2. The variability of the suspiciousness within each race
3. The average thresholds used by departments for each race
4. The variability of the thresholds used by departments for each race
They attempted to infer those unknowns from the data. To start they used estimates that assumed no racial bias in either the suspiciousness or the thresholds. They then used a process known as Bayesian Inference to update their estimates in response to the observed data to find the most plausible ranges of estimates for each of the parameters. [End Mathematical Digression.]
They found that black (and Hispanic) subjects were indeed more likely to carry contraband in the first place (see below for an explanation of how this is possible), but this could not fully explain the higher search rates. Instead, they found that the average threshold of evidence required before executing a search was substantially higher (over 2 times higher) for white drivers than for black and Hispanic drivers. In other words, police officers were willing to search minority drivers based on half as much evidence as their white counterparts. This meant that more innocent minority drivers were searched and as a result, the success rates of those searches were lower than for white drivers. Note that if the officers used the same thresholds for all races, they still would have searched black drivers more often due to their higher average probability of carrying contraband. However, they didn’t stop with just searching them more often, they took it a step further and searched them based on less evidence. I don’t see any way to interpret that as anything other than unfair.
In a related study, they looked out how often black drivers were stopped when it was light compared to when it was dark. The idea was that when it was dark, police officers would be forced to make decisions based on drivers’ behavior rather than their race. They found that the percentage of black drivers who were stopped was significantly lower after dark even after accounting for other factors like time of day. This suggests that the appearance of dark skin likely contributed to some of the decisions to stop drivers in the first place. You can find more details about their analyses here.
Conclusions
In this post, I focused on stops, speeding tickets and searches, and found evidence of bias across the board. I focused on those particular areas because there was lots of available data. However, if discrimination is prevalent in those areas, why would we expect the use of force to be any different? And even if police officers don’t have a lower threshold of evidence for the use of force, the lower threshold of evidence used to justify stopping and searching minorities means more opportunities for confrontations to escalate into violence.
When I started looking into this issue a few weeks ago, I have to confess that part of me was hoping to find that the concerns were overblown. Part of this stems from the (selfish) idea that if other people are at an unfair disadvantage, then it cheapens any success that I achieve in life. Part of this stems from the fact that I want to think the best of people and it is disconcerting to have your faith in institutions that you have long trusted shaken. And part of this stems from the fact that many people in my social circles and on my side of the political spectrum are skeptical of the extent to which racism is a significant issue in policing. It is troubling to think that so many people that I know and trust could be wrong. However, if I am honest with myself, then I should be just as troubled by the possibility that by denying the existence of the problem, I would be dismissing the negative experience of millions of black Americans as imagined. That is why I actually tried to look at the evidence as objectively as possible. And, I have to say, I was floored by just how damning the evidence is. I don’t see any way to avoid the conclusion that black Americans are policed differently than white Americans. These biases may not always be overt or intentional, but they are widespread and severe enough to put African Americans at significant disadvantage. I have to confess that if I shared their experience, I too would find it difficult to trust the police.
When you work with data, particularly messy data, there are always some caveats. In datasets like these, some of the data is missing and some of the data is incomplete. So, if you want to make the case that the data is flawed and that this does not prove the existence of racial bias in policing, then you are perfectly entitled to do that. But the reality is that when you look at the evidence we do have, the fingerprints of discrimination against minorities and particularly African Americans and Hispanics are everywhere. You don’t have to look very hard to find it. And sure, you can try to find some technicality to argue that the evidence is inconclusive, but at that point you are just looking for reasons to escape the reality that discrimination in policing is a serious problem.
PS. The table in the cover image comes from a paper called “THE PROBLEM OF INFRA-MARGINALITY IN OUTCOME TESTS FOR DISCRIMINATION” in the Annals of Applied Statistics from 2017.
PSS. If you were confused by the statement that black drivers were both more likely to carry contraband and that searches of black drivers were less likely to turn up contraband, consider the following example:
In a small town, there are 100 black drivers, 20 of which carry contraband, and 100 white drivers, 10 of which carry contraband. Now suppose that police search 50 black drivers (20 guilty and 30 innocent) and 20 white drivers (10 guilty and 10 innocent). Notice that when the black drivers are searched, police will find contraband 40% of the time, but when white drivers are searched, they will find contraband 50% of the time. Therefore, the following statements can be true simultaneously:
- Black drivers had contraband more often (a larger percentage of the time).
- Police officers searched black drivers more often (a larger percentage of the time).
- Those searches turned up contraband less often (a smaller percentage of the time).
This suggests that the central question should not be about who is searched most often or even whether particular groups are searched disproportionately often, but rather whether those searches are actually justified. The evidence suggests that many of those searches are not.
PSSS. This is not to say that all or even most cops are bad. However, it does suggest that they are human. Police work is far from the only type of work that has a discrimination problem. Discriminatory practices have been documented in many other contexts as well (for example, in hiring: https://www.pnas.org/content/early/2017/09/11/1706255114). Some of this can be attributed to overt racism, but much of it can be explained by subtle biases that people may not even be aware of. As I discussed in some of my earlier posts, we all have a tendency to make generalizations about groups of people and sometimes those generalizations cause us to make decisions that disadvantage others. I share these findings not to denigrate the police or to suggest that this is evidence of widespread corruption, but to show that we all need to be mindful of these tendencies in order to attempt to counteract them. And when we encounter truly bad cops, which I would like to believe are the exception, their behavior should not be excused or covered up. If anything, public servants like the police should be held to a higher standard, and when they fail to live up to those standards, their behavior should be confronted and even prosecuted.