The human factor – why facts are a work in progress

Fact checking is the bread and butter of journalism. The first principle of professional ethics in journalism is people’s right to true information. This leads right into the second principle: the journalist’s dedication to objective reality. In short, journalists are expected by everyone, including themselves, to verify every statement they publish.

In practice, this doesn’t always happen which is why we’re used to editorial comments and corrections in news papers and on internet news sites. There are various reasons why journalists don’t always check the facts. The following three justifications were given in a recent Dutch study:

1. Explicit accordance:
   1. We followed the rules, so it’s not our responsibility if our report turns out to be false after all.
2. Practical accordance
   1. Not enough time, resources, or money was available to check all the facts.
3. Exceptional divergence
   1. there was no reason to check the facts because this clearly wasn’t real news.

Speaking as a programmer, these justifications sound all too familiar. If you ever wonder why computer programs you paid good money for have bugs in them, just take a look at these justifications above. They are as true for programmers as they are for journalists. In fact, they are probably true for most professions since these three justifications can be summed up as follows: Humans aren’t perfect.

Unrealistic expectations 

One of life’s ironies is that imperfect people expect others to be perfect. In fact, they often expect themselves to be perfect… provided the world around them cooperates. And when they don’t, well… clearly this isn’t their fault. Personally, I don’t object to that point of view as it leads to one very important thing: humans strive for the unattainable.  Objectively, the two principles above may be unattainable, but in practice most journalists do strive to adhere to them and are  held to account by both public and editors if they fail to do so.

But what about facts?

If people have unrealistic expectations about journalists, what about facts? Do we hold unrealistic expectations about them as well? In my opinion, yes.

According to Dictionary.com, a scientific fact is defined as follows:
“any observation that has been repeatedly confirmed and accepted as true; any scientific observation that has not been refuted”

This works well in science, but obviously not so much in the real world where some events only happen once and cannot be observed from afar. Journalists have found ways to deal with these limitations, such as the rule of thumb that a story isn’t published unless it has been confirmed by two independent sources.

However, data journalism deals with data and thus allows its professionals to take a more scientific approach to facts, right? No matter how many times and how many people run the same numbers, the result should always remain the same. Unfortunately, the numbers aren’t the starting point but the end result of a human process.

Example: Figures from the Dutch Ministry of Education

Each year, the Dutch ministry of education presents the numbers of youths who leave school without a starting qualification. The numbers for each previous school year are to be handed in by the regional and local authorities before november first. Meanwhile, the national ’Service of Education Execution’ (otherwise known as DUO) also hands in the number of student registrations that are sent to them by each school. Based on these numbers, the ministry officials calculate the national, regional, and local numbers that are used by the government to determine policy.

On january 16, the Dutch government published the following statement on its website (translated from Dutch:

“The number of premature school leavers has dropped significantly during the last school year to just 27.950 youths…. … On the one hand, this decrease is due to the combined efforts of schools, local governments, and other partners. On the other hand this decrease is due to the better measurements which clarify which youths truly leave school prematurely.”

The government admits that a new statistical measurement allowed these numbers to drop. However, what they don’t admit is an observation that was made a few weeks ago by someone at a meeting between DUO and the programmers which develop the software used by local governments:

“The problem with the government is that when they ask a question, they expect an immediate answer. They don’t understand that the data first needs to be gathered.” 

In other words… the numbers presented here are due to people being told that they needed to register their work.  However, people from different areas used different software programs. That is why the government and the different software companies got together in order to try and determine what exactly needed to be registered. But this was of course only one difference. The main difference was the different work processes used by the various local authorities. A big city in the west of the country for example automatically excluded youths that had found a job. A big, underpopulated region in the east automatically sent mail every 6 months to verify whether employed youths without a diploma didn’t want to get back into school after all. Different circumstances require not only different actions, but different terms as well. And that lead to different interpretations of the same questions.

Facts are a work in progress

If a good fact checker is a good reporter, how is he or she to deal with this issue? I’m afraid I’m just going to have to repeat myself from earlier columns: keep the bigger picture in mind and delve deeper into the subject. Do not accept the facts, but ask yourself how they came about and be open about these questions. Both to yourself and the audience.