The human factor – why facts are a work in progress

Fact checking is the bread and butter of journalism. The first principle of professional ethics in journalism is people’s right to true information. This leads right into the second principle: the journalist’s dedication to objective reality. In short, journalists are expected by everyone, including themselves, to verify every statement they publish.

In practice, this doesn’t always happen which is why we’re used to editorial comments and corrections in news papers and on internet news sites. There are various reasons why journalists don’t always check the facts. The following three justifications were given in a recent Dutch study:

  1. Explicit accordance:
    1. We followed the rules, so it’s not our responsibility if our report turns out to be false after all.
  2. Practical accordance
    1. Not enough time, resources, or money was available to check all the facts.
  3. Exceptional divergence
    1. there was no reason to check the facts because this clearly wasn’t real news.

Speaking as a programmer, these justifications sound all too familiar. If you ever wonder why computer programs you paid good money for have bugs in them, just take a look at these justifications above. They are as true for programmers as they are for journalists. In fact, they are probably true for most professions since these three justifications can be summed up as follows: Humans aren’t perfect.

Unrealistic expectations 

One of life’s ironies is that imperfect people expect others to be perfect. In fact, they often expect themselves to be perfect… provided the world around them cooperates. And when they don’t, well… clearly this isn’t their fault. Personally, I don’t object to that point of view as it leads to one very important thing: humans strive for the unattainable.  Objectively, the two principles above may be unattainable, but in practice most journalists do strive to adhere to them and are  held to account by both public and editors if they fail to do so.

But what about facts?

If people have unrealistic expectations about journalists, what about facts? Do we hold unrealistic expectations about them as well? In my opinion, yes.

According to Dictionary.com, a scientific fact is defined as follows:
“any observation that has been repeatedly confirmed and accepted as true; any scientific observation that has not been refuted”

This works well in science, but obviously not so much in the real world where some events only happen once and cannot be observed from afar. Journalists have found ways to deal with these limitations, such as the rule of thumb that a story isn’t published unless it has been confirmed by two independent sources.

However, data journalism deals with data and thus allows its professionals to take a more scientific approach to facts, right? No matter how many times and how many people run the same numbers, the result should always remain the same. Unfortunately, the numbers aren’t the starting point but the end result of a human process.

Example: Figures from the Dutch Ministry of Education

Each year, the Dutch ministry of education presents the numbers of youths who leave school without a starting qualification. The numbers for each previous school year are to be handed in by the regional and local authorities before november first. Meanwhile, the national ’Service of Education Execution’ (otherwise known as DUO) also hands in the number of student registrations that are sent to them by each school. Based on these numbers, the ministry officials calculate the national, regional, and local numbers that are used by the government to determine policy.

On january 16, the Dutch government published the following statement on its website (translated from Dutch:

“The number of premature school leavers has dropped significantly during the last school year to just 27.950 youths…. … On the one hand, this decrease is due to the combined efforts of schools, local governments, and other partners. On the other hand this decrease is due to the better measurements which clarify which youths truly leave school prematurely.”

The government admits that a new statistical measurement allowed these numbers to drop. However, what they don’t admit is an observation that was made a few weeks ago by someone at a meeting between DUO and the programmers which develop the software used by local governments:

“The problem with the government is that when they ask a question, they expect an immediate answer. They don’t understand that the data first needs to be gathered.” 

In other words… the numbers presented here are due to people being told that they needed to register their work.  However, people from different areas used different software programs. That is why the government and the different software companies got together in order to try and determine what exactly needed to be registered. But this was of course only one difference. The main difference was the different work processes used by the various local authorities. A big city in the west of the country for example automatically excluded youths that had found a job. A big, underpopulated region in the east automatically sent mail every 6 months to verify whether employed youths without a diploma didn’t want to get back into school after all. Different circumstances require not only different actions, but different terms as well. And that lead to different interpretations of the same questions.

Facts are a work in progress

If a good fact checker is a good reporter, how is he or she to deal with this issue? I’m afraid I’m just going to have to repeat myself from earlier columns: keep the bigger picture in mind and delve deeper into the subject. Do not accept the facts, but ask yourself how they came about and be open about these questions. Both to yourself and the audience.

Advertisements

7 thoughts on “The human factor – why facts are a work in progress

  1. Great example of one of the pitfalls of data-journalism: wrong interpretations.

    You mention that ‘humans aren’t perfect’ and that ‘humans strive for the unattainable’. Those are good notions to be aware of and I agree with them. The conclusion of your post contains some practical advice about fact checking as a journalist. To what extent do you think the idea of striving for the unattainable apply to that advice, if at all?

    Like

      • Having read all of our blogs, and listened to three more presentations, I think we can all agree that it’s difficult to get to a completely objective truth. And yet, most journalists do strive to be as objective as possible according to d’Alessio’s book. So journalists do strive for the unattainable… and fact checking is a process that allows them to get close.

        Journalists are employed to be critical thinkers however, and so shouldn’t just blindly follow the ‘fact checking rules’, but also look beyond the facts and see how they were generated.

        In the end, you can ask yourself… am I satisfied with this article? If you need to apply any of the justifications, you probably should continue to strive further and forego publication.

        Like

  2. Great blog! “Fact checking is the bread and butter of journalism” – I strongly agree with this killer statement. Of course, human factor is inevitable. It’s hard to check everything you publish is this fast pace world. I like your comparison with programmers. I believe justifications that you mention above is following each of us even in different spheres of our life (like lack of time, close deadlines, just following the rules), so we should always be critical to the content we have in front of us, because we are humans – we do make mistakes. It’s true that when it comes to data journalism there is more time to analyse the numbers but I agree, that results can differ. After all, facts are open for interpretation, so it means – numbers as well. In your opinion, when do you think is enough to analyse numbers in data journalism if every time we get different results. Is it when another person person (or you) get the same results? Or it is a never ending process?

    “Facts are a work in progress”. Agree. Great phrase. Both journalists and audience should keep in it mind.

    Like

    • If people keep running the same numbers through the same statistical analysis and keep ending up with different results, then either the numbers or the analysis isn’t exactly the same. Either the data set was switched (which makes the source questionable) or the thresholds in the analysis differ, which quantifies the different interpretations held by the analysts. And that is a story well worth examining imho. 🙂

      Liked by 1 persoon

  3. Again good story and thoughts from you. So your example is another proof that why data journalism will be popular in the future. People are not perfect but scientists are a group of people who are relative perfect, I think. When data sciences combines with journalism, I do hope there will be more factual and trustable stories produced in the world.

    Like

    • I wouldn’t call scientists relatively perfect, they are humans so have flaws as well.:) That said, the scientific community values critical thinking, repetitions, and validations so they can produce good tools and methods for journalists to use in the journalistic quest for objective reporting.

      Like

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen.

WordPress.com logo

Je reageert onder je WordPress.com account. Log uit / Bijwerken )

Twitter-afbeelding

Je reageert onder je Twitter account. Log uit / Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit / Bijwerken )

Google+ photo

Je reageert onder je Google+ account. Log uit / Bijwerken )

Verbinden met %s