February 19, 2020

Fighting fake news in biotechnology: Sense about Science

By Conor McKechnie and Dodi Axelson

Fighting fake news in biotechnology: Sense about Science

Can you trust the scientific claims on products you buy? Conor's curiosity led him down a three-pronged investigation about what you can and should believe when it comes to science statistics and statements. His journey starts with Alex Clegg from the Sense about Science Ask for Evidence campaign.

CONOR: So, Dodi let me run a quick test with you here.

DODI: Okay.

CONOR: 65% of people who floss their teeth…

DODI: Yeah.

CONOR: …reduce their risk of getting mouth cancer by 28%.

DODI: Wow. Oh my God, tell me more.

CONOR: Well, the thing is, you know, I actually can't because I just made it up. It's total rubbish.

DODI: Oh, that's super frustrating. It is so hard to know when the science behind media stories is true or manipulated.

CONOR: Yes, and it happens a lot and I get really annoyed about it. So, I went on a little bit of a mission to try and find out what's really going on and what we, as just your average person, can do about science being used to tell stories that aren't true.

DODI: Okay. So, I guess that is what matters on today's episode of Discovery Matters.

CONOR: That's exactly right.


CONOR: Okay Dodi, so you know me really quite well but full disclosure, for all of our listeners, I spend a lot of time thinking about how marketers and brands can influence people to spend their money. And I think a lot about the digital media space, of course, because that drives a lot of consumer behavior. But this story actually starts at a makeup counter at the Liberty's store in London where I was looking at a product, which had a, what could only be called a bogus claim on the back of it saying: “Stem cell science supports the use of this product” in some way or another. I can't remember the claim itself, but it just got me really a bit annoyed. So, I decided I would go and speak to people who work in science communication. So, I got hold of a campaigner, a scientist, and a journalist.

DODI: A campaigner, a scientist, and a journalist. Okay, this sounds like a setup. Do they all walk into a bar?

CONOR: It's not a normal podcast episode. There's no bar, because this is deadly serious. And it's deadly serious because look, science, misinformation and misuse 140 000 deaths from measles last year globally for a disease that was actually pretty much licked in countries such as the UK. But it's all a bit depressing for, kind of, the holiday period. So, we're going to stick to the use and abuse of science in business, politics, social media, and makeup. So, for part one we talked to the campaigner.

ALEX CLEGG: So, it all started back in 2007.

CONOR: This is Alex Clegg who works for Sense about Science. It's a charitable organization in the UK that focuses on the promotion of public understanding and engagement in science.

AC: I lead the "Ask for Evidence" campaign that encourages people to go out there and ask for the evidence behind claims they want to know more about.

DODI: Why did you start with Alex to answer these worries that you had?

CONOR: Well, I've known Sense about Science for a good while. And we have to go back to about 2007 as Alex mentioned, when Sense about Science was working with a group of young scientists that were getting really frustrated.

AC: They were getting pretty peeved off about these dubious claims out there or unsubstantiated claims that we were coming across. So ,they decided to take action and begin to challenge these dodgy claims that they came across and they were ringing up organizations and began sending emails and began asking for the evidence.

DODI: And where were they? The scientists finding these claims?

AC: Really all over the place. You know, obviously a lot on the internet. One was a computer, a vibration device that uses a homeopathic-type remedies to release, I think it was vibrations into you, you know? Yeah. It was in the supermarket, online, in adverts that you see when you’re just walking down the street. As we began to have more conversations about it, we realized it wasn't just scientists or people interested in research who were wanting to challenge these claims. We found more and more people were interested.

And so, you know, it was around 2016 where we recruited ten ambassadors to go out and give talks to different community groups and different interest groups about these issues. And we developed a website where you could actually request the evidence from a company, as a tool there, where you can request the evidence. And now it's over 1500 asks, have been logged. We've reached over like 50 000 people. There's a real appetite for it among the general public.

DODI: Okay. That's some pretty extensive and important work there.

CONOR: It is. And it became really, really clear that Alex was a good person to talk to about this. So, I just asked him straight up what had been worrying me so much. It's how well-equipped are we, the average person in the street, to make decisions and know the difference between science and advertising. Between real discovery and jumped-up claims.

AC: You know, I'm quite positive about these things. And I think there is a real appetite for it, but there's obviously work the we can be doing as well. And I think, you know, it's not always necessary for someone to have always the scientific understanding and evidence behind a product, you know. So when your friends recommend a yogurt to you because it helps them lose weight, you know like, that can be fine. But then if the NHS starts rolling out these yogurts because it reduces weight loss, then we need to have a good understanding of the evidence behind that policy decision and that works on the side of the people making that policy and then making sure that the public have the resources to understand that as well.

And I suppose the big thing, which we're doing as part of Ask for Evidence and with Sense about Science is working with young people. It's not always about knowing the nitty gritty, scientific, statistical analysis of a particular paper, but it's about critical thinking and giving people the confidence and saying that, you know, you can critically assess claims that you come across and there's ways that you can go about that and you know. And it's being able to just spot a claim and begin to understand what things like peer review are, what a method analysis is. These are the things where you can begin to have a conversation. And so that's something which we were constantly equipping people to, you know, join in a discussion. But we also say it needs to come from the other side as well. Organizations need to be willing to communicate evidence effectively as well.

CONOR: And I wondered if there was like a particular industry that gets challenged a lot more in the Sense about Science campaign.

AC: One of the surprising things, it is especially brought obviously a lot around health. So, you know, I've mentioned opticians before, but it can do with different drugs, vaccines. But then it can just as much be about transport infrastructure, emotional coaching, anthropogenic global warming, health apps, local election party manifestos. And so you see, this is what the exciting thing is. When it is a public campaign, you can see that a public do care about a whole host of evidence issues.

DODI: Well that's super encouraging in a sense because of course I'm gullible. I believed your statistic about brushing teeth at the beginning of this podcast. And it's also a bit discouraging by some of the things that we see in our field of work in science. But to see that there's a groundswell of people who are asking for evidence in this day and age, I'm just going to say it, it's, that's actually pretty encouraging.

CONOR: Yeah. I felt, I actually felt relieved. I've been grumpy about this for quite awhile, but to hear Alex talk very positively about it was, was really encouraging. But you know, you know me, I never miss an opportunity to be grumpy about something. So, I had to go and find another opinion. So…

DODI: Just to be sure.

CONOR: Yeah, exactly. So, now we move on to part two.

DODI: Yeah.

CONOR: The scientist.

ANA YONG: I'm Ana Yong and I'm a Material Scientist at the National Physical Laboratory.

DODI: All right, so what brought you to Ana?

CONOR: Well, I got to Ana through Alex, actually. She's a scientist and an ambassador for Alex's Ask for Evidence campaign. And I wanted to ask Ana a really simple question. How often in her daily life does she see scientific claims where she thinks: Hmm, that looks a little bit dodgy to me.

AY: Every day I think everywhere you look is the short answer. So I have a PhD and the point of the training that you do to kind of graduate as a doctoral level scientists to be able to look at things and critically assess absolutely everything you read. So, a scientist is trained to just make that decision quite quickly on whether or not something can be trusted, whether it requires further study or whether or not you can say, yep, that seems sound straight off the bat. So, I'd say that I see it every day, but the thought processes are quite fast in terms of whether or not I think something is suspicious.

DODI: So, if I'm not a classically trained scientist, and clearly I'm not, just a member of the public, what questions should I be asking when presented with what looks like a science back claimed for the effectiveness of a product? For example, that that makeup product that you were looking at.

AY: For me, the first thing I do is immediately look to the references that they've used. So if it's something like L'Oreal™ saying that 90% of women thought that this cream made them taller, normally there's an asterisk at the bottom saying, you know, nine out of ten women tested and you can say, well, you know, a group of ten women, that's really not a large enough group to get that kind of valuable statistic from. If that's not immediately available, you can normally see where the references are, where you can Google the research and the research papers will come up as a member of the general public being able to understand the research papers is another thing.

But the way science is communicated normally is published in a scientific journal. And the top paragraph of that publication will have an abstract that basically gives a summary of the findings. And within the abstract you can normally see how overinflated a claim is, if it is overinflated. So, in the abstract it might say ten mice got eczema from eating blueberries and then the you know, the headline might say that the reason that there's an eczema epidemic in the country is because we all eat so many blueberry pancakes. And you think, well, when did they bring pancakes into this? You know, that's not what the articles said. So, once you get comfortable reading abstracts, the evidence will be presented fairly plainly in a, in a scientific paper, in the abstract. So that's where I would go first. And if you can't find a journal article, well that's telling as well.

DODI: So, while you were chatting with Ana, I hope you also asked her about the influencers that got you worried in the first place.

CONOR: Yes, completely and never to miss an opportunity. We dug into that. And actually, her answer was pretty interesting.

AY: I'm not going to blow the whistle on any specific influencers, but I think that they ultimately, in order to sustain their careers as influencers, they have to be promoting products. They have to be advertising products and saying that they love certain things that they're paid to say that they love. Now when you see in an advert, you know that that's an advert telling you that the product is amazing. But when you see a relatable human being through Instagram, which is something that you and I both have, telling you that they love something and that's changed their life, you are more likely to believe them.

And whilst I don't think influencers are necessarily bad people, they do have to promote things in order to sustain their job to make a living. So there's a danger there that people are being led to believe that something is, you know, they need to go out and buy something because an influencer has said that it's great when they're not seeing what goes on behind the scenes in terms of how influencers work and, and how they actually get to that position of having 100 000 followers and not needing to have any other job outside of what they do as an influencer. But it's really interesting for me.

I think I really got into skin care blogs when I developed acne when I was 25 so, you know, I followed everything that the, the first blogger I started following said. Then I got into various blogs and I realized when a new product was launched, they would all say that this was a life changing product. And it became very apparent that they were all distributed this product and told to say it's amazing. And then the next product comes out and they do the same thing and that, I immediately lost trust in whether or not those things work. You know, that ultimately is still advertising.

CONOR: So before we get to the final part, I wondered if Ana thinks things are getting better or if things are getting worse, if people are abusing or using correctly the science in the advertising claims and in their promotion of their products.

AY: I think with the internet, obviously people have a lot more information and that's a great thing. But even with people who understand a scientific evidence, trials, et cetera. Without the training behind it, it's still very difficult to understand what to do with the evidence that's available even if you know how to find it. I'd say that I'm probably not old enough to know whether or not if things are getting better or worse because this is the time that I've been alive.

CONOR: So, let's talk to the journalist who is very often the person responsible for interpreting the claims made by companies and putting them into print or digital media.

DODI: Putting them before the eyes of the public.

CONOR: Exactly.

LIZ STOUT: Hello. Testing. Testing.

DODI: Okay, who's this?

LS: So, my name is Liz Stout. I'm an ex-beauty editor and a freelance journalist who now dabbles in the world of branding and copywriting.

CONOR: So, I met Liz and her dog, Margo, who was like really interested in the microphone and kept uhm, well we shut her behind a door at the end of it. Liz really knows a lot about the use of science and scientific claims in the promotion and sale of beauty products and makeup. Like the one that I was looking at in Liberty's. Other stores are available. So straight off the bat I wanted to know if Liz sees a lot of science in the promotional materials that's pushed to her by PR agencies and companies asking her to promote products.

LS: Yes, more so as a journalist. So you know, if I'm writing a beauty piece and I'll be sent the blurb from a PR, which ranges from being, you know, very scientific in terms of linking to clinical trials that have been done, right through to literally using these sort of pseudo scientific terms that, you know, aren't really saying anything, but it's sort of quite seductive in terms of the fact that they are new and they sound like a sort of new exciting innovation.

DODI: Okay. So, when Liz sees something like that in a press release, for example, how much thinking does she do about the actual science behind the claims that they make about the products?

LS: Well, I think as a journalist you have to. Because actually you won't get your story commissioned by your editor unless it's saying something, unless it's something quite groundbreaking or quite new or quite interesting. We're talking about beauty now. So, any brand that comes to me with something, you know with a new product boast, you have to check it out because ultimately there's accountability from the publishing, publisher point of view. Good journalism and bad journalism is ultimately it's about having evidence to back up your claims. And I think we're trained to do that. So, the answer to your question is I would, yeah, I would want to see evidence. I'd want to see, you know, what these pseudo terms mean and what they're actually sort of claiming to do.

DODI: Does Liz know, are there companies out there that are misusing scientific terminology? Even the illusion of clinical trials by saying, you know, 12 out of 15 consumers prefer X product and then you see that the sample size was, you know, 20 friends and family.

LS: I mean, I think that's sort of, it's quite sort of a Machiavellian view of it. I don't think they are deliberately out to do that. But I think very often in order to, you know, grab attention for a new product or a new concept, I think sometimes science claims are slightly glossed over. They appear more exciting than they actually are, I guess is what I mean. So, you know, something might be called some new name that actually when you look and you read the small print you realize actually it's not anything particularly ground breaking at all. They've just packaged it up, rather cleverly into something that sounds extremely exciting.

And I know of course in the past when I've worked on things and you actually then get experts in to look at the product. Because quite often, you know, as a journalist we've run, tried and tested features. We've tested them on people and that's involved getting, you know, a science person to look at the ingredient list and actually ascertain whether that product is going to do something or not. And then you kind of look at the amounts of ingredients that are in that.

So, something that claims to have, you know, Q10. Actually is the amount of Q10 in there going to do anything? You know, the retinol actually is the, you know, ratio of retinol to other ingredients actually going to be effective? And they're the small print things that I think your average consumer probably doesn't look at. They're just sort of seduced by the sort of grander top line claims that are, that products are making.

CONOR: In the end I kind of got to the root of what has been nagging me, the worry that brought me to the question in the first place. So how does somebody in the early teens distinguish between an influencer who is focused on doing the right thing and telling the truth and an influencer who is maybe not and has other motivations at the back of their mind?

LS: Okay. So, I think for that Conor, I think it's less about, you know, targeting individuals. I think it's more the phenomena of, of the social media, which is more worrying than actually the individuals. Because I think social media as a whole, like Instagram just presents a very glossy aspirational view on things. So, I think the danger, if you're talking about your daughter, and I have a daughter the same age, I think it's more about them that they are seeing, you know, a reality that just isn't a reality. They are seeing images of, you know, lifestyles that just are completely posed and completely, you know, fake. They're just not real. So that, that I think is something to worry about.

I do think there's a growing movement to expose that. So, people like Celeste Barber, you know, she's Australian comedian, she does hilarious Instagram posts where she sends up, basically Instagrammers who are posting these sort of very glamorous shots. And she just basically does her version of them and they're brilliant. They're brilliant parodies of what, you know, what's appearing on these very glam feeds. So, I think there's that, but I think, you know, when you get down to a sort of beauty product level, and I know that's how we started this conversation.

You know what? I think, you know, buying an eyeliner because Kendall Jenner got it. Or you know, buying a lip gloss because I don't know Taylor Swift used it once. I'm not sure that, I personally think that that's too much of a worry. I think it's sort of part of being a teenager. You sort of want to look like you're, you know you want to look like your idols and in our youth we probably just had posters of them on our walls and to that. But I think it's just sort of a natural evolution of that.

CONOR: I came away from discussion with these three thinking that, you know, it's not as bad as grumpy me thinks that is going to be. There are a lot of people out there who really are challenging the claims, asking the questions, the right questions. There are people out there who are helping people to find the answers to those questions and there are good journalists out there who are doing good work. Making sure that when a product makes a claim about what it does and what it doesn't do, that those claims are checked and that they're appropriately challenged.

DODI: Is there a simple litmus test? Is there somewhere where people can go for a yes, this is true or no, it's not.

CONOR: Yeah, completely. You can and there's a whole series of these places that you can go to look for scientific facts. One place, if you're looking for evidence about clinical trials and health claims, a really good place to start is the Cochrane foundation, which does meta studies into clinical trials. If you're really looking at health. One of the other areas to go is to Full fact. It is a UK fact checking website...

DODI: Is that fullfact.org?

CONOR: Yeah, fullfact.org. Yeah and that's if you're looking at claims that are more broadly, not necessarily purely science-based. And then of course there is Sense about Sciences’ Ask for Evidence campaign where you can go and you can download letters and a form that you can send to the organizations that you're asking for them. So that's senseaboutscience.org so you can go there and actually ask for the evidence for any claim that you've seen from any company about anything.

DODI: Okay. So, from the trio of the campaigner, the scientist, and the journalist, I'm not sure if I feel more optimistic or pessimistic, but at least I know where to go.

CONOR: Yeah, and it turned out not to be a joke.

DODI: Any of it.

CONOR: None of it. No jokes.

DODI: Thank you for listening to this episode of Discovery Matters.

CONOR: Thank you. And goodbye.

CONOR: Our executive producer is Andrea Kilin. Discovery Matters is produced in collaboration with Soundtelling. Production and music by Thomas Henley.

Listen to more podcast episodes