Making sense of science in the messy world of every day parenting

[rockhopper72]

@011235813 just wrote this (my emphasis):

As parents who want to follow evidence based parenting practises, it can sometimes be very challenging to use the information presented. And I’m not a scientist and don’t know how to interpret statistics, it’s difficult for me to refute or present alternate research. So to me a big part of this sub logically isn’t just about presenting the science, but understand what it means in the messy world of every day parenting. [Edit: See here for context from @011235813.]

I'm coming at this from the other end, in that I am a scientist, I understand stats and it's difficult for me to write about research in a way is accessible to a non-technical audience. I would really, deeply, honestly, appreciate any thoughts on how it could be done.

A few notes (which can be skipped):

• Statistics is a red herring. It's really bad in much of social science, inc. child development. In at least 85% of cases people are not correcting for multiple hypothesis testing, which is a fancy way of saying that they're doing this and admitting to doing it. Reviewers should stop this but aren't statistically literate either.
• This means you never rely on one paper. Instead we lean on the mass of evidence -- once multiple studies in multiple countries reach the same conclusion, you can start to be confident in it. (Continuing XKCD, if 5 independent studies had found that green jelly beans cause acne, and none had found that any other colors did, there would be collective evidence there.)
• It's not as simple as counting papers on each side. Things like sample size matter. More subtly, you need to look at all the bits of research and see how they fit together. This is hard to explain, so here's an example. I've written about long-term effects of daycare before. We have
  • Psychological studies that follow children over time
  • Biological studies that measure children's stress levels in daycare centers
  • Biological studies that show long-term effects of early life stress
  • Economic studies that show the long-term effects of center daycare in real populations
• ... and all of these fit with each other; the psychological studies find the exact consequences predicted by the biology, and the economic studies show that those consequences arise in the real world. Plus, critically, the exact same factors that cause more biological stress cause larger effects in the psychological and economic studies.
• How on earth do you convey the details of a 'big picture' like this to a skeptical audience? I've tried writing 'annotated bibliographies', where I mention each paper I draw on and how it fits in to the big picture, but it quickly becomes way too long for anyone to read.
• One thing that doesn't work: chest-beating about supposed credentials on Reddit. They. Are. Completely. Unverifiable. (Unless you're crazy enough to doxx yourself.) Often people claiming to be experts on X clearly have no idea about X.

Thoughts would be very welcome.

 

[rockdrik]

@rockhopper72 So I had another look at your article because I really wanted to comment on this. I'm a scientist too (physical sciences, not psychology or anything remotely related to parenting) and I LOVE interpretation and application of science.

Frankly, I thought your article was excellent. It's referenced, but you brought in a lot of interpretation and thoughts that are relatable to parents. What makes you think you missed the mark?

It's never going to be possible to cover a complex topic simply and this certainly qualifies as a complex topic. Different types of writing are appropriate for different purposes and different types of audiences. Online, you can't really control your audience, so you have to just go with being clear on your purpose. I think yours was to summarize a breadth of research into a digestible and thoughtful package, and you succeeded.

I don't think it's surprising that people have things to add, any piece of writing has limitations and people who are interested are going to bring their perspectives. Any individual study may have a clear result, but when it comes to making science useful, it's much more of an art. I'm not sure everyone appreciates this reality. Personally, I have seen very few parenting topics (if any) where science has "the answer". What science has is information for decision making and your article does exactly that.

Not everyone is interested in being unbiased, lots of people come to discussions like this with emotional attachments. There isn't much we can do about that, as scientists, except present the evidence and let people absorb or not absorb and have compassion for their individual human experience that brings them to an emotional reaction.

Finally, I don't think we should strive, with our science writing, to tell the whole story. Doing so fails to hit a practical balance: the idea of letting perfect be the enemy of the good. For example, I think the impacts of child temperament and discussion on the difference between harmful and helpful stress are logic extensions of your work, but it's just not realistic to cover all the relevant ground in one place.

 

[rockhopper72]

@rockdrik That was a very kind and reassuring comment. Thank you.

What makes you think you missed the mark?

Specifically, it was reading this comment from @andrew1221 on r/Parenting:

There’s an actual study that backs up what they’re saying, though it’s methodology is hotly debated over on r/sciencebasedparenting:

When I'm buried in work I leave Reddit for long periods, so I don't see things playing out. If the methodology I'm using is hotly debated by people on this sub, where the rules and moderation keep out most trolls, I really want to know what I can do to help convince them. Maybe I'm being unrealistic.

Thanks again for the comment -- it's helped me put things in perspective.

 

[rockdrik]

@rockhopper72 I have a lot of respect for @godwithpeter's moderation. I've been here since she started this sub and although the rules sound a bit harsh, she just doesn't want people getting harassed for no reason. She's not removing legitimate comments (my experience).

Re convincing them, something to think about is people like to react, but that's not (always) the end of their thinking. People do process and learn and change, so maybe you're doing exactly what you want but just don't know it.

I totally understand where you are coming from. Your article was brave, even wrapped in all your caveats. But it was also useful and open minded. It was a good contribution. To the other comment about simplifying, personally I don't think you should. You have a gift for writing at exactly the level you did and not everyone does. There is ALWAYS something lost in simplification. I'm not saying it's bad to do in general, but I think you doing so would deny the world the value of your specific skills.

 

[introverteen]

@rockdrik Wish I could upvote this more than once

 

[katrina2017]

@rockhopper72 I mean, hotly debated as in “it wasn’t an RCT!” and “people in Quebec have no problems with their childcare” and “my baby went to daycare right from the hospital and was perfectly fine!”. The usual rhetoric. And some stuff about how it’s political and how you’re anonymous. I have yet to see anything seriously scientific in the critique.

Like you, I don’t disclose my credentials or specific job title on the Internet because it could be identifying (and it’s not like it matters), but as someone who is at least somewhat science literate having authored several papers on various topics, your article seemed very balanced and convincing to me. I suppose I’m biased though because I live in a country with good mat leave and don’t have to send my kid to daycare until at least a year old (or later). So the findings don’t challenge my parenting style.

 

[fullongrace]

@katrina2017 This is a vast simplification of the criticism. I have a heavy background in observational causal inference and I saw plenty of strong criticism beyond just writing off anything that isn’t an RCT or anecdotes.

What I did see though is something very common in my experience as the statistical expert working with researchers of other fields which is the misalignment of how important the statistical nuances are.

 

[rockhopper72]

@fullongrace I've replied below your other comment -- please see here.

 

[fullongrace]

@rockhopper72 I’m not sure how the debate about your daycare post Is relevant to this post. The criticism and debate I saw were from folks similar to me that have a heavy statistical background which understand perfectly well the very basics you’ve stated here. I’m a public health statistical researcher with a specialization in causal inference and their concerns can be understood on their merits. So it’s not an issue of you not communicating it well enough, it’s an issue of your interpretations being a reach for the studies you’ve cited and falling into similar traps as you accused Emily Oster.

 

[rockhopper72]

@fullongrace > criticism and debate I saw were from folks similar to me

I'm not sure whether you're referring to specific posts. (As I said, it sounds like I've missed a lot of discussion.) Skimming the comments on my original post, you specifically objected to my use of the word 'robust', and I changed it. It's worth however saying that economists Elango, Garcia, Heckman and Hojman (2015) say:

the authors of the Quebec study perform sensitivity analyses and report robust results

Similarly, economist Steven Lehrer commented:

Baker, Gruber and Milligan’s work is 100 per cent correct. It’s robust. If anything, in our own work, including a paper that came out in Canadian Public Policy and won “best paper award” for that year, it actually finds the effects get larger over time, on average

(Emphases mine.) I didn't want to quote these before because quoting a Nobel Laureate felt like arguing from authority. But I think that you may be giving readers a sense that the BGM Quebec study is considered shaky within economics; that's just not the case.

your interpretations being a reach for the studies you’ve cited

Please be concrete? When I criticised Emily Oster, I quoted her and provided citations pointing out serious errors in what she claimed. I feel that making broad assertions like the ones you have isn't constructive, because they are unfalsifiable.

> I’m a public health statistical researcher

Is it possible that this is the cause of our disagreement? The nature of the evidence is just different in different fields, and my guess is that you and others are trying to apply a standard that is inappropriate here.

_____________________________________

I'm very happy to have a constructive discussion here, on the methods of the Quebec paper or anything else! If you'd like to, though, please read the latest version of the article as I've removed/revised some of the things you may have objected to.

 

[daltonracer]

@rockhopper72 Ex science communicator here - one of the best bits of advice I got was to work to an audience with the scientific/math literacy of a 12 year old. That might seem strangely low - but think about your aunt who hasn't been to school in 30 years. Or your uncle who left at 16 to work or learn a trade. Or your cousins who spent their time reading fantasy novels. They are likely intelligent and smart people - but they aren't necessarily up to date or knowledgeable. So scientists need to simplify to get a point across (As opposed to 'dumb it down').

Engagement is great too! It's how people develop their literacy - we're all mini-scientists to some degree (see endless questions and experimentation by toddlers as exhibit A!) but it gets beaten out of us, or put on a pedestal of 'right vs wrong' which just sends people on tangents. So patience, just like you'd show your 12 year old niece who is critiquing your points, is a good skill to develop too.

 

[rylt]

@daltonracer The suggestion for teachers (not that most of them follow this in my experience) is to make sure none of your parent communication is written above a 4th-5th grade level (so about the level of a 10 year old who is on track academically).

 

[coltonsindy]

@daltonracer

They are likely intelligent and smart people - but they aren't necessarily up to date or knowledgeable. So scientists need to simplify to get a point across (As opposed to 'dumb it down').

I love this!

 

[wadebayo30]

@rockhopper72 while this negates your last point (so you’ll have to take my word for it), I’m a professor in cognitive development, and you’re on the money. Dev psych is getting BETTER about stats, larger sample sizes, pre-registration, etc, but there’s a long way to go. It’s only in the aggregate that you can get a good picture of anything. Also, due to ethical concerns, so much of developmental research is correlational, and often ten years later those correlations are found to be based on third variables (most often things like socioeconomic status). I also do cross-species research and doing that type of research is just so different than what I do in the developmental world. Much less extrapolation/interpretation, stricter stats requirements, etc.

FWIW, the r/science subreddit does verify credentials in order to offer up flair. Not like that couldn’t be falsified I’m sure, but on some account (maybe not this one) I had to provide proof of PhD. Makes me a little more likely to trust flared sources over there.

edit: just to add a few points after reading more of your comments.

-As a scientist, I do not feel comfortable making policy recommendations. That is not my field, that is not my training, I know my lane. We need more people who are specifically trained to make those bridges, but I’m not it.

-Similarly, OPs experience as a statistician provides a really valuable lens, but OP is not a developmental psychologist. My partner is a data scientist and statistician who mostly works in Biology. He defers to me on my area of expertise and I defer to him on his. Sometimes our perspectives very much align and sometimes they do not. Eg, he doesn’t like the way sibling data points are often handled in labs, but there are practical implications to things that require less than perfect statistical modeling for ecologically valid purposes.

-Science is about debate. Debate forces us to create better, more convincing, more powerful paradigms. This is why peer review is so critical. Don’t assume your job is to simply “convince people” and stop at that. You also don’t have all of the data. None of us do. Let the debates happen. It’s part of the process.

 

[rockdrik]

@wadebayo30 I wanted to pick up on your point about not being involved in policy making. In my organization (PhD also), I'm promoted based on my ability to impact the real world with my research, not just publish papers. While I never have the last word on policy, I write clauses that policy makers integrate into the final document and they take my input seriously; my input often goes well beyond the data itself. Being able to do this well comes with experience and also personality. I thought this was the same for most academic positions. Can you comment on this from your experience?

 

[wadebayo30]

@rockdrik There are always requests for a few sentences on “broad applications” in large federal grants and publications. But, no, I have absolutely no requirements for doing this practically for my professorial position or publication records. This may be because of the type of research I do which is in the social/cognitive and neuro realm. I do not publish or interface with applied developmental psychologist, I don’t work with education journals, I don’t study parenting or mental health outcomes. Is it cool when someone in those fields picks up on my work, or in the side projects I have that explicitly collaborate with public health experts, or when a finding goes mainstream and gets into classrooms? Yes. Do I consider myself fully competent to make claims by myself about how my lab findings interact with real world scenarios in which there are variables I know nothing aboht? No.

 

[rockdrik]

@wadebayo30 Interesting. This may be field-dependant. I work in engineering. It's research, but very, very applied. That could be why. Research involving human beings is much more complex so I get it's different.

 

[wadebayo30]

@rockdrik yes, I’m sure it’s field specific, and given that this thread topic is about dev psych, it seemed relevant to note that this isn’t necessarily encouraged in our field. I have colleagues in not just psych but bio and chem at my institution who have been reprimanded for too much engagement in community work and not enough focus on their purely empirical findings as they go up for tenure. It’s been hard work changing that narrative, particular because minority groups in stem (women, BIPOC individuals) are more likely to do community-engaged or applied work.

 

[rockdrik]

@wadebayo30 Wow, thanks for sharing. It would be fascinating to understand why those views are there in the first place.

 

[011235813]

@rockhopper72 I would like to also emphasize my additional response:

I guess what I’m saying is, I worry that the phrasing of this rule will embolden folks who think this place is only for posting research and who try to shut down or belittle comments about feelings/reactions to it) even if they are not attacking), or who are seeking practical applications of the science.

I’m not completely science illiterate - I have a couple scientists in my life and have enough schooling to understand a good deal, but I also don’t have the time/energy to read every paper that seems like BS in the abstract.

Personally, I wish more scientific analysis could be presented with policy change suggestions. And obviously that is not the norm for science, but it might make sense here sometimes.

E.g., if I recall, you’ve posted a lot about daycare. And if, say, daycare isn’t beneficial for kids until they’re 2 or 3, what needs to change in society to make that happen? Multi-year parental leaves? Poverty alleviation through social housing or UBI? Parenting classics and mental health services for all? I feel like presenting that type of info along with the original science would go a long way to helping people listen to the science without feeling judged or being overwhelmed by guilt because they’ll also see why their own situation made following the science impossible. (And then in turn maybe parents start demanding those supports from politicians to make that feasible.)

Just a thought, and I know it’s not something that is immediately possible for every poster. I just think it’s important to accept that with respect to parenting the science always goes hand in hand with the social, and both are needed to make changes in the world.

—

What you quoted above is an incomplete thought from me, I think. My larger point is that it’s not just about I don’t always know how to parse the science”, but I would appreciate when science is presented with practical applications not just for the individual but for our larger systems, which can arguably play a greater part in putting the science to use.