The Flaws in Female-Based Research
How A Lack of Understanding and Poor Experimental Design Lead to Inconsistent and Invalid Findings
There is an immense gender data gap when it comes to research.1 Where the majority of studies are done on men and then just applied to women. Where females are significantly underrepresented in all human studies in all fields. And where men are seen as adequate proxies for women.2
Thankfully, more females are being involved in studies and hormones and the menstrual cycle are being taken into consideration. And in fact the study of the menstrual cycle on the exercise response has been happening since at least 1876 when “Professor Mary Jacobi won the Harvard University Boylston Prize Essay for her observations on menstruation and physical work and rest”.3 So does that mean all is well in the world of research?
Not exactly. Because despite all the improvements that are being made in gender equality, there are still no sport and exercise-related guidelines for fitness and nutrition which are customised for women.
Why? Because the research that has been conducted is inconsistent, invalid, confusing and conflicting.
Lack of consistent methodologies
You would think that with decades of research, consistent methods of studying the menstrual cycle would have been developed. But you would be wrong.
That’s even with researchers, in the early 2000s providing critical commentary on the effects that oestrogen and progesterone has on different aspects of exercise performance as well as the methodological approach used in studies.4
This obviously began to raise an awareness of the flaws within this field and yet this doesn’t change the fact that we are still relying on research that is inconsistent, unreliable and invalid.
Because the fact is, poor experimental design within studies carried out on females is down to one thing — a lack of understanding of the menstrual cycle and hormone changes that females experience over their lifespan.
Understanding the complexities of hormonal changes
Oestrogen and progesterone are sex hormones. But they don’t just deal with reproductive function. They affect the whole body.
Not only do they target the tissues in the body including the epithelial, connective, muscle and nervous tissue but they also influence biological processes including metabolism, breathing, immunity, cognition, gastrointestinal, cardiovascular and nervous system function.
Thus different concentrations of these hormones will “influence the mechanisms that control and regulate cell function and integrated physiologic adaptation in women”.5
To put it simply, oestrogen and progesterone affect everything in the body. But how they affect functions and processes will depend on the amount of these hormones (which are changing all the time).
During the reproductive years, there are five primary factors that will impact the changes in oestrogen and progesterone levels.6
- normal fluctuations across the menstrual cycle
- disruptions to the menstrual cycle caused by illness or disease such as PCOS or endometriosis
- manipulation of the menstrual cycle with external hormones such as HCPs or HRT
- pregnancy
- menopause (and perimenopause)
If this is not understood by researchers then there are a number of mistakes that are being made which again renders the research inconsistent, invalid, confusing and conflicting.
A lack of validity
Elliott-Sale et al., (2021) wrote that “The validity of studies, or lack thereof, in sport and exercise science using women as participants is rarely discussed.” What is valid for one woman may not be valid for another.7
This means that when I look at the literature, the hormone status and reproductive profile of the study participants need to match the information I am looking for.
Another way to think about it is that when looking for injury risks in footballers, it would not be valid if the participants studied were swimmers.
The same is true for women who’s hormone status is different.
Hence we cannot compare the 16 year girl old who’s menstrual cycle has not yet settled with the 30 year old who is post-partum.
We cannot compare the 45 year old with perimenopause symptoms with the 36 year old who has PCOS.
We cannot compare the 60 year old who has been menopausal for the last 10 years with the woman who has endometriosis.
And we cannot compare the 23 year old who is taking HCPs with the naturally cycling female.
Each of these women have a different hormone status and reproductive profile. Each of these women have different amounts of progesterone and oestrogen in their body. And as we know, different amount of hormones affect our physiology in different ways.
Yet I see this all the time when reading through the literature. Or rather, I don’t see it. Because when female subjects are used, rarely is their hormone status or reproductive profile mentioned.
And if we are lucky enough to find that the menstrual-cycle status is considered by researchers, the hormonal profile of the participants is often implied rather than confirmed.
The importance of considering menstrual cycle phase
Another issue we come across when females participants are used in studies is that because women are believed to be so complex they are often tested in the early follicular phase to minimise the effects of oestrogen and progesterone.8
This the when oestrogen and progesterone are at their lowest, rendering women to be more ‘male-like’. Less obscure, if you will.
Or if the researcher has not considered the effect that the menstrual cycle can have on their study, the menstrual cycle phase is not mentioned at all and the female subjects are studied at any point in their cycle.
Thus a woman who is menstruating (↓ oestrogen & ↓ progesterone) may be compared with a woman who is ovulating (↑ oestrogen & ↓ progesterone) or a woman who is in the luteal phase (↑ oestrogen & ↑ progesterone).
But we simply cannot compare these phases. They are hugely different and regardless of what the research shows, every woman will tell you she feels completely different around menstruation compared to every other part of her cycle. And this affects…well, everything!
Inconsistencies in menstrual cycle phase definitions
So what about the studies that do break down the phases? They’re good, right? Actually, that’s probably not the case.
After all, we cannot, or at least, we should not look at just one single study.9 We need to look at multiple studies to see of they are saying the same thing or whether there are contradictions being found.
The problem here is that research has not adopted consistent methods of defining the different phases.
In the simplest terms the menstrual cycle can be divided into 2 phases, the follicular phase and the luteal phase. And these are separated by ovulation (not menstruation as most people think).
And these 2 terms are often used to describe the different phases that a woman is in. She is either in the follicular phase or luteal phase.
The problem here is that classifying the menstrual cycle into just 2 phases does not distinguish the multiple hormonal changes that occur within these 2 phases.
Luckily most of the research does not use this 2 phase approach. Nope, the majority of the research on the menstrual cycle focuses on a 3 phase model.10
- menstruation (↓ oestrogen ↓ progesterone)
- pre-ovulation (↑ oestrogen ↓ progesterone)
- luteal (↑ oestrogen ↑ progesterone)
But again, here we have an issue because this three-phased model approach comes with the assumption that steady-state hormone levels exist. And guess what? They do not.
Thankfully some researchers go further and use sub-phases, such as early follicular, late follicular, ovulatory, early luteal, mid luteal and late luteal.11 This is with or without the inclusion of ovulation as a discrete phase. Which is brilliant as they’re starting to look deeper at all the different hormone states that exist.
So am I happy with this?
Well I would be if every study followed the same methods and used the same definitions. But they do not. At least not at this moment in time.
So when reading the literature we are faced with studies that divide the menstrual cycle into 2 to 7 phases. Thus the inconsistencies in terminology and research design such as that of menstrual cycle phase definition leads to grouping of participants that do not match.
Hence the luteal phase of one study may not match up to the luteal phase of another study. And what may be referred to as menses in one paper may simply be labelled as the follicular phase in another.
The importance of accurately verifying menstrual cycle phase
Another issue I am finding when reading through the research is that of verifying the menstrual cycle phase.
According to Janse DE Jonge, Thompson and Han (2019) less than half of the studies on menstrual cycle & exercise performance measured hormone concentrations.12
And when hormone concentrations are not used to determine the phase, researchers often use calculations instead.
This is where they conform to the idea that every woman is the same, each with a 28 day cycle and thus base ovulation on 14 days after menstruation.
The problem here is that healthy cycles vary in length between 21 days and 37 days.13 This means that assuming every woman ovulates on day 14 is not only foolish but illogical.
Not only does this mean that the phase a woman is assigned to is incorrect but using calculations rather than more scientific methods (such as hormone concentrations or basal body temperature) increases the likelihood that women with anovulatory cycle (those that have not ovulated) are included therefore affecting results and, consequently, our understanding.
The overlooked hormonal transitions
Bruinvels et al.,(2022) also point out that the majority of research disregards the hormonal changes punctuating the transitions between the phases. Where “most of the prior research informing our practices with athletes tends to rely on a three-phased model with the assumption of steady-state hormone levels existing”.14
But as Bruinvels et al.,(2022) also point out these hormone transitions can challenge homeostasis. The changes from high to low concentrations can be rapid and profound, creating a dramatic change in the hormonal environment and consequently, the way a female thinks, feels and behaves.
Recommendations and efforts to improve research
It now goes without saying that the complexity of the menstrual cycle is considered a major barrier when including women in research. And from the information I’ve covered above I’m sure you can understand why.
And yet is simply not appropriate to exclude women from research, especially on the basis of convenience.
Thankfully there are a number of researchers who are doing what they can to improve future studies.
Janse DE Jonge, Thompson and Han (2019) make recommendations to verify menstrual cycle phase in their recent paper.15
Elliott-Sale et al.,(2021) intend to guide readers to adopt good practice when working with women in science studies.16
And Schmalenberger et al.,(2021) make recommendations that they hope will help make study results more meaningful and replicable.17
Granted, there may never be a universal design for women’s research, but we at least need to try.
Final Thoughts
So with all these issues, should we even bother even looking at the research?
I think so. The studies may not be perfect but they give us a place from which to start. Furthermore, even flawed research can still provide valuable insights and lead to further exploration and improvement.
Obviously it’s important to approach any research with a critical eye and consider the potential biases or limitations, but dismissing it entirely would be a disservice to the scientific community and the potential benefits that could come from further investigation.
In addition, simply acknowledging the flaws in research can help guide future studies and lead to more robust and accurate conclusions. Ultimately, it’s important to recognise that research is an ongoing process of discovery and refinement, and each study adds to our collective understanding of the world around us.
Let us also not forget that there are many studies that may not ever get published. See EXPOSING PUBLICATION BIAS↩︎
Elliott-Sale, K.J. et al. (2021) ‘Methodological Considerations for Studies in Sport and Exercise Science with Women as Participants: A Working Guide for Standards of Practice for Research on Women.’, Sports Med, 51(5), pp. 843–861. Available at: LINK.↩︎
Janse de Jonge, X.A.K. (2003) ‘Effects of the menstrual cycle on exercise performance’, Sports Medicine (Auckland, N.Z.), 33(11), pp. 833–851. Available at: LINK.↩︎
Elliott-Sale, K.J. et al. (2021) ‘Methodological Considerations for Studies in Sport and Exercise Science with Women as Participants: A Working Guide for Standards of Practice for Research on Women.’, Sports Med, 51(5), pp. 843–861. Available at: LINK.↩︎
Elliott-Sale, K.J. et al. (2021) ‘Methodological Considerations for Studies in Sport and Exercise Science with Women as Participants: A Working Guide for Standards of Practice for Research on Women.’, Sports Med, 51(5), pp. 843–861. Available at: LINK.↩︎
Elliott-Sale, K.J. et al. (2021) ‘Methodological Considerations for Studies in Sport and Exercise Science with Women as Participants: A Working Guide for Standards of Practice for Research on Women.’, Sports Med, 51(5), pp. 843–861. Available at: LINK.↩︎
Bruinvels, G. et al. (2017) ‘Sport, exercise and the menstrual cycle: where is the research?’, British Journal of Sports Medicine, 51(6), pp. 487–488. Available at: LINK.↩︎
I love the quote from Scott Alexander“Aquinas famously said: beware the man of one book. I would add: beware the man of one study”.↩︎
Bruinvels, G. et al. (2017) ‘Sport, exercise and the menstrual cycle: where is the research?’, British Journal of Sports Medicine, 51(6), pp. 487–488. Available at: LINK.↩︎
Carmichael, M.A., Thomson, R.L., Moran, L.J., Wycherley, T.P., 2021. The Impact of Menstrual Cycle Phase on Athletes’ Performance: A Narrative Review. International Journal of Environmental Research and Public Health 18, 1667. LINK↩︎
Janse DE Jonge, X., Thompson, B. and Han, A. (2019) ‘Methodological Recommendations for Menstrual Cycle Research in Sports and Exercise’, Medicine and Science in Sports and Exercise, 51(12), pp. 2610–2617. Available at: LINK.↩︎
Schmalenberger, K.M. et al. (2021) ‘How to study the menstrual cycle: Practical tools and recommendations.’, Psychoneuroendocrinology, 123, p. 104895. Available at: LINK.↩︎
Bruinvels, G. et al. (2017) ‘Sport, exercise and the menstrual cycle: where is the research?’, British Journal of Sports Medicine, 51(6), pp. 487–488. Available at: LINK.↩︎
Janse DE Jonge, X., Thompson, B. and Han, A. (2019) ‘Methodological Recommendations for Menstrual Cycle Research in Sports and Exercise’, Medicine and Science in Sports and Exercise, 51(12), pp. 2610–2617. Available at: LINK.↩︎
Elliott-Sale, K.J. et al. (2021) ‘Methodological Considerations for Studies in Sport and Exercise Science with Women as Participants: A Working Guide for Standards of Practice for Research on Women.’, Sports Med, 51(5), pp. 843–861. Available at: LINK.↩︎
Schmalenberger, K.M. et al. (2021) ‘How to study the menstrual cycle: Practical tools and recommendations.’, Psychoneuroendocrinology, 123, p. 104895. Available at: LINK.↩︎
