Friday, July 19, 2013

All Things in Moderation. Except Science.

This is my eighth year of blogging. It's a personal thing for me. I enjoy writing about training and racing with power. It's just stuff. Pub chat fodder for those four sad guys that talk about power meters in pubs. You know who you are. It's OK to come out and admit it, there's nothing wrong with it.

Occasionally I post an item that seems to be actually helpful to people, such as items on performance testing or indoor training. By posting I learn things and that's enjoyable. Hopefully others learn too. And I make mistakes. It happens.

I think I've held pretty well to the purpose as stated in my very first post.

I also used the blog to help me through difficult times, as longer term followers would be aware through sharing some of my experiences of getting back to racing after my 2007 accident and amputation. Where possible I tied the two themes together.

It's not a commercial blog and the subject matter is reasonably esoteric, and the following is modest with views numbering in the hundreds of thousands, not millions.

Moderation


I also know that public online interaction forums require moderation. Again it's mostly to keep spammers at bay, but occasionally some stretch the bounds of reasonableness and need to be reeled in or reminded of some basic netiquette. Troll watch duties as well. Sometimes you forget to check the moderation queue for comment that require moderation and they get posted a bit late.

I am a forum moderator myself, at the wattage forum on Google Groups which grew from modest beginnings in 2001 and was originally hosted on Topica. Want to pull apart anything to do with power and cycling? That's the place to do it. The archives are a gold mine of information, and the early days had a high signal to noise ratio. Only very rarely does moderation occur there, it's mostly self policed and the 10,000+ crowd are generally technically or scientifically oriented, so bullshit generally gets called out early and dealt with.

or Censorship?


But what I've never understood is why some forums or blogs dedicated to performance and/or training and racing would want to prevent actual relevant science or scientific discussion from being published/referred to.

I first encountered this a few years ago when I was banned by a triathlon forum owner from posting to the Slowtwitch forum. My crime? I wanted to know why they were persistently censoring the publication of a link to new published scientific research related to pedalling a bike. They didn't like the subject matter, nor my criticism of such censorship, so I was shown the electronic door and all evidence of the matter wiped. I was not offensive, I did not abuse, swear or curse at others, I did not break published forum rules. OK, it's their privately owned business, they can do what they like and they are large so what does it matter?

But it made me think. If you censor science, or discussion of science, is it really a place worth visiting? Well obviously lots think so, because such things are but minor ripples in the ocean of posts about saddle height, cadence, new bike frames, paleo diets, and the inevitable sarcastic pseudonymous responses.

So hooray for science. Perhaps.


Fast forward to a blog followed by many who, like me, enjoy their science in accessible form - the guys at The Science of Sport blog. I've followed it for a number of years, pretty much since it began in 2007. They dissect performance matters from a range of sports, in particular running, and introduce various topics for discussion. All good stuff and I think they do a good job of it.

Eventually they started writing about cycling performance like this one talking about the 2008 TdF and the all pervasive issue of doping. As they began to inform themselves on such issues in the cycling world, they also joined the bandwagon of estimating power from climbing speeds / VAM, and what we can learn from such information. They also began to suggest what is and is not physiologically plausible sans doping.

In the early days of such posts, I helpfully explained to them via the blog comments one of the difficulties in using climbing speed to estimate power, i.e. the large unknown factor of wind, and they got that message loud and clear. They even referenced my comments in subsequent posts. And they have usually referenced the issue of wind confounding such estimates whenever this topic comes up.

In general they have also tried to steer people away from the folly of using individual data points on climbing speed (or power estimations from them) as some means to infer doping status, but rather to consider the longer term trends. All good stuff.

Wham bam, thank you pVAM


Lately though they have been publishing a lot of information derived by the pVAM methodologies, which I am less enamoured with for a couple of reasons.

Firstly, whether intentional or not, it encourages the invalid use of such information as a "dopeometer" and secondly, I'm unconvinced they have sufficiently critically assessed the validity of the pVAM and dpVAM model. I think this is bordering on a desire for publicity than on good science. Not nearly to the extent that Antoine Vayer's occasionally dodgy maths does, but are they pushing the "Publish" button a little too quickly at times?

I get that their blog is designed to provoke discussion about sports science matters, make things accessible and to make people think more critically about stuff, and that's a good thing - but when you represent yourselves as "The Science of Sport" and present methodologies or models under that masthead that have not been peer reviewed or are based on known/published science, or without having done critical assessment against existing models, well I just think more caution is warranted before lending an overweighted amount of credibility to such things. Perhaps greater emphasis is needed on making it clear what is opinion and what is actually science.

I could of course be completely wrong and talking out of my arse. Happy to stand corrected. Most of us out here is the interwebs are not well versed in understanding the credibility differential of such material published by scientists under a heading of "The Science of ...".

It's difficult for me to strongly criticise since I'm not a scientist and not well versed in such peer review, so I just occasionally chip in with my amateur 20 cents worth and hopefully it generates some thoughts, at least in a pub chat kinda way. At the least I have a personal sense of contributing in some way, however insignificant that contribution may actually be. Which is what I did four days ago when I pointed out an error in a statement they made about the pVAM method.

This is what was written on their post:
"The error in the wind component is much larger, which has implications for the assumptions of cdA (drag co-efficients). In the model, however, the relative contribution of the wind during climbing is small, and so the total error is actually not too bad."

I then pointed out the obvious (to me anyway) physics mistake by posting this to the comments:

"The error in CdA isn't the issue, but the wind velocity most definitely is. Ignore that and you may as well throw darts at a W/kg board. That interpretation suggesting the error introduced by wind is, frankly, nonsense.

Even modestly different wind conditions for the same VAM can see estimates of power over a 1W/kg error range. The wind for the Armstrong/Pantani ascent was quite different to this year's ascent."

Perhaps my tone wasn't ideal, but nonsense is nonsense, right? And scientists generally have pretty thick skin.

For anyone wondering what I'm on about, when estimating power from steep climbing speed, if your assumption of the rider's coefficient of drag is (CdA) is wrong, well it doesn't generate a really big error in the estimate of power. It's not "sensitive" to that particular assumption. But it is most definitely sensitive to the wind velocity assumption. Get that wrong and the numbers can easily be wrong, and by a large margin.

Comment MIA


Then yesterday another SoS post about Alpe D'Heuz climbs and the pVAM and dpVAM methods appeared, so I thought I would post a comment with a link to djconnel's cool blog post which introduces a critical appraisal of the pVAM model, from both a physics and physiological basis, by comparing it with actual published and well established science models. I also provided links to a couple of charts about ADH climbing times that were easier to read than the ones they posted up.

As scientists, you think they might be interested in discussing the limitations of a model they are presenting as highly credible, or consider how it might be improved or under what circumstances we need to be very careful in using it.

Well for starters, I suggest using actual physics when estimating power from climbing speed, and also checking how the model stacks up with established physiological models.

Except my comments to them are now being moderated and have not appeared.

As Robert Chung would say, Hmmm.

15 comments:

Duane said...

I haven't been shunned by their blog, but I have been less than impressed by their analysis of cycling. They seem to be a bit out of their element and I find it disingenuous to write about doping during July than to really study the data and write when they have more to say. I can't help but feel that they would rather have a thousand eyes reading mediocre analysis than to have a hundred read a thorough and measured critique of the sport. Besides, I find that they add very little value to the discussion of how science improves the art of cycling training and the physical demands of elite cycling.

I hope they can improve it because we need more scientific analysis of sport. For the vast majority who have no interest in doping, either personally or as it relates to hero worship, could be better served by their publication.

Andrew R. Coggan, Ph.D. said...

"I can't help but feel that they would rather have a thousand eyes reading mediocre analysis than to have a hundred read a thorough and measured critique of the sport."

Some people would rather be known than be right.

Andrew R. Coggan, Ph.D. said...

Thank you, Alex, for "speaking truth to power".

Unknown said...

The Capn seems to be consistently posting "counterpoints" in his tumblr including a link back to your blog just the other day and a link back to djconnel's blog today:

http://veloclinic.tumblr.com/

Dave Clarke, Ph.D. said...

Dear Mr. Simmons,

Thank you for eloquently stating a critical issue that is arising in science, caused by the increasing prevalence of internet-based citizen science. The advantages of "internet science" include the speed of publishing and its near universal accessibility with respect to both generating the knowledge and being able to access it.

The problem, as you indicate, is that internet science does not hold the knowledge published to the standards that traditional science does. Specifically, traditional science requires peer review and publication in formal journals or conference presentations. Only then is the knowledge considered "generated" and proper credit attributed to the authors. This process, which is often lengthy and drawn out, keeps the signal-to-noise ratio as high as humanly possible.

Internet science, in contrast, does not hold authors to the same standard. Anyone can publish anything on a blog. While opportunities exist for comments and debate, the author can easily censor criticism. The reader must also filter through the valid comments from those that are not. Furthermore, the debate about a certain topic can get scattered across many internet sites, which makes coherent following of the debate very difficult. Finally, the authors have little requirement to take professional responsibility for their publishing or for ethical and cordial conduct, like they would if they published in formal academic forums.

I too find it unfortunate that Dr. Tucker now exhibits these pitfalls of internet science. It will be fascinating to witness the effects of internet science on the sport of cycling and, more broadly, how traditional and internet science will merge in the years to come.

Powerfulcycling said...

Science in Sport has "moderated" my comments that disagree with their views, or introduce clear evidence that counters their opinion. While they might believe that they are doing the world a service by pursuing drugs in cycling they continue to ignore the issue of drugs in athletics and marathon running (along with comments that include evidence of drugs in running). Great to read that I'm not alone.

Ross Tucker and Jonathan Dugas said...

Alex

Hi there. Thanks for the articles, and for engaging in the discussion. that is, after all the point.

If I may begin with a response to your feeling that you are being 'censored' - nothing could be further from the truth. The only reason you feel your comments are being moderated is because I started getting overwhelmed with spam, changed settings via Disqus and now I just don't have the time to engage in the comments on that forum. I have tried very hard to move the discussions over to Facebook, where you'll see that I respond to as many posts as I can, even the critical ones. Similarly, the discussion is now on Twitter.

That was a conscious decision on my part because I cannot sustain all three discussions. Remember, this website is a hobby to me, as much as yours is to you. You can tell that from the frequency of posts per month - sometimes I get more done, sometimes less. But you'll also note, if you go all the way back to 2012 and the London Olympics, that I haven't responded to any Disqus discussions. I don't have the capacity. So I apologize if you have felt censored. I'd point out that I have ended most of my most recent posts with the 'encouragement' to follow on Facebook and Twitter to have more discussion, because I wanted to move it there.

Further, I've also noted (and taken advice on this) that most authors don't respond to comments to their articles (see Sports Illustrated opinion pieces), so I feel it is reasonable that I don't get involved there, particularly since it is a hobby to me too.

Regarding some of the other comments here on the site, I've been very clear about the limitations of the method throughout this Tour. I have, at every possible occasion, emphasized that these methods are imperfect. I have emphasized, at every possible occasion, that they need improvement. I have never once said that I seek proof.

Continued below...

Ross Tucker and Jonathan Dugas said...

(Continued from above)

So to Duane, I'm sorry you don't feel they analysis adds anything, and that I seem out of my element. I wish you'd be more specific. The criticism that I'd rather have a thousand eyes on mediocre analysis is harsh, but then of course I would say that. Again, I'd like to know what is mediocre, given that I've never once stepped beyond the bounds of what the numbers can suggest, given their acknowledged weaknesses.

Finally, to Andy Coggan, every time we've had a debate, dating all the way back to maybe 2009, your insults are personal. The first engagement we ever had was an insult where you labeled me a newly minted scientist not worth listening to. I hear and see the same now, but I haven't the energy to fight these battles with you. Similarly now, you accuse me of wanting to be known rather than be right. Your opinion, which you're entitled to, but I believe it to be grossly unfair, and as I'm sure you're aware, I try to maintain a separate scientific career which I'd like to believe has contributed to the field we both work in. I have a number of very credible scientists who I think appreciate what I've contributed to the field, and even this analysis. If you don't believe that, well, there's little I can say in response.

Finally, to Powerful cycling, I've not moderated anything. As I've said to Alex, I just haven't addressed the comments to articles. I should actually disable comments to articles, because it creates an expectation that, frankly, I cannot meet. I don't have the time or energy in amongst other commitments. So I'm sorry for doing that, creating that expectation.

With regards to the issue of "pursuing doping", you do realize that the purpose behind this analysis is in fact to provide evidence that cycling is clean? My message is one of optimism for the sport, not cynicism. I believe in skepticism, but I think the overall tone of the last four years has been optimistic.

With respects to marathon running and doping, if there are specific stories, please let me know. When Tyson Gay and Asafa Powell were caught a week ago, you may have noticed an article where I commented on it, so it's not like it's being ignored. Again, I apologize if you feel censored, or that your comments are being moderated. It's just that I haven't the time - those articles i write take a few hours, and I have at times spent entire days responding to comments. It just can't be done. Unless some kind soul would like to pay me a salary to do it! That's a joke.

Continued below:

Ross Tucker and Jonathan Dugas said...

In any event, I am sorry if you are disappointed by anything written recently. I stand by all of it, because I have tried very hard to be balanced and provide a "non-extreme" viewpoint to the Tour. I have disclosed every error, I have freely admitted the imperfections. I do however believe in the concept of performance analysis, and if I may use the following example to illustrate:

When I work for the SA Rugby 7s team, we analyze our opponents. We can tell you what New Zealand TEND to do off line-outs and penalty moves, we can tell you how Samoa TEND play differently to Fiji and Wales. We can predict what England are LIKELY to do off kickoffs and scrums.

Note the emphasized word - TEND, LIKELY. It's about probability. So when we send the players on the field, they have some knowledge, as a result of analysis, of what to expect. It does not mean we guarantee that New Zealand will attack down the narrow side, and it doesn't mean Samoa will always play touchline to touchline. But it gives us some indication of what to expect. The players are still given responsibility for assessing each situation.

The same is true of this analysis in cycling. The model, which is so readily criticized, is not being put out there as proof (I've only said this a million times. Maybe I should put it in caps - IT'S NOT BEING DONE TO PROVE DOPING)

The process here, is to analyse performances with the view of finding trends that suggest credible performance or doping. Over time, as more data comes in, we will get that. Will we get a standard of data that court of law would accept to send a man to jail? No. Will we get a standard that proves doping? No (but not even SRM data would do that, as I've literally written on the website).

To all those who dismiss as worthless the analysis because it is estimated, you are basically choosing to know nothing because you can't know everything.

You are like a blind man who says he would rather stay 100% blind than be given 50% vision. And before you tell me that 50% vision can be misleading and inaccurate, I'd remind you that you still have other senses - touch, smell, and hearing. Therefore, with 50% vision, and the sensible use of other senses, you are better off than 100% blind with those senses. Why disregard everything because we can't know it with 100% certainty. We may not be able to make out details, but if we show some sense, use those other senses, we can still find considerable value in the process.

It's astonishing to me how many people are extreme on this one. Evolution of knowledge is the objective, and you are all extinct. Evolution gets bored in the absence of uncertainty.

People who dismiss this because it's not perfect, well, next time you turn the key on your expensive car, consider how iterative that process of development was, how the luxury and performance you experience now began. If Henry Ford had refused to build a Model T because it didn't perform a certain way, we'd all be riding horses!

So to wrap up (I'm sorry this is so long - but now you see why I have forced myself not to get involved in the discussion to my articles - I would get no work done), what I'm saying is the following:

I stand by every word written. In fact, I'd emphasize the words written, in particular the concepts of acknowledged uncertainty, and the fact that we cannot accuse or prove guilt based on performance.

But I stand by it, because I want to use this to add insight, to push a new approach to performance analysis in cycling. This is not about being popular (Andy - if I wanted popularity I'd simplify the message and tell people they are clean and be a fan, just look at how unpopular this has made me). This is not about censoring you (I apologize if I have created this expectation).

It is just a process of trying to evolve. And I'm proud of what we have done, collectively. And I'm confident that many a good scientist back this and will vouch for me on it.

Regards
Ross

Ross Tucker and Jonathan Dugas said...

Just a final point, something I want to emphasize again.

The articles are written to provide a) My views and opinions. I've always been very clear on this. So to all of you who disagree and think I'm out of my element, I'm tempted to say "take a number", because I heard teh same when trying to discuss Pistorius, Caster Semenya, Lance Armstrong, match-fixing in cricket, my nation's Olympic HP programme (or lack thereof).

It's pretty clear from the website that the purpose is to add opinion and what I hope is insight, not to seek agreement.

It's also clear that I want to initiate discussion and debate. But that is not the same as promising that I'll avail myself of that to every single comment. I simply cannot do it. I try my best, but had to make a decision not to spend 4 hours a day trying to reply. The result is what you, Alex, have perceived as me putting science out there not wanting "peer review". That's not fair, and it's also untrue. My purpose is to have the first word, or perhaps A word in the debate, and then others can continue. I have seen your articles, so i consider this a success in this case! If the media pick up on it, then provided they report on it responsibly, that's good for the sport, good for sports science. That's why personal insults like Coggan's are so unnecessary - this process, of looking at performance, is opening up a world of insight that many would not have seen or thought of before. And that's progress.

I just can't be the one who responds, I'd lose my job. So I do so on Facebook, because then I can respond when in traffic, or waiting in a queue, or for a meeting to arrive etc. Censorship has nothing to do with it, it's an over-reaction, that I have to say is typical of the extremism that has characterized the last two weeks. It is exhausting.

Unknown said...

The polemics and history of various interchanges aside. I think three points deserve to be considered:

1) It is possible to make some ball park estimates of the oxygen cost of cycling with watts data.
2) These estimates can be considered in the context of how long various power outputs might be physiologically realistic, the effects of a long day or days of riding on what is possible, and the effects of altitude on what is possible.
3) If more biological data on blood parameters is a good thing, it seems to me that the teams and governing bodies might want to post more watts data as well.

That having been said, perhaps a lot of this discussion would be moot if the data were readily available. Why not let the data speak for itself?

Mike Joyner

Alex Simmons said...

Hi Ross,

thanks for responding. I'm back from being away a couple of days (our team time trial championships - well done to my lads for getting on the podium).

I think the confusing thing for me with the comments moderation was four days earlier my comments appeared, and since then they don't, yet other's comments were and still are appearing. So for some reason there was some selective moderation of comments by the system.

I did say to my own Facebook crowd that I was giving benefit of the doubt on that front, and I'm aware of the issues around moderation headaches. Spam and trolls abound. I accept that it wasn't a deliberate decision on your part.

I hope that I was reasonably fair to point out that in general you do try to emphasise the issues with the flaws in estimation of power in the text of what you write, and that all we really can do is inspect trends.

Whilst that's the case in general in your commentary, I still note three things that run counter to this:

- every time these pVAM & dpVAM numbers charted there are no error bars or ranges for power estimates given, implying a level of precision that doesn't exist. The sensitivity of the error due to a key unknown input is large.

- individual performances *are* being highlighted, and as a result an individual implication *is* being made, whether or not that's intended

- the very use of the "dp" prefix is specifically there to highlight doping suspicion of an individual performance, when the error is such that doing so is misleading.

Hence why I think it makes sense for the scientists such as yourself who choose to publicise and lend credibility to such models either perform a proper scientific critical analysis of this methodology, or if you choose not to do such analysis, then present information in a manner that highlights the limitations of showing an individual performance (e.g show numbers with error bars).

I'm no scientist, yet I can see basic flaws in it, so I just think it could use better scrutiny, and until such time as it has had the blowtorch of scientific rigour applied, I think you would do well to be careful about posting charts with individual numbers providing a false sense of precision.

Indeed there is even issues with validation of climb times, lots have suggested many of the pre-2001 ADH times are not correct, especially Pantani.
...

Alex Simmons said...

...
Power tells us what happened, but not why.

Every time the W/kg estimates are posted up on the clinic, the various methods can't even agree within 0.3-0.4W/kg, and they haven't considered the wind, so not sure they tell us what happened all that well either.

I'm also not convinced on the arguments about plausible limits to human cycling performance, but that's my opinion. Too many variables in play to drawing a hard line in the sand.

Also IMO this dpVAM stuff is a red herring that's not really helping the anti-doping cause, as it focuses people's thinking on limiting performance rather than methodologies that do reduce/removes the impact of doping, i.e. a focus on:
- Increased anti-doping resources,
- Improved testing and doping control processes,
- Better forensics and investigative activity/expertise,
- Better culture (e.g. removing the worst influences from involvement at all levels).

Hearing journos now regularly quoting W/kg numbers and proclaiming sans-doping plausibility is just nonsensical (either way). Lemond isn't helping in this respect.

As for the running & Rugby's version of touch football, I don't really read those much, of limited interest to me personally.

I followed the Pistorius stuff more closely and with keener interest given my personal experience of being both an able bodied and an amputee athlete *with power meter data from before and after my amputation*. This has given me a unique insight into the performance impacts of such a physical disability.

Anyway, better that we all have the opportunity to discuss, debate, criticise and learn, than not. Makes for interesting pub chat fodder at least. Thanks for engaging.
Cheers, Alex

Andrew R. Coggan, Ph.D. said...
This comment has been removed by the author.
The Sports Scientists said...

Hi people---coming a bit late to this discussion, but it's a slow Friday at my day job. ;)

Just one thing on the accusations of censorship. The only comments we have ever (EVER) deleted from the site are spammers.

We have never once deleted a comment otherwise. Not once. On any post. Ever.

We do use a feature on Disqus that sends comments with links automatically into moderation. This requires us to approve these messages, which we do, sometimes immediately and sometimes not so quickly.

Andy, I see there was a comment from you last week that was sitting there. It should be posted now.

We totally accept the gripes about internet science and all the other things as everyone being entitled to their opinion.

I think somewhere up above someone said scientists have thick skin, and we agree. . .However this stuff about censoring, it's just not on, it is something we have never engaged in and never plan to.

Jonathan