Show of hands who plays Fantasy (American) Football please… I know I do. And so do 19 of my colleagues here at BBK. (That’s 23.75% of our workforce). We form two separate leagues, each consisting of ten teams apiece, where we battle for FF (Fantasy Football) supremacy.
Our leagues are set up using the excellent, free Yahoo! Sports Fantasy Football online application. It really is quite astonishing how easy to use the system is, and the wealth of player information it places at your fingertips.
Essentially, FF is a statistical nerd's utopia.
For those of you who are not familiar with Yahoo…there are points given for a concise set of offensive occurrences at different skill positions. Yards gained (or lost), touchdowns scored, fumbles (and recoveries). All result in fantasy points for each player. Also, the defensive units can score fantasy points for restricting points scored, intercepting a pass, sacking the quarterback, and so on.
Each week’s matchup against a new fantasy team in your league creates a fascinating delve into a statistical game of chance. You can only work with / play a set number of players, each of whom was “drafted” into your team at the start of the season. Do you play this receiver or that receiver? Who is covering them in defense? Are they carrying an injury? Can I trade for a better player that has greater upside potential?
It is a bewildering choice, but it is all laid bare by the constant update of statistical projections for each player and defensive unit. Through a complex algorithm, the system projects how each player will perform next week. It’s compelling because there seems to be the power of a multi-billion dollar organization behind it driving the numbers and offering clever interpretations of the wins and losses in an eerily “human” postgame breakdown of the cut and thrust of the contest.
And yet, despite the fact that this commercial juggernaut has all this industrial power behind it, it is NEVER right. I know this and can state it with confidence because I use it regularly to pick my team. And it has missed the projection. Every. Single. Time. (And by quite a wide margin, I might add).
This of course is generated from complex systems that are used literally by hundreds of thousands of people, sometimes with high stakes gambling attached. That’s millions of transactions occurring every week, scrutinized by many, many people. And it’s pretty much always wrong.
In our clinical R&D industry, clinical trial feasibility models and site selection algorithms operate in kind of the same way. They are expected to predict future performance of clinical trial sites based on data, except they are used by far fewer individuals and with much less data fed into them. Often that data feed is also speculative (such as number of patients screened per month which is guessed by an interested investigator) as opposed to something objective in Fantasy Football like “receiving yards” which is scrutinized by millions of football fans. The financial markets are the same – they have trillions of interactions daily, with the whole world watching. And still no one can predict what will happen day to day.
Doesn’t sound much like a successful recipe does it? It isn’t – simply because it cannot be. The quality and quantity of the data is simply not good enough. Even if it were better, it could never be a crystal ball. But with each study, we put our faith in them because it seems compelling enough – just like the old Fantasy Footy recommendations.
For certain, it can help set a course that seems more assured or gratifying at the start of the study (or our fantasy football game) and for that there are many reasons to rely on certain data for site selection. What else is there? That’s more difficult to answer.
For me however, I think that what happens during the study or the game is more interesting. In most cases, the game is going to be played regardless of the outcome. How can we use data and specifically human interpretation of that data to improve the situation that we find ourselves in?
For Fantasy Football, I’m nothing more than an onlooker once the board is set and pieces are in motion. I cannot help the players perform their duties on the field and I cannot influence the outcome of the games in any way. But for clinical studies we absolutely can influence the rates at which patients are accrued – regardless of however flawed the feasibility assessment may or may not have been. To me that’s exciting, because I see it work every single day.
Essentially, we have become the Peyton Manning of patient recruitment. Only better looking!