How do you imagine the future of data sharing in healthcare or research?

Each day more and more data is collated. This data could have huge effects for healthcare and research if properly used. But will this knowledge be harnessed and used to its biggest advantage?

Will this knowledge be harnessed?

Will it be used to its biggest advantage?

There is a growing potential in sharing medical data but there is still much work to be done in gaining people's trust and encouraging them that it's the right thing to do: it's for our own good and that of mankind.  Some people may be naturally less inclined or have good reason not to do so.  And rightfully so since there might be a few immediate drawbacks like getting scooped in research or illicit use of someone's medical data to discriminate.  We have a fear of being singled out, of being watched.
Are we willing to accept that data sharing might have some negative effects at the timescale of the individual for the sake of the greater good?
We need to encourage sharing of data, we need to promote the moral trust to think rationally and altruistically.
But it is our responsibility to share the benefits of our research to support greater public involvement to prove that the benefits outweigh the drawbacks.  If we can show that our data sharing is saving lives in the long run.
Convincing people is hard but if we share the good coming of data sharing in healthcare and research then we will be able to distil this positive message and we can guarantee a positive feedback loop.

Building trust takes years but only moments to tear down.  Internet companies such as Facebook and Google have deceived us.  Already people give out a lot of information unwittingly.  We would like to avoid this.
Scientific journals have deceived us: Eg recent articles about most research findings being FP.  Scientists are often much more cynical about science than people outside the field.

Ideally we would like people to know that there are giving information, we do not want to deceive them.  We should make it clear what their data will be used for.

For example Google is able to trace and predict epidemic based on search terms.  That's a good use of our data, one that benefits mankind as a whole.
However we don't like people making a profit at our expense by selling our data to third parties.

But if data is considered private then there will always be market for it.  Better an open market.

Nothing worse then breaking a promise: promising confidentiality or giving the illusion of confidentiality to later sell off the information to a third party.

If everything is in the public domain to start with then there is no need for this black market.

However this is where information sharing can help. Sharing information has had a huge psychological impact on our society.  I believe,  perhaps quite naively, that we are promoting a more open, honest and tolerant society, one were we have nothing to hide.  

Thinking in terms of utility. Bentham: the greater good.

Possibly we need to distinguish population data from individual longitudinal data.
We are scared of being watched.

Promoting scientific honesty, thinking in terms of utility and greater good

It's ok to be wrong.  And it's ok for things to be incomplete.  In fact sometimes we learn new things from a person's draft, about their way of thinking that are not obvious in the final product.  But it's in our nature to fear being single out as being wrong, as an outlier.  In fact we (collectively) learn more from when things don't work then when they do.  The important thing is to learn from our collective mistakes.

Simple when something works we don't need to fix it and so are less motivated to understand how it works.

I believe that the Wikipedia model shows that objectivity and scientific honesty prevails amidst dialogue. Open data encourages scientific dialogue. Complete transparency . Compare the performance of an athlete who trains by himself and one who trains with others.  There are many cases where competition drives progress but also cases where competition distracts from alternative roads less travelled, inhibits diversity and encourages lying and deceit.

Massive parallelisation of collection and analysis

It is pretty clear that everything needs to be parallelised/distributed.
Massively parallel collection of phenotypes.
Collating data efficiently.  Preserving anonymity.
Genetics risk factors will be updated on the fly.
Parellelisation makes consistency harder.

The objectivity of data, unlabelled data
Putting labels on things can be as useful as it can be destructive.  We've learned not to label ourselves, now we need to learn not to label our data.  The issue with data sharing is not so much the data itself but the interpretation of it, the label that comes with it which can be misleading.  For example products when you buy a product in the supermarket it comes with a detailed list of ingredients but not with a risk factor.  A recent example is 23 and me who were sued over their diagnostics.  It's one thing to collect the data, it's another thing to interpret it.  Over-dramatisation carries the risk of causing mass-hysteria.

 I believe we need to encourage data sharing without the interpretation of the data, or at least provide several interpretations of the data.   Every dataset should come with a disclaimer stating that the data is provided as such, that it came off the machine x, has undergone the following steps of QC.
There are many levels of raw data.

Many possible data labels, no label is permanent, many data interpretations
Drowning in data, starving for information
In science there are often many competing hypotheses which, depending on the the data, have posterior probabilities of being true.  In the light of new data, these posterior probabilities might change or new hypotheses might emerge.
This is the increasingly popular Bayesian way of thinking whereby our beliefs are continuously updated in light of new data.

Although we have learned a great deal about genetic data in the last 20 years there is still a lot we don't know.  We have high-level conceptual models of genes, of how the immune system works, of how cancer metastasizes.  But in some cases we still have very little predictive power of how a disease will evolve, how efficient is a vaccination.

Jumping to conclusions to diagnosis for the sake of impact is one of the biggest problem  we are facing in research, lack of objectivity. So called expert judgments overuling objectivity.  Dismissing competing hypotheses, oversimplifying before enough evidence has been gathered, leaps of reasoning, favouring elegant solutions. People blindly following the opinion of so-called experts.  We need a minimum of trust which is the point of peer reviewing to establish of knowledge base.  Some journals are more trusted than others.

Keeping our options open
But what if we lack the expertise to analyse the data?  When do we chose to suspend our disbelief?  A more mundane example, say I bring my car to the garage for a road test.  Do I trust the mechanics diagnosis?  Do I consult a second opinion?

As long as we are allowed to question and have these options at our disposal.  We don't own the earth, we don't own our genetic code, we are merely borrowing it from future generations.
We are only transiently here but we have the chance of contributing to something that may outlive us all.

My naive hope is that the future of data sharing is a much simpler than the present. 

I believe (albeit naively) that humanity has come of age, that our tolerance, understanding, and scientific openness has reached a point where data can be put in the public space without fear of confidentiality, judgement or reprisal.

That we become as open about our genetics and medical problems than about our thoughts, religion, sexual orientation.  That these things don't become newsworthy anymore.  If anything genetics shows us that we are all exceptions, we all carry minor alleles which distinguish us from everyone else.  We are all genetically flawed in some way.  It's normal to be different, the mean doesnt' exist.

I see a future so simple that data sharing is no longer a newsworthy question, but my job is not to predict it but to enable it.

Around nineteen eighty-four

George Orwell’s (Eric Arthur Blair) “1984” can be interpreted as a very cynical story and fervent criticism of totalitarianism.  Yet it at it's heart lies a deeply philosophical theme, one of existentialism and solipsism.

  In the year 1984, the world is divided in three totalitarian superpowers who wage constant war on each other for fictitious reasons.  These three superpowers are Oceania (America, Australia, most of Africa and England), Eurasia (Europe, Russia and most of South Asia) and East-Asia (Japan, China, Mongolia, most of south-east Asia) are all oligarchies with an omnipotent and ubiquitous “Party” in power.

  The story of 1984 is set in Oceania which is under the control of “Ingsoc”, the english socialist regime, whose governing body is known as “The Party”.  The will of the Party is personified as “Big Brother”, a supposedly inspirational leader whom every member of the Party must revere and love unconditionally with a certain degree of fanaticism.

  Big Brother is one, he is ever watchful, he is powerful, wise and uncompromising.  He plays God in the microcosm of Oceania for he defines people’s reality, modifying the past as he please by altering records and destroying evidence.  People must see, think and remember what he wants them to.   Although the regime cannot directly control people’s minds through telepathy nor read people’s thoughts without observing telltale facial expressions, they manage to confine their thinking and reasoning as to deter the slightest impulse, the slightest thought of rebellion against the oligarchy.  And furthermore, they instill an incessant feeling of gratitude towards the Big Brother regime fuelled by hate for the enemies (Eurasia or Eastasia alternatively) of the regime.  The means by which this is achieved are purely psychological: constant surveillance, mindless propaganda due to the eternal state of war, the Spies, the Thoughtpolice, Crimethink…

  Paranoia is no more an illusion.  Any “unorthodox” behaviour is swiftly suppressed and the “culprit” is erased.  One’s existence is not only futile, it is irrelevant.  Since not only one's existence is denied, all trace of its is destroyed: it is as if he never existed.

  As all newly born members of the Party are taken in hand and swiftly brainwashed by Big Brother, the ideas of rebellion against the regime or freedom of will are not even conceivable by newer members of the Party.  For them, the Party, Big Brother are eternal and define reality.
  The mere concept of overthrowing the Party of destroying Big Brother can only come from the minds of those who know otherwise, who trust their own judgment over Big Brother’s deceit.  They are incapable to believe in what they do not see as true.  The Party adheres to the solipsistic view that there is no objective truth: reality is an illusion which exists in one’s mind and nowhere else.  Therefore, before man, before the regime , there is nothing, and after man, they will be nothing.  The Party chooses to regard as truth what one convinces himself of being true and nothing more.  Reality is therefore defined by the Party, and all must “double-think” their way into seeing reality as Big Brother their way into seeing reality as Big Brother choses.  Their opinion must be subdued to that of Big Brother.

  However this process is very difficult for those who are older than the regime.  Winston Smith, for instance, the “hero" of the story is not able to and unwilling of tricking himself into double-thinking.  He, unlike most, is not stranded in the world of Oceania and Big Brother but has a link with the outside.  Winston Smith, has a precious link with the past for he possesses persistent memories and which are in some mysterious way more real than the present.

  Winston knows that reality might be perceived by one’s mind but is actuality exterior to it and exists in itself.  Man might consciously chose what he wants to see, he might chose to live in a world of illusions, but reality will eventually catch up with him and the truth will be uncovered in such a way that will not be able to convince himself otherwise.

  When that time comes he can either embrace the truth and repent or he can disregard it because it is against his interests.  Whatever his choice, it is from this moment  impossible for him to completely ignore reality.

  Winston Smith is one of these people, who chose not to double-think, not to believe that they can be two realities co-existing and that the Party’s is the right one, the true one.  He believes that two plus two make four and will always make four whatever the Party says.  That he is dangerous for the Party, because has higher faith in his judgement, his reason, than that of the Party.

  Eventually Winston Smith is betrayed by the man to whom he had confided, a supposedly rebel like himself who has ensnared him in a web of deceit and false hope.  He is tracked down by the Thought Police and the so-called “Ministry of Love”, a giant prison, where he suffers intense torture.  The only objective of the torture is to make him love Big Brother more than anyone by forcing him to betray everything and everyone he loves.  Winston resists as much as he can but his demise is the dreaded “Room 101” where he faces he is greatest fear.

  And so the individual is nothing against the regime:  he is crushed, brainwashed, and finally dies without the slight remainder of his past life, the smallest proof of his existence.  For in nineteen eighty four, the regime defines not only reality but existence...

Why C++ and Perl are dead

I would like to start by saying never follow people's advice, the greatest adventures in life come from disobeying.

But since this is an opinion article 

The question you should first ask yourself is why?

Maybe you need these languages for code someone else wrote>

What Do you Mean by Efficient Code?
You need more efficient code

code which runs faster?
code which uses less memory?
code is easier to write?

I think the key lesson to remember is that your time is more precious than the computer's time.


Experimental Languages
Both C++ and Perl are ill designed languages which were both designed as "macro" languages to leverage existing programming languages.
I think we could call them experimental languages or maybe prototype languages in their design.
They are organic languages which have grown according to need and lack a grand design
This is precisely the key to their success, they have evolved like a genetic mess to fill in the niches of need.

The point of Perl was to avoid writing long shell and awk programs whereas C++ was at the time the best way to combine the efficiency of C and the object-oriented properties of slower language like Smalltalk.
There are many interesting ideas that came from these languages like templates in C++ and regular expressions in Perl but these have now been taken further.

Lessons Learned from These Languages
In my opinion the only practical reason one should learn these languages is to maintain existing code.
Studying these languages is also a lesson in what works and what doesn't and the principles one should adhere to in designing a language.
You can write your C code to work like C++ code.
You might also find that once you've learned these languages they don't stick in you memory.
The Ruby language was designed with the principle of least surprise or intuition.
You need only remember a few key points and then you can derive everything from first principles.
Which is why language which have an interpreter are much easier to pick up

My Advice for What It's Worth
If you think you need to learn C++, learn C first.  Once you've learned C you will realise C++ is just a coverup for C.
If you think you need to learn Perl, learn Python first.

"Power law: a universal pattern" or "why rare things are diverse" or "is new rare?"

Power law is as universal as the exponential function, you  might have heard of 90-percent-10-percent-rule.  Maybe that would be a better name for it.  The negative exponential law.

$\text{f(x)}={\text{x^}}^{-\text{λ}}\text{for x}>{\text{x}}_0$

Where λ is the exponent of the power law.
An exponential decay process
This pattern arises naturally from count data.

First time I heard of it was in the context of software development whereby 10% of the code takes up 90% of the time to write.  This tends to be the case because programming bugs can be very difficult to discover and when they are found it is usually that the error originates from a single line of code.

However I am now seeing a similar sort of pattern emerge in the context of biological diversity.
For example if we consider the cells in our blood, the great majority (maybe up to 99%) of them are as you would expect, red whereas a small percentage of them are white.
However there is much greater polymorphism (fancy way of saying diversity) in the minority of white blood cells than there is in the red blood cells.  Red bloods cells pretty much all do the same thing whereas white blood cells, which are part the immune system,  are incredibly functionally diverse.
Another example from biology is when looking at polymorphic regions of DNA across populations. If we consider a region across different people the majority of people will probably have the similar sequences but there is a small proportion which will account for most of the diversity.

This is why biology is getting more and more complicated because we are constantly opening small Pandora boxes of diversity as we dig deeper and deeper.  New discoveries are always by definition rare.  This is something that might be obvious to some of you but wasn't to me.

The logic here is they were not rare then we would have found them earlier.
Either there is some intrinsic property for rare events to be heterogeneous because they originate from diverse sources or maybe it's simply because patterns only emerge once you have a sufficient sample size.

And that is I believe a key insight:
Rare things will always seem more heterogeneous because patterns only emerge once you have a sufficient sample size.  As an example if you generate number from a known distribution say the normal distribution you need a sufficient sample size before you ascertain with a certain degree of certitude what the nature of the distribution is (until then you will have competing models in your mind (think Bayesian)).  Also groups can only formed once you have seen something substantial different.
For example if was to place points randomly on a plane you might no see any pattern but say i placed a point much further away from the center of group of points then suddenly because i have increased the scale (zoomed out) you would tend to want to cluster those original points and the new point might be classified as an outlier from the existing random generation process or become a cluster of its own if.
 
An example would be:
Say I was to generate random numbers from a normal distribution with a mean of 0 and a standard deviation of 1.  Only once a sufficient number of points had been generated would you be able to assert whether there was any relationship between these points.  A more visual example is say you had customer coming to sit at invisible tables in a restaurant.  Only once you would have observed enough customers would you have any idea of the number, shape and capacity of these tables.  In this particular context would probably want to make some assumptions about the tables.

In networks, the edge count distribution in scale-free networks follows a power-law.  Which means that there are a few hubs of high connectivity and many nodes of low connectivity.  The hubs can be thought of as clusters.

The distribution of wealth follows a negative exponential distribution.  Or at least used to but is becoming now more multimodal.

The distribution of city size  follows a power law.
The tail of the distribution will get longer as more cities pop up.

Mark McCarthy said at GCD2013: "taken individually variants are rare but genetic variation is common".  This is perhaps  a bit of a tautology but diversity implies that things are different in unique ways meaning that taken individually they are rare, much like the long tail of the negative exponential function.

Another impact of new allele discoveries is that they decrease the overall frequency of older alleles.
This decreases the MAF and consequently the odds ratio since the MAF in the cases stays the same but the OR in the controls decreases.

What about the diversity of alleles in common genes vs the allele diversity in rarer genes?
For example KIR3DL1 KIR3DS1?

The Gini index is measure of statistical dispersion.

And yes you will probably spend 90% of your time looking at the rarest 10%...

Stumpf, M. P. H., & Porter, M. A. (2012). Critical Truths About Power Laws. Science, 335(6069), 665–666. doi:10.1126/science.1216142

Does Less Privacy Promotes a More Trusting, Honest & Tolerant Society? Can intolerance be tolerated in a tolerant society?

You might have heard the quote "privacy is dead - deal with it", I first this heard back in 2004 when I was studying computer science at university.

Now days privacy is becoming less and less of a concern because people don't think it's a case for worry.  But are we treading worryingly close to the ominous Big Brother society.  Some Orwellian aspects of which are already real.

This raises the question of why do we wish to keep things private in first place?
Are we scared of being morally judged or even worse, persecuted?
Should we care if others are making a profit by selling our own private details?
How do deal with paranoia?

Should anything be private anymore?
From a philosophical standpoint one may argue that the need for secrecy stems from distrust and fear.
An open society is a fearless society and perhaps a more tolerant society?
Being quick to judge someone's character is something we should move away from, we want to move away from such biases and assess people for their worth.  Give people the benefit of the doubt, or if we are to have a bias, then at least adopt an optimistic one.

If everyone was truly honest and considerate then they would be no need for privacy but then such a society would require everyone to hold the same faith in others - a sort of moral communism - a state of equilibrium which may easily be disrupted by the most minor glitch in trust or the minority who take advantage of this utopia to fulfil personal goals.  If anything we know that true equilibrium is an illusion and is in fact a seeming state of stability can be achieved by a dynamic system on a lower scale.  Much like how solid objects which follow deterministic trajectories at our scale are constituted of vibrating atoms at a lower scale which follow stochastic movement.

But in the end is it not in our nature to take the initiative and to lead?  Provided we are given the illusion that we have free will and are following our selfish ideals we will be happy.  Let people do as they wish but trust that they will do good.

I am not saying we should not have any strong views one way or another, it is important to stand up for some things that we hold dear.  We all have our buttons.  Ultimately, we need to believe in something to give a sense of purpose (at least I think most people need or are happier with a sense of purpose in their lives).  Yes purpose is meaningless but it is useful to think that way to get things done.  The only thing one should not tolerate is intolerance.  But is the law intolerant?  Why is it ok to ridicule some people and not others?

Perhaps this a naive view but I do feel we need to break the cycle of distrust somehow and not let it escalate...

In my view if everything is public then that's fine.  If we are going to decide what is to be kept and what is to be made public then this may raise concerns.
It's even more of an ethical concern when some things are deceitfully labelled as private when in fact there are not.  Might as well come clean and admit than nothing is fully private.

Disclaimer I have never been the victim of cyber bullying or defacing
A counter point "cyber bullying" for no apparent reason except for the stupidity of group mentality: https://thenib.com/the-internet-s-most-trolled-cartoonist-91a92d9b7585