Category: Deep Learning

Why The Machines Won’t Win – This is a story of human ambition.

 

I have been working on the bleeding edge of AI technology for more than 4 months now. I am working on a project that seeks to further the state of the art in conversational agents and I have been critically reading and critiquing research papers published in top tier conferences like NIPS, ICLR and AAAI. I state all of this to establish that I know what I am talking about.

Whenever I tell someone I am working in deep learning almost inevitable the conversation veers of to their fears of how they believe that technology is beating mankind, how this is the apocalyptic prediction coming true and some version of the suggestion that the progress we’ve made is good enough and we must slow down if not stop all together. A lot of this stems from the way AI advancements are portrayed by the media; ‘ Last bastion of human intelligence  falls’ , ‘Man vs Machine’, ‘How the Computer Beat the Go Master’, ‘The battle between humans and machines’.

What they do not tell you is that humans built the machine in the first place! So it is not AplhaGo that beat Lee Se-dol in the game of Go but the engineers at deep mind. This is a story of human triumph not defeat. There is no need for skepticism or cautious optimism we should be celebrating these advancements and welcoming them with open arms.

The machines are not out there to get you, these technologies are going to augment human ability as technology has been doing since the stone ages. Advancements in artificial intelligence make us smarter and more productive. Better search results help you find the information you need faster, more accurate speech recognition allows you to go hands free, more reliable machine translation helps you consume information that would have been outside your reach. Both technologies and humans are the most productive when their respective strengths are combined to give rise to an unstoppable force. Machine have memory, humans have experience; machines have precision, humans have compassion and empathy; machines bring the tools, humans bring the purpose.

These tools combined with human ambition, perseverance and the super power to dream big are going to open doors to worlds we cannot have imagined in our wildest dreams. So the next time you hear about an AI advancement (which is most definitely going to be tomorrow, given the crazy pace of development) instead of fearing it, think of how you can use this advancement to augment your efforts in making the world a better place.

Further reading:

Google Brain AMA Learnings

Last year the Google brain team organised a Ask-Me-Anything on reddit. It is an amazing AMA which I encourage everyone to read. However in case you do not have the time to go through the whole thing, I present some of the key take aways and learnings from the AMA.

“our research directions have definitely shifted and evolved based on what we’ve learned. For example, we’re using reinforcement learning quite a lot more than we were five years ago, especially reinforcement learning combined with deep neural nets. We also have a much stronger emphasis on deep recurrent models than we did when we started the project, as we try to solve more complex language understanding problems.”

“Machine learning is equal parts plumbing, data quality and algorithm development. (That’s optimistic. It’s really a lot of plumbing and data :).“

Underrated methods:

  • Random Forests and Gradient Boosting
  • Evolutionary approaches
  • The general problem of intelligent automated collection of training data
    Treating neural nets as parametric representations of programs, rather than parametric function approximators.
  • NEAT
  • Careful cleanup of data, e.g. pouring lots of energy into finding systematic problems with metadata

Exciting Work:

  • The problem of robotics in unconstrained environments is at the perfect almost-but-not-quite-working spot right now, and that deep learning might just be the missing ingredient to make it work robustly in the real world.
  • Architecture search is an area we are very excited about. We could be getting to the point where it may soon be computationally feasible to deploy evolutionary algorithms in large scale to complement traditional deep learning pipelines.
  • Excited by the potential for new techniques (particularly generative models) to augment human creativity. For example, neural doodle, artistic style transfer, realistic generative models, the music generation work being done by Magenta.
  • All the recent work in unsupervised learning and generative models.
    anything related to deep reinforcement learning and low sample complexity algorithms for learning policies. We want intelligent agents that can quickly and easily adapt to new tasks.
  • Moving beyond supervised learning. I’m especially excited to see research in domains where we don’t have a clear numeric measure of success. But I’m biased… I’m working on Magenta, a Brain effort to generate art and music using deep learning and reinforcement learning

Resources:

  • https://keras.io/ : Keras is a minimalist, highly modular neural networks library, written in Python and capable of running on top of either TensorFlow or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
  • http://www.arxiv-sanity.com/ : Get the best of airxiv; also find similar papers according to tf-idf
  • /r/MachineLearning
  • https://nucl.ai/blog/neural-doodles/ : Neural Doodles!!