Geometric Deep Learning
An attempt to unify things
Hey! I hope you had an amazing Diwali. I am appearing today to wish you Chhath and talk about Geometric Deep Learning.
The essence of deep learning is based on two principles:
Feature Learning - often Hierarchical
Learning by local gradient-descent - Back-propagation
While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world.
We can understand these phenomenon when we read about topics like Gaussian Annulus theorem, Random Projection theorem and Johnson–Lindenstrauss lemma to understand more about dimensionality and the tools needed to do learning in higher dimensions.
Anyways, the objective of Geometric Deep Learning is to take advantage of the fact that most of the tasks/functions which are needed to be estimated, have an underlying regularities through unified geometric principles that can be applied throughout a wide spectrum of applications.
The more we understand the irregularities/challenges present in ultra-high dimensions, the more we would be able to appreciate the motivation to study geometric deep learning.
Geometric Deep Learning is an umbrella term we introduced in referring to recent attempts to come up with a geometric unification of ML similar to Klein’s Erlangen Programme.
Exploiting the known symmetries of a large system is a powerful and classical remedy against the curse of dimensionality, and forms the basis of most physical theories. Deep learning systems are no exception, and since the early days researchers have adapted neural networks to exploit the low-dimensional geometry arising from physical measurements, e.g. grids in images, sequences in time-series, or position and momentum in molecules, and their associated symmetries, such as translation or rotation.
Geometric Deep Learning is an attempt for geometric unification of a broad class of ML problems from the perspectives of symmetry and invariance.
Throughout our exposition, we will describe these models, as well as many others, as natural instances of the same underlying principle of geometric regularity. Such a ‘geometric unification’ endeavour in the spirit of the Erlangen Program serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.
Please feel free to reach out to me if you need help with data science or data engineering at your organisation, I would be more than happy to help.
If you are liking my articles and want me to write more then feel free to buy me a coffee! 😁
I hope that you enjoyed this read, if so then please share this article with your friends, let’s build a solid community. I will be back next week with another well thought/researched article delivered straight in your inbox. See you!