Seminar Dates
-
Abstract: Modern machine learning, especially deep learning, faces a fundamental question: how to create models that efficiently deliver reliable predictions to meet the requirements of diverse applications running on various systems. This talk will introduce reuse-centric optimization, a novel direction for addressing the fundamental question. Reuse-centric optimization centers around harnessing reuse opportunities for enhancing computing efficiency. It generalizes the principle to a higher level and a larger scope through a synergy between programming systems and machine learning algorithms. Its exploitation of computation reuse spans across the boundaries of machine learning algorithms, implementations, and infrastructures; the types of reuse it covers range from pre-trained Neural Network building blocks to preprocessed results and even memory bits; the scopes of reuse it leverages go from training pipelines of deep learning to variants of Neural Networks in ensembles; the benefits it generates extend from orders of magnitude faster search for a good smaller Convolution Neural Network (CNN) to the elimination of all space cost in protecting parameters of CNNs. This talk will also cover our recent progress on extending the reuse-centric optimization to support deep multi-task learning.
-
Abstract: Many challenges in modern machine learning involve the three fundamental problems of optimization, Bayesian inference, and multi-player games. In this talk, I will discuss how the use of second-order information – e.g, curvature or covariance – can help in all three problems, yet with vastly different roles in each. First, I will present a noisy quadratic model, which qualitatively predicts scaling properties of a variety of optimizers and in particular suggests that second-order optimization algorithms would extend perfect scaling to much bigger batches. Second, I will show how we can derive and implement scalable and flexible Bayesian inference algorithms from standard second-order optimization algorithms. Third, I will describe a novel second-order algorithm that finds desired equilibria and save us from converging to spurious fixed points in two-player sequential games (i.e. bilevel optimization) or even more general settings. Finally, I will conclude how my research would pave the way towards intelligent machines that can learn from data and experience efficiently, reason about their own decisions, and act in our interests.
-
Thursday, Oct 14 2021
12:00PM - 1:00PM - Building 9, Lecture Hall 2 Room 2325
AI for Art, Culture and Heritage by Prof. James She.
[Video]Abstract: In this talk, Prof. James She will share what is possible already with Artificial Intelligence technologies for Art, Culture and Heritage, and also the opportunities in research and potential impacts in the Gulf region. -
Wednesday, Feb 19 2020
12:00PM - 1:00PM - Building 9, Lecture Hall 2 Room 2325
Machine Learning Scan of Complete Genome Sequences of 4,500 Rice Varieties reveals Secrets in the Domestication Process of Crops by Prof. Takashi Gojobori.
[Slides] | [Video]Abstract: In the history of humankind, domestication was invented as anthropogenic evolution that fulfills mankind’s critical food demand. The domestication is simply based on mating processes and subsequent selection processes to pick up better hybrid offspring that have advantageous combinations of genomes. Taking advantage of machine learning classifier, we discovered a number of sub-genomic regions that have been incorporated in the rice genomes through production of hybrid offspring during domestication. This so-called “introgression” event is disclosed as an essential key of domestication process. This eventually leads to construction of the AI-aided Smart Breeding Platform to accumulate all the breeding histories of crop species into an Integrated Breeding Knowledgebase. -
Wednesday, Feb 5 2020
12:00PM - 1:00PM - Building 9, Lecture Hall 1 Room 2322
Optimization and Learning in Computational Imaging by Prof. Wolfgang Heidrich.
[Slides] | [Video]Abstract: Computational imaging systems are based on the joint design of optics and associated image reconstruction algorithms. Historically, many such systems have employed simple transform-based reconstruction methods. Modern optimization methods and priors can drastically improve the reconstruction quality in computational imaging systems. Furthermore, learning-based methods can be used to design the optics along with the reconstruction method, yielding truly end-to-end learned imaging systems, blurring the boundary between imaging hardware and software. -
Wednesday, Nov 20 2019
12:00PM - 1:00PM - Building 9, Lecture Hall 2 Room 2325
Towards an AI Scientist - Foundational Machine Learning for Causality inspired from and applied to Living Systems by Prof. Jesper Tegner.
[Slides] | [Video]Abstract: Nature generates data such as discrete objects, patterns, or as a temporal evolution of a particular process. Fundamentally, scientists search for an explanation; what are the causes, generative mechanisms? Historically, the existence of first principles (e.g., conservation, symmetry, action integrals) has made it possible to formulate explanatory, predictive, quantitative mathematical models in physics and to some extent in chemistry. Yet, in most areas of sciences, and life-sciences in particular, we have data, often sparse, but no equations or first principles. Can we construct intelligent scientific machines that can generate a family of predictive quantitative causal models given a set of data?
In this talk, in the domain of biology and medicine, I will address challenges and opportunities in the light of this question using work in network biology and medicine & deep analysis of cells – the building blocks of living systems. These examples set the stage for data-driven manifold learning targeting interpretable latent spaces. Secondly, explainable causal machine-learning models beyond black-box solutions in part based on “classical mathematics”. Finally, I will present recent work in a discrete setting, on using algorithmic information theory, for the discovery of causality from observations in a rule- based context. -
Wednesday, Nov 13 2019
12:00PM - 1:00PM - Building 9, Lecture Hall 2 Room 2325
Applying symbolic and sub-symbolic approaches to formalized knowledge bases in life sciences by Prof. Robert Hoehndorf.
[Slides] | [Video]Abstract: Knowledge representation is a sub-field of AI which studies how to represent information about a domain so that is can be utilized for a wide range of tasks. In particular the life sciences have created a large amount of formalized knowledge bases. In my talk, I will show how to use information in formalized knowledge bases as background knowledge in statistical analyses and machine learning. I will discuss an algorithm to construct a map from formal theories into vector spaces that preserve the model-theoretic semantics of the theories while enabling new operations within the vector space. Combining methods from knowledge representations with machine learning can be used to generate explanations and exploit background knowledge, which is particularly important in knowledge-intensive disciplines such as biology and medicine. -
Wednesday, Oct 30 2019
12:00PM - 1:00PM - Building 9, Lecture Hall 2 Room 2325
Representation Learning from Graphs: Algorithms and Applications by Prof. Xiangliang Zhang.
[Slides] | [Video]Abstract: Graph/Network embedding is to represent graph vertices or a graph itself as new low-dimensional vectors, and has been playing important roles in diverse network management and analysis applications. Learning representation from graphs faces challenges of the preservation of vertex-vertex relevance, the integration of structure and text information, and the heterogeneity of vertex attributes, and the large size of graphs, etc. This talk will introduce recent solutions to them based on reinforcement learning for intelligently aggregating a vertex’s neighborhood information to represent itself, based on variational autoencoder for encoding the uncertainty, and based on information fusion/propagation for resolving the node heterogeneity. The obtained embedding results will be demonstrated in standard applications of node classification and link prediction, user profiling, as well as recommendation and graph alignment. -
Wednesday, Sep 18 2019
12:00-1:00PM - Building 9, Lecture Hall 2 Room 2325
Imagination Inspired Vision by Prof. Mohamed Elhoseiny.
[Slides] | [Video]Abstract: Imagination is one of the key properties of human intelligence that enables us not only to learn new concepts quickly and efficiently but also to generate creative products like art and music. My research has focused on developing imagination-inspired techniques that empower AI machines to see the world (computer vision) or to create novel products (e.g., fashion and art); “Imagine to See” and “Imagine to Create”. In this talk, I will cover some of my works on these two directions and I will show how they are connected by the developed techniques and that they circle back to benefit each other. Imagine to See: There are over 10,000 living bird species, yet most computer vision datasets of birds have only 200-500 categories. Typically, there are few images available for training classifiers for most of these categories (long-tail classes). How could imagination help understand visual classes with zero/few examples? Many people might not know what “Parakeet Auklet” is but can imagine it when described in language by saying that “Parakeet Auklet is a bird that has an orange bill, dark above and white below.”. If we give this description to an average person, he will be able to select the relevant bird among other different birds, due to our capability to imagine the “Parakeet Auklet” class from the language description. I will cover in the presentation my most recent work on zero-shot learning by generating imaginary data from text descriptions, presented at CVPR18. Imagine to Create: In the short term, Creative AI has a high potential to speed up our rate of generating creative products like paintings, music, animations, etc. as a source of inspiration. I have worked on modeling Creative AI to produce art (ICCC17) and fashion. Our work on both art and fashion grabbed attention from the scientific community, media, and industry. One of the exciting results we achieved recently is that our AI fashion model was able to create new pants with additional arm sleeves (non-existing in the dataset). The surprising aspect of this design is that professional fashion designers found it inspirational for designing new pants, showing how creativity may impact the fashion industry. Our work has been featured by the new scientist magazine, the Facebook F8 conference. Our work also received the best paper award at ECCV18 workshop on fashion and art. I will show that these Creative AI techniques for creating a likable unseen circle back to benefit understanding an unseen (zero-shot learning) in our very recent work to appear at ICCV 2019 . With the success of generative models in both tasks, we got motivated to proposed a data efficient generative model, dubbed as Generative Determinative Point Processes (GDPP), at ICML 2019. -
Wednesday, May 29 2019
12:00-1:00PM - Building 9, Lecture Hall 1 Room 2322
Modern Techniques of Statistical Optimization for Machine Learning by Prof. Tong Zhang.
[Slides] | [Video]Abstract: Many problems in machine learning rely on statistics and optimization. To solve these problems, new techniques are needed. I will show some of these new techniques through some machine learning problems I have recently worked on, such as nonconvex stochastic optimization, distributed training, adversarial attack, generative models, etc. -
Tuesday, April 23 2019
12:00-1:00PM - Building 9, Lecture Hall 2 Room 2325
Convolutional Neural Networks on Graphs by Prof. Xavier Bresson.
[Slides] | [Video]Abstract: In the past years, deep learning methods have achieved unprecedented performance on a broad range of problems in various fields from computer vision to speech recognition. So far research has mainly focused on developing deep learning methods for grid-structured data, while many important applications have to deal with graph-structured data. Such geometric data are becoming increasingly important in computer graphics and 3D vision, sensor networks, drug design, biomedicine, recommendation systems, NLP and computer vision with knowledge graphs, and web applications. The purpose of this talk is to introduce convolutional neural networks architectures on graphs, as well as applications for this class of problems. -
Thursday, March 21 2019
2:00-3:00 PM - Building 9, Lecture Hall 1 Room 2322
Restarting accelerated first-order methods without knowing error bound condition number by Prof. Zheng Qu.
[Slides]Abstract: Many convex optimization problems arising from the context of machine learning and signal processing satisfies the so-called local quadratic growth condition, which can be understood as a generalization of strong convexity. When solving such problems, classical (non-accelerated) gradient and coordinate descent methods automatically have a linear rate of convergence, whereas one needs to know explicitly the strong convexity (or error bound) parameter in order to set accelerated gradient and accelerated coordinate descent methods to have the optimal linear rate of convergence. Setting the algorithm with an incorrect parameter may result in a slower algorithm, sometimes even slower than if we had not tried to set an acceleration scheme.
We show that restarting accelerated proximal gradient algorithms at any frequency gives a globally linearly convergent algorithm. Then, as the rate of convergence depends on the match between the frequency and the quadratic error bound, we design a scheme to automatically adapt the frequency of restart from the observed decrease of the norm of the gradient mapping. Restarting accelerated coordinate descent type methods, as well as accelerated stochastic variance reduced methods, also leads to a geometric rate of convergence, under the more restrictive assumption that the objective function is strongly convex. We propose a well chosen sequence of restarting times to achieve a nearly optimal linear convergence rate, without knowing the actual value of the error bound. -
Wednesday, February 13 2019
12:00-1:00PM - Building 9, Lecture Hall 2 Room 2325
The Machine Learning Hub unveiling event and seminar by Prof. Bernard Ghanem.
[Slides] | [Video]Abstract: The Machine Learning Hub @ KAUST is designed to be the one-stop-shop for machine learning (ML) and artificial intelligence (AI) at KAUST. It is an informal forum for exchanging ideas in these areas, including (but not limited to) theoretical foundations, systems, tools, and applications. It will be providing several offerings to the KAUST community interested in ML and AI, including a regular seminar series where new research in the field is presented, an online social forum dedicated to AI and ML discussions, announcements, brainstorming, and collaborations, and hands-on activities (e.g. tutorials/workshops and hackathons) to bolster the growing need for ML and AI education/training on campus. For more details, please visit ml.kaust.edu.sa. This talk will serve as the first installment in the seminar series, during which a more detailed overview of The Hub will be presented. All who are interested in ML and AI on campus are invited to join.