ML/AI Courses at KAUST (2021/2022)

Spring

• CS 229: Machine Learning - [Syllabus]
  By Prof. Di Wang

  Description: Machine Learning is a science of getting machines to learn, more specifically, designing algorithms that allow computers to learn from empirical data. In the past decade, Machine Learning has successfully made computers to recognize speeches and hand-written characters, to convert spoken words to text, to effectively search our needed information, and to recommend products/books/movies that we potently like. In this class, you will learn the most important machine learning techniques, not only the theoretical foundations of these techniques, but also the practice implementation of them.

  The main topics will include linear and non-linear regression, nonparametric methods, Bayesian methods, support vector machines, kernel methods, Artificial Neural Networks, deep network, model selection, learning theory, VC dimension, clustering, EM, mixture model, dimensionality reduction, PCA, SVD, and reinforcement learning.

  Classes: Mon Wed 10:15

• CS 283: Deep Generative Modeling - [Syllabus]
  By Prof. Mohamed Elhoseiny

  Description: This course focuses on generative modeling. The topics covered in this course include Deep Generative Models (VAEs, GANs), Normalizing Flows, infinitesimal flows (Neural ODEs/SDEs, Deep Equilibrium Models), Energy-based Models, Variations and Combinations of Basic Generative Models, Generative Imitation Learning, Genetic Algorithms, Deep Fakes, Aesthetic guided Reinforcement Learning, Style Transfer, Cycle Consistent Generative Models, Creative Adversarial Networks, Algorithmic Art models, and 3D generative models. Story Generation, Transformer-based Text Generation, GPT based Text Generation, and Transformer GANs. All topics are around generative modeling from computer vision, Music, and NLP domains.

  Classes: Sun 08:30

• CS 323: Deep Learning for Visual Computing - [Syllabus]
  By Prof. Bernard Ghanem

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (e.g. optimization, network architecture, and training best practices) as well as selected applications (e.g. image/video classification, object detection, semantic segmentation, and point cloud segmentation). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Mon Wed 13:15

• CS 332: Federated Learning - [Syllabus]
  By Prof. Peter Richtarik

  Description: This is a PhD level course in a new branch of machine learning: federated learning. In federated learning, machine learning models are trained on mobile or edge devices with an explicit effort to preserve the privacy of users' data. Federated Learning combines areas such as supervised machine learning, privacy, distributed and edge computing, optimization, communication compression and systems. This is a new and fast-growing field with few theoretical results, and early production systems (e.g., Tensor Flow Federated). The aim of this course is to become familiar with the key results and practices of this field. As there is no textbook on this topic, the course material will be based on recent papers.

  Classes: Sun Tue 10:15

• CS 361: Combinatorial Machine Learning - [Syllabus]
  By Prof. Mikhail Moshkov

  Description: The main difference between Combinatorial Machine Learning (CML) and Machine Learning (ML) is the following: usual ML is based on probability theory and mathematical statistics, and is oriented mainly on prediction problem, but in CML we concentrate on the study of classifiers as combinatorial objects, and we consider classifiers not only as predictors but also as algorithms and as a way for knowledge representation.

  The course covers tools for design and analysis of decision trees, decision rules and tests, their applications to supervised machine learning, and related topics including current results of research.

  The main contents are: introduction (basic notions and examples from applications); tools (relationships among decision trees, rules and tests; bounds on complexity of tests, decision rules and trees; algorithms for construction of tests, decision rules and trees); applications (supervised machine learning); some of the additional topics (decision tables with many-valued decisions; approximate decision trees, rules and tests; global and local approaches to the study of problems over infinite sets of attributes; applications to combinatorial optimization, fault diagnosis, pattern recognition, analysis of acyclic programs, data mining, and knowledge representation); current results of research.

  Classes: Mon Thu 15:00

• ErSE 222: Machine Learning in Geoscience - [Syllabus]
  By Prof. Matteo Ravasi

  Description: This course covers a number of machine learning methods and their application to geoscientific problems. The main focus of this course is on describing the fundamental theory and practical applications as well as providing basic best practices for the rigorous development and evaluation of machine learning models.

  At the end of the course, students will know how to appropriately choose which algorithm to use depending on the nature of the problem to solve as well as how to create effective neural network models. Students are expected to have basic experience in programming a high-level language such as Python or MATLAB and have a solid background in linear algebra and inverse problems. Students will be required to use Python in the labs and programming assignments.

  Classes: Tue Thu 10:15

Fall

• CS 220: Data Analytics - [Syllabus]
  By Prof. Robert Hoehndorf

  Description: The course covers basic concepts and algorithms for artificial intelligence, data mining and machine learning. The main contents are: artificial intelligence (task environment, performance measure, and problem solving by searching), data mining (data and patterns, summary statistics and visualization, unsupervised feature selection, and supervised feature selection), and machine learning (cross validation and supervised learning).

  Classes: Mon Wed 8:30

• CS 294E: Contemporary Topics in AI - [Syllabus]
  By Prof. Marco Canini

  Description: The recent successes of AI/ML owe a great debt to the rapid innovations in hardware and software systems and to the broad accessibility to this technology. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. On the one hand, innovations in AI/ML have the potential to improve the systems infrastructure itself by applying AI to improve system design, job scheduling, video streaming or simplify network management. This class covers a broad array of research topics and latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems.

  Classes: Mon 16:45, Thu 13:15

• CS 323: Deep Learning for Visual Computing - [Syllabus]
  By Prof. Peter Wonka

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (optimization, network architecture, compression, …) as well as selected applications (image recognition, segmentation, image synthesis, object detection, object synthesis, mesh segmentation, point cloud processing, …). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Sun Wed 15:00

• CS 331: Stochastic Gradient Descent Methods - [Syllabus]
  By Prof. Peter Richtarik

  Description: Stochastic gradient descent (SGD) in one or another of its many variants is the workhorse method for training modern supervised machine learning models. However, the world of SGD methods is vast and expanding, which makes it hard for practitioners and even experts to understand its landscape and inhabitants. This course is a mathematically rigorous and comprehensive introduction to the field, and is based on the latest results and insights. The course develops a convergence and complexity theory for serial, parallel, and distributed variants of SGD, in the strongly convex, convex and nonconvex setup, with randomness coming from sources such as subsampling and compression. Additional topics such as acceleration via Nesterov momentum or curvature information will be covered as well. A substantial part of the course offers a unified analysis of a large family of variants of SGD which have so far required different intuitions, convergence analyses, have different applications, and which have been developed separately in various communities. This framework includes methods with and without the following tricks, and their combinations: variance reduction, data sampling, coordinate sampling, arbitrary sampling, importance sampling, mini-batching, quantization, sketching, dithering and sparsification.

  Classes: Sun Tue 10:15

• CS 394S: Contemporary Topics in Computer Security - [Syllabus]
  By Prof. Di Wang

  Description: Differential Privacy, with roots in cryptography, is a strong mathematical scheme for privacy preserving and is now becoming a standard for private data analysis which has been deployed in both governments and industries. In this class, you will learn the motivation and the developments of differential privacy, with its application to Machine Learning and Statistics, from both theory and practice.

  Classes: Mon Wed 10:15

• ErSE 253: Data Analysis in Geosciences - [Syllabus]
  By Prof. Ibrahim Hoteit & Prof. Sigurjon Jonsson

  Description: Processing of multidimensional data, spatial statistics including variogram, covariance analysis and modeling, multipoint estimation, spatial interpolation including statistical methods (kriging), time series analysis, uncertainty assessment, cross validation, multivariate analysis including principal component analysis and canonical analysis.

  Classes: Mon Wed 10:15

ML/AI Courses at KAUST (2020/2021)

Spring

• B 322: Computational BioSci & Machine Learning - [Syllabus]
  By Prof. Jesper Tegner

  Description: The course provides a broad and practical overview of selected techniques and concepts in rapidly developing areas such as bioinformatics, computational biology, systems biology, systems medicine, network biology, synthetic biology, data analytics, predictive modelling, machine learning, and machine intelligence. Topics are selected to be of relevance for the computer scientist, working biologist, computational scientist, and applied investigator (Biotechnology and engineering).

  Classes: Mon Thu 15:00

• B 324: Machine Learning for Genomics and Health - [Syllabus]
  By Prof. Jesper Tegner

  Description: Recent progress in machine learning and artificial intelligence is currently transforming genomics, translational medical research, healthcare, and wellness. Huge data-sets are produced at an increasing rate. This include recordings of smart living augmented by sensor devices, medical images, text data in healthcare and social media, and genomics profiling of a range of different biomolecular data. Concurrent with these developments there has over the last 5 years been a stunning production of open source machine learning tools and powerful computational platforms. These advances are currently advancing bioinformatics, computational biology, systems biology, where an area which could be referred to as Digital Medicine in a broad sense is emerging. We expect students with a background in computer science, mathematics, bioscience, and engineering to learn how to use, develop, and to think on how to use ML/AI techniques in what can broadly be referred to as Digital Technologies for Medicine and Health.

  Classes: Sun Wed 16:45

• CS 213: Knowledge Representation and Reasoning - [Syllabus]
  By Prof. Robert Hoehndorf

  Description: The aims of the course are to introduce key concepts of knowledge representation and its role in artificial intelligence, enable students to design and apply knowledge-based systems, and understand the limitations and complexity of algorithms for representing knowledge.
  The course will begin with a review of basic concepts in first order logics, including syntax, semantic, and different deductive systems (Hilbert style systems, sequent calculus). We will then discuss resolution as a method for generating proofs, illustrate implementation strategies and limitations of resolution-based algorithms. As second major formalism we will introduce Description Logics for expressing terminological knowledge and ontologies. Description Logics form one of the foundations of the Semantic Web, and we will discuss a decision procedure for the basic Description Logic ALC as well as algorithms for different variants of ALC. In the third part of the course, we will introduce methods to represent and reason about common sense knowledge, including Default Logic and Circumscription, as well as Answer Set Programming. The final part of the course will be about modal logics.

  Classes: Mon Tue 08:30

• CS 229: Machine Learning - [Syllabus]
  By Prof. Xiangliang Zhang

  Description: Machine Learning is a science of getting machines to learn, more specifically, designing algorithms that allow computers to learn from empirical data. In the past decade, Machine Learning has successfully made computers to recognize speeches and hand-written characters, to convert spoken words to text, to effectively search our needed information, and to recommend products/books/movies that we potently like. In this class, you will learn the most important machine learning techniques, not only the theoretical foundations of these techniques, but also the practice implementation of them.
  The main topics will include linear and non-linear regression, nonparametric methods, Bayesian methods, support vector machines, kernel methods, Artificial Neural Networks, deep network, model selection, learning theory, VC dimension, clustering, EM, dimensionality reduction, PCA, SVD, and reinforcement learning.

  Classes: Mon Wed 10:30

• CS 320: Probabilistic Graphical Models - [Syllabus]
  By Prof. Xin Gao

  Description: This is a research-oriented graduate-level course on PGMs. The course will cover two main types of PGMs, i.e., directed PGMs and undirected PGMs. For directed PGMs, we will cover Bayesian networks, with one of its most important variants, hidden Markov models. For undirected PGMs, we will cover Markov networks (or Markov random fields), with one of its most important variants, conditional random fields. Therefore, the course contains four parts: Bayesian networks, hidden Markov models, Markov networks, and conditional random fields.

  Classes: Wed 15:00

• CS 361: Combinatorial Machine Learning - [Syllabus]
  By Prof. Mikhail Moshkov

  Description: The main difference between Combinatorial Machine Learning (CML) and Machine Learning (ML) is the following: usual ML is based of probability theory and mathematical statistics, and is oriented mainly on prediction problem, but in CML we concentrate on the study of classifiers as combinatorial objects, and we consider classifiers not only as predictors but also as algorithms and as a way for knowledge representation. The course covers tools for design and analysis of decision trees, decision rules and tests, their applications to supervised machine learning, and related topics including current results of research.
  The main contents are: introduction (basic notions and examples from applications); tools (relationships among decision trees, rules and tests; bounds on complexity of tests, decision rules and trees; algorithms for construction of tests, decision rules and trees); applications (supervised machine learning); some of the additional topics (decision tables with many-valued decisions; approximate decision trees, rules and tests; global and local approaches to the study of problems over infinite sets of attributes; applications to combinatorial optimization, fault diagnosis, pattern recognition, analysis of acyclic programs, data mining, and knowledge representation); current results of research.

  Classes: Sun 15:00 Thu 13:15

• CS 323: Deep Learning for Visual Computing - [Syllabus]
  By Prof. Bernard Ghanem

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (e.g. optimization, network architecture, and training best practices) as well as selected applications (e.g. image/video classification, object detection, semantic segmentation, and point cloud segmentation). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Mon Wed 13:15

• CS 332: Federated Learning - [Syllabus]
  By Prof. Peter Richtarik

  Description: This is a PhD level course in a new branch of machine learning: federated learning. In federated learning, machine learning models are trained on mobile devices with an explicit effort to preserve the privacy of users’ data. This is a new field, with a few theoretical results, and early production systems (e.g., Tensor Flow Federated). The aim of this course is the become familiar with the key results of this field. As there is no textbook on this topic, the course material will be based on recent papers.

  Classes: Sun Tue 10:15

• CS 294D: Deep Generative Modeling [Syllabus]
  By Prof. Mohamed Hamdy Elhoseiny

  Classes: Mon Thu 15:00

• ErSE 394A: Advanced Machine Learning Methods in Geoscience [Syllabus]
  By Prof. Gerard Thomas Schuster

  Classes: Mon Wed 13:15

Fall

• CS 220: Data Analytics - [Syllabus]
  By Prof. Xin Gao

  Description: The course covers basic concepts and algorithms for artificial intelligence, data mining and machine learning. The main contents are: artificial intelligence (task environment, performance measure, and problem solving by searching), data mining (data and patterns, summary statistics and visualization, unsupervised feature selection, and supervised feature selection), and machine learning (cross validation and supervised learning).

  Classes: Mon Wed 10:15

• CS 323: Deep Learning for Visual Computing - [Syllabus]
  By Prof. Peter Wonka

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (optimization, network architecture, compression, …) as well as selected applications (image recognition, segmentation, image synthesis, object detection, object synthesis, mesh segmentation, point cloud processing, …). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Sun Wed 15:00

• CS 331: Stochastic Gradient Descent Methods - [Syllabus]
  By Prof. Peter Richtarik

  Description: Stochastic gradient descent (SGD) in one or another of its many variants is the workhorse method for training modern supervised machine learning models. However, the world of SGD methods is vast and expanding, which makes it hard for practitioners and even experts to understand its landscape and inhabitants. This course is a mathematically rigorous and comprehensive introduction to the field, and is based on the latest results and insights. The course develops a convergence and complexity theory for serial, parallel, and distributed variants of SGD, in the strongly convex, convex and nonconvex setup, with randomness coming from sources such as subsampling and compression. Additional topics such as acceleration via Nesterov momentum or curvature information will be covered as well. A substantial part of the course offers a unified analysis of a large family of variants of SGD which have so far required different intuitions, convergence analyses, have different applications, and which have been developed separately in various communities. This framework includes methods with and without the following tricks, and their combinations: variance reduction, data sampling, coordinate sampling, arbitrary sampling, importance sampling, mini-batching, quantization, sketching, dithering and sparsification.

  Classes: Sun Tue 8:30

• CS 394D: Contemporary Topics in Machine Learning
  By Prof. Mohamed Elhoseiny

  Description: This course covers advanced topics in deep learning. This course is intended as a second graduate level course on deep learning and, compared with CS 323, will go into more advanced deep learning techniques. The topics covered in this course includes zero/few-shot learning, long-tail recognition, deep generative models, meta-learning, efficient continual learning, causal deep-learning.

  Classes: Sun Wed 16:45

• ErSE 222: Machine Learning in Geoscience - [Syllabus]
  By Prof. Gerard Thomas Schuster

  Description: The course covers a number of Machine Learning methods and their applications in solving geoscience problems. The main focus is on using supervised learning methods to solve geoscience problems, with an emphasis on the practical use of convolutional neural networks. At the end of the course, the diligent student will know how to design the architecture of a convolutional network or ML method and employ it in solving a particular geoscience problem. Students are expected to have experience in programming a high-level language such as MATLAB and have a background in partial differential equations and linear algebra. Students will use KERAS and PYTHON in the network programming.

  Classes: Sun Thu 10:15

ML/AI Courses at KAUST (2019/2020)

Spring

• B 322: Computational BioSci & Machine Learning - [Syllabus]
  By Prof. Jesper Tegner

  Description: The course provides a broad and practical overview of selected techniques and concepts in rapidly developing areas such as bioinformatics, computational biology, systems biology, systems medicine, network biology, synthetic biology, data analytics, predictive modelling, machine learning, and machine intelligence. Topics are selected to be of relevance for the computer scientist, working biologist, computational scientist, and applied investigator (Biotechnology and engineering).

  Classes: Mon Thu 14:30

• B 390N: Machine Learning for Genomics and Health - [Syllabus]
  By Prof. Jesper Tegner

  Description: Recent progress in machine learning and artificial intelligence is currently transforming genomics, translational medical research, healthcare, and wellness. Huge data-sets are produced at an increasing rate. This include recordings of smart living augmented by sensor devices, medical images, text data in healthcare and social media, and genomics profiling of a range of different biomolecular data. Concurrent with these developments there has over the last 5 years been a stunning production of open source machine learning tools and powerful computational platforms. These advances are currently advancing bioinformatics, computational biology, systems biology, where an area which could be referred to as Digital Medicine in a broad sense is emerging. We expect students with a background in computer science, mathematics, bioscience, and engineering to learn how to use, develop, and to think on how to use ML/AI techniques in what can broadly be referred to as Digital Technologies for Medicine and Health.

  Classes: Sun Wed 14:30

• CS 213: Knowledge Representation and Reasoning - [Syllabus]
  By Prof. Robert Hoehndorf

  Description: The aims of the course are to introduce key concepts of knowledge representation and its role in artificial intelligence, enable students to design and apply knowledge-based systems, and understand the limitations and complexity of algorithms for representing knowledge.
  The course will begin with a review of basic concepts in first order logics, including syntax, semantic, and different deductive systems (Hilbert style systems, sequent calculus). We will then discuss resolution as a method for generating proofs, illustrate implementation strategies and limitations of resolution-based algorithms. As second major formalism we will introduce Description Logics for expressing terminological knowledge and ontologies. Description Logics form one of the foundations of the Semantic Web, and we will discuss a decision procedure for the basic Description Logic ALC as well as algorithms for different variants of ALC. In the third part of the course, we will introduce methods to represent and reason about common sense knowledge, including Default Logic and Circumscription, as well as Answer Set Programming. The final part of the course will be about modal logics.

  Classes: Sun Wed 13:30

• CS 229: Machine Learning - [Syllabus]
  By Prof. Xiangliang Zhang

  Description: Machine Learning is a science of getting machines to learn, more specifically, designing algorithms that allow computers to learn from empirical data. In the past decade, Machine Learning has successfully made computers to recognize speeches and hand-written characters, to convert spoken words to text, to effectively search our needed information, and to recommend products/books/movies that we potently like. In this class, you will learn the most important machine learning techniques, not only the theoretical foundations of these techniques, but also the practice implementation of them.
  The main topics will include linear and non-linear regression, nonparametric methods, Bayesian methods, support vector machines, kernel methods, Artificial Neural Networks, deep network, model selection, learning theory, VC dimension, clustering, EM, dimensionality reduction, PCA, SVD, and reinforcement learning.

  Classes: Mon Wed 10:30

• CS 320: Probabilistic Graphical Models - [Syllabus]
  By Prof. Xin Gao

  Description: This is a research-oriented graduate-level course on PGMs. The course will cover two main types of PGMs, i.e., directed PGMs and undirected PGMs. For directed PGMs, we will cover Bayesian networks, with one of its most important variants, hidden Markov models. For undirected PGMs, we will cover Markov networks (or Markov random fields), with one of its most important variants, conditional random fields. Therefore, the course contains four parts: Bayesian networks, hidden Markov models, Markov networks, and conditional random fields.

  Classes: Mon Wed 10:30

• CS 361: Combinatorial Machine Learning - [Syllabus]
  By Prof. Mikhail Moshkov

  Description: The main difference between Combinatorial Machine Learning (CML) and Machine Learning (ML) is the following: usual ML is based of probability theory and mathematical statistics, and is oriented mainly on prediction problem, but in CML we concentrate on the study of classifiers as combinatorial objects, and we consider classifiers not only as predictors but also as algorithms and as a way for knowledge representation. The course covers tools for design and analysis of decision trees, decision rules and tests, their applications to supervised machine learning, and related topics including current results of research.
  The main contents are: introduction (basic notions and examples from applications); tools (relationships among decision trees, rules and tests; bounds on complexity of tests, decision rules and trees; algorithms for construction of tests, decision rules and trees); applications (supervised machine learning); some of the additional topics (decision tables with many-valued decisions; approximate decision trees, rules and tests; global and local approaches to the study of problems over infinite sets of attributes; applications to combinatorial optimization, fault diagnosis, pattern recognition, analysis of acyclic programs, data mining, and knowledge representation); current results of research.

  Classes: Sun 14:30 Thu 13:00

• CS 390DD: Special Topics in Machine Learning - [Syllabus]
  By Prof. Bernard Ghanem

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (e.g. optimization, network architecture, and training best practices) as well as selected applications (e.g. image/video classification, object detection, semantic segmentation, and point cloud segmentation). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Mon Wed 13:00

• CS 390T: Special Topics in Federated Learning - [Syllabus]
  By Prof. Peter Richtarik

  Description: This is a PhD level course in a new branch of machine learning: federated learning. In federated learning, machine learning models are trained on mobile devices with an explicit effort to preserve the privacy of users’ data. This is a new field, with a few theoretical results, and early production systems (e.g., Tensor Flow Federated). The aim of this course is the become familiar with the key results of this field. As there is no textbook on this topic, the course material will be based on recent papers.

  Classes: Sun Tue 09:00

Fall

• CS 220: Data Analytics - [Syllabus]
  By Prof. Xin Gao

  Description: The course covers basic concepts and algorithms for artificial intelligence, data mining and machine learning. The main contents are: artificial intelligence (task environment, performance measure, and problem solving by searching), data mining (data and patterns, summary statistics and visualization, unsupervised feature selection, and supervised feature selection), and machine learning (cross validation and supervised learning).

  Classes: Mon Wed 10:30

• CS 290E: Special Topic in Artificial Intelligence - [Syllabus]
  By Prof. Marco Canini

  Description: The recent successes of AI/ML owe a great debt to the rapid innovations in hardware and software systems and to the broad accessibility to this technology. These systems have enabled training increasingly complex models on ever larger datasets. In the process, these systems have also simplified model development, enabling the rapid growth in the machine learning community. On the one hand, innovations in AI/ML have the potential to improve the systems infrastructure itself by appling AI to improve system design, job scheduling, video streaming or simplify network management. This class covers a broad array of research topics and latest trends in systems designs to better support the next generation of AI applications, and applications of AI to optimize the architecture and the performance of systems.

  Classes: Mon Thu 13:00

• CS 390DD: Special Topics in Machine Learning - [Syllabus]
  By Prof. Peter Wonka

  Description: This course provides an overview of deep learning applications in visual computing. We will cover some basics of deep learning (optimization, network architecture, compression, ...) as well as selected applications (image recognition, segmentation, image synthesis, object detection, object synthesis, mesh segmentation, point cloud processing, ...). The selection of the applications is expected to change with different course offerings and will be adapted to the latest research papers in computer vision and computer graphics.

  Classes: Sun Wed 14:30

• CS 390FF: Special Topics in Data Sciences - [Syllabus]
  By Prof. Peter Richtarik

  Description: The course is a mathematically rigorous introduction to the emerging field of big data optimization. The focus is on algorithms and assocaited theory. Randomized/stochastic algorithms play a dominant role. The course is based on a novel and unified approach to recent developments in the field developed by the lecturer. Big data optimization is the study of optimization problems described by big quantities of data, where "big" is loosely defined as large enough for traditional approaches to suffer or not be applicable at all. As we live in a digital age where it is increasingly easier to collect and store data in digital form (e.g., transaction records, YouTube clicks, internet activity, Wikipedia, twitter, customer behaviour databases, government records, image collections), big data problems are becoming ubiquitous. New methods and tools are needed to analyze such vast datasets, and optimization algorithms are at the heart of such efforts, underpinning much of data science, including machine learning, operations research and statistical analysis. Alongside computer science and statistics, optimization is one of the pillars of big data analysis. The course will cover topics such as supervised learning, empirical risk minimization, big data problems, stochastic gradient descent, minibatching, importance sampling, arbitrary sampling, variance reduction, quantization and compression for distributed training, convex feasibility problems, high dimensional problems, randomized coordinate descent, and acceleration.

  Classes: Sun Tue 09:00