Each Lecturer will hold three lessons on a specific topic.
The Lecturers below are confirmed.
Dr. Butenko’s research concentrates mainly on global and discrete optimization and their applications. In particular, he is interested in theoretical and computational aspects of continuous global optimization approaches for solving discrete optimization problems on graphs. Applications of interest include network-based data mining, analysis of biological and social networks, wireless ad hoc and sensor networks, energy, and sports analytics.
TopicsConstraint-Based Approaches to Machine Learning
Marco Gori received the Ph.D. degree in 1990 from Università di Bologna, Italy, while working partly as a visiting student at the School of Computer Science, McGill University – Montréal. In 1992, he became an associate professor of Computer Science at Università di Firenze and, in November 1995, he joint the Università di Siena, where he is currently full professor of computer science. His main interests are in machine learning, computer vision, and natural language processing. He was the leader of the WebCrow project supported by Google for automatic solving of crosswords, that outperformed human competitors in an official competition within the ECAI-06 conference. He has just published the book “Machine Learning: A Constrained-Based Approach,” where you can find his view on the field.
He has been an Associated Editor of a number of journals in his area of expertise, including The IEEE Transactions on Neural Networks and Neural Networks, and he has been the Chairman of the Italian Chapter of the IEEE Computational Intelligence Society and the President of the Italian Association for Artificial Intelligence. He is a fellow of the ECCAI (EurAI) (European Coordinating Committee for Artificial Intelligence), a fellow of the IEEE, and of IAPR. He is in the list of top Italian scientists kept by VIA-Academy.
TopicsAdam algorithm, generative models, variational (Bayesian) inference & stochastic optimization
- Current (2018 – …): Senior Research Scientist at Google Brain (San Francisco); generative models, identifiability, among other topics.
- 2015-2018: Research Scientist at OpenAI (San Francisco). Part of the founding team of OpenAI and lead of the Algorithms team, focused on basic research.
- 2013-2017: Ph.D. (cum laude) at University of Amsterdam. Thesis: Variational Inference and Deep Learning: A New Synthesis.
- 2019: The Gerrit van Dijk prijs from the Royal Holland Society of Sciences and Humanities, for my work in machine learning.
- 2019: The ELLIS PhD Award for “outstanding research achievements during the dissertation phase of outstanding students working in the field of artificial intelligence and machine learning”.
- 2017: PhD with ‘cum laude’, highest distinction in the Netherlands, and first time it was awarded at the CS department in 30 years.
- 2015: Google’s first European Doctoral Fellowship in Deep Learning
Of great interest to research and industry is the ability to model and simulate very high-dimensional data, such as images, audio or text. We provide an introduction to a powerful and general set of techniques for high-dimensional modeling, that simultaneously allows for efficient learning, inference and synthesis. We introduce the framework of VAEs, that uses amortized variational inference to efficiently learn deep latent-variable models. We also introduce Normalizing Flows (NFs), an equally useful, and interlinking, technique. NFs allow us to improve variational inference, and can even completely remove the need for variational inference. We explain common methods such as NICE, IAF, and Glow.
Learning generative models with tractable likelihoods is fairly straightforward, as we have shown. But what about learning energy-based models, with intractable partition functions? Few methods exist that scale to high-dimensional data. To this end we introduce Flow Contrastive Estimation (FCE), a new method for estimating energy-based models. FCE was primarily conceived as a version of noise-contrastive estimation (NCE), adding an adaptive noise distribution, making it scale well to high-dimensional data. Secondarily, FCE is also a method for optimizing likelihood-based generative model w.r.t. the Jensen-Shannon divergence, as an alternative to the usual Kullback-Leibler Divergence. Lastly, we show that the FCE method is also a special case of the GAN method, where the generator is given by a flow-based model, and the discriminator is parameterized by contrasting the likelihoods of an energy-based model and the flow-based model.
Is principled disentanglement possible? Equivalently, do nonlinear models converge to a unique set of representations when given sufficient data and compute? We provide an introduction to this problem, and present some surprising recent theoretical results that answer this question in the affirmative. In particular, we show that the representations learned by a very broad family of neural networks are identifiable up to only a trivial transformation. The family of models for which we derive strong identifiability results includes a large fraction of models in use today, including supervised models, self-supervised models, flow-based models, VAEs and energy-based models.
Risto Miikkulainen is a Finnish-American computer scientist and professor at the University of Texas at Austin. In 2016, he was named Fellow of the Institute of Electrical and Electronics Engineers (IEEE) “for contributions to techniques and applications for neural and evolutionary computation”.
Risto Miikkulainen is AVP of Evolutionary Intelligence at Cognizant Technology Solutions. His current research focuses on methods and applications of neuroevolution, as well as neural network models of natural language processing and vision; he is an author of over 400 articles in these research areas.
Honors and Awards
- IEEE CIS Evolutionary Computation Pioneer Award, 2020
- Gabor Award, the International Neural Network Society, 2017
- Outstanding Paper of the Decade Award, International Society for Artificial Life, 2017.
- IEEE Fellow, 2016
- IEEE Computational Intelligence Society Distinguished Lecturer, 2015-2017.
- Deployed Application Award, AAAI/IAAI-2013, AAAI/IAAI-2018
- Best Paper Awards at GECCO-2002, 2003, 2005, 2007, 2008, 2014, 2015, 2017
- Best Paper Awards at CIG-2005, 2006, 2009, 2011
- BotPrize Award (Turing test for game bots), 2012
- Honorable mention, Ziskind-Somerfield Research Award, Society of Biological Psychiatry, 2012
- Winner, Annual Competition of Pseudo-Boolean SAT Solvers at SAT-2010 and SAT-2011
- Bronze Medal, Human Competitive Results Competition, GECCO-2005, GECCO-2017
His research focuses on biologically-inspired computation such as neural networks and evolutionary computation. On one hand, the goal is to understand biological information processing, and on the other, to develop intelligent artificial systems that learn and adapt by observing and interacting with the environment. The three main focus areas are: (1) Neuroevolution, i.e. evolving complex deep learning architectures and recurrent neural networks for sequential decision tasks such as those in robotics, games, and artificial life; (2) Cognitive Science, i.e. models of natural language processing, memory, and learning that, in particular, shed light on disorders such as schizophrenia and aphasia; and (3) Computational Neuroscience, i.e. development, structure, and function of the visual cortex, episodic memory, and language processing.
See the UTCS Neural Networks Research Group website for research projects, publications, demos, and software. A few highlights: TexasExes interview/skit on artificial evolution; O’Reilly Radar Podcast on evolutionary computation; Digital Nibbles interview on BotPrize (i.e. Turing test for game bots); a 2-min soundbite on neuroevolution; the NERO machine learning game; an interactive demo of schizophrenic language model; the Computational Maps in the Visual Cortex book.
TopicsOptimization, Networks & Data Science
Panos M. Pardalos serves as distinguished professor of industrial and systems engineering at the University of Florida. Additionally, he is the Paul and Heidi Brown Preeminent Professor of industrial and systems engineering. He is also an affiliated faculty member of the computer and information science Department, the Hellenic Studies Center, and the biomedical engineering program. He is also the director of the Center for Applied Optimization. Pardalos is a world leading expert in global and combinatorial optimization. His recent research interests include network design problems, optimization in telecommunications, e-commerce, data mining, biomedical applications, and massive computing.
Jose C. Principe (M’83-SM’90-F’00) is a Distinguished Professor of Electrical and Computer Engineering and Biomedical Engineering at the University of Florida where he teaches statistical signal processing, machine learning and artificial neural networks (ANNs) modeling. He is the Eckis Professor and the Founder and Director of the University of Florida Computational NeuroEngineering Laboratory (CNEL) www.cnel.ufl.edu . His primary area of interest is processing of time varying signals with adaptive neural models. The CNEL Lab has been studying signal and pattern recognition principles based on information theoretic criteria (entropy and mutual information). The relevant application domain is neurology, brain machine interfaces and computation neuroscience.
Dr. Principe is an IEEE Fellow. He was the past Chair of the Technical Committee on Neural Networks of the IEEE Signal Processing Society, Past-President of the International Neural Network Society, and Past-Editor in Chief of the IEEE Transactions on Biomedical Engineering. He received the IEEE Neural Network Pioneer Award in 2011. Dr. Principe has more than 800 publications. He directed 99 Ph.D. dissertations and 65 Master theses. He wrote in 2000 an interactive electronic book entitled “Neural and Adaptive Systems” published by John Wiley and Sons and more recently co-authored several books on “Brain Machine Interface Engineering” Morgan and Claypool, “Information Theoretic Learning”, Springer, and “Kernel Adaptive Filtering”, Wiley.
Lecture I – Requisites for a Cognitive Architecture
- Processing in space
- Processing in time with memory
- Top down and bottom processing
- Extraction of information from data with generative models
- Attention mechanisms and fovea vision
Lecture II – Putting it all together
- Empirical Bayes with generative models
- Clustering of time series with linear state models
- Information Theoretic Autoencoders
Lecture III – Beyond Backpropagation: Modular Learning for Deep Networks
- Reinterpretation of neural network layers
- Training each learning without backpropagation
- Examples and advantages in transfer learning
TopicsMachine Learning for Medicine, Data Science and decisions, Artificial Intelligence
Professor van der Schaar is John Humphrey Plummer Professor of Machine Learning, Artificial Intelligence and Medicine at the University of Cambridge and a Turing Fellow at The Alan Turing Institute in London, where she leads the effort on data science and machine learning for personalised medicine. She is an IEEE Fellow (2009). She has received the Oon Prize on Preventative Medicine from the University of Cambridge (2018). She has also been the recipient of an NSF Career Award, 3 IBM Faculty Awards, the IBM Exploratory Stream Analytics Innovation Award, the Philips Make a Difference Award and several best paper awards, including the IEEE Darlington Award. She holds 35 granted USA patents.
The current emphasis of her research is on machine learning with applications to medicine, finance and education. She has also worked on data science, network science, game theory, signal processing, communications, and multimedia.
Five papers accepted at NeurIPS 2019
TensorFlow is a powerful open-source software library for machine learning developed by researchers at Google. It has many pre-built functions and models to ease the task of building and training neural networks, both on single machine and on multiple machines. TensorFlow also has dedicated modules to support the seamless integration of research with production.
This tutorial will cover the fundamentals and contemporary usage of the TensorFlow library for deep learning. It aims to help the audience understand the design choices that led to TensorFlow 1.0 and 2.0, explore the functions it has to offer, and learn how to build and structure models best suited for a deep learning project. Through the tutorials, students will use TensorFlow to build models of different
complexity, from simple linear/logistic regression to convolutional neural networks, recurrent neural networks, and self-attention models to solve tasks such as word embedding, translation, optical character recognition, joint intent slot filling for conversational AI. Students will also learn the best practices to structure a model and manage research experiments.
Over the past two years PyTorch has became the dominant tool for machine learning research, with many of the groundbreaking advancements appearing alongside their PyTorch implementations. Having at least a basic understanding of the library is an asset, as it allows one to easily collaborate with others, develop their own research faster or at least gain a deeper understanding of the resources published every day online.
This course will be a gentle introduction to the PyTorch library and we will go over all of its fundamental abstractions. Those include the way model code is usually structured, how can one go about computing gradients of arbitrary Python functions automatically, making effective use of accelerators such as GPUs and what are the best practices for research implementations. If time allows, we will also take a peek into some more advanced features like the just-in-time compiler.
The Lecturers of the previous editions:
- Ioannis Antonoglou, Google DeepMind, UK
- Roman Belavkin, Middlesex University London, UK
- Yoshua Bengio, Head of the Montreal Institute for Learning Algorithms (MILA) & University of Montreal, Canada
- Sergiy Butenko, Texas A&M University, USA
- Giuseppe Di Fatta, University of Reading, UK
- Marco Gori, University of Siena, Italy
- Yi-Ke Guo, Imperial College London, UK & Founding Director of Data Science Institute
- Phillip Isola, MIT, USA
- Leslie Kaelbling, MIT - Computer Science & Artificial Intelligence Lab, USA
- Ilias S. Kotsireas, Wilfrid Laurier University, Canada
- Peter Norvig, Director of Research, Google
- Panos Pardalos, University of Florida, USA
- Alex 'Sandy' Pentland, MIT & Director of MIT’s Human Dynamics Laboratory, USA
- Marc'Aurelio Ranzato, Facebook AI Research Lab, New York, USA
- Dolores Romero Morales, Copenhagen Business School, Denmark
- Ruslan Salakhutdinov, Carnegie Mellon University, and AI Research at Apple, USA
- Josh Tenenbaum, MIT, USA
- Naftali Tishby, Hebrew University, Israel
- Joaquin Vanschoren, Eindhoven University of Technology, The Netherlands
- Oriol Vinyals, Google DeepMind, UK
- Aleskerov Z. Fuad, National Research University Higher School of Economics, Russia