Machine Learning (ML) is quickly providing new powerful tools for physicists and other natural scientists to extract essential information from large amounts of data, either from experiments or simulations. This IPAM long program will foster nontrivial research and provoke scientific discussion at the interface between ML and Physics. We aim to go beyond simple fitting of physical models from data and move the discussion to (i) using generative ML methods and active learning in order to generate and design complex and novel physical structures and objects, (ii) obtain models that are physically understable, e.g. by maintaining relations of the predictions to the microscopic physical quantities used as an input, (iii) using ML to learn the physical principles and mathematical structures underlying the data, and (iv) developing new ML methods inspired by methods and models developed in Physics.