Here’sthe link for the recorded presentation.
Department ofComputer Science
Data Science Institute
?Contrastivelearning, multi-view redundancy, and linear models?
Abstract: Contrastive learning is a”self-supervised” approach to representation learning that usesnaturally occurring similar and dissimilar pairs of data points to find useful embeddings ofdata. We study contrastive learning in the context of multi-view statisticalmodels.
First, weshow that whenever the views of the data are approximately redundant in theirability predict a target function, a low-dimensional embedding obtained viacontrastive learning affords a linear predictor with near-optimal predictiveaccuracy.
Second, weshow that in the context of topic models, the embedding can be interpreted as alinear transformation of the posterior moments of the hidden topic distributiongiven theobserved words. We also empirically demonstrate that linearclassifiers with these representations perform well in document classificationtasks with very few labeled examples in a semi-supervised setting.This isjoint work with Akshay Krishnamurthy (MSR) and Christopher Tosh (Columbia).
Bio: Daniel Hsu is an associateprofessor in the Department of Computer Science and a member of the DataScience Institute, both at Columbia University. Previously, he was a postdoc atMicrosoft Research New England, and the Departments of Statistics at RutgersUniversity and the University ofPennsylvania. He holds a Ph.D. in ComputerScience from UC San Diego, and a B.S. in Computer Science and Engineering fromUC Berkeley. He was selectedby IEEE Intelligent Systems as one of ?AI’s 10 toWatch? in 2015 and received a 2016 Sloan Research Fellowship.
Daniel’s research interests are inalgorithmic statistics and machine learning. His work has produced the firstcomputationally efficient algorithms for several statistical estimation tasks(including many involving latent variable models such as mixture models, hiddenMarkov models, and topic models), provided new algorithmic frameworks forsolving interactive machine learning problems, and led to the creation ofscalable tools for machine learning applications.
His Ph.D.advisor at UCSD was Sanjoy Dasgupta. Hispostdoctoral stints were with Sham Kakade (atPenn) and Tong Zhang (at Rutgers).