Dimitris Papailiopoulos
Jay & Cynthia Ihlenfeld Associate Professor University of Wisconsin-Madison
Electrical and Computer Engineering
I am the Jay & Cynthia Ihlenfeld Associate Professor of Electrical and Computer Engineering (and CS by courtesy) at the University of Wisconsin-Madison, a faculty fellow at the Grainger Institute, and a faculty affiliate with the Optimization group at the Wisconsin Institute for Discovery.
My research lies in the intersection of machine learning, coding theory, and distributed optimization. I am particularly interested in the theory and practice of large-scale machine learning and the challenges that arise once we aim to build solutions that come with robustness and scalability guarantees.
Before coming to Madison, I spent two wonderful years as a postdoc at UC Berkeley, where I was a member of the AMPLab and BLISS, and had the pleasure to collaborate with
Ben Recht and Kannan Ramchandran.
I received my Ph.D. in 2014 from UT Austin, where I was fortunate to be advised by
Alex Dimakis. Before UT, I spent 3.5 years as a grad student at USC.
Before all that, I received my M.Sc. (2009) and ECE Diploma (2007) from the Technical University of Crete (TUC), located in the beautiful city of Chania.
In 2018, I co-founded the conference on Machine Learning & Systems (MLSys), a new conference that targets research at the intersection of systems and machine learning. In 2018 and 2020, I was the program co-chair for MLSys. In 2019, I also co-chaired the 3rd Midwest Machine Learning Symposium (MMLS).
gScholar
bio,
resume
twitter
Teaching
-
Spring 2022
-
Fall 2019, Fall 2020, Fall 2021
-
Fall 2017, Fall 2018
-
Fall 2016, Spring 2018
-
Spring 2020
-
Spring 2021
Publications
2022
-
GenLabel: Mixup Relabeling using Generative ModelsICML 2022. [arxiv]
-
On the Utility of Gradient Compression in Distributed Training SystemsMLSys 2022. [arxiv]
-
Permutation-Based SGD: Is Random Optimal?ICLR 2022. [arxiv]
-
Finding Nearly Everything within Random Binary NetworksAISTATS 2022. [arxiv]
2021
-
An Exponential Improvement on the Memorization Capacity of Deep Threshold NetworksNeurIPS 2021. [arxiv]
-
PUFFERFISH: Communication-efficient Models at No Extra CostMLSys 2021. [arxiv]
-
Accordion: Adaptive Gradient Communication via Critical Learning Regime IdentificationMLSys 2021. [arxiv]
2020
-
Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is SufficientNeurIPS 2020 (spotlight). [arxiv]
-
Attack Of The Tails: Yes, You Really Can Backdoor Federated LearningNeurIPS 2020. [arxiv]
-
Bad Global Minima Exist and SGD Can Reach ThemNeurIPS 2020 . [arxiv]
-
Closing the Convergence Gap of SGD without ReplacementICML 2020 . [arxiv]
-
Federated Learning with Matched AveragingICLR 2020 (oral). [openreview]
2019
-
DETOX: A Redundancy-based Framework for Faster and More Robust Gradient AggregationNeurIPS 2019. [arxiv]
-
Does Data Augmentation Lead to Positive Margin?ICML 2019. [arxiv]
-
Bad Global Minima Exist and SGD Can Reach ThemICML 2019, Deep Phenomena Workshop (oral). [arxiv]
-
A Geometric Perspective on the Transferability of Adversarial DirectionsAISTATS 2019. [arxiv]
2018
-
ATOMO: Communication-efficient Learning via Atomic SparsificationNeurIPS 2018. [arxiv]
-
The Effect of Network Width on the Performance of Large-batch TrainingNeurIPS 2018. [arxiv]
-
DRACO: Byzantine-resilient Distributed Training via Redundant GradientsICML 2018. [arxiv]
-
Stability and Generalization of Learning Algorithms that Converge to Global OptimaICML 2018. [arxiv]
-
Gradient Coding Using the Stochastic Block ModelISIT 2018. [arxiv]
-
Gradient Diversity: a Key Ingredient for Scalable Distributed LearningAISTATS 2018. [arxiv]
2017
-
Gradient Diversity: a Key Ingredient for Scalable Distributed LearningNIPS 2017, OPT Workshop (oral). [arxiv]
-
UberShuffle: Communication-efficient Data Shuffling for SGD via Coding TheoryNIPS 2017, ML Systems Workshop [pdf]
-
Perturbed Iterate Analysis for Asynchronous Stochastic OptimizationSIAM Journal on Optimization. [arxiv]
-
Speeding-up Distributed Machine Learning Using CodesIEEE Transactions on Information Theory. [arxiv]
-
Coded Computation for Multicore SetupsISIT 2017. [pdf]
2016
-
CYCLADES: Conflict-free Asynchronous Machine LearningNIPS 2016. [arxiv]
-
Speeding-up Distributed Machine Learning Using CodesISIT 2016. [arxiv]
-
On the Worst-Case Approximability of Sparse PCACOLT 2016. [arxiv]
-
Bipartite Correlation Clustering - Maximizing AgreementsAISTATS 2016. [arxiv]
-
Locality and Availability in Distributed StorageIEEE Transactions on Information Theory. [preprint]
-
Optimal Locally Repairable Codes and Connections to Matroid TheoryIEEE Trans. on Information Theory. [preprint]
2015
-
Perturbed Iterate Analysis for Asynchronous Stochastic OptimizationNIPS 2015, OPT Workshop (oral). [long version]
-
Speeding-up Distributed Machine Learning Using CodesNIPS 2015, ML-Systems Workshop. [arxiv]
-
Parallel Correlation Clustering on Big GraphsNIPS 2015. [long version]
-
Sparse PCA via Bipartite MatchingsNIPS 2015. [long version]
-
Orthogonal NMF through Subspace ExplorationNIPS 2015. [long version]
2014
-
Provable Deterministic Leverage Score SamplingKDD 2014. [long version]
-
Finding Dense Subgraphs via Low-Rank Bilinear Optimization
-
Nonnegative Sparse PCA with Provable Guarantees
-
Locality and Availability in Distributed StorageISIT 2014. [long version]
-
Locally Repairable CodesIEEE Transactions on Information Theory, September 2014. ,
-
A Repair Framework for Scalar MDS CodesIEEE Journal on Selected Areas in Communications, May 2014. ,
-
The Sparse Principal Component of a Constant-rank MatrixIEEE Transactions on Information Theory, March 2014. , implementation by Megas.
2013
-
Sparse PCA through Low-rank Approximations
-
Optimal Locally Repairable Codes and Connections to Matroid TheoryISIT 2013. [IEEEXplore], [arXiv]
-
XORing Elephants: Novel Erasure Codes for Big Data
-
Repair Optimal Erasure Codes through Hadamard DesignsIEEE Transactions on Information Theory, May 2013. [IEEEXplore], [arXiv]
-
Maximum-Likelihood Noncoherent PAM DetectionIEEE Transactions on Communications, March 2013. [IEEEXplore], [draft]
2012
-
A Repair Framework for Scalar MDS CodesAllerton 2012. [IEEEXplore], [arXiv]
-
Locally Repairable Codes
-
Feedback in the K-user Interference channel
-
Simple Regenerating Codes: Network Coding for Cloud Storage
-
Maximum-likelihood Blind PAM DetectionICC 2012. [IEEEXplore], [long]
-
Interference Alignment as a Rank Constrained Rank Minimization
2011
-
Repair Optimal Erasure Codes through Hadamard Designs
-
Distributed Storage Codes through Hadamard Designs
-
Sparse Principal Component of a Rank-deficient Matrix
-
Repairing Erasure CodesUSENIX FAST 2011, Work-In-Progress (WiP). [short abstract]
2010
-
Distributed Storage Codes Meet Multiple-Access Wiretap Channels
-
MCMC Methods for Integer Least-Squares ProblemsAllerton 2010. [IEEEXplore], [draft]
-
Interference Alignment as a Rank Constrained Rank Minimization
-
Maximum-likelihood Noncoherent OSTBC Detection with Polynomial ComplexityIEEE Transactions on Wireless Communications, June 2010. [IEEEXplore], [draft]
2007-2009
-
Optimal OSTBC Sequence Detection over Unknown Correlated Fading ChannelsAsilomar 2009. [IEEEXplore], [draft]
-
Efficient maximum-likelihood noncoherent orthogonal STBC detection
-
Polynomial-complexity maximum-likelihood block noncoherent MPSK detectionICASSP 2008. [IEEEXplore], [draft]
-
Near ML detection of nonlinearly distorted OFDM signals