Tom Mitchell in his 1997 Machine Learning textbook defined the well-posed learning problem as follows: “A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” In this talk, I will discuss current tasks, experiences, and performance measures as they pertain to fairness in machine learning. The most popular task thus far has been risk assessment. For example, Jack’s risk of defaulting on a loan is 8, Jill’s is 2; Ed’s risk of recidivism is 9, Peter’s is 1. We know this task definition comes with impossibility results (e.g., see Kleinberg et al. 2016, Chouldechova 2016). I will highlight new findings in terms of these impossibility results. In addition, most human decision-makers seem to use risk estimates for efficiency purposes and not to make fairer decisions. The task of risk assessment seems to enable efficiency instead of fairness. I will present an alternative task definition whose goal is to provide more context to the human decision-maker. The problems surrounding experience have received the most attention. Joy Buolamwini (MIT Media Lab) refers to these as the “under-sampled majority” problem. The majority of the population is non-white, non-male; however, white males are overrepresented in the training data. Not being properly represented in the training data comes at a cost to the under-sampled majority when machine learning algorithms are used to aid human decision-makers. There are many well-documented incidents here; for example, facial recognition systems have poor performance on dark-skinned people. In terms of performance measures, there are a variety of definitions here from group- to individual-fairness, from anti-classification, to classification parity, to calibration. I will discuss our null model for fairness and demonstrate how to use deviations from this null model to measure favoritism and prejudice in the data.

Host-based Intrusion Detection Systems (HIDS) automatically detect events that indicate compromise by adversarial applications. HIDS are generally formulated as analyses of sequences of system events such as bash commands or system calls. Anomaly based approaches to HIDS leverage models of normal (aka baseline) system behavior to detect and report abnormal events and have the advantage of being able to detect novel attacks. In this paper we develop a new method for anomaly-based HIDS using deep learning predictions of sequence-to-sequence behavior in system calls. Our proposed method, called the 𝐴𝐿𝐴𝐷 algorithm, aggregates predictions at the application level to detect anomalies. We compare and contrast various modeling approaches to make these predictions, namely, WaveNet, LSTM, and GRU. We show that the 𝐴𝐿𝐴𝐷 algorithm, empowered with deep learning predictive models, significantly outperforms previous approaches by comparison of ROC curves. To train and evaluate models we use existing event sequence corpora, and present a new corpus for HIDS development, called PLAID, that is more representative of modern systems. As deep learning models are black box in nature we use an alternate approach, allotaxonographs, to characterize and understand differences in baseline vs. attack sequences in HIDS datasets such as PLAID.

The ever-growing accumulation of data makes automated distillation of understandable models from that data ever-more desirable. Deriving equations directly from data using symbolic regression, as performed by genetic programming, continues its appeal due to its algorithmic simplicity and lack of assumptions about equation form. But, genetic programming has not yet been shown capable of transfer learning: the ability to rapidly distill equations successfully on new data from a previously-unseen domain, due to experience performing this distillation on other domains. Given neural networks’ proven ability to transfer learn, here we introduce a neural architecture that, after training, iteratively rewrites an inaccurate equation given its current error, regardless of the domain. We found that trained networks can improve their ability to derive equations from data produced by a test domain, when trained on data from several training domains. Although this phenomenon did not arise in all cases we tested, it does suggest that symbolic regression can more rapidly distill equations from data if exposed to data from a growing set of domains.

Extreme-mass-ratio inspirals (EMRI) are prospective sources for the detection of observational signals from the Laser Interferometer Space Antenna (LISA) mission, built to accurately detect and measure gravitational waves -- ripples in the curvature, and fabrics of space-time. EMRIs are typically comprised of a supermassive black hole (SMBH) one million times more massive than our Sun, and a stellar-origin black hole several orders of magnitude smaller. As the smaller black hole spirals into the supermassive black hole, thousands of gravitational waveforms serve as a precision probe for the extreme space-time curvature of the system. The goal of this research is to model the dynamics of and calculate the gravitational waveforms from “Trojan analog” EMRIs: multiple EMRIs in a single system, locked in 1:1 resonant orbits, analogous to Jupiter’s Trojan asteroids. These Trojan EMRIs hold a mix of unique observational potentials from those of single EMRI systems, that may be detectable from the LISA mission, while simultaneously providing detailed orbital dynamics around a supermassive black hole.

Increasing prevalence of drug resistant pathogens has driven a search for alternatives to traditional antibiotics. Anti-microbial peptides (AMPs), short amino acid chains that kill or inhibit certain classes of microbes, are a core component of many natural immune systems and show promise as supplements or alternatives. However, there are several issues that prevent widespread therapeutic applications of AMPs including potential toxicity to human cells, storage complications, and manufacturing difficulties. One approach that can mitigate these issues is the design of specialized AMPs with desirable properties. We develop a conditional generative machine learning model, AMP-GAN, to facilitate the discovery of candidate AMPs. Using the candidate AMPs generated by AMP-GAN, we use domain knowledge and molecular simulations to identify six particularly promising sequences. Finally, we validate the expected properties of the selected sequences using wet-lab experiments. This pipeline represents a proof of concept, showing how generative machine learning models can facilitate rapid development of specialized AMPs.

Deep learning has produced big advances in artificial intelligence, but trained neural networks often reflect and amplify bias in their training data, and thus produce unfair predictions. We propose a novel measure of individual fairness, called prediction sensitivity, that approximates the extent to which a particular prediction is dependent on a protected attribute. We show how to compute prediction sensitivity using standard automatic differentiation capabilities present in modern deep learning frameworks, and present preliminary empirical results suggesting that prediction sensitivity may be effective for measuring bias in individual predictions.

Probability Generating Functions (PGFs) provide an analytical and probabilistic description of random networks. In the context of emerging infectious diseases and their transmission trees, PGFs become a powerful tool for probabilistic forecasting that naturally accounts for the intrinsic stochasticity of disease transmission. However, transmission trees are incredibly noisy data as they either come from genomic sequencing with imperfect sampling, contact tracing in noisy environments, or biased interactions with other pathogens. It is therefore critical to evaluate the importance of data quality for probabilistic forecasts made with PGFs. This research explores the variation in final epidemic size for various degree distributions (i.e. distributions of secondary infections per case) and associated parameters given different levels of added error. The objective is to characterize the sensitivity to noise of PGFs in different network conditions. Preliminary results show larger uncertainty when adding error to more homogenous degree distributions, suggesting that the PGF framework might be better suited for diseases transmitted through direct contact rather than airborne infections, as the former tends to show more heterogeneous transmission patterns. These initial results lead to analytical perturbation theory approaches, where the perturbations of the PGF coefficients determine sensitivity measures for a desired root. The more sensitive the root, the more errors could yield inaccurate overall outbreak size. Altogether, this project paves the way towards a noisy and probabilistic forecasting framework for emerging epidemics while taking into account confidence in data and a network’s noise sensitivity

We study collective attention paid towards hurricanes through the lens of n-grams on Twitter, a social media platform with global reach. Using hurricane name mentions as a proxy for awareness, we find that the exogenous temporal dynamics are remarkably similar across storms, but that overall collective attention varies widely even among storms causing comparable deaths and damage. We construct ‘hurricane attention maps’ and observe that hurricanes causing deaths on (or economic damage to) the continental United States generate substantially more attention in English language tweets than those that do not. We find that a hurricane’s Saffir-Simpson wind scale category assignment is strongly associated with the amount of attention it receives. Higher category storms receive higher proportional increases of attention per proportional increases in number of deaths or dollars of damage, than lower category storms. The most damaging and deadly storms of the 2010s, Hurricanes Harvey and Maria, generated the most attention and were remembered the longest, respectively. On average, a category 5 storm receives 4.6 times more attention than a category 1 storm causing the same number of deaths and economic damage.

Diarrheal disease is the second leading cause of non-birth related death in children under age 5 years in low- and middle-income countries (LMICs). Recent large-scale studies have demonstrated a high prevalence of simultaneous enteropathogen co-infection during infant diarrheal episodes, raising questions about diarrheal etiology. Additionally, studies of acute diarrhea have found co-infection strengthens pathogenesis. Despite these insights, co-occurrence patterns among enteric pathogens and their relative contributions to disease risk remain unexplored. To identify patterns of enteric co-infection we built a bipartite network of enteric pathogens and individual stool samples collected from 791 infants in the MAL-ED and PROVIDE birth cohorts from Dhaka, Bangladesh. Data on enteric pathogen infection in 1,713 diarrheal and 3,485 asymptomatic stools were obtained using a qRT-PCR TaqMan Array Card panel of 34 enteropathogens. The observed bipartite network is dense, therefore, to isolate the underlying structure of co-infections we compare the observed number of co-occurrences to an ensemble of null communities generated from randomizing the original network by controlling for the frequency of distinct pathogens per observation and the total incidence of each pathogen. Among other pairs that were found using this method, we found co-infection with Campylobacter sp. occurs more frequently than expected with Shigella sp. in diarrheal stools while rotavirus co-infects with enterotoxigenic E.coli (ETEC) less frequently than expected in diarrheal stools. In asymptomatic stools, ETEC is found with atypical enteropathogenic E.coli (EPEC) and typical EPEC more than expected. As we move beyond the One Pathogen, One Response paradigm, interventions to prevent diarrheal disease in infants in LMICs must account for the presence of co-pathogen infections. Identifying significant co-infection patterns will inform treatment approaches and vaccine development to eliminate the excess burden of diarrheal disease.

Sleep loss has been linked to heart disease, diabetes, cancer, and an increase in accidents, all of which are among the leading causes of death in the United States. Population-scale sleep studies have the potential to advance public health by helping to identify at-risk populations, changes in collective sleep patterns, and to inform policy change. Prior research suggests other kinds of health indicators such as depression and obesity can be estimated using social media activity. However, the inability to effectively measure collective sleep with publicly available data has limited large-scale academic studies. Here, we investigate the passive estimation of sleep loss through a proxy analysis of Twitter activity profiles. We use “Spring Forward” events, which occur at the beginning of Daylight Savings Time in the United States, as a natural experimental condition to estimate spatial differences in sleep loss across the United States. On average, peak Twitter activity occurs roughly 45 minutes later on the Sunday following Spring Forward. By Monday morning however, activity curves are realigned with the week before. This represents a compression of the sleep opportunity window, demonstrating a reflection of the hour of sleep lost in Twitter activity.

Major depression is a serious health issue afflicting hundreds of millions of people each year, with many going untreated due to the intense stigma surrounding mental illness. In the present study, we explore trends in words and phrases related to mental health through a collection of 1-, 2-, and 3-grams parsed from a data stream of roughly 10% of all English tweets since 2012. We examine temporal dynamics of mental health language, finding that the usage rank of the phrase ‘mental health' has increased by an order of magnitude since 2013. We also investigate the ambient sentiment of tweets containing the phrase ‘mental health', highlighting specific dates where sentiment deviates substantially from the baseline. Finally, we use the balance of original tweets to retweets to quantify the fraction of appearances of mental health language due to social amplification. Since 2015, appearances of mental health related phrases are increasingly due to retweets, suggesting that stigma associated with discussion of mental health on Twitter has diminished with time.

This talk describes a novel approach to secure federated machine learning using techniques from secure multiparty computation (MPC), and differential privacy (DP). Federated machine learning is a model training technique utilizing a federation, or group, of distributed edge device, each supplying input data without aggregating data samples. On principle, federated learning affords data holders privacy by not requiring them to publish their private data. However, it allows adversaries the possibility to access or reconstruct the inputs. We propose a mixed mode protocol that utilizes both DP and MPC to rapidly train an accurate and privacy-preserving model without the use of a third party. We break federations into groups that use MPC to calculate differentially private gradients. An aggregator collects and combines these private gradients at each iteration, thus training one centralized model. We also compare the performance of various network layer topologies implemented at the transport layer including point-to-point TCP, point-to-point UDP, and star TCP topologies. Our initial results show that our framework is able to scale to 1000s of parties.

Deep learning has rightly captured the imagination of AI researchers and the public alike, for the sudden and unexpected advances it has made in pattern recognition. However, a shift is now evident in the field away from technologies that analyzing incoming patterns, to generative systems that produce useful patterns. I will describe my group’s approach to automated design, which rests on two pillars: evolution and embodiment. To illustrate this approach, I will describe three applications: tasking AI with generating soft robots, crowdsourced robots, and biological robots. Along the way I will describe parts of my own journey from graduate student to professor.