Tag Archives: hedonometer

Hedonometer 2.0

Geography of Happiness for the US

Geography of Happiness for the US

Over the summer of 2014, we have worked very hard to bring many new pieces to our Hedonometer, and we’re pleased to tell you about what we’ve done, and where we’re going next.

Snapshot of Hedonometer 2.0's happiness time series.

Snapshot of Hedonometer 2.0′s happiness time series.

All along, one of the central goals for the Hedonometer has been to provide a new instrument for society’s dashboard, one that measures population-level happiness in real time from any streaming text source. Like flying a plane, where we would never want just one dial with the limits “all good” and “uh-oh”, we need a sophisticated dashboard to quantify how well a population is faring. We want to see unconventional measures like ours added to traditional, easier-to-gauge quantities often concerned with economic activity. Money doesn’t equal happiness. We hope the Hedonometer will enable individuals, journalists, policy makers, corporations, and other research teams in their various pursuits.

Because our Hedonometer works for any large text, we’re able to explore other areas for basic science purposes, particularly the vast realm of sociotechnical systems and the digital humanities. And some of our work will be simply just for fun (hopefully yours and ours).

Harry Potter and the Prisoner of Azkaban.

Harry Potter and the Prisoner of Azkaban.

As you’ll see below, we have many plans for the future. So far, we’ve received crucial support from the NSF and the MITRE Corporation, and we’re always looking for more ways to continue to lift our enterprise. If you’re interested in or have suggestions about funding our work, please contact us.

Okay—here’s what we’ve put together. We now have four main interactive views of emotion up and running:

  1. A completely rebuilt global Twitter happiness time series in English, updated daily and with powerful new word shifts;
  2. An interactive map of happiness for the 50 US States plus DC, also based on Twitter;
  3. A ranked list of cities by happiness for the US (Twitter again);
  4. and an explorable visualization of the emotional plot trajectories of 10,000 books in 10 major languages including Harry Potter along with classic and obscure works.

We’ll go into more depth about how to use and share these visualizations in our following blog posts. As always, Hedonometer stands on a team effort but we have to acknowledge and praise Andy Reagan (@andyreagan) for his incredible efforts in leading the charge to Hedonometer 2.0. Building things is fun.

Some of the many new elements we’re looking to add in the next year are:

  1. Other kinds of real-time, population-scale meters based on word usage including sleep, food consumption, exercise, binge-drinking, and boredom. We’ll apply these meters to Twitter but they could in principle be used on any text.
  2. Global Twitter happiness time series and maps for all 10 languages: English, Spanish, French, German, Brazilian Portuguese, Indonesian, Korean, Simplified Chinese, Arabic, and Russian;
  3. Real-time Twitter happiness at 1 minute time scales.
  4. An interactive world map with the ability to explore at scales of country, state, city, and district (or equivalents).
  5. Simple ways to embed our interactive visualizations into webpages;
  6. A simple interface for uploading and comparing two texts, and for generating shareable visualizations;
  7. Phrase-based rather than word-based analysis in English;
  8. More stand-alone projects such as interactive visualizations of music lyrics over the last 60 years;
  9. Measures based on other major emotions such as fear, disgust, anger, and surprise.

We also have two longer-term, major projects in development:

  1. A fast search facility in the Hedonometer for users to find the emotional spectrum around specific words or phrases. This is a computationally bundensome problem. We’ll be able to show how the emotional texture of how people are talking about an event or a product.
  2. Storybreaker: a real-time extractor of stories and narratives emerging around major events. Our algorithm will include emotion but our goal is to measure frames around issues and ultimately meaningful stories.

One last thing: we’ve moved our blog from onehappybird to compstorylab.org. All old links will still work.

How does movement influence your daily happiness?

Imagine commuting an hour to work, one way, grinding through miles of traffic to get from your suburban home to a desk job in the big city. Excited yet?

Ok, now imagine that you lead a life of leisure traveling the world. You fly coast-to-coast to see a concert, soak in some culture, and drink fine wine. Does this lifestyle seem more appealing?

Lets try to quantify the influence of these travel patterns on individual happiness. We do this using geolocated tweets, which we have previously used to reveal the happiness of cities, and to quantify patterns of movement.

Each point corresponds to a geo-located tweet from 2011. (A) USA (B) Washington, D.C. (C) Los Angeles (D) Earth

Each point corresponds to a geo-located tweet from 2011.
(A) USA (B) Washington, D.C. (C) Los Angeles (D) Earth

First, we find the average location of each individual’s tweets. We call this their expected location. Then we draw circles emanating from this spot, like rings on a dart board. Some messages are written close to home, others from very far away.

Then we collect all of the words written at each distance, roughly 500,000 tweets per ring. Averaging the happiness of words found at each distance, remarkably we find that happiness increases logarithmically with distance from expected location. Tweets authored far from home contain a smaller number of negative words.

Tweets are grouped into ten equally populated bins by the distance from their author's average location, and the average happiness of words written at each distance is plotted. Expressed happiness grows logarithmically with distance from home.

Tweets are grouped into ten equally populated bins by the distance from their author’s average location, and the average happiness of words written at each distance is plotted. Expressed happiness grows logarithmically with distance from home.

Home is where the hate is? What? No.

Below we look at the difference between the happiest and saddest distances from home. Words appearing on the right increase the happiness of the 2500km distance relative to the 1km distance. For example, tweets authored far from an individual’s expected location are more likely to contain the positive words `beach’, `new’, `great’, `park’, `restaurant’, `dinner’, `resort’, `coffee’, `lunch’, `cafe’, and `food’, and less likely to contain the negative words `no’, `don’t’, `not’, `hate’, `can’t’, `damn’, and `never’ than tweets posted close to home. Words going against the trend appear on the left, decreasing the happiness of the 2500km distance group relative to the 1km group.

Word shift graph comparing the lowest average word happiness distance group to the words authored farthest from home.

Word shift graph comparing the lowest average word happiness distance group to the words authored farthest from home.

Tweets written close to home are more likely to contain the positive words `me’, `lol’, `love’, `like’, `haha’, `my’, `you’, and `good’. Moving clockwise, the three insets show that the two text sizes are comparable, the biggest contributor to the happiness difference is the decrease in negative words authored by individuals very far from their average location, and the 50 words listed make up roughly 50% of the total difference between the two bags of words. For you visual learning folks, here is a short video explaining how these word shifts work.

Take home story: people tweeting far from home talk about food more, and they swear less than people tweeting close to home. These people are probably enjoying awesome vacations, and tweeting about it!

In summary, if you are a fellow with a daily commute that makes you feel a little bit sad, you are not alone! Try swearing less. Or ride your bike.

If you are lucky enough to travel often, then keep smiling…maybe send the rest of us some pictures to cheer us up!

For more details on our analysis, check our paper “Happiness and the Patterns of Life: A Study of Geolocated Tweets” recently published in Nature Scientific Reports.

Now Published: The Geography of Happiness

Today we’re pleased to announce that our article “The Geography of Happiness: Connecting Twitter sentiment and expression, demographics, and objective characteristics of place” has been officially published by PLoS ONE.  We wanted to tell you about one key piece we’ve added to the paper and an unusual new Twitter account we’ve created.

After our three blog posts (which coincided with the release of the preprint), we received plenty of media attention, as well as some fantastic feedback from readers (thanks!). One very important question kept coming up: “How well does happiness agree with other measures of well-being?”, or more simply: “Why should we believe you?”

Well, we’re glad you asked.  For the final paper, we’ve added a US state-level comparison between our happiness measure and five other kinds of well-being indices:

  • the Behavioral Risk Factor Surveillance Survey (BRFSS)  for which people were asked to rate their life satisfaction on a scale of 1 to 4 (the BRFSS was explored in this Science paper on well-being from a few years back);
  • Gallup’s health survey-based well-being index;
  • the Peace Index, which aggregates various crime data;
  • the America’s Health Ranking, which aggregates health data; and
  • gun violence, specifically the number of shootings per 100,000 people.

In the figure below, we show a series of scatter plots comparing all pairs of well-being metrics  (happiness runs along the top row).  Each dot represents a US state, and the colors represent strength of correlation or agreement between measures, with blue meaning strong agreement, and red representing no (statistically significant) agreement. (We include the exact Spearman correlation coefficienr and p-value in each scatter plot.)

happinessScatterMatrix1

Scatter matrix showing comparison between different well-being metrics for all US states. The top row shows comparisons with happiness. Colors indicate the strength of correlation between pairs of metrics; shades of blue indicate increasingly significant correlation.

Looking at the top row, we can immediately see that happiness agrees with all measures except for the BRFSS. However, the BRFSS itself doesn’t agree with any other measure except for the Gallup well-being index.  The most striking departure was the BRFSS ranking Louisiana as the happiest state whereas our happiness measure placed it last.  There are a number of possible explanations for these disagreements: one is that the BRFSS data was taken between 2005 and 2008, while all other data is from 2011 only; another is that unlike the other measures, happiness is self-reported in the BRFSS. How would you answer if asked how happy you are? Do you expect that your answer is representative of the population you live in at large? There are certainly many different ways to define “happiness”, as a number of different readers have pointed out.

Of course, this is not to criticize the BRFSS (it remains a significant data source, and Oswald & Wu did fine work analyzing it in their Science paper), but merely to suggest that our word happiness score is measuring something different but perhaps complementary to traditional survey-based techniques. There certainly appears to be plenty of value to observing people “in the wild” via social network data, e.g. with the real-time instrument hedonometer.org.

Finally, to celebrate the publication of our article we created a Twitter feed, @geographyofhapp, dedicated to tweeting the happiest and saddest city every day, and we invite you to follow.  We’re hoping that this is the first research article with its own Twitter account, but perhaps not hoping that it represents the future of scientific publishing…

Where is the happiest city in the USA?

(Update: this work is now published at PLoS ONE)

Is Disneyland really the happiest place on Earth?* How happy is the city you live in? We have already seen how the hedonometer can be used to find the happiest street corner in New York City, now it’s time to let it loose on the entire United States.

We plotted over 10 million geotagged tweets from 2011 (all our results are in this paper, also on the arxiv), coloring each point by the average happiness of nearby words (detail on how we calculate happiness can be found in this article published in PLoS ONE):

Image

As well as cities and the roads between them, we can make out many regions of higher and lower happiness, even within individual cities. As an example, check out this tweet-generated map of the city of Chicago:

Tweet-generated map of Chicago. Click to enlarge.

Tweet-generated map of Chicago. Click to enlarge.

Notice the striking contrast between the relatively happy Central/North Side of the city, and the sadder South Side. You can also find a few airports in this map, and if you look very closely you might even be able to pick out happy and sad terminals!

To quantify this variation in happiness a bit better, let’s look at the average happiness of each state:

Image

Southern states tend to produce sadder words than those in northern New England or out west. Hawaii emerges as the happiest state and Louisiana as the saddest, due to relative differences in the frequencies of happy and sad words used in each state. Here at onehappybird, we characterize such differences by “word shifts”, which are basically word clouds for grown-ups. You can find examples of these, as well as the full list of the average happiness of each state, here (page best viewed using Google Chrome).

Zooming in further to the level of cities, we produced a similar list for 373 cities in the lower 48 states (you can find the full list, as well as maps and word shifts for each city, here). With a score of 6.25, we found the happiest city to be Napa, CA, due to a relative abundance of such happy words as “restaurant”, “wine”, and even “cheers”, along with a lack of profanity.

wordshiftNapaRedacted

At the other end of the spectrum, we found the saddest city to be Beaumont, TX, with a score of 5.82. In general, cities in the south tended to be less happy than those in the north, with a major contributing factor being the relative abundance of profanity used in those cities.

We can go even further than this, and group cities by similarities in word usage. Each square in the heatmap below represents the similarity (Spearman correlation for you mathematically minded onehappybird watchers) between word distributions for the largest cities in the US. Red squares mean that the corresponding cities use words in a similar fashion, while blue means that those cities tend to use different types of words with respect to each other. The colors in the tree diagram at the top signify clusters of cities exhibiting similar word usage (below a certain threshold).

As we might expect for two cities that are geographically nearby, New Orleans and Baton Rouge are clumped together at the bottom right of the figure. On the other hand, New York and Seattle get clumped together as well, suggesting that similarities in language depend on more than just geographical proximity.

cityClusters

You can find more information about happiness and cities, as well as details on the methods used to produce these results, in our arxiv research article. In our next post, we’ll look at how these results are related to various underlying socioeconomic characteristics of cities. What makes a city happy or sad? Can we use Big Data to predict future changes in the demographics, health, or happiness of a city? How does happiness relate to the food you eat?

*By the way, to answer the question at the start of this post: According to this analysis Disneyland is not the happiest place on Earth; it isn’t even the happiest place in Southern California! See if you can find it in this tweet-generated map of LA! Or find your city here.

Hedonometrics

Our paper “Temporal Patterns of Happiness and Information in a Global Social Network: Hedonometrics and Twitter” appears in PLoS ONE this week. Their blog encourages you to tweet for the sake of science!

Among other findings, in this paper we demonstrate that human ratings of the happiness of an individual word correlate very strongly with the average happiness of the words that co-occur with it. This implies that tweets containing particular keywords can be used as an unsolicited public opinion poll.

For example, tweets containing “Tiger Woods” became decidedly less positive after his Thanksgiving disaster in 2009 as the words ‘accident’, ‘crash’, ‘scandal’, and ‘cheating’ are more abundant, while the word ‘love’ appears less often.

Happiness is measured relative to the ambient background of all tweets.

Sad words are blue, happy words are yellow. Up (down) arrow indicates that the word appeared more (less) frequently in tweets containing "Tiger Woods".

Generally, tweets containing personal pronouns tell a positive prosocial story with ‘our’ and ‘you’ outranking ‘I’ and ‘me’ in happiness. The least happy pronoun on our list is the easily demonized ‘they’.

Emoticons in increasing order of happiness are ‘:(’, ‘:-(’, ‘;-)’, ‘;)’, ‘:-)’, and ‘:)’. In terms of increasing information content (diversity of words co-occuring with each emoticon), the order is ‘:(’, ‘:-(’, ‘:)’, ‘:-)’, ‘;)’, and ‘;-)’. We see that happy emoticons co-occur with words of higher levels of both happiness and information but the ordering changes in a way that appears to reflect a richness associated with cheekiness and mischief: the two emoticons involving semi-colon winks are third and fourth in terms of happiness but first and second for information.

A list of the happiness ratings of tweets containing some interesting keywords can be seen here.

And not surprisingly, the happiness of all tweets appearing on a given day of the week correlates well with the happiness ratings humans give each day.

Happiness of tweets appearing on a given day

Human ratings of the happiness of each day of the week

You can download the language assessment by Mechanical Turk (labMT 1.0) word list here. It is a text file containing the set of 10,222 most frequently occurring words in the New York Times, Google Books, music lyrics, and tweets, as well as their average happiness evaluations according to users on Mechanical Turk.  See the paper for details.

Much more to come regarding sociotechnical phenomena…

Happy and we know it

Science Magazine published a piece today framing twitter as a laboratory for research, Social Scientists Wade Into The Tweet Stream, including the above figure showing our hedonometer’s measure of happiness in 2011 as a function of day. Dodds was also interviewed by Science for their weekly podcast, and by Benedict Carey for a New York Times piece, Happy and You Know It? So Are Millions on Twitter.