This post was formerly titled "Syuzhet for Dummies." Lightly updated in February 2025. This blog post has been read about 5000 times as of the spring of 2025.
There was quite a bit of discussion of Matthew Jockers’ “Syuzhet”package on scholary blogs when it was first released in February 2015. When we talked about Syuzhet in class in our DH class that fall, one of my students noted that if this same back-and-forth had taken place in scholarly journals it might have taken years to have the conversation play out. So: score one for scholarly blog conversation.
My goal here is not to add anything substantive to the discussion, but rather to describe
my own learning curve and give others, including students, a few tips as to how
to do things to play around further. My assumption is that the reader is not experienced with R, and I will guide you through installing it and getting started. If you are already an R user, Jockers' documentation will probably be more useful to you than the blog post below.
I’ll start with a brief summary
of the scholarly debate about Jockers’ work and method as I understand it, and then give readers
a step-by-step account of how to emulate the process of using the R programming
language in order to apply them to other texts.
* * *
The 2015 Debate About
Syuzhet
(People who are already familiar with the Syuzhet debate might want to scroll down to "How To Try It Yourself")
Start by reading Matthew Jockers’ intitial post on Syuzhet here (Feb 02 2015):
And the follow-up here (Feb 22, 2015):
A basic point or two that might not be obvious to newbies. First, Jockers is not inventing the idea of sentiment analysis out
of the blue. He is working with algorithms for analyzing sentences from four
separate processes that have been developed to do that.
These are algorithms that are currently mostly used to scrape social media posts online to ascertain consumer responses to commercial products in the marketplace; academics have been using them for very different purposes.
Sentiment analysis approaches like the Bing lexicon (named after CS professor Bing Liu, not Microsoft's "Bing") are derived from a dictionary-based method where a lexicon of words
has assigned sentiment values (presumably a word like “adore” would have a
positive sentiment value, while a word like “detest” would have a negative
one). The “Stanford” natural language processing method is more complex; they have used computational linguistics to
diagram whole sentences and plot their sentiment arc. This ought to be a more sophisticated and accurate
way of reading the sentiment of individual sentences, but in a subsequent test, Jockers decided that the Stanford method actually corresponds less well than dictionary-based
methods like Bing to his own, hand-tagged reading of a large but limited set of trial sentences (the entire text of Joyce's Portrait of the Artist as a Young Man!). In most of his blog posts on Sentiment Analysis, Jockers
bases his examples on the Bing algorithm.
[Update from 2025: at the time I first wrote this, I didn't know about the NRC emotion lexicon. In fact, with its more complex range of affects -- joy, disgust, anger, trust, sadness, surprise, anticipation, fear -- NRC might ultimately have the potential to be more useful to literary critics, though the simple +/- graphs below might not be relevant. Instead, we might need a more complex mode of visualizing an eight-point framework. In Jockers' updated documentation from 2023, scroll down to "get_nrc_sentiment" too learn how to use this.]
Next, read Annie Swafford’s first critique here (there are several). There are two
prongs of the critique, one mathematical (do these sentiment-processing
algorithms read the emotional valence of individual sentences correctly?), and
the other oriented to the method by which Jockers creates his rather
astonishing visualizations of novels.
Jockers responds to the mathematical questions Swafford
raises by arguing that while dictionary-based algorithms do indeed have the
potential to misread individual sentences, most times in traditional literary
works ambiguous sentences will be surrounded by contextual sentences that clarify
the mood. On average, Jockers suggests, the algorithms do a pretty good job
finding the average sentiment of larger chunks of text, while they might indeed be prone to misread very small samples.
On the visualization question, the most damning point Swafford makes is that Jockers’ clear and striking sentiment graphs are
based on a method (Fourier Transformation + a low pass filter) that is likely to introduce false artifacts upon visualization (she calls them "ringing artifacts"). So the problem isn't that these diagrams are too clean and simplified to be useful, the problem is in fact that they might not be true to the text.
Jockers initially pushed back against Swafford (and other
critics who made arguments similar to hers). But in April 2015, in a blog post
called “Requiem for a Low Pass Filter,” he changed his mind about the Fourier
transformation he had been using and advocating. In effect, he acknowledged that the beautiful “foundation
shapes” he had been hoping to use his “Syuzhet” package to create (and allow
others to create) gives us images that are actually too distorted to trust.
All we have is an R package, syuzhet, which does something I would call exploratory data analysis. And it’s hard to evaluate exploratory data analysis in the absence of a specific argument.
For instance, does syuzhet smooth plot arcs appropriately? I don’t know. Without a specific thesis we’re trying to test, how would we decide what scale of variation matters? In some novels it might be a scene-to-scene rhythm; in others it might be a long arc. Until I know what scale of variation matters for a particular question, I have no way of knowing what kind of smoothing is “too much” or “too little.”