Watching Playthroughs as a Means of Evaluating Games

We've heard a lot of feedback specifically about watching playthroughs and how Hugo voters can evaluate or vote on games they have not played or cannot play. However, the issue of how you experience a work for voting purposes is not unique to games. This is a long post, but we want to be clear that the issues around watching playthroughs exist in just about every medium, and the practice is not particularly mysterious or esoteric.

What Playthroughs Are and Types of Playthroughs

A recording or stream of a playthrough shows to the viewer what the player(s) see: the game itself. For digital games this will be the game screen itself, including interface elements where applicable. Streamers will often put an inset of themselves on the screen as well so you can also see the player, but the main focus is the larger portion of the output that shows the game itself. For VR or AR games, this will show wherever the player is looking inside the game environment, with the augmented or rendered elements as the player sees them (barring some field of vision distortion). For analog games, a playthrough will show the state of play, meaning the board, cards, dice, or other relevant physical materials, and often the players themselves. In all cases, the central purpose is for the game itself to be clearly displayed in the playthrough.

There are many styles of playthroughs, with a common category/genre in the world of video games being popularly called Let's Plays or LPs, as in "Let's Play Outer Wilds." A popular type of LP will have a skilled player navigating the game to show off its story and gameplay. These vary widely by gameplay style, presentational style, and use or lack of commentary. Many feature players familiar with the game guiding viewers through all the available content in the game (completionist or 100% runs), often showing multiple results for branching narratives and taking the time to show significant environmental materials such as signs, notes, and in-universe artwork. For roleplaying games, players will generally try to stick to the chosen personality and build of the character, sometimes even soliciting audience input on what that build should be before beginning. Some playthroughs have the player going in cold, giving viewers the experience of someone playing a game for the first time. Entertainment-centered streams or playthroughs are often more about watching someone have fun than necessarily playing the game well. And, of course, speedruns are a wildly popular style of run that, while generally not useful for purposes of evaluating story, gameplay, or SFnal content, nonetheless give a picture of how big the playthrough and streaming communities are. Games Done Quick, the longest-running series of speedrunning marathons/conventions, has raised over $37.8 million for charity since 2010.

Comparisons to Other Media

We gave the example of live theatre earlier, which is a great example of fans using different means to evaluate a work, so we're expanding on it a bit here. A musical like Hadestown is widely regarded as a worthy SF work, but very few people would have been able to experience it fully as intended during its window of Hugo eligibility. Fans may live too far away and not have the means to travel, or live proximate but not be able to afford to go. The COVID-19 pandemic disrupted its release to the broader public. A fan may have sensory limitations or issues that make processing the experience of watching a live performance difficult, unpleasant, or impossible. For these and many other reasons, we broadly accept that reading the script, listening to the soundtrack, and watching clips or full recordings (official or illicit) are valid means towards saying you're a fan of a musical and for judging how good you think it is. These methods are obviously not equivalent to the experience of a live performance, but they are perfectly valid, whether this path is a choice or a necessity. Certainly we wouldn't tell a blind person they didn't have a valid basis for voting on a musical. The experience people have of a work is the experience people have.

Audiobooks provide another example. With an audiobook, you experience the prose at a speed and style not of your choosing. WSFS members agree that this is a valid way of experiencing a prose work. This can make prose more accessible to some readers, or accessible at all, e.g. if someone is low-vision or if their library only carries the audiobook version. Audiobook narrators vary in style and in quality, and different people will prefer different narrators. The same is true for players who are making Let's Plays: as someone watches more streams, they will learn whether people are good players, or if they do a style of playthrough that suits them.

We also evaluate movies watched on our laptops even though in many many ways it's not the intended experience, e.g. not getting to see as much detail or experience the fine points of the sound design. We even feel perfectly equipped to evaluate games without playing them when it comes to sports. Sports fans say, "this is a good sport," without playing it, whether they mean fun to watch, requiring skilled play, or any other metric. We even compare entire sports wholesale ("I prefer European football over American football") without playing them.

In the end, we accept many means of evaluating specific works or even entire genres of entertainment. More to the point, we accept people as fans based on the experience they are capable of or prefer to have.

Evaluating Playthroughs and Finding Good Playthroughs

A common issue even for those well-familiar with playthroughs is finding quality playthroughs and/or streamers that they like. It also must be acknowledged that the gaming community has suffered under virulent toxicity, some of which has even spilled over into the world of the Hugos via the Puppies. Navigating this landscape to find streamers compatible with your viewing and/or playing style is a known issue within the community, just as much as streaming is a wildly popular and accepted means of experiencing games. For purposes of Hugo voting, many voters will want to focus on evaluating the story without overlaid commentary, but as stated previously there are many types of playthrough that cater to different evaluation and entertainment preferences.

Towards this end, both major platforms for playthroughs — Twitch and YouTube — provide means for finding compatible players. Twitch categorizes by the game the streamer is playing, so viewers interested in a particular title can browse streamers who actively play it. Twitch also allows users to tag their streams in various genres and categories. Browsing by lower average concurrent viewership (AVC) will often lead to streams where you have a higher chance of influencing gameplay, and watching the chat in higher AVC streams (as well as listening to any commentary from the streamer) will generally quickly reveal the tenor of the stream. On YouTube, there are many channels devoted entirely to no-commentary playthroughs and game movies. Searching for a given game alongside the term "no commentary playthrough" will yield multiple results for most major or popular releases. The absence of commentary and chat obviates many compatibility factors with players and fellow viewers and allows for a focus on the game and the skill of the player. However, many viewers enjoy commentary, especially from "cold" or first-time playthroughs, in which case just a couple of minutes of watching actual gameplay (or even hearing the player's commentary on opening cutscenes) will generally signal player skill and compatibility.

It sounds like a bit more of an art than a science, and that's true! It's much like finding good audiobook narrators. If you find one you like, you may even follow them to other works that you may not have otherwise given a try, and the same is true for players doing playthroughs.

It's also important to point out that there is a strong community factor in play, especially in the context of Hugo voting. While Hugo administrators are very unlikely to include particular playthroughs or reviews in a Hugo Voter Packet in order to avoid bias (though please read our section on how a games Hugo Voter Packet would work — and has worked!), fans are welcome to recommend things to each other, and we expect this will become part of the community discussion about nominees for Best Game. In fact...

Example of Crowdsourced Videos on the 2020 Best Video Game Finalists

...The Games Hugo campaign has put together a list of some examples of videos that could have been used to evaluate the 2020 Best Video Game finalists without playing them. The biggest factor here is time, which is an aspect we've covered elsewhere in the FAQ, but watching videos such as those below is still generally faster than playing a game cold yourself. The videos below are a mix of playthroughs, game movies, recaps, and reviews that give some perspective on each game and what fans found valuable about them (or not). These are not meant to be recommendations of any particular channels or approaches and more a survey of the sort of easily accessible content available for various titles.

Animal Crossing: New Horizons

Blaseball

Final Fantasy VII Remake

Hades

The Last of Us: Part II

Spiritfarer

Would Playthroughs Themselves Qualify in this Category?

Generally, no. The most popular type of playthrough is one in which the player does not solicit or take into account audience input into how the game is played. This is normal and straightforwardly does not qualify, as there is not an interactive element of the playthrough itself. Machinima likewise do not qualify, as the interactivity is in the base game, not in the finished product that is released to the public, which typically would qualify in the Best Dramatic Presentation or Best Fancast categories, as would typical noninteractive playthroughs.


Critical Role is a fantastic case to examine. There is the show itself, which is broadcast on a regular basis as unscripted recordings of gaming sessions, and there is also the specific "homebrew" version of Dungeons & Dragons 5th Edition used to create the show. Matt Mercer, the Game Master, created not just a specific setting, worldstate, and NPCs, but also custom classes that significantly affect the framework previously provided by D&D5e. This modification is codified and publicly available as Critical Role: Tal’Dorei Campaign Setting and Critical Role: Call of the Netherdeep. They are interactive in nature, providing a framework for users to enact unique narratives in the Critical Role version of D&D5e and play out their own Critical Role games. The show, being non-interactive and requiring a specific cast of people, is a clear candidate for Best Dramatic Presentation or Best Fancast, but the gaming framework used in creating the show would indeed qualify as a "interactive substantial modification of a work".


Interactive playthroughs, such as any given Twitch stream with chat enabled, would generally not qualify either. While the streamer may be providing a particular means and environment for viewers to interact with both the streamer and the game, this experience is generally predicated on the involvement of that specific person, the streamer themselves. However, a playthrough like Twitch Plays Pokémon does not require the presence of any specific people and instead provides a platform for the public to interact with the base game in a unique way. The initial entry in this series allowed over a million people to crowdsource playing Pokémon Red, and the series has continued to provide an interactive framework for thousands of participants to play a series of official and unofficial Pokémon games.

The Bottom Line: Accessibility and Democracy

A crucial aspect of playthroughs is that they democratize access to games in terms of finances, logistics, disability, skill, and any number of other factors.

Watching other people play often helps even those who play the game themselves, and often increases accessibility in other ways. Content that is difficult, gameplay-wise, to access can be shown off by skilled players so that a wider audience is exposed to it. Gamers also often use playthroughs to evaluate whether they want to play — or, more to the point, purchase — a game themselves. In the gaming community, it's a popular and well-accepted way to judge works, and it removes barriers such as ableism, classism, and other exclusionary factors in the same way that an audiobook makes prose accessible to a blind or low-vision person, or a script makes a dramatic presentation accessible to a D/deaf person.

It must be acknowledged that watching playthroughs for evaluating games does elide the characteristic that the Games Hugo campaign maintains sets games apart from all other media: the interactivity. Many games have such widely branching narratives or significantly incorporate haptic elements that watching playthroughs leaves out significant aspects of the intended experience. However, many other games, such as The Last of Us, are entirely linear, with the gameplay simply serving to gate and pace the linear plot, and for those unable to do or not interested in attempting the mechanics of its execution, watching the story play out and how a player navigates the challenges (which character attributes to focus on, what weapons to choose, aiming and shooting, executing stealth) conveys plenty of information about the quality and content of the game. That doesn't make The Last of Us better or worse, or less of a game, and that doesn't make watching playthroughs of such a game more or less valid than others. Games themselves are constantly evolving in their accessibility features. The Last of Us: Part II has robust accessibility features, but that doesn't mean that players who could play the second game but not the first game are less qualified to give opinions on the first game just because they couldn't play it themselves or had a very different gameplay experience. In fact, it gives such players an entirely different and rich aspect on which to evaluate both games qua storytelling experiences for them. It's not up to us to decide that watching playthroughs is not valid, whether it’s a choice made by the audience or a necessity. You might not ever evaluate a game this way for Hugo voting purposes. But it shouldn't stop others from doing it, and for having a category in which to do it.

To be clear, we are not saying that watching playthroughs is equivalent to playing a game. The entire basis of our proposal is that the interactive quality of games sets them apart from any other type of storytelling, and watching a playthrough is simply not the same experience as playing a game yourself. However, we firmly believe that watching playthroughs is a valid basis for voting in the Hugo for any individual voter so inclined. It does not mean everyone, even every gamer or Hugo voter interested in games, has to espouse this view vis-a-vis their own voting. But policing the means by which people experience a work is ableist, classist, and generally exclusionary. It's ableist to say that blind or low-vision people cannot evaluate a novel via its audiobook, or that D/deaf people cannot evaluate a podcast by its script and supporting materials. It's classist to say that those who can't afford to see a live theatrical production can't form opinions on it based on reading the script, listening to the soundtrack, watching clips, or watching recordings of the show. Players with mobility or processing issues (or any number of other conditions or even preferences) may not be able to play a given game themselves, but they can still decide if it's good in their opinion. None of these experiences, obviously, are equivalent, but they are valid.

In the end, it's antithetical to the democratic spirit of the Hugos to say that this well-established means of experiencing games is not a valid way for any given Hugo voter to evaluate games.

We believe a Best Game or Interactive Work Hugo category not just could, but needs to exist. And playthroughs are a valid way to participate in this category.