Developing Literacies in the Digital Humanities Classroom: A Case Study

Developing literacies in the digital humanities classroom include, yet transcend, the ‘traditional’ passive literacies of reading, hearing, and seeing into the active realms of finding, evaluating, creating, engaging and communicating with an audience that may extend beyond institutional boundaries. These skills have been presented as a theoretical framework by Yoram Eshet et al. (2004, 2006, 2012), amongst others. Their model has developed over time to include six digital literacies, generally categorized as technical-procedural, cognitive, and emotional-social:

1. 1. Photo-Visual Literacy – Understanding workflows, instructions, and messages when presented in graphical formats; ranging from individual icons and symbols within a platform, to navigating an entire graphical user interface (GUI).
   2. Reproduction Literacy – Creating new meanings or interpretations from disparate information in various formats.
   3. Information Literacy – Assessing large volumes of data objectively, discounting irrelevant material while demonstrating an awareness of bias and/or falsehood.
   4. Branching Literacy – Navigating a hypermedia environment, including complex or varied knowledge domains, while remaining oriented and focused on core research and learning tasks.
   5. Socio-Emotional Literacy – Understanding the rules of effective, respectful, and sensitive engagement in an online environment, including a willingness to share knowledge while working and learning collaboratively.
   6. Real-Time Thinking Literacy – Processing or interacting with real-time or high-speed digital data (less relevant in the context of an asynchronous class).

This model provides a robust framework for designing a course geared towards cultivating digitally literate students. This article will examine these interrelated literacies in the context of an online no-prerequisite introductory digital humanities course. The five-week asynchronous class was offered in the summer of 2019 through the Informatics Department at the University of Washington and was designed to help students develop an understanding of the digital humanities through the lens of text mining historical documents. Rather than a homogeneous group of Informatics students, the demographics of the class included a mix of thirty-one undergraduates and graduates from a diverse range of departments. Twenty-one unique majors were represented, from electrical engineering to business administration. Many from this group self-identified as having little to no experience working with humanities data, text mining, or data analysis.

Our intent in writing this article is to provide perspectives on the pedagogical strategies and practical challenges of teaching and participating in the course, along with an assessment of the effectiveness of the syllabus in developing core digital literacy competencies.

Course Structure

The Canvas classroom was structured as five week-long modules with staggered release dates. Each module followed a logical but non-linear path that drew in resources from outside the LMS; students developed their branching literacies as they navigated the hypermedia environment on their own time, with content milestones defined by class submission deadlines. Varied pedagogical strategies were employed to meet the needs of a diverse group of learners, including written ‘How-To’ worksheets, tutorial videos created by the instructor and uploaded to YouTube, and tutorial videos created by product developers and uploaded either to YouTube or to the product’s Help Center. Zoom office hours were also offered for those who preferred the option of in-person tutorials, but few students availed themselves of this opportunity.

The class was guided through the process of navigating a digital platform for text analysis–the Gale Digital Scholar Lab–to build corpora or ‘content sets’ of relevant primary source material and to curate and analyze the collected data, before presenting research results in an accessible and engaging format for a hypothetical public audience. In doing so, and to complete the course and learning requirements, students developed and used a combination of five core digital literacy skills detailed above in the Introduction to Digital Humanities Curriculum and Literacies Infographic.

Mapping a Digital Literacy Framework to Course Tasks

Students were expected to identify a research question that they wished to answer using qualitative or quantitative text mining methodologies. The instructor provided a range of sample topics for the group to choose from since the range of archival material available to mine was extensive and potentially overwhelming within the short time-frame of the class. A few students had particular research interests they wished to explore, and were able to do so once they had mapped out the scope of their project appropriately.

In order to gather the data necessary to answer their question, work in class combined ‘traditional’ linear search and retrieve exercises, using primary source databases to find relevant research material, set against a backdrop of a more iterative and cyclical flow of retrieving documents, evaluating and cleaning the OCR text, testing the cleanup using analysis tools, and returning to repeat the process if necessary. Students were required to evaluate and order considerable amounts of information from disparate sources in a short period of time. Repeating the build-clean-analyze-visualize process on multiple occasions, and across different platforms, helped reinforce this mental model for students, moving from an abstract concept into a defined workflow. This construction of knowledge from independent and unordered sources in a non-linear manner, coupled with multiple-domain thinking, exemplifies branching literacy.

The process of building a research dataset consisting of digitized primary source material to explore a chosen research question served to underscore the importance of sourcing relevant and usable data, and of developing information literacy skills. Many students drew on material from newspaper archives and needed to evaluate its credibility and bias. The constraints of the limited time frame required students to constantly define and redefine the scope of their project, including the size of their content sets, to ensure that their research focus was narrow, and data manageable and relevant. Similarly, in order to make informed choices about the documents they were working with, students learned about the process of creating digital archives and OCR texts, developing an awareness of some of the challenges of generating ‘clean’ OCR text as well as the factors that influence the OCR confidence level. The class quickly learned that no matter how they configured their text cleaning tools, an error-free OCR dataset was impossible to achieve. Accepting when a content set was cleaned to the degree that was “good enough” for the purposes of their projects was yet another lesson learned through experience working with the tools and guided discussion, and again involved developing branching literacy skills by iterating through the process of examining OCR text, comparing with original sources, developing cleaning configurations, testing analyses, then returning to tweak configurations as appropriate.

While much of this workflow took place within the Digital Scholar Lab, students were expected to export OCR text and mirror the process using external tools to compare and evaluate the experience and their analysis results. Along with Canvas and the Gale Digital Scholar Lab, students navigated a number of digital platforms with different GUIs, including Lexos, Voyant, and StoryMapJS. Options were provided for students looking for additional challenges, including OpenRefine and Regular Expressions. Navigating these digital work environments required the development of skills related to photo-visual literacy, using intuitive-associative thinking to understand and interact with the visual messaging and workflow in each GUI. Later tasks would also involve this cognitive literacy, specifically the interpretation of visualized data and analyses created using these digital environments.

Assessing whether their research question had been satisfactorily addressed using a particular analysis method or tool configuration was an important factor in teaching students about the potential and the limitations of given digital tools, as well as the optimal parameters for creating significant results. For example, in investigating prevalent themes in their datasets, students used a topic modeling tool in the Digital Scholar Lab, experimenting with the number of topics and number of words per topic that the algorithm returned. When choosing a result of 10 topics, students often found that results were ‘as expected’ or ‘unsurprising,’ whereas choosing a higher number of topics often returned less obvious themes in the datasets. This type of iterative digital work encouraged the class to be thoughtful in their choice of configurations, and on occasion, led to interesting research results.

More often, however, it led to analysis outputs that were either unclear or insignificant, which galvanized students into returning to their primary source data to adjust the size of their content set or to revise tool configurations and re-run their analyses. Additionally, the experience of seeing insignificant or inconclusive results underscored the fact that digital humanities analysis is primarily driven by the scholar using the tools, rather than the tools themselves. Interpreting and refining results engaged students in various literacies, including photo-visual, reproduction, and branching, as well as information literacy since analysis results occasionally brought to light outlying data that needed to be re-assessed for relevance.

Digital humanities projects are typically collaborative undertakings, but the disaggregated online classroom presented challenges to building an engaged learning community. A number of strategies were employed to create a “Community of Inquiry” whereby students were encouraged to regularly engage and interact among themselves and with the TA and the instructor. While most students are well-versed in the socio-emotional literacies of working in an online classroom and were willing to share, evaluate, and collaborate with information and data to create knowledge, nonetheless, class online etiquette was laid out during the first few days of class as a matter of best practice. The discussion board in Canvas became the main gathering point for class brainstorming and communication. Specific expectations for these discussions included student responsibility for asking thoughtful questions of each other, offering encouragement, sharing experiences, and generally driving the learning forward. The class was also expected to share user feedback about the platforms they engaged with, an exercise that was intended to prompt critical thinking about the digital environments they were working in. This process involved drawing on their own experiences and expectations for interacting with websites and digital tools as a basis for evaluation. Furthermore, concepts from foundational digital humanities texts and projects were brought into the conversation to provide broader perspectives intended to shape how the class viewed their ongoing projects. For example, Ben Schmidt’s blog post “Machine Learning at Sea” (2012) provided an in-depth case study of how clustering can be applied to text-mined datasets. After reading and discussing this work, students found themselves better understanding the sort of questions these tools can answer and began conducting more comprehensive exploratory analyses with similar techniques. This additional work was not conducted to fulfill specific requirements of the course, but rather as required to fully realize the vision they had for their final projects, within the context of the ongoing development of each of the five digital literacy skills.

Final Projects

The final project was an exercise in photo-visual communication and reproduction literacy, in the sense that students were engaged in ‘the creative recycling of existing materials’. While this process occurred on a smaller scale throughout the course, for their final projects, and in lieu of a traditional written exam, students created an annotated interactive slideshow (a StoryMap) to present the outcomes of their research. While the structure of the exercise was defined by a detailed project rubric, the choice of images, narrative, and visualizations was the responsibility of the student. That the curriculum had been effective in providing a scaffolded experience for students to develop the five core digital literacy skills was demonstrated by the depth, breadth, and content of final projects which included:

• a title slide with a high-level overview of the topic and appropriate imagery,
• a compelling and logical narrative highlighting the main research points,
• a summary of the archival resources used to build content sets,
• a description of the process of data curation and cleaning, which students had documented by keeping detailed work logs,
• a summary of each of the text mining analyses students carried out with an assessment of which was most useful for answering their research question,
• at least one visualization and analysis result with a discussion about tool configuration choices and consideration about how meaningful the research results were,
• a learning summary to encourage students to reflect on the skills they had learned and used in the class,
• a bibliography.

The range and quality of the work speaks to students’ high level of engagement with both the historical primary source material and the digital tools for text analysis. The projects provide a benchmark for assessing the effectiveness of the curriculum in developing digital literacy skills, while at the same time delivering a worthwhile educational experience. Over thirty final student projects are available at Newbook Digital Texts. The course will be offered again in Summer 2020 with a few workflow tweaks to further expand opportunities to develop the core technical-procedural, cognitive, and emotional-social digital literacies described in this article.

Content sets created by Jared Nistler.

Photo Credits: Curriculum infographic, visualizations, analyses, and StoryMap cover created by Jared Nistler.

Bibliography

Aharon Aviramand, Yoram Eshet-Alkalai. “Towards a Theory of Digital Literacy: Three Scenarios for Next Steps.” European Journal of Open, Distance and E-Learning (2006).

Developing digital literacies https://www.jisc.ac.uk/full-guide/developing-digital-literacies (2014), accessed 02.03.2020

Yoram Eshet-Alkalai. “Digital Literacy: A Conceptual Framework for Survival Skills in the Digital Era”. Journal of Educational Multimedia and Hypermedia (2004) 13(1), 93-106.

Yoram Eshet. “Thinking in the Digital  Era: A Revised Model for Digital Literacy.” Issues in Informing Science and Information Technology (2012) 9, 267-276.

CoI Framework. https://coi.athabascau.ca/coi-model/

Ben Schmidt, ‘Machine Learning at Sea,’ Sapping Attention, 2012 https://sappingattention.blogspot.com/2012/11/machine-learning-on-high-seas.html