Ontmoet enkele van de 100.000+ gebruikers die buitengewone projecten met Transkribus hebben gerealiseerd

Succesverhalen

<p>The <a href="https://www.sg.ch/kultur/staatsarchiv.html">State Archives of the Canton of St.Gallen</a> hold a wealth of historical material documenting the cultural, political, and administrative history of this Swiss region. Among its significant collections are the council meeting minutes, which record decisions made by the St.Gallen government since its founding in 1803. These documents, a mix of handwritten and typewritten pages, are an invaluable resource for researchers, historians, and local citizens alike.</p>

<p>For many years, accessing these materials was limited to those who could visit the archive in person. This often meant navigating through volumes of fragile, handwritten records, which required both time and expertise to interpret. Recognising the need to make these documents more accessible and easier to use, the State Archives launched a digitisation project using <a href="https://www.transkribus.org/sites">Transkribus Sites</a>, a tool designed for transcribing and publishing historical texts online.</p>

<p>We spoke to Remo Cadalbert, the digitisation project manager at the archives, to learn more about the process of bringing this important historical resource online.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/stgallen/01-Archivregal.jpg" alt="The State Archives of the Canton of St.Gallen is home to thousands of pages of archival material." style="max-width: 100%; height: auto;">

<p><em>The State Archives of the Canton of St.Gallen is home to thousands of pages of archival material. Image: Staatsarchiv St.Gallen</em></p>

<h2>Selecting the documents for publishing</h2>

<p>The State Archives house many collections, but the council meeting minutes were a clear choice for this project. "The council meeting minutes of the Canton of St.Gallen are among the most important holdings of the State Archives," Remo explained. "They document the activities of the St.Gallen government since the canton was founded in 1803 and serve as a central source for cantonal history."</p>

<p>The minutes provide an official record of decisions and discussions during more than a century of government activity. From legislative actions to administrative changes, the collection offers a consistent resource for understanding local governance and its evolution over time. Making this information widely available was a key goal of the project, as the demand for access to these records extends beyond academics to genealogists and residents interested in their heritage.</p>

<p>However, digitising the minutes posed a significant challenge due to their size and format. The collection consists of 417 books and nearly 200,000 pages, with a mix of handwritten and typewritten texts. This required an automatic text recognition tool capable of handling not only large volumes but also the complexity of historical handwriting.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/stgallen/02-Bild1.jpg" alt="The government of St.Gallen has been meeting regularly since the canton was founded in 1803." style="max-width: 100%; height: auto;">

<p><em>The government of St.Gallen has been meeting regularly since the canton was founded in 1803. Image: Staatsarchiv St.Gallen</em></p>

<h2>Choosing the right tools</h2>

<p>The archive chose <a href="https://www.transkribus.org/">Transkribus</a>, a platform specialising in handwriting recognition and enrichment. While standard optical character recognition (OCR) systems are designed for printed text, Transkribus can process handwritten or mixed documents, making it better suited for historical materials.</p>

<p>"Transkribus is by far the most efficient way of transcribing the manuscripts and making them available together with the originals," Remo said. The platform relies on artificial intelligence (AI) and neural networks to <a href="https://www.transkribus.org/ai-text-recognition">recognise handwriting styles</a>. By <a href="https://www.transkribus.org/ai-training">training the system</a> on samples from the council minutes, the team could improve its accuracy over time.</p>

<p>Despite its strengths, Transkribus was not without limitations. Many of the pages had been digitised from older microfilm scans, which were not always clear or detailed. Additionally, the handwritten sections varied in style and legibility, which occasionally caused errors in the handwritten text recognition. These inaccuracies required manual correction, adding an extra step to the process.</p>

<p>For Remo, however, the benefits of using a text recognition platform still outweighs the disadvantages. "For the considerable saving in resources and time, one also accepts a certain character error rate (CER)," he noted.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/stgallen/04-Bild3.jpg" alt="Transkribus Sites allows users to view the original document and the digital transcription side by side." style="max-width: 100%; height: auto;">

<p><em>Transkribus Sites allows users to view the original document and the digital transcription side by side. Image: Staatsarchiv St.Gallen</em></p>

<h2>Making the collection accessible with Transkribus Sites</h2>

<p>After the transcription process was completed, the digital files were uploaded to Transkribus Sites, a tool for publishing historical materials online. This allowed the State Archives to create an organised and searchable collection of the council minutes, making them accessible to a much wider audience and eliminating many of the barriers associated with accessing historical documents, such as the need to travel or the ability to read old handwriting.</p>

<p>"The council minutes are now available around the clock, regardless of location," Remo said. This practical change has already benefited researchers and casual users alike. People who once needed to visit the archive in person can now work with the documents remotely, and even those unfamiliar with old scripts can make use of the transcriptions.</p>

<p>You can view the St.Gallen Transkribus Site <a href="https://app.transkribus.org/sites/St.Gallen">here</a>.</p>

<h2>A model for future projects</h2>

<p>The success of the St.Gallen project has positioned Transkribus Sites as a model for other archives looking to digitise their collections. "Transkribus Sites is a tried and tested solution for online publishing and editing," Remo said. "Archives that have large manuscript collections and want to make them accessible to a wide audience should use [it]."</p>

<p>In addition to enabling online access, the platform’s collaborative features open up further possibilities for public engagement. For example, archives could invite volunteers to participate in reviewing and correcting transcriptions, combining <a href="https://www.transkribus.org/en/blog/what-is-citizen-science">citizen science</a> with <a href="https://www.transkribus.org/en/blog/what-is-digital-humanities">digital humanities</a>. "Participants who are interested in old handwriting can deepen their knowledge [...] while helping to improve the accuracy of the transcriptions," Remo explained. This approach not only increases the accuracy of the digital collection but also fosters a stronger connection between the public and historical documents, encouraging people to engage with history in new and practical ways.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/stgallen/07-Kirchenb%C3%BCcher%20online%20Staatsarchiv.jpg" alt="Remo sees potential in using Transkribus for citizen science initiatives at the archive." style="max-width: 100%; height: auto;">

<p><em>Remo sees potential in using Transkribus for citizen science initiatives at the archive. Image: Staatsarchiv St.Gallen</em></p>

<h2>Looking ahead</h2>

<p>The St.Gallen project demonstrates how handwriting recognition technology and online publishing tools are changing the way archives operate. In addition to preserving fragile documents, these tools create new ways for both researchers and the general public to interact with history.</p>

<p>“The progress Transkribus has made in terms of the quality of text recognition using AI in recent years is remarkable—and the system is constantly being improved. I can only recommend it,” Remo said.</p>

<p>With the council minutes <a href="https://app.transkribus.org/sites/St.Gallen">now accessible online</a>, the State Archives of St.Gallen have set a high standard for digital access to historical materials. However, with vast amounts of archival material still to be digitised, the work is far from over.</p>

<p>"We hope to be able to use Transkribus in the future to digitise more handwritten documents," Remo explained. "There is certainly no shortage of archival material!"</p>

<p><em>Thank you, Remo, for sharing your insights into this project.</em></p>

<h2>Interested in using Transkribus at your archive?</h2>

<p>Transkribus has been used in archives around the world to realise projects both big and small. If you are interested in using Transkribus at your archive, then <a href="https://www.transkribus.org/contact/consultation">set up a consultation call</a> with our advisors and find out how Transkribus can benefit your archival work.</p>

<p>The <a href="https://www.nationaalarchief.nl/en">National Archives of the Netherlands</a> used Transkribus to transcribe over 3 million pages of documents. Find out more in <a href="https://www.transkribus.org/success-story/national-archives-of-the-netherlands">this Success Story</a> with digitisation project manager, Liesbeth Keijser.</p>

<a href="https://www.transkribus.org/success-story/national-archives-of-the-netherlands"><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/NAN.jpg" style="max-width: 50%; height: auto;"></a>

Unlocking St.Gallen's archive: Publishing council meeting minutes with Transkribus Sites

<p>Like most national archives, the <a href="https://www.arhivi.gov.lv/en/department/latvian-state-historical-archive">National Archives of Latvia</a> (NAL) is a mine of historical documents detailing the intricate past of the Baltic nation. One of the archive’s most important collections is the <a href="https://www.arhivi.gov.lv/en/riga-city-council-minutes-1603-1980-included-latvian-national-register-unesco-program-memory-world"><em>Minutes of the Riga City Council</em></a> (1603–1890), which have recently been recognised by UNESCO as part of its <a href="https://www.unesco.org/en/memory-world">Memory of the World Programme</a>. These German-language records cover nearly 300 years of decisions made by the Riga City Council, during periods when Riga was ruled by Poland, Sweden, and later the Russian Empire, making them a valuable source of information about the city and its people.</p><p>Until now, this vast collection has only been available in paper form, meaning anyone wanting to view it had to travel to Riga and search through the physical archive. But thanks to the efforts of digitisation expert and archive employee Mairita Lukianska, these important historical records are, for the first time, being made more accessible.</p><p>We talked to Mairita to find out more about the project, her Master’s thesis, and about how a <a href="https://www.transkribus.org/scholarship">Transkribus scholarship</a> has benefited her work.</p><br><img src="https://www.archiv.org.lv/rigaspilsetasarhivs/wp-content/uploads/2022/10/DSC_8630-scaled.jpg" style="width:100%; height:auto;"><em>Just some of the records in the Minutes of the Riga City Council collection. © Latvian State Historical Archive</em><br><h3><strong>A large and important collection</strong></h3><p>As a professional archivist and a Master’s student in <a href="https://www.transkribus.org/blog/what-is-digital-humanities">Digital Humanities</a>, Mairita has a strong interest in using technology to preserve historical documents. “I have always been interested in how we can use the latest innovations in information technology to preserve cultural heritage, especially documentary heritage,” she said. Her project to transcribe the <em>Minutes of the Riga City Council</em> is part of her Master’s research on <a href="https://www.transkribus.org/blog/what-is-handwriting-recognition-and-how-does-it-work">handwritten text recognition (HTR)</a>, which aims to make valuable documents such as these easier to digitise, preserve, and access.</p><p>And the <em>Minutes of the Riga City Council</em> could not be a better material for this purpose. Consisting of 386 books and more than 263,000 pages, the documents provide a detailed record of the city’s political, economic, and social decisions over nearly three centuries. The collection is such an important part of Latvian history that it was recently included in the UNESCO Memory of the World Programme.</p><p>“On 22 March 2022, the Director of the National Archives of Latvia and the Secretary-General of the Latvian National Commission for UNESCO signed a cooperation agreement on the preservation, accessibility, and promotion of the <em>Minutes of the Riga City Council</em>,” Mairita explained. It was this agreement that inspired Mairita to choose the collection for her Master’s thesis and explore how it would be possible to digitally preserve it and make it accessible to everyone.</p><br><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/2posts/8_image.jpg" style="width:100%; height:auto;"><em>The volume of the Minutes of the Riga City Council for the years 1643 to 1645. © Latvian State Historical Archive</em><br><h3><strong>Tackling the challenges of transcription</strong></h3><p>But transcribing historical documents, especially handwritten ones, comes with many challenges. The <em>Minutes of the Riga City Council</em> were written in <a href="https://www.transkribus.org/blog/3-ai-models-for-transcribing-german-text-in-fraktur-kurrent-and-sutterlin">German Kurrent script</a>, a handwriting style that was commonly used during that period. Since the council’s scribes often wrote quickly, their handwriting can sometimes be difficult to read. “One of the biggest challenges is the handwriting. Some handwriting can only be deciphered after careful comparison work,” Mairita explained.</p><p>In addition to the handwriting, the documents contain many abbreviations that were typical of administrative texts of the time. These abbreviations are often derived from medieval Latin and don’t have direct modern equivalents, making the transcription even more complex.</p><p>Because of these challenges, it would take many years to transcribe the collection by hand. However, Mairita knew that this process could be speeded up dramatically with the help of AI tools. As a first step towards making the <em>Minutes of the Riga City Council</em> fully accessible, she chose a sample of 3,170 pages — 1.2% of the entire collection — and set about finding the best workflow for automatically transcribing it.</p><br><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/2posts/11_image.jpg" style="width:100%; height:auto;"><em>The Kurrent script can be tricky to decipher, as shown in this volume from 1710. © Latvian State Historical Archive</em><br><h3><strong>Choosing Transkribus</strong></h3><p>Ideally, that entire workflow could be carried out with just one tool, saving Mairita valuable time and resources. “Transkribus offers a set of tools for document analysis, including <a href="https://help.transkribus.org/layout-recognition">layout and line recognition</a>, <a href="https://help.transkribus.org/tagging">tagging</a>, and <a href="https://help.transkribus.org/downloading">export in various formats</a>. [It] provides an all-in-one solution in one place, eliminating the need to find tools for each task in the workflow.”</p><p>For Mairita, one of the most useful aspects of Transkribus is that it allows users to <a href="https://help.transkribus.org/training-text-models">create their own models</a> based on the handwriting in their specific documents. Given the variety of unique handwriting styles in the Riga City Council minutes, this would be essential for the project. “Training a model was not a difficult challenge on the Transkribus platform. If there were any problems to troubleshoot, I referred to the <a href="https://help.transkribus.org/">Transkribus documentation</a>, which is very easy to navigate and well-structured.”</p><p>Finally, the fact that Transkribus offers <a href="https://www.transkribus.org/scholarship">scholarships to early-career researchers</a> was also an important factor in Mairita’s decision. “The scholarship provided financial support for the research. The application form [...] was easy to fill out. I just needed to decide how many pages I wanted to recognise.”</p><br><h3><strong>Training a model with Transkribus</strong></h3><p>Mairita trained her first <a href="https://www.youtube.com/watch?v=DZtjK3DMa3A">text recognition model</a> with 109 pages of <a href="https://www.transkribus.org/blog/what-is-ground-truth">Ground Truth</a> data, selecting German Kurrent 17th-18th century as a base model. “I [then] applied the model to recognise an additional 200 pages from four different books, [and] submitted the text to the paleographers for editing.”</p><p>As Mairita’s professional background is in project management rather than paleography, she worked closely with experts to ensure that these Ground Truth transcriptions were accurate. The handwritten texts were reviewed by Enija Rubina, a senior researcher at the NAL, and Dr. Manfred von Boetticher, an expert in German paleography. “Editing 200 pages took a year,” Mairita says. “It was a time-consuming process but necessary to continue the research.”</p><p>Once the corrections were made by the paleographers, Mairita trained a new version of the model, using this expanded collection of Ground Truth. By the end of August 2024, she had used this model to successfully transcribe 3,170 pages — or four books of The Minutes of the Riga City Council.</p><br><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/2posts/6_image.png" style="width:100%; height:auto;"><em>A transcribed document with the digital text (screenshot taken from a previous version of Transkribus). © Mairita Lukianska</em><br><h3><strong>Discovering new details about Riga’s past</strong></h3><p>During this process, Mairita has come across all sorts of interesting records, covering everything from the Great Northern War and Napoleon’s 1812 campaign, to small details about everyday life. “The books document significant historical events,” Mairita says, “but they also show how closely Riga was connected to European trade and communication networks.”</p><p>For example, there is the entry about the Italian who caused a stir in 1644 by bringing his Siamese twin to Riga. Or the Hungarian merchants who were investigated in 1773 for selling an unknown cattle medicine. Then there were the strange regulations that were passed by the Riga City Council during this period, focusing on the illegal import of rhubarb or ruling that hot meals were not allowed to be served at weddings within the city’s boundaries.</p><p>While these entries don’t contribute to our understanding of major world events, they do help us to understand how the citizens of Riga lived during this period, what was important to them, and how local and global events shaped their daily lives. Accessing these valuable insights has become much easier thanks to Mairita’s project, particularly for people who lack the paleography skills to read the original documents.</p><br><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/2posts/10_image.jpg" style="width:100%; height:auto;"><em>Typical council records from the year 1643. © Latvian State Historical Archive</em><br><h3><strong>Looking to the future</strong></h3><p>Mairita hopes at some point to make her model public for everyone to use, but wishes to refine it more before doing so. “Currently, the CER for the retrained model on the validation set is 7.50%. I would only make it public if I could improve the model by a few percent more.”</p><p>However, as a result of the project, the archive has organised several outreach events, designed to encourage public involvement in the documents. “The NAL have started offering seminars on German handwriting paleography to teach reading skills for 17th-19th century German handwriting. I also led a workshop on how to use [Transkribus] for handwritten documents.”</p><p>In addition to these events, Mairita and her colleagues plan to use the digitised documents to test different forms of volunteer involvement in the correction and indexing of the transcribed texts, perhaps paving the way for a future <a href="https://www.transkribus.org/en/blog/what-is-citizen-science">citizen science</a> project.</p><br><h3><strong>Advice for future researchers</strong></h3><p>For students or researchers interested in similar projects, Mairita’s advice is to take advantage of the opportunities available, such as the Transkribus scholarship, and to be prepared for the challenges that come with working on large historical collections. “Working with such a platform provides [an] exceptional experience that can be valuable for your [skillset] and career growth.,” she says.</p><p>As she looks ahead, Mairita plans to continue using Transkribus for smaller projects. With the help of new tools and techniques, she hopes to keep uncovering more of Riga’s rich history, ensuring that these important documents remain available for future generations to study, explore, and enjoy.</p><p><em>Thank you for taking the time to talk to us, Mairita!</em></p>

Rhubarb, Siamese twins, and an unknown cattle medicine: Unlocking the history of Riga with Transkribus

<p>Transcribing historical documents is vital yet often painstaking work, particularly if you’re dealing with English secretary hand. This script, widely used in England from the 16th to early 17th centuries, appears in countless documents from the era. However, English secretary hand is notoriously difficult to read, with its intricate and elaborate style leaving many historians and researchers struggling to decipher it. As a result, millions of pages of documents written in this script remain inaccessible, locked away in archives with their secrets still untold.</p><p>Fortunately, recent advancements in AI and machine learning offer promising solutions for unlocking such documents. Emily Kadens, a lawyer and historian at <a href="https://www.northwestern.edu/">Northwestern University</a>, has embarked on a mission to harness this technology, decode the complicated script, and make these historical documents accessible to all. We sat down with Emily to learn more about her journey and the development of the “Egerton” model for English secretary hand.</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2001.jpg" style="width:100%; height:auto;" /><em>English secretary hand is one of the trickiest hands in the English language. © National Archives, London</em><h3>The challenge of English secretary hand</h3><p>Emily’s expertise lies at the intersection of law and history, with a focus on early modern legal documents, many of which were written in English secretary hand. Among them are the records of English equity courts, which, unlike the regular royal common law courts, used a fully written, English-language procedure.</p><p>“These records are incredibly rich sources of information about all aspects of English life, including commerce, agriculture, property, family and marriage, [and] crime,” Emily explained. Over the years, she has manually transcribed over a thousand pages of such documents—a laborious and time-consuming task given the script’s complexity.</p><p>Determined to make these legal documents more accessible, Emily discovered <a href="https://www.transkribus.org/">Transkribus</a>, a platform for transcribing historical documents using AI. “I was thinking about making the 1000+ pages of equity court case files that I had manually transcribed over the years [...] available online. As I looked for a way to do that, I learned about Transkribus and realised that I could use my manual transcriptions as <a href="https://www.transkribus.org/blog/what-is-ground-truth">Ground Truth</a> to [train] a model.”</p><p>However, due to the known complexity of English secretary hand, not everyone was so convinced by Emily’s plan. “A couple of archivists from major libraries holding large quantities of secretary hand documents assured me it was not possible to create a secretary hand model,” Emily said. “But Transkribus created a workable model very quickly.”</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2002.png" style="width:100%; height:auto;" /><em>The Egerton model now contains thousands of transcribed documents. © National Archives, London via Transkribus</em><h3>Building the Egerton model</h3><p>The starting point for that model was a selection of Emily’s manually transcribed documents. Together with a dedicated team of 3 to 4 researchers, Emily embarked on the meticulous task of correcting these transcriptions. After accumulating around 50,000 words, they <a href="https://help.transkribus.org/training-text-recognition-models">trained their first model</a>, aptly named "Egerton" after Thomas Egerton, the Lord Chancellor who presided over many of the equity court cases reported in the documents.</p><p>“We then started picking images [...] of documents, mostly from the equity courts, that were good enough [...] for Transkribus to read, and running them through the model. [We] put every new transcription through three checks before marking it as Ground Truth.”</p><p>As the team gathers more Ground Truth data — much of it from the <a href="https://www.nationalarchives.gov.uk/">National Archives in London</a> — they continually retrain the model. The current version of the Egerton model boasts an impressive 750,000 words and <a href="https://readcoop.eu/how-to-improve-the-cer-of-your-model/">a character error rate (CER)</a> of just 3%. “This fall, we plan to begin a sustained project transcribing one term of depositions from the English Court of Chancery from 1597. That will add about 550 more pages to the model.”</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2003.jpg" style="width:100%; height:auto;" /><em>Emily’s team had to work out a system for transcribing the many different abbreviations and symbols. © National Archives, London</em><h3>The art of standardisation</h3><p>But training a model for English secretary hand wasn’t without its difficulties. One of the biggest hurdles was the lack of standardisation. Unlike other official scripts, secretary hand varied significantly from scribe to scribe. “Many scribes wrote very idiosyncratic forms of secretary hand that to a lay eye seem nothing alike. The extent to which [Egerton] can recognise really different-looking hands is completely amazing.”</p><p>“It has also been fascinating to develop conventions for dealing with the various, often non-standard abbreviations we have encountered,” Emily added. “Because we are prioritising readability in our transcriptions, we have trained Egerton to silently expand almost all abbreviations, but some abbreviations represent more than one word, so we had to figure out how to deal with that.”</p><p>“We now have a 17-page conventions document to explain to transcribers how to handle various situations with abbreviations, capitals, punctuation, the Unicodes and Transkribus tags we use, and the occasional abbreviated Latin words.”</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2004.png" style="width:100%; height:auto;" /><em>Ensuring the layout is recognised correctly helps achieve more accurate transcriptions. © National Archives, London via Transkribus</em><h3>Built on words, not characters</h3><p>As Transkribus deciphers texts word by word, instead of character by character like conventional OCR systems, Emily’s team also had to think more broadly than they used to about Transkribus’ capabilities. “It [...] took us a while to fully understand that <a href="https://medium.com/transkribus/ocr-vs-htr-or-what-is-ai-actually-5c33f21f3145">Transkribus is not doing OCR</a> and what that means for what we can ask the technology to do.”</p><p>“For instance, [in secretary hand ] I/J and U/V were written in the same way. At first, we thought we had to be consistent in our transcriptions and pick either I or J and either U or V.” But because Transkribus learns what words look like, rather than individual characters, Emily’s team realised that the Egerton model could be trained to recognise when a word contained an I instead of a J and a U instead of a V. “So now we use modern spelling for words with capital I, J, U, and V even though the I/J and U/V letter forms are the same.”</p><h3>A work in progress</h3><p>Currently, the Egerton model remains a private tool as Emily and her team continue to refine it. Their focus is now on incorporating data from deposition hands, which are some of the most difficult in terms of readability. “We are trying to put large enough samples of the various bad deposition hands into our Ground Truth to achieve better than a 7% CER for even the most difficult hands. So far, we are making surprisingly good progress on this.”</p><p>Ultimately, though, the team wants to make the model public and is aiming to do this once they hit a million words of training data. “[This] seemed like a huge amount two years ago, but [it] is now a milestone we should hit in early 2025.”</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2005.jpg" style="width:100%; height:auto;" /><em>Emily’s team are continually adding new documents to the model. © National Archives, London</em><h3>Lessons learnt from model training</h3><p>Reflecting on her journey, Emily notes that one of the most valuable lessons she learned was the importance of thinking through transcription conventions from the start. “We have had to rethink earlier decisions and [...] spend a lot of time going back and changing Ground Truth to reflect new choices,” Emily explained. “Things are going to come up as you build, but to the extent that you have thought a lot of it through before you start, you will save time fixing things later.”</p><p>Another key lesson is that training models is not a linear process. “Your character error rate and the quality of the transcriptions will not always seem to be improving,” Emily said. “Egerton took a giant step forward at 300,000 words, and saw another massive improvement at 600,000 words. But not every new model run brings such noticeable change. So don’t get frustrated.”</p><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Image%2006.jpg" style="width:auto; height:auto; display: block; margin: 0 auto;" /><em>The historical documents have also thrown up some surprises outside of the legal field. © Emily Kadens via X</em><h3>From scepticism to success</h3><p>The Egerton model stands as a testament to the potential of AI in historical research, challenging long-held assumptions about complex scripts and creating new possibilities for historians working with documents from the early modern period.</p><p>Despite the Egerton model still being in development, Emily is willing to share her work with others in the field. “I am happy to make the model available privately now to anyone working on English secretary hand material,” she said. If you are interested in using Egerton, Emily invites you to <a href="https://www.law.northwestern.edu/faculty/profiles/EmilyKadens/">reach out to her by email</a> or <a href="https://x.com/ekinthearchives">follow her updates on X</a> (formerly Twitter) as the model continues to evolve.</p><p><em>Thanks Emily for sharing your experiences with us!</em></p><h3>Interested in using Transkribus for your research?</h3><p>Transkribus has been used for research projects around the world, helping to create searchable, digital versions of paper resources. We have several subscription plans aimed at researchers, from our solo 'Scholar' plan to the 'Team' plan, which is ideal for small research groups. You can view all the available subscriptions on our <a href="https://www.transkribus.org/plans">Plans and pricing</a> page.</p><p>When researcher Álvaro Cuéllar set out to transcribe a series of theatrical works from the Spanish Golden Age, he hoped he would find something interesting. But he did not expect to discover a completely new work by one of Spain’s most famous authors, Félix Lope de Vega y Carpio. Find out more about his research project in <a href="https://www.transkribus.org/success-story/discovering-a-lost-play-by-lope-de-vega-alvaro-cuellar">this Success Story.<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/alvaro.jpg" style="width:70%; height:auto;" /></a></p>

Reading English Secretary Hand with AI: the Egerton Model

<p>First, there was the <a href="https://www.google.com/url?q=https://app.transkribus.org/training/text/50870?previous%3Dgallery%26activeTab%3DAll%26searchFilter%3D%26centuries%3D0%26centuries%3D21&amp;sa=D&amp;source=editors&amp;ust=1724254022860494&amp;usg=AOvVaw2yFVjvQIEqS-GjlIfBO9nA">German Giant</a>. Then there was the <a href="https://www.google.com/url?q=https://app.transkribus.org/training/text/55158?previous%3Dgallery%26activeTab%3DAll%26searchFilter%3D%26centuries%3D0%26centuries%3D21%26language%3D%5Bobject%2BObject%5D&amp;sa=D&amp;source=editors&amp;ust=1724254022860799&amp;usg=AOvVaw3vwuYR4_aHOCnRvg8kbBYd">Swedish Lion</a>. And now it’s time for the Norwegian language to get the Transkribus treatment with the new <a href="https://app.transkribus.org/training/text/55080?previous=gallery&activeTab=All&searchFilter=norhand&centuries=0&centuries=21">NorHand model.</a></p><p>This AI text recognition model, developed by the <a href="https://www.nb.no/en/">National Library of Norway</a>, has been trained on over 400 different hands, including the writing of famous authors such as Sigrid Unset. The library has also decided to make the NorHand model public, meaning that everyone can benefit from the high-quality text recognition it provides.</p><p>We spoke to Yngvil Beyer, head of the Language Bank and Digital Humanities Lab at the National Library, to find out more about the creation of the NorHand model.</p><figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/Image%2001.jpg" alt=""/><figcaption><em>Based in Oslo, the National Library of Norway is the country’s leading cultural institutions. &copy; Gorm K. Gaare</em></figcaption></figure><h3>The challenge of digitising handwritten documents</h3><p>Making cultural heritage accessible is a key goal for libraries, and the National Library of Norway (NLN) is no exception. “Digitisation is a crucial way to make cultural heritage widely accessible,” Yngvil explained. For that reason, back in 2005 the library set the goal of digitising all their materials and making them available online. They began this process by digitising printed books using conventional OCR systems, before starting on the library’s collection of newspapers. “We have now digitized almost all the books and over half of the newspapers in our collection.”</p><p>But the library’s handwritten manuscripts presented more of a problem. The OCR systems that had worked successfully for printed materials did not return good results with the handwritten documents. After trying out a few different systems, Yngvil’s team decided to go with Transkribus. “Transkribus was easy for us to adopt. It has an accessible interface, and we can use both our own and their API to ensure good data flow.”</p><figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/Image02.jpg" alt=""/><figcaption><em>The library is home to millions of books, documents, newspapers, and other pieces of cultural heritage. &copy; Nasjonalbiblioteket</em></figcaption></figure><h3>Training the NorHand model</h3><p>Transkribus’ ability to let users train their own AI models was also a key benefit for the library. “For us, it [was] crucial that one can train models without needing machine learning expertise”. This was particularly important as the majority of the training and preparation of training data would be done by library staff and students, who don’t necessarily have a technical background.</p><p>Yngvil’s team made the sensible decision to start small. “When we started using Transkribus, we trained several author-specific models. These worked well for processing material for authors we had a lot of material from, such as well-known authors like Sigrid Undset.”</p><p>After successfully training many of these author-specific models, the team then compiled all of the training data and created one large model, capable of deciphering Norwegian handwriting from the 19th and 20th centuries. “[The model] is now trained on around 400 different authors, and we see that it generalises well, meaning it can effectively recognize text from authors it has not been trained on.”</p><figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/Image03.png" alt=""/><figcaption><em>The NorHand model has a low CER of just 4.0%. &copy; Transkribus</em></figcaption></figure><h3>Tough transcription decisions</h3><p>While the team found the training process to be fairly easy, there were several not-so-easy transcription decisions that had to be made as part of the process. “We quickly realized that we had to make several choices. Some of these choices have been adjusted along the way, such as how we handle corrections, inserted words, and lines that do not follow the reading order.”</p><p>For example, in Transkribus, many people use the strikethrough markup to indicate a deleted word in the text. However, the NLN’s online library does not support the strikethrough markup. Instead, the library presents words that had been marked with a strikethrough in Transkribus simply as part of the text, as shown below:</p><figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/NLN_mismatch.png" alt=""/><figcaption><em>An extract from the NLN’s online library. &copy; Nasjonalbiblioteket</em></figcaption></figure><p><p>“We therefore had to choose between annotating deleted words in Transkribus, or just skip them, or adjust the layout so deleted words are omitted. There is not always a match between the best way [...] to create the best possible model and the best way to make the text available to users.”</p><h3>Putting the model to work</h3><p>The library’s goal was to have a model that they could use to transcribe the vast volumes of handwritten documents in their possession. This goal has definitely been achieved — approximately 25% of the library’s scanned documents have already been processed with Transkribus and published online in the NLN’s <a href="https://www.google.com/url?q=https://www.nb.no/search&amp;sa=D&amp;source=editors&amp;ust=1724254022864853&amp;usg=AOvVaw2emgUOjEJzyx85MvLdpqBn">digital library</a>. What’s more, the digitised documents are fully searchable, making it possible to quickly find all the documents relating to a certain keyword or phrase.</p><figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/NLN_love.jpg" alt=""/><figcaption><em>The NLN’s digital library makes it easy to find documents containing a keyword, such as “love”. &copy; Nasjonalbiblioteket</em></figcaption></figure><p><p>The team also took the community-minded decision to make the model available for everyone to use. “We share both the model and the training data openly, and we know that many other institutions in Norway are now using our model.” You can try out the NorHand model for yourself on the <a href="https://www.google.com/url?q=https://app.transkribus.org/training/text/55080?previous%3Dgallery%26activeTab%3DAll%26searchFilter%3D%26centuries%3D0%26centuries%3D21%26language%3D%5Bobject%2BObject%5D&amp;sa=D&amp;source=editors&amp;ust=1724254022865657&amp;usg=AOvVaw3ASGy7_gZ1oTA-ljEpNo8o">Transkribus platform</a>.</p><h3>An ever-moving beast</h3><p>The first version of the NorHand model was published in September 2023. But the work didn’t stop there. “We wanted to make the model more general, so when we had new transcriptions, we added them and trained a new version of our model.”</p><p>By continually updating the NorHand model in this way, the library can ensure that the model is constantly improving, and is able to recognise an increasingly diverse range of hands. Because of this, it is an effective model for any handwritten Norwegian text from the 19th and 20th centuries.</p><h3>Yngvil’s advice for others</h3><p>The NorHand model has been a successful project, and the team has learnt a lot during the process. We asked Yngvil what tips she would have for other libraries and institutions embarking on a similar project.</p><p>“Build competence and interest within the institution. And start with material that someone is interested in, so that they can, for instance, help proofread the annotations.”</p><p><i>Thank you, Yngvil, for sharing your experience with us!</i></p>

The NorHand Model: a new public AI model from the National Library of Norway

<p>What happens to the historical texts of a living language once the script used to write it is abandoned for another? Does a script even have a future, if people no longer learn how to read it?</p> <p>Undoubtedly, there are many ways to approach these problems, including education and transliteration — translation from one script to another rather than from one language to another. Where a language is endangered, the issue takes on a particular urgency.</p> <p>Machine-reading techniques such as OCR are a good option for making texts accessible. But what if the texts you want to recognise appear in bilingual contexts? Or the spelling pre-dates standardisation of the language? What if texts feature contributions from writers who were unaccustomed to writing or who had only just learned to write themselves?</p> <p>A collaboration between <a href="https://www.nyu.edu/">New York University</a> and the <a href="https://www.universityofgalway.ie/">University of Galway</a> is exploring these questions and more in the <em>An Gaodhal</em> project. One of its many aims is to create the first publicly available OCR model for bilingual texts printed in Irish and English. This new model is doubly innovative as it is also the first to combine multilingual and multiscript functionality in a single OCR model. We spoke to team members Oksana Dereza, Deirdre Ní Chonghaile, and Nicholas Wolf to find out more.</p> <h3>The history of Cló Gaelach</h3> <p>In Ireland, the Irish language, or Gaeilge, was once written in a script known as Cló Gaelach. You can still see it on signs, shopfronts, monuments, and headstones throughout the country. However, during a period of standardisation and modernisation in the 1960s, Cló Gaelach was replaced by Roman letters and the script was no longer taught in schools. "It is remarkable how quickly a script can fade from public memory," says Deirdre. "Still, Cló Gaelach has a strong nostalgic appeal in Ireland — and in its significant diasporic community."</p> <figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/Gaodhal/IMAGE%205.jpeg" alt=""/><figcaption><em>Office of Public Works notice using Cló Gaelach.</em></figcaption></figure> <p>The Irish diaspora is central to the <em>An Gaodhal</em> project. In the late 19th century, large numbers of people moved from Ireland to the U.S., and many of those were native speakers of Irish. By the 1890s, 40% of the world's Irish speakers lived outside Ireland. Due to the numbers of speakers in major cities such as New York, it is not really surprising that the world's first Irish-language newspaper was produced not in Ireland, but in Brooklyn, New York, by Michael J. Logan.</p> <figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/Gaodhal/IMAGE%201.jpg" alt=""/><figcaption><em>"An Bratach Ghealréaltach" — Translation of "The Star-Spangled Banner" by Fr. Eoghan Ó Gramhnaigh, An Gaodhal 13, no. 1 (Sept 1898): 5.</em></figcaption></figure> <h3>A bilingual newspaper for a diasporic community</h3> <p>Logan’s bilingual Irish-English newspaper was called <em>An Gaodhal</em>. It ran monthly from October 1881 to December 1898 and had a readership of around 3,000 people throughout the USA and Ireland. The newspaper contained a variety of texts — such as articles, ads, subscriber names, folklore, poetry, and songs — typically arranged in a unique format with the Irish text and the English translation side by side. This format not only helped Irish speakers to maintain their language skills, it also enabled many bilingual speakers who were literate only in English to learn how to read Irish for the first time.</p> <figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/Gaodhal/IMAGE%203.jpg" alt=""/><figcaption><em>"The Gaelic Alphabet" frequently reprinted in An Gaodhal.</em></figcaption></figure> <p>The only complete set of <em>An Gaodhal</em> in existence, housed at the <a href="https://www.universityofgalway.ie/library/">University of Galway Library</a>, had already been scanned and <a href="https://digital.library.universityofgalway.ie/p/ms/categories/an-gaodhal">published online</a>. However, the text had not yet been extracted from those images, making it impossible to search the collection or perform data analysis. It was this problem that the project team wanted to resolve, by extracting accurate text from all 2,298 pages and creating a digital and searchable corpus. "381 pages feature Irish mostly, 896 English mostly, and 1,019 both languages together," Deirdre said. "Also, the corpus reflects the three major dialects of Irish so, however small its 1.86 million tokens may seem, it presents a welcome diversity in the prospective training data."</p> <h3>Choosing Transkribus</h3> <p>When the project began in January 2023, there were no publicly available OCR models suitable for Cló Gaelach and the pre-standardised spelling of the Irish language, in either monolingual or multilingual contexts. The only related project in existence was a Cló Gaelach training dataset for the Tesseract software, published by <a href="https://github.com/kscanne/tesseract-gle-uncial">Scannell et al. (2020)</a>.</p> <p>As Nicholas Wolf, the principal investigator of the <em>An Gaodhal</em> project, explained: "We knew we would need to develop a bilingual model from scratch, as none were available to us. We decided to train an Irish-only model and then use that model to train a bilingual Irish-English model. Transkribus seemed well placed to serve our needs and it has worked well so far!</p> <p>Deirdre added: "In our two models, the selected unicode characters do not replicate exactly the design of Cló Gaelach (such as Gaelchló provides <a href="https://www.gaelchlo.com/">https://www.gaelchlo.com/</a>); rather, in deference to long-standing practice, Roman type characters — including those with diacritics — were chosen, thus ensuring interoperability between this dataset and others (see <a href="http://corpas.ria.ie/">http://corpas.ria.ie/</a>)."</p> <figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/Gaodhal/IMAGE%204.png" alt=""/><figcaption><em>Original script and Unicode output.</em></figcaption></figure> <h3>A simple solution to training a bilingual model</h3> <p>The project team chose a simple, yet effective, workflow for the model training process. "We first ran a preliminary OCR process using Amazon Textract across all 2,298 pages of the newspaper,” Nicholas explained. “This identified text regions containing English-language content, enabling us to isolate the Irish-language content. We then masked English-language text regions to produce page images with Irish-language text only, which would facilitate training a model for Cló Gaelach."</p> <p>After that, the team set about transcribing the Irish-language content on 60 pages of the newspaper entirely by hand. As there is no fully integrated keyboard for Cló Gaelach available, the team customised the virtual keyboard embedded in Transkribus to allow the required unicode characters to be inputted. A preliminary model, Version 1, was then trained on those 60 pages of Ground Truth; with 18,533 training tokens, it achieved a CER of <1%. Version 2, which had 164,015 training tokens, is now publicly available under the name of <a href="https://readcoop.eu/model/an-gaodhal-gaeilge-irish-monolingual-mode/">An Gaodhal Irish (Gaeilge) Monolingual Model</a>.</p> <p>The team is currently working on the bilingual model.</p> <h3>Lessons learned from layout deviations</h3> <p>Some layout elements of the newspaper — including small tables appearing between text regions, varying line directionality, and curved or acrostic texts — needed the layout analysis to be performed by hand. Given the limitations of the project’s resources and the limited number of such elements, the team decided against training workable baseline, table, or field models. Instead, all layout elements were either automatically generated using the default baseline recognition settings or manually applied and then fully reviewed by a member of the project team before any text recognition was performed.</p> <figure><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/Gaodhal/IMAGE%202.png" alt=""/><figcaption><em>Layout variables — Different table layouts and mix of orthographies in page sections and individual lines.</em></figcaption></figure> <p>Along the way, the team learnt a couple of shortcuts for effective manual layout analysis. “It’s important to ensure line polygons capture all the diacritics and punctuation,” Nicholas told us. “This reduces the need for manual correction afterwards.”</p> <p>“Within the confines of our project resources, we limited the tagging of text regions to page numbers, paragraphs, and marginalia,” he went on to explain. “Where required, tags for ‘gaps’ or for words that were ‘supplied’ or ‘unclear’ were applied. If you go on to train a model, though, don’t forget that lines featuring the 'unclear' tag are excluded from the Ground Truth.”</p> <h3>What the future holds</h3> <p>The project team have already published their Irish model, and are currently completing their bilingual Irish-English model. They are publishing all of their data publicly, including: <a href="https://ultraviolet.library.nyu.edu/records/5ya5n-mc504">full text (ALTO XML)</a>, which is corrected manually; a <a href="https://huggingface.co/ancatmara/bart-ocr-correction-ga-en">BART-based bilingual OCR post-correction model</a> and the dataset with which it was trained; and a <a href="https://aclanthology.org/2024.lt4hala-1.9/">paper published in the LT4HALA @ LREC-COLING 2024 proceedings</a>.</p> <p>The team's expert on computational linguistics, Oksana Dereza, has developed an OCR post-correction model for historical bilingual Irish-English data and is currently working on Named Entity Recognition (NER) for historical Irish. "There have been many developments in the provision of digital tools for the Irish language,” Oksana said. ”And we are delighted that the An Gaodhal project is at the forefront of that wave of innovation. We hope that other under-resourced language communities will take inspiration from this project, especially its approach to multilingualism."</p> <p>For more on this project, you can listen to this 7 minute report on <a href="https://www.rte.ie/radio/radio1/clips/22363424/">Irish radio</a>, and follow #AnGaodhal for updates.</p> <h3>Our Transkribus Tip</h3> <p>“When you get stuck, ask for help, either from the Transkribus team or from its user community, many of whom share their experiences in real-time. Such consideration and generosity meant a lot to the project team, which was working remotely. Long live this ethos of open scholarship!”</p> <h3>Funding</h3> <p>The project is funded by the Robert D. L. Gardiner Foundation, the Irish Institute of New York, Glucksman Ireland House at New York University, and the University of Galway.</p>

Training a bilingual Irish-English model in Transkribus using An Gaodhal

<p>How can a scrapbook full of notes, clippings or illustrations give us an insight into the social history of Latin American women writers? Looking at the scrapbook as more than that, the personal album of Zoila Aurora Cáceres is a personal archive and part of the cultural heritage of Peru and Latin America. And now, a team from the <a href="https://en.departamento-humanidades.pucp.edu.pe/investigacion/proyectos/laboratorio-de-humanidades-digitales">Laboratory of Digital Humanities</a> (Hlab) of the <a href="https://www.pucp.edu.pe/en/">Pontifical Catholic University of Peru</a> is transcribing and cataloguing the album of Zoila Aurora Cáceres with <a href="https://www.transkribus.org/">Transkribus</a>.&nbsp;</p> <p>That team includes Javier Vera, an expert in linguistics and mathematics; Ainaí Morales, specialized in literature; Monica Arakaki, information scientist; Mariana Reyes, historian; and Dina Cornejo, archaeologist. Together, they provided us with a deeper insight into their collaboration and the importance of this project.&nbsp;</p> <h3 class="wp-block-heading" id="h-writer-intellectual-and-feminist-activist-zoila-aurora-caceres">Writer, intellectual and feminist activist Zoila Aurora Cáceres</h3> <p>Zoila Aurora Cáceres (1872–1958), born in Lima, Peru, was an accomplished writer, journalist and social activist. Associated with the modernismo movement, she wrote essays, novels and travel literature, drawing on her own experiences and significant periods in Peruvian history, and her literary and journalistic work has been published in renowned periodicals in France, Spain and South America. But Cáceres' impact extends beyond her literary and intellectual legacy. </p> <p></p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/MO6D-neZipfoVzBXr2CLN6WrXzh44Mrvj0e16sy0OiWvd8hLvY62UW0lDSA0fcNMvyoktgpmVQKu9am3erLMWXUdJV1awA0FGLea7WaC0N96CfSClZz0bF6by4PD8QebSnPvN_i1h6dEygKJCTfnkA" alt=""/><figcaption class="wp-element-caption"><em>Page from Zoila Aurora Cáceres’ album, Laboratory of Digital Humanities.</em></figcaption></figure> <div style="height:20px" aria-hidden="true" class="wp-block-spacer"></div> <p>Born in the late 19th century, Zoila Aurora Cáceres faced the deep-rooted gender bias and inequality of her time, which denied women access to higher education and equal civil rights. Recognising Cáceres' achievement of earning a doctorate at the Université Sorbonne, it is worth noting that the university excluded women from attending lectures until 1880, when Cáceres was already 8 years old. It would be another 18 years before Peru's universities opened their doors to women in 1908.<br></p> <p>Mariana Reyes Mugaburu explains that, “Zoila Aurora Cáceres s not only a major Peruvian writer and intellectual, whose work is fundamental for rethinking the literary history and the history of ideas in the region, but also a prominent feminist activist and associationist committed to the rights and better lives of women.” Through <a href="https://www.transkribus.org/">Transkribus</a>, the <a href="https://en.departamento-humanidades.pucp.edu.pe/investigacion/proyectos/laboratorio-de-humanidades-digitales">Hlab</a> team can use the personal album as a window into Cáceres’ experiences and contributions.</p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/dQ3uUHEu6XgUvwQzafSDFPXQolkYSZpbwnsLyHZ8Ivc8SYHMt-ffNoyJfVT1LsYwt9g2BWnd_FWet5zRcLhy1a2F_hmP88nr1Iy9huNhD2LUitDCqk6ZOjjBDRiUXudjBVL-Wl8yPr9oE3pDnOzyow" alt=""/><figcaption class="wp-element-caption"><em>Page from Zoila Aurora Cáceres’ album including her photo, Laboratory of Digital Humanities.</em></figcaption></figure> <p></p> <h3 class="wp-block-heading" id="h-zoila-aurora-caceres-album">Zoila Aurora Cáceres' Album</h3> <p>From founding several associations for women's education and socio-economic support, campaigning for women's citizenship and civil rights, to supporting women workers, much is known about Caceres' social activism. “Nevertheless, there are many silences in her history that can be filled to some extent with the information contained in her album," states Ainaí Morales.&nbsp;</p> <p>"Zoila Aurora Cáceres's personal album opens up avenues for an alternative and more complex understanding of Peru's intellectual and political history, and especially of women's history at the turn of the century in Latin America," Ainaí continues. The album itself is "a kind of scrapbook that contains personal information, newspaper clippings (with literary texts, reviews of Cáceres' writings, news articles), as well as personal and professional correspondence, [...] and illustrations.” In order to find a way through the complex contents of the album, it had to be structured, transcribed and categorised. </p> <p></p> <p><img src="https://lh7-us.googleusercontent.com/iOdltgEEblWFNSmXeWR0cbjd7kYu2Zqv5e_birlIIwIr6TUjNPg71WW9YRmYMlfsXrgG2Fitc9-guFFieFeWdJcJDv3wKYP9AzJB7StMZSAOZCsdQGhVdHtzV4LJG2Rt4YKOjkw2gMlt0Q0FWb6-Sg" width="565" height="409"></p> <p><em>Page from Zoila Aurora Cáceres’ album, Laboratory of Digital Humanities.</em></p> <p></p> <h3 class="wp-block-heading" id="h-navigating-complexity-with-transkribus">Navigating complexity with Transkribus<br></h3> <p>The album of Zoila Aurora Cáceres and its contents can be understood as a personal archive that contributes to the cultural heritage of Peru and Latin America, offering insights into the literary, cultural and social history of women writers. Working on the systematisation of this personal archive is a multidisciplinary team from the Digital Humanities Laboratory (Hlab).&nbsp;</p> <p><br>As team member Javier Vera explained, “By working with datasets, we aim to array data in such a way that it becomes easily accessible and navigable, without blurring the documents’ complex materiality”. The Hlab team makes clear that “the datasets do not aim to impose new hierarchies in the album, but to showcase its multiplicity.” To transcribe and catalogue the album, Transkribus proved to be the most efficient tool, as it is capable of transcribing text while classifying it and respecting the materiality of the document.</p> <p><img width="337" height="468" src="https://lh7-us.googleusercontent.com/R2cYzGeQwIG9ZTwwP8gior8Grah9P1v0Hquk7-YpzelEZVrjZkxB-rbg_hhlQF-iLzW7qvMYuSjaFMXtAbWCPPeNMaxMpnhRA7TqQbhxUKRT5yBADuJeWxr3y24-1ZBZcwnqtXowdLPkTdCEw35EVg"></p> <p><em>Page from Zoila Aurora Cáceres’ album, Laboratory of Digital Humanities.</em></p> <p></p> <h3 class="wp-block-heading" id="h-working-with-transkribus">Working with Transkribus</h3> <p>The 304-page album holds a variety of items like letters, photos, and newspaper clippings, meaning the Hlab team had to work with printed as well as handwritten text. In the initial research phase, the team transcribed over 50 printed literary texts using the <a href="https://readcoop.eu/model/transkribus-print-multi-language-dutch-german-english-finnish-french-swedish-etc/">Transkribus Print M1</a> model for Spanish, which was found to be the best-performing model. The second phase involved handwritten documents, such as multilingual letters. Filter and <a href="https://help.transkribus.org/searching">search options</a> helped organise this large collection, and the possibility to use <a href="https://help.transkribus.org/textual-tags">textual tags</a> like location or person tags allowed the team to work on creating an interactive map of Cáceres' journey.&nbsp;</p> <p>While the printed text was quite manageable, about half of the documents, including letters and postcards in five languages, required manual transcription due to their diverse nature. Another challenge was the layout of the newspaper clippings. “In this case, dealing with layouts was really challenging! Through trial-and-error, we learned to fine-tune the layout recognition and get more accurate text regions", reports Monica Arakaki.</p> <p>Recognising the challenge of handling mixed materials, Transkribus is currently developing <a href="https://readcoop.eu/introducing-transkribus-super-models-get-access-to-the-text-titan-i/">Super Models</a> that can handle mixed materials and, depending on the Super Model, different languages. In addition, it is already possible to train layout models such as <a href="https://readcoop.eu/introducing-field-models-trainable-layout-ai-in-transkribus/">Field</a> and <a href="https://readcoop.eu/introducing-table-models-trainable-layout-ai-in-transkribus/">Table Models</a> for improved layout recognition with documents such as newspapers.</p> <p></p> <h3 class="wp-block-heading" id="h-achievements-and-next-steps">Achievements and Next Steps</h3> <p>Since the start of the project in July 2023, the Hlab team has been cataloguing each of the documents in the album as well as transcribing them using Transkribus. The team has already completed a quarter of the content, including pages and images, and is aiming to complete the entire album. Having put considerable effort into the project, the hope is to then make the content available to the public. The easiest way to do this would be to publish the album on <a href="https://www.transkribus.org/sites">Transkribus Sites</a>, as the transcription and cataloguing have already been done in the Transkribus platform. Dina Cornejo adds: "it would be very interesting to see Transkribus Sites in action."</p> <p></p> <figure class="wp-block-image is-resized"><img src="https://lh7-us.googleusercontent.com/W9QhPlh-JgA6rtliOB8Q_GpgORhG8PWRBIE5VSuzbxBo6bJNinUMSOy5FB0jGmzs2eKjzHuN71QR0RlJRgIhrS0AJ0zgT60rw1YZKt_ti0dkqR1VPN7qY_Tku__QwCXjtMQ1rEjN-A-BRW-x73TOjA" alt="" width="783" height="587"/><figcaption class="wp-element-caption"><em>Team members of the Laboratory of Digital Humanities</em>.</figcaption></figure> <p></p> <p>We wish the Hlab team all the best with their project and look forward to hopefully seeing Zoila Aurora Cáceres' personal album published on Transkribus Sites in the future!</p> <p>Thank you to Mariana and the Hlab team for talking to us and highlighting a remarkable woman in history.</p> <p></p> <h4 class="wp-block-heading">Mariana’s&nbsp; Transkribus Tips:</h4> <blockquote class="wp-block-quote"> <p></p> <cite>“We recommend relying on the page enumeration provided by Transkribus for a thorough cataloging of all documents, ensuring that not a single item is overlooked.”<br><br>“Initially, using AI for transcription may seem unfamiliar, but with practice, the process becomes much quicker and simpler. Gradually, one discovers the many advantages and possibilities that Transkribus offers.”</cite></blockquote> <div style="height:79px" aria-hidden="true" class="wp-block-spacer"></div> <p>Thumbnail: <em>&nbsp;Zoila Aurora Cáceres, Laboratory of Digital Humanities.</em></p>

Transcribing the personal album of Zoila Aurora Cáceres

<p>Simplicity, citizen engagement and AI-driven transcription were key factors that intrigued Olof Karsvall and the team at the <a href="https://riksarkivet.se/startpage">Swedish National Archives</a> when they discovered <a href="https://transkribus.org/">Transkribus</a>.</p> <p><a href="https://riksarkivet.academia.edu/OlofKarsvall">Olof Karsvall</a>, Research Manager at the Swedish National Archives, has been involved in several research projects, most recently the integration of AI, which has revolutionised the research of archival materials. In this blog post, he shares insights into how the <a href="https://www.transkribus.org/model/swedish-lion-i/">Swedish Lion Ⅰ model</a> is supporting this transformative journey.</p> <p></p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/BPIqeTZXjkabKkDEvFvgimB9dkAd8QdKDPFxVs7pLnfz8WyjuOzP838cpPlcurPfq9GbUJ17ZrmR9X0oT_6Ha_0flkZkiOEZWU5E_NZs9z2TpcLFEAG21CH8MQhEfInSzf8zhq-694XvFNJR1aFPyA" alt=""/><figcaption class="wp-element-caption"><em>Archival Material from the Swedish Lion Ⅰ Project.</em></figcaption></figure> <p></p> <h2 class="wp-block-heading" id="h-a-collaboration-of-archives-researchers-and-universities">A collaboration of archives, researchers and universities</h2> <p>The Swedish Lion Ⅰ text recognition model is a collaborative effort involving institutions such as the <a href="https://riksarkivet.se/startpage">National Archives of Sweden</a> and <a href="https://kansallisarkisto.fi/en/frontpage">Finland</a>, <a href="https://stadsarkivet.stockholm/">Stockholm City Archives</a>, <a href="https://fornskrift.se/">Jämtlands läns fornskriftsällskap</a> and researchers from <a href="https://www.su.se/cmlink/stockholm-university">Stockholm</a> and <a href="https://www.uu.se/en">Uppsala Universities</a>. “As we collectively focus on generating training data for [automatic text recognition] in Swedish, we recognised the advantages of collaboration. Consequently, we merged our training data to create a joint model”, says Olof Karsvall. The fully trained <a href="https://www.transkribus.org/model/swedish-lion-i/">Swedish Lion Ⅰ</a> model can automatically transcribe other documents with similar handwriting, making it a valuable tool for digitising and analysing historical manuscripts and archival materials.&nbsp;&nbsp;</p> <p>At the heart of this collaboration is Transkribus, a platform that allows users to create and train models for specific handwriting styles and historical periods. A key moment came in 2019 when the Stockholm National Archives joined <a href="https://readcoop.org/">READ-COOP SCE</a>, the cooperative behind Transkribus. Olof Karsvall emphasises, “Primarily, it was the ease of use and the opportunity to engage citizens and volunteers in utilising AI for machine transcription that fascinated us”. </p> <p>With external funding from the <a href="https://www.vinnova.se/en/">Swedish Innovation Agency (Vinnova)</a>, and now most recently from the <a href="https://www.raa.se/in-english/">Swedish National Heritage Board</a>, this fascination evolved into a transformative journey, resulting in innovative projects that seamlessly combine citizen science and text recognition, all made possible by Transkribus. In this way, the Swedish Lion Ⅰ model, together with Transkribus, opens up new possibilities for accessing and researching historical documents.</p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/7u4DvFAd36P-2X7Dwj86PXOQ5X5f2g5E0Hot_pr_z1Zb3wndHDr9k5QIqSuWjoZN9cYsq4t0Fa-cPHUNxr8kwLWXuVd_plVpjjk-SBf-0oc7z2k4NJgHvoMxeEhZX1AXY2Dkk2OAmSA20fpSpt2NAQ" alt=""/><figcaption class="wp-element-caption"><em>Archival Material from the Swedish Lion Ⅰ Project and transcription in Transkribus.</em></figcaption></figure> <p></p> <h2 class="wp-block-heading">Expanding research possibilities</h2> <p>When looking more closely at the history of models, it is always interesting to find out what the aim and motivation behind their creation was. Karsvall explains that: “By incorporating texts of diverse types from various historical periods, the goal is for the model to generalise effectively and apply to archival material beyond its original training scope.” To create this model, it was necessary to include a variety of texts from different historical periods.&nbsp; This diversity of training data helps to make the model more effective and applicable to a wide range of archival materials, ensuring better accuracy and performance when transcribing handwritten documents from different periods and styles.</p> <p>The Swedish Lion Ⅰ is envisioned as a basic model for Swedish historical texts, which will simplify access to handwritten materials and support data-driven research.&nbsp;</p> <p></p> <figure class="wp-block-image size-full is-resized" style="text-align:center;"><img src="https://readcoop.eu/wp-content/uploads/2024/02/image.png" alt="" class="wp-image-34514" style="width:50%;height:auto;margin:0 auto;"><figcaption class="wp-element-caption"><em>The Swedish Lion Ⅰ model for Swedish handwriting.</em></figcaption></figure><p></p> <h2 class="wp-block-heading">Training a versatile model&nbsp;</h2> <p>The <a href="https://www.transkribus.org/model/swedish-lion-i/">Swedish Lion Ⅰ model</a>, carefully trained using a wide range of historical documents, particularly court records and minutes from the 1600s, 1700s and 1800s, truly demonstrates the capabilities of Transkribus. Accordings to Karsvall: "Getting started with Transkribus was easy". The software's potential can be seen in the collaborative process of transcribing 3.3 million lines of text from 268 archival volumes. The final model was a result of various projects that have created &nbsp;<a href="https://www.transkribus.org/blog/what-is-ground-truth/">Ground Truth data</a> utilising specialised models and applying manual corrections The model's remarkable <a href="https://help.transkribus.org/character-error-rate-and-learning-curve">Character Error Rate</a> (CER) of just 4% confirms the model's great performance. This is particularly evident in the processing of running text and marginal notes.<br></p> <p>Olof Karsvall acknowledges a challenge in managing diverse documents: “As we manage a wide array of documents, a significant challenge has been the segmentation of regions and lines.” Fortunately, the introduction of new trainable <a href="https://www.transkribus.org/field-models/">Field Models</a> and <a href="https://www.transkribus.org/table-models/">Table Models</a> brought greater accuracy, easier segmentation, and better recognition of layout structures. After three years of careful transcription, manual review, and correction, the Swedish Lion Ⅰ model is now ready and available as a public model!</p> <p><br><a href="https://www.transkribus.org/model/swedish-lion-i/"><strong>-&gt; Swedish Lion Ⅰ Model&nbsp;</strong></a></p> <p></p> <h2 class="wp-block-heading">Next steps for the Swedish Lion Ⅰ Model</h2> <p><br>The hope is that the Swedish Lion I text recognition model will reach new users via Transkribus and stimulate the usage of historical archive material in Swedish. Its collaborative development, involving several institutions, researchers, and volunteers, is already a great inspiration. Karsvall highlights the intention to extend this collaboration, creating a larger model covering older periods and diverse materials, and thereby promoting citizen science. Colleagues and the archival community have already shown growing interest, leading to increased requests to collaborate. The team plans to apply the model to several large collections to meet the expectations of increased accessibility to archives, following the publication of the Swedish Lion Ⅰ model.</p> <p>Thank you Olof Karsvall for the interview and for sharing the journey of the Swedish Lion Ⅰ model!</p> <p></p> <h4 class="wp-block-heading">Olof Karsvall's Transkribus Tips:</h4> <blockquote class="wp-block-quote"> <p>“Seek advice from others who have undertaken similar projects previously”</p> <p>“Share your data; everyone benefits if data can be reused”</p> </blockquote> <div style="height:25px" aria-hidden="true" class="wp-block-spacer"></div> <p>Thumbnail: Olof Karsvall, © Emre Olgun</p><div style="height:25px" aria-hidden="true" class="wp-block-spacer"></div>  <h2 class="wp-block-heading">Read more about training large models:</h2><p><br><a href="https://www.transkribus.org/success-story/norhand"><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/norhand/SuS%20NorHand%20Model.png" style="width:50%;height:auto;"><br>The NorHand Model: a new public AI model from the National Library of Norway</a></p><p><a href="https://www.transkribus.org/success-story/reading-english-secretary-hand-with-ai"><img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/blog/egerton/Thumbnail.png" style="width:50%;height:auto;"><br>Reading English Secretary Hand with AI: the Egerton Model</a></p>

Creating the Swedish Lion Ⅰ text recognition model

<p>Have you ever sent a letter that never arrived? From 1652-1815 the British navy and privateers seized all types of documents from enemy ships, ship logs, cargo lists and even private letters. Hundreds of years later, the <a href="https://www.huygens.knaw.nl/en/home-english">Huygens Institute</a> in the Netherlands started the <a href="https://www.huygens.knaw.nl/en/projecten/dutch-prize-papers-2/">Dutch Prize Papers</a> project to digitise and analyse these historical documents and transcribe them with the help of <a href="https://readcoop.eu/transkribus/">Transkribus</a>. </p> <p><a href="https://www.huygens.knaw.nl/en/medewerkers/marijcke-schillings-2/">Marijcke Schillings</a>, historian, researcher and coordinator of the Dutch Prize Papers told us more about the project from its start in 2016, its documents and how the Transkribus software was involved to create an AI text recognition model.&nbsp;</p> <div style="height:3px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-the-dutch-prize-papers-project">The Dutch Prize Papers project</h2> <h4 class="wp-block-heading"><strong>What are the Prize Papers?</strong></h4> <p>From ship logs, cargo lists, plantation records, and crew interrogations to letters, the documents making up the <a href="https://www.huygens.knaw.nl/en/projecten/dutch-prize-papers-2/">Dutch Prize Papers</a> are anything but trivial. These papers are kept at <a href="https://discovery.nationalarchives.gov.uk/details/r/C142">The National Archives</a> in Kew (London).</p> <p>Marijcke Schillings explains that “the Prize Papers are documents seized by the British navy and privateers from enemy ships in the period 1652-1815”. As a powerful player on the seas, the British Navy, together with privately owned and operated ships, engaged in naval warfare to disrupt enemy trade.</p> <p>“This collection also contains approximately 38,000 Dutch business and private letters,” says Schillings<sup>1</sup>. With the great variety of documents, the Prize Papers therefore offer the possibility of “different types of international research"&nbsp; and insights into “all social strata of society”.</p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/vxswC153TNaZJXJ3WnjDfadO2muQqE2W-l6C7W3apFBZmL1NyK8VEu6wP5JTuSI_KPfUC1RrK6oi8zDHYL8fqdc5-4lxjRaAHqhf6qFmJZABzacP5nAN5Ojqf69Tez-hHPkX8de0CTbcSHylJlSr" alt="C:\downloads\Blog_5_Transkribus_transcriptie.PNG"/><figcaption class="wp-element-caption"><em>PrizePapers Collection, folder 1800-1810_25/HCA32-1210-0033, fragment of a GT-transcription. Transkribus platform</em></figcaption></figure> <p></p> <h4 class="wp-block-heading"><strong>What is the Dutch Prize Papers project?</strong></h4> <p>As the national institute for research on the history and culture of the Netherlands, the <a href="https://www.huygens.knaw.nl/en/projecten/">Huygens Institute</a> (HI) is dedicated to innovative and collaborative research on historical sources and literary texts. The aim of the Dutch Prize Paper project was, firstly, to have a large selection of digitised (Dutch) documents available for research and, secondly, "to make printed and handwritten texts more searchable and readable".&nbsp;</p> <p>At the end of 2015, the Huygens Institute received a substantial subsidy that made it possible to achieve the first step.</p> <p>In June 2019, 72,000 scans or 140,000 pages of mainly Dutch documents from the seventeenth to the early nineteenth century and their metadata became available online on the <a href="https://prizepapers.huygens.knaw.nl/">Dutch Prize Papers</a> website. To improve access to digitised documents, the Huygens Institute therefore created a Virtual Research Environment (VRE).</p> <p>Working towards the second step, Marijcke Schillings and her colleagues from the DPP project turned to Transkribus’ text recognition software. Since several Huygens Institute projects "already had experience with the user-friendly HTR platform and achieved good results", the team decided to start a pilot project, with the primary goal of exploring automatic text recognition.</p> <p><img width="602" height="99" src="https://lh7-us.googleusercontent.com/KcJyVJMTJnz1qos3S8D4TCh7g6hVYGRQFwYltbG6XNVTi6lTGIdEU-jv4mEM5GZ3h7YNL6w7_mfXdSBZBQr67tL9Zk6kTOvkZDFoMbx9s4UmJsD0OOhSzWp3ibzzW-KfPDq0NGgbtdmf1xH9G5bU" alt="C:\downloads\Blog_4_Transkribus_LA.PNG"> </p> <p><em>PrizePapers Collection, folder 1800-1810_24/HCA32-1210-0016b, fragment layout-analysis. Transkribus Platform</em></p> <div style="height:3px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-creating-an-ai-model-with-transkribus">Creating an AI model with Transkribus</h2> <p>For this pilot project, 100 scans of documents from different centuries and written in varying languages were selected to <a href="https://help.transkribus.org/training-models">train a custom text recognition model</a>.</p> <p></p> <h4 class="wp-block-heading"><strong>Layout Ground Truth&nbsp;</strong></h4> <p>After choosing the material, the team started working on creating <a href="https://readcoop.eu/what-is-ground-truth/">Ground Truth</a> pages of the layout, specifically the <a href="https://readcoop.eu/glossary/text-region/">text regions</a> and the <a href="https://readcoop.eu/glossary/baseline/">baselines </a>of the historical pages. Schillings elaborates that the baselines were first placed automatically and then checked manually, as the lines of text tended to be disintegrated or crooked.</p> <p>Using the <a href="https://readcoop.eu/glossary/p2pala/">P2PaLA </a>layout analysis tool the Ground Truth pages were then used to train three structure recognition models. However, when these models were tested, the results were not as accurate as hoped, indicating the need for additional training material. Recognising the challenges with the <a href="https://readcoop.eu/glossary/p2pala/">P2PaLA</a> layout analysis tool, Transkribus has since introduced trainable layout models, such as the <a href="https://readcoop.eu/introducing-field-models-trainable-layout-ai-in-transkribus/">Field Models </a>and <a href="https://readcoop.eu/introducing-table-models-trainable-layout-ai-in-transkribus/">Table Models.</a> These trainable layout models require less training data while being more precise.</p> <p></p> <h4 class="wp-block-heading"><strong>Text Ground Truth</strong></h4> <p>The next step was to create <a href="https://readcoop.eu/what-is-ground-truth/">Ground Truth</a> pages of transcribed text to train the text recognition model. The Ground Truth pages were generated by using existing models and then checked and manually corrected. Based on 100 pages of Ground Truth, the DPP team created two custom text recognition models. “We decided to let create a model including a base model (i.e. <em>IJsberg</em>) first and a second one, exclusive of a base model.”</p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/HuzXtXejGPLP1vjXIqC93AxtVdkvS1j_UA2MSBGlQUTakk5-toaDXDng-MFB04b4v7BKdYawdFgvEDIuuEkAdOZmbm5LU5zlEv-yrDNbscqYH70TZ9_TRH_px3jAjWdtoa8NTAhtZRri08jAxFhV" alt="C:\downloads\NL-HaNA_2.22.24_HCA30-342_0253b-fragment.jpg"/><figcaption class="wp-element-caption"><em>PrizePapers Collection, folder 1700-1799_11_r2-2/HCA30-342-0253b, invoice. Transkribus platform</em></figcaption></figure> <p></p> <p>Model Result Comparison:<br><em>DPP</em>= ede gescheept in het Schip de Gesina Mana, Comyn Cannelis</p> <p><em>DPP2</em>= Dene gescheept en her Schip de Gesena Aana, Comin Corneeir</p> <p>Manual=ende gescheept in het schip De Gesina Maria, Captyn Cornelis</p> <p>As the team expected, the first model, including the base model <em>IJsberg</em>, produced the best results, which is shown in the comparison of text recognition results.</p> <div style="height:3px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-working-with-the-transkribus-platform">Working with the Transkribus Platform&nbsp;</h2> <p>“The experience with the Transkribus tools was very good.” summarises Marijcke Schillings. By creating two multilingual models, the team explored the potential of Handwritten Text Recognition (HTR), which was the primary objective of the pilot project. This effort resulted in a positive evaluation report that showed a significant improvement in readability.</p> <p>&nbsp;Due to challenges in the accuracy of the layout analysis, a different tool called “Loghi” was applied to the documents of the Dutch Prize Papers Project in June 2023, which improved considerably the readability and searchability of the documents.&nbsp;</p> <p>Listening to the feedback of our users, Transkribus now offers an improved and more efficient way of recognising layouts: trainable layout models. The trainable <a href="https://readcoop.eu/introducing-field-models-trainable-layout-ai-in-transkribus/">Field Models</a> and <a href="https://readcoop.eu/introducing-table-models-trainable-layout-ai-in-transkribus/">Table Models</a> are designed to produce accurate results even with complex layouts such as those found in newspapers, index cards or spreadsheets.&nbsp;</p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/NbBzXW-GMTwNb62RNsWBgsEIWfsBaM5Pg1bHHhhwOfBwOK2FBALvFfp29kdYYZEXkvosSCYgmxF2kRDmqA8iYPS_AvkxMhpdc4lnUBJmuflrUkXCT5YDvVspRGxCglpg8BBpGDzkH8Q1hGKwSDRl" alt="C:\downloads\Blog_3_DPP_Vrachtbewijs.PNG"/><figcaption class="wp-element-caption"><em>Website Dutch Prize Papers, HCA30-1056-0072b, Bill of lading</em></figcaption></figure> <div style="height:3px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-creating-opportunities-for-further-research">Creating opportunities for further research</h2> <p>Marijcke Schillings concludes that with this project, the DDP team was able to allow “interested people anywhere to view a small selection of papers”, consisting of more than 100,000 images, that are legible and digitally available.</p> <p>The next step of the DPP project is to focus on making one specific type of document accessible, the bills of lading. Bills of lading were not usually kept after the shipment of goods by sea, clarifies Schillings. They do, however, reappear in the cargoes seized by British privateers<sup>2</sup>.&nbsp;</p> <p>We at Transkribus are delighted to have been part of this pilot and wish the DPP project team continued success in their research into the bills of lading.</p> <p>Thank you Marijcke Schillings for taking the time to talk to us!</p> <p></p> <div style="height:69px" aria-hidden="true" class="wp-block-spacer"></div> <p class="has-small-font-size"><sup>1</sup> R. van Gelder, Zeepost. Nooit bezorgde brieven uit de 17de en 18de eeuw (Amsterdam/Antwerpen 2008) 20-21.</p> <p class="has-small-font-size"><sup>2</sup>&nbsp;“Flessen op papier”, A.P. v[an] V[liet], in: <em>Buitgemaakt en teruggevonden. Nederlandse brieven en scheepspapieren in een Engels archief. Sailing Letters Journaal V</em>. Onder redactie van E. van der Doe, P. Moree, D.J. Tang, met medewerking van P. de Bode (Zutphen 2013) 196-197. </p> <div style="height:69px" aria-hidden="true" class="wp-block-spacer"></div> <p><em>Thumnail: </em>Logo website <a href="https://prizepapers.huygens.knaw.nl/">Dutch Prize Papers</a></p>

Navigating the Transcription of Dutch Prize Papers

<p>There are some Transkribus projects that aren’t only important for history, but for society too. “Trug und Schein” is one of those projects. For over 10 years now, volunteers have been transcribing the 2,600 letters of wartime lovers Roland and Hilde Nordhoff, and making this valuable historical source accessible to the public. But more than that, they have created an environment where such sources can be discussed, contextualised, and shared by senior citizens, young students, and everyone in between.</p>

<p>We spoke to Laura Fahnenbruck, former co-project leader of “Trug und Schein”, to find out more about creating a citizen science project with Transkribus.</p>

<h2>How it all started</h2>

<p>At the heart of the “Trug und Schein” project are two wartime lovers: Roland and Hilde Nordhoff (names are pseudonyms to protect their real identity). She was a factory worker, he was a teacher and led the church choir where the pair first met in 1938. Shortly after getting married in 1940, Roland was called up to fight in the war that had engulfed Europe. “As is so often the case, the couple was separated by the war,” Laura explained. “One was transferred to the front in Eastern Europe; one remained in rural Saxony. Like so many others, they communicated by letter, almost daily.”</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image01.jpg" alt="The choir where Roland and Hilde met." style="width: 100%; height: auto;"><em>The choir where Roland and Hilde met. © Trug und Schein</em>

<p>But the fact that so many of the Nordhoffs’ letters have been preserved is not usual. In total, the team received around 2,600 letters from the Nordhoff’s heirs, some of which are up to 20 pages long. With a collection of this size, the potential for gaining new insights into everyday life in Germany under National Socialism is huge.</p>

<p>“We agreed that this source was something very special. We didn’t want to let [it] continue to gather dust in the attic, but we also didn’t want to lock it away in an archive where only a few people would find their way to it. Therefore, we decided to make it accessible by digitising it and publishing it online as a blog.”</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image02.png" alt="A heart drawn by Roland Nordhoff in one of his many letters." style="width: 100%; height: auto;"><em>A heart drawn by Roland Nordhoff in one of his many letters. © Trug und Schein</em>

<h2>Making history accessible for everyone</h2>

<p>Right from the start, “Trug und Schein” was intended to be a participatory public history project. “Public history is linked to the approach that history is a matter for everyone and not just professional historians,” Laura explained about her team’s mentality. “This is how the idea of publishing, contextualising, and discussing these letters as a citizen science project came about.”</p>

<p>Citizen science is a method that involves volunteers in some aspect of the research process. In this case, the volunteers were tasked with transcribing the large corpus of handwritten letters. “As the manuscripts in the correspondence can no longer be read by everyone today, it was obvious to seek the expertise of people who can still read these writings,” said Laura. Those people turned out to be a mix of senior citizens who could still read the Sütterlin, Kurrent, and modern cursive scripts used in the letters, as well as younger students who were adept with technology, studying German as a foreign language, or both.</p>

<p>But the volunteers were more than just transcribers. “For us, it was always about co-creation. All participants were equally empowered to help determine the course of the project. We wanted to be democratically involved in the negotiation process about the past and have the knowledge of the crowd and the path to this knowledge as our goal.”</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image03.png" alt="A screenshot of one of the 2,600 letters being transcribed with Transkribus." style="width: 100%; height: auto;"><em>A screenshot of one of the 2,600 letters being transcribed with Transkribus. © Laura Fahnenbruck</em>

<h2>Transcribing with Transkribus</h2>

<p>At the beginning of the project, all the letters were transcribed manually. But over time, it became clear that transcribing the whole collection could become a never-ending project. “We needed to make our work easier,” Laura said. “That’s when Transkribus came into play.”</p>

<p>Even though many of the older volunteers hadn’t grown up with computers, they were open to trying this new software to see if it would make their work quicker. The first training course the team organised was offline. Over the course of a week, the volunteers familiarised themselves with the software and the new workflow. From then on, the seniors, together with the principal investigators and student volunteers, were quickly only working online. This allowed the project to evolve into a truly international initiative, with volunteers joining not just from Germany, but also from Austria, the Netherlands, and the USA.</p>

<p>“For some of our participants […] it can be a disaster if the mouse hangs, the browser view changes, or a pop-up window can't be clicked away. It was always important for me to take away the fear that something could break.” To achieve this, Laura and her colleagues wrote a manual, explaining how to use Transkribus and what to do if something unexpected happens. Thankfully, it is rare that anything is actually “broken” by a volunteer: “With Transkribus, there’s nothing that can’t be restored,” Laura said.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image04.png" alt="The three models created by the project." style="width: 100%; height: auto;"><em>The three models created by the project. © Transkribus</em>

<h2>Creating models for everyone to share</h2>

<p>“It was a good thing that we had already transcribed 1,000 letters before Transkribus because they provided the basis for our first own text recognition model.” Back in 2018, the Transkribus team created two models for the project, one for each writer. “Basically, the first models were good enough. It was only when I realised in the course of another Transkribus project that there were not yet that many public models for the 20th century and for modern German handwriting, that I realised our data set is also very useful for others.”</p>

<p>Laura then set about training three new models: one for <a href="https://app.transkribus.org/training/text/58659">Roland’s handwriting</a> with a mix of Kurrent, Sütterlin, and cursive; one for <a href="https://app.transkribus.org/training/text/59628">Hilde’s modern cursive handwriting</a>; and one trained with the training data from <a href="https://app.transkribus.org/training/text/58704">both writers</a>. All three models were highly successful: “Because we always transcribe so accurately and the handwriting is quite consistent, we have achieved really low CERs.”</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image05.png" alt="The search console of Alltag im Krieg." style="width: 100%; height: auto;"><em>The search console of Alltag im Krieg. © Alltag im Krieg</em>

<h2>Joining forces with similar projects</h2>

<p>“Trug und Schein” is well on its way to being completed. About half of the letters have already been published, another quarter have been transcribed, and the last quarter is waiting to be processed. It has resulted in not just a database of transcribed letters but also a theatrical play, a string of academic projects, and even a regular <a href="https://www.tonkuhle.de/tonkuhle-programm/wortsendungen/46676-trug-und-schein-die-sendung.html">audio programme on Radio Tonkuhle</a>.</p>

<p>After a while, however, the project website began to face technical problems. Laura and her colleagues, therefore, decided to join forces with the <a href="https://geschichts-kontor.de/">Culture and History Office</a>, an archive in Bergedorf near Hamburg, who were also working on digitising letters from the period. “Collaborating with a non-profit historical institute has enabled us to secure the long-term digital archiving of our letters.”</p>

<p>Merging these projects required the assistance of yet more volunteers; for example, software engineers who wrote Python programs to transform multiple data sources into XML formats suitable for uploading to the new site. It was always very important in the transcription process to digitise not only the textual information of the letters, but also the visual information that the writers gave, such as the use of a smaller or larger font. This could now also be exported from Transkribus.</p>

<p>Together, the expanded research team of scholars and citizens has created a new searchable digital archive: “<a href="https://alltag-im-krieg.de/startseite">Alltag im Krieg</a>”, or “Everyday Life in the War”. This new website allows users to search through Roland and Hilde’s letters as well as the correspondence of two other German couples during the war. Both scholars and the general public can take advantage of this rich collection of sources and discover more about life in wartime Germany from the people who were there.</p>

<img src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/success-stories/trug-und-schein/Image06.jpg" alt="Roland and Hilde’s story has brought the generations closer together." style="width: 100%; height: auto;"><em>Roland and Hilde’s story has brought the generations closer together. © Trug und Schein</em>

<h2>A project for social good</h2>

<p>Laura and her colleagues set out to digitise a collection of letters and make them accessible to everyone. That aim has definitely been achieved. But in addition, the project has also brought awareness of the social impact that such public history initiatives can have. “Senior citizens can use their skills for the good of society; students can get to know the older generation; people learning German can come into contact with native speakers; and so on.”</p>

<p>“The senior citizens were able to share their experiences – for example, about what their own parents had (not) told them about this time – and the students were able to ask questions that they could no longer ask at home. I repeatedly heard from the team that this engagement was one of the most important achievements of the project,” Laura explained.</p>

<p>“In the end, you get more than just a digitised historical source.”</p>

<p>Thank you, Laura, for sharing your project with us!</p>

Trug und Schein: unlocking the letters of wartime lovers

<p>"According to the inquisitorial rules in the past, everything had to remain secret.” To punish people accused of straying from the Christian belief, the Portuguese Inquisition conducted trials clouded in secrecy and censorship. However, these trials were recorded in court files that are currently stored in the National Archives of Portugal (ANTT), waiting to be transcribed and analysed. Today, thanks to the hard work of the <a href="https://traprinq.mozellosite.com/quem-somos/">TraPrInq</a> project, long-kept secrets of the Portuguese Inquisition are gradually being revealed. </p> <p>In order to uncover the contents of inquisitorial trial records, the TraPrInq team set out to use an AI-powered text recognition model to transcribe these documents dating from 1536 to 1821. <a href="https://www.fcsh.unl.pt/en/college/researchers/herve-didier-christophe-baudry-en/">Hervé Baudry</a>, senior researcher at the <a href="https://www.fcsh.unl.pt/en/unidades-investigacao/cham-nova-fcsh-en/#:~:text=In%20a%20multi%2Dcultural%20and,and%20areas%20of%20History%20(Economic%2C">CHAM-Centre for the Humanities of the New University of Lisbon</a> and coordinator of the <a href="https://traprinq.mozellosite.com/quem-somos/">TraPrInq</a> project, explained to us how it was possible to transcribe the inquisitorial court trial records with the help of <a href="https://readcoop.eu/transkribus/">Transkribus</a>.</p> <div style="height:11px" aria-hidden="true" class="wp-block-spacer"></div> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/fXbua74T8emstqNAyIxEPWeMMkFw_9ucM6F5z5TOYMT8FGq1NLXycsZxOgqSDzG1U4HmHCpdSIlEnHyP4m5kG_MOlwgRf1sKGCrzvPxzV5xU6z7WPY1zSiDA0xsDer0P1DsRuQkgvVI33KXN3OnVIXraJ0mvSXNVFogR4kx__ELCFm1lquqw1hvCqpCN" alt=""/></figure> <p><em>Official </em><a href="https://traprinq.mozellosite.com/"><em>Website</em></a><em> of the </em><a href="https://traprinq.mozellosite.com/o-projeto/"><em>TraPrInq</em></a><em> Project. © TranPrInq</em></p> <div style="height:25px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-the-traprinq-project">The TraPrInq Project</h2> <p>300 years of archival output and more than 40,000 files from these court trials of the Portuguese Inquisition are held by the National Archives of Portugal, waiting to be transcribed and analysed. Recognising the potential of the cour trial records, ‘Processos’, the <a href="https://traprinq.mozellosite.com/o-projeto/">TraPrInq project</a> was created. The unique name ‘TraPrInq’, a nonce word, sums up its mission: "Transcrever os Processos da Inquisição Portuguesa, 1536-1821", which translates as “Transcribing the court trials of the Portuguese Inquisition, 1536-1821”.&nbsp;</p> <p>However, there is a significant challenge in trying to unlock the history hidden within thousands of pages of documents - accessibility.&nbsp; Not everyone can access these historical files or read centuries-old manuscripts.&nbsp;</p> <p>Given his previous knowledge of Transkribus, Hervé Baudry recognised this challenge and thought of a solution: “In 2020, a webinar for Brazilian undergraduates on Inquisitorial Studies led me to conceive a project that could help to overcome the difficulties faced by the non-paleographers to read and consult the archives.” This led to the creation of the TraPrInq project, consisting of a <a href="https://traprinq.mozellosite.com/quem-somos/">team of eleven paleographers</a> from Portugal and Brazil, with the primary goal of creating a versatile <a href="https://help.transkribus.org/training-text-recognition-models">text recognition mode</a>l using Transkribus and making it available to the public.&nbsp;</p> <p></p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/bbEQMfiVvSSBh8P8huuxpd8ArhVMDL8ZDTa57VGatRm1Jd9-6MxmR1bQTrktv5vpE7kQSLdnBihMUXUPcxmdh08kSpXv5sF-iRBFcYf2YbRkNz9ADgHQvfkGlKzjOKrYy6JHA053cCPU6xJGdHMAHd4NPpYogbD12fyfeXYsdffgkijTZONPdSfrs6_G" alt=""/></figure> <p><em>Trial of Manuel Gois, Court Trials of the Portuguese Inquisition,1586. © ANTT</em></p> <div style="height:11px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-the-court-trials-of-the-portuguese-inquisition">The court trials of the Portuguese Inquisition</h2> <p>Making these ‘Processos’ accessible for research and to the public would present a valuable resource, Hervé Baudry explains. “In the Portuguese culture, as in the Spanish and Italian ones, early modern censorship is strongly linked to the Inquisition.” The close link between early modern censorship and the Inquisition in Portuguese, Spanish and Italian cultures can be attributed to the Inquisition's dual role as a religious and political authority. Identifying and suppressing religious deviations from Christianity within the country, the Portuguese Inquisition put people on trial to imprison them on the accusation of heresy or witchcraft. When censorship became part of inquisitorial activities, everyone who had contact with the Inquisition, even inmates who were released due to overcrowded prisons had to sign an oath of secrecy. </p> <p>Apart from the general historical value of discovering information about the past, revealing the contents of documents treated with secrecy can lead to new discoveries and insights. Determined to do so, the TraPrInq team came together to unlock the trials of the Portuguese Inquisition using Transkribus.</p> <p></p> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/GrWdxZn3aBdsjbU-iPiJk02U234xdLM26rm6tV1TvhnZY3xTX4clMVewVX3Zbz3CcCCVNJhR2W6jLf9VCpop4Hs_ax1yuAOfE8WhtJDV0TlIDCjN-n4mQl83EbIwh2Odv_MfSFIKuoElcIAzNyL7GDYiuhWD7k9mRpF6HFtypjvxiVFid15nAmbC9GiY" alt=""/></figure> <p><em>Trial of Isaac Almosnino, Court Trials of the Portuguese Inquisition, 1618. © ANTT</em></p> <div style="height:11px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-transcribing-centuries-old-documents-with-transkribus">Transcribing centuries-old documents with Transkribus</h2> <p>The material used to train the Transkribus text recognition model consisted of handwritten documents of court trials from the Portuguese Inquisition from the years 1536 to 1821. From the existing digital database of the National Archives of Portugal, TraPrInq extracted over 6,000 images and transcribed over a million words for the creation of <a href="https://readcoop.eu/what-is-ground-truth/">Ground Truth</a>.&nbsp;</p> <p>To create the Ground Truth and train an efficient <a href="https://help.transkribus.org/training-models">customised model</a>, firstly a reasonable number of texts had to be transcribed and secondly, texts from different hands had to be allocated among the project members. Hervé Baudry explains further that “there was an imperative for quantity and quality, renewed at each session following training on the model.” Over the course of nine training sessions, starting in June 2022 and ending in July 2023, the team transcribed the entire corpus of 6000 digitised documents. After the fourth training session of the Portuguese text recognition model, the results started to become more efficient, to the point where “manual transcribing could be gradually substituted by only correcting the errors made by the [text recognition] engine”.</p> <p>There was still a struggle when transcribing damaged documents or records with complex layouts. “Such images have a CER above 15-20%”, states Baudry.&nbsp;</p> <p>While there is no way to undo damage to historical documents, Transkribus has already improved layout recognition with new trainable layout models. With the <a href="https://readcoop.eu/introducing-field-models-trainable-layout-ai-in-transkribus/">Field Models</a> (already available on <a href="http://beta.transkribus.eu/">beta.transkribus.eu</a>) and Table Models (coming soon) Transkribus models can be trained to recognise and transcribe tricky layouts such as periodicals, registries, legal records or forms automatically.&nbsp;</p> <p>Overcoming the challenges, the final Character Error Rate (CER) after training was 5.2%, making the accuracy of the Transkribus text recognition model as high as 94.8%.</p> <div style="height:16px" aria-hidden="true" class="wp-block-spacer"></div> <figure class="wp-block-image"><img src="https://lh7-us.googleusercontent.com/RfHjytY9xP09K-lTQP-oRrQyjj-8b1Q3wRzBiU6ZICSDSJHPFP7Yod-Rzlqb9a6L849VLNQ2lann39YucT7mIRMay3BhQhmT3hUlk0cJqqC_qHmtuRqBpFZssRz3r4ejvpvU_hFxO-qI9xcWITOhY7c3PlAAyMJtd0Se3FbFt2mOdL8TtBDxnUHLagO7" alt=""/></figure> <p><em>Public Transkribus text recognition model: Portuguese Handwriting 16th-19th century. </em></p> <div style="height:11px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-a-successful-outcome-of-the-traprinq-project">A successful outcome of the TraprInq project</h2> <p>The goal of the 18-month-long TraPrInq project was to create a public text recognition model that is available to anyone. And indeed, in the summer of 2023, the new model for Portuguese Handwriting from the 16th-19th century was made publicly available for every Transkribus user.</p> <p>Find the Model here: <a href="https://readcoop.eu/model/portuguese-handwriting-16th-19th-century/">Portuguese Handwriting 16th-19th century</a></p> <p>Baudry is satisfied with the outcome of the project: “Members of the team and people from other projects who contacted ours used the model with high efficacy”. Hervé Baudry adds that, more significantly, the texts automatically recognised by this new Portuguese model were not part of the TraPrInq project and came from non-inquisitorial sources. This means that the model is effective with different texts and document types. These results were presented at the final meeting of the project, <a href="https://traprinq.mozellosite.com/coloquio-2023/">Humanidades Digitais e Estudos Inquisitoriais</a> (Conference on Digital Humanities and Inquisitorial Studies) in June 2023.</p> <p>Furthermore, Baudry states that the project served as an initial step toward systematic document transcriptions, to make the digitised and transcribed documents available on Transkribus Sites (former Read&amp;Search).</p> <p>Reflecting on the project, Hervé Baudry expresses his admiration for all the <a href="https://traprinq.mozellosite.com/quem-somos/">participants in the TraPrInq project</a> who, despite the uncertainty of the outcome, “took the plunge anyway, just by accepting my invitation to participate!”. And together they have succeeded in achieving their goal. He would also like to express his gratitude to the late Dirk Alvermann</p> <p>who, at a crucial time for TraPrInq, put all his skills and sympathy at the service of the project.</p> <div style="height:11px" aria-hidden="true" class="wp-block-spacer"></div> <h2 class="wp-block-heading" id="h-unlocking-opportunities-for-future-projects">Unlocking opportunities for future projects</h2> <p><a href="https://readcoop.eu/de/insights/ocr-vs-htr/">Handwritten Text Recognition</a> (HTR) technology is not new to the archival community, and while there is a great deal of interest, there are not always the means to use it in such a capacity.&nbsp; “TraPrInq was a first step toward something massive and ambitious”, Hervé Baudry observes. Until now, only a fraction of the entire 3.4 million pages of the Inquisitorial Archives have been studied, and only a small part of the 40,000 files from the National Archives of Portugal have been transcribed and edited.</p> <p>Baudry explains that this Transkribus model was created as a tool to make inquisitorial archives more accessible, and that hopefully one day all records of the Portuguese Inquisition will be transcribed and made legible for all.</p> <p><a href="https://readcoop.eu/transkribus/">Transkribus</a> offers the potential to extract information and knowledge from historical documents and unlock the written past. With the ability to train custom models using the AI-powered transcription software, projects such as <a href="https://traprinq.mozellosite.com/o-projeto/">TraPrInq</a> have the potential to transcribe historical documents and make them available for further research.&nbsp;</p> <p>When it came to the court trials of the Portuguese Inquisition, Hervé Baudry concluded that Transkribus’ HTR software “was able to break a double secret, one historical, the other, paleographical.”<br></p> <div style="height:25px" aria-hidden="true" class="wp-block-spacer"></div> <h4 class="wp-block-heading"><strong>Hervé Baudry’s Transkribus Tip:</strong></h4> <blockquote class="wp-block-quote"> <p>“Transkribus may look a bit complicated to learn, but in a few hours you can use it, and it is worth it.”</p> </blockquote> <div style="height:41px" aria-hidden="true" class="wp-block-spacer"></div> <p><em>Thumbnail: Portrait photo of Hervé Baudry. </em></p>

Uncovering the secrets of the Portuguese Inquisition with Hervé Baudry

<p>In many cities, archives house rich collections of letters, books, diaries, minutes and other valuable artefacts. However, these sources of knowledge and information are often underused as they are either fragile or the handwriting might be illegible to the untrained eye.</p> <p>The Archivverbund Bautzen faced a similar challenge - nearly 300 books containing the minutes of the weekly city council meetings from 200-400 years ago, providing insights into the city's past, were being under-utilised due to hard-to-read handwriting. For today's article, we spoke with Grit Richter-Laugwitz, head of the <a href="https://www.archivverbund-bautzen.de/">Archivverbund Bautzen</a>, about the task of transcribing Bautzen's council minutes using Transkribus, the importance of these records, and how Transkribus has helped make them accessible to the public online.</p> <p><img src="https://lh4.googleusercontent.com/sXjiWDddTla00MCb-orLFHSx5eYwAIPpm8rvI3RMhoWHpl-J50Y2KiScKnTi_0B6Qjtpx66qOvOexLPsr0IRkT1cyD1_PM3hM5KEY7D7XobIKMZnDPKkxznW-8y_wA6OqMoo4pAmRxajDD5zxP-1LQ" width="602" height="456"></p> <p><em>Grit Richter-Laugwitz, Head of the Archive Association Bautzen. © Holger Hinz</em></p> <p></p> <h2 class="wp-block-heading" id="h-starting-the-project">Starting the project</h2> <p>With weekly entries into the 257 council minute books from 1623-1832, the city of Bautzen has documents that offer insights into (almost) everything that happened in the city during that time. Grit Richter-Laugwitz and the <a href="https://www.archivverbund-bautzen.de/">Archivverbund Bautzen</a> (consisting of the Stadtarchiv and the Staatsfilialarchiv Bautzen,) realised that two centuries of the city's history documented in these records were a rich source of knowledge. Against this background, a project was launched to transcribe and bring to light this collection of historical records.</p> <p>Collaboration was at the core of this initiative, with Transkribus joining Archivverbund Bautzen as a project partner. The financial support for the project came from the ‘<a href="https://www.bibliotheksverband.de/wissenswandel-digitalprogramm-fuer-bibliotheken-und-archive-innerhalb-von-neustart-kultur">WissensWandel</a>’ initiative, a digital programme of the Federal Government Commission for Culture and the Media (BKM). Richter-Laugwitz explains that the programme was designed to help libraries and archives improve access to their collections independently of on-site use and to apply novel (digital) formats of knowledge and information transfer.&nbsp; The aim of this particular project was to make the ‘Ratsprotokolle’ (council minutes), one of the most important archival holdings of the city of Bautzen, more accessible.</p> <p><br></p> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/t0mRlWLb09LiAHp0HvP3BazJTKwNzGB0jenpKNm1yz8_rq_Z7hRd-v7eNbohXWX4B7sbbylNH9trP0mRZ00fvZs9y0AtU5tM515G5KEfVOKcFtBC1-e_KhO_GqqDwnpyP1wyYsdS2oS-DOfExBBnSA" alt=""/><figcaption class="wp-element-caption"><em><em>Ratsprotokolle bis 1832, Nr. 166 (28. Jan. 1732 - 30. Juli 1733). © Archivverbund Bautzen<br></em></em></figcaption></figure> <p></p> <h2 class="wp-block-heading">The Bautzen city council minutes</h2> <p>The council minutes of the city of Bautzen, dating from 1623 to 1832, form an important cornerstone of the city's historical heritage. But why are these records so important when it comes to understanding and researching the city’s past?<br><br>These books document the weekly meetings of the council, which consisted of seven to twelve men from wealthy patrician families, details Richter-Laugwitz, among its members, the council elected a mayor each year. “The council minutes record both the annual council business and the issues discussed at the meetings, which cover the full range of municipal functions and developments.”</p> <p>Richter-Laugwitz clarifies that although catalogued under the generic title of 'Council Minutes', their true content is only revealed by close examination of each entry.&nbsp; From information on sales or tax collection to political developments and issues, these historical documents captured the diversity of the city's affairs and progress.</p> <p>While these records were extensively restored and digitised between 2010 and 2018, with digital access available since 2019, navigating the sometimes poorly structured volumes remained labour-intensive. This has led to the under-utilisation of this resource, as “the collection was only used very sporadically by researchers - despite its significant relevance to a wide range of subjects”.&nbsp;</p> <p>To make the council minutes more accessible to those with limited knowledge of historical writing, the Archivverbund Bautzen approached Transkribus. Using Transkribus' AI-powered text recognition software, it would be possible to transcribe the handwritten records and make the transcriptions available and searchable online.</p> <p></p> <figure class="wp-block-image"><img src="https://lh5.googleusercontent.com/0xy9ezSoEK3bJwYxyJL8BqGvKIJejAb-Yf3sjaxo8_EQgcyg3D8W-LQMjdSZtybJxhkSv__mVYV0eeM0ExOpf2kKkdqFva76Q9fXhP3yZ0m_rcRiWxn0Tmz5r9T3j2LhdOOBYtatbsRf3FIrkDa2Ag" alt=""/><figcaption class="wp-element-caption"><em>Transcription with adjustable view (Transcription overlaying the text). Ratsprotokolle bis 1832, Nr. 254 (4. Jan. 1830 - 30. Dez. 1830). © Archivverbund Bautzen</em></figcaption></figure> <p></p> <h2 class="wp-block-heading">Transcribing 55.000 pages with Transkribus</h2> <p>With the Bautzen city council minutes having already been digitised by an external company in 2019-2021, Richter-Laugwitz tells us that she contacted Transkribus on the recommendation of Dirk Alvermann, head of the Greifswald University Archive. After detailed discussions and a successful application through the ‘WissensWandel’ funding programme, the historical documents were handed over to Transkribus for the text recognition and transcription process in March 2021.</p> <p>As mentioned above, the material consisted of 257 books containing approximately 55,000 pages of German handwriting from 1623-1832. This immense corpus of text was then transcribed using the Transkribus Early Kurrent model, an AI text recognition model for German Kurrent handwriting. After the initial transcription process in 2021, it became clear that a better transcription result was needed, as well as further improvement of the metadata (e.g. display of signatures on the search page). To achieve this, a custom AI model was trained using Early Kurrent as the base model, which was then tested to see if it gave better results. As the transcription of the new model showed a definite improvement to the transcription, the entire collection of documents was revised in the summer of 2023.</p> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/MwPs8dFdV0EjhjoSI7NL0mfC3E_5mljWBCZ-nyy75wBWH4nznNHcq9ENAtP7_0nRRJg9zeKSNyJBGCr9pJ8PNH4w9VLnrWYmWui7FW2cl_s7K2fkz_hj01xp6UjSokWDc7QUPGtnO8ua8WckMfbwjw" alt=""/><figcaption class="wp-element-caption"><em>Available search option on the <a href="https://transkribus.eu/r/bautzen-ratsprotokolle/#/">website</a> of the Archive Association Bautzen.&nbsp;</em></figcaption></figure> <p></p> <h2 class="wp-block-heading" id="h-exploring-bautzen-s-council-minutes-with-transkribus">Exploring Bautzen's council minutes with Transkribus&nbsp;</h2> <p>The successful outcome is the now digitised and transcribed records of 200 years of the city's history, uploaded to a <a href="https://www.transkribus.org/sites">Transkribus Sites</a> (previously called Read&amp;Search) website, which makes the documents from your Transkribus collection accessible online,&nbsp; so that they can be searched and accessed by anyone, anywhere.&nbsp; You can find the&nbsp; complete volumes of the Bautzen City Council Minutes on the Transkribus Sites website <a href="https://transkribus.eu/r/bautzen-ratsprotokolle/#/">Bautzen Ratsprotokolle</a>. The council minutes contain practically all the events considered noteworthy at the time that took place in the city, and now anyone can easily search all 257 books for any topic or term of interest. By simply entering search terms or names in the search engine of the website, the system will search all documents for corresponding entries.&nbsp;</p> <p>Researchers will find Transkribus’ search function and the added metadata in the <a href="https://transkribus.eu/r/bautzen-ratsprotokolle/#/">Bautzen Ratsprotokolle</a> website particularly useful, as it allows not just the viewing of documents but quick and accurate access to specific topics or names. Particularly in the case of large collections of documents, having a searchable corpus means that historical research becomes more focused and efficient. Richter-Laugwitz explains that digitised and transcribed archival records are of great value for historical research, as is the connection of the Transkribus Sites website to other portals. As part of the project, Transkribus has created permalinks to <a href="https://transkribus.eu/r/bautzen-ratsprotokolle/#/">Bautzen Ratsprotokolle</a>, which can be found in the portals <a href="https://www.archivportal-d.de/objekte?query=Ratsprotokoll+bautzen&amp;offset=0&amp;rows=20">Archivportal-D</a> and <a href="https://www.archivverbund-bautzen.findbuch.net/php/main.php">Findbuch</a>. By linking Transkribus Sites to existing portals or databases, we aim to provide added value to archives and researchers alike.</p> <p></p> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/jGP83dDB8rLZJpVfhTxbYUWyKpLCohEmqkE2MHuLg1dX9hiFivZlRuDjFtBDrq_I4ok4SmH5ZkPtZB3Ci3OvVdGlk3m7Dq1SJ77SWAwp634a-c6p03kdKXrd8qvNjHmMCf54AVFXATNN9OCnDDS0Wg" alt=""/><figcaption class="wp-element-caption"><em>Ratsprotokolle bis 1832, Nr. 256 (2. Jan. 1832 - 9. Juni 1832)</em></figcaption></figure> <p></p> <h2 class="wp-block-heading">A successful collaboration</h2> <p>While Transkribus proved to be a great choice for making historical documents accessible, Richter-Laugwitz explains that the lack of experience with the capabilities and limitations of the software was a challenge leading to many questions during the project. However, the collaboration with colleagues from Transkribus was very straightforward and constructive, so that questions or concerns could be resolved, leading to a satisfactory result, says Richter-Laugwitz.&nbsp;</p> <p>The project even attracted a lot of media attention, including reports in the regional press about the use of AI to research Bautzen's history,&nbsp; reported Richter-Laugwitz. The shared posts on <a href="https://twitter.com/StadtBautzen">Twitter</a> (now called X) also attracted a lot of interest and were shared and positively commented on several times nationwide. Richter-Laugwitz and the team at Archivverbund Bautzen are also looking to explore further opportunities with Transkribus, particularly for the digitised chronicles of the city of Bautzen.</p> <p>The successful cooperation between the Archivverbund Bautzen and Transkribus shows a shared commitment to preserving and sharing local history. We at Transkribus are delighted to have been part of this project to unlock Bautzen's written past.&nbsp;</p> <p>Many thanks to Grit Richter-Laugwitz for talking to us!</p> <p></p> <h3 class="wp-block-heading"><strong>Richter-Laugwitz’s Transkribus Tip:</strong></h3> <blockquote class="wp-block-quote"> <p>Start bravely, the possibilities are diverse and impressive.</p> </blockquote> <p></p> <p> </p> <p> <br><br></p> <p> </p> <p>Thumbnail Image: <em>Grit Richter-Laugwitz, Head of the Archive Association Bautzen. © Holger Hinz</em></p>

Making 200 years of Bautzen's city history accessible

Making 200 years of Bautzen’s city history accessible

<p>Working with historical documents written in Ottoman Turkish has long been a difficult task. Not only is it considered a 'dead' script today, but from accessibility to legibility, there are certainly challenges for those trying to research Ottoman Turkish texts. However, the Digital Ottoman Corpora team has taken on these challenges of transcribing, digitising, and analysing collections of Ottoman Turkish print to make Ottoman Turkish sources more accessible for research. We spoke to Suphan Kirmizialtin, who told us more about the Digital Ottoman Corpora project and how they even published the first text recognition model of Ottoman Turkish print.</p> <h2 class="wp-block-heading" id="h-the-digital-ottoman-corpora-project">The Digital Ottoman Corpora Project</h2> <p>Suphan Kirmizialtin, a historian specialising in the Ottoman Empire and researching the intersection of gender and modernisation in the Middle East, is leading the Digital Ottoman Corpora team in this project. Within the last few years, Kirmizialtin’s work shifted to digital crowdsourcing projects in cultural heritage as well as text recognition and textual analysis of historical archives. This brings us to today's project of the <a href="https://www.digitalottomancorpora.org/">Digital Ottoman Corpora</a>, where we talk about preserving and understanding Ottoman history through technology.</p> <p>Accessing historical sources is crucial for gaining a deeper understanding of the past. The question of accessibility becomes rather complex with Ottoman Turkish archives, due to the replacement of this language with a latinised and Turkified version in the late 1920s. Today, Ottoman Turkish, now considered a 'dead' script, is accessible only to specialists, unless the documents are transcribed into 'Modern Turkish'. With this in mind, the Digital Ottoman Corpora set the following goals for their project:</p> <ol> <li>Improve the accessibility of the Ottoman Turkish historical archives to both researchers and the public</li> <li>Help to create a digital research infrastructure for Ottoman Turkish.</li> </ol> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/aSffL7T01I7HRnD5FFCIc2Isa19_lkowryfWaL0sMwlKguN_eWYAJII5zQJ3poAcBBCyr7WnG0aJV0IyPaH8POA5O14MlPnWoBXcTv_StO3MYcrIXUOwBiBqTyokdvm5tliXUlgWOVkE_ESH28Gndg" alt=""/><figcaption class="wp-element-caption"><em>Official Logo of Digital Ottoman Corpora. © Digital Ottoman Corpora</em></figcaption></figure> <p>Transforming handwritten or printed historical documents into transcribed and digital format enables researchers to easily search, analyse, and extract valuable information from large volumes of historical texts. “Overall, I can say that the Digital Ottoman Corpora project is part of a larger effort to use computational methods to disseminate and analyse historical texts. By making these important resources more widely available, we hope to contribute to a deeper understanding of the past and its relevance to contemporary society.”, explains Suphan Kirmizialtin. When combined with textual analysis, transcribed and digitised texts can reveal patterns, trends, and discussions that enhance our understanding of historical events, social dynamics, and cultural phenomena.</p> <h2 class="wp-block-heading">Challenges of Ottoman Turkish Script</h2> <p>For researchers and scholars looking for valuable information within historical documents, Ottoman Turkish, a literary language written in Arabic script and deeply influenced by Arabic and Persian, is intriguing but intricate. In 1928, it was replaced by 'Modern Turkish', transforming the blended writing, vocabulary, and grammatical structures into a Latin script. This transition also replaced Persian and Arabic loan words with Turkish equivalents, shifting text directionality and writing systems.</p> <p>Original Ottoman Turkish documents were written and read from right-to-left, but transcriptions in Modern Turkish follow a left-to-right format. Beyond the directionality, the writing system switch poses a significant challenge for historians transcribing Ottoman Turkish. Compared to Modern Turkish, Ottoman Turkish's writing system is complex; word pronunciation isn't explicitly stated, but merely implied.</p> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/wIGnVyWf2zgWgeuA6cWGQCNw4cxGbLwWfjowp-vuFGrn-HpHfH1NqLdXwkOS3zSKXzic4XpUEQIQoJGaNyeK6dtsHiZVtBg7aPuMh_oo5z-SRG9E54czCA3zNQzeRZ8h-9kALgnAOWc2R8FdwdJEUQ" alt=""/><figcaption class="wp-element-caption"><em>Late 19th – early 20th century Ottoman Turkish periodicals. © Digital Ottoman Corpora</em></figcaption></figure> <p></p> <p>Because the transcription process relies more on educated decisions about how to solve this linguistic puzzle, Kirmizialtin describes transcribing Ottoman Turkish as more of an art form than an academic exercise. Faced with all these challenges that the language and therefore the historical documents present, the Digital Ottoman Corpora set out to transform research on Ottoman Turkish texts, and to make these types of materials more accessible and available to a wider audience.</p> <h2 class="wp-block-heading">Unlocking Ottoman Turkish Historical Archives</h2> <p>Without digital access, without a document catalog or structure, looking for valuable information within archives and documents almost becomes a Sisyphean task. Having spent most of the research time manually going&nbsp; through each document to identify relevant research material for her dissertation research, Suphan Kirmizialtin was just too familiar with how time-consuming and inefficient analysing and transcribing historical documents can be.</p> <figure class="wp-block-image"><img src="https://lh3.googleusercontent.com/mNDm3oQq_CbpZlmOsGGK1VHXou3dCncxi-SKcldju2Fmw9XGKhXAoBtd7bN3JamLCHha4iReosXqJpmsQ9XC7tgHpNxOspcBAsZyYygTGqfhL3Ti1pmUWHivO9KtAbRVKmhQmfltq_u0KFRuD_8hqg" alt=""/><figcaption class="wp-element-caption"><em>Recognised Layout of Ottoman Turkish periodicals. © Digital Ottoman Corpora</em></figcaption></figure> <p></p> <p>Curious to see how this process can be made more time-efficient, Kirmizialtin started looking for options on how to improve this. Discovering Transkribus, Kirmizialtin recognized that it not only has the potential to save time for individual researchers, but also offers the capacity to fulfill multiple needs, potentially transforming the way research is conducted in Ottoman studies. “Ottoman Turkish has not been represented in textual digital humanities in a meaningful way, mainly due to the lack of machine-readable corpora in this language. [...] we aim to produce searchable and computable historical text collections in Ottoman Turkish that can then be studied with textual analysis and data visualisation tools.”</p> <h2 class="wp-block-heading">Preparing the Material</h2> <p>Having considered the challenges, the Digital Ottoman Corpora team started preparing their work together with Transkribus’ AI-powered text recognition software in order to create machine-readable and accessible text. With the goal in mind to train and publish&nbsp; a text recognition model with Transkribus, the team hopes &nbsp; to “raise awareness of HTR in the Ottomanist community and encourage more scholars to participate in the digital corpora creation project for Ottoman Turkish”.</p> <p>The first step was to select and prepare the text material for the transcription process. The material chosen to train the automated text recognition model consisted of six Ottoman Turkish periodicals from the late 19th and early 20th centuries and an Ottoman Turkish dictionary. These journals covered topics ranging from politics to literature and culture, providing "valuable insights into the social and intellectual history of the late Ottoman Empire".</p> <p>As previously mentioned, a major challenge is the transition from the Arabic script used in Ottoman Turkish to the Latin script used in modern Turkish, along with the difference in directionality (right-to-left vs. left-to-right). To overcome this, the team defined a standardised transcription scheme, taking into account both the requirements of Transkribus’ transcription engine and human readability.</p> <h2 class="wp-block-heading">Transcribing the Documents</h2> <p>After choosing the most suitable transcription method for the project, the next step was to create training data.&nbsp; The creation of training data involves providing a large number of correctly transcribed pages as <a href="https://readcoop.eu/what-is-ground-truth/">ground truth</a>, which is used to train an automatic text recognition model that is capable of recognising the script. In this case, the generation of training data was a mixed approach, including manual transcription and the repurposing of existing Latinised texts through normalising spellings, correcting errors and standardising transcription schemes.</p> <figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/GHWRXlyeFArhTfBic0P3NvJdmcHxCUchZLCg7vjM-7K8HyiqfVyKeYRLfcLXGp7IJvYf52r689sSj1O8RUrl7kkR4XH8I3D8fmWszxGPHOQN076AIVZKuvuaaxjBK4he4BdpRHK8NcZyx42GR3EwTA" alt=""/><figcaption class="wp-element-caption"><em>Transcription of Ottoman Turkish periodicals with Transkribus. © Digital Ottoman Corpora</em></figcaption></figure> <p></p> <p>&nbsp;An additional obstacle that was unique to this project was the contrasting text orientation between Ottoman Turkish and modern Turkish. For this project of transforming Ottoman Turkish journals into transcribed and digital Modern Turkish Latin script text, the transcription software would need to support matching right-to-left images with left-to-right transcription. At first, Transkribus didn't support the alignment of right-to-left images with left-to-right transcription. However, Suphan Kirmizialtin was able to develop a workaround to reverse the direction of the transcribed text in Modern Turkish. By working closely with the Transkribus development team, another feature has been added to Transkribus that drastically reduced the time and effort required to train a text recognition model for Ottoman Turkish.</p> <p>“While we may not have arrived at the ultimate solution to all these complexities of Ottoman Turkish, I believe that adopting an algorithmically-compatible transcription scheme and consistently applying it in ground truth creation holds the key to successful HTR training in Transkribus.”<em>&nbsp;</em></p> <p>The effort of Suphan Kirmizialtin and the Digital Ottoman Corpora team has resulted in the first text recognition model for Ottoman Turkish with a CER on the validation set of 7.21%.</p> <p></p> <p class="has-medium-font-size"><strong>Try the model here:</strong> <a href="https://readcoop.eu/model/ottoman-turkish-print/">Ottoman Turkish Print </a></p> <h2 class="wp-block-heading">From Ottoman Turkish Periodicals to the First Public Model for Ottoman Turkish Print</h2> <p>Although the Digital Ottoman Corpora project faced some skepticism about working with automated transcription, as well as challenges due to the nature of the language they were working with, the team was able to publish the first automated text recognition model for Ottoman Turkish print. The collaboration with Transkribus also demonstrated the potential of HTR's software in making Ottoman Turkish texts and Ottoman cultural heritage accessible to a wider audience. With the success of this project, the Digital Ottoman Corpora team hopes to start a conversation within the community about important issues such as creating consistent guidelines for converting Ottoman Turkish texts from the original Arabic script to Latin script which are compatible with algorithms and automated systems.&nbsp;</p> <p>With this innovative project, Suphan Kirmizialtin and the Digital Ottoman Corpora team are at the forefront of the fascinating field of automated transcription of Ottoman Turkish printed texts. They are redefining the approach to accessing, analysing and interacting with Ottoman Turkish texts, paving the way for new discoveries and a deeper understanding of this rich cultural heritage.</p> <p>Thank you for talking to us Suphan, we will be following the development of the project with great interest! To find out more about the project, please visit the official <a href="https://www.digitalottomancorpora.org/">Digital Ottoman Corpora website.</a> </p> <h3 class="wp-block-heading">Suphan Kirmizialtin’s Transkribus Tips:</h3> <blockquote class="wp-block-quote"> <p>“Create detailed documentation and metadata for the HTR training process. I read somewhere that ‘metadata is a love note to the future’ and I completely agree!”</p> </blockquote> <blockquote class="wp-block-quote"> <p>“Avail yourself of the experience and wisdom of the awesome Transkribus user community.”</p> </blockquote>

From Script to Digital - Transforming Research for Ottoman Turkish Texts with Suphan Kirmizialtin

From Script to Digital – Transforming Research for Ottoman Turkish Texts with Suphan Kirmizialtin


<p>When Álvaro Cuéllar set out to transcribe a series of theatrical works from the Spanish Golden Age, he hoped he would find something interesting. But he did not expect to discover a completely new work by one of Spain’s most famous authors, Félix Lope de Vega y Carpio.</p>



<p>A prolific playwright, novelist, and poet, Lope de Vega was a leading figure in Spain’s Golden Age era. His plays include the renowned <a href="http://www.europeana.eu/portal/record/2022701/lod_oai_www_bibliotecavirtualmadrid_org_1380_ent1.html"><em>El acero de Madrid</em></a> (The Steel of Madrid), <a href="http://www.europeana.eu/portal/record/03486/urn_resolver_pl_urn_urn_nbn_de_bvb_12_bsb10608251_0.html"><em>El perro del Hortelano</em></a> (The Gardener&#8217;s Dog), and <a href="http://www.europeana.eu/portal/record/90901/EB7536FDFAFBEC697AE53215A5A3B7F407D0123E.html"><em>La viuda valenciana</em></a> (The widow from Valencia). The discovery by Cuéllar and his colleague, Germán Vega, adds a brand-new work to this list: <em>La francesa Laura</em> (The Frenchwoman Laura).</p>



<p>News of the find spread quickly. “Artificial intelligence attributes an anonymous work from the National Library&#8217;s manuscript collection to Lope de Vega,” reported <a href="https://elpais.com/cultura/2023-01-31/la-inteligencia-artificial-atribuye-a-lope-de-vega-una-obra-anonima-del-fondo-de-manuscritos-de-la-biblioteca-nacional.html">El Pais</a>, followed by similar articles from <a href="https://www.theguardian.com/world/2023/feb/05/artificial-intelligence-uncovers-lost-work-by-titan-of-spains-golden-age">The Guardian</a>, <a href="https://edition.cnn.com/style/article/ai-spanish-play-scli-intl/index.html">CNN</a>, and a host of other media outlets. Everyone wanted to know more about how it was possible to make such a discovery using just the power of AI.</p>


<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-798x1024.png" alt="" class="wp-image-32027" width="338" height="434" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-798x1024.png 798w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-234x300.png 234w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-768x985.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-9x12.png 9w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2-600x770.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image2.png 800w" sizes="(max-width: 338px) 100vw, 338px" /><figcaption><em>Lope de Vega was one of the most prolific playwrights of the Spanish Golden Age. © BNE</em></figcaption></figure></div>


<p>And they are about to find out. We sat down with&nbsp; Álvaro to discover how the team went about digitising such a vast collection of manuscripts and about finding a long-lost play by Lope de Vega.</p>



<h2 class="wp-block-heading">Authorship in the Spanish Golden Age: the ETSO project</h2>



<p>Meet Álvaro Cuéllar. In his position at the <a href="https://www.univie.ac.at/en/">University of Vienna</a>, Álvaro researches literature from Spain’s Golden Age: a period in the late 16th and 17th centuries known for its high artistic activity and achievement. However, it is also a period plagued with authorship problems. There are many manuscripts from the era buried in libraries and archives, yet to be attributed to a particular writer or poet.</p>



<figure class="wp-block-image size-full"><img decoding="async" width="720" height="480" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image6.png" alt="" class="wp-image-32029" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image6.png 720w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image6-300x200.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image6-18x12.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image6-600x400.png 600w" sizes="(max-width: 720px) 100vw, 720px" /><figcaption><em>The manuscript was part of an anonymous collection at the Spanish National Library. © BNE</em></figcaption></figure>



<p>Álvaro’s project, <a href="https://etso.es/">ETSO</a> — <em>Estilometría aplicada al Teatro del Siglo de Oro</em> (Stylometry applied to the Golden Age Theater) — aims to shed new light on these authorship problems. Together with colleague <a href="https://etso.es/contacto?sent_to=german_vega">Germán Vega García-Luengos</a> from the <a href="https://universityofvalladolid.uva.es/">University of Valladolid</a>, Álvaro analyses Golden Age theatrical manuscripts and compares the results against a corpus of works by dramatists from the era. “Our aim is to restore lost or misattributed works to the author. This includes canonical authors like Lope de Vega, but also the other 350 dramatists we have in our database.”</p>



<p>To do this, the team uses a method called stylometry. Stylometry analyses the different aspects of a writer’s particular style, for example how often they use certain words or how many clauses their sentences tend to have. Once a stylometry profile has been created for an author, other texts can be analysed to see how closely they fit that profile, and conclusions can then be made about who wrote the text. </p>



<figure class="wp-block-image size-full"><img decoding="async" width="948" height="589" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5.png" alt="" class="wp-image-32030" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5.png 948w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5-300x186.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5-768x477.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5-18x12.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image5-600x373.png 600w" sizes="(max-width: 948px) 100vw, 948px" /><figcaption><em>The character list from the play. © BNE</em></figcaption></figure>



<p>What makes the ETSO project different is that this whole process is carried out digitally. The team first creates digital versions of the prints and manuscripts with Transkribus, before using a second AI platform for the stylometric analysis and comparison. The success of this method could set a precedent for future projects of this kind.</p>



<h2 class="wp-block-heading">Transcribing the manuscripts</h2>



<p>The first step in the project was to transcribe the documents: over 1000 prints and 400 manuscripts in total. Many of these, including the Lope de Vega manuscript, came from the <a href="https://www.bne.es/en/about-us">Spanish National Library</a> in Madrid. “The Spanish National Library has dedicated a tremendous amount of resources to digitising its Spanish Golden Age theatre collections,” Álvaro explained. “When we approached the library, they had already digitised most of the thousands of pages we needed. The problem was that the documents were scanned, but not transcribed. That’s when we used Transkribus.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="621" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-1024x621.png" alt="" class="wp-image-32028" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-1024x621.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-300x182.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-768x466.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-18x12.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3-600x364.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image3.png 1304w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption><em>The team’s model both transcribed and modernised the handwritten text. © BNE</em></figcaption></figure>



<p>Because the collection contained both printed texts and handwritten manuscripts, two separate models needed to be created for the transcriptions. In reality, though, Álvaro ended up creating three. “Our first model was able to transcribe Spanish Golden Age prints with incredible success (1% CER). The problem was that we needed these texts with modernised orthography. So this first model was not useful for us, because it only transcribed the original spelling of the texts.”</p>



<p>After a bit more research into Transkribus, Álvaro found a solution. “I realised that I could train Transkribus not only to transcribe texts but also to modernise them at the same time. This seems problematic but because Transkribus works by using groups of letters instead of individual characters, the modernisation was quite successful.”</p>



<p>“Combining editions of the texts and the documents, I was able to train a model with 2 million words that could transcribe and modernise Spanish Golden Age prints (3% CER) and a model trained with 3 million words that was able to transcribe and modernise Spanish Golden Age manuscripts (9% CER).”&nbsp;</p>



<h3 class="wp-block-heading">You can find all three models on our website:</h3>



<p><a href="https://readcoop.eu/model/spanish-golden-age-prints-1-0/">Spanish Golden Age Prints 1.0</a></p>



<p><a href="https://readcoop.eu/model/spanish-golden-age-theatre-prints-spelling-modernization-1-0/">Spanish Golden Age Prints (Spelling Modernization) 1.0</a></p>



<p><a href="https://readcoop.eu/model/spanish-golden-age-theatre-prints-1-0/">Spanish Golden Age Manuscripts (Spelling Modernization) 1.0</a></p>



<h2 class="wp-block-heading">Analysing the stylometry</h2>



<p>But the transcription was only half the work. Álvaro and his team also had to analyse the stylometry of the 1400 documents, to see if any of them could be attributed to authors in the ETSO database.</p>



<p>To do this, they used a digital tool called <a href="https://cran.r-project.org/web/packages/stylo/index.html">Stylo</a>. “Stylo was developed by Maciej Eder, Jan Rybicki and Mike Kestemont and it is able to compare texts by their usage of words. This is extremely useful for our research and has been proven to be very effective. For example, it correctly classified 99% of the texts written by Lope de Vega in our last control experiments.”</p>



<p>Conveniently for the researchers, Stylo is almost as good at analysing automatic transcriptions as it is at analysing fully edited versions. “We found it extraordinary that the automatic transcriptions gave us approximately the same results as the perfectly edited texts. In the case of <em>La francesa Laura</em>, the relation with Lope de Vega was surprisingly strong, even with the automatic transcription.”</p>



<h2 class="wp-block-heading">An astonishing discovery</h2>



<p>Álvaro never set out to discover a play by such a famous author. But the moment he did is one he will never forget. “I was processing a bunch of manuscripts, as I do every day. Then one of these manuscripts, <em>La francesa Laura</em>, aligned unexpectedly with Lope de Vega in a very strong way. I sent a message to my colleague Germán Vega and I told him that we had something special, but that we had to be extremely cautious about it because it was an automatic transcription and we had to study the text carefully first.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="315" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-1024x315.png" alt="" class="wp-image-32031" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-1024x315.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-300x92.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-768x236.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-18x6.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7-600x184.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image7.png 1366w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption><em>The newly discovered play is entitles “La Francesca Laura”, or “The Frenchwoman Laura”. © BNE</em></figcaption></figure>



<p>Studying the text took two years of meticulous historical-philological analysis. “We read the text very closely and looked for parallel expressions and ideas between this text and other works by Lope de Vega and the other 350 dramatists we have in our database. We also proceeded with metrical approximations, orthology, rhythms, thematics, dating and so on. All of them gave us the same result: a crystal-clear correlation between this play and the repertoire of Lope de Vega.”</p>



<p>Certain that this was, in fact, a new work by Lope de Vega, the team finally shared their findings with the world. “We did not expect such a repercussion in the national and international news. Maybe the most rewarding has been that three theatrical companies have shown interest in representing the play, which would be extraordinary.”</p>



<h2 class="wp-block-heading">The project continues…</h2>



<p>Álvaro and his team have already had a pretty amazing result, but the project doesn’t stop there. “We have to keep going with the general objectives of the project: collecting all the works of Spanish Golden Age theatre and trying to shed light on authorship problems.”</p>



<p>Bear in mind, too, that it took two years between the technology first attributing <em>La francesa Laura </em>to Lope de Vega and the researchers being sure enough about the attribution to announce it to the world. “That means that in two years, you are going to see what we are working on right now, which is also quite exciting.”</p>



<p>Thanks for talking to us Álvaro, and we look forward to seeing what will happen next with the project.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="682" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-1024x682.png" alt="" class="wp-image-32032" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-1024x682.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-300x200.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-768x512.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-1536x1023.png 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-18x12.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4-600x400.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/image4.png 1600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption><em>Álvaro has already been invited to speak about the project at various conferences. © Pio Baruque Fotógrafos</em></figcaption></figure>


When Álvaro Cuéllar set out to transcribe a series of theatrical works from the Spanish Golden Age, he hoped he would find something interesting. But he did not expect to discover a completely new work by one of Spain’s most famous authors, Félix Lope de Vega y Carpio.

Discovering a lost play by Lope de Vega: Álvaro Cuéllar


<p>The <a href="https://www.zh.ch/de/direktion-der-justiz-und-des-innern/staatsarchiv.html">State Archives of Zurich</a> is the central archive for the Canton of Zurich, Switzerland. They are responsible for storing and preserving documents from councils, courts, administrations, and many other public bodies in the canton, giving a rich insight into the last 1150 years of Zurich’s history.&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="683" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-1024x683.jpg" alt="" class="wp-image-31071" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-1024x683.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-768x512.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-1536x1024.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-2048x1365.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_1-600x400.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption">The State Archives of Zurich is one of the largest archives in Switzerland. © Staatsarchiv Zürich</figcaption></figure>



<p></p>



<p>But naturally, you can only access these insights if you can access the documents themselves. And due to the sheer number of documents in the archive, this is not always easy to do. In the past, there was no way to quickly search whole premodern collections for relevant documents, meaning you would have to search through shelves and boxes, trying to find the papers you need.&nbsp;</p>



<p>To solve this problem and make it easier to access premodern documents, the archive is currently using <a href="https://readcoop.eu/transkribus">Transkribus</a> and <a href="https://readcoop.eu/readsearch/">read&amp;search</a> to create digital versions of their collections. We spoke to Christian Sieber, who leads the digitisation team at the Zurich archive, about their first Transkribus project and their experience with the software.</p>



<h2 class="wp-block-heading" id="h-50-000-pages-of-meeting-minutes-transcribed">50,000 pages of meeting minutes transcribed</h2>



<p>The State Archives’ history with Transkribus goes back many years—they were one of the partners in the original <a href="https://readcoop.eu/about/">READ research project</a>, which ran from 2016 to 2019. Back then, the Zurich archive mainly did manual transcription, which is much less time-efficient, and they immediately saw the benefit of automatic transcription. “In the past, we used to transcribe handwritten texts manually – and we know how much effort it takes,” he explained. “So it was clear to us early on that we would want to use Transkribus for our own projects in the future.”&nbsp;&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="683" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-1024x683.jpg" alt="" class="wp-image-31072" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-1024x683.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-768x512.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-1536x1024.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-2048x1365.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_4-600x400.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption">The archive is home to documents from the last thousand years of Swiss history. © Staatsarchiv Zürich</figcaption></figure>



<p></p>



<p>By 2019, the archive was ready to start their first large-scale Transkribus project. The documents chosen for this were meeting minutes from the Zurich Council in the 18th century, as they provide insights into almost every interaction between the city council and its inhabitants at that time. However, as this collection was over 50,000 pages, it was going to take some work. The first thing the team did was create an AI model. They manually transcribed 203,189 words and got down to an CER of just 4.80% on the Train Set. This created a solid foundation for transcribing the rest of the documents.&nbsp;</p>



<p>Like with any large-scale project, there were some hiccups along the way. “One of the challenges in the texts were marginalia, whose layout was not always correctly recognized,” Christian explained. “Another was the various writers who worked on the protocols. But with Transkribus, we were able to process over 50,000 pages of Zurich Council minutes in just three years, which would never have been possible with manual transcription.”</p>



<h2 class="wp-block-heading">Making the collection digitally available</h2>



<p>Of course, transcribing is only half the story. To make collections like these accessible to researchers and the public, the transcriptions need to be published in an online database—preferably one that is both reliable and easily searchable. For the State Archives of Zurich, the best option for this was to use Tranksribus’ sister platform: <a href="https://ratsmanuale-zuerich.transkribus.eu/">read&amp;search</a>.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="675" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-1024x675.jpg" alt="" class="wp-image-31074" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-1024x675.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-300x198.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-768x506.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-1536x1012.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-2048x1349.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/zurich-600x395.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>“It was important for us to have a standard solution, which is used by many other projects and will be continuously developed,” Christian said about their decision-making process. “With read&amp;search, READ-COOP offered a publishing solution that convinced us.”</p>



<p>As the archive had already done all their transcriptions with Transkribus, their digital collections could be simply uploaded <a href="https://ratsmanuale-zuerich.transkribus.eu/">to read&amp;search</a>, along with any metadata and tags that had been assigned during transcription. That means that, for example, if someone wants to find all council meeting records involving a certain council member, they can simply search for that person’s name and find the records they are mentioned in. </p>



<p>“In the past weeks, we have already received positive feedback from many researchers. Some have suggested to manually correct the texts with Citizen Science and thus further improve them.”</p>



<h2 class="wp-block-heading">The future looks bright</h2>



<p>This project is just the first of a few that the archive has planned. Starting next year, the team will begin digitising the Zurich Council minutes from the 15th to the 17th centuries, creating an online database of nearly 400 years of council documents. Over the next few years, the plan is to make all of the archive’s key documents freely accessible online, so that researchers and other interested parties can find the information they need as quickly and as easily as possible.</p>



<p>“By digitising our collections, we are creating an invaluable resource which meets the great demand from researchers from all over the world interested in the history of Zurich.”</p>



<h2 class="wp-block-heading">Christian’s Transkribus advice</h2>



<blockquote class="wp-block-quote">
<p>“Make sure you have a good overview of the texts you want to digitise. This makes it easier to plan everything to meet your requirements, and not waste time and money along the way. In short: the better you know your texts, the better you can plan the project.”</p>
</blockquote>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="683" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-1024x683.jpg" alt="" class="wp-image-31073" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-1024x683.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-768x512.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-1536x1024.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-2048x1365.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/11/Staatsarchiv_Zurich_5-600x400.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption">The documents at the archive provide a wealth of insights into the history of Zurich. © Staatsarchiv Zürich</figcaption></figure>


State Archives of Zurich is the central archive for the Canton of Zurich, Switzerland. They are responsible for storing and preserving documents from councils, courts, administrations, and many other public bodies in the canton, giving a rich insight into the last 1150 years of Zurich’s history.

How the State Archives of Zurich published 50,000 pages with read&search

How the State Archives of Zurich published 50,000 pages with read&#038;search


<p>While Transkribus may be our most well-known product, it is only part of the READ-COOP eco-system. In fact, we have a range of different technologies, each designed for a different stage in the digitisation process. One of these is the <a href="https://readcoop.eu/scantent/">ScanTent</a>: a portable scanning station that makes it possible to create high-quality images of historical documents using just a smartphone. These images can then be uploaded directly to Transkribus, ready to be converted into searchable digital text.</p>



<p>At the <a href="https://www.bnf.fr/fr">Bibliothèque nationale de France</a> (BnF) in Paris, visitors are now able to scan documents quickly and autonomously using one of the library’s 40 ScanTents. We wanted to know more about how the ScanTents work in practice, so we spoke to Laurent Perronnet, logistics manager in the Collections Department, to discover how the tents have changed scanning at the BnF for the better.</p>



<figure class="wp-block-image size-full"><img decoding="async" width="640" height="427" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/BnF_Bibliotheque_nationale_de_France_-_Salon_du_Livre_de_Paris_2015.jpg" alt="" class="wp-image-30681" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/BnF_Bibliotheque_nationale_de_France_-_Salon_du_Livre_de_Paris_2015.jpg 640w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/BnF_Bibliotheque_nationale_de_France_-_Salon_du_Livre_de_Paris_2015-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/BnF_Bibliotheque_nationale_de_France_-_Salon_du_Livre_de_Paris_2015-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/BnF_Bibliotheque_nationale_de_France_-_Salon_du_Livre_de_Paris_2015-600x400.jpg 600w" sizes="(max-width: 640px) 100vw, 640px" /><figcaption>The BnF is one of the oldest libraries in Europe. © ActuaLitté, distributed under a CC-BY 2.0 license</figcaption></figure>



<h2 class="wp-block-heading" id="h-scanning-facilities-at-the-bnf">Scanning facilities at the BnF</h2>



<p>The BnF is the national library of the French Republic, and it has existed in one form or another since the Middle Ages. As the national repository of France, it collects, archives and preserves everything that is published or edited in the country, as well as housing various historic and royal collections. In total, the institution looks after over 40 million documents, making it a mine of information for researchers in almost any field.</p>



<p>One service that is often needed by both researchers and other visitors to the library is scanning. Being able to take high-quality scans of documents and books allows people to use the information stored in them at home, without taking the actual publications out of the library. Scans can also be used for many other purposes, for example, to provide a visual reference of the original document, or for use with software such as<a href="/transkribus/"> Transkribus</a>.</p>



<p>“In the past, we had a contract with a service provider for scanning documents,” Laurent explained. But as that contract came to an end, the library wanted to explore other scanning options available to them, particularly as smartphone photography had developed so much in the meantime. The team looked into several innovative scanning solutions and got hold of some prototypes to try. “We tested three different products but the ScanTent was the best in terms of size, brightness, ease of use and quality of life over time.”</p>



<h2 class="wp-block-heading">First impressions of the ScanTent</h2>



<p>The ScanTent is basically a small white tent of approximately A3 size with a black base and built-in LED lighting. On the top of the tent is a non-slip mount with a viewing hole that looks down into the tent. To create an image, simply place your smartphone on top of the mount, with the lens positioned over the hole, and take a photo in the normal way. Users can also choose to use the <a href="http://docscan.transkribus.eu/">DocScan app</a>. This specially developed app recognises when a page has been turned and automatically takes a picture, allowing you to hold a document with two hands, and let your phone do the rest. </p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="768" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-1024x768.jpg" alt="" class="wp-image-30682" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-1024x768.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-300x225.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-768x576.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-1536x1152.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-2048x1536.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0910-600x450.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>The ScanTent has built-in LED lighting for optimal image quality. © Laurent Perronnet</figcaption></figure>



<p></p>



<p>When the BnF received their first ScanTents in 2020, the reaction was mixed. “There were some problems with the first model of the tents,” Laurent explained. “The fabric was fragile at the level of the feet and the power cable was too short. But then the product was improved and now it is very popular with our members.” The BnF now have 40 of the new and improved tents spread out across their various sites. These are used not only by visitors but by employees too—for example, the restoration workshops use the ScanTent to create before and after images of their restored works.</p>



<h2 class="wp-block-heading">A new way to scan</h2>



<p>The main advantage of scanning with the ScanTent is that it is more flexible and autonomous than with more conventional library scanners. Not only do you use your own smartphone but you can also move the ScanTent to wherever you need it, as opposed to having to take all the documents over to the scanner. “At the beginning, the members did not know the product and were a little reluctant about being so autonomous,” said Laurent about the first few weeks of using the ScanTent. But very quickly, people realised that this new way of scanning was actually a big benefit to their work. “Now we regularly get positive feedback from our members. They like that it is easy to use and also that it is free.”</p>



<p>“Overall, the ScanTent is a very good-quality product.”</p>



<p>Thank you, Laurent, for the interview.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="768" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-1024x768.jpg" alt="" class="wp-image-30683" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-1024x768.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-300x225.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-768x576.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-1536x1152.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-2048x1536.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/IMG_0882-600x450.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>There are 40 ScanTents spread throughout the BnF. © Laurent Perronnet</figcaption></figure>



<h2 class="wp-block-heading">Laurent’s advice for the ScanTent:</h2>



<blockquote class="wp-block-quote"><p>Remember to plug in the tent so that you have good lighting, and also to download the DocScan app. You can find instructions next to each ScanTent at the BnF.</p></blockquote>


While Transkribus may be our most well-known product, it is only part of the READ-COOP eco-system. In fact, we have a range of different technologies, each designed for a different stage in the digitisation process. One of these is the ScanTent.

How the ScanTent revolutionised scanning at the BnF


<p>The majority of Transkribus models are also trained to read just one language — after all, most historical documents are written in one language. But what if your document contains three (or even more) languages? Can you train a model to read all three at the same time?</p>



<p>The answer is yes, as Jeff Rusten and his team at Cornell University recently proved. Rusten’s research group have been working on a dictionary of all the words used by the ancient Greek comedic playwright, Aristophanes. The unpublished lexicon was put together in 1910 by Ernst Wüst, a German classicist, and is entirely in his handwriting. It is written in three different languages: ancient Greek, Latin, and Wüst’s native German.&nbsp;</p>



<p>As no public model currently exists for this language combination, Rusten’s team had to train their own multilingual model from scratch. We spoke to Rusten and his colleague, Ethan Della Rocca, about how they trained their multilingual model and used it to transcribe Wüst’s lexicon.</p>



<figure class="wp-block-image size-full"><img decoding="async" width="473" height="579" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114613.jpg" alt="" class="wp-image-32166" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114613.jpg 473w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114613-245x300.jpg 245w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114613-10x12.jpg 10w" sizes="(max-width: 473px) 100vw, 473px" /><figcaption>Wüst’s entire lexicon was written by hand. © K G Saur Verlag</figcaption></figure>



<p></p>



<h2 class="wp-block-heading" id="h-wust-s-lexicon-to-aristophanes">Wüst’s lexicon to Aristophanes</h2>



<p>Aristophanes’ comedies from the fifth century BC became famous not just for their satirical perspective on life in ancient Athens, but also for their unusual linguistic style. “Aristophanes&#8217; language is unique in classical Greek, combining high poetry and intellectual terminology with colloquialisms, terms from daily life, and even obscenities,” Rusten explained. “His lexical corpus was prized and studied already in antiquity.”</p>



<p>However, studying Aristophanes’ language in full requires having access to a comprehensive lexicon, detailing meanings, semantic relationships, and usage with key examples. “It was thought there was no adequate lexicon but in Cornell’s library we found a collection from 1984 of microfiche images of Ernst Wüst&#8217;s unpublished manuscript.” For the uninitiated, a microfiche is a small piece of photographic film containing images of pages of written material. Before digitisation, they were used to store materials, such as newspapers, without taking up too much physical space in an archive. In the 1980s, there was a push to use microfiches to ensure that manuscripts on decaying paper would be preserved forever.&nbsp;</p>



<p>One of those manuscripts was Wüst’s lexicon to Aristophanes from ca. 1910. “The publisher K G Saur decided to preserve the lexicon, without which it might have been lost forever, as the original manuscript has not been located to date. The publication had never been reviewed or even mentioned by any subsequent scholar, probably because accessing it was too daunting in the 1980s, not to mention reading it.”&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="456" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-1024x456.jpg" alt="" class="wp-image-32167" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-1024x456.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-300x134.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-768x342.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-18x8.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425-600x267.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114425.jpg 1225w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Wüst’s lexicon had been stored as microfiches since the 1980s. © K G Saur Verlag</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Creating a digital lexicon</h2>



<p>Of course, the best way to truly preserve Wüst’s lexicon is to create a digital version of it, to allow scholars to quickly search through the manuscript’s 1500 pages and also provide additional information about the individual entries. “A digital version can link each word with its basic meaning, word family, semantics and distribution in the author’s works, as well as a complete list of all occurrences, sorted by work and hyperlinked to the passage that is their context.”&nbsp;</p>



<p>Rusten’s team had already created an online platform, Lexeis.org, making it possible to publish digital versions of lexicons in this way. “We have already done this for Thucydides and Plato, and now Aristophanes. Adding Wüst’s digitized lexicon will greatly improve the platform.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="281" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-1024x281.jpg" alt="" class="wp-image-32168" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-1024x281.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-300x82.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-768x211.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-1536x422.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-18x5.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340-600x165.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114340.jpg 1919w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>The Lexeis.org platform shows both definitions and occurrences for words used by Aristophanes. © Lexeis.org</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Choosing a transcription platform</h2>



<p>This wasn’t the first time the team had used technology to try and speed up the transcription process. “We had already tried a modern Greek OCR package but ancient Greek has many more accents and other diacritics, which are difficult for OCR platforms to distinguish. We had also used Tesseract to transcribe Ast’s Plato lexicon (which is also in ancient Greek, Latin, and German) but we only had limited success.”</p>



<p>With Wüst’s lexicon, the team came across Transkribus quite by chance. “We discovered the platform while searching for information about the German &#8220;Kurrent&#8221; script. Then we attended a five-hour training workshop held at Yale by Sara Mansutti, which showed that Transkribus had many clear advantages over other methods.”</p>



<p>“Firstly, it deals with handwritten scripts as well as print. It also provides a computing platform for training transcription models for specific scripts and hands, based on neural networks. We liked that the methods are clearly explained and accessible to non-specialists, with videos of past workshops and instructional web pages. Finally, it makes past users’ models available for use in transcriptions and re-training by new users. This saves a lot of time when creating models.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="441" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-1024x441.jpg" alt="" class="wp-image-32169" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-1024x441.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-300x129.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-768x330.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-18x8.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702-600x258.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114702.jpg 1262w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>The team used Transkribus to create automatic transcriptions for the majority of the pages. © K G Saur Verlag</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Preparing the lexicon for transcription</h2>



<p>Once they had decided to use Transkribus, the next challenge for the team was to create digital scans of the 1500 microfiches of Wüst’s lexicon. “Today there are high-quality microfiche digital scanning readers available. With the help of Cornell Library equipment and staff, and after much trial and error, we were able to produce acceptable jpeg images (detailed enough to zoom in on, but still under Transkribus’ 10MB limit) of all 1500 pages in just over four weeks.”</p>



<p>Armed with their digital scans, the team then selected 50 of the pages to transcribe manually as Ground Truth. Della Rocca explained their strategy. “We aimed to include pages with different structures, including both short entries with much space between each entry and extended entries with little space between the lines. One of the main considerations was getting good coverage of the different letters when placed in the starting position of the line, as it makes identifying each article and its respective lemma much easier.”</p>



<p>“We also selected pages representative of the varying degrees of marginalia present in the text, although we excluded marginalia too faint to discern—maybe someday we will locate the original manuscript!”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="431" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-1024x431.jpg" alt="" class="wp-image-32170" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-1024x431.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-300x126.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-768x323.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-18x8.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515-600x252.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/Screenshot-2023-03-07-114515.jpg 1222w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>A wide variety of pages were chosen as training data. © K G Saur Verlag</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Training a multilingual model</h2>



<p>The manual transcription of the 50 selected pages was carried out partly by student workers at the university, who had skills in ancient Greek and Latin but not always in German. Rusten recalled: “The initial problem for us as Americans was the unfamiliarity with the German Kurrent script. We used the <a href="https://readcoop.eu/model/transkribus-german-handwriting-m1/">German Handwriting M1</a> model to produce an initial transcription of the pages we were working on. It did a good job with the German, but misrecognized the Latin and Greek as German also.”&nbsp;</p>



<p>“This meant we had to manually transcribe the parts that had been misrecognized. Using the images we loaded in Transkribus’ collaborative online workspace, our student workers had to locate and transcribe the script or language they knew best for each page, starting with Greek from scratch, then Latin also from scratch, and finally correcting the automatically transcribed German. Once a student had transcribed everything they could, they turned the page over to the next transcriber.”&nbsp;</p>



<p>“Because we had different people focus on different scripts, rather than force everyone to try to do every script, the transcription took five of us just six weeks of full-time work.”</p>



<p>Della Rocca supervised the final stage. “After several tries at optimizing the model, our lowest character error rate (CER) is 7.21%. This is approximately a 36% reduction from our initial model’s CER of 11.20%, and we are trying to improve it further by running more training and experimenting with different configurations. This has greatly reduced our post-transcription correction time per page.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="544" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-1024x544.jpg" alt="" class="wp-image-32171" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-1024x544.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-300x159.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-768x408.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-1536x816.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-18x10.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten-600x319.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/03/rusten.jpg 1785w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Jeff Rusten (second from right), Ethan Della Rocca (left) and the rest of the Lexeis.org team. © Jeff Rusten</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Publishing the lexicon online</h2>



<p>At the moment, the team is preparing the lexicon for publication on Lexeis.org. “The transcription still needs to be tagged in XML format for correction and analysis and linked to our XML text of Aristophanes. Then we can upload it to the platform. So the work is only just now becoming known but the support from granting agencies has been gratifying, and we look forward to publicizing it among scholars as soon as it is corrected and documented.”</p>



<p>The team are very happy with the way the project has turned out. “We only wish we had found Transkribus earlier! Henceforth we will use it for all of our OCR work, including print. We are currently re-transcribing a text which we had previously processed, to use the Transkribus model’s much more accurate Greek to correct our existing version.”</p>



<p>Della Rocca has some advice for other research groups wanting to train a multilingual model. “Stay calm if the initial training attempt fails — simply try other configurations. When first training your model, experiment with using different base models, as you’ll find that some may be more helpful than others. Then, once you have a working model, keep refining it with more transcribed and corrected pages to make it as accurate as possible.” We couldn’t agree more!</p>



<p>Thank you to Jeff and Ethan for talking to us about the project, and we wish you all the best with Lexeis.org.</p>



<p>For more about the project, check out Jeff Rusten’s talk about how they digitised Wüst’s lexicon: <a href="https://www.youtube.com/watch?v=kdNg-j7eKJk">https://www.youtube.com/watch?v=kdNg-j7eKJk</a></p>


Training a multilingual model in Transkribus: Jeff Rusten


<p>“I want to organise a Transkribus workshop but I don’t know where to start. Could you give me some advice?” This is a common question we get at READ-COOP. Learning to digitise historical documents can be a challenge for students, colleagues and volunteers in citizen science projects, and a workshop is a great way to help them get to grips with the process.</p>



<p>But organising a workshop is easier said than done. Which topics should be covered? What format should the workshop take? How interactive should it be?</p>



<p>To help answer these questions, we sat down with researcher Jan Odstrčilík from the Institute for Medieval Studies at the <a href="https://www.oeaw.ac.at/en/1/austrian-academy-of-sciences">Austrian Academy of Sciences</a>. In the winter semester of 2022, Jan brought together a group of colleagues from all over Europe and the US to organise the institute’s first HTR Winter School. This multi-day workshop was designed to give students all the skills they need to digitise historical documents, using both Transkribus and various other HTR tools.</p>



<p>In this post, Jan shares his recipe for organising a successful Transkribus workshop as well as some vital tips for organising a similar event.</p>



<h2 class="wp-block-heading" id="h-the-big-idea">The big idea</h2>



<p>Like many big ideas, the inspiration to organise an HTR Winter School came as a result of collaboration. Jan was already working with Helmut Reimitz from <a href="https://www.princeton.edu/">Princeton University</a> on a model for medieval scripts when the pair came across Tim Geelhaar from the <a href="https://www.uni-bielefeld.de/">University of Bielefeld</a>, who was creating a model for Carolingian minuscule with some pretty good results.&nbsp;</p>



<p>“We started to have discussions and to cooperate. We noticed interesting and different challenges for the Carolingian minuscule and the late medieval Latin, and realised how useful it could be to organise some type of a course.” With that, the idea for the winter school was born.&nbsp;</p>



<p>The team was then joined by highly experienced <a href="https://readcoop.eu/success-stories/tobias-hodel/">Tobias Hodel</a> from the <a href="https://www.unibe.ch/index_eng.html">University of Bern. </a>There were also co-organisers from the <a href="https://www.univie.ac.at/en/">University of Vienna</a> and the <a href="https://www.avcr.cz/en/">Czech Academy of Sciences</a> in Prague,<a href="https://www.unibe.ch/index_eng.html"> </a>making it possible to create a workshop with four different focus points: Carolingian minuscule, Late Medieval Latin, Medieval Germanand Medieval Czech (see <a href="https://www.oeaw.ac.at/en/imafo/read/discovering-the-power-of-ai-for-reading-medieval-manuscripts-htr-winter-school-2022">the list of all organizers</a>) .While the workshop was designed with PhD students in mind, the team decided to open it up to anybody who wanted to learn about handwritten text recognition of medieval manuscripts, resulting in quite a diverse range of participants.&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="485" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-1024x485.jpg" alt="" class="wp-image-31610" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-1024x485.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-300x142.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-768x363.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-1536x727.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-2048x969.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-18x9.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_125309-600x284.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>The participants of the HTR Winter School came from many countries and disciplines. © Jan Odstrčilík</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Choosing the format</h2>



<p>The next step was to decide how long the event should be. “Most workshops that I have experienced are strictly time-limited to a couple of hours and thus only allow a brief introduction,” Jan explained. “We wanted to offer a comprehensive experience: a mix where participants could not only learn the basics of Transkribus but where they could work on specific manuscripts, inspect them, discuss them, transcribe them, train models and then — at the end — make the results public.”&nbsp;</p>



<p>“But this requires time, so we decided to go for four virtual meetings every two to three weeks, which we concluded with a workshop in Vienna. This provided the participants enough time to absorb all the new information, as well as time to transcribe the necessary pages for training between individual meetings.”</p>



<p>The team decided to cover a diverse range of topics during the workshop, to give the participants comprehensive knowledge of HTR as a whole. “The virtual meetings focused closely on handwritten text recognition, including both theory and technical aspects. We followed that with the basics of the Transkribus program that were gradually deepened with new information (advanced layout, tags, training of the models, etc.) at each subsequent virtual meeting.”</p>



<p>There were several reasons behind the choice to use Transkribus as the main HTR platform for the workshop. “Firstly, Transkribus is currently the most accessible platform for independent researchers and small teams: anybody can use it regardless of the performance of their computer and without the need for any server. Moreover, the user community around Transkribus is very active and supportive, which was from our viewpoint a major factor.”&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="485" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-1024x485.jpg" alt="" class="wp-image-31609" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-1024x485.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-300x142.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-768x363.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-1536x727.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-2048x969.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-18x9.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221221_092018-600x284.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>A session at the Winter School in full swing. © Jan Odstrčilík</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">Learning through doing</h2>



<p>Right from the start, it was clear that the HTR Winter School should be as interactive as possible. The team wanted to give the students the chance to work on manuscripts in exactly the same way as they would in a real research environment.&nbsp;</p>



<p>“We wanted to give the students a common goal: Each of the four groups worked on selected manuscripts and had to collectively decide on the best methodology for the preparation of the models. We reserved part of every virtual meeting for group work. Then during the in-person workshop in Vienna,&nbsp; participants learnt how to use <a href="https://teipublisher.com/index.html">TEI-Publisher</a> to display editions and they published the Ground Truths using <a href="https://zenodo.org/">Zenodo</a> and <a href="https://htr-united.github.io/">HTR United</a>.”</p>



<p>Despite the focus on digital technology, it was important for the team that the participants also spent time with original manuscripts in physical form. “Although we mainly dealt with computer tools, the material world is still irreplaceable. That&#8217;s why we visited the <a href="https://www.onb.ac.at/en/">Austrian National Library</a> so that participants could see the manuscripts they had been working with until then only in a digital form. There is still something magical and deeply affecting in the ability to look and touch these objects. You could observe the immense excitement in the group.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="485" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-1024x485.jpg" alt="" class="wp-image-31608" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-1024x485.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-300x142.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-768x363.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-1536x727.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-2048x969.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-18x9.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2023/02/20221220_141134-600x284.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Viewing the manuscripts at the Austrian National Library. © Jan Odstrčilík</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">The end result</h2>



<p>But of course, the big question is: how did it all go? According to the students, very well. “Based on our survey after the workshop, the participants appreciated the hybrid format as well as the different teams and the diverse composition of participants. They also liked the connection between the theory and practice and the ability to see manuscripts in person.”</p>



<p>The hybrid format, mixing virtual sessions with an in-person workshop, was very popular with the organisational team, too. “This kind of format is demanding but rewarding. Both the participants and we as organisers learned a lot.”</p>



<p>For Jan though, one of the highlights came right at the end of the course, when the different groups presented the results of their work throughout the winter school. “Each of the teams chose a slightly different approach and it was great to see what everybody achieved.”</p>



<p>“Our experience from the workshop in Vienna is summarized by the video made by one of our participants, Giuseppe Paternicò.”</p>



<iframe src="https://drive.google.com/file/d/17HQSGCbeHga-UqLUPzsHVwkWV0VmDmlq/preview" width="640" height="480" allow="autoplay"></iframe>



<p>© Giuseppe Paternico</p>



<p>Jan’s tips for organising your own Transkribus workshop</p>



<p>Despite the success of the event, there was something Jan and his team would do differently next time. “Even with a lot of time in the schedule, we still ran short of it. This was especially apparent in the in-person workshop where we wanted students to learn how to publish their transcriptions and Ground Truths. We succeeded thanks to Tobias Hodel, who took care of this part, but it was a little bit in a hurry. Those things are simply very time-consuming and so it is necessary to reserve enough time for this last step.”</p>



<p>Jan also had one final tip for anyone organising a workshop involving a lot of digital tools. “It was great and necessary to have a dedicated person (in our case, Leon Pürstinger) who took care of all the digital stuff, like Zoom and the recordings.”</p>



<p>Thanks, Jan, for talking to us and congratulations on the successful event!</p>


How to organise a successful Transkribus workshop: Jan Odstrčilík

<p>The largest, and richest, collection of documents kept inside the Amsterdam City Archives are no doubt the Notarial Archives, starting from 1578. It spans over 30.000 volumes and after 8,6 millions of scans, digitization is only halfway. Handwritten Text Recognition would be revolutionary in unveiling its content. But would it prove to be useful for such a huge and varied collection, containing documents in all formats, spanning over several centuries, sometimes damaged by fire or water, and, not in the least, written by hundreds of different hands? Transkribus gave us the option to grow and experiment.&nbsp;</p>



<p>The citizen-science project <a href="https://readcoop.eu/crowdsourcing-transkribus-amsterdam/">Crowd Leert Computer Lezen</a> on the platform &#8216;VeleHanden&#8217; combined the powers of the crowd with the HTR tooling of Transkribus. The volunteers provided 10.000 scans of Ground Truth transcriptions in a very short time. We were then able to train notary-specific models (works for selected sets), we combined data in general models (very satisfactory for 18th c. documents) and also combined them with existing basemodels (even better!).</p>



<p>The next step was processing HTR in large quantities and make them available to the public.&nbsp;The first few hundreds of thousands of pages are now searchable thanks to the <a href="https://transkribus.eu/r/amsterdam-city-archives/#/">Transkribus Read&amp;Search interface</a>. Even if the HTR is far from perfect, the advanced search options for fuzzy search with high tolerance reveals many treasures.&nbsp;</p>



<p>As Amsterdam was (and is) an international hub, the HTR results are of interest to scholars worldwide. Within hours, they discovered new details of a mixed and growing population in long-gone neighbourhoods, proof of new global connections, the provenance of numerous european paintings, and even a Surinam-Indian chief was found to be inside an Amsterdam notary office in 1682. </p>



<p>But the most high-profile story to come out of these archives concerns a painting by the famous Dutch artist, Rembrandt van Rijn. For years, scholars have been trying to identify the subject of “Portrait of a Man with Arms Akimbo”, a painting of an unknown gentleman previously thought to be a Dutch sailor or admiral. However, documents in the Notarial Archives suggest a rather different story. A document from 1663 about the settling of the estate of Amsterdam craftsman Jacob Wesselsz Wiltingh states that “Rembrandt the painter” was paid a sum of money for “painting the deceased”. No further details about this painting or what it looks like are given. But since the subjects of most Rembrandt paintings are accounted for, many leading scholars now believe that Wiltingh is indeed the “Man with Arms Akimbo”, and the dress and time period would certainly fit. <br>If you want to find out more about this mysterious Rembrandt painting, you can read the full story <a href="https://www.rembrandthuis.nl/nl/kroniek/rembrandt-paints-master-carpenter-jacob-wesselsz-wiltingh-an-unknown-rembrandt-from-the-archive-of-the-amsterdam-notaries/">here</a> or visit <a href="https://alleamsterdamseakten.nl/">Alle Amsterdamse Akten</a> for more historical stories from this magnificent city.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="979" height="1024" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--979x1024.jpg" alt="" class="wp-image-21707" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--979x1024.jpg 979w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--287x300.jpg 287w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--768x803.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--1469x1536.jpg 1469w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--1959x2048.jpg 1959w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--11x12.jpg 11w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--600x627.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--57x60.jpg 57w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Schriftausschnitt--86x90.jpg 86w" sizes="(max-width: 979px) 100vw, 979px" /></figure>


<script type="text/javascript" id="hs-script-loader" async defer src="//js-eu1.hs-scripts.com/25701986.js"></script>

The largest, and richest, collection of documents kept inside the Amsterdam City Archives are no doubt the Notarial Archives, starting from 1578. It spans over 30.000 volumes and after 8,6 millions of scans, digitization is only halfway.

Amsterdam Notary Archives out of the dark


<p>Some Transkribus projects are just a few pages long. Many are a few hundred or thousand pages long. But the latest Transkribus project at the <a href="https://www.nationaalarchief.nl/en">National Archives of the Netherlands</a> involved a whopping 3 million pages of documents. And this is just the beginning. Over the next few years, the Dutch archive aims to scan about 10% of its entire collection—that’s more than 10 million scans a year—and transcribe at least a part of the collection to make it more accessible.</p>



<p>We spoke to Liesbeth Keijser, Project Manager for Digitisation at the National Archives of the Netherlands, to discover more about digitising such large collections of documents with Transkribus.</p>




<h2 class="wp-block-heading">Welcome to the National Archives of the Netherlands</h2>



<p>Based in the Dutch coastal city of The Hague, the National Archives of the Netherlands is the country’s largest archive. It is home to hundreds of years of governmental and official documents, as well as private documents relevant to the history of the Netherlands. Millions of pages are looked after at the archive. In fact, the collection is so large that if you were to line it all up in a row, it would stretch for over 140km!</p>




<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="683" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-1024x683.jpg" alt="" class="wp-image-30624" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-1024x683.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-768x512.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-1536x1024.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-2048x1365.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/tdf-20211005_121-600x400.jpg 600w" sizes="(max-width: 1024px) 100vw, 1024px" />
<figcaption><i>The National Archives of the Netherlands is the country’s largest archive © Tineke Dijkstra</i></figcaption></figure>



<p></p>



<p>However, most of the archive’s documents are still paper-based, which makes accessing them difficult in two ways. Firstly, you have to travel to The Hague to browse the archive. Secondly, and probably more importantly, there is no way to quickly search whole collections for specific information. Instead of simply typing a search term into a database, you have to manually search through collections of papers, which is infinitely more time-consuming.</p>



<p>With that in mind, the National Archives embarked on an ambitious digitisation strategy. “Our plan is to scan 10% of our archives over the next 15 years,” digitisation manager Liesbeth explained. “That will add up to more than 100 million scans in a couple of years.”&nbsp;</p>



<p>To make the scans more accessible, the archive is using handwritten text recognition technology to automatically transcribe the handwritten text and convert it into a digital text file. They decided to start with a collection of 3 million pages, mainly records regarding the Dutch East India Company in the 17th and 18th centuries and notarial deeds from the 19th century. This first project would set the groundwork for later parts of the digitisation strategy.</p>




<h2 class="wp-block-heading" id="h-creating-an-ai-model-with-transkribus">Creating an AI model with Transkribus</h2>



<p>The National Archives started working with handwriting recognition technology about five years ago and the team have been pleasantly surprised by how easy it is. “Using Transkribus and creating a custom AI model was actually quite straightforward,” Liesbeth said. At the start, we were aiming for a CER [character error rate] of 20%, we would have been happy with that. But after creating 6000 pages of training data, we got down to a CER of 7%, which was even better for us.”</p>



<p>In keeping with Transkribus’ cooperative values, Liesbeth’s team also decided to make their AI model public, so that other people can benefit from their work. Their model, <a href="https://readcoop.eu/model/dutch-handwriting-17th-19th-century/">Dutch Handwriting 17th-19th century</a>, now contains almost 1.5 million words and can be used by any Transkribus user working with similar documents.</p>



<a href="https://www.transkribus.org/event/webinar-voor-beginners-januari-2025"><img src="https://blog.transkribus.org/hs-fs/hubfs/Webinar%20Banner.png?width=1536&height=540&name=Webinar%20Banner.png" alt="Webinar voor beginners in het Nederlands" style="max-width: 100%; height: auto;"></a>




<h2 class="wp-block-heading">Publishing the transcriptions</h2>



<p>For Liesbeth and her team, the transcription was actually the less complicated step of the project. “Transcribing everything was the easy part,” she explained. “Publishing everything online was a lot more complex, both from an archival and a technical perspective.” Deciding how to organise everything into a logical online format was one challenge, finding people with the right development skills to build exactly what the archive needed was yet another.</p>




<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="630" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-1024x630.png" alt="" class="wp-image-30625" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-1024x630.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-300x185.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-768x473.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-18x12.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7-600x369.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Knipsel-7.png 1472w" sizes="(max-width: 1024px) 100vw, 1024px" />
<figcaption><i>Over 3 million pages were automatically transcribed during the project, which ran from 2020 to 2021. © Zoeken in transcripties</i></figcaption></figure>



<p><p><p>After considering different solutions, the team decided to build a custom system divided into a back end and a separate front end by two suppliers. The result was the “<a href="https://zoekintranscripties.nl/">Zoeken in transcripties</a>” platform. Although the project is still ongoing, the platform already provides access to a wealth of documents, making it much easier for researchers and interested persons to find the information they need. The team also added named entity recognition to the system, so that it would automatically enrich the transcriptions with named entities such as people and places.</p>



<p>“Ideally, we would have a platform that integrates seamlessly with our existing IT infrastructure. That isn’t quite possible yet, but we are still pretty happy with the results so far.”</p>




<h2 class="wp-block-heading">The benefits of digitisation</h2>



<p>And it is not just Liesbeth’s team who is happy with the new digitised collection. “We’re still collating exact data about user satisfaction, but our impression is that people like the new system.”</p>



<p>“A good example of this was the bittersweet feedback we got from some academic researchers. They really liked that so many documents were suddenly so easily accessible. But because they suddenly had so many new sources to work with, they realised they had to scrap their current conclusions and start again. I think this shows just how much impact a digitisation project like this can have on academic research.”</p>



<p>Thank you, Liesbeth, for talking to us!</p>



<p><h3>Liesbeth’s Transkribus Tip:</h3></p>



<blockquote class="wp-block-quote"><p>“When embarking on a project like this, make sure there is someone in your team who has a background in AI. It is hard to compare different technologies if you don’t understand the differences between them, so make sure the team has that knowledge before you start.”</p></blockquote>



<p>Cover image: © Anne Reitsma Fotografie</p>


Some Transkribus projects are just a few pages long. Many are a few hundred or thousand pages long. But the latest Transkribus project at the National Archives of the Netherlands involved a whopping 3 million pages of documents.

Transcribing 3 million scans at the National Archives of the Netherlands


<p>If you’re a researcher in Austria, you may have heard of the prestigious <a href="https://www.fwf.ac.at/en/research-funding/fwf-programmes/start-programme">START Grant</a>. Run by the Austrian Science Fund or FWF, the grant is awarded each year to a handful of post-doctoral researchers carrying out groundbreaking projects in their field. In 2022, one of those researchers was William Barton, a classical philologist currently based at the <a href="https://neolatin.lbg.ac.at/institut/the-ludwig-boltzmann-institute-for-neo-latin-studies/?lang=en">Ludwig Boltzmann Institute for Neo-Latin Studies</a> in Innsbruck. William will carry out the project at the University of Innsbruck’s Institute for <a href="https://www.uibk.ac.at/latinistik-graezistik/">Classical Philology and Neo-Latin Studies</a>.</p>



<p>William will use the funding to transcribe, translate and study the secret diary of Karl Benedikt Hase: a lively and rather infamous professor of Greek philology in the 19th century. By creating a digital version of the diary with Transkribus, William and his team hope to discover new information about both Hase’s life and the lives of other Philhellenes in the early 1800s. We sat down with William to find out more about this award-winning project and learn about his experience with Transkribus.</p>


<div class="wp-block-image">
<figure class="aligncenter size-full is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Karl_Benedikt_Hase_IZ_43-1864_S_272.jpg" alt="" class="wp-image-29655" width="327" height="399" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Karl_Benedikt_Hase_IZ_43-1864_S_272.jpg 575w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Karl_Benedikt_Hase_IZ_43-1864_S_272-246x300.jpg 246w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Karl_Benedikt_Hase_IZ_43-1864_S_272-10x12.jpg 10w" sizes="(max-width: 327px) 100vw, 327px" /><figcaption>Karl Benedikt Hase was a key figure in Paris’s philhellenic scene</figcaption></figure></div>


<h2 class="wp-block-heading" id="h-the-philhellenic-scene-of-the-1800s">The philhellenic scene of the 1800s</h2>



<p>Ancient Greek culture might have peaked over 2000 years ago but it has fascinated people ever since. From science to art to architecture, the ideas of Ancient Greece have always had a large influence on modern Europe, and this was particularly true at the beginning of the 19th century.</p>



<p>“Greece was on everyone’s mind in the early 1800s,” William explained. “The Greek revolution was happening and with it came a renewed fascination for the country and its history, a movement known as ‘philhellenism’, or ‘love for Greek culture’.”&nbsp;</p>



<p>One of the most prominent Philhellenes at that time was Karl Benedikt Hase. Born in Germany, Hase was a professor of Greek who spent much of his life and career in Paris, France. Hase was famous for his knowledge of the Greek language and culture, as well as his place at the centre of Paris’ philhellenic movement. But he was infamous for something else: forging Greek manuscripts. In one well-known example, a Russian statesman wrote to Hase asking for historical Greek documents about Crimea, which Hase didn’t have. “So he simply produced something he thought the Russian would be interested in,” William explained. “And he happily took the money for it, too.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="834" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-1024x834.png" alt="" class="wp-image-30593" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-1024x834.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-300x244.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-768x626.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-15x12.png 15w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837-600x489.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/TB_Hase_1837.png 1316w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>An excerpt from Hase&#8217;s diary, which is preserved at the <a href="https://www.klassik-stiftung.de/goethe-und-schiller-archiv/">Goethe und Schiller Archive</a>, Weimar</figcaption></figure>



<h2 class="wp-block-heading">The search for Hase’s diary</h2>



<p>Researchers like William have known for a long time that Hase kept a secret diary. “Byzantine researchers in the 1970s were fascinated by this topic of Hase and his forgeries. And they knew he had a diary, but they couldn’t find it,” William explained. “So I thought I would give myself a chance at finding it instead.”&nbsp;</p>



<p>Luckily for William, the researchers had left a good trail of clues. “They had described where they had already looked and it became clear that the diary was probably in one of three places: Paris, Lyon, or in the area of Weimar/Jena, in Germany.” William set about trawling through the indexes of archives in all three cities, to see if there was anything that matched the description of Hase’s diary. “In Weimar, there was a document listed as ‘notebooks in Greek by Hase’. So I went to the archive to check and it was indeed Hase’s secret diary.” One of the biggest mysteries of philhellenism in the 19th century had just been solved.</p>



<h2 class="wp-block-heading">“Transkribus is a huge benefit for us”</h2>



<p>So why is this such a breakthrough? “There’s a lot we can learn from Hase’s diary,” William explained. “We can learn more about Paris’ large philhellenic network and what Hase thought about the other Philhellenes he knew. What’s more, the diary should tell us about Hase’s private life and his famous forgeries.”</p>



<p>But before any discoveries can be made, Hase’s Ancient Greek diary needs to be transcribed and digitised, which is no small undertaking. “The diary is a handwritten document and it’s enormous. In total, there are about 2,500 pages to transcribe, all written in Hase’s heavily abbreviated Greek&nbsp; hand.” Living and working in Innsbruck, William was familiar with Transkribus and knew the advantages the software could bring to the project. “Transkribus’ unique ability to be trained for this kind of very complex manuscript work is a huge benefit for us.”</p>



<p>“Our first model was based on just 10 pages, and already the software was able to make some sensible transcriptions. Now after 70 pages, it’s really getting there. I no longer sit there typing every word out, I just run the text through Transkribus, edit it and set this as the new Ground Truth. It’s even learnt to transcribe all the diacritics — Ancient Greek’s set of accents and linguistic markers — which is a lot faster than manually typing them in.”</p>



<h2 class="wp-block-heading">Publishing the manuscripts online</h2>



<p>Another big benefit for William and his team is the tagging feature. “The ability to tag certain information inside Transkribus, like a person or a place, is really helpful. For example, one of the things we want to do is to cross reference the people Hase mentions in the diary with other historical sources from the time. If all the names in the diary are tagged, then we can simply search for a particular name and instantly see all the diary entries mentioning that person. And that saves us a lot of time.”</p>



<p>The final step of the project will be to publish the diary online using the <a href="https://readcoop.eu/readsearch">Read and Search</a> platform so that people around the world can access Hase’s diary. And this kind of collaboration can only benefit a project of this scale. “I’m an expert in Ancient Greek and its modern uses,” William explained. “My chief contribution to this project is to read and transcribe the text, then to offer some preliminary studies. But publishing everything online means that experts in other fields can take advantage of the work I’ve done and take the research one step further. Or that’s my hope, anyway!”</p>



<h2 class="wp-block-heading">William’s Transkribus Tip:</h2>



<blockquote class="wp-block-quote"><p>When you train a model in Transkribus, you have to prepare a series of Ground Truth texts. Normally, this involves transcribing some of your handwritten texts manually. However, we saved time by finding original handwritten manuscripts by Hase which had later been published, giving us a handwritten and a (very accurate) printed version of the same text. This meant we were able to feed Transkribus with a lot of very sure Ground Truth from the start, saving us time and effort.&nbsp;</p></blockquote>


<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-1024x682.jpeg" alt="" class="wp-image-29653" width="523" height="348" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-1024x682.jpeg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-300x200.jpeg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-768x512.jpeg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-1536x1024.jpeg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-2048x1365.jpeg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-18x12.jpeg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/fwf-barton_01-scaled-1-600x400.jpeg 600w" sizes="(max-width: 523px) 100vw, 523px" /><figcaption>William Barton works at the Ludwig Boltzmann Institute for Neo-Latin Studies in Innsbruck © Ludwig Boltzmann Institute</figcaption></figure></div>

If you’re a researcher in Austria, you may have heard of the prestigious START Grant. Run by the Austrian Science Fund or FWF, the grant is awarded each year to a handful of post-doctoral researchers carrying out groundbreaking projects in their field. In 2022, one of those researchers was William Barton, a classical philologist currently based at the Ludwig Boltzmann Institute for Neo-Latin Studies in Innsbruck.

Uncovering the secret life of Karl Benedikt Hase with START Grant winner William Barton


<p>Meet Henning Schuler. Henning is a historian specialising in modern intellectual and social history, who has just completed his PhD at the <a href="https://www.eui.eu/en/home">European University Institute</a> in Florence, Italy.&nbsp;</p>



<p>When Henning first considered using Transkribus for his dissertation, some of his historian colleagues were sceptical. How could software ever read texts as well as a person? But he decided to give Transkribus a go, and now a few years down the line, he’s very glad that he did.</p>



<p>After receiving a <a href="https://readcoop.eu/transkribus/scholarship/">Transkribus Scholarship</a>, we spoke to Henning to find out more and discover how Transkribus makes life easier for historians.</p>



<h2 class="wp-block-heading" id="h-the-challenge-of-transcribing-handwritten-documents">The challenge of transcribing handwritten documents</h2>



<p>The eighteenth century was a time of change. The world’s trade networks were rapidly growing and increasing numbers of European merchants were travelling to new and exotic locations to buy and sell products. It is these merchants that Henning Schuler decided to research for his PhD. “In my dissertation, I studied British and French merchants who lived in Aleppo at the end of the eighteenth century. I wanted to shed some new light on the way in which Europeans conceptualised themselves and the seemingly expanding world they inhabited in the eighteenth century,” Henning explained. “And luckily, I had quite a lot of handwritten sources for my project.”</p>



<p>But while having a large corpus is good in terms of quality of research, it does present some logistical problems. “I had so many different sources that it was completely impossible to transcribe even meaningful amounts of them myself.” So Henning decided to see if there were any software solutions that could help speed up the process.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="815" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-1024x815.png" alt="" class="wp-image-29644" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-1024x815.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-300x239.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-768x612.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-15x12.png 15w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English-600x478.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-English.png 1247w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Example of English material Henning was dealing with</figcaption></figure>



<p></p>



<h2 class="wp-block-heading">First impressions of Transkribus</h2>



<p>Very soon, he came across Transkribus. “I started to use Transkribus out of curiosity, to see whether it worked, how good it was, and whether it might be potentially useful,” Henning said. His first impression was not overly positive. “Initially I found the transcription engine not to be sufficient for my sources”. But he then discovered the benefits of training your own AI model. “Transkribus offers the function to train a model specifically for your sources. Once I had done this, the accuracy was much better and I ended up using it on the majority of my sources.”</p>



<p>Of course, there are some things Transkribus can’t do. “As a historian, I never regard a transcription as an alternative to reading the actual sources. A transcription doesn’t show you the condition of the text, alterations, errors made by the writer etc. But as an additional tool, Transkribus was great. It allowed me to quickly search for something that I had forgotten, do some quick analyses of word frequency or copy a long quote instead of transcribing it by hand.”</p>



<p>“In a few cases, when I was struggling with the reading of a word, Transkribus even helped me find the correct reading. As a friend working in image recognition told me: oftentimes, things that are difficult for humans are easy for computers, and vice-versa.”</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="790" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-1024x790.png" alt="" class="wp-image-29645" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-1024x790.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-300x232.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-768x593.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-16x12.png 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French-600x463.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Example-French.png 1332w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Example of French Henning is researching </figcaption></figure>



<h2 class="wp-block-heading">Why Transkribus is good for historians</h2>



<p>Henning has now successfully completed his dissertation and is looking to publish his results. But despite his success with Transkribus, he experienced a lot of scepticism from his colleagues. “I have to say that I was surprised by the incredulity I encountered in many other historians. When I talked to colleagues about it, they did not believe that a computer could easily read texts that they themselves needed a long time to decipher.”</p>



<p>“But I would encourage all historians, particularly if you are struggling with transcription, to use Transkribus. After all, there should be no hesitation or shame in using digital tools to make our lives as historians easier.”</p>



<h2 class="wp-block-heading">Henning’s Transkribus Tip:</h2>



<blockquote class="wp-block-quote"><p>Transkribus takes some getting used to. But for larger bodies of manuscript sources, it is well worth taking the time to prepare your sources and train your own AI model. This helped me to really get the most out of the software.</p></blockquote>



<figure class="wp-block-image size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-768x1024.jpg" alt="" class="wp-image-29646" width="259" height="345" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-768x1024.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-225x300.jpg 225w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-1152x1536.jpg 1152w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-1536x2048.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-9x12.jpg 9w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-600x800.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/08/Photo-Henning-scaled.jpg 1920w" sizes="(max-width: 259px) 100vw, 259px" /><figcaption>Henning Schuler</figcaption></figure>


Meet Henning Schuler. Henning is a historian specialising in modern intellectual and social history, who has just completed his PhD at the European University Institute in Florence, Italy.

“There should be no shame in using digital tools to make our lives easier”: Henning Schuler


<p>Languages evolve over time: even first-year linguistics students could tell you that. But understanding how they evolve is slighty more challenging.&nbsp;&nbsp;</p>



<p>At the <a href="https://ijp.pan.pl/en">Institute of Polish Language &#8211; Polish Academy of Sciences</a>, a team of researchers is trying to gather insights into the evolution of the Polish language. They are creating a digital linguistic corpus of Polish texts from the 17th and 18th centuries, to make it easier to analayse the language used at this time. And as the corpus is designed to be entirely digital, they decided to transcribe texts using Transkribus.</p>



<p>We spoke to team member Dr Ewa Rodek to find out more about this exciting project in the field of Polish linguistics.</p>



<h2 class="wp-block-heading" id="h-a-unique-linguistic-corpus">A Unique Linguistic Corpus</h2>



<p>Ewa Rodek has always been a fan of languages and their evolution: “I really like the history of language, especially historical lexicography and literary culture.” Luckily for Ewa, she was able to turn her passion into a career at the Institute of Polish Language &#8211; Polish Academy of Sciences. Her team is currently working on a digital corpus of Polish texts from the 17th and 18th centuries, the so-called <a href="https://www.korba.edu.pl/query_corpus/?lang=en">KorBa project</a>. This unique corpus contains Polish texts of many different genres and styles from the Baroque and Enlightenment eras. Once completed, it will contain over 25 million tokens tagged according to their structure and morphology, making it the largest diachronic corpus of this type.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="576" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-1024x576.jpg" alt="" class="wp-image-29464" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-1024x576.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-300x169.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-768x432.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-1536x864.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-18x10.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts-600x338.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/06/01_1_Texts.jpg 1920w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>An impression of the variety of the material Ewa and her Team are dealing with</figcaption></figure>



<p></p>



<p>“KorBa is the first diachronic corpus of this size in Polish,” Ewa told us. “Its rich lexis makes it important for the scientific community. But its also important for our team because it is the main material base for the creation of the <a href="http://www.sxvii.edu.pl">eSXVII</a>.” The eSXVII is short for the Electronic Dictionary of the 17th- and 18th-century Polish, which the same team has been working on since 2004. “The corpus is integrated with eSXVII in such a way that the user can easily switch from an entry in the dictionary, to a specific search in the corpus, where the user can see how many times the word has been used and in what context.” In short, this project will give researchers much more information about the historical usage of individual words than they previously had.</p>



<h2 class="wp-block-heading">Choosing Transkribus For Transcription</h2>



<p>Historical texts like the ones in this project are not always easy to transcribe. “Our material is not exactly homogenous,” Ewa explained. “The manuscripts often have several different typefaces and even different languages on the same page — Polish coexists with Latin, German or French. Some of the documents also have stains or are damaged.” That’s why, at the start of the project, Ewa’s team chose to transcribe the documents manually. But that wasn’t as successful as they had hoped. “I have been involved in the workflow from the beginning, so I know the problems you encounter using manual transcribers, especially regarding delays. The transcription and proofreading of documents in Gothic script also requires specialist knowledge, which makes it quite expensive.”&nbsp;</p>



<p>The team had already ruled out using OCR technology — “I had a lot of experience with OCR software and I knew that it would not improve our work” — but then Ewa discovered the HTR possibilities of Transkribus. The software would not only make things quicker, but the team were able to use it even though they had no experience in coding or software. “It was particularly important that we could manage the work ourselves and didn’t have to ask our IT colleagues for help. Using Transkribus was also much cheaper than hiring transcribers.”&nbsp;</p>



<h2 class="wp-block-heading">Transcribing The Documents</h2>



<p>And thankfully, Ewa did not regret her decision to use Transkribus. The team started by creating their own AI models, and they achieved some pretty impressive character error rates. “The biggest advantage of Transkribus is that it learns very quickly. We developed two models &#8211; one for printed texts (with a CER of 0.29%), the other for manuscripts (with a CER of 1.8%),” Ewa explained. And this was despite the various fonts, languages and conditions of the documents: “Transkribus offset these difficulties and handled them very well.”</p>



<p>The fact Transkribus can be used easily by a whole team was also a plus for this large project. “The ability to work in teams, administer models and document collections is very helpful,” Ewa said. “Exporting a pdf file from an image with an underlying text layer is extremely powerful as well. I will definitely use this option in my future projects.”</p>



<h2 class="wp-block-heading">The Next Steps</h2>



<p>Ewa’s team might have completed the transcription of the texts, but the KorBa project isn’t quite finished yet. “We have now finished the longest part of the process &#8211; that is collecting and transcribing the texts. Now we are preparing the training material for the tagger software and then we have to combine the results of the first and second editions of the project.” It sounds like the team are going to be busy for a while yet!</p>



<p>Thank you to Ewa Rodek and her team from the KorBa project for talking to us!</p>



<h2 class="wp-block-heading">Ewa’s Trankribus Tip</h2>



<blockquote class="wp-block-quote"><p><em>Before training the model it is important to establish which characters you want to retain and which are just the writer&#8217;s spelling mannerisms, particularly if the alphabet in your documents isn’t standardised. For example, the character ÿ appeared in our material. After a while, we realised that this wasn’t a ligature from the combination of the letters ij, but was used interchangeably with the letter y. Therefore, we stopped marking the character ÿ as a separate letter and transcribed it simply as y. By establishing a list of characters like this, you can avoid errors in the Ground Truth and so create a more accurate transcription.</em></p><cite>Ewa Rodek, Institute of Polish Language &#8211; Polish Academy of Sciences</cite></blockquote>


Languages evolve over time: even first-year linguistics students could tell you that. But understanding how they evolve is slighty more challenging.

Creating a linguistic corpus with Transkribus: Ewa Rodek


<p>The Fondazione Banco di Napoli preserves the Historical Archive of the Banco di Napoli, the largest complex of historical-economic documentation in the world. This enormous treasure of paper collects the documents of the eight Neapolitan public banks (1539-1809), the Banco delle Due Sicilie (1809-1861) and the Banco di Napoli. This is an extraordinarily rich documentation, where in addition to the names of the customers of the ancient counters, it is also possible to find the reasons for each economic transaction. A huge fresco on the daily life of baroque and contemporary Naples.</p>



<p>In order to enrich this precious source for the history of the Mediterranean, the foundation has launched a project for the digitization and transcription of the indices of the names of the customers of the Neapolitan banks. The Pandetta project (this is the name of customer index) started in 2018, using the Transkribus software and taking advantage of the technology developed by the European Cooperative Society READ. Furthermore, in July 2019 the Banco di Napoli Foundation became a founding member of the READ cooperative.</p>



<p><br>At the moment sixty pandette have been digitized and transcribed, for the banks active from 1586 to 1599. There are over 6,000 pages for a total of about 190,000 transcribed name of ancient customers rercords.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="683" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-1024x683.jpg" alt="" class="wp-image-27018" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-1024x683.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-768x513.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette-600x400.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/11/pandette.jpg 1500w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Example of a Pandetta</figcaption></figure>



<p>This material will be made available as soon as possible and will be offered, through a web portal, to the public and scholar community. In this way, it will be possible, via digital search, find the presence of an account, a character or a surname within the customer lists of the Neapolitan bancks. This great work would not have been possible, especially considering the reasonable time in which it was completed (two years), without the use of the growing HTR technologies offered by the READ cooperative. In fact, during the processing, transcription models capable of returning only 5% of Character Error Rating were trained.</p>



<figure class="wp-block-image"><img alt=""/></figure>


The Fondazione Banco di Napoli preserves the Historical Archive of the Banco di Napoli, the largest complex of historical-economic documentation in the world. This enormous treasure of paper collects the documents of the eight Neapolitan public banks (1539-1809).

The Pandetta Project - Fondazione Banco di Napoli

The Pandetta Project &#8211;  Fondazione Banco di Napoli


<p>Over the last 150 years, large edition series have shaped hanseatic research. Around 1900, a group of historians had collected and brought between book covers a vast amount of material up to the 1530s. From the 16<sup>th</sup> century onwards the material was not manageable though. The inventories of selected archives at least provided an overview of the available material.</p>



<p>Digitalisation has reshuffled the cards; especially Transkribus opens up new possibilities for making the almost endless amounts of unedited material accessible to a wider (research) community. At the FGHO, the Research Center for Hanse and Baltic History, we mostly focus on the so called “recesses”, the minutes of resolutions passed at the Hanseatic Days and other low German town conventions, as at these conventions a lot of hanseatic history ‘was made’ at a time.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="543" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-1024x543.png" alt="" class="wp-image-26738" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-1024x543.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-300x159.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-768x407.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-1536x814.png 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-18x10.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen-600x318.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshot-Tagging-1914-Bremen.png 1916w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>In just two to three years and with just a small team, we were able to transcribe several hundred pages of archival material, to label them with person and place names and to make the material available on a <a href="https://transkribus.eu/r/rezesse-niederdeutscher-staedtetage/#/">Read&amp;Search website</a> – and more pages are in progress. So far, we have transcribed mostly manually, i.a. in order to provide material for special handwriting models. In this process students of the University of Greifswald assisted us. Furthermore, with our Citizen-Science project ‘Read.Hanse.Sources!’, we involved interested citizens – a small but committed team which will hopefully grow in the future. None of it would have been possible without Transkribus. Since it is very important for us to communicate hanseatic research to the outside world or to provide opportunities for public participation, the involvement of the interested public is a great opportunity for scientific communication. At the same time, it helps us to make continuous progress with the project through activities for which our colleagues hardly have any time left.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="576" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-1024x576.png" alt="" class="wp-image-26739" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-1024x576.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-300x169.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-768x432.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-1536x864.png 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-18x10.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage-600x338.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Screenshto-Niederdeutsche-Stadtetage.png 1920w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>What will the future bring? Of course we want to use HTR models and further develop them ourselves, but we also want to continue working on the material with volunteers. Not only do we want indexes of persons and places, we also want to label topics. E.g., parallel manuscripts of the conventions which will be made accessible. Moreover, we want to make accessible at least one manuscript of each convention between 1537 and 1669. With the help of Transkribus and the services offered by READ-COOP, within a few years, a small research unit is able to make accessible sources on hanseatic history to an extent that has not been possible since the beginnings of hanseatic research around 1900. Furthermore, open data will be generated and will serve as a foundation for many interested parties and for further work on the material (e.g.  critical editions, text analysis, and much more).</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="663" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-1024x663.jpg" alt="" class="wp-image-26737" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-1024x663.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-300x194.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-768x498.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-1536x995.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web-600x389.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Prasentation-Web.jpg 1656w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>


Over the last 150 years, large edition series have shaped hanseatic research. Around 1900, a group of historians had collected and brought between book covers a vast amount of material up to the 1530s. From the 16<sup>th</sup> century onwards the material was not manageable though.

Recesses of Low German Hanseatic Days: A Hanseatic journey with Transkribus


<p>A central goal of the research project 'Mapping Medieval Vienna' is to make the Viennese land registers of the 15th century available to the public. This is because the land register entries contain a great deal of information that is highly relevant to the economic and social history of the late Middle Ages. From the middle of the 14th century, after the Viennese Council had come into possession of the city&#8217;s land rights, meticulous records were kept of all house transactions, which means that both the prices of the houses and detailed information on their location as well as on their inhabitants have been handed down.</p>



<p></p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="525" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-1024x525.png" alt="" class="wp-image-26696" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-1024x525.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-300x154.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-768x394.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-1536x788.png 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-18x9.png 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-600x308.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-117x60.png 117w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v-175x90.png 175w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/Workflow-bei-der-Bearbeitung-von-WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.5v.png 1912w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Workflow</figcaption></figure>



<p><br>This bundle of information makes it possible to draw a very concrete picture of the social topography of the city of Vienna. For example, it is possible to locate the properties mentioned in the land registers, so that the exact neighbourhood conditions in individual streets can be reconstructed and even georeferenced. It is also possible to take a closer look at the financial mechanisms of the housing market or to analyse the actual living conditions in the medieval houses.</p>



<p>So far, such aspects could only be partially investigated, since due to the sheer mass of material, only the land registers of the years from 1368 to 1419 were published in the form of registers. Thanks to the HTR technology of Transkribus, however, we are now able to publish a selection of complete land registers in a digital edition. In the preliminary stages of the project, we were able to train an initial model that reliably recognises the late Gothic cursive of the various Viennese scribes by transcribing about 30 folios from one of the land registers. This base model is used for the land registers that have been selected for editing in order to generate a raw version of the transcription at the push of a button, which only needs to be checked and slightly normalised with little effort. Due to the constantly growing amount of training material, the base model is also continuously being expanded, making it more and more accurate. Currently, the character error rate (CER) is below the 2% mark.</p>



<p>In this way, Transkribus makes a significant contribution to enabling transcription to be completed in the shortest possible time. In a second decisive step, the relevant text information is also marked up according to the TEI/XML standard. Thus, a graph database can be built up in which all persons, institutions, properties and prizes will be recorded. Transcription and mark-up will ultimately serve as the basis for the online edition of the sources. This will enable users to view the land registers digitally and access the database behind them.</p>



<p>The project is supported by the German Research Foundation (DFG) and is based at the Freie Universität Berlin in the Department of Medieval History with a focus on the High and Late Middle Ages. Further information can be found at: <a href="https://www.geschkult.fu-berlin.de/e/fmi/bereiche/mittelalter/ab_ertl/Mapping-Vienna.html">https://www.geschkult.fu-berlin.de/e/fmi/bereiche/mittelalter/ab_ertl/Mapping-Vienna.html</a> &nbsp;&nbsp;&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="746" height="1024" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-746x1024.jpg" alt="" class="wp-image-26697" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-746x1024.jpg 746w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-219x300.jpg 219w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-768x1054.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-1119x1536.jpg 1119w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-1493x2048.jpg 1493w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-9x12.jpg 9w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-600x823.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-44x60.jpg 44w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-66x90.jpg 66w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/10/WStLA_Grundbuch_Stadt_Wien_B1_6a_fol.100r-1-scaled.jpg 1866w" sizes="(max-width: 746px) 100vw, 746px" /><figcaption>Example of a &#8220;Grundbuch&#8221;</figcaption></figure>



<p></p>


A central goal of the research project 'Mapping Medieval Vienna' is to make the Viennese land registers of the 15th century available to the public.

Mapping Medieval Vienna: The digital edition of historical land registers supported by Transkribus


<p>Over the course of its history, the colony of New France covered large portions of North America. Its main center was in the city of Québec, which was then the capital of Canada, but the secondary administrations of Louisiana, Acadia, Newfoundland and Isle Royale (today’s Cape Breton Island) benefited from a broad autonomy and generated their own documentation. Today, New France’s archives are scattered in many government (national, provincial and municipal), religious and private (libraries, museums) archive centers, mainly in Canada, the United States and France, but also elsewhere in Europe and America. This wide dispersion of archives makes it particularly difficult to obtain a global portrait of the colonial regime and the mechanisms underlying the production and circulation of its documentation. New France’s documentary corpus is also characterized by its great material diversity, being composed of administrative and private correspondences, as well as judicial, notarial, religious, cartographic, linguistic and literary works, all types of sources that require specific expertise to be decrypted and analyzed.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="725" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-1024x725.jpg" alt="" class="wp-image-24938" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-1024x725.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-300x212.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-768x543.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-600x425.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-85x60.jpg 85w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3-127x90.jpg 127w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_3.jpg 1320w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>« Ratification de la paix entre les Sauvages &amp; les Iroquois a Montreal le 4 aoust 1701 », Archives nationales d’Outre-Mer (Aix-en-Provence), Série C11A, vol. 19, f. 43.</figcaption></figure>



<p>Nouvelle-France numérique is a project that seeks to overcome these problems and revolutionize research on New France by creating the means for pooling expertise and documentation and thus making New France’s corpora easier to access, to search and to process by AI.</p>



<p>Led by historians Maxime Gohier, Léon Robichaud and Jean-François Palomino, as well as philologist Catherine Broué, Nouvelle-France numérique brings together a multidisciplinary team of specialists in history, linguistics, geography, literature and digital humanities. Our collaborators and partners are based in various universities in Quebec, Canada, the United States and Europe, as well as the main institutions preserving New France’s archives (Bibliothèque et Archives nationales du Québec, Archives nationales d’Outre-Mer, Library and Archives Canada, Musée de la civilization de Québec, the Rare Books Library and the Documents and Archives Management Division of the University of Montreal).</p>



<p>Since its early days, in 2018, Nouvelle-France numérique has been associated with the READ-Coop and the development of Transkribus. The project relies on Transkribus to share the documents provided by our partner archive centers, to transcribe them and to mark up the metadata following TEI standards.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="569" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-1024x569.jpg" alt="" class="wp-image-24937" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-1024x569.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-300x167.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-768x427.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-16x9.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-600x334.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-108x60.jpg 108w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1-162x90.jpg 162w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_1.jpg 1320w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Commission par la Compagnie de la Nouvelle-France à Louis D’Aillebout pour la distribution des terres aux colons, 29 mars 1649, Archives de l’Université de Montréal, Collection Louis-François-Georges Baby, série O (commissions civiles) P0058/O,002.</figcaption></figure>



<p>With the help of our collaborators, we have been able to transcribe different types of documents and to train several HTR models adapted to modern French (17th and 18th centuries) and to different styles of handwritings. These models have allowed us to transcribe, to date, tens of thousands of pages of different types of documents, achieving Character Error Rate (CER) below 5%, some of which have been made accessible to every users in Transkribus (see model “New France 17<sup>th</sup>-18<sup>th</sup> Century”). In the coming years, we plan to improve these models and make them both more efficient and flexible, in order to accommodate a wider variety of writing styles. In addition, because of the central role played by Indigenous peoples in the history of New France, the project also aims at developing recognition models adapted to different indigenous languages.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="579" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-1024x579.jpg" alt="" class="wp-image-24939" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-1024x579.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-300x170.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-768x434.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-16x9.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-600x339.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-106x60.jpg 106w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2-159x90.jpg 159w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/07/quebec_2.jpg 1320w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>Nouvelle-France numérique benefits from financial support from Canadian <a href="https://www.sshrc-crsh.gc.ca/">Social Sciences and Humanities Research Council</a>, through the “Partnership Development” program. The website can be consulted at the following address: <a href="http://www.nouvellefrancenumerique.info/">www.nouvellefrancenumerique.info</a> (in French).</p>



<p></p>


Over the course of its history, the colony of New France covered large portions of North America. Its main center was in the city of Québec, which was then the capital of Canada, but the secondary administrations of Louisiana, Acadia, Newfoundland and Isle Royale (today’s Cape Breton Island) benefited from a broad autonomy and generated their own documentation.

Nouvelle-France numérique: Collaboration and partnership arising from AI


<p>From the digitization with the <a href="https://readcoop.eu/scantent/">ScanTent</a>, to the use of artificial intelligence for the recognition of the handwritten forms through to the publication of the documents using <a href="https://readcoop.eu/readsearch/">read&amp;search</a> &#8211; with the project &#8220;Military Personnel Master Data Sheets&#8221; of the Tyrolean State Archives, READ-COOP SCE pulled out all the stops with its <a href="https://readcoop.eu/transkribus/">Transkribus</a> platform. Now, more than 98,000 files are available to the public. They reflect the fates of around 90,000 young soldiers from the historical region of Tyrol who did their military service in the Austro-Hungarian army in the late 19th century. This unique collection is of great importance for scholars and many family historians and has already been in great demand.</p>



<p>Christoph Haidacher, Director of the Tyrolean State Archives, comments: &#8220;For the Tyrolean State Archives, the cooperation with Dr. Günter Mühlberger and READ-COOP SCE offered a unique opportunity to make a collection that is in great demand throughout the European region of Tyrol accessible online, beyond the confines of our reading room, and thus to all interested parties. The availability of selected archive holdings on the worldwide web in conjunction with artificial handwriting recognition means pioneering a path that the Tyrolean State Archives are happy to pursue with its partners.</p>



<p> The project was carried out in several stages: In January and February 2020, the data sheets from South Tyrol and Trentino were scanned, and in February and March 2021, the sheets from North Tyrol were scanned &#8211; together, a little more than 98,000 individual scans. Each scan represents the first page of a personnel file.</p>



<div class="wp-block-image"><figure class="aligncenter size-full"><img decoding="async" width="605" height="326" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_.jpg" alt="" class="wp-image-24808" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_.jpg 605w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_-300x162.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_-16x9.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_-600x323.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_-111x60.jpg 111w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_-167x90.jpg 167w" sizes="(max-width: 605px) 100vw, 605px" /><figcaption>Registration sheet of the Imperial and Royal Army.</figcaption></figure></div>



<p>The file usually consists of several sheets, which were bound with staples or adhesive tape. Due to an otherwise much-increased workload, only the file&#8217;s first page was digitized in each case. Scans of the remaining pages can be ordered for a small fee within the framework of an on-demand procedure.</p>



<p>Several ScanTents were used for the digitization, usually five to seven students were working at the same time.</p>



<div class="wp-block-image"><figure class="aligncenter size-full"><img decoding="async" width="454" height="279" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning.jpg" alt="" class="wp-image-24809" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning.jpg 454w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning-300x184.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning-16x10.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning-98x60.jpg 98w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_scanning-146x90.jpg 146w" sizes="(max-width: 454px) 100vw, 454px" /><figcaption>Digitisation using ScanTent and smartphone</figcaption></figure></div>



<p>Using the <a href="https://play.google.com/store/apps/details?id=at.ac.tuwien.caa.docscan&amp;hl=en">DocScan</a> app, the pages were scanned and cropped, and immediately uploaded into the Transkribus platform. The naming of the files was done according to the labeling of the original boxes. The entire collection could thus be completely digitized in less than 20 working days.</p>



<p>In a further step, a special model was trained to recognize the layout. Here, the P2PaLa tool was used. Since the military  personnel master data sheets, &#8220;Grundbuchsblätter&#8221; in German, are complex forms that have changed over the years and are sometimes very inconsistent, handwriting recognition was deliberately limited to the name, birthplace, and year of birth. These three fields, present in all form types, were trained on several hundred examples. The fields are identified by the software quite reliably, as the following example shows (&#8220;Name&#8221; = name, &#8220;Ort&#8221; = (birth-)place, &#8220;Jahrgang&#8221; = year of birth). </p>



<div class="wp-block-image"><figure class="aligncenter size-full is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2.jpg" alt="" class="wp-image-24810" width="794" height="428" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2.jpg 605w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2-300x162.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2-16x9.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2-600x323.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2-111x60.jpg 111w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp2-167x90.jpg 167w" sizes="(max-width: 794px) 100vw, 794px" /><figcaption>Text and structure recognition with Transkribus</figcaption></figure></div>



<p>In parallel to creating the layout model, a special handwriting model was trained to recognize the actual family and first names, i.e., the writing within the previously identified &#8220;name&#8221; fields. For this purpose, the first 10 pages of each document in the South Tyrolean collection were transcribed manually and some documents in their entirety, as training data. Furthermore, name data from other projects, such as the one on the Franciscan Cadastre, were also added, and the general model &#8220;Transkribus German Kurrent M2&#8221; was used as a base model. Only the name field was recognized with this special model &#8211; a feature that is probably still far too little known.</p>



<div class="wp-block-image"><figure class="aligncenter size-full is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1.jpg" alt="" class="wp-image-24811" width="353" height="486" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1.jpg 286w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1-218x300.jpg 218w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1-9x12.jpg 9w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1-44x60.jpg 44w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/grundbuch_bsp1-65x90.jpg 65w" sizes="(max-width: 353px) 100vw, 353px" /><figcaption>Recognition for a structural field</figcaption></figure></div>



<p>Transkribus allows different models to be applied to individual structural fields. The two remaining fields, i.e., place of birth and year of birth, were recognized with the standard &#8220;German Kurrent M2&#8221; model. The recognition quality of about 89% is satisfactory given the rather challenging form fields, the many different font types and writers, and the relatively small set of training data. The automated processing, which was carried out directly in the platform, took several weeks, including waiting times due to the many individual jobs.</p>



<p>The last part then consisted of making the data available using the &#8220;read&amp;search&#8221; web interface. This process could be completed relatively quickly, as the documents were already in Transkribus and what was left was setting up the according website with the required background images, description texts, fonts, colors etc.</p>



<p>The military personnel master data sheets of the Tyrolean State Archives can now be accessed online at: <a href="https://transkribus.eu/r/grundbuchblaetter/">http://transkribus.eu/r/grundbuchblaetter/</a>.</p>



<p>A crowd-sourcing component is to be added to the project in the coming months. Through the &#8220;citizen@science&#8221; tool, which is currently being developed, interested users will be allowed to volunteer to help correct the recognized text as well as complete the remaining form data, thus significantly enriching this valuable historical resource further.</p>


From the digitization with the ScanTent, to the use of artificial intelligence for the recognition of the handwritten forms through to the publication of the documents using read&amp;search - with the project 'Military Personnel Master Data Sheets' of the Tyrolean State Archives, READ-COOP SCE pulled out all the stops with its Transkribus platform.

How to make a complete collection accessible with Transkribus. A best-practice example from the Tyrolean State Archives.

How to make a complete collection accessible with Transkribus. A best-practice example from the Tyrolean State Archives


<p>My first encounter with Transkribus was driven by envy. In 2014 I helped to draft an unsuccessful Horizon 2020 proposal at Trinity College Dublin for a call that was won by the READ consortium. I subsequently became involved in a major transcription project for the Irish Manuscripts Commission and attended READ’s 2018 user conference in Innsbruck to establish (1) did Transkribus work and (2) expecting that it did not, how was READ’s original proposal better than ours.<br>Transkribus, of course, worked very well and is now a key component of a suite of software tools we are using to realise Beyond 2022, Ireland’s virtual record treasury. On 30 June 1922, at the outbreak of Ireland’s civil war that followed independence from the United Kingdom, the Public Record Office of Ireland was completely destroyed. On the centenary of this cultural catastrophe, 30 June 2022, the Beyond 2022 Project, funded by the government of Ireland and based at Trinity College Dublin, will unveil a virtual reconstruction of this building and digital surrogates of many of its contents. The copies are official or academic transcriptions sourced from over 50 libraries and archives around the world. Although some of these copies are printed, the majority are manuscript and range from contemporary 13th century transcriptions to official copies made throughout the 19th century. The material is mainly in English or Latin.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-898x1024.jpg" alt="" class="wp-image-24746" width="560" height="638" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-898x1024.jpg 898w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-263x300.jpg 263w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-768x876.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-11x12.jpg 11w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-600x684.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-53x60.jpg 53w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_-79x90.jpg 79w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/06/beyond2022_.jpg 912w" sizes="(max-width: 560px) 100vw, 560px" /><figcaption>Beyond 2022&#8217;s English language model, transcribing a page from the 1641 Depositions</figcaption></figure></div>



<p><br>We began by producing bespoke HTR models for specific large series of transcriptions, principally unpublished calendars of early material produced by the Irish Record Commissioners, 1810-1830. As these are written in consistent copperplate hand to very high standards, the results form Transkribus are excellent. Our next steps were to produce models tailored to the more cursive hands used by Victorian antiquaries. These enthusiasts sometimes produced 10,000 pages of transcriptions, entirely on their own and for their own research. We are fortunate to have found several collections of these transcriptions made for private research in libraries as far afield as Chicago. The senior officials who ran Ireland on behalf of the British crown normally made copies of the official records produced during their tenure and left the copies in Ireland when they moved on, taking the originals with them. These also consist of large collections of around 10,000 pages, usually the work of one or two officials working carefully, and thus also ideal for a Transkribus approach. We have recently bundled our hand-specific models into a single base model that produces excellent results from most official documents in English, 1600-1900. This model will be made publicly available to all Transkribus users on 20 June 2021, the 99th anniversary of the destruction of the Public Record office of Ireland.<br>The Beyond 2022 workflow consists of receiving digital images of historical printed and manuscript texts from our archival partners, ‘stitching’ these documents to their destroyed equivalents and placing them back on the virtual shelves of the PROI. We have produced a detailed 3D rendered model of the building and recovered the shelving arrangements for the 140,000 folders, boxes, bound volumes and parchment rolls that were its contents. We can, therefore, place the original at the exact location in the building where it was destroyed. With Transkribus, we can produce high quality searchable text that is then analysed by our own Natural Language Processing system that in turn produces triples of entities that are the basis for a knowledge graph for Irish history. This final step is not possible without Transkribus sitting in the middle producing millions of words of high quality text.<br>Trinity College Library joined the READ COOP on behalf of the university as READ transitioned from an ERC funded research project to an independently funded one. Beyond 2022 is one of several projects based at Trinity that uses the COOP’s services, and many more are on the way.<br>For more on Beyond 2022 see our website: <a target="_blank" href="https://beyond2022.ie/" rel="noreferrer noopener">https://beyond2022.ie/</a></p>


My first encounter with Transkribus was driven by envy. In 2014 I helped to draft an unsuccessful Horizon 2020 proposal at Trinity College Dublin for a call that was won by the READ consortium.

David Brown on Transkribus - the Beyond 2022 Project


<p>The National Library of Finland has reprocessed almost two million historical newspaper pages with the Transkribus automatic text recognition workflow in cooperation with READ-COOP. The greatly improved recognition results convinced the Library of a workflow which was developed to its current state in the NewsEye project. The University of Innsbruck led this development. Text recognition in general, and high-accuracy recognition in particular, is of immense importance for the quality and usability of digitized historical sources.</p>



<p>The material in this reprocessing cooperation project with READ-COOP included a little under two million pages of Finnish newspapers dating from 1771 to 1914. The languages of the materials are Finnish and Swedish, according to the languages used in Finland during this timeframe. Now all the Finnish newspapers published in Finland starting from the first newspaper published in 1771 until the newspaper titles from 1914 and a selection of newspapers from 1915 to 1918 have been reprocessed.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="462" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-1024x462.png" alt="" class="wp-image-24074" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-1024x462.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-300x135.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-768x346.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-16x7.png 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library-600x270.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/Digital-Library.png 1471w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure>



<p>The newly reprocessed newspapers will gradually replace the older versions, with lower optical character recognition results, in the publication and presentation system of the National Library of Finland, starting from summer 2021. The Library will launch an information campaign regarding the quality improvements. We are also aiming to process more newspapers from 1914 onwards, but this decision will follow later.</p>



<p>The improvement of the text recognition results has been considerable and we are currently calculating the exact figures. These will be published on<a href="https://digi.nationallibrary.fi"> https://digi.nationallibrary.fi</a> .</p>



<p>The work in this cooperation has been financed by the EU’s European Regional Development Fund / Leverage for the 2014-2020 funding period.</p>



<div class="wp-block-image"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_1.png" alt="" class="wp-image-24073" width="146" height="103" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_1.png 634w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_1-300x212.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_1-16x12.png 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_1-600x425.png 600w" sizes="(max-width: 146px) 100vw, 146px" /></figure></div>



<div class="wp-block-image"><figure class="alignleft size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_2.png" alt="" class="wp-image-24072" width="138" height="152" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_2.png 354w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_2-272x300.png 272w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/05/EU_2-11x12.png 11w" sizes="(max-width: 138px) 100vw, 138px" /></figure></div>



<p></p>


The National Library of Finland has reprocessed almost two million historical newspaper pages with the Transkribus automatic text recognition workflow in cooperation with READ-COOP.

Improved Text Recognition for Finnish Historical Newspapers with Transkribus


<p>In 2016 I joined the READ project for the state archives of Zurich. Within the large project, I became part of the dissemination working group and responsible for the alignment of more than 100’000 pages of handwritten minutes from the Zürich executives of the 19<sup>th</sup> century. Thanks to READ, I could not only travel Europe &amp; the US for more than 50 Transkribus related workshops and talks. Furthermore, I got in contact with numerous scholars, archivists, librarians, and scientists trying to get the most out of HTR, KWS, (semantically enhanced) Layout Analysis, and much more. I had the privilege of seeing written cultural heritage in its incredible variety and discussing its specificity with experts.</p>



<p>One of the consequences of using, thinking, and talking about machine learning daily was to encounter this approach and its advantages and problems in-depth and shape my research accordingly. The result of my usage of Transkribus was thus not only hundreds of HTR+ and PyLAIA models as well as the preparation of thousands of pages of Ground Truth (see, e.g., the public model StAZH_RRB_German_Kurrent_XIX based on 26 million words). It’s rather the insight that it’s our duty as scholars to use and critically analyze deep learning not only to make cultural heritage accessible but help understand the technology and its pitfalls for our future benefit.</p>



<p>Regarding Transkribus, I understand the platform as ready to use if several hundreds of images need to be processed and a stable environment is essential. For a scholarly edition project (koenigsfelden.uzh.ch), we used Transkribus as our hub for transcriptions, resulting in some HTR models as a by-product. At the end of my tenure at the state archives of Zürich, we started a variety of projects building on HTR+ and p2pala to prepare vast amounts of pre-modern text and use semantic annotations to speed up archival indexing. For the whole GLAM field, I believe this is the way to go.</p>



<p>In 2019 – in no small part thanks to the success of READ – I was offered a tenure track position at the University of Bern with the task to provide the faculty with approaches to digital humanities. Since then, I have been using Transkribus in teaching and currently think about the next steps in text annotation, including Named Entity Recognition (esp. for historical languages) and Content Extraction (e.g., using Topic Modeling).</p>



<p>Want to know more? I published in German and English about Transkribus, HTR, and consequences of the use of machine learning in the humanities (besides some stuff about the Middle Ages 😉</p>



<p>See my page at the University of Bern <a href="https://www.dh.unibe.ch/ueber_uns/personen/prof_dr_hodel_tobias/index_ger.htm">here</a>, and my <a href="https://orcid.org/0000-0002-2071-6407">ORCiD profile</a>, or follow me on <a href="https://twitter.com/thist_be/">Twitter</a>.</p>



<div class="wp-block-image"><figure class="aligncenter size-large"><img decoding="async" width="1024" height="623" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-1024x623.png" alt="" class="wp-image-23331" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-1024x623.png 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-300x182.png 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-768x467.png 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-16x10.png 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-600x365.png 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-99x60.png 99w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem-148x90.png 148w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/karte_dissem.png 1128w" sizes="(max-width: 1024px) 100vw, 1024px" /></figure></div>


In 2016 I joined the READ project for the state archives of Zurich. Within the large project, I became part of the dissemination working group and responsible for the alignment of more than 100’000 pages of handwritten minutes from the Zürich executives of the 19<sup>th</sup> century.

My READ/Transkribus Story: Tobias Hodel


<p>Legal history is a research field that focuses on the evolution of legal systems, and the reasons behind the implementation of different laws. Luckily for legal historians, law is also a field that is known for its thorough documentation. That means there is a wealth of legal documentation available for researchers, stretching back several hundred years.</p>



<p>However, accessing this (mostly paper) documentation isn’t always easy. Firstly, documentation is often housed across several archives in different locations, and a researcher must travel personally to the archive in order to view the documents. Even the documents that have already been digitised are spread out across the internet and often hidden behind paywalls.</p>



<p>At the <a href="https://www.uzh.ch/en.html">University of Zurich</a> , a group of researchers decided it was time to rectify this problem. <a href="https://www.ius.uzh.ch/de/staff/professorships/alphabetical/boente.html">Prof. Dr. Walter Boente</a> — chair for private law at the university — and his team decided to produce a digital corpus of legal texts and publish it online using the <a href="https://readcoop.eu/readsearch/">read&amp;search</a> platform. By doing so, they have created one of the largest digital collections of historical legal documents available on the internet.&nbsp;</p>



<h2 class="wp-block-heading" id="h-transcribing-the-documents">Transcribing the documents</h2>



<p>Of course, digitising every document out there in one go would be a rather large project, particularly as the team had not embarked on a project of this type before. Instead, they decided to begin with a few smaller sample collections, such as papers written by lawyers of historical importance or documents concerning a particular law.&nbsp;</p>



<p>The starting point was to bring documents stored digitally in other archives and libraries over to the new platform. In addition, many documents needed to be digitised for the first time, and the team was lucky to have the support of many institutions, who prepared their documents ready for use with Transkribus.</p>



<p>Once the documents were accessible on Iurisprudentia, they were then transcribed using the Transkribus platform. The team started by using the public models offered by Transkribus to automatically transcribe the historical documents. But with the help of the crowd, more and more documents were transcribed or corrected manually. Using the resulting Ground Truth, the team retrained the general models and thereby adopted them to the needs of the project.&nbsp;</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="502" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-1024x502.jpg" alt="" class="wp-image-22089" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-1024x502.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-300x147.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-768x377.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-1536x754.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-16x8.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example-600x294.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia-example.jpg 1879w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption>Example of an Edition that is made accessible thanks to the iurisprudentia project </figcaption></figure>



<h2 class="wp-block-heading">Making the collections accessible</h2>



<p>Another key goal for the Iurisprudentia team was accessibility. After all, it was the lack of access to legal historical documents that prompted the team to start the project in the first place. The aim was to have all the transcribed documents available in one place, without a pay wall, and for users to be able to quickly search entire collections for specific information.</p>



<p>After looking at several options, the team decided to use the read&amp;search platform to publish the corpus. Their main reason was the ease of use that read&amp;search offered. Documents can be uploaded straight from Transkribus, as well as any tags and metadata that have been implemented. The team therefore could rely on the infrastructure set up centrally by READ-COOP. For the Iursprudentia team, this remains one of the biggest benefits.</p>



<p>“The read&amp;search platform is an all-round carefree package,” Professor Boente explained. “Every few weeks I see new tools popping up that seem to offer better solutions than read&amp;search. That is sometimes very tempting. Nevertheless, it would take a lot of investment and further development to be able to use these tools productively.”</p>



<figure class="wp-block-video"><video autoplay loop poster="" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/iurisprudentia.mp4" playsinline></video><figcaption>A journey through time with the iurisprudentia web application</figcaption></figure>



<h2 class="wp-block-heading">An ever-expanding corpus</h2>



<p>The Iurisprudentia platform currently stores 22 different collections of documents, spanning hundreds of years of legal history. Although the platform was originally intended just for documents concerning Swiss, German and Austrian law, several international collections have now been launched. The first of these, a Polish-language collection on civil law, has already been added and there are other collaborations in the pipeline.</p>



<p>It has been several years since the project began, but Professor Boente is still very happy with their decision to use read&amp;search for the project. “To date, I have not found a better offer than read&amp;search. My advice is therefore to concentrate on the content of your project. If, contrary to expectations, you find something better in the future, you can continue to use all of your data stored with read&amp;search—without converting it. It doesn&#8217;t get any more sustainable than that!”</p>



<p></p>



<p></p>



<p></p>


Legal history is a research field that focuses on the evolution of legal systems, and the reasons behind the implementation of different laws. Luckily for legal historians, law is also a field that is known for its thorough documentation.

Iurisprudentia - A Journey through Time


<p>The Vienna City Library has already implemented three projects with Transkribus. Their first project was the Lehmann address books. All of Vienna&#8217;s main tenants from 1859 to 1942 are listed in these Viennese address books, which is why they are a valuable source for research into the history of the city of Vienna. The address books were digitized from microfilm in 2011 and put online without OCR. In order to ensure a better searchability, &#8220;Lehmann&#8221; was digitized again from the original in 2018 and a full-text recognition was carried out. However, the recognition of addresses and names in the Fraktur script with common OCR programs results in an error rate that is too high, therefore the library decided to work with Transkribus. Now there are approximately 200,000 pages of the Lehmann books which are searchable <a href="https://www.digital.wienbibliothek.at/wbrobv/periodical/titleinfo/2316398">here</a>.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-870x1024.jpg" alt="" class="wp-image-22083" width="435" height="512" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-870x1024.jpg 870w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-255x300.jpg 255w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-768x904.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-1306x1536.jpg 1306w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-10x12.jpg 10w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470-600x706.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Lehmann-e1617950623470.jpg 1337w" sizes="(max-width: 435px) 100vw, 435px" /><figcaption>A page from the Lehmann address books © Wienbibliothek im Rathaus</figcaption></figure></div>



<p>Another project was on the text recognition of handwriting. The Vienna Library owns the estate of Franz Grillparzer, which was bequeathed to the City of Vienna in 1878 as a gift. On the occasion of the 150th anniversary of Grillparzer&#8217;s death, which will be celebrated in January 2022, the Vienna City Library is planning to make the Grillparzer manuscripts available online. For this purpose, the letters and manuscripts written by Grillparzer were sent to Transkribus for full-text recognition. After creating a model, around 15,000 scans were successfully recognized.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-825x1024.jpg" alt="" class="wp-image-22085" width="413" height="512" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-825x1024.jpg 825w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-242x300.jpg 242w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-768x953.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-1238x1536.jpg 1238w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-10x12.jpg 10w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1-600x744.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Grillparzer-1.jpg 1500w" sizes="(max-width: 413px) 100vw, 413px" /><figcaption>A Manuscript of Franz Grillparzer © Wienbibliothek im Rathaus</figcaption></figure></div>



<p>The last project the library carried out with Transkribus was the text recognition of around 20,000 obituary notes. The challenge with these announcements was the different fonts, languages and formats of the obituaries. In addition to that, presorting was not possible due to the sheer volume. Attempts with common OCR programs had failed because the various formats within a death notice were not recognized. With Transkribus, however, it was finally possible to carry out successful text recognition of all death notices. It is planned for them to go online by the end of 2021. </p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-1024x746.jpg" alt="" class="wp-image-22082" width="512" height="373" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-1024x746.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-300x219.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-768x560.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-1536x1119.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-2048x1493.jpg 2048w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/04/Transkribus_Partezettel-600x437.jpg 600w" sizes="(max-width: 512px) 100vw, 512px" /><figcaption>Example of an obituary note © Wienbibliothek im Rathaus</figcaption></figure></div>



<p></p>


The Vienna City Library has already implemented three projects with Transkribus. Their first project was the Lehmann address books. All of Vienna's main tenants from 1859 to 1942 are listed in these Viennese address books, which is why they are a valuable source for research into the history of the city of Vienna.

Transkribus Projects at the Vienna City Library


<p>The aim of the Greifswald project is the complete digitization and full-text indexing of the verdict files of the Greifswald Faculty of Law, namely the statement of reasons for the judgments of the assessors at the Wismar Tribunal from the period of the Wismar Council Court from 1580 to 1880. In total there are around 250,000 pages of historical documents.</p>



<p>Such court sources often form the basis for various research fields of criminal law history and historical crime research with their references to research observing everyday life, as well as mentality- and gender research. They also provide detailed information about the changing legal thinking between the 16th and 19th centuries. Interestingly, they are not just about the &#8220;big criminal cases&#8221;, but rather about the &#8220;ordinary people&#8221; and their everyday conflicts with the law. The cases range from murder and manslaughter, robbery and theft, all kinds of assault and insults, broken marriage vows, inheritance disputes and guardianship, to economic crimes and matters of faith.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="1024" height="598" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-1024x598.jpg" alt="" class="wp-image-21557" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-1024x598.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-300x175.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-768x448.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-1536x897.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-16x9.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-600x350.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-103x60.jpg 103w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia-154x90.jpg 154w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/Justitia.jpg 1684w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption><em>Justitia in front of the Baltic Sea © Rechtsprechung im Ostseeraum</em></figcaption></figure>



<p>Due to their sheer quantity, their scope, and the special internal structure, the court and trial files in the archives are often only cataloged flat and can therefore only be researched to a limited extent. Even the arrangement of the process files makes it difficult to use them systematically. The inquiries from lawyers with the case descriptions and the associated legal information, counter statements and submissions from the prosecutor and defendants, witness statements and interrogation protocols are so manifold that these files are often only indexed according to the parties in litigation (who against whom?), hardly according to location, and almost never according to offenses.</p>



<p>The given text corpus therefore poses quite a challenge when it comes to the big questions (Who? What?, When?, Where?, How? And why?). They cannot be elicited that easily – which actually portrays a typical case for digital humanities &#8230; and also for archives.</p>



<p>This is where full-text indexing comes into play with the help of the tools of the Transkribus platform. The ability to develop an own HTR (Handwritten Text Recognition) models that fit the varied material perfectly and that are capable of reliably coping with and transcribing a text volume of up to 250,000 pages initially sounded like a utopia. Meanwhile, however, a large part of the material has been processed in Transkribus. It soon should also be searchable by using KWS (Keyword Spotting). The team describes the experiences they made in this project <a href="https://rechtsprechung-im-ostseeraum.archiv.uni-greifswald.de/">in a blog</a> and of course they share their best HTR models with the Transkribus community.</p>



<p>The Greifswald University Archives, the Archives of the Hanseatic City of Wismar, the Mecklenburg-Western Pomerania State Archives, the Greifswald University Library and the READ-COOP cooperate in this project. The project is funded by the Deutschen Forschungsgemeinschaft (DFG).</p>


The aim of the Greifswald project is the complete digitization and full-text indexing of the verdict files of the Greifswald Faculty of Law, namely the statement of reasons for the judgments of the assessors at the Wismar Tribunal from the period of the Wismar Council Court from 1580 to 1880.

Greifswald: Making legal sources from 1580 to 1880 accessible


<p>The project REPUBLIC aims to create a searchable online edition of the resolutions of the States-General of the Dutch Republic (1576-1796). This federal assembly, in which each of the seven United Provinces had a seat, gathered almost every day in The Hague to decide on common matters like defense, foreign policy, finance, trade, and religion. Although small in size, the Dutch Republic is considered to have been one of the Great Powers of the Early Modern Age, mainly due to its prominent position in global trade. Therefore the resolutions of the States-General are a highly important source for national and international history.</p>



<figure class="wp-block-image size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-1024x800.jpg" alt="" class="wp-image-21438" width="768" height="600" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-1024x800.jpg 1024w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-300x234.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-768x600.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-1536x1199.jpg 1536w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1-600x469.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic1.jpg 1921w" sizes="(max-width: 768px) 100vw, 768px" /><figcaption>States-General of the Dutch Republic © Project REPUBLIC</figcaption></figure>



<p>In the project, Transkribus provides a fundamental role. Ultimately, some 425,000 pages of handwritten resolutions will be transcribed by HTR through the platform. During the first phase, Huygens ING have created a ground truth corpus of around 1,000 pages, with which several HTR-models have been trained. The <a href="https://readcoop.eu/model/republic-7/">latest model</a>, which Huygens ING proudly releases for common use, is performing at a staggering 2.99% Character Error Rate. With the help of volunteers Huygens ING aims at lowering this percentage even further during the next phase of the project. Because of the excellent HTR results they are confident that the ultimate goal, making this important source (re)searchable from all sorts of perspectives, will become reality.</p>



<figure class="wp-block-image size-large"><img decoding="async" width="680" height="1005" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic2.jpg" alt="" class="wp-image-21439" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic2.jpg 680w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic2-203x300.jpg 203w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic2-8x12.jpg 8w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/proj_republic2-600x887.jpg 600w" sizes="(max-width: 680px) 100vw, 680px" /><figcaption>A Resolution from 1598 © Project REPUBLIC</figcaption></figure>


The project REPUBLIC aims to create a searchable online edition of the resolutions of the States-General of the Dutch Republic (1576-1796). This federal assembly, in which each of the seven United Provinces had a seat, gathered almost every day in The Hague to decide on common matters like defense, foreign policy, finance, trade, and religion.

REPUBLIC: The resolutions of the States-General of the Dutch Republic (1576-1796)


<p><strong>Digital Publications are the latest addition to the DH Lab of the Bibliotheca Hertziana – Max Planck Institute for Art History in Rome (https://www.biblhertz.it), and the goal is to to publish the Institute’s Open Access content online.</strong></p>



<p>Since lot of Art History sources are antique books, and the library of our institute has been devoting a lot of resources scanning our “Rara” books collection (http://dlib.biblhertz.it), it became natural to imagine a way to access them not only as digitized images but also as transcribed content. This will allow authors to quote them directly, but also to enhance cross referencing, content checking and accessibility for people relying on TTS tools.</p>



<p>Older books present several challenges to standard OCR, the technology normally used to recognize text in books. Not only are some characters and ligatures difficult to train (just think about the slight difference between the letter “f” and the elongated s “ſ”, or the use of “u” as “v” in lower letters, and “V” as “U” in capital letters), but there are also abbreviations and symbols with a special meaning. In fact, especially in the fifteen and sixteen centuries, most books contained the exact scribal abbreviations that were common in manuscripts.</p>



<p>This means that approaching the transcription one character at time, as OCR does, would simply create a huge error rate and no way to search for abbreviated words. On the contrary, even if HTR is intended for handwriting, it can be trained perfectly to adapt to the context, and expand abbreviations or distinguish between letters that seem or are identical.</p>



<p>For this reason, we teamed up with the READ-COOP and planned a complete neural text recognition of our existing digitisations. The goal is to create new models that will be able not only to transcribe all the content, but also to recognize a book’s main structure: extract the list of images, distinguish between main text and commentaries, and a lot more. The transcriptions will be available in the IIIF viewer, but also in an online <a href="https://readcoop.eu/readsearch/">Read&amp;Search</a> platform, together with the digitized books from the Kunsthistorisches Institut in Florence and the Max-Planck-Institut für Wissenschaftsgeschichte in Berlin, they can be searched and analysed through machine learning for data mining.</p>



<p>Another project relying on <a href="https://readcoop.eu/transkribus/download/">Transkribus </a>is a digital edition of manuscripts, where alongside the manual transcription of the content, the tagging of information is needed. Thanks to the easy tag management available in the Expert client, the team can work together and edit the text, insert semantic information and identify relevant named entities like people, places, dates or artworks that are mentioned in the text. Thanks to direct TEI export or XSLT conversion, the digital edition can be created almost without any further post processing.</p>



<p>Before starting this project, I was already using Transkribus for my own research, and now I encourage Hertziana researchers to use it as much as possible when accessing content is important, or when working on a digital edition.</p>



<div class="wp-block-image"><figure class="aligncenter size-large is-resized"><img decoding="async" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana.jpg" alt="" class="wp-image-21231" width="840" height="630" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana.jpg 1019w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-300x225.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-768x577.jpg 768w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-16x12.jpg 16w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-600x450.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-80x60.jpg 80w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2021/03/image_2-Hertziana-120x90.jpg 120w" sizes="(max-width: 840px) 100vw, 840px" /><figcaption><em>Source: <a href="https://www.biblhertz.it/2660543/teaser-1557389517.jpg">https://www.biblhertz.it/2660543/teaser-1557389517.jpg</a></em></figcaption></figure></div>



<p></p>


Digital Publications are the latest addition to the DH Lab of the Bibliotheca Hertziana – Max Planck Institute for Art History in Rome (https://www.biblhertz.it), and the goal is to to publish the Institute’s Open Access content online.

Transkribus at the Bibliotheca Hertziana – Max Planck Institute for Art History


<p>Mathematics, computing, music, poetry: Ada Lovelace was a person of many talents. The 19th-century mathematician — and daughter of infamous Romantic poet, Lord Byron — is widely regarded as one of the pioneers of modern computing thanks to her work on Charles Babbage’s “Analytical Engine”. One of Lovelace’s most groundbreaking ideas was that the machine could be used for more than just arithmetic: something that, in today’s world of smartphones and servers, turned out to be very true indeed.</p>



<p>However, despite Lovelace’s technical achievements, we know surprisingly little about her life. While there is a wealth of papers and documents stored in the Lovelace archive, much of it is inaccessible, making it difficult to conduct scholarly research on the sources. But that is about to change.</p>



<p>Jessica Cook, a PhD candidate in the English department at <a href="https://www.ucla.edu/">UCLA</a>, is currently transcribing the entire Ada Lovelace archive to uncover more about the life of this amazing mathematician. She hopes that by making it easier to access Lovelace’s personal writings, we will have a better insight into Lovelace’s thinking, and into how her poetry influenced her revolutionary ideas about computing. We had a chat with Jessica to find out more about this unique research project and her experience with Transkribus.</p>



<h2 class="wp-block-heading" id="h-accessing-the-ada-lovelace-archive">Accessing the Ada Lovelace Archive</h2>



<p>There hasn’t always been so much interest in the life and work of Ada Lovelace. In the past, much more attention was paid to her male collaborator, Charles Babbage, with Lovelace seen as a mere assistant. “Many accounts portrayed Lovelace as a naive and somewhat star-struck young woman with no advanced mathematical knowledge and little true visionary insight,” Jessica explained. “But beginning in the late 1970s, a number of studies began to delve more fully into Lovelace’s intellectual legacy. Together they convincingly demonstrate that she did indeed possess the requisite technical knowledge to significantly shape the conceptualization of the Analytical Engine.”</p>



<figure class="wp-block-image size-full"><img decoding="async" width="640" height="427" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Bodleian_Library_views_35803949781.jpg" alt="" class="wp-image-30638" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Bodleian_Library_views_35803949781.jpg 640w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Bodleian_Library_views_35803949781-300x200.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Bodleian_Library_views_35803949781-18x12.jpg 18w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Bodleian_Library_views_35803949781-600x400.jpg 600w" sizes="(max-width: 640px) 100vw, 640px" /><figcaption>The Bodleian Library at Oxford University is responsible for preserving the entire Lovelace collection. @ <a href="https://www.flickr.com/people/82323317@N00">subherwal</a>, distributed under a <a href="http://creativecommons.org/licenses/by/2.0/">CC-BY 2.0 license</a></figcaption></figure>



<p></p>



<p>Yet despite this increased interest in Lovelace’s work, scholars still struggle to get a full picture of her life. This is not due to a lack of historical sources. At the <a href="https://www.bodleian.ox.ac.uk/home">Bodleian Library</a> in Oxford, there is a large collection of letters, notes, and other papers all written by and to the remarkable mathematician, providing a wealth of insights into her life. But accessing this rich collection remains a challenge. As the Lovelace archive has never been published in a full scholarly edition, researchers find it hard to conduct effective research on it. “Most scholars working on Lovelace today only have access to an incomplete and often misleading portrait of her life,” Jessica explained. “Until this gap is rectified, our knowledge of this remarkable woman will always be necessarily incomplete.” Convinced of the potential of the Lovelace archive, Jessica boldly decided to rectify this gap herself and create a fully transcribed digital version of the Lovelace archive.</p>



<h2 class="wp-block-heading">How to transcribe 14,000 pages</h2>



<p>Of course, digitising an archive on this scale is no small feat. The Lovelace archive contains around 14,000 pages, which would take a solo researcher like Jessica years to manually transcribe. Therefore, she started looking for a more time-effective method and found she had two main options. “The first was to crowd-source, which has been successfully used by digital humanities projects like the <a href="https://le.ac.uk/dickens-code">Dickens Code Project</a> ,” Jessica explained. “There are some incredible benefits that can come from this kind of public-facing model, but it also comes with some significant pitfalls.” </p>



<p>Crowd-sourcing is quite a resource-intensive method. It requires materials to be professionally digitised, it involves a lot of outreach and training and you have to cooperate with many team members of varying levels of involvement and expertise. “All these tasks cost time and money. As such, this kind of model may not be the best fit for graduate students, postdoctoral students, or others who operate outside the auspices of tenured academia.”</p>



<p>Jessica’s second option was to use software to do the transcription automatically. After much searching on academic blogs and forums, she came across the <a href="https://readcoop.eu/transkribus">Transkribus</a> handwriting recognition software. Tranksribus uses AI models to turn the text in handwritten documents into digital text, which can then be stored as a database and made searchable through tags and metadata. “By training AI models to perform much of the actual transcription work, I can use the keyword search feature to find material specific to my immediate dissertation research goals, instead of relying on guesswork for where relevant material might be located.” </p>



<p>The fact that Transkribus was developed by a team of academics who understood the process of historical archive research was also a key factor for Jessica, as were the scholarship options available to her. The <a href="https://readcoop.eu/transkribus/scholarship/">Transkribus Scholarship Programme</a> awards free credits to young researchers looking to use Transkribus for their project, and due to the academic importance of Jessica’s project, she was granted enough credits to cover the transcription of all pages. “Thanks to the scholarship, I will be able to use Transkribus to transcribe the entirety of my collection.”</p>



<h2 class="wp-block-heading">Creating the Lovelace AI model with Transkribus</h2>



<p>Jessica’s first step in the transcription process was to learn exactly what is possible with Transkribus. “I would urge anyone starting a Transkribus project to take the time to fully read the website and listen to all the recorded Q&amp;As on <a href="https://www.youtube.com/channel/UC-txVgM31rDTGlBnH-zpPjA">YouTube</a>,” she advised. There was so much relevant information available that Jessica started collating everything she read into a personal training manual for future reference. “And most importantly, I reached out to the READ-COOP team a lot with questions, and they have been very helpful in answering my queries.”  </p>



<p>Armed with her newfound knowledge, Jessica then set about training her AI model with Transkribus. So far, she has transcribed about a tenth of the collection and has been very happy with the initial results. But in some ways, working with the software was different to how she expected. “Initially, I wanted to get the Lovelace model to perform almost as well as I could read and transcribe. As my work has progressed, my view of this joint operation has shifted. While the Lovelace model sometimes misses words or letters that seem relatively easy to the human eye, many times it has recognized words that both previous scholars and I have incorrectly deciphered. This process has truly been one of collaboration, where the human and the machine bring different expertises.”</p>



<p>Training the Lovelace model correctly is particularly important in this stage of the project, as it sets the foundations for future project stages. The Lovelace archive contains not only papers from Lovelace herself but also her husband, William King, and her mother, Lady Noel Byron, which also need to be transcribed. “I will use the Lovelace model as a base model to train additional models on the handwriting of other major contributors,” explained Jessica—a strategy that will save her both time and transcription effort in the months to come, and sets the groundwork for further research into the collection after her PhD.</p>



<h2 class="wp-block-heading">A meaningful project</h2>



<p>There is a beautiful symmetry to this project. Ada Lovelace was one of the first people to suggest that the computers of the future would be able to do much more than just calculate numbers. Almost 200 years later, Jessica Cook is using computers to transcribe documents, enrich them, publish them, and by doing so, uncover more about the life of this remarkable woman. </p>



<figure class="wp-block-image size-full"><img decoding="async" width="640" height="640" src="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait.jpg" alt="" class="wp-image-30639" srcset="https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait.jpg 640w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-300x300.jpg 300w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-150x150.jpg 150w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-12x12.jpg 12w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-600x600.jpg 600w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-100x100.jpg 100w, https://s3.eu-central-1.amazonaws.com/readcoop.cis.public-assets.prod/WP/uploads/2022/10/Ada_Lovelace_Chalon_portrait-32x32.jpg 32w" sizes="(max-width: 640px) 100vw, 640px" /><figcaption>Mathematician Ada Lovelace was one of the pioneers of modern computing. @ <a href="https://en.wikipedia.org/wiki/en:Alfred_Edward_Chalon">Alfred Edward Chalon</a>, distributed under a <a href="http://creativecommons.org/licenses/by/2.0/">CC-BY 2.0 license</a></figcaption></figure>



<p></p>



<p>“Without overestimating Lovelace’s contributions to computer science, I feel that it is appropriate to say that her intellectual openness and creativity opened a small window onto the world of miraculous digital possibility that we now find ourselves in. I am still working through the full conceptual implications of this question, but on a personal level I would like to think that this Transkribus project is a proud fulfillment of Lovelace’s early insights.”   </p>



<h2 class="wp-block-heading">Jessica’s Transkribus Tip</h2>



<blockquote class="wp-block-quote"><p>To anyone thinking about embarking on a similar project: go for it. Don’t be discouraged by those who dismiss large-scale digital projects as endeavours that are too much work to be undertaken by graduate students. It is important to know your limits, but I also think that the future of digital research hinges upon scholars who focus on solutions-oriented approaches to challenges.   </p></blockquote>


Mathematics, computing, music, poetry: Ada Lovelace was a person of many talents. The 19th-century mathematician — and daughter of infamous Romantic poet, Lord Byron — is widely regarded as one of the pioneers of modern computing thanks to her work on Charles Babbage’s “Analytical Engine”

Creating a digital scholarly edition of the Lovelace papers with Jessica Cook

Leverage the power of Transkribus to get the most out of your historical documents.

Succesverhalen

Start unlocking the past with Transkribus