Newsletter 2021/1

2021-02-02, Andreas
Tags: ,


We’re happy to send our first newsletter covering the past year of activities and developments within e-editiones and would like to encourage institutions or individuals who are not yet members to join forces.

e-editiones as a Society

The e-editiones society was founded on 4 May 2020 by more than 20 international partners. Our institutional members are academies, archives and libraries, edition projects and their sponsors, as well as companies. Among individual members we are proud to count archivists, librarians, editors, researchers from the (digital) humanities, and software developers. After our first attempt to obtain a non-profit association status from the St. Gallen tax office failed, e-editiones was recognized as a non-profit association with tax exemption, following an amendment to the statutes (5th November 2020). A huge thanks to the Karl Barth Foundation for generously sponsoring the necessary legal advice. Building on these foundations and developments and events listed below, our still young society is off to a great start. We hope to broaden the community and increase our membership in 2021.

More Information

2020 Events

e-editiones organized various meetings and workshops in 2020 for members and the wider community to get to know each other and exchange ideas.

Meetings

  • 04.05.2020: Virtual foundation of e-editions and first community event for members
  • 18.05.2020: First Public Community Event: Presentation of e-editiones by Andreas Kränzle with a short introduction to TEI Publisher for digital editions by Wolfgang Meier, discussion moderated by Joe Wicentowski Video
  • 20.10.2020: TEI «Vanilla» by Magdalena Turska Report

Workshops

  • Three-part online Course: Learn TEI Publisher by Wolfgang Meier
    • Learn TEI Publisher 1: «Stay Home Learn TEI Publisher From Scratch» , 8th Jun 2020 Video
    • Learn TEI Publisher 2, 15th Jun 2020 Video
    • Learn TEI Publisher 3, 22nd Jun 2020 Video
  • Music is in the air – MEI and TEI Publisher by Giuliano Di Bacco and Dennis Ried, 8th Jul 2020 Video
  • Common eXist-db/TEI Publisher. Deployment Scenarios by Olaf Schreck and Lars Windauer covers different deployment scenarios of eXist-db and TEI Publisher, 9th Sep 2020 Video
  • Open Source Advent: Presentation of a number of open source developments, 11th – 31st Dec 2020 Link

More Information

Communication

The association immediately set up communication facilities for its members and the community. For technical questions about TEI Publisher, the central communication channel is Slack. Recurring issues and questions are incorporated into the new FAQ page. For announcements and important messages, we use Twitter. Follow us!

More Information

TEI Publisher Developments: Versions 6 and 7

In 2020 two major versions of TEI Publisher were released: versions 6 and 7. The plan for these versions was born during a meeting in Basel in November 2019, when several Swiss edition projects using TEI Publisher came together to discuss how the software could be developed into an even more sustainable platform. A number of them later became founding members of e-editiones. Their primary goal was to enable institutions to manage and maintain a larger number of editions, either on the same or several servers. To reduce maintenance costs, keeping editions up to date with newer TEI Publisher versions needed to be as simple as possible. Editions relying on different versions needed to be able to run alongside each other.

Two areas had to be addressed to achieve these goals:

  1. On the client side TEI Publisher was already a collection of small, modular web components, which together formed the user interface. These components had to be extracted into a separate library, so projects could benefit from bug fixes and new components without having to update the rest of the edition’s application.
  2. On the server side TEI Publisher needed a well defined API. Such an API would provide a standardized communication channel, through which the web components on the client could talk to the server. To accommodate future updates and support backwards compatibility, the API would need to be versioned, ensuring that both sides could speak the same language.

Fortunately the Swiss Nationale Infrastruktur für Editionen – Infrastructure nationale pour les éditions agreed to support the outlined ideas, funding a substantial part of the work.

With the release of TEI Publisher 7, both areas mentioned above have been addressed and the goal has been fully reached.

Contributions also came from member institutions: the Collection of Swiss Law Sources Online and the St. Galler Missiven were built on an early version of TEI Publisher 4. Both funded a major update of their code to version 7. A number of regressions and new bugs were encountered during the migration, and appropriate fixes were contributed back into the TEI Publisher code base.

The Karl Barth Gesamtausgabe likewise served as a test bed during TEI Publisher 7 development. Furthermore, the Karl Barth project financed the Microsoft Word to TEI transformation, which is now part of TEI Publisher.

The source code of TEI Publisher and its related repositories have been moved under the roof of e-editiones and are hosted within its GitHub organization.

Other Software Packages

Beyond TEI Publisher itself, e-editiones released a number of other software packages:

  • The Open API routing library at the heart of TEI Publisher 7, called Roaster, proved to be useful even outside the humanities and has already been adopted by a project in the medical realm.
  • Born out of the Karl Barth Gesamtausgabe, e-editiones published an extension for the Visual Studio Code editor to help editors work on TEI. This includes an entity explorer to help with the markup of entities in a text by querying external authority databases from within the XML editor.
  • The Cross Search package provides a blueprint for a portal incorporating multiple editions, supporting cross-edition search facilities. This will become the basis for future hosting services.

Community Contributions

Last but not least, we would like to point out the many contributions we received with respect to translations: TEI Publisher has now been translated to more than 20 languages!

We hope to see similar interest in other areas, e.g., the newly established FAQ website. We would like to encourage everyone to add entries and turn this into a knowledge base for all users. It already hosts questions and answers concerning encoding, workflows, or best practice recommendations.

The Future

Continuing our cooperative development model, the following new components and features are scheduled to appear in the forthcoming minor version of TEI Publisher:

  • A component for displaying mathematical formulae, sponsored by the Bernoulli project (Uni Basel).
  • A feature to automatically register persistent DOI identifiers with every resource uploaded to TEI Publisher (financed by the DIPF).

e-editiones is actively apply for funding: a first proposal has been sent to a Swiss foundation, and we’re waiting for feedback. The features we hope to obtain sponsorship for include:

  • Better navigation within an edition by allowing arbitrary identification schemes for resources.
  • Speaking and bookmarkable URLs.
  • Improved accessibility.
  • A timeline component.
  • Components to improve the editorial workflow:
    • Git integration into the user interface for synchronization.
    • An annotation feature allowing editions to perform semantic markup tasks directly on the rendered text.

If you have other features on your list, please do not hesitate to propose them. We can achieve more at lower costs if we all work together. If you need a certain feature and plan to apply for funding, why not ask other projects which may have similar requirements? e-editiones is a welcoming umbrella under which your project can come together with other projects from different academic disciplines, coordinate needs, and potentially convince funding agencies to look beyond the perspective of a single project.

1 Comment on Newsletter 2021/1

New Visual Studio Code extension for TEI Publisher

Enhancing TEI documents with semantic markup, i.e. encode places, people or terms, can be a time consuming and tedious task. For the Karl Barth edition project, we have thus created a new extension for Visual Studio Code (vscode) and published it as Open Source.

Where it all started: a Word plugin

Within the Karl Barth edition, texts to be published in new volumes are encoded in Microsoft Word and transformed to TEI using TEI Publisher’s powerful docx2TEI conversion feature based on the TEI processing model.

To simplify semantic encoding, eXist Solutions created a Word plugin for the project, which allows editors to easily encode entities occurring within the text: select a string representing an entity, click a button and the plugin will run a query against the database of the Karl Barth Gesamtausgabe, showing potential matches for each possible category.

With another click, the selected entity is inserted into the Word document, including a reference to the entity in the database. This proved to be a huge timesaver: what took minutes before – switching to a different window, doing a search and copying the ID – is now done in seconds. A screenshot of the Word plugin is shown below:

MS Word screenshot

The plugin also allows editors to get a preview of how the current text would look like in TEI and how it would be rendered to HTML by TEI Publisher’s ODD-based transformations. The preview feature sends the underlying Word XML to a TEI Publisher instance, which transforms it first to TEI, and then – if requested – to HTML. Editors can thus see the text as it would later appear on the website.

Porting it to Visual Studio Code

However, the Word-workflow only covers the new volumes: we still have 54 already published volumes in TEI, which need to be extended. Parts of this is done automatically (via entity detection), but there’s still a lot of manual editing involved. Thus the question arised if the Word plugin could be ported to an XML editor.

For sure, oXygen would have been the obvious choice, but since the Word Plugin was already written in javascript/typescript, we decided to give Visual Studio Code a try. It turned out to be rather straightforward and the first usable version was done within one evening. Thrilled by the success, we invested two more days to make the plugin more generic, in particular, add connectors for other authority databases. Beyond the Karl Barth Gesamtausgabe, we now have connectors for the German GND, metagrid, Geonames and Google places.

Enity lookup screencast

The TEI snippet to be inserted for each of the entity types can be configured in the settings. For example, you could replace <persName> with <name type="person"> if you like.

Previews

The HTML preview feature has been ported as well: pressing a keyboard shortcut or clicking the toolbar icon will present you with a list of ODDs available for transformations, and then open a new editor panel showing the transformation result in HTML. All you need is a TEI Publisher 7 instance which has a matching ODD installed. The plugin uses the general API introduced with TEI Publisher 7, which will also be exposed automatically by any application generated from it.

HTML preview screencast

Snippets

Finally, it is rather straightforward to create reusable snippets in vscode. To start with, we implemented the one shortcut we use most in oXygen and miss everywhere else: wrap the current selection with an element (the Scholarly XML extension described below also has this, but it doesn’t hurt to have it twice in case you’re not using that).

And since we have to encode a lot of foreign texts, we also added one for <foreign>. Doing more would be easy: maybe you have some suggestions? Just write them into our github issues.

Combine with other extensions

If you combine this with other extensions available on the vscode marketplace, you get rather decent XML editing support – not as sophisticated as with oXygen, but good enough for some serious work.

For schema-based validation and context sensitive suggestions one can choose between two extensions: Scholary XML is lightweight and provides nice suggestions with documentation. It only understands relax-ng though and does not support catalogs, which means you need to reference the schema via a processing instruction in every TEI file. The second option, XML Language Support by Red Hat, supports XML schema and catalogs, which you can use to associate the TEI namespace with a schema once and for all. On the downside, it uses more system resources as it starts a background Java process.

To set up a catalog, just create a file catalog.xml somewhere with e.g. the following content and change the vscode configuration property xml.catalogs to point to it:

<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">
  <uri
      name="http://www.tei-c.org/ns/1.0"
      uri="https://tei-c.org/Vault/P5/current/xml/tei/custom/schema/xsd/tei_all.xsd" />
</catalog>

Finally, if you use TEI Publisher as a basis for your own editions, you definitely want to install the Language Server and Client for XQuery/eXistdb. It supports working on XQuery files, syncing your local changes with the database, installing XAR files and more.

Get it

The extension is available from Microsoft’s Visual Studio Marketplace. To install it in your own Visual Studio Code, just open the Extensions tab and search for tei-publisher-vscode.

All source code is available on the e-editiones github and we would like to invite others to further extend it, e.g. by adding connectors to other authority APIs. Connectors are simple javascript/typescript classes with just two methods. See the existing connectors for inspiration.

If you have questions, suggestions or would like to discuss further extensions: you’ll find us in the e-editiones slack rooms.

2 Comments on New Visual Studio Code extension for TEI Publisher

TEI “Vanilla” meeting summary

Following the spontaneous exchange on e-editiones Slack channel, we have held an online community meeting to discuss the emerging TEI Vanilla proposal on October 20th. It was a pleasure to see such a broad interest and diverse representation of participants, spanning the entire range from TEI newcomers to current and former TEI Council and Board members.

Continue reading “TEI “Vanilla” meeting summary”

1 Comment on TEI “Vanilla” meeting summary