The "TUebingen System of Text Processing Programs" TUSTEP, developed at the University of Tuebingen Computing Center since the late 60es, is a professional toolbox for scholarly processing textual data. Long-term availability, platform independence for procedures and data, coverage of all steps of a typical humanities research project, flexibility based on a consequent modular design are important design principles.

The work began in 1966 when we first designed a series of functions and subroutines for character and string handling in FORTRAN (compatible, in their first version, to those previously developed by the Deutsches Rechenzentrum in Darmstadt) and implemented them on the mainframe of the Computing Center of the University of Tübingen. This made programming easier for projects such as the Metrical Analysis of Latin Hexameter Poetry, the Concordance to the Vulgate, or the edition and indexes to the works of Heinrich Kaufringer.

Proceeding from the experiences gained from those projects, the next step in supporting projects was to no longer rely on programming in FORTRAN or other "high level" languages, but to provide a toolbox consisting of several program modules, each covering one "basic operation" required for processing textual data. The function of each program is controlled by user-supplied parameters; the programs themselves may be combined in a variety of ways, thereby allowing the user to accomplish tasks of the most diversified kind. It was in 1978 when these programs got the name TUSTEP.

We have chosen the term "textdata processing" in order to distinguish between TUSTEP's prime field of application and what is commonly understood by the term text processing or word processing. Naturally, TUSTEP is also equipped with the same functions needed for preparing documents (such as input, editing, formatting, printing of texts, also those in non-latin alphabets); these functions are required for the documentation and for the preparation of publications in all fields of scholarly work, including both humanities and sciences. However, TUSTEP has been developed in particular to serve those academic fields where the texts themselves are the object of scholarly research: philology, literary studies, linguistics, historical sciences, librarianship: i.e. fields of research where not only new texts are to be produced and published as the result of scholarly work, but where existing texts (including literary texts and historical sources) are to be preserved for the future in the form of new critical editions, are to be analyzed in terms of language, style, contents, or are to be catalogued in bibliographical form.

The basic operations required by those tasks include:

• Automatic collation of different versions of a text;
• text correction not only by using an editor, but also in batch mode by means of correction instructions prepared beforehand (by manual transcription, or by program);
• decomposing texts into elements (e.g. word forms) according to rules provided by the user;
• building logical enities (e.g. bibliographic records) consisting of more than one line of text;
• sorting such elements or entities (according to non-latin alphabetical rules and other sorting criteria as well);
• preparing indexes by building entries from the sorted elements;
• processing textual data by selecting records or elements, by replacing strings or text parts, by rearranging, completing, compressing and comparing text parts on the basis of rules and conditions provided by the user, by retrieving numerical values which are already given in the text (like calendar-dates) or which can be derived from it (such as the number of words in a paragraph);
• transforming textual data from TUSTEP files into file formats used by other systems (e.g. for statistical analysis or for electronic publication).

The tasks which can be accomplished with the help of TUSTEP range from composing a brief seminar paper to preparing extensive bibliographies, lexica, indexes, concordances, dictionaries, critcal editions and of course monographs; the final output can be formatted for fotocomposition in a quality one is accustomed to in letterpress printing, or can be prepared in a form (e.g. XML, HTML) and encoding (e.g. Unicode) which is required for electronic publishing.

In addition to programs for the aforementioned textdata processing operations, TUSTEP features all necessary organizational functions such as file handling and defining new commands, functions which are normally covered by the job control language (JCL) of the respective operating system (OS). Thus, an identical user interface independent of the computer and its OS is provided. This not only saves the user the trouble of having to relearn when he switches to a computer with a different operating system, but also allows him to adopt existing TUSTEP command sequences unchanged.

TUSTEP is constantly being improved and expanded in order to facilitate solutions for new problems in the field of scholarly textdata processing and to take advantage of new developments of hardware and operating systems. Recently added features include a CGI interface and improved support for SGML / TEI / XML markup.

The following list contains a selection of TUSTEP programs for the basic operations of textdata processing and of organizational commands. The names in square brackets are the names of the commands.

• EDITING
  Entering, modifying, replacing and searching textdata on the screen using the interactive editor [#EDIT]
  Entering, modifying, searching, automatic checking of structured data by means of on-screen forms (including those displayed by www-browsers) defined by the user [#MACRO]
  Automatic correction of textdata with previously defined correction instructions [#CORRECT]
   
  
• COMPARING
  Comparing different versions of a text; listling and storing the differences [#COMPARE]
  Listing, in synoptic lines, the basic text and the differences contained in other versions of the same text [#COLLATE]
   
  
• PROCESSING TEXT
  Selecting, substituting, rearranging, supplementing, compressing and comparing text parts based on given rules and conditions; performing mathematical calculations using numbers (including calendar dates) which are either already given in the text or can be derived from it; output in various formats (including those required for electronic publishing or for subsequent processing outside of TUSTEP) [#COPY]
  Replacing short forms contained in a text by full text (words, lines, passages) located in a file and identifiable by corresponding short forms [#INSERT]
  Maintaining and updating cross-references [#NUMBER]
   
  
• PREPARING INDEXES
  Preparing index entries by decomposing text units into their elements, or by extracting marked text parts; if necessary, supplementing and modifying text parts; adding the reference to the source text; distinguishing between different types of entries; adding sort fields reflecting freely definable sort criteria and sort alphabets the reference; distinguishing between different types of entries [#PINDEX]
   
  
• PRESORTING
  Creating sort units by combining text parts which are logically related; adding sort fields which are needed to determine the sequence of the text units in the subsequent sorting and which reflect the sort criteria (selection and sequence of certain parts of the text units), the sort values for any string of characters and the sort alphabets [#PRESORT]
   
  
• SORTING
  Rearranging data records in ascending or descending order according to the sort criteria contained in the sort fields [#SORT]
  Merging of presorted files [#MERGE]
   
  
• GENERATING INDEXES AND CONCORDANCES
  Reducing multiple and perhaps hierarchically structured index entries or text units after sort has been run; supplementing and substituting text parts and references, thereby distinguishing between different types of entries; calculating absolute and relative frequencies [#GINDEX]
   
  
• GENERATING LISTINGS
  Preparing output for printing
  - in the form in which the data are recorded in the file; control characters (tags) are not interpreted but printed [#GLISTING]
  - in a format and an arrangement which can be defined by control characters (tags) contained in the text, using the entire inventory of typefaces and characters available for the selected printer; with automatic hyphenation and line division as well as other page layout features, including line justification and footnote placement [#FORMAT]
  - for forms (e.g. address stickers, catalogue cards, standardized letters, office forms) [#GFORMS]
   
  
• TYPESETTING
  Transforming textdata into typographic codes for output on PostScript printers or (for professional composing) on a typesetting device, including automatic line division (with or without line justification and tabular settings) and automatic page makeup with running titles, headings, text, insertions in smaller type, graphs and pictures, marginal notes, footnotes and up to nine critical apparatuses at page bottom; variety of typefaces and special characters available [#SATZ]

• FILE MANAGEMENT
  Creating, opening, closing, renaming, deleting files [#CREATE, #OPEN, #CLOSE, #RENAME, #ERASE]
   
  
• DATA TRANSFER
  Import of textdata from other programs into TUSTEP files (e.g. ASCII files) and conversion to TUSTEP format and vice versa [#CONVERT]
   
  
• ARCHIVING, BACKUP
  Archiving files (also different versions of the same file) in a "tape file" (which can also be used for compatible data transfer between computers having different operating systems); listing the contents of a tape file [#MTREAD, #MTWRITE, #MTCOPY, #MTINFORM, #MTLABEL, #MBTEST, #*MBUPDATE]
   
  
• JOB CONTROL
  Executing and controlling sequences of commands and programs; defining and executing user-defined commands ("macros") [#EXECUTE, #MACRO]