diff options
author | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
---|---|---|
committer | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
commit | e0c6872cf40896c7be36b11dcc744620f10adf1d (patch) | |
tree | 60335e10d2f4354b0674ec22d7b53f0f8abee672 /usergrps/uktug/baskervi/5_4/raman.tex |
Initial commit
Diffstat (limited to 'usergrps/uktug/baskervi/5_4/raman.tex')
-rw-r--r-- | usergrps/uktug/baskervi/5_4/raman.tex | 530 |
1 files changed, 530 insertions, 0 deletions
diff --git a/usergrps/uktug/baskervi/5_4/raman.tex b/usergrps/uktug/baskervi/5_4/raman.tex new file mode 100644 index 0000000000..b0e9f1c08c --- /dev/null +++ b/usergrps/uktug/baskervi/5_4/raman.tex @@ -0,0 +1,530 @@ +\def\dash{---} +\let\Dash\dash +\def \ifundefined#1{\expandafter\ifx\csname#1\endcsname\relax } +%Use to generate a new row in tables with a horizontal line separating +%them: +\newcommand{\newrow}{\\ \hline} +\newcommand{\headrow}{\\ \hline \hline} +\newcommand{\mdash}{---} +\newcommand{\lisparg}[1]{{\em #1\/}} +\newcommand{\lispname}[1]{{\sf #1\/}} +\newtheorem{theorem}{Theorem} +\newtheorem{algorithm}{algorithm} +\newtheorem{lemma}{Lemma} +\newtheorem{definition}{Definition} +\newtheorem{corollary}{Corollary} +\newtheorem{conjecture}{Conjecture} +\newcommand{\nonterm}[1]{\mbox{${\scriptstyle <}{\mbox{\em #1\/}}{\scriptstyle >}$}} +\newcommand{\bld}[1]{{\bf #1}} +\newcommand{\type}[1]{{\tt #1}} +\newcommand{\pz}{\phantom{0}} +\newcommand{\inference}[2]{\frac{#1}{#2}} +\newcommand{\induction}[2]{\frac{#1}{#2}} +\newcommand{\kronecker}{\raisebox{1pt}{$ \:\otimes \:$}} +\newcommand{\subst}[3]{{#1[#2/#3]}} +\newcommand{\id}[1]{\mbox{{\sf #1\/}}} +\newcommand\french[1]{{\it #1\/}} +\newcommand{\afl}{{AFL}} +\newcommand{\term}[1]{\mbox{\sf #1\/}} +\newcommand{\divides}[2]{#1/#2} +\newcommand{\subgroup}{\triangleright } +%slide title. +\newcommand{\itidetitle}[1]{\center \framebox{\large\bf #1}} + +%section reference. +\newcommand{\sref}[1]{Section~$\ref{#1}$} +\newcommand\cref[1]{Chapter~$\ref{#1}$} +\newcommand{\aref}[1]{Appendix~$\ref{#1}$} +%integral d +\newcommand{\varint}[1]{\,d#1} +%quantifiers: cs611 +\newcommand{\all}[2]{\forall #1\!\!:\!#2.\:} +\newcommand{\exist}[2]{\exists #1\!\!:\!#2.\:} +%integrals +\newcommand{\dx}{\,dx} +\newcommand{\dy}{\,dy} +\newcommand{\dz}{\,dz} +\newcommand{\dt}{\,dt} + +\newcommand{\naive}{na{\"\i}ve{}} +\providecommand\AmSTeX{$\cal A\kern-.1667em\lower.5ex\hbox{$\cal + M$}\kern-.075emS$-\TeX} +\providecommand{\amstex}{\AmSTeX{}} +\newcount\TestCount +\providecommand{\La}{\TestCount=\the\fam \leavevmode L\raise.42ex + \hbox{$\fam\TestCount\scriptstyle\kern-.3em A$}} +\providecommand\AllTeX{(\La)\TeX} +\providecommand{\alltex}{\AllTeX{}} + +\providecommand{\latex}{\LaTeX{}} + +\providecommand{\tex}{\TeX{}} + +\providecommand{\macro}[1]{{\cal M}_#1} + +\providecommand{\rfb}{{\sc RFB}\footnote{Recordings for the +Blind}} + +% +\newcommand{\www}[1]{{\it WWW}: {\small #1}} +\newcommand{\email}[1]{{\it E-mail\/}: $\langle\hbox{\tt#1}\rangle$} +\newcommand{\phone}[1]{{\it Phone\/}: {\tt #1}} + +\newcommand{\voicemail}[1]{{\it Voice-mail\/}: {\tt #1}} +\newcommand\homepage{\sf http://www.research.digital.com/CRL/personal/raman/raman.html} +\newcommand{\faxno}[1]{{\it Fax\/}: {\tt #1}} + +\newcommand{\textalk}{\rm T\kern -.1667em\lower .5ex\hbox {E}\kern% + -.125emXT\kern -.1667em\lower .5ex\hbox {A}\kern -.125em L\kern -.125em K} + + + + +\newcommand{\Dectalk}{{\sc dectalk}} +\newcommand{\Sparc}{{\sc sparc}} + + +\title{An Audio View of \alltex{} Documents} +\author[T. V. Raman]{T.\ V.\ Raman\\ +Digital Equipment Corporation\\ + Cambridge Research Lab\\ + One Kendall Square, Building 650\\ + Cambridge, MA 02139\\ +\emph{Email:} \texttt{raman@crl.dec.com} +} +\begin{Article} +\begin{abstract} + \aster{} \dash Audio System For Technical Readings\dash is a computing + system that produces audio renderings from the {\em same\/} \alltex{} source + used to produce the printed document. \cite{Raman:TB13-3-372-377} + described our preliminary + work on this project. At the time, correct handling of user-defined + \alltex{} macros was described as one of the key issues in building a fully + extensible audio rendering system. + \aster{} \cite{raman-phd-thesis} has now been fully implemented. + This paper reports on the approach used + in \aster{} to handle user-defined macros. + +\aster{} treats macro definitions as introducing new object types +into the document logical structure. The \alltex{} macro consists of two +parts; a declaration, and a series of \TeX{} commands that the macro expands +into. The macro expansion is nothing but a visual rendering rule that +specifies how \TeX{} should display instances of the object represented by the +macro. + +\aster{} provides an equivalent mechanism for extending the class of +logical structures that are recognized. Once \aster{} has been told about a +user-defined macro, audio rendering rules for the new object type introduced +by this macro can be defined in AFL (Audio Formatting Language). + +The approach used not only makes \aster{} fully extensible; +it points out a unique advantage of \alltex\dash the ability of the +author to encode semantic meaning into the markup by extending the document +model in ways appropriate to the specific document instance that is being +encoded. + \end{abstract} +\section{Introduction}\label{s:introduction} + + +\begin{center} + \asterlogo +\end{center} + +\aster\dash Audio System For Technical Readings\dash is a computing +system that aurally renders electronic documents marked up in the \alltex{} family of +markup languages (see~\cite{raman-phd-thesis} for details). \aster{} uses the +structural markup present in the electronic source to advantage in producing +high-quality, interactive audio renderings. This paper focuses on a specific aspect of the +problem; namely that of flexibly rendering the extended document logical +structure encapsulated in a \alltex{} document. + + +One primary advantage of \alltex{} is the flexibility it provides the author +in defining logical structures that are specific to a particular document +instance. In this sense, the class of logical structures that can be +encapsulated in a \alltex{} document is extensible. \alltex{} macros allow +an author to abstract away the layout details. At the same time, they provide +a powerful mechanism for defining new constructs that are not already present +in the document style (DTD in SGML parlance) in use. As a consequence, + when introducing a new piece of mathematical notation, an author can first define +a new \alltex{} macro that produces a desired layout, and then use this newly +defined construct throughout the document. + +The flexibility of the \alltex{} macro facility initially proved a major +stumbling block in building a fully extensible audio rendering system. A +system that attempts to produce aural renderings by {\em mapping\/} the +built-in \alltex{} commands to an equivalent aural representation faces the +severe shortcoming of not being able to render documents that contain +user-defined macros. At the same time, it is impossible to translate such +user-defined \alltex{} macros into a suitable aural representation. This is +because \tex{} in its full glory is a Turing-complete programming language, and +saying ``we can translate a general \tex{} macro to audio'' is equivalent to +saying that ``Given a \tex{} program, we can predict the result''. Being able +to achieve the above without actually running \tex{} on the program (document +fragment) would amount to being able to solve the Halting Problem! + +In the rest of this paper, we describe the solution used in \aster{} to +circumvent this difficulty. The solution we used in fact turns the presence of +user-definable \alltex{} macros into an advantage. +Such user-defined constructs allow \aster{} to glean even more information +about the document logical structure than would be possible if the document +were encoded using only the built-in \alltex{} operators; as a consequence, +the audio renderings produced are also significantly better. + +\section{Document Models in \protect\aster{}}\label{s:represent} + + +\aster{} produces audio renderings by first extracting the document +logical structure. In this model, all forms of rendering, \ie visual, +aural, etc.\ are regarded as a projection of the structure present in +the information being conveyed onto the medium being used to +communicate the information. Thus, typesetting a document requires +visual formatting\Dash projecting the information structure onto a +two-dimensional visual tablet; aural rendering requires presenting the +structure using various features of the auditory display. + +The recognizer used in \aster{} extracts logical structure present in +documents encoded in the \alltex{} family of languages. An important +feature of this recognizer is that it works on the entire gamut of +encodings, ranging from plain ASCII documents, \ie no explicit markup, +up to documents containing completely unambiguous encodings of the +logical structure. + + +The basic document model used in \aster{} is the attributed tree. +Each hierarchical level of the document is modeled as a node in this +tree. Each node can have content, children and attributes. Using +object-oriented terminology, each different kind of node of the tree +is called an {\em object\/} and represents a document element. Thus, +``chapter'', ``section'', ``paragraph'', and ``sentence'' are all +objects. If a document contained five sections, its representation in +\aster{} would have five instances of object ``section''. This +object-oriented terminology is used because \aster{} actually uses +CLOS objects in this fashion. The use of an object-oriented language +was instrumental in allowing us to develop and implement the ideas in +\aster{} incrementally and effectively. + +This attributed tree +structure is augmented to represent mathematical content; we call this +augmented representation the {\em quasi-prefix form}, +(see figure~\ref{fig:math-object} below). +Expressions that are completely unambiguous, \eg $x+y$, are captured in their +prefix form. In addition to linearizing the underlying tree structure, +mathematical notation uses {\em visual attributes\/} such as superscripts and +subscripts, whose interpretation is context-dependent. We extend the prefix +form to capture such visual attributes\Dash hence the name {\em + quasi\/}-prefix. +\begin{minipage}{\linewidth} +\makeatletter\def\@captype{figure}\makeatletter + \begin{center} +\begin{tabular}[h]{|rcl|}\hline +left-superscript & accent & superscript \\ + &$\displaystyle \nwarrow$ \hfill + $\displaystyle \uparrow$ + \hfill $\displaystyle \nearrow$ & \\ +& {\bf math object } & \\ + & $\displaystyle \swarrow$ \hfill + $\displaystyle \downarrow$ + \hfill $\displaystyle \searrow$ & \\ +left-subscript & underbar & subscript \\ \hline +\end{tabular} +\end{center} +\caption{A math object with attributes. Each of the attributes + themselves contain math objects.} + \label{fig:math-object} +\end{minipage} + +The next section describes how this model is extended to encapsulate the use +of user-defined constructs in \alltex. +\section{Extended Logical Structure}\label{s:macros} + +The \alltex{} facility can be used to extend the document logical structure by +defining new constructs. Thus, an author preparing a manuscript on inference +logic might define +\begin{verbatim} +\newcommand{\inference}[2]{{#1\over#2}} +\end{verbatim} +\noindent and write +\begin{verbatim} +\inference{x}{y} +\end{verbatim} +\noindent and use this construct throughout the document. + +Notice that defining the \verb|\inference| as shown above and using it to +encode inference statements is distinct from and more powerful than just using +the \tex{} built-in operator \verb|\over| throughout the document. +A commonly mentioned advantage in this context is that using the newly +defined construct \verb|\inference| will permit the author to easily change +the notation used to denote {\it inference}. +Notice, that this is in fact the same as saying that +\begin{quote} + If distinct elements in a document instance are marked up using distinct + constructs, then it is possible to recognize and process these elements + in a multiplicity of ways. +\end{quote} +In \aster, the \alltex{} facility of defining a second \verb|\inference| macro +that produces a different layout for {\it inference\/} can be generalized to +the notion of different {\em audio renderings\/} for {\it inference}. + + + As explained above (``Document models''), \aster{} achieves its + aural renderings by building a rich internal representation of the + document content. In this representation, each document + element\footnote{We use the term {\em element\/} loosely to mean a + logical unit of the document. } $E$ is represented by an instance + of object $O_E$. \aster{} provides a predefined type $O_E$ for each + of the built-in constructs in \alltex. Thus, we could represent the + use of \verb|\inference| defined above in terms of object $O_{\rm + over}$. However, notice that this would mean losing valuable + information. When building up the internal representation, the + additional semantic information provided by the author's use of the + \verb|\inference| construct is very useful. In addition, expanding + all \alltex{} macros results in a pure layout representation, which + is not appropriate for producing aural renderings + (see~\cite{Raman:TB13-3-372-377}). If we were to represent + instances of \verb|\inference| in terms of $O_{\rm over}$, \aster{} + would be forced to render \verb|\inference| the same as the + \verb|\over| construct. Though the author in this particular + example may have chosen to use the same visual rendering for + inferences that is normally used for fractions, the same may not + carry over well to the aural domain. + + +\subsection*{Representing Extended Logical Structure}\label{s:extend} + + + \aster{} solves the problem of representing and rendering the + extended logical structure arising from user-definable macros by + considering each macro definition as introducing a new object type. + Instances of a macro $M$, are represented by instances of object + $O_M$. Thus, in the example shown above, the definition of the + construct \verb|\inference| introduces a new object type $O_{\rm + inference}$. The \alltex{} macro consists of two parts; a + declaration, and a series of \TeX{} commands that the macro expands + into. The macro expansion is nothing but a visual rendering rule + that specifies how \TeX{} should display instances of the object + represented by the macro. + + +\aster{} provides an equivalent mechanism for extending the class of logical +structures that are recognized. Once \aster{} has been told about a +user-defined macro, audio rendering rules for the new object type introduced +by this macro can be defined in AFL (Audio Formatting Language). Notice that +such audio rendering rules have to be defined by the user, just as the +\alltex{} macro is defined by hand. It is not possible in general to translate +the \tex{} macro into a set of audio rendering rules. This is because the +\tex{} macro is capable of performing any arbitrary computation permitted by +the operators present in the \tex{} language \cite{knuth84}\dash a +Turing-complete programming language. +\section{Rendering Information}\label{s:rendering} +\aster{} renders information by applying {\em rendering rules\/} to the +internal representation described above (``Document models''). + The system of rendering rules used in \aster{} +and the language in which they are written (AFL\dash Audio Formatting +Language) are described in detail in~\cite{raman-phd-thesis}. In a sense, AFL +is to audio formatting as Postscript is to visual formatting, although AFL is +a much smaller language. + +Here, we show a +small example of such a rendering rule for a user-defined macro. In the +following, we use \term{CLOS} generic function \term{read-aloud}. For the +present, let us assume that function \term{read-aloud} executes the necessary +actions to render its argument. + + + After extending \aster{} to process the \alltex{} macro + \verb|\inference| shown above (``Logical structure''), we can define + +{\small +\begin{verbatim} + (defmethod read-aloud((inference inference)) + "Sample rendering for object inference." + (read-aloud (argument 1 inference)) + (read-aloud "implies") + (read-aloud (argument 2 inference))) +\end{verbatim} +} +\noindent Given $\inference{A}{B}$, this produces ``A implies B''. + +If we wished to produce a rendering that inverts the order in which the +arguments to macro \verb|\inference| are rendered, we would define: + +{ +\small\begin{verbatim} + (defmethod read-aloud((inference inference)) + "Renders inference with arguments reversed." + (read-aloud "We know") + (read-aloud (argument 2 inference)) + (read-aloud "because") + (read-aloud (argument 1 inference))) +\end{verbatim} +} +\noindent which produces ``We know B because A''. + +Switching between these two rendering rules has the effect of inverting a +proof-tree! +Notice that writing a new rendering rule for an object $O_E$ has the same +effect as redefining the \alltex{} macro that corresponds to $E$. + +\aster{} makes it easy to write several rendering rules for the same object +and also allows rendering rules to be partitioned into rendering {\em + styles}. Such {\em styles\/} can be thought of as being analogous to +\latex{} styles, but with one important difference. Due to the +non-interactive nature of traditional paper documents, a paper is typically +typeset in a given style. It is not possible for the reader to change the +style in which the document is typeset. +Typically, we do not feel the shortcoming of not being able to change the way +a mathematical expression is rendered when reading a printed paper because the +eye is capable of reading the various parts of an expression in any order that +is convenient. However, when listening to an aural presentation, the listener +does not have this flexibility. In other words, an active reader peruses a +printed paper, a passive display, whereas in the case of audio, these roles +are reversed\dash the aural display scrolls {\em actively\/} past a passive +listener. + +\aster{} overcomes these difficulties by being a fully interactive system. +It is possible for the listener to interrupt the rendering, change the +rendering style in use, and listen to the document. In an interactive session +with \aster{}, switching between rendering styles (a collection of rendering +rules for different objects) and invoking individual rendering rules can be +done with a few keystrokes, making it easy for a listener to obtain many +different views of a document. +This facility enables {\em active\/} listening. + +\aster{} derives its power from representing document content as objects and +by allowing multiple user-defined rendering rules for individual object types. +These rules can cause any number of audio events (ranging from speaking a +simple phrase, to playing a digitized sound). The pitch of the voice, the +physical head-size of the virtual speaker, the volume, and many other +parameters can be changed by rendering rules, making it easy to create sound +cues to help display structure. +In fact, the design of \aster{} does not restrict the system to producing +purely aural renderings; there is nothing to preclude us from defining +renderings that produce truly multimodal output; \ie renderings where the +traditional visual rendering is augmented with aural feedback. We conjecture +that such multimodal renderings may prove very useful for persons with +learning impairments. + +To give an example of a multimodal rendering, the logo for \aster{} is +\begin{center} + \asterlogo{} + \end{center} + \noindent and is produced by \alltex{} macro \verb|\asterlogo|. + After appropriately extending \aster{} to recognize this macro, we + can define an audio rendering rule for object {\em asterlogo\/} that + produces a bark when rendering instances of this macro. Thus, the + same piece of markup \verb|\asterlogo| produces the picture of + Aster\footnote{Aster is my guide-dog. } when rendered visually, and + an appropriate sound\footnote{The bark is that of a generic dog, + Aster is too well trained to bark, and could not therefore be + recorded.} when rendered aurally. + + + This feature was exploited to advantage when producing the audio formatted + version of the author's thesis. The dedication page of the thesis contains + a large picture of Aster, and the audio formatted version\footnote{An audio + formatted version of the thesis produced by \aster{} (about 6 hours) is + being distributed by RFB\dash Recordings For The Blind\dash as the first fully + computer-generated talking book. } contains a verbal description of the + picture, accompanied by the sound of Aster panting in the background. You + can listen to this example on the WWW\dash visit the \aster{} home + page by following the link to the \aster{} demonstration + from my home + page\footnote{\URL|http://www.research.digital.com/CRL/personal/raman/raman.html|} + and clicking on the picture of Aster. + +Several ideas come together to make all this possible. First, logical +structure is of paramount importance\dash not its display on any one +particular medium. The more a document makes structure explicit, +the better the document can be displayed on (projected onto) several +different media. + +Next, the use of \alltex{} macros to encode structure makes it +possible to have a system like \aster, in which the internal +structure can be extended to fit a document. This allows the encoding +of the structure in a flexible, uniform, and consistent representation +such as an attributed tree, with the addition of the quasi-prefix form +for dealing with mathematics. + +Finally, providing different rendering rules and styles and a +flexible way to switch among them makes it possible to obtain multiple +views of a document in an interactive fashion. + +\section{Conclusion}\label{s:conclusion} + + + The approach used in \aster{} to exploit the additional semantic + information present in the electronic encoding in the form of + user-defined constructs points to an important feature of markup + systems like \alltex{} that is currently missing to a certain extent + in systems like SGML. When \aster{} was at its inception, I firmly + believed that one should use a semantic-oriented DTD to encode a + document in order to be able to produce high-quality audio + renderings. I still believe this; however the work on \aster{} does + point out one shortcoming with the fixed document DTD model. Given + that mathematical and technical notation is being invented all the + time, a fixed DTD forces the author to encode new constructs using + {\em only\/} primitives that are provided by the DTD. As a + consequence, authors end up using a presentation-oriented encoding + even though the DTD in use is one that is semantically oriented. + + + To make this concrete, consider the case of the {\it inference\/} + construct described above (``Logical structure''). If the document + were being encoded using a fixed non-extensible DTD that only + provides a {\it fraction\/} element, the author would be forced to + encode {\it inference\/} using this element. + +Since in general it is not possible to define an all-encompassing DTD that +covers every possible kind of math notation (those currently known and those +yet to be discovered) extensibility of the DTD as provided by \alltex{} is of +vital importance. + +Another good example of this facility in \alltex{} being put to good +use is the Hyper\tex{} system \mdash an extension to \tex{} that +allows the user to view his legacy \alltex{} documents as online +hypertext. Conceptually, we can think of \verb|\ref| and +\verb|\label| as being object types; traditionally, these cause +specific marks to appear on paper when rendered visually by \tex; to a +system like Hyper\tex{} these turn into {\em active\/} links that a +user can follow interactively. + +The ability to produce multiple renderings of the same object provided by +\aster{} was introduced in the context of aural presentations. However, such +multiple presentations become equally relevant when interactively perusing +online documents visually. For instance, when reading a document that +presents a complex proof, a user may wish to have the same proof displayed as +an outline in one window, and as a proof-tree in another +(see~\cite{lamport:proofs93}). In the case of paper documents, the user has +to use her imagination to achieve such multiple views \mdash though she is +aided in this by the visual notation. + In the interactive scenario presented by electronic +documents, the previewer can provide some additional functionality to aid in +this process. + +\begin{thebibliography}{} +\bibitem[Knuth 1984]{knuth84} +Knuth, D.~E. +\newblock {\em The \TeX{}book}, volume~A of {\em Computers and Typesetting}. +\newblock Addison-Wesley, Reading, Massachusetts, 1984. + +\bibitem[Lamport 1993]{lamport:proofs93} +Lamport, L. +\newblock ``How to write a proof''. +\newblock Technical Report~94, DEC Systems Research Center, Palo Alto, {CA}, + 1993. +\newblock To appear in {\em American Mathematical Monthly}. + +\bibitem[Raman 1992]{Raman:TB13-3-372-377} +Raman, T.~V. +\newblock ``An audio view of \TeX\ documents''. +\newblock {\em TUGBoat} {\bf 13}(3), 372--377, 1992. + +\bibitem[Raman 1994]{raman-phd-thesis} +Raman, T.~V. +\newblock {\em Audio System for Technical Readings}. +\newblock Ph.D. thesis, Cornell University, 1994. + +\end{thebibliography} + +\end{Article} |