summaryrefslogtreecommitdiff
path: root/web/funnelAC/hackman/h_ch4.tex
blob: b4df3706a876772db09885e0cc81d6b1ff115725 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
%==============================================================================%
%                              Start of Ch4.tex                                %
%==============================================================================%
%
% Copyright
% ---------
% Copyright (C) 1992 Ross N. Williams.
% This file contains a chapter of the FunnelWeb Hacker's Manual.
% See the main TeX file for this manual for further information.
%
%==============================================================================%

\chapter{FunnelWeb Future}
\label{chapfuture}\xx{FunnelWeb}{future}

Previous chapters have explained the design of FunnelWeb and given reasons
for why it is the way it is. This chapter goes through each of the
aspects of FunnelWeb, and explores their problems, how they can be solved,
and lists opportunities for improvement.

\section{Documentation}
\x{documentation}

\thing{An official example:} A official example program written using
FunnelWeb should be constructed and made available.\xx{official}{example}

\thing{Index program:} In order to typeset the documentation, a portable
index sorting program is required. One should be written and added to the
distribution kit.\xx{index}{program} Perhaps this could be the official
example!

\section{Command Line Interface}
\xx{command}{interface}

\thing{Buffer length:} Currently the FunnelWeb shell uses the
\p{COMLINE\_MAX} constant to size its internal command line buffers.
This is untenable. The maximum length of a shell command line should
not be machine dependent.

\thing{Antiquated Features:} As the FunnelWeb language develops, it is likely
that some changes will have to be made that will render one or more
language constructs obsolete. When this happens, it may be necessary to
add a command line option that has the power to turn on and off warnings or
errors flagging antiquated features.\xx{antiquated}{features}

\section{Shell Interpreter}
\xx{shell}{interpreter}\xx{command}{interpreter}

\thing{SETALL command:} When writing FunnelWeb scripts, it is sometimes
desirable to set \i{all} of FunnelWeb's options\x{options}
to some particular value
so that the script is not vulnerable to changes in FunnelWeb's default
values which might occur from time to time. To this end, it may be worth
creating a \dqp{SETALL} command that is identical to the \dqp{SET} command
except that it will generate an error if the value of an option is not
specified explicitly.\xx{setall}{command}

\thing{Recursion test:} A test should be added to test for recursive
invocation in a shellscript.\xx{recursion}{test}

\thing{Diagnostic counting:} The code for counting diagnostics in the
script interpreter is rather messy and perhaps even buggy. It should be
cleaned up and commented.\xx{diagnostic}{counting}

\thing{Argument counting:} In the \p{command.c} module, there is a variable
that counts the arguments to a command. Currently it takes the value of
the number of parameters including the command verb. This has turned out
to make the code less readable. It should be changed to be the number of
arguments to the command verb.

\thing{Make facility:} It may be worth building some sort of make facility
into the script language so as to support machines such as the Macintosh
that do not already have this facility.\x{make}

\thing{Signature file:} One problem with using FunnelWeb in conjunction with
an external \p{Make} facility is that a user might change a FunnelWeb source
file without making changes that will affect the files that it generates.
If FunnelWeb is then run and the \dqp{+D} option is on, then the output
files will be deleted (to avoid further \p{Make} propagations).
If \p{Make} then has a production linking the \p{.fw} file to its output
files, then it may conclude that the output files are still out of date.
To solve the problem, FunnelWeb could be changed to write a \p{.sig} file
whenever it processes a \p{.fw} file. The \p{Make} production could then
be wired up to link
the \p{.fw} file to the \p{.sig} file instead of to the output
files.\xx{signature}{file}
  
\section{Language Design}
\xx{language}{design}

Some proposed changes to FunnelWeb do not correspond to
any particular component of FunnelWeb
and are really to do with the design of the input language.

\thing{Output or file?:} The \dqp{@O} special sequence for defining an output
file is somewhat non-mnemonic and can be confused with zero (\dqp{0}). Perhaps
it should be replaced by the \dqp{@F} sequence.

\thing{Syntax of section names:} Currently section names use
the same syntax as macro names. For example \dqp{@<Sloth@>}. It can be
argued that angle brackets should be reserved only for macro names and that
some other syntax should be found for delimiting section names. This is not
a clear issue. It could also be argued that they are both names, and that
because sections can inherit their names from the macros they contain, that
the names are of the same \dq{type}.\xx{section}{syntax}

\thing{One macro per section:} One particular style of using FunnelWeb is
to have at most one macro definition per section. It may be worth adding
a pragma that instructs FunnelWeb to enforce this.

\thing{Should @\{ suppress EOL?:} When defining a macro in FunnelWeb, it
seems to be rule rather than the exception that the
\dqp{@\{} be followed by \dqp{@-}. This suppresses the EOL on the definition
line, allowing the first line of the macro to be placed immediately
above and in line with the other
lines without introducing an EOL at the start of the macro text.
One option is to introduce a pragma to determine whether to suppress EOLs
following \dqp{@\{}.

\thing{Pragma syntax:} It is not clear how \dq{loose} the syntax of pragmas
should be. Perhaps they should be case insensitive.\xx{pragma}{syntax}

\thing{Conditionals:} Depending on demand, it may be worth reintroducing
some sort of conditional feature into FunnelWeb. However, it would have to
be very simple to compete with the various ways in which conditionals
can already be fudged within FunnelWeb as it stands.\x{conditionals}

\thing{File markers:} It might be worth modifying the language so that
a special syntactical marker is required at the start and end of files.
This will assist in detecting truncations and other
corruptions.\xx{file}{markers}

\thing{Formal parameter lists:} It might be worth changing over to a
syntax for formal parameter lists that does not require the \p{@(} and
\p{@)}. However, they could be retained as optional for backward
compatibility.\xx{parameter list}{formal}

\section{Scanner/Mapper}
\x{scanner}\x{mapper}

\thing{All non-contiguous mappings:} Currently FunnelWeb requires that all
input files be mapped into a contiguous lump of memory. This caused problems
for two reasons. First, to do this, one has to allocate the memory first,
and to do that, you have to know how long the file is, and it turns out that
finding out the length of a file in a portable manner is very inefficient.
Second, although IBM PC compatibles may have megabytes of memory, it is
segmented into blocks of at most 64K. This means that FunnelWeb currently
cannot read a file longer than 64K on a PC.\xx{contiguous}{memory}

These problems could be avoided if the mapper and scanner were reorganized to
allow input files to be read in and stored as a linked list of chunks of text
rather than a contiguous block.

\thing{EOL is unspecifiable:} FunnelWeb uses ASCII character decimal ten (10)
internally to represent logical end-of-line and is currently organized so that
if one of these is inserted into the text by the user using a
\dqp{@\circumflex{}D(10)}, it will be written out as a logical end of line,
rather than as a single ASCII character 10. This should be
fixed.\xx{representation}{EOL}

\thing{Allow mnemonics for unprintables:} FunnelWeb allows users to insert
unprintable characters into the output using the \dqp{@\circumflex{}D(ddd)}
special sequence. Perhaps it would be changed so that it understands ASCII
standard mnemonics such as \dqp{LF} as well as ASCII
numbers.\xx{ASCII}{mnemonics}

\thing{Version pragma:} A \dqp{version} pragma should be added that allows
the user to specify in the input file the version of FunnelWeb that was
around when the input file was created. At a later date, such a construct
would be very useful for determining how an input file should be updated
if the FunnelWeb language has changed between
versions.\xx{version}{pragma}

\section{Parser}
\x{parser}

There are no proposals to change the parser except as a consequence of
other proposals.

\section{Analyser}
\x{analyser}

\thing{Recursion detection:} Currently the FunnelWeb analyser flags, with
an error, all macros with an infinite expansion. This would be best
changed to flagging all macros that are directly or indirectly recursive.
To do this, Tarjan's algorithm\paper{Tarjan72}\xn{Robert Endre}{Tarjan}
for the detection of strong components should
be installed.\xx{recursion}{detection}

\thing{Once only macros:} By default FunnelWeb prevents a macro from being
called more than once unless it has a \dqp{@M} associated with it. However,
FunnelWeb does allow a macro that calls such a macro to be called more
than once. Perhaps this \dq{loophole} should be plugged somehow.

  
\section{Tangle}
\x{tangle}

The Tangler is one of the cleanest components of FunnelWeb, as basically all it
has to do is expand some very well-defined macros.

\thing{Text indentation:} Currently FunnelWeb supports \i{no indentation}
and \i{blank indentation}. A third form could be added if it was considered
necessary. \i{Text indentation} is the same as \i{blank indentation} except
that instead of indenting with blanks, FunnelWeb would indent with the
text to the left of the called macro. This facility could be useful for
commenting out large bodies of text in languages that do not have
multi-line comments (e.g. Ada). A discussion of the pros and cons of this
form of indentation appears earlier.\xx{text}{indentation}

\section{Weave}
\x{weave}

Perhaps FunnelWeb's weakest aspect is its typesetting facility.


\thing{Align table of contents:} When FunnelWeb generates a table of contents,
the section numbers are not quite aligned with the start of the controlling
heading above them.\x{table of contents}

\thing{Typesetting strength:} It should be possible to specify the
level of typesetting strength for headings so that short documents do not
look overdone when typeset. A new pragma would be good for
this.\xx{strength}{typesetting}

\thing{Typeset a portion:} Sometimes it is desirable to typeset just a
portion of a program. A command line option could be added to do
this. The option could accept as its argument, a string containing a list
of section numbers or heading names.

\thing{Generic typesetter option:} In addition to building in a number
of different versions of Weave, one for each popular typesetter, it would be
possible to add a special generic format where the typeset output is
expressed in terms of \i{FunnelWeb macros}. The user could then specify
macro definitions for a non-supported typesetter and run the output
through FunnelWeb Tangle to get a typeset file in a format suitable for the
unsupported typesetter.\xx{typesetter}{generic}

\thing{Suppression of include files:} It should be possible to specify
in the input file that particular included files not appear in the
typeset output. Currently, the fact that an inclusion has occurred is
not even represented in the typeset output. Suppression of inclusions
is particularly necessary where a library of macros has been included
at the top of each of a group of source files.\xx{include file}{suppression}

\thing{Cross reference of identifiers:} WEB provides a list of identifiers
and a list of all the definitions in which they are used. A similar
feature could be added to FunnelWeb.\xx{cross}{references}

\thing{Support for non-printables:} Currently FunnelWeb does not provide
support for typesetting the special \dqp{@\circumflex(num)} sequences.
This should be added.

\thing{Support for @+ sequences:} Currently Weave does not see \dqp{@+}
sequences as such. Instead it perceives them as ordinary EOLs. However,
there are arguments for typesetting them specially.

\thing{Typeset text in macro bodies:} One of the much-loved features
of WEB is the way that it allows the user to switch recursively between
document and program formats. FunnelWeb does not allow this, but
should. In FunnelWeb, the delimiters \dqp{@\{} and \dqp{@\}} are already
used consistently to delimit macro text. The \dqp{@[} and \dqp{@]} sequences
have been reserved for the delimitation of documentation text.

\thing{Non-tt typesetting:} The current version of FunnelWeb sets all its
macro text in \p{tt font}. This is both a blessing and a curse. It is a
blessing because it connects the reader directly to the code, with no
complicated intermediary. It is a curse because it looks ugly compared to
the beautifully typeset programs produced by other literate programming
tools.

The difficulty with adding such beautiful typesetting is that it is
necessarily language-specific. Keywords and syntax differ from language
to language and it would not be easy to come up with a set of language
independent rules.

One approach is to write a set of Weave back-ends, one for each language.
Another approach is to \i{generate} back ends. This is the approach
taken in the \i{Spider} system\paper{Ramsey89}.\x{spider}
In the \i{Spider} system,
the programmer writes production rules for converting lexical components
of the program text into typesetter instructions. The \i{Spider} program
reads these rules and generates a new version of WEB suited for the target
language.

For FunnelWeb a slightly different system is proposed in which Spider-like
rules appear in the input file and are used directly by Weave to perform the
typesetting. An intermediate abstract typesetting language could be used
so that the productions can be made language specific, but not typesetter
specific.

\section{Lister}
\x{lister}

\thing{Glue factor:} A glue factor could be added that determines how many
lines can be in between two diagnostics in the listing before the two
groups of lines are joined together in the listing with no intervening
ellipsis.\xx{glue}{factor}

\section{Diagnostics}
\x{diagnostics}

\thing{Advisory information:} Some of FunnelWeb's diagnostics provide
a detailed explanatory paragraph. While this information might be useful the
first time, it has the capacity to clog up a listing file if the user has
made the same error many times. To solve this problem, FunnelWeb could be
modified so that such explanations are only displayed the first time the
error occurs.

\thing{Abort after n errors:} A facility could be added to prevent
FunnelWeb's scanning, parsing, and analysing phases from continuing if
a certain number of errors have already been issued.


\section{Speed}
\x{speed}\x{efficiency}

\thing{Measurement of speed:} Although FunnelWeb can generate a breakdown
of where it is spending its time, it does not give a final rating in
lines per minute. This should be added.

\thing{Find the hot spots:} Although FunnelWeb has been designed to allow
high speed, not much effort has so far been made to make it fast. This should
be done.

\thing{Change some declarations:} FunnelWeb is full of variable declarations
where the variables are wider than they need be. Replacing these might speed
it up.

\section{Correctness}
\x{correctness}

\thing{Bounds analysis:} Not much effort has gone into the design of
FunnelWeb's input boundaries. An analysis should be made of FunnelWeb's
behaviour when the following quantities are stretched:

\begin{itemize}
\item Input line length.
\item Input file size.
\item Number of macros.
\item Length of macro.
\end{itemize}

In particular, FunnelWeb's behaviour at 32K and 64K boundaries should be
observed.

\thing{Stack detection:} Macintosh THINK-C provides just 6K of memory for
the stack. It might be worth adding checks to make sure that the stack is
not being blown.

\section{Test Suite}
\xx{test}{suite}

The following tests should be added to the test suite:

\begin{verbatim}
Lister
------
   LR01: Test with a full listing with no diagnostics.
   LR02: Test with a full listing with diagnostics.
   LR03: Test with an abbreviated listing with no diagnostics.
   LR04: Test with an abbreviated listing with diagnostics.
   LR05: Test error context system with nearby diagnostics.

Boundary Cases
--------------
Static analysis might preclude the need for most of these tests.
   BC01: Test what happens when memory runs out.
   BC02: Test on a file with a single line of a megabyte.
   BC03: Test on a file of a megabyte of EOLs.
   BC04: Generate an output file with an extremely long line.
   BC05: Generate an output file with one million lines.
   BC06: Test on a file with very many macros.

General
-------
   GN01: A large legal input file exercising as many features as possible.
         1. Test listing file.
         2. Test output files.
         3. Test typeset file.
   GN... A selection of ten real-life FunnelWeb files.
\end{verbatim}

\section{Machine-Specific Changes}

\thing{Icon for the Macintosh:} Currently no icon is supplied for the
Macintosh version of FunnelWeb. An icon depicting a spider or a funnelled
web of some kind would seem appropriate.

\section{Summary}

This chapter has describes some of the problems with FunnelWeb and some of the
opportunities that exist for improving it. The direction and speed in which
FunnelWeb will develop will depend largely on user feedback. If you have any
strong feelings on where FunnelWeb should go, please email them to
Ross Williams (\p{ross@spam.adelaide.edu.au}).


%==============================================================================%
%                              Start of Ch4.tex                                %
%==============================================================================%