diff options
| author | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
|---|---|---|
| committer | Norbert Preining <norbert@preining.info> | 2019-09-02 13:46:59 +0900 |
| commit | e0c6872cf40896c7be36b11dcc744620f10adf1d (patch) | |
| tree | 60335e10d2f4354b0674ec22d7b53f0f8abee672 /support/splint | |
Initial commit
Diffstat (limited to 'support/splint')
82 files changed, 26719 insertions, 0 deletions
diff --git a/support/splint/COPYING b/support/splint/COPYING new file mode 100644 index 0000000000..94a9ed024d --- /dev/null +++ b/support/splint/COPYING @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + <program> Copyright (C) <year> <name of author> + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +<http://www.gnu.org/licenses/>. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +<http://www.gnu.org/philosophy/why-not-lgpl.html>. diff --git a/support/splint/INSTALL b/support/splint/INSTALL new file mode 100644 index 0000000000..11e85736ed --- /dev/null +++ b/support/splint/INSTALL @@ -0,0 +1,59 @@ +There is no specific installation procedure to follow for SPLinT, although +there are a few dependencies to keep in mind. To build all the tools +in the package, you will need the usual suite of build tools (gcc, +make and friends), bison and flex (which usually come separately), +cweb and cwebmac.tex (supplied by the texlive-extra-utils in Mint, +for example), and various fonts (cyrillic, such as lhr10, etc., found in +texlive-lang-cyrillic, and extra sizes of Computer Modern, supplied by +texlive-fonts-extra). The only purpose for the cyrillic fonts is to supply +a decent looking pair of `french quotes' or `guillemot's used in +./tex/btokenset.sty. + +As of version 3.0 of bison, the arrays yyprhs and yyrhs are no longer +exported as part of the bison output (indeed, they are no longer created +internally, either, so even writing a `plugin' would not help). A very +elaborate scheme, akin to the one used to extract the actions can be +implemented to reproduce those arrays but this seems too high a price +to pay for the arrogance and shortsightedness of bison maintainers and +developers. Therefore, as of this release, this package is only +intended to be used with bison version 2.7 or lower. So far, the +latest release of flex (2.5.39) is still compatible with SPLinT. In +case the local version of bison (and possibly, flex, in the future) is +incompatible with splint, it is recommended that a local version is +compiled and used. For this purpose, make variables BISON_ROOT and +FLEX_ROOT can be set to the appropriate locations in makefile.loc. The +appropriate versions of bison and flex can be downloaded from +http://ftp.gnu.org/gnu/bison/ and http://flex.sourceforge.net/, +respectively. The installation instructions in those packages are easy +to follow (the standard ./configure --prefix=<local bin directory> &&\ +make && cp src/bison <local bin directory> && cp -a data/* \ +<local bin directory>/share/bison should work). + +The arrays (yyprhs and yyrhs) only affect the error reporting and the +`symbolic switch' output. The former dependence can be eliminated +(following the route taken by bison itself), however, the latter one +is a much more serious issue. If one is not using symbolic names for +grammar terms, the arrays can be ignored. The approach taken by the +curent version of bison is to use the state stack and yystos, yyr1, +and yyr2 arrays instead. Note that this is somewhat inconsistent with +the purpose of the debugging output since the error reporting routines +rely on the correct state of the state stack (yyssa) rather than on a +static set of grammar rules. Use an older version of bison. + +The setup involved is minimal, but for the build procedure to work, +all the style files (that reside in ./tex and ./cweb) should be visible +to \TeX\ (i.e. you should be able to \input them). Building everything +from scratch also requires Perl to be installed (see the scripts in the +./scripts directory). Building after the result of `make mostlyclean' +requires a C compiler and CWEB only. + +* A note about make: while using make to perform quick rebuilds after + minor modifications mostly works, this style of project is not very + suitable for make's capabilities. Make is really good at + codifying the build procedures that form a tree. In the case of + SPLinT, the same commands often have to be run several times. What + is even worse, running those commands repeatedly modifies the + prerequisites for earlier targets. Thus, view make as simply a + convenient way of recording every step needed to build the project + and its various parts, not as an efficient building tool: it is + always safer to say 'make distclean' and repeat the build.
\ No newline at end of file diff --git a/support/splint/Makefile b/support/splint/Makefile new file mode 100644 index 0000000000..05209cf6f8 --- /dev/null +++ b/support/splint/Makefile @@ -0,0 +1,39 @@ +SPLINT_ROOT = $(shell pwd) +SPLINT_EXAMPLES_DIRS = expression xxpression symbols ld + +DO_SUBMAKE = for dir in ${SPLINT_EXAMPLES_DIRS}; do cd ${SPLINT_ROOT}/examples/$$dir && ${MAKE} $(1); done + +include ${SPLINT_ROOT}/makefile.inc +include ${SPLINT_ROOT}/makefile.loc + +# output a list of all control sequences defined in the package + +lists: tex/*.sty + perl scripts/cslist.pl $^ > cseqs.lst + +manual: + cd ${SPLINT_ROOT}/cweb && ${MAKE} splint.pdf + +docs: + cd ${SPLINT_ROOT}/cweb && ${MAKE} splint.pdf && ${MAKE} ssffo.pdf + $(call DO_SUBMAKE,docs) + +# clean will erase all automatically generated files in the current directory + +clean: clean_core + -rm -f cseqs.lst + +# mostlyclean will leave all the generated table and token equivalence files + +mostlyclean: + -cd cweb && ${MAKE} clean_temp && rm -f ctablesout b?out ltout smallp_out \ + smalll_out lstabout bo.c np.c + $(call DO_SUBMAKE,mostlyclean) + +# distclean will erase all automatically generated files + +distclean: clean + rm -f splint.tar.bz2 + cd cweb && ${MAKE} clean + $(call DO_SUBMAKE,clean) + diff --git a/support/splint/README b/support/splint/README new file mode 100644 index 0000000000..9a0111b681 --- /dev/null +++ b/support/splint/README @@ -0,0 +1,82 @@ +To make the licensing part clear, SPLinT is GPL v.~3: + +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +SPLinT is a \TeX\ package for building parsers and scanners in +\TeX. Bison and Flex are used to create the tables that drive the +automata programmed as \TeX\ macros (thus, after the parser/scanner +has been completed it can be used in plain \TeX, without invoking any +outside software). The documentation included with the package has a +detailed description of the features of the produced parsers and +scanners. To get started, run `make' to create a manual (which can be +found as cweb/splint.pdf in the cweb/ directory) and take a look at the +`examples' directory. Here is a short +description of the various files included in the collection. + +Files: + +cweb/* - executables and documentation: + bs.w - Bison Sourcer(er): table code core for bison + fk.w - Flex Kit(ten): table code core for flex + common.w - common code for table generators + bo.w - parser for the bison grammar + lo.w - lexer for the bison grammar + np.w - scanner and parser for token names + ssffo.w - lexer for state grabbing + mkeparser.w - parser output `driver' + mkscanner.w - lexer output `driver' + splint.w - documentation + +tex/* - \TeX\ macros + yy*.sty, yx*.sty, flex.sty - automata machinery + trt1.sty - `\TeX\ runtime': temporary register definitions + xarithm.sty - expandable arithmetic for parsing macros + grabstates.sty - macros for state grabbing + ?tokenset.sty - token typesetting definitions + dcols.sty - multiple column output + limbo.sty - limbo section macros + frontmatter.sty - macros to typeset the cover page + +examples/* - various examples: + count - an example of robust token counting macros. + expression - a simple expression parser built with the package. + ld - a typesetting parser for the \GNU\ linker, ld, with a very + detailed implementation manual. + symbols - a demonstration of the features of the bison parser + included in the package. + types - expandable arithmetic (e.g. addition and subtraction + macros that can be used inside \edef), tree data structure; + incomplete and slow, merely a proof of concept; only + standard plain \TeX\ is used + xxpression - an extended version of the `expression' + example above, with a demonstration of symbolic name + mechanism setup. + +scripts/* - helper (perl, for now) scripts to aid in pre- and + post-processing of various files + +makefile.inc - common definitions +makefile.loc - placeholder file for local make rules +COPYING - license information +INSTALL - cursory instalation notes +README - this file +TODO - plans for future development +VERSION - the current version + +The most recent version of this software can be downloaded at + + math.tntech.edu/alex/splint.tar.bz2 diff --git a/support/splint/TODO b/support/splint/TODO new file mode 100644 index 0000000000..42c0b2b51f --- /dev/null +++ b/support/splint/TODO @@ -0,0 +1,27 @@ +... ongoing + +o Typo and style fixes + +... sometime in the future + +o Change generic macro names (such as \table, \symstream, etc) to something more specific +* Add \bgrulealign and \egrulealign macros to align rules across sections (a simple way to + implement this is given as an example in the included package, a more flexible method would + have to wait) +* Rewrite yytexlex.sty for better \TeX\ pretty printing macros (* the new macros are still + not ideal but are very unlikely to change in the near future) +o Rewrite limbo.sty to provide a more logical structure +* Add indexing features to \TeXx macros (* there is a mechanism to do this) +o Change CWEB macros so that \pdfoutput=0 does not cause conflicts (this is a bug in cwebmac.tex) + +... very remote future + +o Make glr parser generation possible (* probably not feasible with + the current state of bison design, likely to require a fork or even + a completely different tool) +o Make the `core' parsing and scanning macros prefix-expandable + +... very very remote future + +o Make automatic translation of \Cee\ into \TeX\ possible so that + bison routines can be simply translated on the fly.
\ No newline at end of file diff --git a/support/splint/VERSION b/support/splint/VERSION new file mode 100644 index 0000000000..993f095645 --- /dev/null +++ b/support/splint/VERSION @@ -0,0 +1 @@ +1.05 diff --git a/support/splint/cweb/Makefile b/support/splint/cweb/Makefile new file mode 100644 index 0000000000..ec9973fb66 --- /dev/null +++ b/support/splint/cweb/Makefile @@ -0,0 +1,115 @@ +SPLINT_ROOT = $(shell pwd)/.. + +include ${SPLINT_ROOT}/makefile.inc + +all: ${SPLINT_PTABLES} ${SPLINT_LTABLES} + +b%out: mkeparser.c b%.c + ${CC} ${BISON_STATE} -DPARSER_FILE=\"$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +b%.yy: bo.x + ${CTANGLE} $< + +%yytab.tex: b%out + ${SPLINT_DRIVER_DIR}/$< --optimize-actions $@ + +ltab.tex: ltout + ${SPLINT_DRIVER_DIR}/$< --optimize-actions $@ + +ltout: mkscanner.c lo_states.h lo.c + ${CC} -DLEXER_FILE=\"$(lastword $^)\" -o $@ $< + +ssffo.ll lo.ll: \ +%.ll: %.x + ${CTANGLE} $< && rm $(patsubst %.x, %.c, $^) + +lo.c: lo.l + ${FLEX} -o $@ $< + +mkscanner.c mkeparser.c: \ +%.c: %.w + ${CTANGLE} $< + +# name parser + +smallp_out: mkeparser.c small_parser.c + ${CC} ${BISON_STATE} -DPARSER_FILE=\"$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +smalll_out: mkscanner.c small_lexer.c + ${CC} -DLEXER_FILE=\"$(lastword $^)\" -o $@ $< + +small_tab.tex: smallp_out + ${SPLINT_DRIVER_DIR}/$< --optimize-actions $@ + +small_dfa.tex: smalll_out + ${SPLINT_DRIVER_DIR}/$< --optimize-actions $@ + +small_parser.yy small_lexer.ll: np.x + @${CTANGLE} $< + +bo.tex: bo.x + ${CWEAVE} -x $< + +splint.tex \ +splint.idx \ +splint.scn: splint.w bo.x lo.x np.x common.w bs.w fk.w philosophy.w references.w + ${CWEAVE} $< + +ssffo.tex \ +ssffo.idx ssffo.scn: ssffo.x + ${CWEAVE} $< + +bo.tok: bo.tex ltab.tex byytab.tex + ${TEX} ${MODEBOOTSTRAP} \\input $< + +ssffo.pdf: %.pdf: ${SPLINT_DOC_PREREQS_XREF} + ${PDFTEX} $*.tex + +ssffo.dvi: %.dvi: ${SPLINT_DOC_PREREQS_XREF} + ${TEX} $*.tex + +splint.gdx: %.gdx: ${SPLINT_DOC_PREREQS_XREF} + @echo "Making the bison and TeX indices ..." + ${TEX} $*.tex + +splint.pdf: %.pdf: ${SPLINT_DOC_PREREQS_XREF} %.gdy + ${PDFTEX} \\input $*.tex && touch $*.gdy && touch $*.pdf + +splint.dvi: %.dvi: ${SPLINT_DOC_PREREQS_XREF} %.gdy + ${TEX} $*.tex && touch $*.gdy && touch $*.dvi + +${SPLINT_ROOT}/tex/btokenset.sty: # stupid make weirdness + @ + +# state parsing + +lstabout: mkscanner.c ssffo.c + ${CC} -DLEXER_FILE=\"$(lastword $^)\" -o $@ $< + +lstab.tex: lstabout + ${SPLINT_DRIVER_DIR}/$< --optimize-actions $@ + +lo.tex: lo.x + ${CWEAVE} $< + +lo_states.h: lo.tex lstab.tex byytab.tex + ${PDFTEX} $< + +# clean will erase all automatically generated files in the current directory + +clean: clean_core + -rm -f ctablesout b?out ltout smallp_out \ + smalll_out lstabout + +include ${SPLINT_ROOT}/makefile.loc + +# since bg.yy is not an intermediate file in examples/symbols/Makefile, repeated 'make all' +# remakes bg.yy thereby forcing make to update byytab.tex, etc., which results in remaking +# of bo.tok, lo.tex, eventually leading to remaking of splint.pdf; +# the special target below tells make to treat bg.yy as if it were not an intermediate file + +.PRECIOUS: bg.yy bg.y + +# the files below appear as targets but are really intermediaries for other files + +.INTERMEDIATE: smallp_out smalll_out lstabout ltout splint.gdx diff --git a/support/splint/cweb/bo.w b/support/splint/cweb/bo.w new file mode 100644 index 0000000000..2185f2f65e --- /dev/null +++ b/support/splint/cweb/bo.w @@ -0,0 +1,2479 @@ +% Copyright 2012-2014, Alexander Shibakov +% Copyright 2002-2014 Free Software Foundation, Inc. +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\input limbo.sty +\input frontmatter.sty +\def\optimization{5} +\input yy.sty +% multi-column output +\input dcols.sty + +\let\hostparsernamespace\mainnamespace % the namespace where tokens are looked up + % for typesetting purposes +\let\currentparsernamespace\parsernamespace + \let\parsernamespace\mainnamespace + \let\currenttokeneq\tokeneq + %\def\tokeneq#1#2{\prettytoken{#1}} + \let\tokeneq\prettywordpair@@ + \let\optstrextra\optstrextraesc + \input bo.tok % re-use token equivalence table to set the typesetting of tokens + \let\tokeneq\currenttokeneq + \input btokenset.sty + % index entries + \let\parsernamespace\indexpseudonamespace + \prettywordpair{emptyrhs}{$\circ$ {\rm(empty rhs)}}% + \prettywordpair{inline_action}{$\diamond$ {\rm(inline action)}}% + \prettywordpair{TOKEN}{{\tt TOKEN} {\rm(example)}}% + \prettywordpair{token}{{\tt "token"} {\rm(example)}}% +\let\parsernamespace\currentparsernamespace + +\immediate\openout\exampletable=\jobname.exl + +\def\nontitle#1{{\ttl #1}} +\def\cite[#1]{% + \def\next{#1}\setbox0=\hbox{l}% + [\ifx\next\empty$\,$\hbox{\vrule width\wd0 height\ht0 depth\dp0}$\,$\else \locallink{#1bibref}#1\endlink\fi]% +} + +\let\oldN\N +\let\N\textN +\let\M\textM + +\defreserved{Y}{\.{Y}} +\showlastactiontrue + +@**Introduction. +\setupfootnotes +\splint\footnote{I was tempted to call the package {\tt ParLALRgram} +which stands for Parsing {\sc LALR} Grammars or {\tt PinT} for +`Parsing in \TeX' but both sounded too generic.} (Simple Parsing and +Lexing in \TeX, or, following the great GNU +tradition of creating recursive names, \splint\ Parses Languages +in \TeX) is a system (or +rather a m\'elange of systems) designed to +facilitate developing parsing macros in \TeX\ and (to a lesser +degree) documenting parsers written in other languages. As +an application, a parser for \bison\ input file syntax has been +developed, along with a macro collection that makes it possible to +design and pretty print \bison\ grammars using \CWEB. + +Developing software in \CWEB\ involves two programs. The first of these is +\CTANGLE\ that outputs the actual code, intended to be in +\Cee. In reality, \CTANGLE\ cares very little about the language it +produces. Exceptions are \Cee\ comments and |@[#line@]| directives that might +confuse lesser software, although \bison\ is all too happy to swallow them +(there are also some \Cee\ specific constructs that \CTANGLE\ tries to +recognize). \CTANGLE's main function is to rearrange the text of the +program as written by the programmer (in a way that, hopefully, +emphasizes the internal logic of the code) into an appropriate +sequence (e.g.~all variable declaration must textually precede their +use). All that is required to adopt \CTANGLE\ to produce \bison\ +output is some very rudimentary post- and pre-processing. + +Our main concern is thus \CWEAVE\ that not only pretty prints the +program but also creates an index, cross-references all the +sections, etc. Getting \CWEAVE\ to pretty print a language other than +\Cee\ requires some additional attention. A true digital warrior would +probably try to decipher \CWEAVE's output `in the raw' but, alas, my +WebFu is not that strong. The loophole comes in the form of a rarely +(for a good reason) used \CWEB\ command: the verbatim (\.{@@=...@@>}) +output. The material to be output by this construct undergoes minimal +processing and is put inside \.{\\vb\{}$\ldots$\.{\}}. All that is +needed now is a way to process this virtually straight text inside \TeX. + +@*1 Using the \bison\ parser. +The process of using \splint\ for writing parsing macros in \TeX\ is +treated in considerable detail later in this document. A shorter +(albeit somewhat outdated but still applicable) version of this +process is outlined in \cite[Sh]. We begin, +instead, by explaining how one such parser can be used to pretty print a +\bison\ grammar. Following the convention mentioned above and putting +all non-\Cee\ code inside \CWEAVE's verbatim blocks, consider the +following (meaningless) code fragment. The fragment contains a mixture +of \Cee\ and \bison\ code, the former appears outside of the verbatim blocks. +\begindemo +^@@= non_terminal: @@> +^@@= term.1 term.2 {@@> a = b; @@=}@@> +^@@= **H term.3 other_term {@@> $$ = $1; @@=}@@> +^@@= **H still more terms {@@> f($1); @@=}@@> +^@@= ; @@> +\enddemo +The fragment above will appear as (the output of \CTANGLE\ can be +examined in \.{sill.y}) +@<A silly example@>= +@G +non_terminal: + term.1 term.2 {@> a = b; @=} +| term.3 other_term {@> $$ = $1; @=} +| still more terms {@> f($1); @=} +; +@g + +@ $\ldots$ if the syntax is correct. +In case it is a bit off, the parser will give up and +you will see a different result. The code in the fragment below is easily +recognizable, and some parts of it (all of \Cee\ code, in fact) are +still pretty printed in \CWEAVE. Only the verbatim portion is left +unprocessed. +@<A silly example@>= +@G +whoops + term.1 term.2 {@>@+ a = b; @+@=} +| term.3 other_term {@>@+ $$ = $1; @+@=} +| still more terms {@>@+ f($1); @+@=} +; +@g + +@ The \TeX\ header that makes such output possible is quite plain. In this case +(i.e.\ this very file) it begins as +\begindemo +^\input limbo.sty +^\input frontmatter.sty +^\input yy.sty +\nooutput +\enddemo +The first two lines are presented here merely for completeness: there is +no parsing-relevant code in them. The line that +follows loads the macros that implement the parsing and scanning +machinery. This is enough to set up all the basic +mechanisms used by the parsing and lexing macros. The rest of the header +provides a few definitions to fine tune the typesetting of +grammar productions. It starts with +\begindemo +^\let\currentparsernamespace\parsernamespace +^ \let\parsernamespace\mainnamespace +^ \let\currenttokeneq\tokeneq +^ \def\tokeneq#1#2{\prettytoken{#1}} +^ \input bo.tok % re-use token equivalence table to set the typesetting of tokens +^ \let\tokeneq\currenttokeneq +^ \input btokenset.sty +\nooutput +\enddemo +We will have a chance to discuss all the \.{\\}$\ldots$\.{namespace} +macros later, at this point it will suffice to say that the lines +above are responsible for controlling the typesetting of term names. The +file \.{bo.tok} consists of a number of lines like the ones below: +\begindemo +^\tokeneq {STRING}{{34}{115}{116}{114}{105}{110}{103}{34}} +^\tokeneq {PERCENT_TOKEN}{{34}{37}{116}{111}{107}{101}{110}{34}} +\nooutput +\enddemo +The cryptic looking sequences of integers above are strings of {\sc ASCII} +codes of the letters that form the name \bison\ uses when it needs to +refer to the corresponding token (thus, the second one is +\toksa{}\numberstochars{34}{37}{116}{111}{107}{101}{110}{34}\end +\.{\the\toksa} which might help explain why such an elaborate scheme +has been chosen). The macro \.{\\tokeneq} is defined in +\.{yymisc.sty}, which in turn is input by \.{yy.sty} but what about +the token names themselves? In this case they were extracted +automatically from the \CWEB\ source file by the parser during the +\CWEAVE\ processing stage. All of these definitions can be +overwritten to get the desired output (say, one might want to typeset +\.{ID} in a roman font, as `identifier'; all that needs to be done is +a macro that says \.{\\prettywordpair\{ID\}\{\{\\rm +identifier\}\}}). The file \.{btokenset.sty} input above contains a +number of such definitions. + +@ To round off this short overview, I must mention a caveat associated +with using the macros in this collection: while one of the greatest +advantages of using \CWEB\ is its ability to rearrange the code in a +very flexible way, the parser will either give up or produce +unintended output if this feature is abused while describing the +grammar. For example, in the code below +@<A silly example@>= +@G +next_term: + stuff @> @<Rest of line@> @={@> a = f( x ); @=} +@g +@<A production@>@; + +@ the line titled |@<A production@>| is intended to be a rule defined +later. Notice that while it seems that the parser was able to recognize +the first code fragment as a valid \bison\ input, it misplaced the +|@<Rest of line@>|, having erroneously assumed it to be a part of +the action code for this grammar (later on we will go into the details of +why it is necessary to collect all the non-verbatim output of \CWEAVE, +even the one that contains no interesting \Cee\ code; hint: it has +something to do with money (\.{\$}), also known as math and the way +\CWEAVE\ processes the `gaps' between verbatim sections). The production +line that follows did not fare as well: the parser gave up. There +is simply no point in including such a small language fragment as a +valid input for the grammar the parser uses to process the verbatim +output. +@<A production@>= +@G + more stuff in this line {@> @[b = g(y);@]@=} +@g + +@ Finally, if you forget that only the verbatim part of the output is +looked at by the parser you might get something unrecognizable, such +as +@<Rest of line@>= + but not all of it + +@ To correct this, one can provide a more complete grammar fragment to +allow the parser to complete its task successfully. In some cases, +this imposes too strict a constraint on the programmer. Instead, the +parser that pretty prints \bison\ grammars allows one to add {\it +hidden context\/} to the code fragments above. The context is added +inside \.{\\vb} sections using \CWEB's \.{@@t}$\ldots$\.{@@>} facility. The \CTANGLE\ +output is not affected by this while the code above can now be typeset as: +@<A silly example@>= +@G +next_term: + stuff @> @t}\vb{\formatlocal{\let\peekstash\stashtoterm}}{@> @<Rest of line@> @t}\vb{FAKE}{@> @={@> a = f( x ); @=} +@g +@<A production@>@; + +@ $\ldots$ even a single line can now be displayed properly. +@<A production@>= +@G +@t}\vb{\formatlocal{\skipheader} FAKE:}{@> + more stuff in this line {@> b = g( y ); @=} +@g + +@ With enough hidden context, even a small rule fragment can be +typeset as intended. The `action star' was inserted to reveal some of +the context. +@<Rest of line@>= +@G +@t}\vb{\formatlocal{\skipheader} FAKE:}{@> + but not all of it +@t}\vb{\{\stashed{$\star$}\}}{@> +@g +@ What makes all of this even more confusing is that \CTANGLE\ will +have no trouble outputting this as a(n almost, due to the +intentionally bad \.{whoops} production above) valid \bison\ file +(as can be checked by looking into \.{sill.y}). The author +happens to think that one should not fragment the software into pieces +that are too small: \bison\ is not \Cee\ so it makes sense to write +\bison\ code differently. However, if the logic behind your code +organization demands such fine fragmentation, hidden context provides +you with a tool to show it off. A look inside the source of this +document shows that adding hidden context can be a bit ugly so it is +not recommended for routine use. The short example above is output in +the file below. +@(sill.y@>= + @<A silly example@>@; + +@*1 On debugging. This concludes a short introduction to the \bison\ +grammar pretty printing using this macro collection. It would be +incomplete, however, without a short reference to debugging\footnote{Here +we are talking about debugging the output produced by \CWEAVE\ when +the included \bison\ parser is used, {\it not\/} debugging parsers +written with the help of this software: the latter topic is covered in more +detail later on}. There is a +fair amount of debugging information that the macros can output, +unfortunately, very little of it is tailored to the {\it use\/} of the +macros in the \bison\ parser. Most of it is designed to help {\it +build\/} a new parser. If you find that the parser gives up too often +or even crashes (the latter is most certainly a bug in the parser +itself), the first approach is to make sure that your code {\it +compiles\/} i.e.\ forget about the printed output and try to see if +the `real' \bison\ accepts the code (just the syntax, no need to +worry about conflicts and such). + +If this does not shed any light on why the macros seem to fail, turn +on the debugging output by saying \.{\\trace$\ldots$true} for various +trace macros. This can produce {\it a lot\/} of output, even for +small fragments, so turn it on only for a section at a time. If you +need still {\it more\/} details of the inner workings of the parser +and the lexer, various other debugging conditionals are available. For +example, \.{\\yyflexdebugtrue} turns on the debugging output for the +scanner. There are a number of such conditionals that are discussed in +the commentary for the appropriate \TeX\ macros. + +Remember, what you are seeing at this point is the parsing process of +the \bison\ input file, not the one for {\it your\/} grammar (which +might not even be complete at this point). However, if this fails, you +are on your own: drop me a line if you figure out how to fix any bugs +you find. + +@*1 Terminology. We now list a few definitions of the concepts used +repeatedly in this documentation. Most of this terminology is +rather standard. Formal precision is not the goal here, and intuitive +explanations are substituted whenever possible. +{% +\def\aterm#1{\item{\sqebullet}{\ttl #1}: \ignorespaces}% +\setbox0=\hbox{\sqebullet\enspace} +\parindent=0pt +\advance\parindent by \wd0 +\smallskip +\aterm{bison parser} while, strictly speaking, not a formally defined +term, this combination will always stand for one of the parsers generated +by this package designed to parse a subset of the `official' grammar for +\bison\ input files. All of these parsers are described later in +this documentation. The term {\it main parser\/} will be +used as a substitute in example documentation for the same purpose. + +\aterm{driver} a generic but poorly defined concept. In this +documentation it is used predominantly to mean both the \Cee\ code and +the resulting executable that outputs the \TeX\ macros that contain the +parser tables, token values, etc., for the parsers built by the user. It +is understood that the \Cee\ code of the `driver' is unchanged and the +information about the parser itself is obtained by {\it including\/} the \Cee\ +file produced by \bison\ in the `driver' (see the examples supplied +with the package). + +\aterm{lexer} a synonym for {\it scanner}, a subroutine that performs the {\it +lexical analysis\/} phase of the parsing process, i.e.\ groups various +characters from the input stream into parser {\it tokens}. + +\aterm{namespace} this is an overused bit of terminology meaning a +set of names grouped together according to some relatively +well defined principle. In a language without a well developed type +system (such as \TeX) it is usually accompanied by a specially designed +naming scheme. {\it Parser namespaces\/} are commonly used in this +documentation to mean a collection of all the data structures describing a +parser and its state, including tables, stacks, etc., named by using the +`root' name (say \.{\\yytable}) and adding the name of the parser (for +example, \.{[main]}). To support this naming scheme, a number of +macros work in unison to create and rename the `data macros' accordingly. + +\aterm{symbolic switch} a macro (or an associative array of macros) +that let the \TeX\ parser generated by the package associate {\it +symbolic term names\/} with the terms. Unlike the `real' parser, the +parser created with this suite requires some extra setup as explained +in the included examples (one can also consult the source for this +documentation which creates but does not use a symbolic switch). + +\aterm{symbolic term name} a (relatively new) way to refer to stack +values in \bison. In addition to using the `positional' names such as +\.{\$}$n$ to refer to term values, one can utilize the new syntax: +\.{\$}\.{[}{\it name\/}\.{]}. The `{\it name}' can be assigned by the +user or can be the name of the nonterminal or token used in the +productions. + +\aterm{term} in a narrow sense, an `element' of a grammar. Instead of +a long winded definition, an example, such as \prodstyle{ID} should +suffice. Terms are further classified into {\it terminals\/} (tokens) +and {\it nonterminals\/} (which can be intuitively thought of as +composite terms). + +\aterm{token} in short, an element of a set. Usually encoded as an +integer by most parsers, an indivisible {\it term\/} +produced for the parser by the scanner. \TeX's scanner uses a more +sophisticated token classification, for example, $($character code, +character category$)$ pairs, etc. + +} +@** Languages, scanners, parsers, and \TeX. % Or $\ldots$ +$$\vbox{\halign to\hsize{\kern-1.5pt\it#\hfil\tabskip0pt plus1fil\cr +Tokens and tables keep macros in check.\cr +Make 'em with \bison, use \.{WEAVE} as a tool.\cr +Add \TeX\ and \CTANGLE, and \Cee\ to the pool.\cr +Reduce 'em with actions, look forward, not back.\cr +Macros, productions, recursion and stack!\cr +\noalign{\vskip2pt} +\omit\hfil\eightpoint Computer generated (most likely)\cr}} +$$ +\def\recount#1{${}^{(#1)}$}% +In order to understand the parsing routines in this collection, +it would help to gain some familiarity with the internals of the +parsers produced by \bison\ for its intended target: \Cee. A person +looking inside a parser delivered by \bison\ would +quickly discover that the parsing procedure itself (|yyparse|) +occupies a rather small portion of the file. If (s)he were to further +reduce the size of the file by removing all the preprocessor +directives intended to anticipate every conceivable combination of the +operating system, compiler, and \Cee\ dialect, and various reporting +and error logging functions it would become very clear that the most +valuable product of \bison's labor is a collection of integer {\it +tables\/} that control the actions of the parser routine. Moreover, +the routine itself is an extremely concise and well-structured loop +composed of |goto|'s and a number of numerical conditionals. If one +were to think of a way of accessing arrays and processing conditionals +in the language of one's choice, once the tables produced by \bison\ +have been converted into a form suitable for the consumption by the +appropriate language engine, the parser implementation becomes +straightforward. Or nearly so. + +The {\it scanning\/} (or {\it lexing\/}) step of this process---a way +to convert a stream of symbols into a stream of integers, also +deserves some attention here. There are a number of excellent tools +written to automate this step in much the same fashion as \bison\ +automates the generation of parsers. One such tool, \flex, though +(in the opinion of this author) slightly lacking in the simplicity and +elegance as compared to \bison, was used to implement the lexer for +this software suite. Lexing in \TeX\ will be discussed in considerable +detail later in this manual. + +The language of interest in our case is, of course, \TeX, so our +future discussion will revolve around the five elements mentioned +above: \recount{1}data structures (mainly arrays and stacks), +\recount{2}converting +\bison's output into a form suitable for \TeX's consumption, +\recount{3}processing raw streams of \TeX's tokens and converting them into +streams of parser tokens, \recount{4}the implementation of \bison's +|yyparse| in \TeX, and, finally, \recount{5}producing \TeX\ output via {\it +syntax-directed translation} (which requires an appropriate +abstraction to represent \bison's actions inside \TeX). We shall +begin by discussing the parsing process itself. + +@*1 Arrays, stacks and the parser. +Let us briefly examine the programming environment offered by \TeX. +Designed for typesetting, \TeX's remarkable language +provides a layer of macro processing atop of a set of commands that +produce the output fulfilling its primary mission: delivering page +layouts. In The \TeX book, macro {\it expansion\/} is likened to +mastication, whereas \TeX's main product, the typographic output is the +result of its `digestion' process. Not everything that goes through +\TeX's digestive tract ends up leaving a trace on the final page: a +file full of \.{\\relax}'s will produce no output, even though +\.{\\relax} is not a macro, and thus would have to be processed by +\TeX\ at the lowest level. + +It is time to describe the details of defining suitable data structures +in \TeX. At first glance, \TeX\ provides rather standard means of +organizing and using general memory. At the core of its generic +programming environment is an array of \.{\\count}$\,n$ {\it +registers\/}, which may be viewed as general purpose integer variables +that are randomly accessible by their indices. The integer arithmetic +machinery offered by \TeX\ is spartan but is very adequate for the sort of +operations a parser would perform: mostly additions and +comparisons. + +Is the \.{\\count} array a good way to store tables in \TeX? Probably +not. The first factor is the {\it size\/} of this array: only 256 +\.{\\count} registers exist in a standard \TeX\ (the actual number of +such registers on a typical machine running \TeX\ is significantly +higher but this author is a great believer in standards, and to his +knowledge, none of the standardization efforts in the \TeX\ world has +resulted in anything even close to the definitive masterpiece that is +The \TeX book). The issue of size can be mitigated to some extent by +using a number of other similar arrays used by \TeX\ (\.{\\catcode}, +\.{\\uccode}, \.{\\dimen}, \.{\\sfcode} and others can be used for +this purpose as long as one takes care to restore the `sane' values +before control is handed off to \TeX's typesetting mechanisms). If a +table has to span several such arrays, however, the complexity of +accessing code would have to increase significantly, and the issue of +size would still haunt the programmer. + +The second factor is the use of several registers by \TeX\ for special +purposes (in addition, some of these registers can only store a +limited range of values). Thus, the first 10 \.{\\count} registers are +used by plain \TeX\ for (well, {\it intended\/} for, anyway) the +purposes of page accounting: their values would have to be carefully +saved and restored before and after each parsing call, +respectively. Other registers (\.{\\catcode} in particular) have even +more disrupting effects on \TeX's internal mechanisms. While all of +this can be managed (after all, using \TeX\ as an arithmetic engine +such as a parser suspends the need for any typographic or other +specialized functions controlled by these arrays), the added +complexity of using several memory banks simultaneously and the speed penalty +caused by the need to store and restore register values make this +approach much less attractive. + +What other means of storing arrays are provided by \TeX? Essentially, +only three options remain: \.{\\token} registers, macros holding whole +arrays, and associative arrays accessed through +\.{\\csname}$\,\ldots\,$\.{\\endcsname}. In the first two cases if care +is taken to store such arrays in an +appropriate form one can use \TeX's \.{\\ifcase} primitive to access +individual elements. The trade-off is the speed of such +access: it is {\it linear\/} in the size of the array for most +operations, and worse than that for others, such as removing the last +item of an array. Using clever ways +of organizing such arrays, one can improve the linear access time to +$O(\log n)$ by simply modifying the access macros but at the moment, a +straightforward \.{\\ifcase} is used after expanding a list macro or +the contents of a \.{\\token}$\,n$ register in an {\it un\/}optimized +parser. An {\it optimized\/} parser uses associative arrays. + +The array discussion above is just as applicable to {\it stacks\/} +(indeed, an array is the most common form of stack +implementation). Since stacks pop up and disappear frequently (what +else are stacks to do?), list macros are usually used to store +them. The optimized parser uses a separate \.{\\count} register to +keep track of the top of the stack in the appropriate associative +array. + +Let us now switch our attention +to the code that implements the parser and scanner {\it functions\/}. +If one has spent some time writing \TeX\ macros of any sophistication +(or any macros, for that matter) (s)he must be familiar with the general +feeling of frustration and the desire to `just call a function here and move +on'. Macros produce {\it tokens\/}, however, and tokens must either +expand to nothing or stay and be contributed to your input, or worse, +be out of place and produce an error. One way to sustain a stream +of execution with macros is {\it tail recursion\/} (i.e.~always expanding the +{\it last token left standing}). + +As we have already discussed, \bison's +|yyparse()| is a well laid out loop organized as a sequence of +|goto|'s (no reason to become religious about structured programming +here). This fact, and the following well known trick, make \Cee\ to \TeX\ +translation almost straightforward. + +% The macro mess below looks painful but this is the only place such layout is used +% The approach can be easily generalized and put in limbo.sty but it seems +% a bit redundant at this point. + +\newcount\piccount +\newdimen\lasthsize + +\setbox5=\vtop{ +\demomargin=0pt +\let\demoastyle\empty +\begindemo +^label A: ... +\nooutput +^ if**L**Krm(condition)**N +^ goto C; +\nooutput +^label B: ... +\nooutput +^ goto A; +\nooutput +^label C: ... +\nooutput +\enddemo +} +\dp5=\z@@ + +\setbox3=\vtop{ +\demomargin=0pt +\let\demoastyle\empty +\begindemo +^\if**L**Krm(condition)**N +^ \let\next=\labelC +^\else +^ \let\next=\labelAtail +\enddemo +} +\dp3=\z@@ + +\newdimen\lastdepth + +\def\startfitpar{% + \bgroup + \lasthsize=\hsize + \advance\lasthsize-1.5in + \vsize=\baselineskip + \topskip=\z@@ + \setbox0\box2 % empty it + % this sounds good at first but there is no good way to pull the insertions out after the + % box manipulations that follow; + % insertions will thus be contributed to whatever page was being worked on when the + % picture insertions {\it started}; hence, if these happen to start at the very top of the page, + % any insertion that follows will be contributed to the previous page; we correct this for footnotes + % below + % \holdinginserts=1 + \output{% + \global\setbox2=\vbox{ + \ifvoid2 + \else + \prevdepth=\dp2 + \unvbox2 + \fi + \lastdepth=\dp255 + \unvbox255 + % this would be tempting, however, the \eject that follows should disappear + % in addition, one really should not be playing with page breaking in the middle of + % such tricky insertions + % \penalty\outputpenalty + % \kern-\lastdepth % to make sure \baselineskip is accounted for + }% + }\eject + \output{% + \setbox0=\vbox{% + \unvbox255% + }% \lastbox would almost work ... if not for insertions + \global\advance\piccount1 + \global\setbox2=\vbox{% + \prevdepth=\dp2 \unvbox2 + \hbox to\hsize{% + \ifnum\piccount<15 + \hbox to1.5in{% + \ifnum\piccount=1 + \ \box5 + \fi + \hfill}% + \fi + \box0 \hfill + \ifnum\piccount=1 + \box3 \ % + \fi + \ifvoid\footins % reinsert footnotes + \else + \insert\footins{\unvbox\footins}% + \fi + }% + }% + }% + \parshape=15 + 0pt 2.7in + 0pt 2.7in + 0pt 2.7in + 0pt 2.7in + 0pt 2.7in + 0pt 2.7in + 0pt 2.7in + 0pt \lasthsize + 0pt \lasthsize + 0pt \lasthsize + 0pt \lasthsize + 0pt \lasthsize + 0pt \lasthsize + 0pt \lasthsize + 0pt \hsize +} + +\def\endfitpar{% + \par + \eject + \egroup + % see the comment above + % \holdinginserts=0 + \prevdepth=\dp2 + \unvbox2 +} + +\startfitpar +\noindent Given the code on the left (where |goto|'s +are the only means of branching but can appear inside conditionals), +one way to translate it into \TeX\ is to define a set of macros (call +them \.{\\labelA}, \.{\\labelAtail} and so forth for clarity) that end in +\.{\\next} (a common name for this purpose). Now, \.{\\labelA} will +implement the code that comes between \.{label A:} and \.{goto C;}, +whereas \.{\\labelAtail} is responsible for the code after \.{goto C;} +and before \.{label B:} +(provided no other |goto|'s intervene which can always be +arranged). The conditional which precedes \.{goto C;} can now be written in +\TeX\ as presented on the right, where (condition) is an appropriate +translation of the corresponding condition +in the code being translated (usually, one of `$=$' or `$\not=$'). Further +details can be extracted from the \TeX\ code that implements these +functions where the corresponding \Cee\ code is presented alongside +the macros that mimic its functionality% +\footnote{Running the risk of overloading the reader with details, the author +would like to note that the actual implementation follows a {\it slightly\/} different +route in order to avoid any \.{\\let} assignments or changing the +meaning of \.{\\next}}. +This concludes an overview of the general approach, +It is time to consider the way characters get consumed +on the lower levels of the macro hierarchy and the interaction between the different +layers of the package. +\endfitpar + +@*1 \TeX\ into tokens. +Thus far we have covered the ideas +behind items \recount{1} and \recount{4} on our list. It is time to +discuss the lowest level of processing done by these macros: +converting \TeX's tokens into the tokens consumed by the parser, +i.e.\ part\recount{3} of the plan. Perhaps, it would be most appropriate +to begin by defining the term {\it token}. + +As commonly defined, a token is simply an element of a set. Depending on +how much structure the said set possesses, a token can be represented by +an integer or a more complicated data structure. In the discussion +below, we will be dealing with two kinds of tokens: the tokens +consumed by the parsers and the \TeX\ tokens seen by the input +routines. The latter play the role of {\it characters\/} that combine +to become the former. \bison's internal representation for its tokens +is non-negative integers so this is what a scanner must +produce. + +\TeX's tokens are a good deal more sophisticated: they can be +either pairs $(c_{\rm ch}, c_{\rm cat})$, where $c_{\rm ch}$ is the +character code and $c_{\rm cat}$ is \TeX's category code ($1$ and $2$ for +group characters, $5$ for end of line, etc.), or {\it control +sequences\/}, such as \.{\\relax}. Some of these tokens (control +sequences and {\it active}, i.e.~category~13 characters) can have +complicated internal structure (expansion). The situation is further +complicated by \TeX's \.{\\let} facility, which can create +`character-like' control sequences, and the lack of conditionals +to distinguish them from the `real' characters. Finally, not all pairs +can appear as part of the input (say, there is no $(n, 0)$ token for +any $n$, in the terminology above). + +The scanner expects to see {\it characters} in its input, which are +represented by their {\sc ASCII} codes, i.e.~integers between $0$ and +$255$ (actually, a more general notion of the Unicode character is +supported but we will not discuss it further). Before character codes +appear as the input to the scanner, however, and make its integer +table-driven mechanism `tick', a lot of work must be done to collect +and process the stream of \TeX\ tokens produced after \CWEAVE\ is done +with your input. This work becomes further complicated when the +typesetting routines that interpret the parser's output must sneak +outside of the parsed stream of text (which is structured by the +parser) and insert the original \TeX\ code produced by \CWEAVE\ into +the page. + +\splint\ comes with a customizeable input routine of +moderate complexity (\.{\\yyinput}) that classifies all \TeX\ tokens +into seven categories: `normal' spaces (i.e.~category~10 tokens, +skipped by \TeX's parameter scanning mechanism), +`explicit' spaces (includes the control sequences \.{\\let} to \.{\ }, +as well as \.{\\\ }), groups ({\it avoid} using \.{\\bgroup} and \.{\\egroup} in +your input but `real', \.{\{}$\ldots$\.{\}} groups are fine), active +characters, normal characters (of all character categories that can +appear in \TeX\ input, including \.{\$}, \.{\^}, \.{\#}, \.{a}--\.{Z}, +etc.), single letter control sequences, and multi-letter control +sequences. Each of these categories can be processed separately to +`fine-tune' the input routine to the problem at hand. The input +routine is not very fast, instead, flexibility was the main +goal. Therefore, if speed is desirable, a customized input routine +is a great place to start. As an example, a minimalistic +\.{\\yyinputtrivial} macro is included. + +When \.{\\yyinput} `returns' by calling \.{\\yyreturn} (which is a +macro you design), your lexing routines have access to three +registers: \.{\\yycp@@}, that holds the character value of the +character just consumed by \.{\\yyinput}, \.{\\yybyte}, that most of +the time holds the token just removed from the input, +and \.{\\yybytepure}, that (again, with very few +exceptions) holds a `normalized' version of the read character (i.e.~a +character of the same character code as \.{\\yycp@@}, and category~11 +(to be even more precise (and to use nested parentheses), `normalized' +characters have the same category code as the current category code of +\.{@@})). + +Most of the time it is the character code one needs (say, in the case +of \.{\\\{}, \.{\\\}}, \.{\\\&} and so on) but under some circumstances the +distinction is important (outside of \.{\\vb\{}$\ldots$\.{\}}, the sequence +\.{\\1} has nothing to do with the digit `\.{1}'). This mechanism +makes it easy to examine the consumed token. It also forms +the foundation of the `hidden context' passing mechanism described later. + +The remainder of this section discusses the internals of \.{\\yyinput} +and some of the design trade-offs one has to make while working on +processing general \TeX\ token streams. It is typeset in `small print' +and can be skipped if desired. +\smallskip +\begingroup +\abovedisplayskip=5pt% +\abovedisplayshortskip=2pt% +\belowdisplayskip=5pt% +\belowdisplayshortskip=2pt% +\fnotesstart=1 +\fnotesspan=2 +\noofcolumns=2 +\icgap=1em% +\eightpoint +\linecount=73 +\setmcparams +\def\.#1{{\chardef\\=`\\\chardef\&=`\&\tt #1}}% +\dsskip=0pt% +\begindoublecols +To examine every token in its path (including spaces that are easy to +skip), the input routine uses one of the two well-known {\sc \TeX}nologies: +\.{\\futurelet\\next\\examinenext} or equally effective +\hbox{\.{\\afterassignment\\next\\let={\tt\char"20}}}. +Recursively inserting one of these sequences, \.{\\yyinput} can go +through any list of tokens, as long as it knows where to stop +(i.e.~return an end of file character). The +signal to stop is provided by the \.{\\yyeof} +primitive which should not appear in any `ordinary' text +presented for parsing, other than for the purpose of providing such a +stop signal. Even the dependence on \.{\\yyeof} can be eliminated if +one is willing to invest the time in writing macros that juggle \TeX's +\.{\\token} registers and only limit oneself to input from such +registers (which is, aside from an obvious efficiency hit, a strain on +\TeX's memory, as you have to store multiple (3 in the general case) +copies of your input to be able to back up when the lexer makes a +wrong choice). There does not seem to be a way of doing it unless the +text has been stored in a \.{\\token} register first (or storing the +whole input as a {\it parameter\/} for the appropriate macro: this +scheme is remarkably powerful and leads to {\it expandable\/} versions +of very complicated macros, although the amount of effort required to +write such macros grows at a frightening rate). All of these are +non-issues for the text inside \.{\\vb\{}$\ldots$\.{\}} and the care that +\.{\\yyinput} takes in processing characters inside such lists is an +overkill. In a more `hostile' environment (such as the one encountered +by the now obsolete \.{\\Tex} macros), this extra attention to detail pays +off in the form of a more robust input mechanism. + +One subtlety deserves a special mention here, as it can be important +to the designer of `higher-level' scanning macros. Two types of tokens +are extremely difficult to deal with whenever \TeX's own lexing +mechanisms are used: (implicit) spaces and even more so, braces. We +will only discuss braces here, however, almost everything that follows +applies equally well to spaces (category 10 tokens to be precise), with +a few simplifications (or complications, in a couple of places). To +understand the difficulty, let's consider one of the approaches above: +$$ +\.{\\futurelet\\next\\examinenext}. +$$ +The macro \.{\\examinenext} +usually looks at \.{\\next} and inserts another macro (usually also called +\.{\\next}) at the very end of its expansion list. This macro usually +takes one parameter, to consume the next token. This mechanism works +flawlessly, until the lexer encounters a \.{\{}br\.{,}sp\.{\}}ace. The \.{\\next} +sequence, seen by \.{\\examinenext} contains a lot of information +about the brace ahead: it knows its category code (left brace, so $1$), its +character code (in case there was, say a \.{\\catcode`\\[=1{\tt\char`\ }} +earlier) but not whether it is a `real' brace (i.e.\ a character +\.{\{}$_1$) or an implicit one (a \.{\\bgroup}). There is no way to find +that out until the control sequence `launched' by \.{\\examinenext} +sees the token as a parameter. + +If the next token is a `real' brace, however, +\.{\\examinenext}'s successor will never see the token itself: the +braces are stripped by \TeX's scanning mechanism. Even if it finds a +\.{\\bgroup} as the parameter, there is no guarantee that the actual +input was not \.{\{\\bgroup\}}. One way to handle this is by using +\.{\\string} ahead of any consumption of the next token. If prior to +expanding \.{\\string} care has been taken to set the \.{\\escapechar} +appropriately (remember, we know the character code in advance), as +soon as one sees a character with \.{\\escapechar}'s character code, +(s)he knows that an implicit brace has just been seen. One added +complication to all this is that a very determined programmer can +insert an {\it active\/} character (using, say, the \.{\\uccode} +mechanism) that has the {\it same\/} character code as the {\it +brace\/} token that it has been \.{\\let} to! Setting this possibility +aside, the \.{\\string} mechanism (or, its cousin, \.{\\meaning}) is +not perfect: both produce a sequence of category 12 and 10 tokens. If +it is indeed a brace character that we just saw, we can consume the next +token and move on but what if this was a control sequence? After all, +just as easily as \.{\\string} makes a sequence into characters, +\.{\\csname}$\,\ldots\,$\.{\\endcsname} pair will make any sequence of +characters into a control sequence. Huh~$\ldots$ + +What we need is a backup mechanism: if one has a copy of the +token sequence ahead, one can use \.{\\string} to see if it is a real +brace first, and if it is, consume it and move on (the active character +case can be handled as the implicit case below, with one extra backup +to count how many tokens have been consumed). At this point one has to {\it +reinsert\/} the brace in case, at some point, a future `back up' +requires that the rest of the tokens are removed from the output (to +avoid `\.{Too many \}'s}' complaints from \TeX). This can be done by using +the \.{\\iftrue\{\\else\}\\fi} trick but of course, some bookkeeping is +needed to keep track of how far inside the brace groups we +are. + +If it is an implicit brace, more work is needed: read all the +characters that \.{\\string} produced (an maybe more), then remember +the number of characters consumed. Remove the rest of the input using +the method described above and restart the scanning from the same point +knowing that the next token can be scanned as a parameter. + +Another strategy is to design a general enough macro that counts +tokens in a token register and simply recount the tokens after every +brace was consumed. + +Either way, it takes a lot of work. If anyone would +like to pursue the counting strategy, simple counting macros +are provided in \.{/examples/count/count.sty}. +The macros in this example +supply a very general counting mechanism that does not depend on +\.{\\yyeof} (or {\it any\/} other token) being `special' and can count the +tokens in any token register, as long as none of those tokens is an +\.{\\outer} control sequence. In other words, if the macro is used +immediately after the assignment to the token register, it should +always produce a correct count. + +Needless to say, if such a general mechanism is desired, one has to +look elsewhere. The added complications of treating spaces (\TeX\ +tends to ignore them most of the time) make this a torturous exercise +in \TeX's macro wizardry. The included \.{\\yyinput} has two ways of +dealing with braces: strip them or view the whole group as a +token. Pick one or write a different \.{\\yyinput}. Spaces, implicit +or explicit are reported as a specially selected character code and +consumed with a likeness of +$$ +\hbox{\.{\\afterassignment\\moveon\\let\\next={\tt\char`\ }}}. +$$ + +Now that a steady stream of character codes is arriving at \.{\\yylex} +after \.{\\yyreturn} the job of converting it into numerical tokens +is performed by the {\it scanner} (or {\it lexer\/}, or {\it tokenizer\/}, +or even {\it tokener}), discussed in the next section. +\enddoublecols +\endgroup + +@*1 Lexing in \TeX. In a typical system that uses a parser to process +text, the parsing pass is usually split into several stages: the raw +input, the lexical analysis (or simply {\it lexing}), and the parsing +proper. The {\it lexing\/} (also called {\it scanning}, we use these +terms interchangeably) clumps various sequences of characters into +{\it tokens\/} to facilitate the parsing stage. The reasons for this +particular hierarchy are largely pragmatic and are partially historic +(there is no reason that {\it parsing\/} cannot be done in multiple +phases, as well, although it usually isn't). + +If one remembers a few basic facts from the formal language theory, it +becomes obvious that a lexer, that parses {\it regular\/} languages, +can (theoretically) be replaced by an {\sc LALR} parser, that parses {\it +context-free\/} ones (or some subset thereof, which is +still a super set of all regular languages). A common justification given for +creating specialized lexers is efficiency and speed. The +reality is somewhat more subtle. While we do care about the efficiency of +parsing in \TeX, having a specialized scanner is important for +a number of different reasons. + +The real advantage of having a dedicated scanner is the ease with which it +can match incomplete inputs and back up. A parser can, of course, +{\it recognize\/} any valid input that is also acceptable to a lexer, as well +as {\it reject\/} any input that does not form a valid token. Between +those two extremes, however, lies a whole realm of options that a +traditional parser will have great difficulty exploring. Thus, to +mention just one example, it +is relatively easy to set up a DFA\footnote{Which stands for +Deterministic Finite Automaton, a common (and mathematically unique) +way of implementing a scanner for regular languages. Incidentally {\sc +LALR} mentioned above is short for Look Ahead Left to Right.} +so that the {\it longest\/} +matching input is accepted. The only straightforward way to do this +with a traditional parser is to parse longer and longer inputs again +and again. While this process can be optimized to a certain degree, +the fact that a parser has a {\it stack\/} to maintain limits its +ability to back up. + +As an aside, the mechanism by which \CWEB\ assembles its `scraps' +into chunks of recognized code is essentially iterative lexing, +very similar to what a human does to make sense of complicated +texts. Instead of trying to match the longest running piece of text, +\CWEB\ simply looks for patterns to combine inputs into larger +chunks, which can later be further combined. Note that this is not +quite the same as the approach taken by, say {\sc GLR} parsers, where +the parser must match the {\it whole\/} input or declare a +failure. Where a \CWEB-type parser may settle for the first available +match (or the longest available) a {\sc GLR} parser must try {\it +all\/} possible matches or use an algorithm to reject the majority of +the ones that are bound to fail in the end. + +This `\CWEB\ way' is also different from a traditional `strict' {\sc +LR} parser/scanner approach and certainly deserves serious +consideration when the text to be parsed possesses some rigid +structure but the parser is only allowed to process it one small +fragment at a time. + +Returning to the present macro suite, the lexer produced by \flex\ +uses integer tables similar to those employed by \bison\ so the +usual {\sc\TeX}niques used in implementing \.{\\yyparse} are fully +applicable to \.{\\yylex}. + +An additional advantage provided by having a \flex\ scanner implemented +as part of the suite is the availability of the original \bison\ scanner written +in \Cee\ for the use by the macro package. + +This said, the code generated by \flex\ contains a few idiosyncrasies +not present in the \bison\ output. These `quirks' mostly involve +handling of end of input and error conditions. A quick glance at the +\.{\\yylex} implementation will reveal a rather extensive collection of +macros designed to deal with end of input actions. + +Another difficulty one has to face in translating \flex\ output into +\TeX\ is a somewhat unstructured namespace delivered in the final +output (this is partially due to the \POSIX\ standard that \flex\ +strives to follow). One consequence of this `messy' approach is that the +writer of a \flex\ scanner targeted to \TeX\ has to declare \flex\ +`states' (more properly called {\it subautomata}) twice: first for the +benefit of \flex\ itself, and then again, in the {\it \Cee\ preamble\/} +portion of the code to output the states to be used by the action code +in the lexer. \.{Define\_State($\ldots$)} macro is provided for this +purpose. This macro can be used explicitly by the programmer or be +inserted by a specially designed parser. +Using \CWEB\ helps to keep these declarations together. + +The `hand-off' from the scanner to the parser is implemented +through a pair of registers: \.{\\yylval}, a token register +containing the value of the returned token and \.{\\yychar}, a +\.{\\count} register that contains the numerical value of the +token to be returned. + +Upon matching a token, the scanner passes one crucial piece of +information to the user: the character sequence representing the token +just matched (\.{\\yytext}). This is not the whole story +though. There are three more token sequences that are made available +to the parser writer whenever a token is matched. + +The first of these is simply a `normalized' version of +\.{\\yytext} (called \.{\\yytextpure}). In most cases it +is a sequence of \TeX\ tokens with the same character codes as the one +in \.{\\yytext} but with their category codes set to 11. In +cases when the tokens in \.{\\yytext} are {\it not} +$(c_{\rm ch}, c_{\rm cat})$ pairs, a few simple +conventions are followed, some of which will be explained below. This +sequence is provided merely for convenience and its typical use is to +generate a key for an associate array. + +The other two sequences are special `stream pointers' that provide +access to the extended scanner mechanism in order to implement passing +of `formatting hints' to the parser without introducing any changes to +the original grammar. As the mechanism itself and the motivation +behind it are somewhat subtle, let me spend a few moments discussing +the range of formatting options desirable in a generic pretty-printer. + +Unlike strict parsers employed by most compilers, a parser designed +for pretty printing cannot afford being too picky about the structure +of its input (\cite[Go] calls such parsers `loose'). To provide +a simple illustration, an isolated identifier, such as `\.{lg\_integer}' +can be a type name, a variable name, or a structure tag (in a language like +\Cee\ for example). If one expects the pretty printer to typeset this +identifier in a correct style, some context must be supplied, as +well. There are several strategies a pretty printer can employ to get +a hold of the necessary context. Perhaps the simplest way to handle +this, and to reduce the complexity of the pretty printing algorithm is +to insist on the user providing enough context for the parser to do +its job. For short examples like the one above, this is an acceptable +strategy. Unfortunately, it is easy to come up with longer snippets of +grammatically deficient text that a pretty printer should be expected +to handle. Some pretty printers, such as the one employed by \CWEB\ +and its ilk (the original \.{WEB}, \.{FWEB}), use a very flexible +bottom-up technique that tries to make sense of as large a portion of +the text as it can before outputting the result (see also \cite[Wo], +which implements a similar algorithm in \LaTeX). + +The expectation is that this algorithm will handle the majority (about +90\%? it would be interesting to carry out a study in the spirit of +the ones discussed in \cite[Jo] to find out) of the +cases with the remaining few left for the author to correct. The +question is, how can such a correction be applied? + +\CWEB\ itself provides two rather different mechanisms for handling +these exceptions. The first uses direct typesetting commands (for +example, \.{@@/} and \.{@@\#} for canceling and +introducing a line break, resp.) to change the typographic output. + +The second (preferred) way is to supply {\it hidden context\/} to the +pretty-printer. Two commands, \.{@@;} and +\.{@@[}$\ldots$\.{@@]} are used for this purpose. The +former introduces a `virtual semicolon' that acts in every way like a +real one except it is not typeset (it is not output in the source file +generated by \CTANGLE, either but this has nothing to do with pretty +printing, so I will not mention \CTANGLE\ anymore). For +instance, from the parser's point of view, if the preceding text was +parsed as a `scrap' of type {\it exp}, the addition of \.{@@;} +will make it into a `scrap' of type {\it stmt\/} in \CWEB's +parlance. The second construct (\.{@@[}$\ldots$\.{@@]}), +is used to create an {\it exp\/} scrap out of whatever happens to be +inside the brackets. + +This is a powerful tool at the author's disposal. Stylistically, +this is the right way to handle exceptions as it forces the writer to +emphasize the {\it logical\/} structure of the formal +text. If the pretty printing style is changed +extensively later, the texts with such hidden contexts should be able to +survive intact in the final document (as an example, using a break +after every statement in \Cee\ may no longer be considered +appropriate, so any forced break introduced to support this convention +would now have to be removed, whereas \.{@@;}'s would simply +quietly disappear into the background). + +The same hidden context idea has another important advantage: with +careful grammar fragmenting (facilitated by \CWEB's or any other +literate programming tool's `hypertext' structure) and a more diverse +hidden context (or even arbitrary hidden text) mechanism, it is +possible to use a strict parser to parse incomplete language +fragments. For example, the productions that are needed to parse +\Cee's expressions form a complete subset of the grammar. If the +grammar's `start' symbol is changed to {\it expression\/} (instead of +the {\it translation-unit\/} as it is in the full \Cee\ grammar), a +variety of incomplete \Cee\ fragments can now be parsed and +pretty-printed. Whenever such granularity is still too `coarse', +carefully supplied hidden context will give the pretty printer enough +information to adequately process each fragment. A number of such {\it +sub}-parsers can be tried on each fragment (this may sound +computationally expensive, however, in practice, a carefully chosen +hierarchy of parsers will finish the job rather quickly) until a +correct parser produced the desired output (this approach is similar +to, although not quite the same one employed by the {\it General LR +parsers}). + +This somewhat lengthy discussion brings us to the question directly +related to the tools described in this article: how does one provide +typographical hints or hidden context to the parser? + +One obvious solution is to build such hints directly into the +grammar. The parser designer can, for instance, add new tokens +(say, \.{BREAK\_LINE}) to the grammar and extend the +production set to incorporate the new additions. The risk of +introducing new conflicts into the grammar is low (although not +entirely non-existent, due to the lookahead limitations of LR(1) +grammars) and the changes required are easy, although very tedious, to +incorporate. + +In addition to being labor intensive, this solution has two other +significant shortcomings: it alters the original grammar and hides its +logical structure; it also `bakes in' the pretty-printing conventions +into the language structure (making `hidden' context much less +`stealthy'). It does avoid the `synchronicity problem' mentioned +below. + +A marginally better technique is to introduce a new regular expression +recognizable by the scanner which will then do all the necessary +bookkeeping upon matching the sequence. All the difficulties with +altering the grammar mentioned above apply in this case, as well, only +at the `lexical analysis level'. At a minimum, the set of tokens +matched by the scanner would have to be changed. + +A much better approach involves inserting the hints at the input stage and +passing this information to the scanner and parser as part of the token `values'. The +hints themselves can masquerade as characters ignored by the scanner +(white space, for example) and preprocessed by a specially designed +input routine. The scanner then simply passes on the values to the +parser. This makes hints, in effect, invisible. + +The difficulty lies in synchronizing the token production with the +parser. This subtle complication is very familiar to anyone who has +designed \TeX's output routines: the parser and the lexer are not +synchronous, in the sense that the scanner might be reading several +(in the case of the general LR$(n)$ parsers) tokens ahead of the +parser before deciding on how to proceed (the same way \TeX\ can +consume a whole paragraph's worth of text before exercising its page +builder). + +If we simple-mindedly let the scanner return every hint it has encountered +so far, we may end up feeding the parser the hints meant for the token +that appears {\it after\/} the fragment the parser is currently working +on. In other words, when the scanner `backs up' it must correctly back +up the hints as well. + +This is exactly what the scanner produced by the tools in this package +does: along with the main stream of tokens meant for the parser, it +produces two hidden streams (called the \.{\\format} stream and +the \.{\\stash} stream) and provides the parser with two +strings (currently only strings of digits are used although arbitrary +sequences of \TeX\ tokens can be used as pointers) with the promise +that {\it all the `hints' between the beginning of the corresponding +stream and the point labeled by the current stream pointer appeared +among the characters up to and, possibly, including the ones matched +as the current token}. The macros to extract the relevant parts of the +streams (\.{\\yyreadfifo} and its cousins) are provided for the +convenience of the parser designer. The interested reader can consult +the input routine macros for the details of the internal +representation of the streams. + +In the interest of full disclosure, let me point out that this simple +technique introduces a significant strain on \TeX's +computational resources: the lowest level macros, the ones that handle +character input and are thus executed (sometimes multiple times), for +{\it every\/} character in the input stream are rather complicated and +therefore, slow. Whenever the use of such streams is not desired a simpler +input routine can be written to speed up the process (see +\.{\\yyinputtrivial} for a working example of such macro). + +Finally, while probably not directly related to the present +discussion, this approach has one more interesting feature: after the +parser is finished, the parser output and the streams exist +`statically', fully available for any last minute preprocessing or for +debugging purposes, if necessary. Under most circumstances, the parser +output is `executed' and the macros in the output are the ones reading +the various streams using the pointers supplied at the parsing stage +(at least, this is the case for all the parsers supplied with the +package). + +@*1 Inside semantic actions: switch statements and `functions' in \TeX. +Now you have a lexer for your input, and a grammar ready to be put into +action (we will talk about actions a bit later). It is time to discuss +how the tables produced by \bison\ get converted into \TeX\ {\it macros\/} +that drive the parser in {\it \TeX}. + +The tables that drive the \bison\ input parsers +are collected in various \.{\{b,d,f,g,n\}yytab.tex} and \.{small\_tab.tex}. Each +one of these files contains the tables that implement a specific parser +used during different stages of processing. +Their exact function is well explained +in the source file produced by \bison\ ({\it how} this is done is +explained elsewhere, see \cite[Ah] for a good reference). It would +suffice to mention here that there are three types of tables in this +file: \recount{1}numerical tables such as \.{\\yytable} and +\.{\\yycheck} (both are either \TeX's token registers in an +unoptimized parser or associate arrays in an optimized version of such +as discussed below), +\recount{2}a string array \.{\\yytname}, and \recount{3}an action +switch. The action switch is what gets called when the parser does a +{\it reduction}. It is easy to notice that the numerical tables come +`premade' whereas the string array consisting of token names +is difficult to recognize. This is intentional: this form of initialization +is designed to allow the widest range of +characters to appear inside names. The macros that do this reside in +\.{yymisc.sty}. The generated table files also contain +constant and token declarations used by the parser. + +The description of the process used to output \bison\ tables in an +appropriate form continues in the section about +\locallink{bsfile}outputting \TeX\ tables\endlink, we pick it up here +with the description of the syntax-directed translation and the +actions. The line +$$ +\.{\\switchon\\next\\in\\currentswitch} +$$ +is responsible for calling an appropriate action in the current +switch, as is easy to infer. A {\it switch\/} is also a macro that +consists of strings of \TeX\ tokens intermixed with \TeX\ macros +inside braces. Each group of macros +gets executed whenever the character or the group of characters in +\.{\\next} matches a substring preceding the braced group. If there +are two different substrings +that match, only the earliest group of macros gets expanded. +Before a state is +used, a special control sequence, +\.{\\setspecialcharsfrom\\switchname} can be used to put the \TeX\ +tokens in a form suitable for the consumption by \.{\\switchon}'s. The +most important step it performs is it {\it turns every token in the +list into a character with the same character code and category +12\/}. Thus \.{\\\{} becomes \.{\{}$_{12}$. There are other ways of +inserting tokens into a state: enclosing a token or a string of tokens in +\.{\\raw...\\raw} adds it to the state macro unchanged. If you have +a sequence of category 12 characters you want to add to the state, put +it after \.{\\classexpand} (such sequences are usually prepared by the +\.{\\setspecialchars} macro that uses the token tables generated by +\bison\ from your grammar). + +You can give a case a readable label (say, \.{brackets}) and enclose +this label in \.{\\raw}$\ldots$\.{\\raw}. A word of caution: an `a' +inside of \.{\\raw}$\ldots$\.{\\raw} (which is most likely an +\.{a}$_{11}$ unless you played with category codes before loading the +\.{\\switchon} macros) and the one outside it are two different +characters, as one is no longer a letter (category 11) in the eyes of +\TeX\ whereas the other one still is. For this reason one should not +use characters other than letters in h\.{\{}is\.{,}er\.{\}} state +names: the way a state picks an action does not distinguish between, +say, a `\.{(}' in `\.{(letter)}' and a stand alone `\.{(}' and may +pick an action that you did not intend. This applies even if `\.{(}' +is not among the characters explicitly inserted in the state macro: if +an action for a given character is not found in the state macro, the +\.{\\switchon} macro will insert a current \.{\\default} action +instead, which most often you would want to be \.{\\yylex} or +\.{\\yyinput} (i.e.\ skip this token). If `\.{(}' or `\.{)}' matches +the braced group that follows `\.{(letter)}' chaos may ensue (most +likely \TeX\ will keep reading past the \.{\\end} or \.{\\yyeof} that +should have terminated the input). Make the names of character +categories as unique as possible: the \.{\\switchon} is simply a +string matching mechanism, with the added distinction between +characters of different categories. + +Finally, the construct \.{\\statecomment}{\it +anything\/}\.{\\statecoment} allows you to insert comments in the +state sequence (note that the state {\it name\/} is put at the +beginning of the state macro (by \.{\\setspecialcharsfrom}) +in the form of a special control sequence +that expands to nothing: this elaborate scheme is needed because +another control sequence can be \.{\\let} to the state macro which +makes the debugging information difficult to decipher). The debugging +mode for the lexer implemented with these macros is activated by +\.{\\tracedfatrue}. + +The functionality of the \.{\\switchon} macros (for `historical' +reasons, one can also use \.{\\action} as a synonym) has been +implemented in a number of other macro packages (see \cite[Fi] that +discusses the well-known and widely used \.{\\CASE} and \.{\\FIND} +macros). The macros in this collection have the additional property +that the only assignments that persist after the \.{\\switchon} +completes are the ones performed by the user code inside the selected +case. + +This last property of the switch macros is implemented using another +mechanism that is part of this macro suite: the `subroutine-like' +macros, \.{\\begingroup}$\ldots$\.{\\tokreturn}. For examples, an +interested reader can take a look at the macros included with the +package. A typical use is +\.{\\begingroup}$\ldots$\.{\\tokreturn\{\}\{\\toks0 \}\{\}} which will +preserve all the changes to \.{\\toks0} and have no other side effects +(if, for example, in typical \TeX\ vernacular, \.{\\next} is used +to implement tail recursion inside the group, after the +\.{\\tokreturn}, \.{\\next} will still have the same value it +had before the group was entered). This functionality comes at the +expense of some computational efficiency. + +This covers most of the routine computations inside semantic actions, +all that is left is a way to `tap' into the stack automaton +built by \bison\ using an interface similar to the special +\.{\$$n$} variables utilized by the `genuine' \bison\ parsers +(i.e.\ written in \Cee\ or any other target language supported by +\bison). + +This role is played by the several varieties of \.{\\yy$\,p$} command +sequences (for the sake of completeness, $p$ stands for one of \.{($n$)}, +\.{[{\rm name}]}, \.{]{\rm name}[} or $n$, here $n$ is a +string of digits, and a `name' is any name acceptable as a symbolic +name for a term in \bison). Instead +of going into the minutia of various flavors of \.{\\yy}-macros, let me +just mention that one can get by with only two `idioms' and still +be able to write parsers of arbitrary sophistication: +\.{\\yy($n$)} can be treated as a token register containing the +value of the $n$-th term of the rule's right hand side, $n>0$. The left +hand side of a production is accessed through \.{\\yyval}. A +convenient shortcut is \.{\\yy0\{{\rm \TeX\space material}\}} which +will expand the `\TeX\ material' inside the braces. Thus, a simple way +to concatenate the values of the first two production terms is +\.{\\yy0\{\\the\\yy(1)\\the\\yy(2)\}}. The included \bison\ +parser can also be used to provide support for `symbolic names', +analogous to \bison's \.{{\$}[{\rm name}]} but a +bit more effort is required on the user's part to initialize such support. +Using symbolic names can make the parser more readable and maintainable, +however. + +There is also a \.{\\bb$\,n$} macro, that provides access to the term +values in the `natural order' (e.g.~\.{\\bb1} is the last term read). Its +intended use is with the `inline' rules (see the main parser for +such examples). As of version \.{3.0} \bison\ no longer outputs +|yyrhs| and |yyprhs|, which makes it impossible to produce the +|yyrthree| array necessary for processing such rules in the `left to right' +order. One might also note that the new notation is better suited for +the inline rules since the value that is pushed on the stack is that +of \.{\\bb0}, i.e.~the term implicitly inserted by \bison. Be aware +that there are no \.{\\bb[$\cdot$]} or \.{\\bb($\cdot$)} versions of +these macros, for obvious reasons. A less obvious feature of this +macro is its `nonexpandable' nature. This means they cannot be used +inside \.{\\edef}. Thus, the most common use pattern is +\.{\\bb$\,n$\{\\toks$\,m$\}} with a subsequent expansion of +\.{\\toks$\,m$}. Making these macros expandable is certainly possible +but does not seem crucial for the intended limited use pattern. + +Naturally, a parser writer may need a number of other data +abstractions to complete the task. Since these are highly dependent on +the nature of the processing the parser is supposed to provide, we +refer the interested reader to the parsers included in the package as +a source of examples of such specialized data structures. + +One last remark about the parser operation is worth making here: +the parser automaton itself does not make any \.{\\global} +assignments. This (along with some careful semantic action writing) +can be used to `localize' the effects of the parser operation and, +most importantly, to create `reentrant' parsers that can, e.g.\ call +{\it themselves\/} recursively. + +@*1 `Optimization'. +By default, the generated parser and scanner keep all of their tables +in separate token registers. Each stack is kept in a single macro (this +description is further complicated by the support for parser {\it +namespaces\/} that exists even for unoptimized parsers but this +subtlety will not be mentioned again---see the macros in the package +for further details). Thus, every time a table +is accessed, it has to be expanded making the table access latency +linear in {\it the size of the table}. The same holds for stacks and +the action `switches', of +course. While keeping the parser tables (which are immutable) in token +registers does not have any better rationale than saving the control +sequence memory (the most abundant memory in \TeX), this way of +storing {\it stacks} does have an advantage when multiple parsers get +to play simultaneously. All one has to do to switch from one parser to +another is to save the state by renaming the stack control sequences +accordingly. + +When the parser and scanner are `optimized', all these control +sequenced are `spread over' appropriate associative arrays. One caveat +to be aware of: the action switches for both the parser and the scanner +have to be output differently (a command line option is used to +control this) for optimized and unoptimized parsers. While it is +certainly possible to optimize only some of the parsers (if your +document uses multiple) or even only some {\it parts\/} of a given +parser (or scanner), the details of how to do this are rather +technical and are left for the reader to discover by reading the +examples supplied with the package. At least at the beginning it is +easier to simply set the highest optimization level and use it +consistently throughout the document. + +@*1 {\it \TeX\/} with a different {\sl slant} or do you C an escape?. +%\def\texnspace{other} +Some \TeX\ productions below probably look like alien script. +The authors of \cite[Er] cite a number of reasons pretty printing of +\TeX\ in general is a nearly impossible task. The macros included with +the package follow a very straightforward strategy and do not try to +be very comprehensive. Instead, the burden of presenting \TeX\ code in +a readable form is placed on the programmer. Appropriate hints can be +supplied by means of indenting the code, using assignments ($=$) where +appropriate, etc. If you would rather look at straight \TeX\ +instead, the line \.{\\def\\texnspace\{other\}} at the beginning of +this section can be uncommented and +|TeX_( "/noexpand/inmath{/yy0{/yy1{}}}" );| becomes +\def\texnspace{other}% +|TeX_( "/noexpand/inmath{/yy0{/yy1{}}}" );|. +\def\texnspace{texline}% +There is, however, more to this story. A look at the actual file will +reveal that the line above was typed as +$$ +\.{TeX\_( "/noexpand/inmath\{/yy0\{/yy1\{\}\}\}" );} +$$ +The `escape character' is leaning the other way! +The lore of \TeX\ is uncompromising: `\.{\\}' is {\it the\/} escape +character. What is the reason to avoid it in this case? + +The mystery is not very deep: `\.{/}' was chosen as an escape character +by the parser macros (a quick glance at \.{?yytab.tex} will reveal as +much). There is, of course, nothing sacred (other than tradition, +which this author is trying his hardest to follow) about what character code +the escape character has. The reason to look for the alternative is straightforward: `\.{\\}' is +a special character in \Cee, as well (also an `escape' in fact). The line +\.{TeX\_( "..." );} is a {\it macro-call\/} but $\ldots$ in \Cee. This +function simply prints out (almost `as-is') the line in +parenthesis. An attempt at \.{TeX\_( "\\noexpand" );} would result in +\numberlinestrue +\begindemo +^ +^oexpand +\enddemo +\numberlinesfalse +Other escape combinations\footnote{Here is a full list of {\it +defined\/} escaped characters in \Cee: \.{\\a}, \.{\\b}, \.{\\f}, \.{\\n}, +\.{\\r}, \.{\\t}, \.{\\v}, \.{\\}{$[$\it octal digit$]$}, \.{\\'}, +\.{\\"}, \.{\\?}, \.{\\\\}, \.{\\x}, \.{\\u}, \.{\\U}. Note that the +last three combinations must be followed by a specific string of +characters to appear in the input without generating errors.} are +even worse: most are simply undefined. If anyone feels trapped without +an escape, however, the same line can be typed as +$$ +\.{TeX\_( "\\\\noexpand\\\\inmath\{\\\\yy0\{\\\\yy1\{\}\}\}" );} +$$ +Twice the escape! + +If one were to look closer at the code, another oddity stands +out: there are no \.{\$}'s anywhere in sight. +The big money, \.{\$} is a beloved character in +\bison. It is used in action code to reference the values of the +appropriate terms in a production. If mathematics pays your bills, use +\.{\\inmath} instead. + +@*1 The \bison\ parser(s). Let's take a short break for a broad overview of the input file. +The basic structure is that of an ordinary \bison\ file that produces +plain \Cee\ output. The \Cee\ actions, however, are programmed to output \TeX. + +@s TeX_ TeX +@s TeXa TeX +@s TeXb TeX +@s TeXf TeX +@s TeXfo TeX +@s TeXao TeX + +@(bg.yy@>= +@G Switch to generic mode. +%{@> @<Grammar parser \Cee\ preamble@> @=%} + @> @<Grammar parser \bison\ options@> @= +%union {@> @<Union of grammar parser types@> @=} +%{@> @<Grammar parser \Cee\ postamble@> @=%} + @> @<Tokens and types ...@> @= +%% + @> @<Fake start symbol for rules only grammar@> @= + @> @<Parser common productions@> @= + @> @<Parser grammar productions@> @= +%% +@g + +@ Bootstrap mode is next. The reason for a separate bootstrap parser is to +collect the minimal amount of information to `spool up' the `production' +parsers. To understand the mechanics and the reasons behind it, consider what happens +following a declaration such as \.{\%token TOKEN "token"} +(or, as it would be typeset by the macros in this package +`\prodstyle{\%token} \.{TOKEN} \.{token}'; see the index entries for +more details)% +\idxinline{TOKEN}\idxinline{token}. +The two names for the same token are treated very differently. \.{TOKEN} becomes +an |enum| constant in the \Cee\ parser generated by \bison. Even when +that parser becomes part of the `driver' program that outputs the \TeX\ +version of the parser tables, there is no easy way to output the {\it +names\/} of the appropriate |enum| constants. The other name +(\.{"token"}) becomes an entry in the |yytname| array. These names +can be output by either the `driver' or \TeX\ itself after the +\.{\\yytname} table has been input. The scanner, on the other hand, +will use the first version (\.{TOKEN}). Therefore, it is important to +establish an equivalence between the two versions of the name. In the +`real' parser, the token values are output in a special header +file. Hence, one has to either parse the header file to establish the +equivalences or find some other means to find out the numerical values +of the tokens. + +One approach is to parse the file containing the {\it declarations\/} +and extract the equivalences between the names from it. This is the +function of the bootstrap parser. Since the lexer is reused, some +token values need to be known in advance (and the rest either ignored +or replaced by some `made up' values). These tokens are `hard coded' +into the parser file generated by \bison\ and output using a special +function. The switch `|@[#define@]@; BISON_BOOTSTRAP_MODE|' tells the `driver' +program to output the hard coded token values. +@q Bizarre looking way of typing #define is due to the awkward way@> +@q \CWEB\ treats switching in and out of $-mode in inline \Cee@> + +Note that the equivalence of the two versions of token names would +have to be established every time a `string version' of a token is +declared in the \bison\ file and the `macro name version' of the token +is used by the corresponding scanner. To establish this equivalence, +however, the bootstrapping parser below is not always necessary (see +the \.{xxpression} example, specifically, the file \.{xxpression.w} in +the \.{examples} directory for an example of using a different parser +for this purpose). The reason it is necessary here is that a parser +for an appropriate subset of the \bison\ syntax is not yet available +(indeed, {\it any\/} functional parser for a \bison\ syntax subset +would have to use the same scanner (unless you want to write a custom +scanner for it), which would need to know how to output tokens, for +which it would need a parser for a subset of \bison\ syntax $\ldots$ +it is a `chicken and egg'). Hence the name `bootstrap'. Once a +functional parser for a large enough subset of the \bison\ input +grammar is operational, {\it it\/} can be used to pair up the token +names. + +The second function of the bootstrap parser is to collect information +about the scanner's states. The mechanism is slightly different for +states. While the token equivalences are collected purely in +`\TeX\ mode', the bootstrap parser collects all the state names into a +special \Cee\ header file. The reason is simple: unlike the token +values, the numerical values of the scanner states are not passed to +the `driver' program in any data structure and are instead defined as +ordinary macros. The header file is the information the `driver' file +needs to output the state values. + +An additional subtlety in the case of state value output is that the +main lexer for the \bison\ grammar utilizes states extensively and thus +cannot be easily used with the bootstrap parser before the state +values are known. The solution is to substitute a very simple scanner barely +capable of lexing state declarations. Such a scanner is implemented +in \.{ssffo.w} (the somewhat cryptic name stands for `{\bf s}imple {\bf s}canner +{\bf f}or {\bf f}lex {\bf o}ptions'). +\saveparseoutputtrue +@(bb.yy@>= +@G Switch to generic mode. +%{ + @> @<Grammar parser \Cee\ preamble@> @= + @> @/#define BISON_BOOTSTRAP_MODE @= +%} + @> @<Grammar parser \bison\ options@> @= +%union {@> @<Union of grammar parser types@> @=} +%{@> @<Bootstrap parser \Cee\ postamble@> @=%} + @> @<Tokens and types ...@> @= +%% + @> @<Fake start symbol for bootstrap grammar@> @= + @> @<Parser bootstrap productions@> @= + @> @<\flex\ options parser productions@> @= + @> @<List of symbols@> @= + @> @<Definition of \prodstyle{symbol}@> @= +%% +@g + +@ The prologue parser is responsible for parsing various grammar +declarations as well as parser options. +\saveparseoutputfalse +%\traceparserstatestrue +%\tracestackstrue +%\tracerulestrue +%\traceactionstrue +\saveparseoutputtrue +@(bd.yy@>= +@G Switch to generic mode. +%{@> @<Grammar parser \Cee\ preamble@> @=%} + @> @<Grammar parser \bison\ options@> @= +%union {@> @<Union of grammar parser types@> @=} +%{@> @<Grammar parser \Cee\ postamble@> @=%} + @> @<Tokens and types ...@> @= +%% + @> @<Fake start symbol for prologue grammar@>@; + @> @<Parser common productions@> @= + @> @<Parser prologue productions@> @= +%% +@g + +@ Full \bison\ input parser is used when a complete \bison\ file is +expected. It is also capable of parsing a `skeleton' of such a file, +similar to the one that follows this paragraph. +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\checktablefalse +\saveparseoutputfalse +@(bf.yy@>= +@G Switch to generic mode. +%{@> @<Grammar parser \Cee\ preamble@> @=%} + @> @<Grammar parser \bison\ options@> @= +%union {@> @<Union of grammar parser types@> @=} +%{@> @<Grammar parser \Cee\ postamble@> @=%} + @> @<Tokens and types ...@> @= +%% + @> @<Parser common productions@> @= + @> @<Parser prologue productions@> @= + @> @<Parser grammar productions@> @= + @> @<Parser full productions@> @= +%% +@g + +@ The first two options are essential for the parser operation. The +start symbol can be set implicitly by listing the appropriate +production first. +@q %define lr.type canonical-lr @> +@q Make not on this and lexing too much lookahead and the \stashed trick@> +@q Explain other options @> +@<Grammar parser \bison\ options@>= +@G +%token-table +%debug +%start input +@g + +@*2 Grammar rules. Most of the original comments present in +the grammar file used by \bison\ itself have been preserved and appear in +{\it italics\/} at the beginning of each appropriate section. + +To facilitate the {\it bootstrapping\/} of the parser (see above), some +declarations have been separated into their own sections. Also, a +number of new rules have been introduced to create a hierarchy of +`subparsers' that parse subsets of the grammar. We begin by listing +most of the tokens used by the grammar. Only the string versions are +kept in the |yytname| array, which, in part is the reason for a +special bootstrapping parser as explained earlier. +@<Tokens and types for the grammar parser@>= +@G +%token GRAM_EOF 0 "end of file" +%token STRING "string" + +%token PERCENT_TOKEN "%token" +%token PERCENT_NTERM "%nterm" + +%token PERCENT_TYPE "%type" +%token PERCENT_DESTRUCTOR "%destructor" +%token PERCENT_PRINTER "%printer" + +%token PERCENT_LEFT "%left" +%token PERCENT_RIGHT "%right" +%token PERCENT_NONASSOC "%nonassoc" +%token PERCENT_PRECEDENCE "%precedence" + +%token PERCENT_PREC "%prec" +%token PERCENT_DPREC "%dprec" +%token PERCENT_MERGE "%merge" +@g +@<Global Declarations@>@; + +@ We continue with the list of tokens below, following the layout of +the original parser. +@<Global Declarations@>= +@G +%token + PERCENT_CODE "%code" + PERCENT_DEFAULT_PREC "%default-prec" + PERCENT_DEFINE "%define" + PERCENT_DEFINES "%defines" + PERCENT_ERROR_VERBOSE "%error-verbose" + PERCENT_EXPECT "%expect" + PERCENT_EXPECT_RR "%expect-rr" + PERCENT_FLAG "%<flag>" + PERCENT_FILE_PREFIX "%file-prefix" + PERCENT_GLR_PARSER "%glr-parser" + PERCENT_INITIAL_ACTION "%initial-action" + PERCENT_LANGUAGE "%language" + PERCENT_NAME_PREFIX "%name-prefix" + PERCENT_NO_DEFAULT_PREC "%no-default-prec" + PERCENT_NO_LINES "%no-lines" + PERCENT_NONDETERMINISTIC_PARSER + "%nondeterministic-parser" + PERCENT_OUTPUT "%output" + PERCENT_REQUIRE "%require" + PERCENT_SKELETON "%skeleton" + PERCENT_START "%start" + PERCENT_TOKEN_TABLE "%token-table" + PERCENT_VERBOSE "%verbose" + PERCENT_YACC "%yacc" +; + +%token BRACED_CODE "{...}" +%token BRACED_PREDICATE "%?{...}" +%token BRACKETED_ID "[identifier]" +%token CHAR "char" +%token EPILOGUE "epilogue" +%token EQUAL "=" +%token ID "identifier" +%token ID_COLON "identifier:" +%token PERCENT_PERCENT "%%" +%token PIPE "|" +%token PROLOGUE "%{...%}" +%token SEMICOLON ";" +%token TAG "<tag>" +%token TAG_ANY "<*>" +%token TAG_NONE "<>" +%token INT "integer" +%token <param> PERCENT_PARAM "%param"; +@g + +@ Extra tokens for typesetting \flex\ state +declarations and options are declared in addition to the ones that a +standard \bison\ parser recognizes. +@<Tokens and...@>= +@G +%token FLEX_OPTION FLEX_STATE_X FLEX_STATE_S +@g + +@ We are ready to describe the top levels of the parse tree. The first +`sub parser' we consider is a `full' parser, that is the parser that +expects a full grammar file, complete with the prologue, declarations, +etc. This parser can be used to extract information from the grammar +that is otherwise absent from the executable code generated by +\bison. This includes, for example, the `name' part of +\.{\$}\.{[}{\rm name}\.{]}. +This parser is therefore used to generate the `symbolic +switch' to provide support for symbolic term names similar to +`genuine' \bison's \.{\$}\.{[}$\ldots$\.{]} syntax. +@<Parser full productions@>= +@G +@t}\vb{\inline}{@> +input: + prologue_declarations + "%%" grammar epilogue.opt {@> @<Finish the input setup@> @=} +; +@g + +@ The action of the parser in this case is simply to separate the +accumulated `parse tree' from the auxiliary information carried by the +parser on the stack. +@<Finish the input setup@>= + @[TeX_( "/getsecond{/yy(3)}/to/toksa" );@]@; /* extract grammar contents */ + @[TeX_( "/yy0{/the/toksa}/table=/yy(0)" );@]@; + +@ Another subgrammar deals with the syntax of isolated \bison\ rules. This is +the most commonly used `subparser' since a rules cluster is the most +natural `unit' to include in a \CWEB\ file. +@<Fake start symbol for rules only grammar@>= +@G +@t}\vb{\inline}{@> +input: + grammar epilogue.opt {@> TeX_( "/getsecond{/yy(1)}/to/table" ); @=} +; +@g + +@ The bootstrap parser has a very narrow set of goals: it is concerned +with \prodstyle{\%token} declarations only in +order to supply the token information to the lexer (since, as noted +above, such information is not kept in the |yytname| array). +The parser can also parse \prodstyle{\%nterm} declarations but the +bootstrap lexer ignores the \prodstyle{\%nterm} token, since the +\bison\ grammar does not use one. +It also extends the syntax of a \prodstyle{grammar\_declaration} by allowing a +declaration with or without a semicolon at the end (the latter is only +allowed in the prologue). This works since the token declarations have +been carefully separated from the rest of the grammar in different +\CWEB\ sections. The range of tokens output by the bootstrap +lexer is limited, hence most of the other rules are ignored. +@<Fake start symbol for bootstrap grammar@>= +@G +@t}\vb{\inline}{@> +input: + grammar_declarations {@> TeX_( "/table=/yy(1)" ); @=} +; +@t}\vb{\resetf}{@> +grammar_declarations: + symbol_declaration semi.opt {@> @<Carry on@> @=} +| flex_declaration semi.opt {@> @<Carry on@> @=} +| grammar_declarations + symbol_declaration semi.opt {@> TeX_( "/yy0{/the/yy(1)/the/yy(2)}" ); @=} +| grammar_declarations + flex_declaration semi.opt {@> TeX_( "/yy0{/the/yy(1)/the/yy(2)}" ); @=} +; +@t}\vb{\inline\flatten}{@> +semi.opt: {} | ";" {}; +@g + +@ The following is perhaps the most common action performed by the +parser. It is done automatically by the parser code but this feature +is undocumented so we supply an explicit action in each case. +@<Carry on@>= + @[TeX_( "/yy0{/the/yy(1)}" );@]@; + +@ Next, a subgrammar for processing prologue declarations. Finer +differentiation is possible but the `subparsers' described here work +pretty well and impose a mild style on the grammar writer. +@<Fake start symbol for prologue grammar@>= +@G +@t}\vb{\inline}{@> +input: + prologue_declarations epilogue.opt {@> TeX_( "/getsecond{/yy(1)}/to/table" ); @=} +| prologue_declarations + "%%" "%%" EPILOGUE {@> TeX_( "/getsecond{/yy(1)}/to/table" ); @=} +| prologue_declarations + "%%" "%%" {@> TeX_( "/getsecond{/yy(1)}/to/table" ); @=} +; +@g + +@ {\it Declarations: before the first \prodstyle{\%\%}}. We are now +ready to deal with the specifics of the declarations themselves. The +\.{\\grammar} macro is a `structure', whose first `field' is the +grammar itself, whereas the second carries the type of the last +declaration added to the grammar. +@<Parser prologue productions@>= +@G +prologue_declarations: + {@> TeX_( "/yy0{/nx/grammar{}{/nx/empty}}" ); @=} +| prologue_declarations + prologue_declaration {@> @<Attach a prologue declaration@> @=} +; +@g + +@ @<Attach a prologue declaration@>= + @<Attach a productions cluster@>@; + +@ Here is a list of most kinds of declarations that can appear in the +prologue. The scanner returns the `stream pointers' for all the +keywords so the declaration `structures' pass on those pointers to the +grammar list. The original syntax has been left intact even though for +the purposes of this parser some of the inline rules are unnecessary. +@<Parser prologue productions@>= +@G +prologue_declaration: + grammar_declaration {@> @<Carry on@> @=} +| "%{...%}" {@> TeX_( "/yy0{/nx/prologuecode/the/yy(1)}" ); @=} +| "%<flag>" {@> TeX_( "/yy0{/nx/optionflag/the/yy(1)}" ); @=} +| "%define" variable value {@> TeX_( "/yy0{/nx/vardef{/the/yy(2)}{/the/yy(3)}/the/yy(1)}" ); @=} +| "%defines" {@> TeX_( "/yy0{/nx/optionflag{defines}{}/the/yy(1)}" ); @=} +| "%defines" STRING {@> @[TeX_( "/toksa{defines}" );@]@+@<Prepare one parametric option@> @=} +| "%error-verbose" {@> TeX_( "/yy0{/nx/optionflag{error verbose}{}/the/yy(1)}" ); @=} +| "%expect" INT {@> @[TeX_( "/toksa{expect}" );@]@+@<Prepare one parametric option@> @=} +| "%expect-rr" INT {@> @[TeX_( "/toksa{expect-rr}" );@]@+@<Prepare one parametric option@> @=} +| "%file-prefix" STRING {@> @[TeX_( "/toksa{file prefix}" );@]@+@<Prepare one parametric option@> @=} +| "%glr-parser" {@> TeX_( "/yy0{/nx/optionflag{glr parser}{}/the/yy(1)}" ); @=} +| "%initial-action" "{...}" {@> TeX_( "/yy0{/nx/initaction/the/yy(2)}" ); @=} +| "%language" STRING {@> @[TeX_( "/toksa{language}" );@]@+@<Prepare one parametric option@> @=} +| "%name-prefix" STRING {@> @[TeX_( "/toksa{name prefix}" );@]@+@<Prepare one parametric option@> @=} +| "%no-lines" {@> TeX_( "/yy0{/nx/optionflag{no lines}{}/the/yy(1)}" ); @=} +| "%nondeterministic-parser" {@> TeX_( "/yy0{/nx/optionflag{nondet. parser}{}/the/yy(1)}" ); @=} +| "%output" STRING {@> @[TeX_( "/toksa{output}" );@]@+@<Prepare one parametric option@> @=} +@t}\vb{\flatten}{@> +| "%param" {} + params {@> TeX_( "/yy0{/nx/paramdef{/the/yy(3)}/the/yy(1)}" ); @=} +@t}\vb{\fold}{@> +| "%require" STRING {@> @[TeX_( "/toksa{require}" );@]@+@<Prepare one parametric option@> @=} +| "%skeleton" STRING {@> @[TeX_( "/toksa{skeleton}" );@]@+@<Prepare one parametric option@> @=} +| "%token-table" {@> TeX_( "/yy0{/nx/optionflag{token table}{}/the/yy(1)}" ); @=} +| "%verbose" {@> TeX_( "/yy0{/nx/optionflag{verbose}{}/the/yy(1)}" ); @=} +| "%yacc" {@> TeX_( "/yy0{/nx/optionflag{yacc}{}/the/yy(1)}" ); @=} +| ";" {@> TeX_( "/yy0{/nx/empty}" ); @=} +; + +params: + params "{...}" {@> TeX_( "/yy0{/the/yy(1)/nx/braceit/the/yy(2)}" ); @=} +| "{...}" {@> TeX_( "/yy0{/nx/braceit/the/yy(1)}" ); @=} +; +@g + +@ This is a typical parser action: encapsulate the `type' of the +construct just parsed and attach some auxiliary info, in this case the +stream pointers. +@<Prepare one parametric option@>= + @[TeX_( "/yy0{/nx/oneparametricoption{/the/toksa}{/the/yy(2)}/the/yy(1)}" );@]@; + +@ Some extra declarations to typeset \flex\ options and +declarations. These are not part of the \bison\ syntax but their +structure is similar enough that they can be included in the grammar. +@<Parser prologue productions@>= +@G +prologue_declaration: + flex_declaration {@> @<Carry on@> @=} +; +@g +@<\flex\ options parser productions@>@; + +@ The syntax of \flex\ options was extracted from \flex\ documentation +so it is not guaranteed to be correct. +@<\flex\ options parser productions@>= +@G +flex_declaration: + FLEX_OPTION flex_option_list {@> @<Define \flex\ option list@> @=} +| flex_state symbols.1 {@> @<Define \flex\ states@> @=} +; + +flex_state: + FLEX_STATE_X {@> TeX_( "/yy0{/nx/flexxstatedecls/the/yy(1)}" ); @=} +| FLEX_STATE_S {@> TeX_( "/yy0{/nx/flexsstatedecls/the/yy(1)}" ); @=} +; + +flex_option_list: + flex_option {@> @<Carry on@> @=} +| flex_option_list flex_option {@> @<Add a \flex\ option@> @=} +; + +flex_option: + ID {@> TeX_( "/yy0{/nx/flexoptionpair{/the/yy(1)}{}}" ); @=} +| ID "=" symbol {@> TeX_( "/yy0{/nx/flexoptionpair{/the/yy(1)}{/the/yy(3)}}" ); @=} +; +@g + +@ @<Define \flex\ option list@>= + @[TeX_( "/yy0{/nx/flexoptiondecls{/the/yy(2)}/the/yy(1)}" );@]@; + +@ @<Define \flex\ states@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/getthird{/yy(1)}/to/toksc" );@]@; + @[TeX_( "/yy0{/the/toksa{/the/yy(2)}{/the/toksb}{/the/toksc}}" );@]@; + +@ @<Add a \flex\ option@>= + @[TeX_( "/getsecond{/yy(2)}/to/toksa" );@]@; /* the identifier */ + @[TeX_( "/getfourth{/toksa}/to/toksb" );@]@; /* the format pointer */ + @[TeX_( "/getfifth{/toksa}/to/toksc" );@]@; /* the stash pointer */ + @[TeX_( "/yy0{/the/yy(1)/nx/hspace{/the/toksb}{/the/toksc}/the/yy(2)}" );@]@; + +@ {\it Grammar declarations}. These declarations can appear in both +prologue and the rules sections. Their treatment is very similar to +prologue-only options. +@<Parser common productions@>= +@G +grammar_declaration: + precedence_declaration {@> @<Carry on@> @=} +| symbol_declaration {@> @<Carry on@> @=} +| "%start" symbol {@> @[TeX_( "/toksa{start}" );@]@+@<Prepare one parametric option@> @=} +| code_props_type "{...}" generic_symlist {@> @<Assign a code fragment to symbols@> @=} +| "%default-prec" {@> TeX_( "/yy0{/nx/optionflag{default prec.}{}/the/yy(1)}" ); @=} +| "%no-default-prec" {@> TeX_( "/yy0{/nx/optionflag{no default prec.}{}/the/yy(1)}" ); @=} +| "%code" "{...}" {@> TeX_( "/yy0{/nx/codeassoc{code}{}/the/yy(2)/the/yy(1)}" ); @=} +| "%code" ID "{...}" {@> TeX_( "/yy0{/nx/codeassoc{code}{/the/yy(2)}/the/yy(3)/the/yy(1)}" ); @=} +; + +code_props_type: + "%destructor" {@> TeX_( "/yy0{{destructor}/the/yy(1)}" ); @=} +| "%printer" {@> TeX_( "/yy0{{printer}/the/yy(1)}" ); @=} +; +@g + +@ @<Assign a code fragment to symbols@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; /* name of the property */ + @[TeX_( "/getfirst{/yy(2)}/to/toksb" );@]@; /* contents of the braced code */ + @[TeX_( "/getsecond{/yy(2)}/to/toksc" );@]@; /* braced code format pointer */ + @[TeX_( "/getthird{/yy(2)}/to/toksd" );@]@; /* braced code stash pointer */ + @[TeX_( "/getsecond{/yy(1)}/to/tokse" );@]@; /* code format pointer */ + @[TeX_( "/getthird{/yy(1)}/to/toksf" );@]@; /* code stash pointer */ + @[TeX_( "/yy0{/nx/codepropstype{/the/toksa}{/the/toksb}{/the/yy(3)}{/the/toksc}{/the/toksd}{/the/tokse}{/the/toksf}}" );@]@; + +@ @<Tokens and types...@>= +@G +%token PERCENT_UNION "%union"; +@g + +@ @<Parser common productions@>= +@G +@t}\vb{\inline\flatten}{@> +union_name: + {@> TeX_( "/yy0{}" ); @=} +| ID {@> @<Carry on@> @=} +; + +grammar_declaration: + "%union" union_name "{...}" {@> @<Prepare union definition@> @=} +; + +symbol_declaration: + "%type" TAG symbols.1 {@> @<Define symbol types@> @=} +; +@t}\vb{\resetf\flatten}{@> +precedence_declaration: + precedence_declarator tag.opt symbols.prec {@> @<Define symbol precedences@> @=} +; + +precedence_declarator: + "%left" {@> TeX_( "/yy0{/nx/preckind{left}/the/yy(1)}" ); @=} +| "%right" {@> TeX_( "/yy0{/nx/preckind{right}/the/yy(1)}" ); @=} +| "%nonassoc" {@> TeX_( "/yy0{/nx/preckind{nonassoc}/the/yy(1)}" ); @=} +| "%precedence" {@> TeX_( "/yy0{/nx/preckind{precedence}/the/yy(1)}" ); @=} +; +@t}\vb{\inline}{@> +tag.opt: + {@> TeX_( "/yy0{}" ); @=} +| TAG {@> @<Carry on@> @=} +; +@g + +@ @<Prepare union definition@>= + @[TeX_( "/yy0{/nx/codeassoc{union}{/the/yy(2)}/the/yy(3)/the/yy(1)}" );@]@; + +@ @<Define symbol types@>= + @[TeX_( "/yy0{/nx/typedecls{/the/yy(2)}{/the/yy(3)}/the/yy(1)}" );@]@; + +@ @<Define symbol precedences@>= + @[TeX_( "/getthird{/yy(1)}/to/toksa" );@]@; /* format pointer */ + @[TeX_( "/getfourth{/yy(1)}/to/toksb" );@]@; /* stash pointer */ + @[TeX_( "/getsecond{/yy(1)}/to/toksc" );@]@; /* kind of precedence */ + @[TeX_( "/yy0{/nx/precdecls{/the/toksc}{/the/yy(2)}{/the/yy(3)}{/the/toksa}{/the/toksb}}" );@]@; + +@ The bootstrap grammar forms the smallest subset of the full grammar. +@<Parser common productions@>= + @<Parser bootstrap productions@>@; + +@ These are the two most important rules for the bootstrap parser. +@<Parser bootstrap productions@>= +@G +@t}\vb{\flatten}{@> +symbol_declaration: + "%nterm" {} symbol_defs.1 {@> TeX_( "/yy0{/nx/ntermdecls{/the/yy(3)}/the/yy(1)}" ); @=} +@t}\vb{\fold\flatten}{@> +| "%token" {} symbol_defs.1 {@> TeX_( "/yy0{/nx/tokendecls{/the/yy(3)}/the/yy(1)}" ); @=} +; +@g + +@ {\it Just like \prodstyle{symbols.1} but accept \prodstyle{INT} for +the sake of \POSIX}. Perhaps the only point worth mentioning here is +the inserted separator (\.{\\hspace}). Like any other separator, it takes +two parameters, stream pointers. In this case, however, both pointers are null +since there seems to be no other meaningful assignment. If any +formatting or stash information is needed, it can be extracted by the +symbols themselves. +@<Parser common productions@>= +@G +symbols.prec: + symbol.prec {@> @<Carry on@> @=} +| symbols.prec symbol.prec {@> TeX_( "/yy0{/the/yy(1)/nx/hspace{0}{0}/the/yy(2)}" ); @=} +; + +symbol.prec: + symbol {@> TeX_( "/yy0{/nx/symbolprec{/the/yy(1)}{}}" ); @=} +| symbol INT {@> TeX_( "/yy0{/nx/symbolprec{/the/yy(1)}{/the/yy(2)}}" ); @=} +; +@g + +@ {\it One or more symbols to be \prodstyle{\%type}'d}. +@<Parser common productions@>= + @<List of symbols@>@; + +@ @<List of symbols@>= +@G +symbols.1: + symbol {@> @<Carry on@> @=} +| symbols.1 symbol {@> TeX_( "/yy0{/the/yy(1)/nx/hspace{0}{0}/the/yy(2)}" ); @=} +; +@g + +@ @<Parser common productions@>= +@G +generic_symlist: + generic_symlist_item {@> @<Carry on @> @=} +| generic_symlist generic_symlist_item {@> TeX_( "/yy0{/the/yy(1)/nx/hspace{0}{0}/the/yy(2)}" ); @=} +; +@t}\vb{\flatten\inline}{@> +generic_symlist_item: + symbol {@> @<Carry on@> @=} +| tag {@> @<Carry on@> @=} +; + +tag: + TAG {@> @<Carry on@> @=} +| "<*>" {@> @<Carry on@> @=} +| "<>" {@> @<Carry on@> @=} +; +@g + +@ {\it One token definition}. +@<Parser bootstrap productions@>= +@G +symbol_def: + TAG {@> @<Carry on@> @=} +@t}\vb{\flatten}{@> +| id {@> TeX_( "/yy0{/nx/onesymbol{/the/yy(1)}{}{}}" ); @=} +| id INT {@> TeX_( "/yy0{/nx/onesymbol{/the/yy(1)}{/the/yy(2)}{}}" ); @=} +| id string_as_id {@> TeX_( "/yy0{/nx/onesymbol{/the/yy(1)}{}{/the/yy(2)}}" ); @=} +| id INT string_as_id {@> TeX_( "/yy0{/nx/onesymbol{/the/yy(1)}{/the/yy(2)}{/the/yy(3)}}" ); @=} +; +@g + +@ {\it One or more symbol definitions}. +@<Parser bootstrap productions@>= +@G +symbol_defs.1: + symbol_def {@> @<Carry on@> @=} +| symbol_defs.1 symbol_def {@> @<Add a symbol definition@> @=} +; +@g + +@ @<Add a symbol definition@>= + @[TeX_( "/getsecond{/yy(2)}/to/toksa" );@]@; /* the identifier */ + @[TeX_( "/getfourth{/toksa}/to/toksb" );@]@; /* the format pointer */ + @[TeX_( "/getfifth{/toksa}/to/toksc" );@]@; /* the stash pointer */ + @[TeX_( "/yy0{/the/yy(1)/nx/hspace{/the/toksb}{/the/toksc}/the/yy(2)}" );@]@; + +@ {\it The grammar section: between the two +\prodstyle{\%\%}'s}. Finally, the following few short sections define +the syntax of \bison's rules. +@<Parser grammar productions@>= +@G +grammar: + rules_or_grammar_declaration {@> @<Start with a production cluster@> @=} +| grammar rules_or_grammar_declaration {@> @<Attach a productions cluster@> @=} +; +@g + +@ {\it As a \bison\ extension, one can use the grammar declarations in the +body of the grammar}. What follows is the syntax of the right hand +side of a grammar rule. +@<Parser grammar productions@>= +@G +rules_or_grammar_declaration: + rules {@> @<Add a productions cluster@> @=} +| grammar_declaration ";" {@> @<Carry on@> @=} +| error ";" {@> TeX_( "/errmessage{parsing error!}" ); @=} +; +@t}\vb{\flatten\inline}{@> +rules: + id_colon named_ref.opt {@> TeX_( "/relax" ); @=} + rhses.1 {@> @<Complete a production@> @=} +; +@t}\vb{\resetf}{@> +rhses.1[o]: + rhs {@> @<Start the right hand side@> @=} +| rhses.1[a] "|"[b] {@> @<Insert local formatting@> @=}[c] + rhs[d] {@> @<Add a right hand side to a production@> @=} +| rhses.1 ";" {@> @<Add an optional semicolon@> @=} +; +@g + +@ The next few actions describe what happens when a left hand side is +attached to a rule. +@<Start with a production cluster@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/yy0{/nx/grammar{/the/yy(1)}{/the/toksa}}" );@]@; + +@ @<Attach a productions cluster@>= + @[TeX_( "/getthird{/yy(1)}/to/toksa" );@]@; /* type of the last rule */ + @[TeX_( "/getsecond{/yy(1)}/to/toksc" );@]@; /* accumulated rules */ + @[TeX_( "/getfirst{/yy(2)}/to/toksb" );@]@; /* type of the new rule */ + @[TeX_( "/let/default/positionswitchdefault" );@]@; + @[TeX_( "/switchon{/the/toksb}/in/positionswitch" );@]@; /* determine the position of the first token in the group */ + @[TeX_( "/edef/next{/the/toksa}" );@]@; + @[TeX_( "/edef/default{/the/toksb}" );@]@; /* reuse \.{\\default} */ + @[TeX_( "/ifx/next/default" );@]@; + @[TeX_( " /let/default/separatorswitchdefaulteq" );@]@; + @[TeX_( " /switchon{/the/toksa}/in/separatorswitcheq" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /concat/toksa/toksb" );@]@; + @[TeX_( " /let/default/separatorswitchdefaultneq" );@]@; + @[TeX_( " /switchon{/the/toksa}/in/separatorswitchneq" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yy0{/nx/grammar{/the/toksc/the/postoks/the/toksd/the/yy(2)}{/the/toksb}}" );@]@; + +@ @<Add a productions cluster@>= + @[TeX_( "/getsecond{/yy(1)}/to/toksa" );@]@; /* \.{\\prodheader} */ + @[TeX_( "/getsecond{/toksa}/to/toksb" );@]@; /* \.{\\idit} */ + @[TeX_( "/getfourth{/toksb}/to/toksc" );@]@; /* format stream pointer */ + @[TeX_( "/getfifth{/toksb}/to/toksd" );@]@; /* stash stream pointer */ + @[TeX_( "/getthird{/yy(1)}/to/toksb" );@]@; /* \.{\\rules} */ + @[TeX_( "/yy0{/nx/oneproduction{/the/toksa/the/toksb}{/the/toksc}{/the/toksd}}" );@]@; + +@ @<Complete a production@>= + @[TeX_( "/getfourth{/yy(1)}/to/toksa" );@]@; /* format stream pointer */ + @[TeX_( "/getfifth{/yy(1)}/to/toksb" );@]@; /* stash stream pointer */ + @[TeXb( "/yy0{/nx/pcluster{/nx/prodheader{/the/yy(1)}{/the/yy(2)}" );@]@; + @[TeXao( "{/the/toksa}{/the/toksb}}{/the/yy(4)}}" );@]@; + +@ It is important to format the right hand side properly, since we +would like to indicate that an action is inlined by an +indentation. The `format' of the \.{\\rhs} `structure' includes the +stash pointers and a `boolean' to indicate whether the right hand side ends +with an action. Since the action can be implicit, this decision has to +be postponed until, say, a semicolon is seen. +No formatting or stash pointers are added for such implicit action. +@<Start the right hand side@>= + @[TeX_( "/rhsbool{/yy(1)}/to/toksa /the/toksa" );@]@; + @[TeX_( "/getthird{/yy(1)}/to/toksb" );@]@; /* the format pointer */ + @[TeX_( "/getfourth{/yy(1)}/to/toksc" );@]@; /* the stash pointer */ + @[TeX_( "/ifrhsfull" );@]@; + @[TeX_( " /yy0{/nx/rules{/the/yy(1)}{/the/toksb}{/the/toksc}}" );@]@; + @[TeX_( "/else" );@]@; /* it does not end with an action, fake one */ + @[TeX_( " /rhscont{/yy(1)}/to/toksa" );@]@; /* rules */ + @[TeX_( " /edef/next{/the/toksa}" );@]@; + @[TeX_( " /ifx/next/empty" );@]@; + @[TeX_( " /toksa{/emptyterm}" );@]@; + @[TeX_( " /fi" );@]@; + @[TeXb( " /yy0{/nx/rules{/nx/rhs{/the/toksa/nx/rarhssep{0}{0}" );@]@; + @[TeXfo( " /nx/actbraces{}{}{0}{0}/nx/bdend}{}{/nx/rhsfulltrue}}{/the/toksb}{/the/toksc}}" );@]@; + @[TeX_( "/fi" );@]@; + +@ Using standard notation, here is what the middle action +does. +@<Old `Insert local formatting'@>= + @[TeX_( "/rhscont{/yy(1)}/to{/yy(0)}" );@]@; + @[TeX_( "/yy0{/the/yy(0)/nx/midf/the/yy(2)}" );@]@; + +@ However, if the length of the rule preceding the inline action +is not known a different way of accessing the stack is necessary. +@<Insert local formatting@>= + @[TeX_( "/bb2{/toksa}/bb1{/toksb}" );@]@; + @[TeX_( "/rhscont{/toksa}/to{/yy(0)}" );@]@; + @[TeX_( "/yy0{/the/yy(0)/nx/midf/the/toksb}" );@]@; + +@ No pointers are provided for an {\it implicit\/} action. +@<Add a right hand side to a production@>= + @[TeX_( "/rhsbool{/yy(4)}/to/toksa /the/toksa" );@]@; + @[TeX_( "/ifrhsfull" );@]@; + @[TeX_( " /yy0{/nx/rules{/the/yy(3)/nx/rrhssep/the/yy(2)/the/yy(4)}/the/yy(2)}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /rhscont{/yy(4)}/to/toksa" );@]@; + @[TeX_( " /edef/next{/the/toksa}" );@]@; + @[TeX_( " /ifx/next/empty" );@]@; + @[TeX_( " /toksa{/emptyterm}" );@]@; + @[TeX_( " /fi" );@]@; + @[TeXb( " /yy0{/nx/rules{/the/yy(3)/nx/rrhssep/the/yy(2)" );@]@; + @[TeXf( " /nx/rhs{/the/toksa/nx/rarhssep{0}{0}" );@]@; /* streams have already been grabbed */ + @[TeXfo( " /nx/actbraces{}{}{0}{0}/nx/bdend}{}{/nx/rhsfulltrue}}/the/yy(2)}" );@]@; + @[TeX_( "/fi" );@]@; + +@ @<Add an optional semicolon@>= + @<Carry on@>@; + +@ @<Tokens and types...@>= +@G +%token PERCENT_EMPTY "%empty"; +@g + +@ The centerpiece of the grammar is the syntax of the right hand side +of a production. Various `precedence hints' must be attached to an +appropriate portion of the rule, just before an action (which can +be inline, implicit or both in this case). +@<Parser grammar productions@>= +@G +rhs: + {@> @<Make an empty right hand side@> @=} +| rhs symbol named_ref.opt {@> @<Add a term to the right hand side@> @=} +| rhs "{...}" named_ref.opt {@> @<Add an action to the right hand side@> @=} +| rhs "%?{...}" {@> @<Add a predicate to the right hand side@> @=} +| rhs "%empty" {@> @<Add \prodstyle{\%empty} to the right hand side@> @=} +| rhs "%prec" symbol {@> @<Add a precedence directive to the right hand side@> @=} +| rhs "%dprec" INT {@> @<Add a \prodstyle{\%dprec} directive to the right hand side@> @=} +| rhs "%merge" TAG {@> @<Add a \prodstyle{\%merge} directive to the right hand side@> @=} +; + +named_ref.opt: + {@> @<Create an empty named reference@> @=} +| BRACKETED_ID {@> @<Create a named reference@> @=} +; +@g + +@ @<Make an empty right hand side@>= + @[TeX_( "/yy0{/nx/rhs{}{}{/nx/rhsfullfalse}}" );@]@; + +@ @<Add a term to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhscnct{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/edef/next{/the/toksb}" );@]@; + @[TeX_( "/ifx/next/empty" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /getfourth{/yy(2)}/to/toksc" );@]@; + @[TeX_( " /getfifth{/yy(2)}/to/toksd" );@]@; + @[TeX_( " /appendr/toksb{{/the/toksc}{/the/toksd}}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeXb( "/yy0{/nx/rhs{/the/toksa/the/toksb" );@]@; + @[TeXao( "/nx/termname{/the/yy(2)}{/the/yy(3)}}{/nx/hspace}{/nx/rhsfullfalse}}" );@]@; + +@ @<Add an action to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhsbool{/yy(1)}/to/toksb /the/toksb" );@]@; + @[TeX_( "/ifrhsfull" );@]@; /* the first half ends with an action */ + @[TeX_( " /appendr/toksa{/nx/arhssep{0}{0}/nx/emptyterm}" );@]@; /* no pointers to streams */ + @[TeX_( "/fi" );@]@; + @[TeX_( "/edef/next{/the/toksa}" );@]@; + @[TeX_( "/ifx/next/empty" );@]@; + @[TeX_( " /toksa{/emptyterm}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/getfirst{/yy(2)}/to/toksb" );@]@; /* the contents of the braced code */ + @[TeX_( "/getsecond{/yy(2)}/to/toksc" );@]@; /* the format stream pointer */ + @[TeX_( "/getthird{/yy(2)}/to/toksd" );@]@; /* the stash stream pointer */ + @[TeXb( "/yy0{/nx/rhs{/the/toksa/nx/rarhssep{/the/toksc}{/the/toksd}" );@]@; + @[TeXf( " /nx/actbraces{/the/toksb}{/the/yy(3)}{/the/toksc}{/the/toksd}/nx/bdend}" );@]@; + @[TeXfo( " {/nx/arhssep}{/nx/rhsfulltrue}}" );@]@; + +@ @<Add a predicate to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhsbool{/yy(1)}/to/toksb /the/toksb" );@]@; + @[TeX_( "/ifrhsfull" );@]@; /* the first half ends with an action */ + @[TeX_( " /appendr/toksa{/nx/arhssep{0}{0}/nx/emptyterm}" );@]@; /* no pointers to streams */ + @[TeX_( "/fi" );@]@; + @[TeX_( "/edef/next{/the/toksa}" );@]@; + @[TeX_( "/ifx/next/empty" );@]@; + @[TeX_( " /toksa{/emptyterm}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/getfirst{/yy(2)}/to/toksb" );@]@; /* the contents of the braced code */ + @[TeX_( "/getsecond{/yy(2)}/to/toksc" );@]@; /* the format stream pointer */ + @[TeX_( "/getthird{/yy(2)}/to/toksd" );@]@; /* the stash stream pointer */ + @[TeXb( "/yy0{/nx/rhs{/the/toksa/nx/rarhssep{/the/toksc}{/the/toksd}" );@]@; + @[TeXf( " /nx/bpredicate{/the/toksb}{}{/the/toksc}{/the/toksd}/nx/bdend}" );@]@; + @[TeXao( "{/nx/arhssep}{/nx/rhsfulltrue}}" );@]@; + +@ @<Add \prodstyle{\%empty} to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhscnct{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/edef/next{/the/toksb}" );@]@; + @[TeX_( "/ifx/next/empty" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /getfourth{/yy(2)}/to/toksc" );@]@; + @[TeX_( " /getfifth{/yy(2)}/to/toksd" );@]@; + @[TeX_( " /appendr/toksb{{/the/toksc}{/the/toksd}}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeXb( "/yy0{/nx/rhs{/the/toksa/the/toksb" );@]@; + @[TeXao( "/nx/emptyterm}{/nx/hspace}{/nx/rhsfullfalse}}" );@]@; + +@ @<Add a precedence directive to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhscnct{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/rhsbool{/yy(1)}/to/toksc /the/toksc" );@]@; + @[TeX_( "/ifrhsfull" );@]@; + @[TeX_( " /yy0{/nx/sprecop{/the/yy(3)}/the/yy(2)}" );@]@; /* reuse \.{\\yyval} */ + @[TeX_( " /supplybdirective/toksa/yyval" );@]@; /* the directive is `absorbed' by the action */ + @[TeX_( " /yy0{/nx/rhs{/the/toksa}{/the/toksb}{/nx/rhsfulltrue}}" );@]@; + @[TeX_( "/else" );@]@; + @[TeXb( " /yy0{/nx/rhs{/the/toksa" );@]@; + @[TeXao( "/nx/sprecop{/the/yy(3)}/the/yy(2)}{/the/toksb}{/nx/rhsfullfalse}}" );@]@; + @[TeX_( "/fi" );@]@; + +@ @<Add a \prodstyle{\%dprec} directive to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhscnct{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/rhsbool{/yy(1)}/to/toksc /the/toksc" );@]@; + @[TeX_( "/ifrhsfull" );@]@; + @[TeX_( " /yy0{/nx/dprecop{/the/yy(3)}/the/yy(2)}" );@]@; /* reuse \.{\\yyval} */ + @[TeX_( " /supplybdirective/toksa/yyval" );@]@; /* the directive is `absorbed' by the action */ + @[TeX_( " /yy0{/nx/rhs{/the/toksa}{/the/toksb}{/nx/rhsfulltrue}}" );@]@; + @[TeX_( "/else" );@]@; + @[TeXb( " /yy0{/nx/rhs{/the/toksa" );@]@; + @[TeXao( "/nx/dprecop{/the/yy(3)}/the/yy(2)}{/the/toksb}{/nx/rhsfullfalse}}" );@]@; + @[TeX_( "/fi" );@]@; + +@ @<Add a \prodstyle{\%merge} directive to the right hand side@>= + @[TeX_( "/rhscont{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/rhscnct{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/rhsbool{/yy(1)}/to/toksc /the/toksc" );@]@; + @[TeX_( "/ifrhsfull" );@]@; + @[TeX_( " /yy0{/nx/mergeop{/the/yy(3)}/the/yy(2)}" );@]@; /* reuse \.{\\yyval} */ + @[TeX_( " /supplybdirective/toksa/yyval" );@]@; /* the directive is `absorbed' by the action */ + @[TeX_( " /yy0{/nx/rhs{/the/toksa}{/the/toksb}{/nx/rhsfulltrue}}" );@]@; + @[TeX_( "/else" );@]@; + @[TeXb( " /yy0{/nx/rhs{/the/toksa" );@]@; + @[TeXao( "/nx/mergeop{/the/yy(3)}/the/yy(2)}{/the/toksb}{/nx/rhsfullfalse}}" );@]@; + @[TeX_( "/fi" );@]@; + +@ @<Create an empty named reference@>= + @[TeX_( "/yy0{}" );@]@; + +@ @<Create a named reference@>= + @<Carry on@>@; + +@ Identifiers. +{\it Identifiers are returned as |uniqstr| values by the scanner. +Depending on their use, we may need to make them genuine symbols}. We, +on the other hand simply copy the values returned by the scanner. +@<Parser bootstrap productions@>= +@G +id: + ID {@> @<Turn an identifier into a term@> @=} +| CHAR {@> @<Turn a character into a term@> @=} +; +@g + +@ @<Parser common productions@>= + @<Definition of \prodstyle{symbol}@>@; + +@ @<Definition of \prodstyle{symbol}@>= +@G +symbol: + id {@> @<Turn an identifier into a symbol@> @=} +| string_as_id {@> @<Turn a string into a symbol@> @=} +; +@g + +@ @<Parser grammar productions@>= +@G +@t}\vb{\inline}{@> +id_colon: + ID_COLON {@> @<Prepare the left hand side@> @=} +; +@g + +@ A string used as an \prodstyle{ID}. +@<Parser bootstrap productions@>= +@G +@t}\vb{\inline}{@> +string_as_id: + STRING {@> @<Prepare a string for use@> @=} +; +@g + +@ The remainder of the action code is trivial but we reserved the +placeholders for the appropriate actions in case the parser gains some +sophistication in processing low level types (or starts expecting +different types from the scanner). +@<Turn an identifier into a term@>= + @<Carry on@>@; + +@ @<Turn a character into a term@>= + @<Carry on@>@; + +@ @<Turn an identifier into a symbol@>= + @<Carry on@>@; + +@ @<Turn a string into a symbol@>= + @<Carry on@>@; + +@ @<Prepare the left hand side@>= + @<Carry on@>@; + +@ @<Prepare a string for use@>= + @<Carry on@>@; + +@ {\it Variable and value. +The \prodstyle{STRING} form of variable is deprecated and is not \.{M4}-friendly. +For example, \.{M4} fails for \.{\%define "[" "value"}.} +@<Parser prologue productions@>= +@G +@t}\vb{\flatten\inline}{@> +variable: + ID {@> @<Carry on@> @=} +| STRING {@> @<Carry on@> @=} +; + +value: + {@> TeX_( "/yy0{}" ); @=} +| ID {@> @<Carry on@> @=} +| STRING {@> @<Carry on@> @=} +| "{...}" {@> TeX_( "/yy0{/nx/bracedvalue/the/yy(1)}" ); @=} +; +@g + +@ @<Parser common productions@>= +@G +@t}\vb{\flatten\inline}{@> +epilogue.opt: + {@> TeX_( "/yy0{}" ); @=} +| "%%" EPILOGUE {} +; +@g + +@ \Cee\ preamble for the grammar parser. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<Grammar parser \Cee\ preamble@>= + +@ \Cee\ postamble for the grammar parser. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<Grammar parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) yyprint (file, type, value) + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ @<Bootstrap parser \Cee\ postamble@>= + @<Grammar parser \Cee\ postamble@>@; + @<Bootstrap token output@>@; + +@ @<Bootstrap token output@>= + void bootstrap_tokens( char *bootstrap_token_format ) { + +#define _register_token_d(name) fprintf( tables_out, bootstrap_token_format, #name, name, #name ); + @<Bootstrap token list@>@; +#undef _register_token_d + + } + +@ \namedspot{bootstraptokens}Here is the minimal list of tokens needed +to make the lexer operational just enough to extract the rest of the +token information from the grammar. +@<Bootstrap token list@>= + _register_token_d(ID)@; + _register_token_d(PERCENT_TOKEN)@; + _register_token_d(STRING)@; + +@q The tokens below are not required to make a minimal bootstrapping parser work @> +@q but they do appear in the rules the parser will encounter while extracting @> +@q token information. @> +@q _register_token_d(INT) /* only encountered in GRAM_EOF definition which is never used */ @> +@q _register_token_d(CHAR) /* \bison\ never declares character tokens */ @> +@q _register_token_d(SEMICOLON) /* can be omitted in prologue */ @> +@q _register_token_d(TAG) /* only encountered in the definition of PERCENT_PARAM */ @> + +@ Union of types. +@<Union of grammar parser types@>= diff --git a/support/splint/cweb/bs.w b/support/splint/cweb/bs.w new file mode 100644 index 0000000000..a2003d81fa --- /dev/null +++ b/support/splint/cweb/bs.w @@ -0,0 +1,706 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1\bison\ specific routines. +The placeholder code left blank in the common routines is filed in +with the code relevant to the output of parser tables in the following sections. + +@*2Tables. \namedspot{bsfile}Here are all the parser table names. Some tables are not output but adding +one to the list in the future will be easy: it does not even have to +be done here. +@<Parser table names@>= + _register_table_d(yytranslate)@; + _register_table_d(yyr1)@; + _register_table_d(yyr2)@; + _register_table_d(yydefact)@; + _register_table_d(yydefgoto)@; + _register_table_d(yypact)@; + _register_table_d(yypgoto)@; + _register_table_d(yytable)@; + _register_table_d(yycheck)@; + _register_table_d(yytoknum)@; + _register_table_d(yystos)@; + _register_table_d(yytname)@; + _register_table_d(yyprhs)@; + _register_table_d(yyrhs)@; + +@ One special table requires a little bit more preparation. This is a +table that lists the depth of the stack before an implicit terminal. It +is not one of the tables that is used by \bison\ itself but is needed +if the symbolic name processing is to be implemented (\bison\ has +access to this information `on the fly'). The `new' \bison\ (starting +with version~\.{3.0}) does not generate |yyprhs| and |yyrhs| or any +other arrays that contain similar information, so we fake them here if +such a crippled version of \bison\ is used. + +@<Variables and types local to the parser@>= + unsigned int yyrthree[YYNRULES + 1] = { 0 }; +#ifdef BISON_IS_CRIPPLED + unsigned int yyrhs[YYNRULES + 1] = { -1 }; + unsigned int yyprhs[YYNRULES + 1] = { 0 }; +#endif + +@ We populate this table below $\ldots$ +@<Parser defaults@>= +#ifndef BISON_IS_CRIPPLED + assert( YYNRULES + 1 == sizeof(yyprhs)/sizeof(yyprhs[0]) ); + + { int i, j; + + for ( i = 1; i <= YYNRULES; i++ ) { + + for ( j = 0; yyrhs[ yyprhs[i] + j ] != -1; j++ ) { + + assert( yyprhs[i] + j < sizeof(yyrhs) ); + assert( j < yyr1[i] ); + + if ( @<This is an implicit term@> ) { + + @<Find the rule that defines it and set |yyrthree|@>@; + + } + + } + } + } +#endif + +@ @<This is an implicit term@>= + ( strlen( yytname[ yyrhs[yyprhs[i]+j] ] ) > 1 ) && + ( yytname[ yyrhs[yyprhs[i]+j] ][0] == '$' ) && + ( yytname[ yyrhs[yyprhs[i]+j] ][1] == '@@' ) + +@ @<Find the rule that defines it...@>= + int rule_number; + + for ( rule_number = 1; rule_number < YYNRULES; rule_number++ ) { + + if ( yyr1[rule_number] == yyrhs[yyprhs[i]+j] ) { + + yyrthree[rule_number] = j; + break; + + } + } + + assert( rule_number < YYNRULES ); + +@ $\ldots$ and add its name to the list. +@<Parser table names@>= + _register_table_d(yyrthree)@; + +@*2Actions. There are several ways of making |yyparse()| execute all portions of +the action code. The one chosen here makes sure that none of the +tables gets written past its last element. To see how it works, it +might be helpful to `walk through' \bison's output to see how each +change affects the generated parser. +@<Output parser semantic actions@>= + if ( output_desc.output_actions ) { + + int i, j; + + fprintf( tables_out, "%s", action_desc.preamble ); + + if ( !bare_actions ) { + + yypact[0] = YYPACT_NINF; + yypgoto[0] = -1; + yydefgoto[0] = YYFINAL; + + } + + for ( i = 1; i < sizeof(yyr1)/sizeof(yyr1[0]); i++ ) { + + fprintf( tables_out, action_desc.act_setup, i, yyr2[i] - 1 ); + + if ( action_desc.print_rule ) { + + action_desc.print_rule( i ); + + } + + if ( yyr2[i] > 0 ) { + + if ( action_desc.action1 ) { + + fprintf( tables_out, "%s", action_desc.action1 ); + + } + } + + for ( j = 2; j <= yyr2[i]; j++ ) { + + if ( action_desc.actionn ) { + + fprintf( tables_out, action_desc.actionn, j ); + } + + } + + if ( !bare_actions ) { + + yyr1[i] = YYNTOKENS; + yydefact[0] = i; + yyr2[i] = 0; + yyparse(YYPARSE_PARAMETERS); + + } + + fprintf( tables_out, action_desc.act_suffix, i, yyr2[i] - 1 ); + + } + + fprintf( tables_out, "%s", action_desc.postamble ); + + if ( action_desc.cleanup ) { + + action_desc.cleanup( &action_desc ); + + } + + } + +@*2Constants. +@<Parser constants@>= + _register_const_d(YYEMPTY)@; + _register_const_d(YYPACT_NINF)@; + _register_const_d(YYEOF)@; + _register_const_d(YYLAST)@; + _register_const_d(YYNTOKENS)@; + _register_const_d(YYNRULES)@; + _register_const_d(YYNSTATES)@; + _register_const_d(YYFINAL)@; + +@*2Tokens. +Similar techniques are employed in token output. Tokens are parser +specific (the scanner only needs their numeric values) so we need {\it +some\/} flexibility to output them in a desired format. For special +purposes (say changing the way tokens are typeset) we can control the +format tokens are output in. +@<Variables and types local to the parser@>= + char *token_format_char = NULL; + char *token_format_affix = NULL; + char *token_format_suffix = NULL; + char *bootstrap_token_format = NULL; + +@ @<Parser specific option list@>= + _register_option("token-format-char", required_argument, 0, TOKEN_FORMAT_CHAR, "")@; + _register_option("token-format-affix", required_argument, 0, TOKEN_FORMAT_AFFIX, "")@; + _register_option("token-format-suffix", required_argument, 0, TOKEN_FORMAT_SUFFIX, "")@; + _register_option("bootstrap-token-format", required_argument, 0, BOOTSTRAP_TOKEN_FORMAT, "")@; + +@ @<Higher index parser specific options@>= + TOKEN_FORMAT_CHAR,@[@] + TOKEN_FORMAT_AFFIX,@[@] + TOKEN_FORMAT_SUFFIX,@[@] + BOOTSTRAP_TOKEN_FORMAT,@[@] + +@ @<Handle parser output options@>= + case TOKEN_FORMAT_CHAR:@; + token_format_char = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(token_format_char, optarg); + break; + + case TOKEN_FORMAT_AFFIX:@; + token_format_affix = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(token_format_affix, optarg); + break; + + case TOKEN_FORMAT_SUFFIX:@; + token_format_suffix = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(token_format_suffix, optarg); + break; + + case BOOTSTRAP_TOKEN_FORMAT:@; + bootstrap_token_format = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(bootstrap_token_format, optarg); + break; + +@ @<Parser specific output descriptor fields@>= + bool output_tokens:1; + +@ No tokens are output by default. +@<Parser specific default outputs@>= + @[@].output_tokens = 0,@[@] + +@ The only part of the code below that needs any explanation is the +`bootstrap' token output. In \bison\ every token has three attributes: +its `macro name' (say, \.{STRING}) that is used by the parse code +internally, its `print name' (\.{"string"} to continue the example) +that \bison\ uses to print the token names in its diagnostic messages, +and its numeric value (that can be assigned implicitly by \bison\ +itself or explicitly by the user). Only the `print names' are kept in +the |yytname| array so to reuse the scanner used by \bison\ we either +have to extract the token `macro names' from the \Cee\ code ourselves +to pass them on to the lexer, or use a special `stripped down' version +of a \bison\ grammar parser to extract the names from the parser's +\bison\ grammar. To do this, some token names would still need to be +known to the scanner. These tokens are selected by hand to make the +`bootstrapping' parser operational. The token list for the \bison\ +grammar parser can be examined as part of the appropriate +\locallink{bootstraptokens}driver file\endlink. +@<Output parser tokens@>= + if ( output_desc.output_tokens ) { + + int i; + int length; + char token; + char *token_name; + bool too_creative = false; + + for ( i = 258; i < sizeof(yytranslate)/sizeof(yytranslate[0]) ; i++ ) { + + token_name = yytname[yytranslate[i]]; + + if ( token_name ) { + + fprintf( tables_out, token_format_affix, yytranslate[i], i ); + + length = 0; + + while ( (token = *token_name) ) { + + if ( token_format_char ) { + + length += fprintf( tables_out, token_format_char, (unsigned int)token ); + + } + + if ( token < 040 || token == 0177 ) { + + too_creative = true; + + } + + token_name++; + } + + fprintf( tables_out, token_format_suffix, too_creative ? ".unprintable." : yytname[yytranslate[i]] ); + + } + } + } + +#ifdef BISON_BOOTSTRAP_MODE + fprintf( tables_out, "\\bootstrapmodetrue\n" ); + fprintf( tables_out, "%% token values needed to bootstrap the parser\n" ); + bootstrap_tokens( bootstrap_token_format ); +#endif + +@ The size of the token name table is useful to determine, say, how +many `named' tokens the parser uses. +@<Output parser constants@>= + fprintf( tables_out, "\\constset{YYTRANSLATESIZE}{%d}%%\n", (int)(sizeof(yytranslate)/sizeof(yytranslate[0])) ); + +@*2Output modes. +The code below can be easily extended and modified to output parser +tables, actions, and constants in a language of one's choice. We are +only interested in \TeX, however, thus other modes are very +rudimentary or non-existent at this point. + +@*3 Token only mode. +Token only output mode does exactly what is expected: outputs token +names and values in the format of your choosing. + +@<Parser specific output modes@>= + TOKEN_ONLY_OUT,@[@] + +@ @<Handle parser related output modes@>= + case TOKEN_ONLY_OUT:@; + @<Prepare token only output environment@>@; + break; + +@ @<Parser specific option list@>= + _register_option("token-only-mode", no_argument, 0, TOKEN_ONLY_MODE, "")@; + +@ @<Higher index parser specific options@>= + TOKEN_ONLY_MODE,@[@] + +@ @<Configure parser output modes@>= + case TOKEN_ONLY_MODE:@; + mode = TOKEN_ONLY_OUT; + break; + +@ @<Prepare token only output environment@>= + if ( !token_format_char ) { + + token_format_char = "{%u}"; + + } + + if ( !token_format_affix ) { + + token_format_affix = "%% token: %d, token value: %d\n\\prettytoken@@{"; + + } + + if ( !token_format_suffix ) { + + token_format_suffix = "}%% %s\n"; + + } + + output_desc.output_tokens = 1; + +@*3Generic output. Generic output is not programmed yet. + +@<Parser specific output modes@>= + GENERIC_OUT,@[@] + +@ @<Handle parser related output modes@>= + case GENERIC_OUT:@; + printf( "This mode is not supported yet\n" ); + exit(0); + break; + +@*3\TeX\ output. The \TeX\ mode is the main reason for this software. + +@<Parser specific output modes@>= + TEX_OUT,@[@] + +@ @<Handle parser related output modes@>= + case TEX_OUT:@; + @<Set up \TeX\ table output for parser tables@>@; + @<Prepare \TeX\ format for semantic action output@>@; + @<Prepare \TeX\ format for parser constants@>@; + @<Prepare \TeX\ format for parser tokens@>@; + break; + +@ Some tables require name adjustments due to \TeX's +reluctance to treat digits as part of a name. +@<Set up \TeX\ table output for parser tables@>= +#define _register_table_d(name) tex_table(name); + @<Table names@>@; +#undef _register_table_d + + yyr1_desc.name = "yyrone"; + yyr2_desc.name = "yyrtwo"; + +@ The memory allocated for the |yytname| table is released at the end. +@<Helper functions declarations for for parser output@>= + void yytname_cleanup( struct table_d *table ); + int yytname_formatter_tex( FILE *stream, int index ); + int yytname_formatter( FILE *stream, int index ); + +@ There are a number of helper functions to output complicated names +in \TeX. The safest way seems to be to output those as sequences of +{\sc ASCII} codes to accommodate names like \.{\$}\.{end} +safely. \TeX's \.{\^\^}$\ldots$ convention is supported as well. +@<Helper functions for parser output@>= + void yytname_cleanup( struct table_d *table ) { + + free( table->separator ); + free( table->null ); + + } + + int yytname_formatter_tex( FILE *stream, int index ) { + + char *token_name = yytname[index]; + unsigned char token; + int length = 0; + + fprintf( stream, "\\addname " ); + + while ( (token = *token_name) ) { + + if ( token < 040 || token == 0177 ) { /* unprintable characters */ + + fprintf( stream, "^^%c", token < 0100 ? (unsigned char)(token + 0100) : (unsigned char)(token - 100) ); + length += 3; + + } else { + + fprintf( stream, "%c", token ); + length++; + + } + + token_name++; + + } + fprintf( stream, "\n" ); + + return length; + + } + + int yytname_formatter( FILE *stream, int index ) { + + char *token_name; + unsigned char token; + int length = 0; + bool too_creative = false; /* to indicate if the name is too dangerous to print */ + + fprintf( stream, "\\addname" ); + + if ( index >= 0 ) { /* this is not the last name */ + + token_name = yytname[index]; + + if ( token_name == NULL ) { + + token_name = "$impossible"; + + } + + while ( (token = *token_name) ) { + + length += fprintf( stream, "{%u}", (unsigned int)token ); + + if ( token < 040 || token == 0177 ) { + + too_creative = true; + + } + + token_name++; + + } + + fprintf( stream, "%% %s\n", too_creative ? ".unprintable." : yytname[index] ); + + } else { /* this is the last name */ + + token_name = yytname[-index]; + + if ( token_name == NULL ) { + + token_name = "$impossible"; + + } + + while ( (token = *token_name) ) { + + length += fprintf( stream, "{%u}", (unsigned int)token ); + token_name++; + + if ( token < 040 || token == 0177 ) { + + too_creative = true; + + } + + } + + fprintf( stream, "%% %s\n\\end\n%%\n", too_creative ? ".unprintable." : + ( yytname[-index] ? yytname[-index] : "end of array" ) ); + + } + + return length; + + } + +@ @<Set up \TeX\ table output for parser tables@>= + yytname_desc.preamble = "%%\n\\newtable{yytname}{}\\tempca0\\relax%% a robust way to add the yytname array\n"; + yytname_desc.separator = NULL; + yytname_desc.postamble = NULL; + yytname_desc.null = NULL; + yytname_desc.null_postamble = NULL; + yytname_desc.optimized_numeric = NULL; + yytname_desc.prettify = false; + yytname_desc.formatter = yytname_formatter; + + yytname_desc.cleanup = NULL; + + output_desc.output_yytname = 1; + +@ @<Prepare \TeX\ format for semantic action output@>= + + if ( optimize_actions ) { + + action_desc.preamble = "%\n% the big switch\n%\n"@/ + "\\catcode`\\/=0\\relax % see the documentation for an explanation of this trick\n"@/ + "\\def\\yybigswitch#1{%%\n"@/ + " \\csname dobisonaction\\number #1\\parsernamespace\\endcsname\n"@; + "}\\stashswitch{yybigswitch}%%\n"; + action_desc.act_setup = "\n\\expandafter\\def\\csname dobisonaction%d\\parsernamespace\\endcsname{%%\n%%"; + action_desc.act_suffix = "}%% end of rule %d\n"; + action_desc.action1 = NULL; + action_desc.actionn = NULL; + action_desc.postamble = "\n\\catcode`\\/=12\\relax\n\n"; + action_desc.print_rule = print_rule; + action_desc.cleanup = NULL; + output_desc.output_actions = 1; + + } else { + + action_desc.preamble = "%\n% the big switch\n%\n"@/ + "\\catcode`\\/=0\\relax % see the documentation for an explanation of this trick\n"@/ + "\\def\\yybigswitch#1{%%\n"@; + " \\ifcase#1\\relax\n"; + action_desc.act_setup = " \\or %% (rule %d) "; + action_desc.act_suffix = ""; + action_desc.action1 = NULL; + action_desc.actionn = NULL; + action_desc.postamble = " \\else\n \\fi\n}\\stashswitch{yybigswitch}%%\n\\catcode`\\/=12\\relax\n\n"; + action_desc.print_rule = print_rule; + action_desc.cleanup = NULL; + output_desc.output_actions = 1; + +} + +@ Grammar rules are listed in a readable form alongside the action +code to make it possible to quickly find an appropriate action. The +rules are not output if a crippled \bison\ is used. +@<Helper functions for parser output@>= + void print_rule( int n ) { + + int i; + + fprintf( tables_out, "%s%s: ", (n < 10 && !optimize_actions ? " " : ""), yytname[yyr1[n]] ); + +#ifndef BISON_IS_CRIPPLED + i = yyprhs[n]; + + if ( yyrhs[i] < 0 ) { + + fprintf( tables_out, "<empty>" ); + + } else { + + while( yyrhs[i] > 0 ) { + + fprintf( tables_out, "%s ", yytname[yyrhs[i]] ); + i++; + + } + + } +#endif + + fprintf( tables_out, "\n" ); + + } + +@ \TeX\ constant output is another place where the techniques described above are applied. +As before, the macro handles the repetitive work of initialization, declaration, etc in +each place where the corresponding constant is mentioned. The one exception is \.{YYPACT\_NINF}, +which has to be handled separately because the underscore in its name makes it difficult to +use it as a command sequence name. +\def\YYPACTxNINFxdesc{\.{YYPACT\_NINF\_}\\{desc}} + +@s YYPACT_NINF_desc TeX + +@<Prepare \TeX\ format for parser constants@>= +#define _register_const_d(c_name) @[c_name##_desc.format = "\\constset{%s}{%d}%%\n"; \ + c_name##_desc.name = #c_name; \ + output_desc.output_##c_name = 1;@] + @<Parser constants@>@; +#undef _register_const_d +YYPACT_NINF_desc.name = "YYPACTNINF"; + +@ Token definitions round off the \TeX\ output mode. + +@<Prepare \TeX\ format for parser tokens@>= + + token_format_char = NULL; /* do not output individual characters */ + + if ( !token_format_affix ) { + + token_format_affix = "\\tokenset{%d}{%d}"; + + } + + if ( !token_format_suffix ) { + + token_format_suffix = "%% %s\n"; + + } + + + if ( !bootstrap_token_format ) { + + bootstrap_token_format = "\\expandafter\\def\\csname token\\parsernamespace %s\\endcsname{%d}%% %s\n"; + + } + + /* |output_desc.output_tokens = 1;| is no longer necessary as it is done entirely in \TeX */ + +@*2 Command line options. +We start with the most obvious option, the one begging for help. + +@<Higher index parser specific options@>= + LONG_HELP,@[@] + +@ @<Parser specific option list@>= + _register_option("help", no_argument, 0, LONG_HELP, "")@; + +@ @<Shortcuts for command line options affecting parser output@>= + "h" + +@ @<Handle parser output options@>= + case 'h': /* short help */@; + fprintf(stderr, "Usage: %s [options] output_file\n", argv[0]); + exit(0); + break; /* should not be needed */ + + case LONG_HELP:@; + fprintf(stderr, "%s [--mode=TeX:options] output_file outputs tables\n" + " and constants for a TeX parser\n", argv[0]); + exit(0); + break; /* should not be needed */ + +@ @<Parser specific option list@>= + _register_option("debug", optional_argument, 0, 'b', "")@; + _register_option("mode", required_argument, 0, 'm', "")@; + _register_option("table-separator", required_argument, 0, 'z', "")@; + + _register_option("format", required_argument, 0, 'f', "")@; /* name? */ + _register_option("table", required_argument, 0, 't', "")@; /* specific table */ + _register_option("constant", required_argument, 0, 'c', "")@; /* specific constant */ + _register_option("name-length", required_argument, 0, 'l', "")@; /* change |MAX_NAME_LENGTH| */ + _register_option("token", required_argument, 0, 'n', "")@; /* specific token */ + _register_option("run-parse", required_argument, 0, 'p', "")@; /* run the parser */ + _register_option("parse-file", required_argument, 0, 'i', "")@; /* input for the parser */ + +@ The string below is a list of short options. + +@<Shortcuts for command line options affecting parser output@>= + "z:m:f:t:" + +@ A few options can be discussed immediately. + +@<Variables and types local to the parser@>= + char *table_separator = "%s "; + +@ @<Handle parser output options@>= + case 'm': /* output mode */@; + switch( optarg[0] ) { + + case 'T': + case 't':@; + mode = TEX_OUT; + break; + + case 'b': + case 'B': + case 'g': + case 'G':@; + mode = GENERIC_OUT; + break; + + default:@; + break; + + } + break; + + case 'z': + table_separator = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(table_separator, optarg); + break; diff --git a/support/splint/cweb/common.w b/support/splint/cweb/common.w new file mode 100644 index 0000000000..3ab5af93b4 --- /dev/null +++ b/support/splint/cweb/common.w @@ -0,0 +1,788 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@** Forcing \bison\ and \flex\ to output \TeX. +Instead of implementing a \bison\ (or \flex) `plugin' for outputting +\TeX\ parser, the code that follows produces a separate executable +that outputs all the required tables after the inclusion of an +ordinary \Cee\ parser produced by \bison\ (or a scanner produced by +\flex). The actions in both \bison\ parser and \flex\ scanner are +assumed to be merely |printf()| statements that output the `real' +\TeX\ actions. The code below simply cycles through all such actions to +output an `action switch' appropriate for use with \TeX. In every +other respect, the included parser or scanner can use any features +allowed in `real' parsers and scanners. + +@*1 Common routines. +The `top' level of the scanner and parser `drivers' is very similar, +and is therefore separated into a few sections that are common to both +drivers. The layout is fairly typical and follows a standard +`initialize-input-process-output-clean up' scheme. The logic behind each +section of the program will be explained in detail below. + +The section below is called |@<\Cee\ postamble@>| because the output of +the tables can happen only after the \bison\ (or \flex) generated +\.{.c} file is included and all the data structures are known. + +The actual `assembly' of each driver has to be done separately due to +some `singularities' of the \CWEB\ system and the design of this +software. All the essential routines are presented in the sections +below, though. +@<\Cee\ postamble@>= + +@<Outer definitions@>; +@<Global variables and types@>@; +@<Auxiliary function declarations@>@; +@<Auxiliary function definitions@>@; + +int main( int argc, char **argv ) { + + @<Local variable and type declarations@>@; + @<Establish defaults@>@; + @<Command line processing variables@>@; + @<Process command line options@>@; + + switch( mode ) { + + @<Various output modes@>@; + + default: + break; + + } + + if ( tables_out ) { + + @<Perform output@>@; + @<Output action switch, if any@>@; + + } else { + + fprintf( stderr, "No output, exiting\n" ); + exit(0); + + } + + @<Clean up@>@; + + return 0; + +} + +@ Not all the code can be supplied at this stage (most of the routines +here are at the `top' level so the specifics have to be `filled-in' by +each driver), so many of the sections +above are placeholders for the code provided by a specific +driver. However, we still need to supply a trivial definition here to +placate \CWEAVE\ whenever this portion of the code is used isolated in +documentation. +@<Various output modes@>= + +@ Standard library declarations for memory management routines, some +syntactic sugar, command line processing, and variadic functions are +all that is needed. +@<Outer definitions@>= +#include <stdlib.h> +#include <stdbool.h> +#include <stdarg.h> +#include <assert.h> +#include <string.h> + +@ This code snippet is a payment for some poor (in my view) philosophy +on the part of the \bison\ and \flex\ developers. There used to be an +option in \bison\ to output just the tables and the action code but it +had never worked correctly and it was simply dropped in the latest +version. Instead, one can only get access to \bison's goodies as part +of a tangled mess of |@[#define@]|'s and error processing code. Had the +tables and the parser function itself been considered separate, well +isolated sections of \bison's output, there would simply be no reason +for dirty tricks like the one below, one would be able to write custom +error processing functions, unicorns would roam the Earth and pixies +would hand open sourced tablets to everyone. At a minimum, it would +have been a much cleaner, modular approach. + +As of version~\.{3.0} of \bison\ some critical arrays, namely, +|yyprhs| and |yyrhs| are no longer generated (even internally) which +significantly reduces \bison's useability as a parser generator. As an +example, the |yyrthree| array, which is necessary for processing +`inline' actions is computed in \.{bs.w} using the two arrays +mentioned in the previous sentence. There does not seem to be any +other way to access this information. A number of tools (GNU and +otherwise) have taken the path of narrowing the field of application +to a few use cases invisioned by the maintainers. This includes +compilers, as well. + +There is a strange +reluctance on the part of the \gcc\ team to output any intermediate +code other than the results of preprocessing and assembly. I have seen +an argument that involves some sort of appeal to making the code +difficult to close source but the logic of it escaped me completely +(well, there {\it is\/} logic to it, however choosing poor design in +order to punish a few bad players seems like a rather inferior option). + +Ideally, there should be no such thing as a parser generator, or a +compiler, for that matter: all of these are just basic table driven +rewriting routines. Tables are hard but table driven code should not +be. If one had access to the tables themselves, and some canonical +examples of code driven by such tables, like |yyparse()| and +|yylex()|, the flexibility of these tools would improve +tremendously. Barring that, this is what we have to do {\it now}. + +There are several ways to gain write access to the data declared |const| +in \Cee, like passing its address to a function with no prototype. All +these methods have one drawback: the loopholes that make them possible +have been steadily getting on the chopping block of the \Cee\ +standards committee. Indeed, |const| data should be constant. Even if +one succeeds in getting access, there is no reason to believe that the +data is not allocated in a write-only region of the memory. The +cleanest way to get write access then is to eliminate |const| +altogether. The code should have the same semantics after that, and +the trick is only marginally bad. + +The last two definitions are less innocent (and, at least the +second one, are prohibited by the ISO standard (clause 6.10.8(2), +see~\cite[ISO/C11])) but +\gcc\ does not seem to mind, and it gets rid of warnings about +dropping a |const| qualifier whenever an |assert| is +encountered. Since the macro is not recursively expanded, this will +only work if $\ldots$|FUNCTION__| is treated as a pseudo-variable, as +it is in \gcc, not a macro. + +@d const +@d __PRETTY_FUNCTION__ @[(char *)__PRETTY_FUNCTION__@] +@d __FUNCTION__ @[(char *)__FUNCTION__@] + +@ The output file has to be known to both parts of the code, so it is +declared at the very beginning of the program. We also add some +syntactic sugar for loops. +@q The line below is simply irresistible, one should put it on a T-shirt@> +@s FOREVER TeX + +@d FOREVER for(;;) + +@<Common code for \Cee\ preamble@>= +#include <stdio.h> + FILE *tables_out; + +@ The clean-up portion of the code can be left empty, as all it does +is close the output file, which can be left to the operating system but +we take care of it ourselves to keep out code `clean'\footnote{In case +the reader has not noticed yet, this is a weak attempt at humor to +break the monotony of going through the lines of \CTANGLE'd code}. +@<Clean up@>= + fclose( tables_out ); + +@ There is a descriptor controlling the output of the program as +a whole. The code below is an example of a literate programming +technique that will be used repeatedly to maintain large structures +that can grow during the course of the program design. Note that the +name of each table is only mentioned once, the rest of the code is +generic. + +Technically speaking, all of this can be done with \Cee\ preprocessor +macros of moderate complexity, taking advantage of its expansion rules +but it is not nearly as transparent as the \CWEB\ approach. +@<Local variable and type declarations@>= + struct output_d { + + @<Output descriptor fields@>@; + + }; + + struct output_d output_desc = { @<Default outputs@> }; + +@ To declare each table field in the global output descriptor, all one has to do +is to provide a general pattern. +@<Output descriptor fields@>= +#define _register_table_d(name) @[bool output_##name:1;@] + @<Table names@>@; +#undef _register_table_d + +@ Same for assigning default values to each field. +@<Default outputs@>= +#define _register_table_d(name) @[.output_##name = 0,@] /* do not output any tables by default */ + @<Table names@>@; +#undef _register_table_d + +@ Each descriptor is populated using the same approach. +@<Local variable and type declarations@>= +#define _register_table_d(name) @[struct table_d name##_desc = {0};@] + @<Table names@>@; +#undef _register_table_d + +@ The flag \.{--optimize-tables} affects the way tables are output. +@<Global variables...@>= + static int optimize_tables = 0; + +@ It is set using the command line option below. +@<Raw option list@>= + _register_option("optimize-tables", no_argument, &optimize_tables, 1, "")@; + +@ The reason to implement the table output routine as a macro is to avoid +writing separate functions for tables of different types of data +(stings as well as integers). The output is controlled by each table's +{\it descriptor\/} defined below. A more sophisticated approach is +possible but this code is merely a `patch' so we are not after full +generality\footnote{A somewhat cleaner way to achieve the same effect +is to use the \.{\_Generic} facility of \Cee11.}. + +@d output_table(table_desc, table_name, stream) + if ( output_desc.output_##table_name ) { + + int i, j = 0; + + if ( optimize_tables ) { + + fprintf( stream, "\\setoptopt{%s}%%\n", table_desc.name ); + + if ( !table_desc.optimized_numeric ) { + + fprintf( stream, "\\beginoptimizednonnumeric{%s}%%\n", table_desc.name ); + + } + + for( i = 0; i < sizeof(table_name)/sizeof(table_name[0]) - 1; i++) { + + if ( table_desc.formatter ) { + + table_desc.formatter( stream, i ); + + } else { + + fprintf( stream, table_desc.optimized_numeric, table_desc.name, i, table_name[i] ); + + } + + } + + if ( table_desc.formatter ) { + + table_desc.formatter( stream, -i ); + + } else { + + fprintf( stream, table_desc.optimized_numeric, table_desc.name, i, table_name[i] ); + + } + + if ( table_desc.cleanup ) { + + table_desc.cleanup( &table_desc ); + + } + + } else { + + fprintf( stream, table_desc.preamble, table_desc.name ); + + for( i = 0; i < sizeof(table_name)/sizeof(table_name[0]) - 1; i++) { + + if ( table_desc.formatter ) { + + j += table_desc.formatter( stream, i ); + + } else { + + if ( table_name[i] ) { + + j += fprintf( stream, table_desc.separator, table_name[i] ); + + } else { + + j += fprintf( stream, "%s", table_desc.null ); + + } + + } + + if ( j > MAX_PRETTY_LINE && table_desc.prettify ) { + + fprintf( stream, "\n" ); + j = 0; + + } + } + + if ( table_desc.formatter ) { + + table_desc.formatter( stream, -i ); + + } else { + + if ( table_name[i] ) { + + fprintf( stream, table_desc.postamble, table_name[i] ); + + } else { + + fprintf( stream, "%s", table_desc.null_postamble ); + + } + + } + + if ( table_desc.cleanup ) { + + table_desc.cleanup( &table_desc ); + + } + } + } + +@<Global variables and types@>= + struct table_d { + + @<Generic table desciptor fields@>@; + + }; + +@ @<Generic table desciptor fields@>= + char *name; + char *preamble; + char *separator; + char *postamble; + char *null_postamble; + char *null; + char *optimized_numeric; + bool prettify; + int (*formatter)( FILE *, int ); + void (*cleanup)( struct table_d * ); + +@ Tables are output first. The action output code must come last since +it changes the values of the tables to achieve its goals. Again, a +different approach is possible, that saves the data first but +simplicity was deemed more important than total generality at this +point. +@<Perform output@>= + @<Output all tables@>@; + +@ One more application of `gather the names first then process' technique. +@<Output all tables@>= +#define _register_table_d(name) @[output_table(name##_desc, name, tables_out);@] + @<Table names@>@; +#undef _register_table_d + +@ Tables will be output by each driver. Placeholder here, for +\CWEAVE's piece of mind. +@<Table names@>= + +@ Action output invokes a totally new level of dirty code. If tables, +constants, and tokens are just data structures, actions are executable +commands. We can only hope to cycle through all the actions, which is +enough to successfully use \bison\ and \flex\ to generate \TeX. The +|switch| statement containing the actions is embedded in the parser +function so to get access to each action one has to coerce |yyparse()| to +jump to each case. Here is where we need the table manipulation. The +appropriate code is highly specific to the program used (since \bison\ +and \flex\ parsing and scanning functions had to be `reverse +engineered' to make them do what we want), +so at this point we simply declare the options controlling the level +of detail and the type of actions output. +@<Global variables...@>= + static int bare_actions = 0; /* (|static| for local variables) and |int| to pacify the compiler + (for a constant initializer and compatible type) */ + static int optimize_actions = 0; + +@ The first of the following options allows one to output an action switch without the +actions themselves. It is useful when one needs to output a \TeX\ +parser for a grammar file that is written in \Cee. In this case it +will be impossible to cycle through actions (as no setup code has been +executed), so the parser invocation is omitted. + +The second option splits the action switch into several macros to speed up +the processing of the action code. + +The last argument of the `flexible' macro below is supposed to be an +extended description of each option which can be later utilized by a +|usage()| function. +@<Raw option list@>= + _register_option("bare-actions", no_argument, &bare_actions, 1, "")@; + _register_option("optimize-actions", no_argument, &optimize_actions, 1, "")@; + +@ The rest of the action output code mimics that for table output, starting with +the descriptor. To make the output format more flexible, this +descriptor should probably be turned into a specialized routine. +@<Global variables and types@>= + struct action_d { + + char *preamble; + char *act_setup; + char *act_suffix; + char *action1; + char *actionn; + char *postamble; + void (*print_rule)( int ); + void (*cleanup)( struct action_d * ); + + }; + +@ @<Output descriptor fields@>= + bool output_actions:1; + +@ Nothing is output by default, including actions. +@<Default outputs@>= + @[@].output_actions = 0,@[@]@; + +@ @<Local variable and type declarations@>= + struct action_d action_desc = {0}; + +@ Each function below outputs the \TeX\ code of the appropriate +action when the action is `run' by the action output switch. +The main concern in designing these functions is to make the +code easier to look at. Further explanation is given in the grammar +file. If the parser is doing its job, this is the only place where one +would actually see these as functions (or, rather, macros). + +In compliance with paragraph 6.10.8(2)\footnote{[$\ldots$] {\it Any +other predefined macro names shall begin with a leading underscore +followed by an uppercase letter or a second underscore.}} of the \ISO\ +\Cee11 standard the names of these macros do not start with an +underscore, since the first letter of \.{TeX} is +uppercase\footnote{One might wonder why one of these functions is +defined as a \CWEB\ macro while the other is put into the preamble `by +hand'. It really makes no difference, however, the reason the second +macro is defined explicitly is \CWEB's lack of awareness of `variadic' +macros which produces undesirable typesetting artefacts.}. +\def\TeXx{\hbox{\.{TeX\_}}} +\def\TeXa{\hbox{\.{TeXa}}} +\def\TeXb{\hbox{\.{TeXb}}} +\def\TeXf{\hbox{\.{TeXf}}} +\def\TeXfo{\hbox{\.{TeXfo}}} +\def\TeXao{\hbox{\.{TeXao}}} +\def\TeXxx{\hbox{\.{TeX\_\_}}} +@s TeX__ TeX +@d TeX_( string ) fprintf( tables_out, " %s%%\n", string ) +@d TeXb( string ) TeX_( string ) +@d TeXa( string ) TeX_( string ) +@d TeXf( string ) TeX_( string ) +@d TeXfo( string ) TeX_( string ) +@d TeXao( string ) TeX_( string ) + +@q \CWEB\ is not aware of variadic macros, so this has to be done the old way@> +@<\Cee\ preamble@>= +#define TeX__( string, ... ) @[fprintf( tables_out, " " string "%s\n", __VA_ARGS__, "%" )@] + +@ We begin with a few macros to facilitate the output +of tables in the format that \TeX\ can understand. As there is no +perfect way to represent an array in \TeX\ a rather weak compromise +was settled upon. Further explanation of this choice is given in the \TeX\ +file that implements the \TeX\ parser for the \bison\ input grammar. + +@d tex_table_generic(table_name) + table_name##_desc.preamble = "\\newtable{%s}{%%\n"; + table_name##_desc.separator = "%d\\or "; + table_name##_desc.postamble = "%d}%%\n"; + table_name##_desc.null_postamble = "0}%%\n"; + table_name##_desc.null = "0\\or "; + table_name##_desc.optimized_numeric = "\\expandafter\\def\\csname %s\\parsernamespace %d\\endcsname{%d}%%\n"; + table_name##_desc.prettify = true; + table_name##_desc.formatter = NULL; + table_name##_desc.cleanup = NULL; + output_desc.output_##table_name = 1; + +@d tex_table(table_name) + tex_table_generic(table_name); + table_name##_desc.name = #table_name; + +@ An approach paralleling the table output +scheme is taken with constants. Since constants are \Cee\ {\it +macros\/} one has to be careful to avoid the temptation of using +constant {\it names\/} directly as names for fields in +structures. They will simply be replaced by the constants' +values. When the names are concatenated with other tokens, however, +the \Cee\ preprocessor postpones the macro expansion until the +concatenation is complete (see clauses 6.10.3.1, 6.10.3.2, and +6.10.3.3 of the \ISO\ \Cee\ Standard, \cite[ISO/C11]). Unless the result of the +concatenation is still expandable, the expansion will halt. +@<Global variables and types@>= + struct const_d { + + char *format; + char *name; + + }; + +@ @<Local variable and type declarations@>= +#define _register_const_d(c_name) @[struct const_d c_name##_desc;@] + @<Constant names@>@; +#undef _register_const_d + +@ @<Output descriptor fields@>= +#define _register_const_d(c_name) @[bool output_##c_name:1;@] + @<Constant names@>@; +#undef _register_const_d + +@ @<Default outputs@>= +#define _register_const_d(c_name) @[@[@].output_##c_name = 0,@[@]@] + @<Constant names@>@; +#undef _register_const_d + +@ @<Perform output@>= + fprintf( tables_out, "%%\n%% constant definitions\n%%\n" ); + @<Output constants@>@; + +@ @<Output constants@>= +{ + + int any_constants = 0; +#define _register_const_d(c_name) \ + \ + if ( output_desc.output_##c_name ) { \ + const_out( tables_out, c_name##_desc, c_name)@; \ + any_constants = 1; \ + } + + @<Constant names@>@; + +#undef _register_const_d + + if ( any_constants ); /* this is merely a placeholder statement */ + +} + +@ Constants are very driver specific, so to make \CWEAVE\ happy $\ldots$ + @<Constant names@>= + +@ A macro to help with constant output. +@d const_out(stream, c_desc, c_name) fprintf(stream, c_desc.format, c_desc.name, c_name); + +@ Action switch output routines modify the automata tables and +therefore have to be output last. Since action output is highly +automaton specific, we leave this section blank here, to pacify +\CWEAVE\ in case this file is typeset by itself. +@<Output action switch, if any@>= + +@*2 Error codes. + +@<Global variables and types@>= + enum err_codes{ @<Error codes@>@, LAST_ERROR }; + +@ @<Error codes@>= + NO_MEMORY, BAD_STRING, BAD_MIX_FORMAT,@[@] + +@ A lot more care is necessary to output the token table. A number of precautions are taken +to ensure that a maximum possible range of names can be passed safely to \TeX. This involves some +manipulation of \.{\\catcode}'s and control characters. The +complicated part is left to \TeX\ so the output code can be kept +simple. The helper function below is used to `combine' two strings. + +@d MAX_PRETTY_LINE 100 + +@<Auxiliary function declarations@>= + char *mix_string( char *format, ... ); + +@ @<Auxiliary function definitions@>= + char *mix_string( char *format, ... ) { + + char *buffer; + size_t size = 0; + int length = 0; + int written = 0; + char *formatp = format; + va_list ap, ap_save; + + va_start( ap, format ); + va_copy( ap_save, ap ); + + size = strnlen( format, MAX_PRETTY_LINE * 5 ); + + if ( size >= MAX_PRETTY_LINE * 5 ) { + + fprintf( stderr, "%s: runaway string?\n", __func__ ); + exit(BAD_STRING); + + } + + while ( (formatp = strstr( formatp, "%" )) ) { + + switch( formatp[1] ) { + + case 's':@; + length = strnlen( va_arg( ap, char * ), MAX_PRETTY_LINE * 5 ); + + if ( length >= MAX_PRETTY_LINE * 5 ) { + + fprintf( stderr, "%s: runaway string?\n", __func__ ); + exit(BAD_STRING); + + } + + size += length; + size -=2; + formatp++; + break; + + case '%':@; + size--; + formatp +=2; + + default: + + printf( "%s: cannot handle %%%c in mix string format\n", __func__, formatp[1] ); + exit( BAD_MIX_FORMAT ); + + } + + } + + buffer = (char *)malloc( sizeof(char) * size + 1 ); + + if ( buffer ) { + + written = vsnprintf( buffer, size + 1, format, ap_save ); + + if ( written < 0 || written > size ) { + + fprintf( stderr, "%s: runaway string?\n", __func__ ); + exit(BAD_STRING); + + } + + } else { + + fprintf( stderr, "%s: failed to allocate memory for the output string\n", __func__ ); + exit(NO_MEMORY); + + } + + va_end( ap ); + va_end( ap_save ); + + return buffer; + + } + +@*2Initial setup. Depending on the output mode (right now only \TeX\ +and `tokens only' (in the \bison\ `driver') are supported) the format of each table, action +field and token has to be set up. + +@<Local variable and type declarations@>= + enum output_mode {@<Output modes@>@, LAST_OUT}; + +@ And to calm down \CWEAVE\ $\ldots$ +@<Output modes@>= + +@ \TeX\ is the main output mode. +@<Establish defaults@>= + enum output_mode mode = TEX_OUT; + +@*2Command line processing. This program uses a standard way of parsing the command +line, based on |getopt_long|. At the heart of the setup are the array below with a +couple of supporting variables. + +@<Outer definitions@>= +#include <unistd.h> +#include <getopt.h> +#include <string.h> + +@ @<Local variable and type declarations@>= + const char *usage = "%s [options] output_file\n"; + +@ @<Command line processing variables@>= + int c, option_index = 0;@# + + enum higher_options{NON_OPTION = 0xFF, @<Higher index options@>@, LAST_HIGHER_OPTION}; + + static struct option long_options[] = { @/@[ + @<Long options array@>@;@/ + {0, 0, 0, 0} @] + };@# + +@ The main loop of the command line option processing follows. This +can be used as a template for setting up the option processing. The +specific cases are added to in the course of adding new features. + +@<Process command line options@>= + opterr = 0; /* we do our own error reporting */ + + FOREVER { + + c = getopt_long (argc, argv, ":" @<Short option list@>, long_options, &option_index); + + if (c == -1) break; + + switch (c) { + + case 0:@; /* it is a flag, the name is kept in |long_options[option_index].name|, + and the value can be found in |long_options[option_index].val| */ + break; + + @t}\4{@>@<Cases affecting the whole program@>; + @t}\4{@>@<Cases involving specific modes@>; + + case '?':@; + fprintf (stderr, "Unknown option: `%s', see `Usage' below\n\n", argv[optind - 1]); + fprintf(stderr, usage, argv[0]); + exit(1); + break; + + case ':':@; + fprintf (stderr, "Missing argument for `%s'\n\n", argv[optind - 1]); + fprintf(stderr, usage, argv[0]); + exit(1); + break; + + default:@; + printf ("warning: feature `%c' is not yet implemented\n", c); + } + + } + + if (optind >= argc) + { + + fprintf( stderr, "No output file specified!\n" ); + + } else { + + tables_out = fopen( argv[optind++], "w" ); + + } + + if (optind < argc) + { + + printf ("script files to be loaded: "); + while (optind < argc) + printf ("%s ", argv[optind++]); + putchar ('\n'); + + } + +@ @<Long options array@>= +#define _register_option(name, arg_flag, loc, val, exp) @[{name, arg_flag, loc, val},@[@]@] + @<Raw option list@>@; +#undef _register_option + +@ In addition to spelling out the full command line option name (such +as \.{--help}) |getopt_long| gives the user a choice of using a +shortcut (say, \.{-h}). As individual options are treated in drivers +themselves, there are no shortcuts to supply at this point. We leave +this section (and a number of others) empty to be filled in with the +driver specific code to pacify \CWEAVE. + +@<Short option list@>= + +@ Some options have one-letter `shortcuts', whereas others only exist +in `fully spelled-out' form. To easily keep track of the latter, a +special enumerated list is declared. To add to this list, simply add +to the \CWEB\ section below. +@<Higher index options@>= + +@ @<Cases affecting the whole program@>= + +@ @<Cases involving specific modes@>= + diff --git a/support/splint/cweb/fk.w b/support/splint/cweb/fk.w new file mode 100644 index 0000000000..04685ff9aa --- /dev/null +++ b/support/splint/cweb/fk.w @@ -0,0 +1,510 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1 \flex\ specific routines. The output of the scanner automaton +follows the steps similar to the ones taken during the parser output. +The major difference is in the output of actions and constants. +@*2 Tables. +As in the case of a parser we start with all the table names. +@<Scanner table names@>= + _register_table_d(yy_accept)@; + _register_table_d(yy_ec)@; + _register_table_d(yy_meta)@; + _register_table_d(yy_base)@; + _register_table_d(yy_def)@; + _register_table_d(yy_nxt)@; + _register_table_d(yy_chk)@; + +@*2Actions. The scanner function, |yylex()|, has been reverse +engineered to execute all portions of +the action code. The method chosen here makes sure that none of the +tables gets written past its last element. +@<Variables and types local to the scanner driver@>= + int max_yybase_entry = 0; + int max_yyaccept_entry = 0; + int max_yynxt_entry = 0; + int max_yy_ec_entry = 0; + +@ The `exotic' scanner constants treated below are the constants used +to control the scanner code itself. Unfortunately they are not given +any names that can be used by the `driver' to output them in a simple +way. +@<Compute exotic scanner constants@>= + + { + int i; + + for ( i = 0; i < sizeof( yy_base )/sizeof( yy_base[0] ); i++ ) { + + if ( yy_base[i] > max_yybase_entry ) { + + max_yybase_entry = yy_base[i]; + + } + + } + + for ( i = 0; i < sizeof( yy_nxt )/sizeof( yy_nxt[0] ); i++ ) { + + if ( yy_nxt[i] > max_yynxt_entry ) { + + max_yynxt_entry = yy_nxt[i]; + + } + + } + + for ( i = 0; i < sizeof( yy_accept )/sizeof( yy_accept[0] ); i++ ) { + + if ( yy_accept[i] > max_yyaccept_entry ) { + + max_yyaccept_entry = yy_accept[i]; + + } + } + + for ( i = 0; i < sizeof( yy_ec )/sizeof( yy_ec[0] ); i++ ) { + + if ( yy_ec[i] > max_yy_ec_entry ) { + + max_yy_ec_entry = yy_ec[i]; + + } + + } + + } + +@ @<Output scanner actions@>= + if ( output_desc.output_actions ) { + + int i, j; + yyscan_t fake_scanner; + + fprintf( tables_out, "%s", action_desc.preamble ); + + if ( !bare_actions ) { + + if ( yylex_init( &fake_scanner ) ) { + + printf( "Cannot initialize the scanner\n" ); + + } + + yy_ec[0] = 0; + yy_base[1] = max_yybase_entry; + yy_chk[max_yybase_entry] = 1; + yy_nxt[max_yybase_entry] = 1; + + } + + for ( i = 1; i <= max_yyaccept_entry; i++ ) { + + fprintf( tables_out, action_desc.act_setup, i ); + + if ( i == YY_END_OF_BUFFER ) { + + fprintf( tables_out, " %% YY_END_OF_BUFFER\n%s\n", " \\yylexeofaction" ); + + } else { + + fprintf( tables_out, "\n" ); + + if ( !bare_actions ) { + + (( struct yyguts_t *)fake_scanner)->yy_hold_char = 0; + yy_accept[1] = i; + yylex( NULL, fake_scanner ); + + } + } + + fprintf( tables_out, action_desc.act_suffix, i ); + + } + + fprintf( tables_out, " %% end of file states:\n%s\n", + " %#define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1)" + ); + + if ( max_eof_state == 0 ) { /* in case the user has not declared any states */ + + max_eof_state = YY_STATE_EOF( INITIAL ); + + } + + for ( ; i <= max_eof_state; i++ ) { + + fprintf( tables_out, action_desc.act_setup, i ); + + if ( !bare_actions ) { + + fprintf( tables_out, "\n" ); + + (( struct yyguts_t *)fake_scanner)->yy_hold_char = 0; + yy_accept[1] = i; + yylex( NULL, fake_scanner ); + + } + + fprintf( tables_out, action_desc.act_suffix, i ); + + } + + fprintf( tables_out, "%s", action_desc.postamble ); + + if ( action_desc.cleanup ) { + + action_desc.cleanup( &action_desc ); + + } + + } + + @<Compute magic constants@>@; + @<Output states@>; + fprintf( tables_out, "\\constset{YYECMAGIC}{%d}%%\n", yy_ec_magic ); + fprintf( tables_out, "\\constset{YYMAXEOFSTATE}{%d}%%\n", max_eof_state ); + +@ @<Error codes@>= + BAD_SCANNER,@[@] + +@ @<Variables and types local to the scanner driver@>= + int yy_ec_magic; + +@ The `magic' constants are similar to the `exotic' ones mentioned +above except the methods used to compute them rely on reverse +engineering the scanner function. Since this changes the scanner +tables it has to be done after the `driver' has finished going through all +the actions. +@<Compute magic constants@>= + { + int i, j; + char fake_yytext[ YY_MORE_ADJ + 1 ]; + + yyscan_t yyscanner; + struct yyguts_t *yyg; + + if ( yylex_init( &yyscanner ) ) { + + printf( "Cannot initialize the scanner\n" ); + exit( BAD_SCANNER ); + + } + + yyg = (struct yyguts_t *)yyscanner; + yyg->yy_start = 0; + yy_set_bol(0); + yyg->yytext_ptr = fake_yytext; + yyg->yy_c_buf_p = yyg->yytext_ptr + 1 + YY_MORE_ADJ; + + fake_yytext[YY_MORE_ADJ] = 0; /* |*yy_cp = 0;| */ + + yy_accept[0] = 0; + yy_base[0] = 0; + + for ( i = 0; i < sizeof( yy_chk )/sizeof( yy_chk[0] ); i++ ) { + + yy_chk[i] = 0; + + } + + for ( i = 0; i < sizeof( yy_nxt )/sizeof( yy_nxt[0] ); i++ ) { + + yy_nxt[i] = i; + + } + + yy_ec_magic = yy_get_previous_state( yyscanner ); + + } + +@*2State names. There is no easy way to output the symbolic names for +states, so this has to be done by hand while the actions are output. The +state names are accumulated in a list structure and are printed out +after the action output is complete. + +Note that parsing the scanner file is only partially helpful (even though the +extended parser and scanner can recognize the \.{\%x} option). All that can +be done is output the state {\it names\/} but not their numerical +values, since all such names are macros whose values are only +known to the \flex\ generated scanner. + +@d Define_State( st_name, st_num ) do { + + struct lexer_state_d *this_state; + + this_state = malloc( sizeof(struct lexer_state_d) ); + this_state->name = st_name; + this_state->value = st_num; + this_state->next = NULL; + + if ( last_state ) { + + last_state->next = this_state; + last_state = this_state; + + } else { + + last_state = state_list = this_state; + + } + + if ( YY_STATE_EOF( st_num ) > max_eof_state ) { + + max_eof_state = YY_STATE_EOF( st_num ); + + } + +} while (0); + +@<Scanner variables and types for \Cee\ preamble@>= + int max_eof_state = 0; + + struct lexer_state_d { + + char *name; + int value; + struct lexer_state_d *next; + + }; + + struct lexer_state_d *state_list = NULL; + struct lexer_state_d *last_state = NULL; + +@ @<Output states@>= + { + + struct lexer_state_d *current_state; + struct lexer_state_d *next_state; + + current_state = next_state = state_list; + + if ( current_state ) { + + fprintf( tables_out, "\\def\\setflexstates{%%\n" + " \\stateset{INITIAL}{%d}%%\n", INITIAL ); + + while ( current_state ) { + + fprintf( tables_out, " \\stateset{%s}{%d}%%\n", + current_state->name, current_state->value); + + current_state = current_state->next; + + free( next_state ); + next_state = current_state; /* the |name| field is not + deallocated because it is not + allocated on the heap */ + + } + + fprintf( tables_out, "}%%\n%%\n" ); + + } + + } + +@*2Constants. +@<Scanner constants@>= + _register_const_d(YY_END_OF_BUFFER_CHAR)@; + _register_const_d(YY_NUM_RULES)@; + _register_const_d(YY_END_OF_BUFFER)@; + +@*2Output modes. +The output modes are the same as those in the parser driver with some minor +changes. + +@*3Generic output. Generic output is not programmed yet. +@<Scanner specific output modes@>= + GENERIC_OUT,@[@] + +@ @<Handle scanner output modes@>= + case GENERIC_OUT:@; + printf( "This mode is not supported yet\n" ); + exit(0); + break; + +@*3\TeX~mode. The \TeX\ mode is the main focus of this software. +@<Scanner specific output modes@>= + TEX_OUT,@[@] + +@ @<Handle scanner output modes@>= + case TEX_OUT:@; + @<Set up \TeX\ format for scanner tables@>@; + @<Set up \TeX\ format for scanner actions@>@; + @<Prepare \TeX\ format for scanner constants@>@; + break; + +@ @<Set up \TeX\ format for scanner tables@>= + tex_table_generic(yy_accept); + yy_accept_desc.name = "yyaccept"; + tex_table_generic(yy_ec); + yy_ec_desc.name = "yyec"; + tex_table_generic(yy_meta); + yy_meta_desc.name = "yymeta"; + tex_table_generic(yy_base); + yy_base_desc.name = "yybase"; + tex_table_generic(yy_def); + yy_def_desc.name = "yydef"; + tex_table_generic(yy_nxt); + yy_nxt_desc.name = "yynxt"; + tex_table_generic(yy_chk); + yy_chk_desc.name = "yychk"; + +@ @<Set up \TeX\ format for scanner actions@>= + + if ( optimize_actions ) { + + action_desc.preamble = "%\n% the big switch\n%\n"@/ + "\\catcode`\\/=0\\relax\n%\n"@/ + "\\def\\yydoactionswitch#1{%%\n"@/ + " \\let\\yylextail\\yylexcontinue\n"@/ + " \\csname doflexaction\\number #1\\parsernamespace\\endcsname\n"@/ + " \\yylextail\n"@; + "}\\stashswitch{yydoactionswitch}%\n"; + action_desc.act_setup = "\n\\expandafter\\def\\csname doflexaction%d\\parsernamespace\\endcsname{%%\n" + " \\YYRULESETUP"; + action_desc.act_suffix = "}%% end of rule %d\n"; + action_desc.action1 = NULL; + action_desc.actionn = NULL; + action_desc.postamble = "\\catcode`\\/=12\\relax\n%\n"; + action_desc.print_rule = NULL; + action_desc.cleanup = NULL; + output_desc.output_actions = 1; + + } else { + + action_desc.preamble = "%\n% the big switch\n%\n"@/ + "\\catcode`\\/=0\\relax\n%\n"@/ + "\\def\\yydoactionswitch#1{%%\n \\let\\yylextail\\yylexcontinue\n"@; + " \\ifcase#1\\relax\n"; + action_desc.act_setup = " \\or\n" + " \\YYRULESETUP %% (rule %d) "; + action_desc.act_suffix = " %% end of rule %d\n"; + action_desc.action1 = NULL; + action_desc.actionn = NULL; + action_desc.postamble = " \\else\n \\fi\n \\yylextail\n}\\stashswitch{yydoactionswitch}%\n\\catcode`\\/=12\\relax\n%\n"; + action_desc.print_rule = NULL; + action_desc.cleanup = NULL; + output_desc.output_actions = 1; + + } + +@ \TeX\ constant output is another place where the techniques described above are applied. A few names +are handled separately, because they contain underscores. +\def\YYxENDxOFxBUFFERxCHARxdesc{\.{YY\_END\_OF\_BUFFER\_CHAR\_}\\{desc}} +\def\YYxNUMxRULESxdesc{\.{YY\_NUM\_RULES\_}\\{desc}} +\def\YYxENDxOFxBUFFERxdesc{\.{YY\_END\_OF\_BUFFER\_}\\{desc}} + +@s YY_END_OF_BUFFER_CHAR_desc TeX +@s YY_NUM_RULES_desc TeX +@s YY_END_OF_BUFFER_desc TeX + +@<Prepare \TeX\ format for scanner constants@>= +#define _register_const_d(c_name) @[c_name##_desc.format = "\\constset{%s}{%d}%%\n"; \ + c_name##_desc.name = #c_name; \ + output_desc.output_##c_name = 1;@] + @<Scanner constants@>@; +#undef _register_const_d + + YY_END_OF_BUFFER_CHAR_desc.name = "YYENDOFBUFFERCHAR"; + YY_NUM_RULES_desc.name = "YYNUMRULES"; + YY_END_OF_BUFFER_desc.name = "YYENDOFBUFFER"; + +@ @<Output exotic scanner constants@>= + fprintf( tables_out, "\\constset{YYMAXREALCHAR}{%ld}%%\n", sizeof( yy_accept )/(sizeof( yy_accept[0] )) - 1 ); + fprintf( tables_out, "\\constset{YYBASEMAXENTRY}{%d}%%\n", max_yybase_entry ); + fprintf( tables_out, "\\constset{YYNXTMAXENTRY}{%d}%%\n", max_yynxt_entry ); + fprintf( tables_out, "\\constset{YYMAXRULENO}{%d}%%\n", max_yyaccept_entry ); + fprintf( tables_out, "\\constset{YYECMAXENTRY}{%d}%%\n", max_yy_ec_entry ); + +@*2 Command line options. +We start with the most obvious option, the one begging for help. + +@<Higher index scanner specific options@>= + LONG_HELP,@[@] + +@ @<Scanner specific option list@>= + _register_option("help", no_argument, 0, LONG_HELP, "")@; + +@ @<Shortcuts for command line options affecting scanner output@>= + "h" + +@ @<Handle scanner output options@>= + case 'h': /* short help */@; + fprintf(stderr, "Usage: %s [options] output_file\n", argv[0]); + exit(0); + break; /* should not be needed */ + + case LONG_HELP:@; + fprintf(stderr, "%s [--mode=TeX:options] output_file outputs tables\n" + " and constants for a TeX scanner\n", argv[0]); + exit(0); + break; /* should not be needed */ + +@ @<Scanner specific option list@>= + _register_option("debug", optional_argument, 0, 'b', "")@; + _register_option("mode", required_argument, 0, 'm', "")@; + _register_option("table-separator", required_argument, 0, 'z', "")@; + + _register_option("format", required_argument, 0, 'f', "")@; /* name? */ + _register_option("table", required_argument, 0, 't', "")@; /* specific table */ + _register_option("constant", required_argument, 0, 'c', "")@; /* specific constant */ + _register_option("name-length", required_argument, 0, 'l', "")@; /* change |MAX_NAME_LENGTH| */ + _register_option("token", required_argument, 0, 'n', "")@; /* specific token */ + _register_option("run-scan", required_argument, 0, 'p', "")@; /* run the scanner */ + _register_option("scan-file", required_argument, 0, 'i', "")@; /* input for the scanner */ + +@ The string below is a list of short options. +@<Shortcuts for command line options affecting scanner output@>= + "b::z:m:f:t:" + +@ A few options can be immediately discussed. +@<Variables and types local to the scanner driver@>= + int debug_level = 0; + char *table_separator = "%s "; + +@ @<Handle scanner output options@>= + case 'b': /* debug (level) */@; + debug_level = optarg ? atoi(optarg) : 1; + break; + + case 'm': /* output mode */@; + switch( optarg[0] ) { + + case 'T': + case 't':@; + mode = TEX_OUT; + break; + + case 'b': + case 'B': + case 'g': + case 'G':@; + mode = GENERIC_OUT; + break; + + default:@; + break; + + } + break; + + case 'z': + table_separator = (char *)malloc( (strlen(optarg) + 1)*sizeof(char) ); + strcpy(table_separator, optarg); + break; diff --git a/support/splint/cweb/lo.w b/support/splint/cweb/lo.w new file mode 100644 index 0000000000..b28711423c --- /dev/null +++ b/support/splint/cweb/lo.w @@ -0,0 +1,797 @@ +@q Copyright 2012-2014 Alexander Shibakov@> +@q Copyright 2002-2014 Free Software Foundation, Inc.@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1 The scanner for grammar syntax. +\ifx\parsernamespace\UNDEFINED + \input limbo.sty + \input grabstates.sty + \immediate\openout\stlist=lo_states.h +\fi +The fact that \bison\ has a relatively straightforward grammar is +due to the sophistication of its scanner. The primary reason for this +increased complexity is \bison's awareness +of syntax variations in its input files. In addition to the grammar +syntax, the parser has to be able to deal with extended \Cee\ syntax +inside \bison's actions. + +Since the names of the scanner {\it states\/} reside in the common +namespace with other variables, in order to make the \TeX\ version of +the scanner aware of the numerical values of the states, a special +procedure is required. It is executed as part of \flex's user +initialization code but the data for it has to be collected +separately. The procedure is declared in the preamble section of the scanner. + +Below, we follow the same convention (of italicizing the original +comments) as in the code for the parser. +@(lo.ll@>= +@G + @> @<Grammar lexer definitions@> @= +%{@> @<Grammar lexer \Cee\ preamble@> @=%} + @> @<Grammar lexer options@> @= +%% + @> @<Grammar token regular expressions@> @= +%% +@O +void define_all_states( void ) { + @<Collect state definitions for the grammar lexer@>@; +} +@o +@g + +@ It is convenient to abbreviate some commonly used subexpressions. +@<Grammar lexer definitions@>= + @<Grammar lexer states@>@; +@G +letter [.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_] +notletter [^.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_]{-}[%\{] +id {letter}({letter}|[-0-9])* +int [0-9]+ +@g + +@ {\it Zero or more instances of backslash-newline. Following \gcc, allow +white space between the backslash and the newline}. +@<Grammar lexer definitions@>= +@G +splice (\\[ \f\t\v]*\n)* +@g + +@ {\it An equal sign, with optional leading whitespaces. This is used in some +deprecated constructs}. +@<Grammar lexer definitions@>= +@G +eqopt ([[:space:]]*=)? +@g + +@ This is how the code for state value output is put inside the +routine mentioned above. The state information is collected by a +special small scanner that is coupled with the bootstrap parser. This +way, all the necessary token information comes `hardwired' in the +bootstrap parser, and the small scanner itself does not use any state +manipulation and thus can get away without any state setup. It can, +however, scan just enough of the \flex\ syntax to extract the state +information from it (only the state {\it names\/} are needed) and +output it in the form of a header file for the `real' lexer output +`driver' to use. +@<Collect state definitions for the grammar lexer@>= +#define _register_name( name ) @[Define_State( #name, name )@] +#include "lo_states.h" +#undef _register_name + +@ {\it A \Cee-like comment in directives/rules}. +@<Grammar lexer states@>= +@G +%x SC_YACC_COMMENT +@g + +@ {\it Strings and characters in directives/rules}. +@<Grammar lexer states@>= +@G +%x SC_ESCAPED_STRING SC_ESCAPED_CHARACTER +@g + +@ {\it A identifier was just read in directives/rules. Special state +to capture the sequence `\.{identifier:}'}. +@<Grammar lexer states@>= +@G +%x SC_AFTER_IDENTIFIER +@g + +@ {\it \POSIX\ says that a tag must be both an id and a \Cee\ union member, but +historically almost any character is allowed in a tag. We +disallow \prodstyle{NUL}, as this simplifies our implementation. We match +angle brackets in nested pairs: several languages use them for +generics/template types}. +@<Grammar lexer states@>= +@G +%x SC_TAG +@g + +@ {\it +\def\aterm{\item{\sqbullet}\ignorespaces}% +\setbox0=\hbox{\sqbullet\enspace}% +\parindent=0pt +\advance\parindent by \wd0 +Four types of user code: +\aterm prologue (code between \.{\%\{} \.{\%\}} in the first section, before \prodstyle{\%\%}); + +\aterm actions, printers, union, etc, (between braced in the middle section); + +\aterm epilogue (everything after the second \prodstyle{\%\%}); + +\aterm predicate (code between \.{\%?\{} and \.{\}} in middle section); +}% +@<Grammar lexer states@>= +@G +%x SC_PROLOGUE SC_BRACED_CODE SC_EPILOGUE SC_PREDICATE +@g + +@ {\it \Cee\ and \Cee++ comments in code}. +@<Grammar lexer states@>= +@G +%x SC_COMMENT SC_LINE_COMMENT +@g + +@ {\it Strings and characters in code}. +@<Grammar lexer states@>= +@G +%x SC_STRING SC_CHARACTER +@g + +@ Bracketed identifiers support. +@<Grammar lexer states@>= +@G +%x SC_BRACKETED_ID SC_RETURN_BRACKETED_ID +@g + +@ @<Grammar lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + +@ The code for the generated scanner is highly dependent on the options +supplied. Most of the options below are essential for the scheme +adopted in this package to work. +@<Grammar lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="lo.c" +@g + +@*2 Tokenizing with regular expressions. +Here is a full collection of regular expressions employed by the scanner. +@<Grammar token regular expressions@>= + @<Scan grammar white space@>@; + @<Scan \flex\ directives and options@>@; + @<Scan \bison\ directives@>@; + @<Do not support zero characters@>@; + @<Scan after an identifier, check whether a colon is next@>@; + @<Scan bracketed identifiers@>@; + @<Scan a Yacc comment@>@; + @<Scan a \Cee\ comment@>@; + @<Scan a line comment@>@; + @<Scan a \bison\ string@>@; + @<Scan a character literal@>@; + @<Scan a tag@>@; + @<Decode escaped characters@>@; + @<Scan user-code characters and strings@>@; + @<Strings, comments etc.\ found in user code@>@; + @<Scan code in braces@>@; + @<Scan prologue@>@; + @<Scan the epilogue@>@; + @<Add the scanned symbol to the current string@>@; + +@ @<Scan grammar white space@>= +@G +<INITIAL,SC_AFTER_IDENTIFIER,SC_BRACKETED_ID,SC_RETURN_BRACKETED_ID> +{ + /* {\it Comments and white space.} */ + "," {@> @[TeX_( "/yycomplain{stray `,' treated as white space}/yylexnext" );@]@=} + [ \f\n\t\v] | + "//".* {@> @[TeX_( "/yylexnext" );@]@=} +@= "/*" {@> @[TeX_( "/contextstate/YYSTART /yyBEGIN{SC_YACC_COMMENT}/yylexnext" );@]@=}@>@/ + /* {\it |@[#line@]| directives are not documented, and may be withdrawn or modified in future versions of \bison.} */ + ^"#line "{int}(" \"".*"\"")?"\n" {@> @[TeX_( "/yylexnext" );@]@=} +} +@g + +@ {\it For directives that are also command line options, the regex must be +\.{"\%..."} after \.{"[-\_]"}'s are removed, and the directive must match the \.{--long} +option name, with a single string argument. Otherwise, add exceptions +to \.{../build-aux/cross-options.pl}}. For most options the scanner +returns a pair of pointers as the value. + +@<Scan \bison\ directives@>= +@G +<INITIAL> +{ + "%binary" {@> @[TeX_( "/yylexreturnptr{PERCENT_NONASSOC}" );@]@=} + "%code" {@> @[TeX_( "/yylexreturnptr{PERCENT_CODE}" );@]@=} + "%debug" {@> @[@<Set \prodstyle{\%debug} flag@>@]@=} + "%default-prec" {@> @[TeX_( "/yylexreturnptr{PERCENT_DEFAULT_PREC}" );@]@=} + "%define" {@> @[TeX_( "/yylexreturnptr{PERCENT_DEFINE}" );@]@=} + "%defines" {@> @[TeX_( "/yylexreturnptr{PERCENT_DEFINES}" );@]@=} + "%destructor" {@> @[TeX_( "/yylexreturnptr{PERCENT_DESTRUCTOR}" );@]@=} + "%dprec" {@> @[TeX_( "/yylexreturnptr{PERCENT_DPREC}" );@]@=} + "%empty" {@> @[TeX_( "/yylexreturnptr{PERCENT_EMPTY}" );@]@=} + "%error-verbose" {@> @[TeX_( "/yylexreturnptr{PERCENT_ERROR_VERBOSE}" );@]@=} + "%expect" {@> @[TeX_( "/yylexreturnptr{PERCENT_EXPECT}" );@]@=} + "%expect-rr" {@> @[TeX_( "/yylexreturnptr{PERCENT_EXPECT_RR}" );@]@=} + "%file-prefix" {@> @[TeX_( "/yylexreturnptr{PERCENT_FILE_PREFIX}" );@]@=} + "%fixed-output-files" {@> @[TeX_( "/yylexreturnptr{PERCENT_YACC}" );@]@=} + "%initial-action" {@> @[TeX_( "/yylexreturnptr{PERCENT_INITIAL_ACTION}" );@]@=} + "%glr-parser" {@> @[TeX_( "/yylexreturnptr{PERCENT_GLR_PARSER}" );@]@=} + "%language" {@> @[TeX_( "/yylexreturnptr{PERCENT_LANGUAGE}" );@]@=} + "%left" {@> @[TeX_( "/yylexreturnptr{PERCENT_LEFT}" );@]@=} + "%lex-param" {@> @[@<Return lexer parameters@>@]@=} + "%locations" {@> @[@<Set \prodstyle{\%locations} flag@>@]@=} + "%merge" {@> @[TeX_( "/yylexreturnptr{PERCENT_MERGE}" );@]@=} + "%name-prefix" {@> @[TeX_( "/yylexreturnptr{PERCENT_NAME_PREFIX}" );@]@=} + "%no-default-prec" {@> @[TeX_( "/yylexreturnptr{PERCENT_NO_DEFAULT_PREC}" );@]@=} + "%no-lines" {@> @[TeX_( "/yylexreturnptr{PERCENT_NO_LINES}" );@]@=} + "%nonassoc" {@> @[TeX_( "/yylexreturnptr{PERCENT_NONASSOC}" );@]@=} + "%nondeterministic-parser" {@> @[TeX_( "/yylexreturnptr{PERCENT_NONDETERMINISTIC_PARSER}" );@]@=} + "%nterm" {@> @[TeX_( "/yylexreturnptr{PERCENT_NTERM}" );@]@=} + "%output" {@> @[TeX_( "/yylexreturnptr{PERCENT_OUTPUT}" );@]@=} + "%param" {@> @[@<Return lexer and parser parameters@>@]@=} + "%parse-param" {@> @[@<Return parser parameters@>@]@=} + "%prec" {@> @[TeX_( "/yylexreturnptr{PERCENT_PREC}" );@]@=} + "%precedence" {@> @[TeX_( "/yylexreturnptr{PERCENT_PRECEDENCE}" );@]@=} + "%printer" {@> @[TeX_( "/yylexreturnptr{PERCENT_PRINTER}" );@]@=} + "%pure-parser" {@> @[@<Set \prodstyle{\%pure-parser} flag@>@]@=} + "%require" {@> @[TeX_( "/yylexreturnptr{PERCENT_REQUIRE}" );@]@=} + "%right" {@> @[TeX_( "/yylexreturnptr{PERCENT_RIGHT}" );@]@=} + "%skeleton" {@> @[TeX_( "/yylexreturnptr{PERCENT_SKELETON}" );@]@=} + "%start" {@> @[TeX_( "/yylexreturnptr{PERCENT_START}" );@]@=} + "%term" {@> @[TeX_( "/yylexreturnptr{PERCENT_TOKEN}" );@]@=} + "%token" {@> @[TeX_( "/yylexreturnptr{PERCENT_TOKEN}" );@]@=} + "%token-table" {@> @[TeX_( "/yylexreturnptr{PERCENT_TOKEN_TABLE}" );@]@=} + "%type" {@> @[TeX_( "/yylexreturnptr{PERCENT_TYPE}" );@]@=} + "%union" {@> @[TeX_( "/yylexreturnptr{PERCENT_UNION}" );@]@=} + "%verbose" {@> @[TeX_( "/yylexreturnptr{PERCENT_VERBOSE}" );@]@=} + "%yacc" {@> @[TeX_( "/yylexreturnptr{PERCENT_YACC}" );@]@=} + + /* {\it deprecated} */ + "%default"[-_]"prec" {@> @[TeX_( "/yypdeprecated{\\%default-prec}" );@]@=} + "%error"[-_]"verbose" {@> @[TeX_( "/yypdeprecated{\\%define parse.error verbose}" );@]@=} + "%expect"[-_]"rr" {@> @[TeX_( "/yypdeprecated{\\%expect-rr}" );@]@=} + "%file-prefix"{eqopt} {@> @[TeX_( "/yypdeprecated{\\%file-prefix}" );@]@=} + "%fixed"[-_]"output"[-_]"files" {@> @[TeX_( "/yypdeprecated{\\%fixed-output-files}" );@]@=} + "%name"[-_]"prefix"{eqopt} {@> @[TeX_( "/yypdeprecated{\\%name-prefix}" );@]@=} + "%no"[-_]"default"[-_]"prec" {@> @[TeX_( "/yypdeprecated{\\%no-default-prec}" );@]@=} + "%no"[-_]"lines" {@> @[TeX_( "/yypdeprecated{\\%no-lines}" );@]@=} + "%output"{eqopt} {@> @[TeX_( "/yypdeprecated{\\%output}" );@]@=} + "%pure"[-_]"parser" {@> @[TeX_( "/yypdeprecated{\\%pure-parser}" );@]@=} + "%token"[-_]"table" {@> @[TeX_( "/yypdeprecated{\\%token-table}" );@]@=} + + /* {\it Semantic predicate.} */ + "%?"[ \f\n\t\v]*"{" {@> @[TeX_( "/yyBEGIN{SC_PREDICATE}/yylexnext" );@]@=} + + "%"{id}|"%"{notletter}([[:graph:]])+ {@> @[@<Possbly complain about a bad directive@>@]@=} + + "=" {@> @[TeX_( "/yylexreturnptr{EQUAL}" );@]@=} + "|" {@> @[TeX_( "/yylexreturnptr{PIPE}" );@]@=} + ";" {@> @[TeX_( "/yylexreturnptr{SEMICOLON}" );@]@=} + + {id} {@> @[@<Prepare an identifier@>@]@=} + {int} {@> @[TeX_( "/edef/next{/yylval{/nx/anint{/the/yytext}" );@]@; + @> @[TeX_( "{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @> @[TeX_( "/yylexreturn{INT}" );@]@=} + 0[xX][0-9abcdefABCDEF]+ {@> @[TeX_( "/edef/next{/yylval{/nx/hexint{/the/yytext}" );@]@; + @> @[TeX_( "{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @> @[TeX_( "/yylexreturn{INT}" );@]@=} + + /* {\it Identifiers may not start with a digit. Yet, don't silently accept \.{1FOO} as \.{1 FOO}.} */ + {int}{id} {@> @[TeX_( "/yycomplain{invalid identifier: /the/yytext}" );@] + @> @[TeX_( "/yyerrterminate" );@]@=} + + /* {\it Characters.} */ + "'" {@> @[TeX_( "/yyBEGIN{SC_ESCAPED_CHARACTER}/yylexnext" );@]@=} + + /* {\it Strings.} */ + "\"" {@> @[TeX_( "/yyBEGIN{SC_ESCAPED_STRING}/yylexnext" );@]@=} + + /* {\it Prologue.} */ + "%{" {@> @[@<Start assembling prologue code@>@]@=} + + /* {\it Code in between braces.} Originally preceded by \.{\\STRINGGROW} but it is omitted here. */ + "{" {@> @[TeX_( "/lonesting/z@@/yyBEGIN{SC_BRACED_CODE}/yylexnext" );@]@=} + + /* {\it A type.} */ + "<*>" {@> @[TeX_( "/yylexreturnptr{TAG_ANY}" );@]@=} + "<>" {@> @[TeX_( "/yylexreturnptr{TAG_NONE}" );@]@=} + "<" {@> @[TeX_( "/lonesting=/z@@/yyBEGIN{SC_TAG}/yylexnext" );@]@=} + + "%%" {@> @[@<Switch sections@>@]@=} + "[" {@> @[TeX_( "/let/bracketedidstr=/empty" );@]@; + @> @[TeX_( "/bracketedidcontextstate/YYSTART" );@] + @> @[TeX_( "/yyBEGIN{SC_BRACKETED_ID}/yylexnext" );@]@=} + + <<EOF>> {@> @[TeX_( "/yyterminate% EOF in INITIAL" );@]@=} + + [^\[%A-Za-z0-9_<>{}\"\'*;|=/, \f\n\t\v]+|. {@> @[@<Process a bad character@>@]@=} +} +@g + +@ Some additional constructs needed to typeset simple \flex\ +declarations. This is not part of the original \bison\ scanner. +@<Scan \flex\ directives and options@>= +@G +<INITIAL> +{ + "%option" {@> @[TeX_( "/yylexreturnptr{FLEX_OPTION}" );@]@=} + "%x" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_X}" );@]@=} + "%s" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_S}" );@]@=} +} +@g + +@ We present the `bad character' code first, before going into the details +of the character matching by the rest of the lexer. +@<Process a bad character@>= + @[TeX_( "/edef/next{/nx/csname lexspecial[/the/yytextpure]/nx/endcsname}" );@]@; + @[TeX_( "/expandafter/expandafter/expandafter/ifx/next/relax" );@]@; + @[TeX_( " /iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( " /fi" );@]@; + @[TeX_( " /yylexreturn{$undefined}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /expandafter/lexspecialchar/expandafter{/next}{/the/yyfmark}{/the/yysmark}/yylexnext" );@]@; + @[TeX_( "/fi" );@]@; + +@ @<Set \prodstyle{\%debug} flag@>= + @[TeX_( "/edef/next{/yylval{{parse.trace}{debug}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_FLAG}" );@]@; + +@ @<Return lexer parameters@>= + @[TeX_( "/edef/next{/yylval{{lex-param}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_PARAM}" );@]@; + +@ @<Set \prodstyle{\%locations} flag@>= + @[TeX_( "/edef/next{/yylval{{locations}{}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_FLAG}" );@]@; + +@ @<Return lexer and parser parameters@>= + @[TeX_( "/edef/next{/yylval{{both-param}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_PARAM}" );@]@; + +@ @<Return parser parameters@>= + @[TeX_( "/edef/next{/yylval{{parse-param}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_PARAM}" );@]@; + +@ @<Set \prodstyle{\%pure-parser} flag@>= + @[TeX_( "/edef/next{/yylval{{api.pure}{pure-parser}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{PERCENT_FLAG}" );@]@; + +@ @<Possbly complain about a bad directive@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid directive: /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexnext" );@]@; + +@ @<Prepare an identifier@>= + @[TeX_( "/edef/next{/yylval{/nx/idit{/the/yytextpure}{/the/yytext}" );@]@; + @[TeX_( " {/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/let/bracketedidstr=/empty" );@]@; + @[TeX_( "/yyBEGIN{SC_AFTER_IDENTIFIER}/yylexnext" );@]@; + +@ @<Switch sections@>= + @[TeX_( "/advance/percentpercentcount/@@ne" );@]@; + @[TeX_( "/ifnum/percentpercentcount=/tw@@" );@]@; + @[TeX_( " /yyBEGIN{SC_EPILOGUE}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturnptr{PERCENT_PERCENT}" );@]@; + +@ @<Start assembling prologue code@>= + @[TeX_( "/edef/next{/postoks{{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yyBEGIN{SC_PROLOGUE}/yylexnext" );@]@; + +@ {\it Supporting \.{\\0} complexifies our implementation for no expected added value}. + +@<Do not support zero characters@>= +@G +<SC_ESCAPED_CHARACTER,SC_ESCAPED_STRING,SC_TAG> +{ + \0 {@> @[TeX_( "/yycomplain{invalid null character}/yylexnext" );@]@=} +} +@g + +@ @<Scan after an identifier, check whether a colon is next@>= +@G +<SC_AFTER_IDENTIFIER> +{ + "[" {@> @[@<Process the bracketed part of an identifier@>@]@=} + ":" {@> @[@<Process a colon after an identifier@>@]@=} + <<EOF>> {@> @[@<End the scan with an identifier@>@]@=} + . {@> @[@<Process a character after an identifier@>@]@=} +} +@g + +@ @<Process the bracketed part of an identifier@>= + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /bracketedidcontextstate/YYSTART /yyBEGIN{SC_BRACKETED_ID}" );@]@; + @[TeX_( " /let/next=/yylexnext" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /ROLLBACKCURRENTTOKEN" );@]@; + @[TeX_( " /yyBEGIN{SC_RETURN_BRACKETED_ID}" );@]@; + @[TeX_( " /def/next{/yylexreturn{ID}}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ @<Process a colon after an identifier@>= + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /yyBEGIN{SC_RETURN_BRACKETED_ID}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{ID_COLON}" );@]@; + +@ @<Process a character after an identifier@>= + @[TeX_( "/ROLLBACKCURRENTTOKEN" );@]@; + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /yyBEGIN{SC_RETURN_BRACKETED_ID}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{ID}" );@]@; + +@ @<End the scan with an identifier@>= + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /yyBEGIN{SC_RETURN_BRACKETED_ID}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/ROLLBACKCURRENTTOKEN" );@]@; + @[TeX_( "/yylexreturn{ID}" );@]@; + +@ @<Scan bracketed identifiers@>= +@G +<SC_BRACKETED_ID> +{ + <<EOF>> {@> @[@<Complain about unexpected end of file inside brackets@>@]@=} + {id} {@> @[@<Process bracketed identifier@>@]@=} + "]" {@> @[@<Finish processing bracketed identifier@>@]@=} + [^\].A-Za-z0-9_/ \f\n\t\v]+|. {@> @[@<Complain about improper identifier characters@>@]@=} +} +@g + +@ @<Process bracketed identifier@>= + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /edef/bracketedidstr{/nx/idit{/the/yytextpure}" );@]@; + @[TeX_( " {/the/yytext}{/the/yyfmark}{/the/yysmark}}" );@]@; + @[TeX_( " /let/next=/yylexnext" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /def/next{/yycomplain{unexpected " );@]@; + @[TeX_( " identifier in bracketed name: /the/yytext}/yylexnext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ @<Finish processing bracketed identifier@>= + @[TeX_( "/yyBEGINr/bracketedidcontextstate" );@]@; + @[TeX_( "/ifx/bracketedidstr/empty" );@]@; + @[TeX_( " /def/next{/yycomplain{an identifier expected}/yylexnext}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /ifnum/bracketedidcontextstate=/yylexstate{INITIAL}/relax" );@]@; + @[TeX_( " /expandafter/yylval/expandafter{/bracketedidstr}" );@]@; + @[TeX_( " /let/bracketedidstr=/empty" );@]@; + @[TeX_( " /def/next{/yylexreturn{BRACKETED_ID}}" );@]@; + @[TeX_( " /else" );@]@; + @[TeX_( " /let/next=/yylexnext" );@]@; + @[TeX_( " /fi" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ @<Complain about improper identifier characters@>= + @[TeX_( "/yycomplain{invalid character(s) in bracketed name: /the/yytext}/yyerrterminate" );@]@; + +@ @<Complain about unexpected end of file inside brackets@>= + @[TeX_( "/yyBEGINr/bracketedidcontextstate" );@]@; + @[TeX_( "/yycomplain{unexpected end of file inside brackets}/yyerrterminate" );@]@; + +@ @<Scan bracketed identifiers@>= +@G +<SC_RETURN_BRACKETED_ID> +{ + . {@> @[@<Return a bracketed identifier@>@]@=} +} +@g + +@ @<Return a bracketed identifier@>= + @[TeX_( "/ROLLBACKCURRENTTOKEN" );@]@; + @[TeX_( "/expandafter/yylval/expandafter{/bracketedidstr}" );@]@; + @[TeX_( "/let/bracketedidstr=/empty" );@]@; + @[TeX_( "/yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/yylexreturn{BRACKETED_ID}" );@]@; + +@ {\it Scanning a Yacc comment. The initial \.{/*} is already eaten}. +@<Scan a Yacc comment@>= +@G +<SC_YACC_COMMENT> +{ + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file in " );@] + @> @[TeX_( " a comment}/yyerrterminate" );@]@=} + "*/" {@> @[TeX_( "/yyBEGINr{/contextstate}/yylexnext" );@]@=} + .|\n {@> @[TeX_( "/yylexnext" );@]@=} +} +@g + +@ {\it Scanning a \Cee\ comment. The initial \.{/*} is already eaten}. +@<Scan a \Cee\ comment@>= +@G +<SC_COMMENT> +{ + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file in " );@] + @> @[TeX_( " a comment}/yyerrterminate" );@]@=} + "*"{splice}"/" {@> @[TeX_( "/STRINGGROW/yyBEGINr/contextstate/yylexnext" );@]@=} +} +@g + +@ {\it Scanning a line comment. The initial \.{//} is already eaten}. +@<Scan a line comment@>= +@G +<SC_LINE_COMMENT> +{ + <<EOF>> {@> @[TeX_( "/yyBEGINr/contextstate /ROLLBACKCURRENTTOKEN" );@] + @> @[TeX_( " /yylexnext" );@]@=} + "\n" {@> @[TeX_( "/STRINGGROW/yyBEGINr/contextstate /yylexnext" );@]@=} + {splice} {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} +} +@g + +@ {\it Scanning a \bison\ string, including its escapes. +The initial quote is already eaten}. +@<Scan a \bison\ string@>= +@G +<SC_ESCAPED_STRING> +{ + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file in " );@] + @> @[TeX_( " a string}/yyerrterminate" );@]@=} + "\"" {@> @[@<Finish a \bison\ string@>@]@=} + "\n" {@> @[TeX_( "/yycomplain{unexpected end of line in " );@] + @> @[TeX_( " a string}/yyerrterminate" );@]@=} +} +@g + +@ @<Finish a \bison\ string@>= + @[TeX_( "/STRINGFINISH" );@]@; + @[TeX_( "/edef/next{/yylval{/nx/stringify{/the/laststring}" );@]@; + @[TeX_( "{/the/laststringraw}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/yylexreturn{STRING}" );@]@; + +@ {\it Scanning a \bison\ character literal, decoding its escapes. +The initial quote is already eaten}. +@<Scan a character literal@>= +@G +<SC_ESCAPED_CHARACTER> +{ + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file in " );@] + @> @[TeX_( " a literal}/yyerrterminate" );@]@=} + "'" {@> @[@<Return an escaped character@>@]@=} + "\n" {@> @[TeX_( "/yycomplain{unexpected end of line in " );@] + @> @[TeX_( " a literal}/yyerrterminate" );@]@=} +} +@g + +@ @<Return an escaped character@>= + @[TeX_( "/STRINGFINISH" );@]@; + @[TeX_( "/edef/next{/yylval{/nx/charit{/the/laststring}{/the/laststringraw}" );@]@; + @[TeX_( " {/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/STRINGFREE" );@]@; + @[TeX_( "/yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/yylexreturn{CHAR}" );@]@; + +@ {\it Scanning a tag. The initial angle bracket is already eaten}. +@<Scan a tag@>= +@G +<SC_TAG> +{ + ">" {@> @[@<Finish a tag@>@]@=} + ([^<>]|->)+ {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + "<" {@> @[@<Raise nesting level@>@]@=} + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file in " );@] + @> @[TeX_( " a literal}/yyerrterminate" );@]@=} +} +@g + +@ @<Finish a tag@>= + @[TeX_( "/advance/lonesting/m@@ne" );@]@; + @[TeX_( "/ifnum/lonesting</z@@" );@]@; + @[TeX_( " /STRINGFINISH" );@]@; + @[TeX_( " /edef/next{/yylval{/nx/tagit{/the/laststring}{/the/laststringraw}" );@]@; + @[TeX_( " {/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( " /STRINGFREE" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( " /def/next{/yylexreturn{TAG}}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /STRINGGROW/let/next=/yylexnext" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ This is a slightly different rule from the original scanner. We do not perform |yyleng| computations, +so it makes sense to raise the nesting level one by one. +@<Raise nesting level@>= + @[TeX_( "/STRINGGROW" );@]@; + @[TeX_( "/advance/lonesting/@@ne" );@]@; + @[TeX_( "/yylexnext" );@]@; + +@ @<Decode escaped characters@>= +@G +<SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER> +{ + \\[0-7]{1,3} {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\x[0-9abcdefABCDEF]+ {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\a {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\b {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\f {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\n {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\r {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\t {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\v {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + + /* {\it \.{\\\\[\\"\\'?\\\\]} would be shorter, but it confuses |xgettext|.} */ + \\("\""|"'"|"?"|"\\") {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + + \\(u|U[0-9abcdefABCDEF]{4})[0-9abcdefABCDEF]{4} {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} + \\(.|\n) {@> @[TeX_( "/yycomplain{invalid character after " );@] + @> @[TeX_( " /\\-escape: /the/yytext}/yylexnext" );@]@=} +} +@g + +@ @<Scan user-code characters and strings@>= +@G +<SC_CHARACTER,SC_STRING> +{ + {splice}|\\{splice}[^\n\[\]] {@> @[TeX_( "/STRINGGROW/yylexnext" );@]@=} +} + +<SC_CHARACTER> +{ + "'" {@> @[TeX_( "/STRINGGROW /yyBEGINr{/contextstate}/yylexnext" );@]@=} + \n {@> @[TeX_( "/yycomplain{unexpected end of line instead of " );@] + @> @[TeX_( " a character}/yyerrterminate" );@]@=} + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file instead of " );@] + @> @[TeX_( " a character}/yyerrterminate" );@]@=} +} + +<SC_STRING> +{ + "\"" {@> @[TeX_( "/STRINGGROW /yyBEGINr{/contextstate}/yylexnext" );@]@=} + \n {@> @[TeX_( "/yycomplain{unexpected end of line instead of " );@] + @> @[TeX_( " a character}/yyerrterminate" );@]@=} + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file instead of " );@] + @> @[TeX_( " a character}/yyerrterminate" );@]@=} +} +@g + +@ @<Strings, comments etc.\ found in user code@>= +@G +<SC_BRACED_CODE,SC_PROLOGUE,SC_EPILOGUE,SC_PREDICATE> +{ + "'" {@> @[TeX_( "/STRINGGROW /contextstate/YYSTART" );@] + @> @[TeX_( " /yyBEGIN{SC_CHARACTER}/yylexnext" );@]@=} + "\"" {@> @[TeX_( "/STRINGGROW /contextstate/YYSTART" );@] + @> @[TeX_( " /yyBEGIN{SC_STRING}/yylexnext" );@]@=} + "/"{splice}"*" {@> @[TeX_( "/STRINGGROW /contextstate/YYSTART" );@] + @> @[TeX_( " /yyBEGIN{SC_COMMENT}/yylexnext" );@]@=} + "/"{splice}"/" {@> @[TeX_( "/STRINGGROW /contextstate/YYSTART" );@] + @> @[TeX_( " /yyBEGIN{SC_LINE_COMMENT}/yylexnext" );@]@=} +} +@g + +@ {\it Scanning some code in braces (actions, predicates). The +initial \.{\{} is already eaten}. +@<Scan code in braces@>= +@G +<SC_BRACED_CODE,SC_PREDICATE> +{ + "{"|"<"{splice}"%" {@> @[TeX_( "/STRINGGROW /advance/lonesting/@@ne /yylexnext" );@]@=} + "%"{splice}">" {@> @[TeX_( "/STRINGGROW /advance/lonesting/m@@ne /yylexnext" );@]@=} + + /* {\it Tokenize \.{<<\%} correctly (as \.{<<} \.{\%}) rather than incorrectly (as \.{<} \.{<\%}).} */ + "<"{splice}"<" {@> @[TeX_( "/STRINGGROW /yylexnext" );@]@=} + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of line " );@] + @> @[TeX_( " inside braced code}/yyerrterminate" );@]@=} +} + +<SC_BRACED_CODE> +{ + "}" {@> @[@<Add closing brace to the braced code@>@]@=} +} + +<SC_PREDICATE> +{ + "}" {@> @[@<Add closing brace to a predicate@>@]@=} +} +@g + +@ Unlike the original lexer, we do not return the closing brace as part of the +braced code. + +@<Add closing brace to the braced code@>= + @[TeX_( "/advance/lonesting/m@@ne" );@]@; + @[TeX_( "/ifnum/lonesting</z@@" );@]@; + @[TeX_( " /STRINGFINISH" );@]@; + @[TeX_( " /edef/next{/yylval{{/the/laststring}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( " /def/next{/yylexreturn{BRACED_CODE}}" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /STRINGGROW" );@]@; + @[TeX_( " /let/next=/yylexnext" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ @<Add closing brace to a predicate@>= + @[TeX_( "/advance/lonesting/m@@ne" );@]@; + @[TeX_( "/ifnum/lonesting</z@@" );@]@; + @[TeX_( " /STRINGFINISH" );@]@; + @[TeX_( " /edef/next{/yylval{{/the/laststring}{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( " /yyBEGIN{INITIAL}" );@]@; + @[TeX_( " /def/next{/yylexreturn{BRACED_PREDICATE}}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /STRINGGROW" );@]@; + @[TeX_( " /let/next=/yylexnext" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/next" );@]@; + +@ {\it Scanning some prologue: from \.{\%\{} (already scanned) to \.{\%\}}}. +@<Scan prologue@>= +@G +<SC_PROLOGUE> +{ + "%}" {@> @[@<Finish braced code@>@]@=} + <<EOF>> {@> @[TeX_( "/yycomplain{unexpected end of file " );@] + @> @[TeX_( " inside prologue}/yyerrterminate" );@]@=} +} +@g + +@ @<Finish braced code@>= + @[TeX_( "/STRINGFINISH" );@]@; + @[TeX_( "/edef/next{/yylval{{/the/laststring}/the/postoks{/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/yylexreturn{PROLOGUE}" );@]@; + +@ {\it Scanning the epilogue (everything after the second \prodstyle{\%\%}, which +has already been eaten)}. +@<Scan the epilogue@>= +@G +<SC_EPILOGUE> +{ + <<EOF>> {@> @[@<Handle end of file in the epilogue@>@]@=} +} +@g + +@ @<Handle end of file in the epilogue@>= + @[TeX_( "/ROLLBACKCURRENTTOKEN" );@]@; + @[TeX_( "/STRINGFINISH" );@]@; + @[TeX_( "/yylval=/laststring" );@]@; + @[TeX_( "/yyBEGIN{INITIAL}" );@]@; + @[TeX_( "/yylexreturn{EPILOGUE}" );@]@; + +@ {\it By default, grow the string obstack with the input}. +\ifbootstrapmode % only if this file is used to extract state information + \immediate\closeout\stlist +\fi +@<Add the scanned symbol to the current string@>= +@G +<SC_COMMENT,SC_LINE_COMMENT,SC_BRACED_CODE,SC_PREDICATE,SC_PROLOGUE,SC_EPILOGUE, + SC_STRING,SC_CHARACTER,SC_ESCAPED_STRING,SC_ESCAPED_CHARACTER>. | + <SC_COMMENT,SC_LINE_COMMENT,SC_BRACED_CODE,SC_PREDICATE, + SC_PROLOGUE,SC_EPILOGUE>\n {@> @[TeX_( "/STRINGGROW /yylexnext" );@]@=} +@g diff --git a/support/splint/cweb/mkeparser.w b/support/splint/cweb/mkeparser.w new file mode 100644 index 0000000000..937a9e5498 --- /dev/null +++ b/support/splint/cweb/mkeparser.w @@ -0,0 +1,123 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\input limbo.sty +\input yy.sty + +\let\N\textN + +@**Parser tables. This is the main table output code. Its core comes from the Bison +Sourcer(er?) (\.{bs.w}) which will be included as soon as this paragraph is +over. Since the main function of this code is to {\it dump\/} tables +produced by \bison, and a {\it bison\/} is a large buffalo like +animal, {\it and\/} the only reason it has to be done like this is due +to the less than optimal choices of a few developers, feel free to +interpret the acronym as something easier to remember. + +@ @<Table names@>= + @<Parser table names@>@; + +@ @<Constant names@>= + @<Parser constants@>@; + +@ @<Local variable and type declarations@>= + @<Variables and types local to the parser@>@; + +@ @<Establish defaults@>= + @<Parser defaults@>@; + +@ @<Auxiliary function declarations@>= + @<Helper functions declarations for for parser output@>@; + +@ @<Auxiliary function definitions@>= + @<Helper functions for parser output@>@; + +@ @<Short option list@>= + @<Shortcuts for command line options affecting parser output@>@; + +@ @<Raw option list@>= + @<Parser specific option list@>@; + +@ @<Higher index options@>= + @<Higher index parser specific options@>@; + +@ @<Cases affecting the whole program@>= + @<Handle parser output options@>@; + +@ @<Output descriptor fields@>= + @<Parser specific output descriptor fields@>@; + +@ @<Default outputs@>= + @<Parser specific default outputs@>@; + +@ @<Various output modes@>= + @<Handle parser related output modes@>@; + +@ @<Output modes@>= + @<Parser specific output modes@>@; + +@ @<Output constants@>= + @<Output parser constants@>@; + +@ @<Cases involving specific modes@>= + @<Configure parser output modes@>@; + +@ @<Perform output@>= + @<Output parser tokens@>@; + +@ @<Output action switch, if any@>= + @<Output parser semantic actions@>@; + +@ @<\Cee\ preamble@>= + @<Common code for \Cee\ preamble@>@; + +@i common.w +@i bs.w + +@*1 Parser dependent settings. +This is it for the core table output functions. +To make all this into a working code in this +case, lexing and error function declarations are supplied. +@<Auxiliary function declarations@>= +#ifndef HAS_SCANNER + int yylex(void); + int yyerror(void); +#endif + +@ @<Auxiliary function definitions@>= +#ifndef HAS_SCANNER + int yylex(void){} + int yyerror(void){} +#endif + +@ \let\B\oldB % \Cee\ mode mixes all up +@c + +@<\Cee\ preamble@>@; + +#include PARSER_FILE + +@<\Cee\ postamble@>@; + +@** Index (for {\tt \jobname}). +\def\readcontents{% + {% + \acrofalse +% \def\jobname{bparser}\input bparser.toc +% \def\jobname{ftablesout}\input ftablesout.toc + }% + \input \contentsfile +}
\ No newline at end of file diff --git a/support/splint/cweb/mkscanner.w b/support/splint/cweb/mkscanner.w new file mode 100644 index 0000000000..22c39e9342 --- /dev/null +++ b/support/splint/cweb/mkscanner.w @@ -0,0 +1,102 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +@**Lexer tables. This is the main table output code. +Its core comes from the Flex Kit(ten?) (\.{fk.w}) +which will be included as soon as this paragraph is +over. Feel free to +interpret the acronym as something easier to remember. + +@<Table names@>= + @<Scanner table names@>@; + +@ @<Local variable and type declarations@>= + @<Variables and types local to the scanner driver@>@; + +@ @<Establish defaults@>= + @<Compute exotic scanner constants@>@; + +@ @<Output action switch, if any@>= + @<Output scanner actions@>@; + +@ @<Constant names@>= + @<Scanner constants@>@; + +@ @<Output constants@>= + @<Output exotic scanner constants@>@; + +@ @<\Cee\ preamble@>= + @<Common code for \Cee\ preamble@>@; + @<Scanner variables and types for \Cee\ preamble@>@; + +@ @<Output modes@>= + @<Scanner specific output modes@>@; + +@ @<Various output modes@>= + @<Handle scanner output modes@>@; + +@ @<Cases affecting the whole program@>= + @<Handle scanner output options@>@; + +@ @<Raw option list@>= + @<Scanner specific option list@>@; + +@ @<Higher index options@>= + @<Higher index scanner specific options@>@; + +@ @<Short option list@>= + @<Shortcuts for command line options affecting scanner output@>@; + +@i common.w +@i fk.w + +@*1 Lexer dependent settings. +This is it for the core table output functions. +To make all this into a working code in this +case, no function declarations are supplied. +@<Auxiliary function declarations@>= + +@ @<Auxiliary function definitions@>= + +@ @<\Cee\ preamble@>= + void define_all_states( void ); + +@ The lexer takes no parameters in this case but if one reuses a +lexer written for a different purpose, the situation may be different. +%\let\B\oldB % \Cee\ mode mixes all up + +@d YYPARSE_PARAMETERS +@d YY_USER_INIT define_all_states(); +@d yyterminate() TeX_( "/yyterminate" ); return YY_NULL + +@c + +@<\Cee\ preamble@>@; +typedef int YYSTYPE; +#define YY_BREAK return 0; + +#include LEXER_FILE + +@<\Cee\ postamble@>@; + +@** Index (for {\tt \jobname}). +%\def\readcontents{% +% {% +% \acrofalse +% \def\jobname{bparser}\input bparser.toc +% }% +% \input \contentsfile +%}
\ No newline at end of file diff --git a/support/splint/cweb/np.w b/support/splint/cweb/np.w new file mode 100644 index 0000000000..aa20e551c7 --- /dev/null +++ b/support/splint/cweb/np.w @@ -0,0 +1,380 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1 The name parser. What follows is an example parser for the name +processing. This approach (i.e. using a `full blown' parser/scanner +combination) is probably not the best way to implement such machinery +but its main purpose is to demonstrate a way to create a separate +parser for local purposes. +% We include the macros here since this file is intended to be +% included by the documentation `aggregator' so putting bare \TeX\ +% at the beginning of the file runs the risk of producing and error +% of having \TeX\ material inside a \Cee\ section. +\let\currentparsernamespace\parsernamespace + \let\parsernamespace\smallnamespace + \let\hostparsernamespace\smallnamespace + \input stokenset.sty +\let\parsernamespace\currentparsernamespace +@(small_parser.yy@>= +@G Switch to generic mode. +%{@> @<Name parser \Cee\ preamble@> @=%} + @> @<Bison options@> @= +%union {@> @<Union of parser types@> @=} +%{@> @<Name parser \Cee\ postamble@> @=%} + @> @<Token and types ...@> @= +%% + @> @<Parser productions@> @= +%% +@g + +@ @<Bison options@>= +@G +%token-table +%debug +%start full_name +@g + +@ @<Token and types declarations@>= +@G +%token PERCENT_IDENTIFIER +%token IDENTIFIER +%token OPTIONAL +%token NO_ATTR +%token INTEGER +%token EXTENDED +%token WILDCARD +@g + +@ @<Parser productions@>= +@G +full_name: + identifier_string suffixes.opt {@> @<Compose the full name@> @=} +; + +identifier_string: + PERCENT_IDENTIFIER {@> @<Attach option name@> @=} +| IDENTIFIER {@> @<Start with an identifier@> @=} +| '<' IDENTIFIER '>' {@> @<Start with a tag@> @=} +| '\'' WILDCARD '\'' {@> @<Start with a quoted string@> @=} +| '\'' '>' '\'' {@> @<Start with a \prodstyle{'>'} string@> @=} +| '\'' '<' '\'' {@> @<Start with a \prodstyle{'<'} string@> @=} +| '\'' '.' '\'' {@> @<Start with a \prodstyle{'.'} string@> @=} +| '\'' '_' '\'' {@> @<Start with an \prodstyle{'\_'} string@> @=} +| '\'' '-' '\'' {@> @<Start with a \prodstyle{'-'} string@> @=} +| qualifier {@> @<Turn a qualifier into an identifier@> @=} +| identifier_string IDENTIFIER {@> @<Attach an identifier@> @=} +| identifier_string qualifier {@> @<Attach qualifier to a name@> @=} +| identifier_string INTEGER {@> @<Attach an integer@> @=} +; + +suffixes.opt: + {@> TeX_( "/yy0{}" ); @=} +| '.' {@> TeX_( "/yy0{/nx/dotsp/nx/sfxnone}" ); @=} +| '.' suffixes {@> @<Attach suffixes@> @=} +| '.' qualified_suffixes {@> @<Attach qualified suffixes@> @=} +; + +suffixes: + IDENTIFIER {@> @<Start with a named suffix@> @=} +| INTEGER {@> @<Start with a numeric suffix@> @=} +| suffixes '.' {@> @<Add a dot separator@> @=} +| suffixes IDENTIFIER {@> @<Attach a named suffix@> @=} +| suffixes INTEGER {@> @<Attach integer suffix@> @=} +| qualifier '.' {@> TeX_( "/yy0{/nx/sfxn/the/yy(1)/nx/dotsp}" ); @=} +| suffixes qualifier '.' {@> TeX_( "/yy0{/the/yy(1)/nx/sfxn/the/yy(2)/nx/dotsp}" ); @=} +; + +qualified_suffixes: + suffixes qualifier {@> @<Attach a qualifier@> @=} +| qualifier {@> @<Start suffixes with a qualifier@> @=} +; + +qualifier: + OPTIONAL {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| NO_ATTR {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| EXTENDED {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +; +@g + +@ @<Compose the full name@>= + @[TeX_( "/yy0{/the/yy(1)/the/yy(2)}/namechars/yyval" );@]@; + +@ @<Attach option name@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/optstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with an identifier@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with a tag@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{</the/toksa>}{</the/toksb>}}" );@]@; + +@ @<Start with a quoted string@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/chstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with a \prodstyle{'<'} string@>= + @[TeX_( "/yy0{/nx/chstr{<}{<}}" );@]@; + +@ @<Start with a \prodstyle{'>'} string@>= + @[TeX_( "/yy0{/nx/chstr{/greaterthan}{/greaterthan}}" );@]@; + +@ @<Start with an \prodstyle{'\_'} string@>= + @[TeX_( "/yy0{/nx/chstr{/uscoreletter}{/uscoreletter}}" );@]@; + +@ @<Start with a \prodstyle{'-'} string@>= + @[TeX_( "/yy0{/nx/chstr{-}{-}}" );@]@; + +@ @<Start with a \prodstyle{'.'} string@>= + @[TeX_( "/yy0{/nx/chstr{.}{.}}" );@]@; + +@ @<Turn a qualifier into an identifier@>= + @<Start with an identifier@>@; + +@ @<Attach an identifier@>= + @[TeX_( "/getsecond{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/appendr/toksa{/space}" );@]@; + @[TeX_( "/getfirst{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/concat/toksa/toksb" );@]@; + @[TeX_( "/getthird{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/appendr/toksb{/space}" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksc" );@]@; + @[TeX_( "/concat/toksb/toksc" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Attach qualifier to a name@>= + @<Attach an identifier@> + +@ @<Attach an integer@>= + @<Attach an identifier@>@; + +@ @<Attach suffixes@>= + @[TeX_( "/yy0{/nx/dotsp/the/yy(2)}" );@]@; + +@ @<Attach qualified suffixes@>= + @<Attach suffixes@>@; + +@ @<Start with a named suffix@>= + @[TeX_( "/yy0{/nx/sfxn/the/yy(1)}" );@]@; + +@ @<Start with a numeric suffix@>= + @[TeX_( "/yy0{/nx/sfxi/the/yy(1)}" );@]@; + +@ @<Add a dot separator@>= + @[TeX_( "/yy0{/the/yy(1)/nx/dotsp}" );@]@; + +@ @<Attach integer suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxi/the/yy(2)}" );@]@; + +@ @<Attach a named suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxn/the/yy(2)}" );@]@; + +@ @<Attach a qualifier@>= + @[TeX_( "/yy0{/the/yy(1)/nx/qual/the/yy(2)}" );@]@; + +@ @<Start suffixes with a qualifier@>= + @[TeX_( "/yy0{/nx/qual/the/yy(1)}" );@]@; + +@ \Cee\ preamble. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<Name parser \Cee\ preamble@>= + +@ \Cee\ postamble. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<Name parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) @[yyprint (file, type, value)@] + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ Union of types. + +@<Union of parser types@>= + +@*1 The name scanner. +%\checktabletrue +@(small_lexer.ll@>= +@G + @> @<Lexer definitions@> @= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% +@O +void define_all_states( void ) { + @<Collect all state definitions@>@; +} +@o +@g + +@ @<Lexer definitions@>= + @<Lexer states@>@; +@G +letter [_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] +wc ([^\\\'\"]{-}[_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0-9]|\\.) +id {letter}({letter}|[-0-9])* +int [0-9]+ +@g + +@ @<Collect all state definitions@>= +#define _register_name( name ) @[Define_State( #name, name )@] +/* nothing for now */ +#undef _register_name + +@ Strings and characters in directives/rules. +@<Lexer states@>= +@G +%x SC_ESCAPED_STRING SC_ESCAPED_CHARACTER +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + +@ @<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="small_lexer.c" +@g + +@ @<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +% +\yyskipparsetrue +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ This collection of regular expressions might seem redundant, and in +its present state, it certainly is. However, if later on the +typesetting style for some of the keywords would need to be adjusted, +such changes would be easy to implement, since the template is already +here. +\yyskipparsefalse % this is not necessary +@<Scan identifiers@>= +@G +"%binary" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%code" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%debug" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%default-prec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%define" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%defines" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%destructor" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%dprec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%empty" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%error-verbose" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%expect" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%expect-rr" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%file-prefix" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%fixed-output-files" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%initial-action" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%glr-parser" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%language" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%left" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%lex-param" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%locations" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%merge" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%name-prefix" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%no-default-prec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%no-lines" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%nonassoc" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%nondeterministic-parser" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%nterm" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%output" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%param" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%parse-param" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%prec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%precedence" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%printer" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%pure-parser" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%require" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%right" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%skeleton" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%start" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%term" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%token" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%token-table" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%type" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%union" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%verbose" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%yacc" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%default"[-_]"prec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%error"[-_]"verbose" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%expect"[-_]"rr" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%fixed"[-_]"output"[-_]"files" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%name"[-_]"prefix" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%no"[-_]"default"[-_]"prec" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%no"[-_]"lines" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%pure"[-_]"parser" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%token"[-_]"table" {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} +"%"({letter}|[0-9]|[-_]|"%"|[<>])+ {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} + +"opt" {@> @[TeX_( "/yylexreturnval{OPTIONAL}" );@]@=} +"na" {@> @[TeX_( "/yylexreturnval{NO_ATTR}" );@]@=} +"ext" {@> @[TeX_( "/yylexreturnval{EXTENDED}" );@]@=} + +[<>._\'] {@> @[TeX_( "/yylexreturnchar" );@]@=} +{wc} {@> @[TeX_( "/yylexreturnval{WILDCARD}" );@]@=} + +{id} {@> @[@<Prepare to process an identifier@>@]@=} +{int} {@> @[TeX_( "/yylexreturnval{INTEGER}" );@]@=} + +"\"" {@> @[TeX_( "/yylexnext" );@]@=} +. {@> @[@<React to a bad character@>@]@=} +@g + +@ @<Prepare to process an identifier@>= + @[TeX_( "/yylexreturnval{IDENTIFIER}" );@]@; + +@ @<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; diff --git a/support/splint/cweb/philosophy.w b/support/splint/cweb/philosophy.w new file mode 100644 index 0000000000..4d30f2d765 --- /dev/null +++ b/support/splint/cweb/philosophy.w @@ -0,0 +1,223 @@ +@**Philosophy. +This section should, perhaps, be more appropriately called {\it rant\/} but +{\it philosophy\/} sounds more academic. The design of any software involves +numerous choices, and \splint\ is no exception. Some of these choices +are explained in the appropriate places in the package files. This +section collects a few `big picture' viewpoints that did not fit elsewhere. + +@*1 On typographic style. +It must seem quite perplexing to some readers that a +document with a focus on {\it pretty-printing\/} displays such a +wanton disregard for good typographic taste. Haphazard choice of +styles to present programming constructs, random overabundance of +fonts on almost every single page are just a few among the many typographic sins +and design guffaws so amply manifested on these pages. The author has +to take full responsibility for the lack of taste in this +opus and has only one argument in his defense: this is not +merely a book for a good night read but a piece of technical +documentation. + +In many ways, the goal of this document is somewhat contrary to that +of a well-written manual: to display the main features +prominently and in logical order. After all, this is a package that is +intended to help {\it write\/} such manuals so it inevitably must +display some use cases that demonstrate a variety of typographic styles +possible to achieve with \splint. Needless to say, {\it variety\/} and +{\it consistency\/} seldom go hand in hand and it is consistency that +makes for a pretty page. One of the objectives has been to demonstrate a +number of quite technical programming constructs so one should keep in +mind that it is assumed that the reader will want to look up the input +files to see how some (however ugly and esoteric) typographic effects +had been achieved. + +On the other hand, to use a clich\'e, beauty is in the eyes of the +beholder so what makes a book readable (or even beautiful) may well +depend on the background of the reader. As an example, letterspacing +as a typographic device is almost universally reviled in Western +typography (aside from a few niche uses such as setting titles). In +Russian, however (at least until recently), letterspacing has been +routinely used for emphasis (or, as a Russian would put it, +e$\,$m$\,$p$\,$h$\,$a$\,$s$\,$i$\,$s) in lieu of, say, {\it italics}. Before +I hear any objections from typography purists, let me just say that +this technique fits perfectly with the way emphasis works in the Russian speech: a +speaker slowly enunciates the sounds of each word (incidentally, +emphasizing {\it emphasis\/} is a perfect example of why this method +would fail in most English texts). Letterspaced +sentences are easy to find on a page and set a special reading rhythm, +which is an added bonus in many cases, although it does violate the +`universally gray pages are a must' dogma. + +@*1 Why GPL. +The choice of license for this project goes beyond merely showing the +source. \TeX, by its very nature is an open source language, so it +is not a matter of hiding anything from the user or a potential +developer. The \Cee\ code is a different matter but the source is not +that complicated. Reducing the licensing issue to the ability of +someone else to see the source code is a great +oversimplification. Without getting into too many details of so-called +`open source licenses' (other than GPL) and arguing with their advocates, let me simply +express my lack of understanding at the arguments that purport that +BSD-style licenses introduce more freedom by allowing a software +vendor to incorporate the BSD-licensed software into their +products. What benefit does one derive from such `extension' of software +freedom? Perhaps the hope that the `open source' (for the lack of a +better term) will prompt the vendor to follow the accepted free (or +any other, for that matter!) software standards and make its software +more interoperable with the free alternatives? A well-known software +giant's {\it embrace, extend, extinguish\/} philosophy shows how na\"{\i}ve and +misplaced such hopes are. + +I am not going to argue for the benefits of free software at length, either +(such benefits seem self-evident to me, although the readers should +feel free to disagree). Let me just point out that software companies +enjoy quite a few freedoms that we, as software consumers elect to +afford them. Among such freedoms are the ability to renege on any +promises made to potential users and withdraw any guarantees that such +users might enjoy. Free software, of course, does not provide any +guarantees, either but `you get what you paid for'. As a result of +such `release of any responsibility', the claims of increased +reliability or better support for the commercial software sound a +bit hollow. Another well spread tactic is user brainwashing and +changing the culture (usually for the worse) in order to promote new +`user-friendly' features of commercial software. Instead of taking +advantage of computers as cognitive machines we have come to view +them as advanced media players that we interact with through +artificial, unnatural interfaces. Meaningless terminology (`UX' for +`user experience'? What in the world is `user experience'?) +proliferates, and programmers are happy to deceive themselves with +their newly discovered business prowess. + +One would hope that the somewhat higher standards of the `real' +manufacturers might percolate to the software world, however, the +reality is very different. Not only has life-cycle `engineering' +got to the point where manufacturers can predict the life spans of +their products precisely, embedded software in those products has +become an enabling technology that makes this `life design' much +easier. + +In effect, by embedding software in their products, hardware +manufacturers not only piggy-back on software's perceived complexity, +and argue that such complex systems cannot be made reliable, they have +an added incentive to uphold this image. The software weighs nothing, +memory is cheap, consumers are easy to deceive, thus `software is +expensive' and `reliable software is prohibitively so'. Designing reliable +software is quite possible, though, just look at programmable +thermostats, simple cellphones and other `invisible' gadgets we +enjoy. The `software ideology' with its `IP' lingo is spreading like a +virus even through the world of real things. We now expect products to +break and are too quick to forgive sloppy engineering that goes into +everyday things. We are also getting used to the idea that it is the +manufacturers that get to dictate the terms of use for `their' products +and that we are merely borrowing `their' stuff. + +The GPL was conceived as an antidote to this scourge. This document is a +remarkable piece of `legal engineering': a self-propagating license +with a clearly outlined set of goals. While by itself it does not +guarantee reliability or quality, it does inhibit the spread of the `IP' +(which is sometimes sarcastically, though quite perceptively, +`deabbreviated' as {\sl I}maginary {\sl P}roperty) disease through +software. + +The industry has adapted, of course. So called (non GPL) `open source +licenses', that are supposed to be an improvement on GPL, +are a sort of `immune reaction' to the free software +movement. Convince and confuse enough apathetic users and the +protections granted by GPL are no longer visible. + +@*1 Why not \Cee$++$ or OOP in general. +The choice of the language was mainly driven by \ae sthetic motives: +\Cee$++$ has a bloated and confusing standard, partially supported by +various compilers. It seems that there is no agreement on what +\Cee$++$ really is or how to use some of its constructs. This is all +in contrast to \Cee\ with its well defined and concise body of +specifications and rather well established stylistics. The existence +of `obfuscated \Cee' is not good evidence of deficiency and \Cee$++$ +is definitely not immune to this malady. + +Object oriented design has certainly taken on an aura of a religious +dictate, universally adhered to and forcefully promoted by its +followers. Unfortunately, the definition of what constitutes an +`object-oriented' approach is rather vague. A few abstract concepts are +commonly tossed about to give the illusion of a well developed +abstraction (such as `polymorphism', `encapsulation', and so on) but +definitions vary in both length and contents, depending on the source. + +On a syntactic level, some features of object-oriented languages are +undoubtedly very practical (such as a |this| pointer in \Cee$++$), +however, many of those features can be effectively emulated with some +clever uses of an appropriate preprocessor (there are a few +exceptions, of course, |this| being one of them). The rest of the +`object-oriented philosophy' is just that: a design philosophy. Before +that we had structured programming, now there are patterns, extreme, +agile, reactive, etc. They might all find their uses, however, there +are always numerous exceptions (sometimes even global variables and +|goto|'s have their place, as well). + +A pedantic reader might point out a few object-oriented features even +in the \TeX\ portion of the package and then accuse the author of +being `inconsistent'. I am always interested in possible improvements +in style but I am unlikely to consider any changes based solely on the +adherence to any particular design fad. + +In short, OOP was not shunned simply because a `non-OOP' language was +chosen, instead, whatever approach or style was deemed most effective +was used. The author's judgment has not always been perfect, of course, +and given a good reason, changes can be made, including the choice of +the language. `Make it object-oriented' is neither a good reason nor a +clearly defined one, however. + +@*1 Why not $*$\TeX. +Simple. I never use it and have no idea of how packages, classes, +etc., are designed. I have heard it has impressive mechanisms for +dealing with various problems commonly encountered in \TeX. Sadly, my +knowledge of $*$\TeX\ machinery is almost nonexistent. This may change +but right now I have tried to make the macros as generic as possible, +hopefully making $*$\TeX\ adaptation easy. + +The following quote from \cite[Ho] makes me feel particularly uneasy +about the current state of development of various \TeX\ variants: +``{\it Finally, to many current programmers\/ \.{WEB} source simply feels over-documented +and even more important is that the general impression is that of a finished book: +sometimes it seems like\/ \.{WEB} actively discourages development. This is +a subjective point, but nevertheless a quite important one.}'' + +{\it Discouraging development\/} seems like a good thing to +me. Otherwise we are one step away from encouraging writing poor +software with inadequate tools merely `to encourage development'. + +The feeling of a \.{WEB} source being {\it over-documented\/} is most +certainly subjective, and, I am sure, not shared by all `current +programmers'. The advantage of using \.{WEB}-like tools, however, is +that it gives the programmer the ability to place the vital +information where it does not distract the reader (`developer', +`maintainer', call it whatever you like) from the logical flow of the +code. + +Some of the complaints in \cite[Ho] are definitely justified, +although it seems that a better approach would be to write an improved +tool similar to \.{WEB}, rather than give up all the flexibility such +a tool provides. + +@*1 Why \CWEB. +\CWEB\ is not as polished as \TeX\ but it works and has a +number of impressive features. It is, regrettably, a `niche' tool and +a few existing extensions of \CWEB\ and software based on similar ideas +do not enjoy the popularity they deserve. Literate philosophy has been +largely neglected even though it seems to have a more logical +foundation than OOP. Under these circumstances, \CWEB\ seemed to be +the best available option. + +@*1 Why not GitHub, Bitbucket, etc. +Git is an incredible tool and is used extensively in the development +of \splint. The distribution archive is a Git repository. The use of +centralized services such as GitHub, however, seems redundant. The +standard cycle, `clone-modify-create pull request' works the same even +when `clone' is replaced by `download'. Thus, no functionality is +lost. This might change if the popularity of the package unexpectedly +increases. + +On the other hand, GitHub and its cousins are commercial entities, +whose availability in the future is not guaranteed (nothing is +certain, of course, no matter what distribution method is +chosen). Keeping \splint\ as an archive of a Git repository seems like +an efficient way of being ready for an unexpected change. diff --git a/support/splint/cweb/references.w b/support/splint/cweb/references.w new file mode 100644 index 0000000000..76dc718d3b --- /dev/null +++ b/support/splint/cweb/references.w @@ -0,0 +1,73 @@ +@** Bibliography. This list of references is not meant to be +exhaustive or complete. These are merely the papers and the books +mentioned in the body of the program above. Naturally, this project +has been influenced by many outside ideas but it would be impossible +to list them all due to time and (human) memory limitations. + +{% +\def\BASIX{{B\kern-.7ptA\kern-.7ptS\kern-.3pt\lower1.3pt\hbox{I}\kern-.3pt X}} +\def\MF{{\tt METAFONT}} +\def\bterm#1{\item{[#1]\namedspot{#1bibref}\quad}\ignorespaces}% +\setbox0=\hbox{[ISO/C11]\quad} +\parindent=0pt +\advance\parindent by \wd0 +\ninepoint +\smallskip +\centerline{\dinkus}% +\smallskip + +\bterm{Ah}Alfred V.~Aho et al., {\it Compilers: Principles, +Techniques, and Tools}, Pearson Education, 2006. + +\bterm{Bi}Charles Donnelly and Richard Stallman, {\it Bison, The +Yacc-compatible Parser Generator}, The Free Software Foundation, 2013. +\url{http://www.gnu.org/software/bison/} + +\bterm{DEK1}Donald E.~Knuth, {\it The \TeX book}, Addison-Wesley Reading, Massachusetts, 1984. + +\bterm{DEK2}Donald E.~Knuth {\it The future of \TeX\ and \MF}, TUGboat {\bf 11} (4), p.~489, 1990. + +\bterm{Do}Jean-luc Doumont, {\it Pascal pretty-printing: an example of ``preprocessing with \TeX''}, +TUGboat {\bf 15} (3), 1994---Proceedings of the 1994 TUG Annual Meeting + +\bterm{Er}Sebastian Thore Erdweg and Klaus Ostermann, {\it Featherweight \TeX\ and Parser Correctness}, +Proceedings of the Third International Conference on Software Language Engineering, +pp.\ 397--416, Springer-Verlag Berlin, Heidelberg {\bf 2011}. + +\bterm{Fi}Jonathan Fine, {\it The \.{\\CASE} and \.{\\FIND} macros}, +TUGboat {\bf 14} (1), pp.~35--39, 1993. + +\bterm{Go}Pedro Palao Gostanza, {\it Fast scanners and self-parsing in \TeX}, +TUGboat {\bf 21} (3), 2000---Proceedings of the 2000 Annual Meeting. + +\bterm{Gr}Andrew Marc Greene, {\it \BASIX---an interpreter written in \TeX}, TUGboat {\bf 11} (3), +1990---Proceedings of the 1990 TUG Annual Meeting. + +\bterm{Ha}Hans Hagen, {\it Lua\TeX: Halfway to version~1}, TUGboat +{\bf 30} (2), pp.~183--186, 2009. \url{http://tug.org/TUGboat/tb30-2/tb95hagen-luatex.pdf}. + +\bterm{Ho}Taco Hoekwater, {\it Lua\TeX\ says goodbye to Pascal}, TUGboat {\bf 30} (3), +pp.~136--140, 2009---Euro\TeX\ 2009 Proceedings. + +\bterm{Ie}R.~Ierusalimschy et al., {\it Lua~5.1 Reference Manual}, +{\tt Lua.org}, August 2006. \url{http://www.lua.org/manual/5.1/}. + +\bterm{ISO/C11}{\it ISO/IEC 9899---Programming languages---C (C11)}, December~2011, draft available at +\url{http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf} + +\bterm{Jo}Derek M.~Jones, {\it The New C Standard: An Economic and +Cultural Commentary}, available at \url{http://www.knosof.co.uk/cbook/cbook.html}. + +\bterm{La}{\it The \.{l3regex} package: regular expressions in \TeX}, The \LaTeX3\ Project. + +\bterm{Pa}Vern Paxson et al., {\it Lexical Analysis With Flex, for +Flex~2.5.37}, July~2012. \url{http://flex.sourceforge.net/manual/}. + +\bterm{Sh}Alexander Shibakov, {\it Parsers in \TeX\ and using \CWEB\ for general pretty-printing}, +TUGboat {\bf 35} (1), 2014, available as part of the documentation supplied with \splint. + +\bterm{Wo}Marcin Woli\'nski, {\it {\tt Pretprin}---a \LaTeX2$\epsilon$ package for +pretty-printing texts in formal languages}, +TUGboat {\bf 19} (3), 1998---Proceedings of the 1998 TUG Annual Meeting. + +}
\ No newline at end of file diff --git a/support/splint/cweb/splint.w b/support/splint/cweb/splint.w new file mode 100644 index 0000000000..05fe5c42fc --- /dev/null +++ b/support/splint/cweb/splint.w @@ -0,0 +1,102 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. +\newwrite\gindex + +@i bo.x +@i lo.x +@i np.x +@i common.w +@i bs.w +@i fk.w +@i philosophy.w +@i references.w + +\let\N\oldN +\let\hostparsernamespace\mainnamespace % to typeset examples in the text + % properly +@**Index. This section is, perhaps, the most valuable product of +\CWEB's labors. It lists references to definitions (set in {\it +italic}) as well as uses for each \Cee\ identifier used in the +source. Special facilities have been added to extend indexing to +\bison\ grammar terms and \TeX\ control sequences encountered in +\bison\ actions. Definitions of tokens (via \prodstyle{\%token}, +\prodstyle{\%nterm} and \prodstyle{\%type} directives) are +%$\underline{\hbox{underlined}}$ +typeset in {\bf bold}. +The \bison\ and \TeX\ entries are put +in distinct sections of the index in order to keep the separation +between the \Cee\ entries and the rest. It may be worth noting that +the {\it definition\/} of the symbol is listed under both its `macro +name' (such as \.{CHAR}, typeset as \prodstyle{CHAR} in the case of +the grammar below), as well as its `string' name if present (to +continue the previous example, \.{"char"} is synonymous with +\prodstyle{CHAR} after a declaration such as `\prodstyle{\%token} +\prodstyle{CHAR} \.{"char"}'), while the {\it use\/} of the term lists +whichever token form was referenced at the point of use (both forms +are accessible when the entry is typeset for the index and a macro can +be written to mention the other form as well). The original syntax of +\bison\ allows the programmer to declare tokens such as +\prodstyle{'\{'} and \prodstyle{'\}'} and the indexing macros honor +this convention even though in a typeless environment such as the +one the present typesetting parser is operating in such declarations +are redundant. The indexing macros also record the use of such +character tokens. The quotes indicate +that the `string' form of the token's name was used. A section set in +{\it italic\/} references the point where the corresponding term +appeared on the left hand side of a production. A production: +\let\TeXx\TeXxi +\def\gatoks{% + \omit\hfil&\omit\hfil&\omit\hfil\hbox to2em{\hfil}&\omit\hfil\cr + \noalign{\vskip-\baselineskip}% +}% +\beginmprod +left_hand_side: + term.1 term.2 term.3 \{\stashed{|TeX_("/dosomething/yy(1)");|}\} +\endmprod +inside the \TeX\ part of a \CWEB\ section will generate several +index entries, as well, including the entries for any +material inside the action, mimicking \CWEB's behavior for the +{\it inline \Cee\/} (\.{\yl}$\ldots$\.{\yl}). Such entries (except for +the references to the \Cee\ code inside actions) are labeled with $^\circ$, +to provide a reminder of their origin. + +This parser collection, as well as the indexing facilities therein have been +designed to showcase the broadest range of options available to the user +and thus it does not always exhibit the most sane choices one could make (for +example, using a full blown parser for term {\it names\/} is poor +design but it was picked to demonstrate multiple parsers in one +program). The same applies to the way the index is constructed (it +would be easy to only use the `string' name of the token if +available, thus avoiding referencing the same token twice). + +\TeX\ control sequences are listed following the index of all \bison\ +entries. The two indices are separated by a {\it dinkus} +(\dinkus). Since it is nearly impossible to determine at what point a +\TeX\ macro is defined (and most of them are defined outside of the +\CWEB\ sources), only their uses are listed (to be more precise, {\it +every\/} appearance of a macro is assumed to be its use). In a few cases, a +`graphic' representation for a control sequence is also listed (for +example, {\termindexfalse\def\texnspace{texline}\inlineTeXx{/getfirst}} represents +{\termindexfalse\def\texnspace{other}\inlineTeXx{/getfirst}}). The index entries are ordered alphabetically. The +latter may not be entirely obvious in the cases when the `graphical +representation' of the corresponding token manifests a significant +departure from its string version (such as |TeX_("/yy(1)");| +instead of {\def\texnspace{other}|TeX_("/yy(1)");|}). +\closeout\gindex +\let\inx\inxmod +\let\fin\finmod +\displaytokenrawtrue +\def\topofcontents{\null\vskip-3\baselineskip\centerline{C{\sc ONTENTS} (\sc\uppercase\expandafter{\title})}\medskip} diff --git a/support/splint/cweb/ssffo.w b/support/splint/cweb/ssffo.w new file mode 100644 index 0000000000..56e710b8c6 --- /dev/null +++ b/support/splint/cweb/ssffo.w @@ -0,0 +1,118 @@ +@q Copyright 2012-2015, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> +\input limbo.sty +\def\optimization{5} +\input yy.sty + +\let\oldN\N +\let\N\textN +\let\M\textM +\def\hostparsernamespace{[unreacheable]} + +@** A simple scanner for \flex\ options. +This is a `bare-bones' scanner for a subset of the `extended' \bison\ +grammar that parses, well, some of the `extensions', namely, the +\flex\ state declarations. It does not use the state mechanism +itself, and is supposed to be used with the bootstrapping parser, even +though it is not strictly necessary. It parses state declarations as +long as they are separated into their own \CWEB\ sections and extracts +the {\it names\/} of the states. The \flex\ scanner output `driver' +does the rest after including the produced header file. + +If reusing the existing scanner for \bison\ were not a priority a +proper way to design a scanner like this is to make it a subset of the +existing scanner code. This way portions of the program would be made +more reusable and the overall design made more consistent. +@s TeX_ TeX +@(ssffo.ll@>= +@G + @> @<Lexer definitions@> @= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% +@g + +@ A couple of handy abbreviations to get started. Note that the +definition of a letter is more restrictive in this case since we only +need to grab the states of an existing \bison\ lexer. For a +bootstrapping scanner like this it is beneficial to fail early while +scanning something that is not in its attention domain: it results in +faster bootstrapping and lower chance of accidentally parsing +something that should not have been. Making the syntax and the grammar +more restrictive helps to acheive this, as well as makes the overall +design simpler. +@<Lexer definitions@>= +@G +letter [_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] +id {letter}({letter}|[-0-9])* +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + + void define_all_states( void ){} + +@ A standard combination of options to match the \TeX\ code that +drives the scanner. +@<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="ssffo.c" +@g + +@ There are not that many regular expressions to list, since the range +of tokens recognized by this routine is not very wide. +@<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ The rest of it are either identifiers or \.{\%}-options. +@<Scan identifiers@>= +@G +{id} {@> @<Return an identifier@> @=} +"%x" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_X}" );@] @=} +"%s" {@> @[TeX_( "/yylexreturnptr{FLEX_STATE_S}" );@] @=} +. {@> @<React to a bad character@> @=} +@g + +@ @<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; + +@ The lexer returns standard \.{\\yyunion} types. +@<Return an identifier@>= + @[TeX_( "/edef/next{/yylval{/nx/idit{/the/yytextpure}{/the/yytext}" );@]@; + @[TeX_( " {/the/yyfmark}{/the/yysmark}}}/next" );@]@; + @[TeX_( "/yylexreturn{ID}" );@]@; + +@** Index. +\def\TeXx{\TeX\ material}
\ No newline at end of file diff --git a/support/splint/doc/ldman.pdf b/support/splint/doc/ldman.pdf Binary files differnew file mode 100644 index 0000000000..22a07ae754 --- /dev/null +++ b/support/splint/doc/ldman.pdf diff --git a/support/splint/doc/splint.pdf b/support/splint/doc/splint.pdf Binary files differnew file mode 100644 index 0000000000..101984caa1 --- /dev/null +++ b/support/splint/doc/splint.pdf diff --git a/support/splint/doc/tb109shibakov.pdf b/support/splint/doc/tb109shibakov.pdf Binary files differnew file mode 100644 index 0000000000..094e15fb55 --- /dev/null +++ b/support/splint/doc/tb109shibakov.pdf diff --git a/support/splint/examples/count/count.sty b/support/splint/examples/count/count.sty new file mode 100644 index 0000000000..bb8998e64f --- /dev/null +++ b/support/splint/examples/count/count.sty @@ -0,0 +1,552 @@ +% the sequences in this file have the eventual goal of implementing macros that count and compare token sequences +% (as either parameters or contents of token registers) in expandable manner. +% currently, a new sequence is prepared first, consisting of `markers' +% that indicate whether the token in the coresponding position of the +% original sequence is a potential blank space or brace (closing or +% opening); +% the macros then make the determination about the last possible blank +% on whether this is a true space or not; the goal is to make this +% macro independent of the value of \escapechar +% +% as of now the implementation is far from refined although an expandable conting macro +% can be easily produced + +\catcode`\@=11 +\input ../../tex/yycommon.sty +\input ../../tex/yymisc.sty + +%%%%%%%%%%%%% the material before this point should be included from appropriate files + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- `parsed' sequence + +\def\yypreparsetokensequenc@#1#2#3{% + \yystringempty{#2}{#1{#3}}{\yypreparsetokensequen@@{#1}{#2}{#3}}% +} + +\def\yypreparsetokensequen@@#1#2#3{% remaining sequence is nonempty + \yystartsinbrace{#2}{\yydealwithbracedgroup{#1}{#2}{#3}}{\yypreparsetokensequ@n@@{#1}{#2}{#3}}% +} + +\def\yydealwithbracedgroup#1#2#3{% the first token of the remaining sequence is a brace + \iffalse{\fi\yydealwithbracedgro@p#2}{#1}{#3}% +} + +\def\yydealwithbracedgro@p#1{% + \yypreparsetokensequenc@{\yyrepackagesequence}{#1}{}% +} + +% #1 -- parsed sequence +% this is a sequence to `propagate expansion' into the next parameter. +% the same can be achieved by packaging the whole sequence with a +% \csname ... \endcsname pair and using a simple \expandafter + +\def\yyrepackagesequence#1{% + \yyrepackagesequenc@{}#1\end +} + +% #1 -- `packaged' sequence (\expandafter\expandafter\expandafter ? ...) +% #2 -- the next category 12 character or \end + +\def\yyrepackagesequenc@#1#2{% + \ifx#2\end + \yybreak{\yyrepackagesequ@nc@{#1\expandafter\expandafter\expandafter}}% + \else + \yybreak{\yyrepackagesequenc@{#1\expandafter\expandafter\expandafter#2}}% + \yycontinue +} + +% #1 -- `packaged' sequence (\expandafter\expandafter\expandafter ? ...) +% this macro is followed by the remainder of the original sequence with a so far +% unmatched right brace, the `call stack' and the parsed sequence. + +\def\yyrepackagesequ@nc@#1{% + \expandafter\expandafter\expandafter\yyrepackagesequ@nc@swap#1{\expandafter\eatone\string}% +} + +% #1 -- parsed sequence without packaging + +\def\yyrepackagesequ@nc@swap#1#{% + \yyrepackagesequ@nc@sw@p{#1}% +} + +% #1 -- parsed `inner' sequence +% #2 -- remainder of the original sequence +% #3 -- `call stack' +% #4 -- parsed sequence so far + +\def\yyrepackagesequ@nc@sw@p#1#2#3#4{% + \yypreparsetokensequenc@{#3}{#2}{#4[#1]}% +} + +% `braced group' thread ends here + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- `parsed' sequence + +\def\yypreparsetokensequ@n@@#1#2#3{% the remaining group in #2 is nonempty and does not start with a brace + \yystartsinspace{#2}{\yyconsumetruespace{#1}{#2}{#3}}{\yypreparsetokenseq@@n@@{#1}{#2}{#3}}% +} + +\def\yyconsumetruespace#1#2#3{% + \expandafter\yyconsumetruespac@swap\expandafter{\eatonespace#2}{#1}{#3.}% +} + +\def\yyconsumetruespac@swap#1#2#3{% + \yypreparsetokensequenc@{#2}{#1}{#3}% +} + +% `group starting with a true (character code 32, category code 10) space' thread ends here + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- `parsed' sequence + +\def\yypreparsetokenseq@@n@@#1#2#3{% a nonempty group, that does not start with a brace or a true space + \yymatchblankspace{#2}{\yyrescanblankspace{#2}{#1}{#3}}{\yypreparsetokens@q@@n@@{#1}{#2}{#3}}% +} + +% #1 -- remaining sequence +% #2 -- `call stack' +% #3 -- `parsed' sequence + +\def\yyrescanblankspace#1#2#3{% + \expandafter\expandafter\expandafter + \yyrescanblankspac@swap + \expandafter\expandafter\expandafter{\expandafter\yynormalizeblankspac@\meaning#1}{#2}{#3*}% +} + +\def\yyrescanblankspac@swap#1#2#3{% + \yystartsinspace{#1}{% + \expandafter\yyrescanblankspac@sw@p\expandafter{\eatonespace#1}{#2}{#3}% + }{% + \expandafter\yyrescanblankspac@sw@p\expandafter{\eatone#1}{#2}{#3}% + }% +} + +\def\yyrescanblankspac@sw@p#1#2#3{% + \yypreparsetokensequenc@{#2}{#1}{#3}% +} + +% `group starting with a blank space' ends here + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- `parsed' sequence + +\def\yypreparsetokens@q@@n@@#1#2#3{% nonempty group starting with a non blank, non brace token + \expandafter\yypreparsetokens@q@@n@@swap\expandafter{\eatone#2}{#1}{#30}% +} + +\def\yypreparsetokens@q@@n@@swap#1#2#3{% + \yypreparsetokensequenc@{#2}{#1}{#3}% +} + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- `parsed' sequence + +\def\yydebracesequence#1#2#3{% + \yybracesleft#3[\end{#1}{#2}% +} + +% #1 -- tokens preceding the brace +% #2 -- tokens following the brace +% #3 -- `call stack' +% #4 -- remaining sequence + +\def\yybracesleft#1[#2\end#3#4{% + \if9#29% + \yybreak{\yyfirstoftwo}% + \else + \yybreak{\yysecondoftwo}% + \yycontinue{#3{#4}}{\yydebracesequenc@swap{#4}{#3}{#1#2}}% +} + +\def\yydebracesequenc@swap#1#2#3{% + \expandafter\expandafter\expandafter + \yydebracesequenc@sw@p + \expandafter\expandafter\expandafter{\yygrabprebrace#1}{#2}{#3}% +} + +\def\yydebracesequenc@sw@p#1#2#3{% + \yydebracesequ@nc@sw@p#3\end{#1}{#2}% +} + +\def\yydebracesequ@nc@sw@p#1[\end#2#3{% + \yydebracesequence{#3}{#2}{#1}% +} +\def\yygrabprebrace#1#{% + \yygrabprebrac@{#1}% +} + +\def\yygrabprebrac@#1#2{#1#2} + +% the `debracing group' ends here + +% string comparison macros below are woefully inefficient and can be replaced by a much +% shorter and faster version; they are easy to analyze, though; + +% #1 -- string of category code 12 or 10 characters +% #2 -- string of category code 12 or 10 characters + +\def\yycomparesimplestrings#1#2{% + \yystringempty{#1}{% + \yystringempty{#2}{\yyfirstoftwo}{\yysecondoftwo}% + }{\yycomparesimplestrings@{#1}{#2}}% +} + +\def\yycomparesimplestrings@#1#2{% the first string is nonempty + \yystringempty{#2}{\yysecondoftwo}{\yycomparesimplestrings@@{#1}{#2}}% +} + +\def\yycomparesimplestrings@@#1#2{% both strings are nonempty + \yystartsinspace{#1}{% + \yystartsinspace{#2}{\yyabsorbfirstspace{#1}{#2}}{\yysecondoftwo}% + }{% + \yystartsinspace{#2}{\yysecondoftwo}{\yyabsorbfirstnonspace{#1}{#2}}% + } +} + +\def\yyabsorbfirstspace#1#2{% + \expandafter\yyabsorbfirstspac@swap\expandafter{\eatonespace#1}{#2}% +} + +\def\yyabsorbfirstspac@swap#1#2{% + \expandafter\yyabsorbfirst@swap\expandafter{\eatonespace#2}{#1}% +} + +\def\yyabsorbfirstnonspace#1#2{% + \expandafter\yyabsorbfirstnonspac@swap\expandafter{\eatone#1}{#2}% +} + +\def\yyabsorbfirstnonspac@swap#1#2{% + \expandafter\yyabsorbfirst@swap\expandafter{\eatone#2}{#1}% +} + +\def\yyabsorbfirst@swap#1#2{% + \yycomparesimplestrings{#2}{#1}% +} + +% `compare strings of category code 12' thread ends here + +% #1 -- a balanced sequence of tokens +% #2 -- a `parsed version' of the same sequence +% +% #1 should be such that \eatone#1 does not lead to an immediate error + +\def\yycomparetailsignatures#1#2{% + \yypreparsetokensequenc@{\yycomparesimplestrings{#2}}{#1}{}% +} + +% #1 -- `call stack' +% #2 -- remaining sequence +% #3 -- remaining parsed sequence +% #4 -- resolved sequence + +\def\yyresolvespaces#1#2#3#4{% + \yystringempty{#2}{#1{#4}}{\yyresolveoneblank#3\end{#2}{#4}{#1}}% +} + +\def\yyresolveoneblank#1{% + \if#1*% + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {\yyresolveonebl@nk}{% + \ifx#1.% + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {\yyresolveonebl@nk}% + {\yyconsumebothnonblanks}% + }% +} + +% #1 -- the remainder of the parsed sequence +% #2 -- remaining sequence +% #3 -- resolved sequence + +\def\yyresolveonebl@nk#1\end#2#3{% + \expandafter\yycomparetailsignatures\expandafter{\eatone#2}{#1}% + {\expandafter\yyresolvespac@sswap\expandafter{\eatone#2}{#1}{#3+}}% + {\expandafter\yyresolveonebl@nklight\expandafter{\eatone#2}{#1}{#3-}}% +} + +% #1 -- remaining sequence +% #2 -- remaining parsed sequence +% #3 -- resolved sequence +% #4 -- `call stack' + +\def\yyresolvespac@sswap#1#2#3#4{% + \yyresolvespaces{#4}{#1}{#2}{#3}% +} + +\def\yyconsumebothnonblanks#1\end#2#3{% + \expandafter\yyconsumebothnonblanksswap\expandafter{\eatone#2}{#1}{#30}% +} + +\def\yyconsumebothnonblanksswap#1#2#3#4{% + \yyresolvespaces{#4}{#1}{#2}{#3}% +} + +% #1 -- remaining sequence +% #2 -- the remainder of the parsed sequence +% #3 -- resolved sequence + +\def\yyresolveonebl@nklight#1#2#3{% + \yyresolveonebl@nklighttest#2\end{#1}{#3}% +} + +\def\yyresolveonebl@nklighttest#1#2\end#3#4{% + \yycomparetailsignatures{#3}{#2}% + {% + \if#1*% + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {\yyresolvespac@sswap{#3}{#2}{#4+}}% + {\yyresolvespac@sswap{#3}{#2}{#40}}% + }% + {\yyresolveonebl@nklight{#3}{#2}{#4-}}% +} + +% `space resolution code' ends here + +% #1 -- remaining parsed sequence +% #2 -- analysed sequence + +\def\yyanalysetokens@#1#2{% + \yystringempty{#1}{{#2}}% + {\yyanalysetok@ns@#1\end{#2}}% +} + +\def\yyanalysetok@ns@#1#2\end{% + \ifx#1.% + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {\yygrabablank{#2}}% + {% + \ifx#1[% not a space, an opening brace + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {% + \yydisableobrace{#2}% + }{% + \ifx#1]% not a space, a closing brace + \expandafter\yyfirstoftwo + \else + \expandafter\yysecondoftwo + \fi + {% + \yydisablecbrace{#2}% + }{% neither space nor brace + \yygrabtokenraw{#2}% + }% + }% + }% +} + +% #1 -- remaining parsed sequence +% #2 -- analysed sequence +% #3 -- next token + +\def\yygrabtokenraw#1#2#3{% + \expandafter\yyanalysetokens@swap\expandafter{\meaning#3}{#1}{#2}% +} + +\def\yyanalysetokens@swap#1#2#3{% + \yyanalysetokens@{#2}{#3t#1e}% +} + +\def\yygrabablank#1#2 {% + \yyanalysetokens@{#1}{#2s0e}% +} + +% #1 -- remaining parsed sequence +% #2 -- analysed sequence + +\def\yydisablecbrace#1#2{% + \yydisablecbrac@{}#1\relax#2\end +} + + +\def\yydisablecbrac@#1#2{% + \ifx#2\end + \yybreak{\yydisablecbrac@@{#1\expandafter\expandafter\expandafter}}% + \else + \yybreak{\yydisablecbrac@{#1\expandafter\expandafter\expandafter#2}}% + \yycontinue +} + +\def\yydisablecbrac@@#1{% + \expandafter\expandafter\expandafter + \yydisablecbrace@@@#1\end + \expandafter\expandafter\expandafter + {\iffalse}\fi\string +} + +\def\yydisablecbrace@@@#1\relax#2\end#3{% + \yystartsinspace{#3}% + {\expandafter\yyanalysetok@nsswap\expandafter{\eatonespace#3}{#1}{#2c1e}}% + {\expandafter\yyanalysetok@nsswap\expandafter{\eatone#3}{#1}{#2c2e}}% +} + +\def\yyanalysetok@nsswap#1#2#3{% + \iffalse{\fi\yyanalysetokens@{#2}{#3}#1}% +} + +% #1 -- remaining parsed sequence +% #2 -- analysed sequence + +\def\yydisableobrace#1#2{% + \yydisableobrac@{}#1\relax#2\end +} + + +\def\yydisableobrac@#1#2{% + \ifx#2\end + \yybreak{\yydisableobrac@@{#1\expandafter\expandafter\expandafter}}% + \else + \yybreak{\yydisableobrac@{#1\expandafter\expandafter\expandafter#2}}% + \yycontinue +} + +\def\yydisableobrac@@#1{% + \expandafter\expandafter\expandafter + \yydisableobrace@@@#1\end + \expandafter\expandafter\expandafter + {\iffalse}\fi\string +} + +\def\yydisableobrace@@@#1\relax#2\end#3{% + \yystartsinspace{#3}% + {\expandafter\yyanalysetok@nsswap\expandafter{\eatonespace#3}{#1}{#2o1e}}% + {\expandafter\yyanalysetok@nsswap\expandafter{\eatone#3}{#1}{#2o2e}}% +} + +% \ifx1\expandafter\eatone\string\11% check if \escapechar is nontrivial + +\uccode`\ =`\- + +\uppercase{\def\dotspace{ }} + +\toksa\expandafter\expandafter\expandafter{\expandafter\meaning\dotspace} + +\toksb{-} + +\toksc{#2} + +\toksd\toksa + +\yyreplacestring\toksb\in\toksa\with\toksc + +\toksc{} +\yyreplacestring\toksb\in\toksd\with\toksc + +\expandafter\def\expandafter\yymatchblankspac@\expandafter#\expandafter1\the\toksd{% + \yystringempty{#1}{\expandafter\yysecondofthree\expandafter{\string}}% + {\expandafter\yythirdofthree\expandafter{\string}}% +} + +\edef\yymatchblankspace#1{% is it \catcode 10 token? + \noexpand\iffalse{\noexpand\fi + \noexpand\expandafter + \noexpand\yymatchblankspac@ + \noexpand\meaning#1\the\toksd}% +} + +% the idea behind the sequence below is that a leading character of category code 10 +% is replaced either by a character of category code 10 and charachter code 32 or a character +% of category code 12 and character code other than 32 +% note that while it is tempting to replace the definition below by something that ends in +% ... blank space #2{ ... with the hope of absorbing the result of \meaning in one step, +% this will not give the desired result in case of an active character, +% say, `~' that had been \let to the normal blank space + +\expandafter\def\expandafter\yynormalizeblankspac@\expandafter#\expandafter1\the\toksd{} + +%% test code begins here + +\tracingmacros=3 +\tracingonline=3 + +\catcode`\ =13\relax% +\def\actspace{ }% +\catcode`\ =10\relax% + +\catcode`\.=13\relax% +\def\actdotspace{.}% +\catcode`\.=12\relax% + +\edef\makefunkydotspace{\let\expandafter\noexpand\actdotspace= \dotspace} +\edef\makefunkyspace{\let\expandafter\noexpand\actspace= \space} + +\makefunkyspace +\makefunkydotspace + +\def\identity#1{#1} + +%\def\tempseq{\space\dotspace\space\space\dotspace\expandafter\noexpand\actspace\expandafter\noexpand\actdotspace +% \end\noexpand\fi\noexpand\else\noexpand\iffalse} +%\def\tempseq{\space\dotspace e} + +%\edef\mypars@{% +% \noexpand\yypreparsetokensequenc@{\noexpand\identity}% +% {\tempseq}% +% {}% +%} + +%\edef\myparse{\mypars@} + +%\toksa\expandafter{\myparse} + +%\showthe\toksa + +%\edef\myresolve{\noexpand\yyresolvespaces{\errmessage}% +% {\tempseq}% +% {\the\toksa}{} +%} + +%\show\myresolve + +%\myresolve + +\catcode`\<=1 +\catcode`\>=2 +\uccode`\<=32 +\uccode`\>=32 + +\uppercase{\edef\temptest{<> \space\space\dotspace\expandafter\noexpand\actspace\expandafter\noexpand\actdotspace{{} {{}{{ u o l k kk + \end\noexpand\fi\noexpand\else\noexpand\iffalse{}} }}}}} + +%\uppercase{\edef\temptest{\dotspace E <>}} + +\show\temptest + +\def\displaypreparse#1{% + \expandafter\errmessage\expandafter{\romannumeral-1\yypreparsetokensequenc@{\yyanalysetokens@}{#1}{}{}#1}% +} + +%\def\displaydebraceparse#1{% +% \yypreparsetokensequenc@{\yydebracesequence{\displayseq}{#1}}{#1}{}% +%} + +%\def\displayseq#1{\toksa{#1}\showthe\toksa} + +\expandafter\displaypreparse\expandafter{\temptest} + +\errmessage{stop!} + +%\expandafter\displaydebraceparse\expandafter{\temptest} + +\end diff --git a/support/splint/examples/expression/Makefile b/support/splint/examples/expression/Makefile new file mode 100644 index 0000000000..0e2f33c2ba --- /dev/null +++ b/support/splint/examples/expression/Makefile @@ -0,0 +1,59 @@ +# this Makefile uses a flat directory structure for demonstration +# purposes; the main directory (../..) contains a slightly more +# modular organization. + +SPLINT_ROOT = $(shell pwd)/../.. + +include ${SPLINT_ROOT}/makefile.inc +include ${SPLINT_ROOT}/makefile.loc + +# rules specific to this example + +PARSER = expp +LEXER = expl + +all: expression.pdf ptab.tex ltab.tex + +step1: expp.y expl.l +step2: ptabout ltabout +step3: ltab.tex ptab.tex + +ptabout: ${SPLINT_ROOT}/cweb/mkeparser.c ${PARSER}.c + ${CC} -DPARSER_FILE=\"../examples/expression/$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +ptab.tex: ptabout + $< --optimize-actions $@ + +ltabout: ${SPLINT_ROOT}/cweb/mkscanner.c ${LEXER}.c + ${CC} -DLEXER_FILE=\"../examples/expression/$(lastword $^)\" -o $@ $< + +ltab.tex: ltabout + $< --optimize-actions $@ + +${LEXER}.c: ${LEXER}.l + ${FLEX} -o $@ $< + +test.tex ${PARSER}.yy ${LEXER}.ll: expression.x + @${CTANGLE} $< + ${UNLINE} test.txx test.tex + +expression.tex: expression.x + @${CWEAVE} $< + +expression.pdf: expression.tex ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} ${SPLINT_ROOT}/cweb/bo.tok etoks.sty + @${PDFTEX} $< + +test: test.tex ptab.tex ltab.tex + tex test.tex + +docs: expression.pdf + +mostlyclean: clean_temp + -rm -f ptabout ltabout expression.c *.txx + +clean: clean_core + -rm -f ptabout ltabout *.txx + +distclean: clean + cd ${SPLINT_ROOT} && ${MAKE} clean + diff --git a/support/splint/examples/expression/etoks.sty b/support/splint/examples/expression/etoks.sty new file mode 100644 index 0000000000..7326245338 --- /dev/null +++ b/support/splint/examples/expression/etoks.sty @@ -0,0 +1,3 @@ +\prettywordpair{IDENTIFIER}{{$\langle$\rm identifier$\rangle$}} +\prettywordpair{INTEGER}{{\bf int}} +\prettywordpair{BOGUS}{{\tt Oh, @\%\$\&*!}} diff --git a/support/splint/examples/expression/expression.sty b/support/splint/examples/expression/expression.sty new file mode 100644 index 0000000000..9f9195aafb --- /dev/null +++ b/support/splint/examples/expression/expression.sty @@ -0,0 +1,25 @@ +\def\optimization{5} + +\input yycommon.sty % general routines for stack and array access +\input yymisc.sty % helper macros (stack manipulation, table processing, value stack pointers) +\input yyinput.sty % input functions +\input yyparse.sty % parser machinery +\input flex.sty % lexer functions +\input yyfaststack.sty % sped up stack access functions +\input yyboth.sty % parser initializatio, optimization + +\let\yylexreturn\yylexreturnregular +\let\setflexstates\relax +\let\parsernamespace\empty + +\genericparser + name: main, + ptables: ptab.tex, + ltables: ltab.tex, + tokens: {}, + asetup: {}, + dsetup: {}, + rsetup: {}, + optimization: \optimizeall;% + +\tomainparser diff --git a/support/splint/examples/expression/expression.w b/support/splint/examples/expression/expression.w new file mode 100644 index 0000000000..c775741184 --- /dev/null +++ b/support/splint/examples/expression/expression.w @@ -0,0 +1,262 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> +\input limbo.sty +\def\optimization{5} +\input yy.sty + +@** Parser file. This an example parser for expressions. It takes +advantage of some of the features of \splint\ generated parsers, +although anything that takes more than a straightforward setup is +omitted. + +The top-level structure of the input file presents no surprises and is +presented below. +\let\currentparsernamespace\parsernamespace + \let\parsernamespace\smallnamespace + \let\hostparsernamespace\smallnamespace + \input etoks.sty +\let\parsernamespace\currentparsernamespace +\def\texnspace{[other]} +@s TeX_ TeX + +@(expp.yy@>= +@G Switch to generic mode. +%{@> @<\.{expression} parser \Cee\ preamble@> @=%} + @> @<Bison options@> @= +%union {@> @<Union of parser types@> @=} +%{@> @<\.{expression} parser \Cee\ postamble@> @=%} + @> @<Token and types ...@> @= +%% + @> @<Parser productions@> @= +%% +@g + +@ The \prodstyle{\%token-table} option is not merely a debugging help, +as it is in the case of the `real' \bison\ parsers and cannot be +omitted . The name table it is responsible for setting up is used as +a set of keys for various associative arrays. Token declarations are +parsed by a bootstrap parser during the \TeX\ processing stage to +establish equivalences between the names kept in |yytname| and the +macro names used internally by the parsers built by \bison. The reason +this is necessary is not very complicated: either version of the token +name can be used in the grammar while the `driver' program +(\.{mkeparser.c}) only has access to the names in |yytname|. In +general, this is important whenever the grammar uses a different set of +token names from the lexer or when diagnostics messages are output. An +important case is the symbolic name switch: before the rules can be +listed to create the switch, the token numerical values must be +known. If the parser is only aware of the |yytname| listed names and the +grammar being parsed uses the `internal' names, the listing macros +will fail. The array, |yytname| is used in a few functions inside the +`driver', as well, so omitting this option would make building the +parser impossible. +@<Bison options@>= +@G +%token-table +%debug +%start value +@g + +@ To continue the token name discussion, this parser uses internal +names only so this is what will appear in the |yytname| array. No +bootstrapping is necessary. The typesetting of the tokens can be +adjusted using \.{\\prettywordpair} macros (see the included +\.{etoks.sty} file for examples). +@<Token and types declarations@>= +@G +%token IDENTIFIER +%token INTEGER +%token BOGUS +@g + +@ Here is the whole grammar, simply additive expressions with two +levels of precedence. +@<Parser productions@>= +@G +value: + expression {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +; + +expression: + term {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| expression '+' term {@> @<Add a term@> @=} +; + +term: + atom {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| term '*' atom {@> @<Multiply by an atom@> @=} +; + +atom: + IDENTIFIER {@> @<Assign variable value to an atom@> @=} +| INTEGER {@> @<Assign value to an atom@> @=} +| '(' expression ')' {@> TeX_( "/yy0{/the/yy(2)}" ); @=} +; +@g + +@ @<Add a term@>= + @[TeX_( "/tempca/the/yy(1)/relax" );@]@; + @[TeX_( "/tempcb/the/yy(3)/relax" );@]@; + @[TeX_( "/advance/tempca by /tempcb" );@]@; + @[TeX_( "/yy0{/the/tempca}" );@]@; + +@ @<Multiply by an atom@>= + @[TeX_( "/tempca/the/yy(1)/relax" );@]@; + @[TeX_( "/tempcb/the/yy(3)/relax" );@]@; + @[TeX_( "/multiply/tempca by /tempcb" );@]@; + @[TeX_( "/yy0{/the/tempca}" );@]@; + +@ @<Assign variable value to an atom@>= + @[TeX_( "/getsecond{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/toksb/expandafter/expandafter/expandafter{/expandafter" );@]@; + @[TeX_( " /number/csname/the/toksa/endcsname}" );@]@; + @[TeX_( "/yy0{/the/toksb}" );@]@; + +@ @<Assign value to an atom@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/yy0{/the/toksa}" );@]@; + +@ \Cee\ preamble. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<\.{expression} parser \Cee\ preamble@>= + +@ \Cee\ postamble. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<\.{expression} parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) @[yyprint (file, type, value)@] + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ Union of types. Empty as well. + +@<Union of parser types@>= + +@** The lexer file. The scanner for the grammar above is even +simpler. Identifiers are interpreted as variable names that expand to +appropriate values. +%\checktabletrue +@(expl.ll@>= +@G + @> @<Lexer definitions@> @= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% + +@ @<Lexer definitions@>= +@G +letter [_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] +id {letter}({letter}|[-0-9])* +int [0-9]+ +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + + void define_all_states( void ){} + +@ @<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="expl.c" +@g + +@ @<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ @<Scan identifiers@>= +@G +{id} {@> @[TeX_( "/yylexreturnval{IDENTIFIER}" );@]@=} +{int} {@> @[TeX_( "/yylexreturnval{INTEGER}" );@]@=} +[+*()] {@> @[TeX_( "/yylexreturnchar" );@]@=} +. {@> @[@<React to a bad character@>@]@=} +@g + +@ @<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; + +@*Test file. The test file includes a handy list of debugging options +that can be activated to see the inner workings of the parser and +scanner routines. +@(test.txx@>= +@G +\input expression.sty + +\iffalse + \tracedfatrue + \traceparserstatestrue + \tracestackstrue + \tracerulestrue + \traceactionstrue + \tracelookaheadtrue + \traceparseresultstrue + \tracebadcharstrue + \yyflexdebugtrue + \yyinputdebugtrue +\fi + +\def\varone{10} +\def\expression{1 + 3 * ( 5 + 7 ) + varone} +\basicparserinit\expandafter\yyparse \expression \yyeof\yyeof\endparseinput\endparse + +{ + \newlinechar`^^J + \immediate\write16{^^Jexpression: \expression^^Jthe value: \the\yyval^^J^^J} +} + +\bye +@g + +@**Index. +\immediate\closeout\exampletable +\def\Tex{\TeX\ output} +\def\TeXx{\TeX\ output} diff --git a/support/splint/examples/ld/Makefile b/support/splint/examples/ld/Makefile new file mode 100644 index 0000000000..a23f191b9c --- /dev/null +++ b/support/splint/examples/ld/Makefile @@ -0,0 +1,118 @@ +# this Makefile uses a flat directory structure for demonstration +# purposes; the main directory (../..) contains a slightly more +# modular organization. + +SPLINT_ROOT = $(shell pwd)/../.. + +PARSER = ldp +LEXER = ldl +MANUAL = ldman + +LD_CORE_PREREQS = ${SPLINT_XTEXSTYLES} ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} ${SPLINT_ROOT}/cweb/bo.tok +LD_DOC_PREREQS = %.tex %.sty ${PARSER}.tok ldunion.sty ldint.sty ldfrontmatter.sty ${LD_CORE_PREREQS} ptab.tex ltab.tex ld_small_tab.tex +LD_DOC_PREREQS += ld_small_dfa.tex ltokenset.sty lstokenset.sty +LD_DOC_PREREQS_XREF = ${LD_DOC_PREREQS} %.scn %.idx + +include ${SPLINT_ROOT}/makefile.inc +include ${SPLINT_ROOT}/makefile.loc + +# rules specific to this example + +ptabout: ${SPLINT_ROOT}/cweb/mkeparser.c ${PARSER}.c + ${CC} -DPARSER_FILE=\"../examples/ld/$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +ptab.tex: ptabout + $< --optimize-actions --optimize-tables $@ + +ltabout: ${SPLINT_ROOT}/cweb/mkscanner.c ldl_states.h ${LEXER}.c + ${CC} -DLEXER_FILE=\"../examples/ld/$(lastword $^)\" -o $@ $< + +ltab.tex: ltabout + $< --optimize-actions --optimize-tables $@ + +${LEXER}.c: ${LEXER}.l + ${FLEX} -o $@ $< + +${PARSER}.yy : ldgram.x + @${CTANGLE} $< + +${LEXER}.ll: ldlex.x + @${CTANGLE} $< + +${MANUAL}.stx: ${MANUAL}.x ldgram.x ldlex.x ldlexo.x ldgramo.x ldnp.x + @${CTANGLE} $< + +# term name parser for ld grammar + +ldsmallp_out: ${SPLINT_ROOT}/cweb/mkeparser.c ld_small_parser.c + ${CC} ${BISON_STATE} -DPARSER_FILE=\"../examples/ld/$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +ldsmalll_out: ${SPLINT_ROOT}/cweb/mkscanner.c ld_small_lexer.c + ${CC} -DLEXER_FILE=\"../examples/ld/$(lastword $^)\" -o $@ $< + +ld_small_tab.tex: ldsmallp_out + $< --optimize-actions --optimize-tables $@ + +ld_small_dfa.tex: ldsmalll_out + $< --optimize-actions --optimize-tables $@ + +ld_small_parser.yy \ +ld_small_lexer.ll: ldnp.x + @${CTANGLE} $< + +# numeric parser for ld grammar + +ldnump_out: ${SPLINT_ROOT}/cweb/mkeparser.c ld_num_parser.c + ${CC} ${BISON_STATE} -DPARSER_FILE=\"../examples/ld/$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +ldnuml_out: ${SPLINT_ROOT}/cweb/mkscanner.c ld_num_lexer.c + ${CC} -DLEXER_FILE=\"../examples/ld/$(lastword $^)\" -o $@ $< + +ld_num_tab.tex: ldnump_out + $< --optimize-actions --optimize-tables $@ + +ld_num_dfa.tex: ldnuml_out + $< --optimize-actions --optimize-tables $@ + +ld_num_parser.yy \ +ld_num_lexer.ll: ldnump.x + @${CTANGLE} $< + +${MANUAL}.tex: ${MANUAL}.x ldgram.x ldlex.x ldlexo.x ldgramo.x ldnp.x + ${CWEAVE} $< + +${MANUAL}.sty: ${MANUAL}.stx + ${UNLINE} $< $@ + +${MANUAL}.gdx: %.gdx: ${LD_DOC_PREREQS_XREF} ${PARSER}.tok + @echo "Making the bison and TeX indices ..." + ${TEX} $*.tex + +${MANUAL}.pdf: %.pdf: ${LD_DOC_PREREQS_XREF} %.gdy + ${PDFTEX} $< && touch $*.gdy && touch $*.pdf + +#${PARSER}.tok \ +#ldl_states.h: ${MANUAL}.tex ${MANUAL}.sty ${LD_CORE_PREREQS} +# ${TEX} ${MODEBOOTSTRAP} \\input $< + +${PARSER}.tok: ldgram.tex ${MANUAL}.sty ${LD_CORE_PREREQS} + ${TEX} ${MODEBOOTSTRAP} \\input $< + +ldl_states.h: ldlex.tex ${MANUAL}.sty ${LD_CORE_PREREQS} + ${TEX} ${MODEBOOTSTRAP} \\input $< + +ldgram.tex ldlex.tex: \ +%.tex: %.x + ${CWEAVE} -x $< + +docs: ${MANUAL}.pdf + +mostlyclean: clean_temp + -rm -f ptabout ltabout ldsmalll_out ldsmallp_out ldnump_out ldnuml_out ${MANUAL}.c + +clean: clean_core + -rm -f ptabout ltabout ldsmalll_out ldsmallp_out ldnump_out ldnuml_out *.stx ldman.sty + +distclean: clean + cd ${SPLINT_ROOT} && ${MAKE} clean + diff --git a/support/splint/examples/ld/ldfrontmatter.sty b/support/splint/examples/ld/ldfrontmatter.sty new file mode 100644 index 0000000000..e4fd94bcf2 --- /dev/null +++ b/support/splint/examples/ld/ldfrontmatter.sty @@ -0,0 +1,2 @@ +% this file is left blank on purpose to avoid referencing non-free fonts +% it would contain the \TeX\ macros that generate the cover page for \ld\ parser manual diff --git a/support/splint/examples/ld/ldgram.w b/support/splint/examples/ld/ldgram.w new file mode 100644 index 0000000000..4fb9a9a876 --- /dev/null +++ b/support/splint/examples/ld/ldgram.w @@ -0,0 +1,1083 @@ +@q Copyright 2012-2015 Alexander Shibakov@> +@q Copyright 2002-2014 Free Software Foundation, Inc.@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@** The parser. +\ifx\parsernamespace\UNDEFINED + \def\tokendeffile{ldp.tok}% + \input ldman.sty + \input limbo.sty + \input dcols.sty + \setupfootnotes + \def\MRI{} + \def\ld{} +\fi +\immediate\openout\exampletable=\jobname.exl\relax% +The outline of the grammar file below does not reveal anything unusual +in the general layout of \ld\ grammar. The first section lists all the +token definitions, \prodstyle{\%union} styles, and some \Cee\ +code. The original comments that come with the grammar file of the +linker have been mostly left intact. They are typeset in {\it +italics\/} to make them easy to recognize. + +@s TeX_ TeX +@s TeXa TeX +@s TeXb TeX +@s TeXf TeX +@s TeXfo TeX +@s TeXao TeX + +@(ldp.yy@>= +@G Switch to generic mode. +%{@> @<\ld\ parser \Cee\ preamble@> @=%} + @> @<\ld\ parser \bison\ options@> @= +%union {@> @<Union of grammar parser types@> @=} +%{@> @<\ld\ parser \Cee\ postamble@> @=%} + @> @<Token and type declarations@> @= +%% + @> @<\ld\ parser productions@> @= +%% +@g + +@ Among the options listed in this section, \prodstyle{\%token-table} +is the most critical for the proper operation of the parser and must be enabled +to supply the token information to the lexer (the traditional way +of passing this information along is to use a \Cee\ header file with +the appropriate definitions). The start symbol does not have to be +given explicitly and can be indicated by listing the appropriate rules +at the beginning. + +Most other sections of the grammar file, with the exception of the +rules are either empty or hold placeholder values. The functionality +provided by the code in these sections in the case of a \Cee\ parser +is supplied by the \TeX\ macros in \.{ldman.sty}. +@<\ld\ parser \bison\ options@>= +@G +%token-table +%debug +%start script_file +@g + +@ @<\ld\ parser \Cee\ preamble@>= + +@ @<Union of grammar parser types@>= + +@ @<\ld\ parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) yyprint (file, type, value) + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ @<\ld\ parser productions@>= + @<\GNU\ \ld\ script rules@>@; + @<Grammar rules@>@; + +@ The tokens are declared first. This section is also used to supply +numerical token values to the lexer by the original parser, as well as +the bootstrapping phase of the typesetting parser. Unlike the native (\Cee) parser for +\ld\ the typesetting parser has no need for the type of each token +(rather, the type consistency is based on the weak dynamic type system +coded in \.{yyunion.sty} and \.{ldunion.sy}). Thus all the tokens used +by the \ld\ parser are put in a single list. +@<Token and type declarations@>= +@G +%token INT +%token NAME LNAME +%token END +%token ALIGN_K BLOCK BIND QUAD SQUAD LONG SHORT BYTE +%token SECTIONS PHDRS INSERT_K AFTER BEFORE +%token DATA_SEGMENT_ALIGN DATA_SEGMENT_RELRO_END DATA_SEGMENT_END +%token SORT_BY_NAME SORT_BY_ALIGNMENT SORT_NONE +%token SORT_BY_INIT_PRIORITY +%token '{' '}' +%token SIZEOF_HEADERS OUTPUT_FORMAT FORCE_COMMON_ALLOCATION OUTPUT_ARCH +%token INHIBIT_COMMON_ALLOCATION +%token SEGMENT_START +%token INCLUDE +%token MEMORY +%token REGION_ALIAS +%token LD_FEATURE +%token NOLOAD DSECT COPY INFO OVERLAY +%token DEFINED TARGET_K SEARCH_DIR MAP ENTRY +%token NEXT +%token SIZEOF ALIGNOF ADDR LOADADDR MAX_K MIN_K +%token STARTUP HLL SYSLIB FLOAT NOFLOAT NOCROSSREFS +%token ORIGIN FILL +%token LENGTH CREATE_OBJECT_SYMBOLS INPUT GROUP OUTPUT CONSTRUCTORS +%token ALIGNMOD AT SUBALIGN HIDDEN PROVIDE PROVIDE_HIDDEN AS_NEEDED +%token CHIP LIST SECT ABSOLUTE LOAD NEWLINE ENDWORD ORDER NAMEWORD ASSERT_K +%token LOG2CEIL FORMAT PUBLIC DEFSYMEND BASE ALIAS TRUNCATE REL +%token INPUT_SCRIPT INPUT_MRI_SCRIPT INPUT_DEFSYM CASE EXTERN START +%token VERS_TAG VERS_IDENTIFIER +%token GLOBAL LOCAL VERSIONK INPUT_VERSION_SCRIPT +%token KEEP ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS ALIGN_WITH_INPUT +%token EXCLUDE_FILE +%token CONSTANT +%token INPUT_DYNAMIC_LIST +%right PLUSEQ MINUSEQ MULTEQ DIVEQ '=' LSHIFTEQ RSHIFTEQ ANDEQ OREQ '?' ':' UNARY +%left OROR ANDAND '|' '^' '&' EQ NE '<' '>' LE GE LSHIFT RSHIFT '+' '-' '*' '/' '%' '(' +@g + +@*1 Grammar rules, an overview. +The first natural step in transforming an existing parser into a +`parser stack' for pretty printing is to understand the `anatomy' of +the grammar. Not every grammar is suitable for such a transformation +and in almost every case, some modifications are needed. The +parser and lexer implementation for \ld\ is not terrible although it does +have some idiosynchasies that could have been eliminated by a careful +grammar redesign. Instead of invasive rewriting of significant +portions of the grammar, the approach taken here merely omits some +rules and partitions the grammar into several subsets, each of which +is supposed to handle a well defined logical section of an \ld\ script +file. + +One example of a trick used by the \ld\ parser that is not appropriate for a +pretty printing grammar implements a way of handling the choice of the +format of an input file. After a command line option that selects the +input format has been read (or the format has been determined using +some other method), the first token output by the lexer branches the +parser to the appropriate portion of the full grammar. + +Since the token never appears as part of the input file there is no +need to include this part of the main grammar for the purposes of +typesetting. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +%\checktabletrue +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +\checktablefalse +% +\saveparseoutputfalse +@<Ignored grammar rules@>= +@G +file: + INPUT_SCRIPT script_file + | INPUT_MRI_SCRIPT mri_script_file + | INPUT_VERSION_SCRIPT version_script_file + | INPUT_DYNAMIC_LIST dynamic_list_file + | INPUT_DEFSYM defsym_expr +; +@g + +@ @<Grammar rules@>= +@G +@t}\vb{\inline\flatten}{@> +filename: + NAME {@>@[TeX_( "/yy0{/noexpand/ldfilename{/the/yy(1)}}" );@]@=} +; +@g + +@ The simplest parser subset is intended to parse symbol definitions +given in the command line that invokes the linker. Creating a parser +for it involves almost no extra effort so we leave it in. + +Note that the simpliciy is somewhat deceptive as the syntax of +\prodstyle{exp} is rather complex. That part of the grammar is needed +elsewhere, however, so symbol definitions parsing costs almost nothing +on top of the already required effort. The only practical use for this +part of the \ld\ grammar is presenting examples in text. + +The\namedspot{pingpong} \TeX\ macro \.{\\ldlex@@defsym} switches the lexer state to +\.{DEFSYMEXP} (see \locallink{stateswitchers}all the state switching +macros\endlink\ in the chapter about the lexer implementation +below). Switching lexer states from the parser presents some +difficulties which can be overcome by careful design. For example, the +state switching macros can be invoked before the lexer is called and +initialized (when the parser performs a {\it default action\/}). +@<Inline symbol definitions@>= +@G +defsym_expr: + {@>@[TeX_( "/ldlex@@defsym" );@]@=} + NAME '=' exp {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; +@g + +@ {\it Syntax within an \MRI\ script file}\footnote{As explained at the +beginning of this chapter, the text in {\it italics\/} was taken from +the original comments by \ld\ parser and lexer programmers.}. The parser for typesetting +is only intended to process \GNU\ \ld\ scripts and does not concern +itself with any additional compatibility modes. For this reason, all +support for \MRI\ style scripts has been omitted. One use for the +section below is a small demonstration of the formatting tools that +change the output of the \bison\ parser. +@<\MRI\ style script rules@>= +@G +@t}\vb{\inline\flatten}{@> +mri_script_file: + {@>@[TeX_( "/ldlex@@mri@@script" );@]@=} + mri_script_lines {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; + +mri_script_lines: + mri_script_lines mri_script_command NEWLINE + | + ; +@t}\vb{\resetf}{@> +mri_script_command: + CHIP exp + | CHIP exp ',' exp + | NAME {} + | LIST {} + | ORDER ordernamelist {} + | ENDWORD {} +@t}\vb{\flatten}{@> + | PUBLIC NAME '=' exp {} + | PUBLIC NAME ',' exp {} + | PUBLIC NAME exp +@t}\vb{\resetf}{@> + {} + | FORMAT NAME {} +@t}\vb{\flatten}{@> + | SECT NAME ',' exp {} + | SECT NAME exp {} + | SECT NAME '=' exp +@t}\vb{\resetf}{@> + {} +@t}\vb{\flatten}{@> + | ALIGN_K NAME '=' exp {} + | ALIGN_K NAME ',' exp +@t}\vb{\resetf}{@> + {} +@t}\vb{\flatten}{@> + | ALIGNMOD NAME '=' exp {} + | ALIGNMOD NAME ',' exp {} +@t}\vb{\resetf}{@> + {} + | ABSOLUTE mri_abs_name_list + | LOAD mri_load_name_list + | NAMEWORD NAME {} +@t}\vb{\flatten}{@> + | ALIAS NAME ',' NAME {} + | ALIAS NAME ',' INT {} +@t}\vb{\resetf}{@> + {} + | BASE exp {} + | TRUNCATE INT {} + | CASE casesymlist + | EXTERN extern_name_list +@t}\vb{\flatten}{@> + | INCLUDE filename {@>@<Peek at a file@>@=} + mri_script_lines END +@t}\vb{\resetf}{@> + {@>@<Close the file@>@=} + | START NAME {} + | + ; +@t}\vb{\inline\flatten}{@> +ordernamelist: + ordernamelist ',' NAME {} + | ordernamelist NAME {} + | + ; + +mri_load_name_list: + NAME {} + | mri_load_name_list ',' NAME {} + ; + +mri_abs_name_list: + NAME {} + | mri_abs_name_list ',' NAME {} + ; + +casesymlist: + {} + | NAME + | casesymlist ',' NAME + ; +@g + +@ {\it Parsed as expressions so that commas separate entries.} The +core of the parser consists of productions describing \GNU\ \ld\ linker +scripts. The first rule is common to both \MRI\ and \GNU\ formats. +@<\GNU\ \ld\ script rules@>= +@G +extern_name_list: + {@>@[TeX_( "/ldlex@@expression" );@]@=} + extern_name_list_body {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; + +extern_name_list_body: + NAME {} + | extern_name_list_body NAME {} + | extern_name_list_body ',' NAME {} + ; + +@ The top level productions simply define a script file as a list of +script commands. +@<\GNU\ \ld\ script rules@>= +@G +script_file: + {@>@[TeX_( "/ldlex@@both" );@]@=} + ifile_list {@>@[TeX_( "/getsecond{/yy(2)}/to/ldcmds/ldlex@@popstate" );@]@=} + ; + +ifile_list: + ifile_list ifile_p1 {@>@<Add the next command@>@=} + | {@>@[TeX_( "/yy0{{}{}}" );@]@=} + ; +@g + +@ @<Add the next command@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/getfirst{/yy(2)}/to/toksc/getsecond{/yy(2)}/to/toksd" );@]@; + @[TeXb( "/yytoksempty{/toksb}{/yy0{/the/yy(2)}}" );@]@; + @[TeXao( "{/yy0{{/the/toksc}{/the/toksb/noexpand/ldcommandseparator{/the/toksa}{/the/toksc}/the/toksd}}}" );@]@; + +@*1 Script internals. +There are a number of different commands. For typesetting purposes, +the handling of most of these can be significantly +simplified. In the \prodstyle{GROUP} command there is no need to +perform any actions upon entering the group, for +instance. \prodstyle{INCLUDE} presents a special challenge. In +the original grammar this command is followed by a general list of script +commands (the contents of the included file) terminated by +\prodstyle{END}. The `magic' of opening the file and inserting its +contents into the stream being parsed is performed by the lexer and +the parser in the +background. The typesetting parser, on the other hand, only has to +typeset the \prodstyle{INCLUDE} command itself and has no need for +opening and parsing the file being included. We can simply change the +grammar rule to omit the follow up script commands but that would +require altering the existing grammar. \namedspot{pretendbuffersw}Since the command list +(\prodstyle{ifile\_list}) is allowed to be empty, +we simply \locallink{pretendbufferswlex}fake\endlink\ the +inclusion of the file in the lexer by immediately outputting +\prodstyle{END} upon entering the appropriate lexer state. One +advantage in using this approach is the ability, when desired, to +examine the included file for possible cross-referencing information. + +Each command is packaged with a qualifier that records its type for +the rule that adds the fragment to the script file. +@<\GNU\ \ld\ script rules@>= +@G +ifile_p1: + memory {@>@[TeX_( "/yy0{{mem}{/the/yy(1)}}" );@]@=} + | sections {@>@[TeX_( "/yy0{{sect}{/the/yy(1)}}" );@]@=} + | phdrs + | startup + | high_level_library + | low_level_library + | floating_point_support + | statement_anywhere {@>@[TeX_( "/yy0{{stmt}{/noexpand/ldstatement{/the/yy(1)}}}" );@]@=} + | version + | ';' {@>@[TeX_( "/yy0{{none}{}}" );@]@=} + | TARGET_K '(' NAME ')' {} + | SEARCH_DIR '(' filename ')' {} + | OUTPUT '(' filename ')' {} + | OUTPUT_FORMAT '(' NAME ')' {} + | OUTPUT_FORMAT '(' NAME ',' + NAME ',' NAME ')' {} + | OUTPUT_ARCH '(' NAME ')' {} + | FORCE_COMMON_ALLOCATION {} + | INHIBIT_COMMON_ALLOCATION {} + | INPUT '(' input_list ')' + | GROUP {} + '(' input_list ')' {} + | MAP '(' filename ')' {} + | INCLUDE filename {@>@<Peek at a file@>@=} + ifile_list END {@>@<Add an \prodstyle{INCLUDE} statement@>@=} + | NOCROSSREFS '(' + nocrossref_list ')' {} + | EXTERN '(' extern_name_list ')' + | INSERT_K AFTER NAME {} + | INSERT_K BEFORE NAME {} + | REGION_ALIAS '(' NAME ',' NAME ')' {} + | LD_FEATURE '(' NAME ')' {} + ; + +input_list: + NAME {} + | input_list ',' NAME {} + | input_list NAME {} + | LNAME {} + | input_list ',' LNAME {} + | input_list LNAME {} + | AS_NEEDED '(' {} + input_list ')' {} + | input_list ',' AS_NEEDED '(' {} + input_list ')' {} + | input_list AS_NEEDED '(' {} + input_list ')' {} + ; + +sections: + SECTIONS '{' sec_or_group_p1 '}' {@>@[TeX_( "/yy0{/noexpand/ldsections{/the/yy(3)}}" );@]@=} + ; + +sec_or_group_p1: + sec_or_group_p1 section {@>@<Add the next section@>@=} + | sec_or_group_p1 statement_anywhere {@>@<Add the next statement@>@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +statement_anywhere: + ENTRY '(' NAME ')' {} + | assignment end {@>@<Carry on@>@=} + | ASSERT_K {@>@[TeX_( "/ldlex@@expression" );@]@=} + '(' exp ',' NAME ')' {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; +@g + +@ @<Add the next section@>= + @[TeXb( "/yytoksempty{/yy(1)}{/yy0{/the/yy(2)}}" );@]@; + @[TeXao( "{/yy0{/the/yy(1)/noexpand/ldsectionseparator/the/yy(2)}}" );@]@; + +@ @<Add the next statement@>= + @[TeXb( "/yytoksempty{/yy(1)}{/yy0{/the/yy(2)}}" );@]@; + @[TeXao( "{/yy0{/the/yy(1)/noexpand/ldsectionseparator/ldstatement{/the/yy(2)}}}" );@]@; + +@ This is the default action performed by the parser when the parser +writer does not supply one. For a minor gain in efficiency, this +definition can be made empty. +@<Carry on@>= + @[TeX_( "/yy0{/the/yy(1)}" );@]@; + +@ @<Peek at a file@>= + @[TeX_( "/ldlex@@script" );@]@; + @[TeX_( "/ldfile@@open@@command@@file{/yy(2)}" );@]@; + +@ @<Close the file@>= + @[TeX_( "/yy0{/noexpand/ldinclude{/the/yy(2)}}/ldlex@@popstate" );@]@; + +@ @<Add an \prodstyle{INCLUDE} statement@>= + @[TeX_( "/yy0{{inc}{/noexpand/ldinclude{/the/yy(2)}}}/ldlex@@popstate" );@]@; + +@ \tracebadcharstrue +{\it The \prodstyle{'*'} and \prodstyle{'?'} cases are there because the lexer returns them as +separate tokens rather than as \prodstyle{NAME}.} +\tracebadcharsfalse +@<Grammar rules@>= +@G +wildcard_name: + NAME {@>@<Carry on@>@=} + | '*' {@>@[TeX_( "/yy0{{*}{*}}" );@]@=} + | '?' {@>@[TeX_( "/yy0{{?}{?}}" );@]@=} + ; +@g + +@ @<Grammar rules@>= +@G +wildcard_spec: + wildcard_name {} + | EXCLUDE_FILE '(' exclude_name_list ')' wildcard_name + {} + | SORT_BY_NAME '(' wildcard_name ')' {} + | SORT_BY_ALIGNMENT '(' wildcard_name ')' + {} + | SORT_NONE '(' wildcard_name ')' + {} + | SORT_BY_NAME '(' SORT_BY_ALIGNMENT '(' wildcard_name ')' ')' + {} + | SORT_BY_NAME '(' SORT_BY_NAME '(' wildcard_name ')' ')' + {} + | SORT_BY_ALIGNMENT '(' SORT_BY_NAME '(' wildcard_name ')' ')' + {} + | SORT_BY_ALIGNMENT +@t}\vb{\breakline}{@> + '(' SORT_BY_ALIGNMENT '(' wildcard_name ')' ')' + {} + | SORT_BY_NAME +@t}\vb{\breakline}{@> + '(' EXCLUDE_FILE '(' exclude_name_list ')' wildcard_name ')' + {} + | SORT_BY_INIT_PRIORITY '(' wildcard_name ')' + {} + ; + +sect_flag_list: + NAME {} + | sect_flag_list '&' NAME {} + ; + +sect_flags: + INPUT_SECTION_FLAGS '(' sect_flag_list ')' {} + ; + +exclude_name_list: + exclude_name_list wildcard_name {} + | wildcard_name {} + ; + +file_NAME_list: + file_NAME_list opt_comma wildcard_spec {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/ldspace/noexpand/ldregexp{/the/yy(2)}}" );@]@=} + | wildcard_spec {@>@[TeX_( "/yy0{/noexpand/ldregexp{/the/yy(1)}}" );@]@=} + ; + +input_section_spec_no_keep: + NAME {} + | sect_flags NAME {} + | '[' file_NAME_list ']' {} + | sect_flags '[' file_NAME_list ']' {} + | wildcard_spec '(' file_NAME_list ')' {@>@[TeX_( "/yy0{/noexpand/ldregexp{/the/yy(1)}(/the/yy(3))}" );@]@=} + | sect_flags wildcard_spec '(' file_NAME_list ')' + {} + ; +@g + +@ @<Grammar rules@>= +@G +input_section_spec: + input_section_spec_no_keep {@>@<Carry on@>@=} + | KEEP '(' {} + input_section_spec_no_keep ')' {@>@[TeX_( "/yy0{/mathop{/hbox{/noexpand/ttl keep}(/the/yy(4))}}" );@]@=} + ; + +statement: + assignment end + | CREATE_OBJECT_SYMBOLS {} + | ';' {@>@[TeX_( "/yy0{}" );@]@=} + | CONSTRUCTORS {} + | SORT_BY_NAME '(' CONSTRUCTORS ')' {} + | input_section_spec + | length '(' mustbe_exp ')' {} + | FILL '(' fill_exp ')' {} + | ASSERT_K {@>@[TeX_( "/ldlex@@expression" );@]@=} + '(' exp ',' NAME ')' end {@>@[TeX_( "/ldlex@@popstate" );@]@=} + | INCLUDE filename {@>@<Peek at a file@>@=} + statement_list_opt END {@>@<Close the file@>@=} + ; + +statement_list: + statement_list statement {@>@<Attach a statement to a statement list@>@=} + | statement {@>@<Start a statement list with a statement@>@=} + ; + +statement_list_opt: + {@>@[TeX_( "/yy0{}" );@]@=} + | statement_list {@>@<Carry on@>@=} + ; +@g + +@ @<Attach a statement to a statement list@>= + @[TeX_( "/yy0{/the/yy(1)/yytoksempty{/yy(2)}{}{/yytoksempty{/yy(1)}{}{/noexpand/ldor}{/the/yy(2)}}}" );@] + +@ @<Start a statement list with a statement@>= + @[TeX_( "/yy0{/yytoksempty{/yy(1)}{}{{/the/yy(1)}}}" );@] + +@ @<Grammar rules@>= +@G +@t}\vb{\inline\flatten}{@> +length: + QUAD {} + | SQUAD {} + | LONG {} + | SHORT {} + | BYTE {} + ; + +fill_exp: + mustbe_exp {@>@<Carry on@>@=} + ; + +fill_opt: + '=' fill_exp {@>@[TeX_( "/yy0{/noexpand/ldfill{/the/yy(2)}}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +@t}\vb{\resetf}{@> +assign_op: + PLUSEQ {@>@[TeX_( "/yy(0)={/MRL{+{/K}}}" );@]@=} + | MINUSEQ {@>@[TeX_( "/yy(0)={/MRL{-{/K}}}" );@]@=} +@t}\vb{\inline\flatten}{@> + | MULTEQ {@>@[TeX_( "/yy(0)={/MRL*{/K}}" );@]@=} + | DIVEQ {@>@[TeX_( "/yy(0)={/MRL{{/div}{/K}}}" );@]@=} + | LSHIFTEQ {@>@[TeX_( "/yy(0)={/MRL{{/ll}{/K}}}" );@]@=} + | RSHIFTEQ {@>@[TeX_( "/yy(0)={/MRL{{/gg}{/K}}}" );@]@=} + | ANDEQ {@>@[TeX_( "/yy(0)={/Xandxeq}" );@]@=} + | OREQ {@>@[TeX_( "/yy(0)={/Xorxeq}" );@]@=} + ; + +end: ';' | ',' + ; + +opt_comma: ',' | + ; +@t}\vb{\resetf}{@> +@g + +@ Assignments are not expressions as in \Cee. +@<Grammar rules@>= +@G +assignment: + NAME '=' mustbe_exp {@>@<Process simple assignment@>@=} + | NAME assign_op mustbe_exp {@>@<Process compound assignment@>@=} + | HIDDEN '(' NAME '=' mustbe_exp ')' {} + | PROVIDE '(' NAME '=' mustbe_exp ')' {} + | PROVIDE_HIDDEN '(' NAME '=' mustbe_exp ')' {} + ; +@g + +@ @<Process simple assignment@>= + @[TeX_( "/yy0{/noexpand/ldassignment{/noexpand/ldregexp{/the/yy(1)}}{/K}{/the/yy(3)}}" );@] + +@ @<Process compound assignment@>= + @[TeX_( "/yy0{/noexpand/ldassignment{/noexpand/ldregexp{/the/yy(1)}}{/the/yy(2)}{/the/yy(3)}}" );@] + +@ @<Grammar rules@>= +@G +memory: + MEMORY '{' memory_spec_list_opt '}' {@>@[TeX_( "/yy0{/noexpand/ldmemory{/the/yy(3)}}" );@]@=} + ; + +memory_spec_list_opt: + memory_spec_list {@>@<Carry on@>@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +memory_spec_list: + memory_spec_list opt_comma memory_spec {@>@[TeX_( "/yy0{/the/yy(1)/the/yy(3)}" );@]@=} + | memory_spec {@>@[TeX_( "/yy0{/the/yy(1)}" );@]@=} + ; + + +memory_spec: + NAME {} + attributes_opt ':' + origin_spec +@t}\vb{\breakline}{@> + opt_comma length_spec {@>@[TeX_( "/yy0{/noexpand/ldmemoryspec{/the/yy(1)}{/the/yy(3)}{/the/yy(5)}{/the/yy(7)}}" );@]@=} + | INCLUDE filename {@>@<Peek at a file@>@=} + memory_spec_list_opt END {@>@<Close the file@>@=} + ; +@g + +@ @<Grammar rules@>= +@G +origin_spec: + ORIGIN '=' mustbe_exp {@>@[TeX_( "/yy0{/noexpand/ldoriginspec{/the/yy(3)}}" );@]@=} + ; + +length_spec: + LENGTH '=' mustbe_exp {@>@[TeX_( "/yy0{/noexpand/ldlengthspec{/the/yy(3)}}" );@]@=} + ; + +attributes_opt: + + {@>@[TeX_( "/yy0{}" );@]@=} + | '(' attributes_list ')' {@>@[TeX_( "/yy0{/the/yy(2)}" );@]@=} + ; + +attributes_list: + attributes_string {@>@[TeX_( "/yy0{/the/yy(1)}" );@]@=} + | attributes_list attributes_string {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/ldspace/the/yy(2)}" );@]@=} + ; + +attributes_string: + NAME {@>@[TeX_( "/yy0{/noexpand/ldattributes{/the/yy(1)}}" );@]@=} + | '!' NAME {@>@[TeX_( "/yy0{/noexpand/ldattributesneg{/the/yy(2)}}" );@]@=} + ; + +startup: + STARTUP '(' filename ')' {} + ; + +high_level_library: + HLL '(' high_level_library_NAME_list ')' + | HLL '(' ')' {} + ; + +high_level_library_NAME_list: + high_level_library_NAME_list opt_comma filename + {} + | filename {} + ; + +low_level_library: + SYSLIB '(' low_level_library_NAME_list ')' {} + ; + +low_level_library_NAME_list: + low_level_library_NAME_list opt_comma filename + {} + | {} + ; + +floating_point_support: + FLOAT {} + | NOFLOAT {} + ; + +nocrossref_list: + {} + | NAME nocrossref_list {} + | NAME ',' nocrossref_list {} + ; + +mustbe_exp: + {@>@[TeX_( "/ldlex@@expression" );@]@=} + exp {@>@[TeX_( "/ldlex@@popstate/yy0{/the/yy(2)}" );@]@=} + ; +@g + +@*1 {\ifheader\ninepoint\fi\prodstyle{SECTIONS}} and expressions. +The linker supports an extensive range of expressions. The precedence +mechanism provided by \bison\ is used to present the composition of +expressions out of simpler chunks and basic building blocks tied +together by algebraic operations. +@<Grammar rules@>= +@G +exp : + '-' exp %prec UNARY {@>@[TeX_( "/yy0{{-/the/yy(2)}}" );@]@=} + | '(' exp ')' {@>@[TeX_( "/yy0{(/the/yy(2))}" );@]@=} + | NEXT '(' exp ')' %prec UNARY {@>@[TeX_( "/yy0{/hbox{/nx/ssf next}(/the/yy(3))}" );@]@=} + | '!' exp %prec UNARY {@>@[TeX_( "/yy0{{/noexpand/CM/the/yy(2)}}" );@]@=} + | '+' exp %prec UNARY {@>@[TeX_( "/yy0{{+/the/yy(2)}}" );@]@=} + | '~' exp %prec UNARY {@>@[TeX_( "/yy0{{/noexpand/R/the/yy(2)}}" );@]@=} + | exp '*' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/times/the/yy(3)}" );@]@=} + | exp '/' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand///the/yy(3)}" );@]@=} + | exp '%' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/div/the/yy(3)}" );@]@=} + | exp '+' exp {@>@[TeX_( "/yy0{/the/yy(1)+/the/yy(3)}" );@]@=} + | exp '-' exp {@>@[TeX_( "/yy0{/the/yy(1)-/the/yy(3)}" );@]@=} + | exp LSHIFT exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/ll/the/yy(3)}" );@]@=} + | exp RSHIFT exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/gg/the/yy(3)}" );@]@=} + | exp EQ exp {@>@[TeX_( "/yy0{/the/yy(1)=/the/yy(3)}" );@]@=} + | exp NE exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/not=/the/yy(3)}" );@]@=} + | exp LE exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/leq/the/yy(3)}" );@]@=} + | exp GE exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/geq/the/yy(3)}" );@]@=} + | exp '<' exp {@>@[TeX_( "/yy0{/the/yy(1)</the/yy(3)}" );@]@=} + | exp '>' exp {@>@[TeX_( "/yy0{/the/yy(1)>/the/yy(3)}" );@]@=} + | exp '&' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/AND/the/yy(3)}" );@]@=} + | exp '^' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/XOR/the/yy(3)}" );@]@=} + | exp '|' exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/OR/the/yy(3)}" );@]@=} + | exp '?' exp ':' exp {@>@<Process a primitive conditional@>@=} + | exp ANDAND exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/V/the/yy(3)}" );@]@=} + | exp OROR exp {@>@[TeX_( "/yy0{/the/yy(1)/noexpand/W/the/yy(3)}" );@]@=} +@g + +@ More atomic expression types specific to the linker's function. +@<Grammar rules@>= +@G +exp : + DEFINED '(' NAME ')' {} + | INT {} + | SIZEOF_HEADERS {} + | ALIGNOF '(' NAME ')' {} + | SIZEOF '(' NAME ')' {} + | ADDR '(' NAME ')' {} + | LOADADDR '(' NAME ')' {} + | CONSTANT '(' NAME ')' {} + | ABSOLUTE '(' exp ')' {} + | ALIGN_K '(' exp ')' {@>@[TeX_( "/yy0{/nx/mathop{/hbox{/nx/ssf align}}(/the/yy(3))}" );@]@=} + | ALIGN_K '(' exp ',' exp ')' {} + | DATA_SEGMENT_ALIGN '(' exp ',' exp ')' {} + | DATA_SEGMENT_RELRO_END '(' exp ',' exp ')' {} + | DATA_SEGMENT_END '(' exp ')' {} + | SEGMENT_START '(' NAME ',' exp ')' {} + | BLOCK '(' exp ')' {} + | NAME {@>@[TeX_( "/yy0{/noexpand/ldregexp{/the/yy(1)}}" );@]@=} + | MAX_K '(' exp ',' exp ')' {} + | MIN_K '(' exp ',' exp ')' {} + | ASSERT_K '(' exp ',' NAME ')' {} + | ORIGIN '(' NAME ')' {} + | LENGTH '(' NAME ')' {} + | LOG2CEIL '(' exp ')' {} + ; +@g + +@ @<Process a primitive conditional@>= +@q TeX_( "/yy0{/hbox{/nx/ttl let }/nx/xi(0)=/the/yy(5): /nx/xi(/nx/CM0)=/the/yy(3)/hbox{ /nx/ttl do }/xi(/the/yy(1))}" );@> +@[TeX_( "/yy0{/hbox{/nx/ttl do }/xi(/the/yy(1))/hbox{ /nx/ttl where }" );@> +@[TeX_( " {/let/nx/{/nx/bigbracedel/nx/xi(x)=/nx/cases{/the/yy(5)& if /inmath{x=0}/cr/the/yy(3)& if /inmath{x/nx/not=0}}}}" );@] + +@ @<Grammar rules@>= +@G +memspec_at_opt: + AT '>' NAME {@>@[TeX_( "/yy0{/the/yy(3)}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +opt_at: + AT '(' exp ')' {@>@[TeX_( "/yy0{/the/yy(3)}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +opt_align: + ALIGN_K '(' exp ')' {@>@[TeX_( "/yy0{/the/yy(3)}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +opt_align_with_input: + ALIGN_WITH_INPUT {@>@[TeX_( "/yy0{align with input}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +opt_subalign: + SUBALIGN '(' exp ')' {@>@[TeX_( "/yy0{/the/yy(3)}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +sect_constraint: + ONLY_IF_RO {@>@[TeX_( "/yy0{only/_if/_ro}" );@]@=} + | ONLY_IF_RW {@>@[TeX_( "/yy0{only/_if/_rw}" );@]@=} + | SPECIAL {@>@[TeX_( "/yy0{special}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +@g + +@ {\it The \prodstyle{GROUP} case is just enough to support the \gcc\ +|svr3.ifile| script. It is not intended to be full support. I'm +not even sure what \prodstyle{GROUP} is supposed to mean.} A careful +analysis of the productions below reveals some +pitfalls in the parser/lexer interaction setup that uses the state switching +macros (or functions in the case of the original parser). The switch +to the \.{EXPRESSION} state at the end of the production for +\prodstyle{section} is invoked before \prodstyle{opt\_comma} which can +be empty. This means that the next (lookahead) token (which could be a +\prodstyle{NAME} in a different context) might be read before the +lexer is in the appropriate state. In practice, the names of the sections +and other \prodstyle{NAME}s are usually pretty straightforward so this +parser idiosynchrasy is unlikely to lead to a genuine problem. Since +the goal was to keep the original grammar intact as much as possible, +it was decided to leave this production unchanged. +@<Grammar rules@>= +@G +section: + NAME {@>@[TeX_( "/ldlex@@expression" );@]@=} + opt_exp_with_type opt_at opt_align opt_align_with_input +@t}\vb{\breakline}{@> + opt_subalign {@>@[TeX_( "/ldlex@@popstate/ldlex@@script" );@]@=} + sect_constraint + '{' {} + statement_list_opt + '}' {@>@[TeX_( "/ldlex@@popstate/ldlex@@expression" );@]@=} + memspec_opt memspec_at_opt phdr_opt fill_opt + {@>@[TeX_( "/ldlex@@popstate" );@]@=} + opt_comma {@>@<Record a named section@>@=} + | OVERLAY {@>@[TeX_( "/ldlex@@expression" );@]@=} + opt_exp_without_type opt_nocrossrefs opt_at opt_subalign + {@>@[TeX_( "/ldlex@@popstate/ldlex@@script" );@]@=} + '{' {} + overlay_section '}' {@>@[TeX_( "/ldlex@@popstate/ldlex@@expression" );@]@=} + memspec_opt memspec_at_opt phdr_opt fill_opt + {@>@[TeX_( "/ldlex@@popstate" );@]@=} + opt_comma {@>@<Record an overlay section@>@=} + | GROUP {@>@[TeX_( "/ldlex@@expression" );@]@=} + opt_exp_with_type {@>@[TeX_( "/ldlex@@popstate" );@]@=} + '{' sec_or_group_p1 '}' + | INCLUDE filename {@>@<Peek at a file@>@=} + sec_or_group_p1 END + {@>@<Close the file@>@=} + ; +@g + +@ @<Record a named section@>= + @[TeXb( "/yy0{/nx/ldnamedsection{/the/yy(1)}{/the/yy(3)}{/the/yy(4)}" );@]@; + @[TeXf( " {{/the/yy(5)}{/the/yy(6)}{/the/yy(7)}}" );@]@;/* alignment */ + @[TeXf( " {/the/yy(9)}{/the/yy(12)}" );@]@; + @[TeXfo( " {{/the/yy(15)}{/the/yy(16)}{/the/yy(17)}{/the/yy(18)}}}" );@]@; /*memory specifiers */ + +@ @<Record an overlay section@>= + +@ @<Grammar rules@>= +@G +type: + NOLOAD {@>@[TeX_( "/yy0{noload}" );@]@=} + | DSECT {@>@[TeX_( "/yy0{dsect}" );@]@=} + | COPY {@>@[TeX_( "/yy0{copy}" );@]@=} + | INFO {@>@[TeX_( "/yy0{info}" );@]@=} + | OVERLAY {@>@[TeX_( "/yy0{overlay}" );@]@=} + ; + +atype: + '(' type ')' {@>@[TeX_( "/yy0{/noexpand/ldtype{/the/yy(2)}}" );@]@=} + | '(' ')' {@>@[TeX_( "/yy0{/noexpand/ldtype{}}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; +@g + +@ {\it The \prodstyle{BIND} cases are to support the \gcc\ |svr3.ifile| +script. They aren't intended to implement full support for the +\prodstyle{BIND} keyword. I'm not even sure what \prodstyle{BIND} +is supposed to mean.} +@<Grammar rules@>= +@G +opt_exp_with_type: + exp atype ':' {@>@[TeX_( "/yy0{{}{/the/yy(1)}{}{}{/the/yy(2)}}" );@]@=} + | atype ':' {@>@[TeX_( "/yy0{{}{}{}{}{/the/yy(1)}}" );@]@=} + | BIND '(' exp ')' atype ':' {@>@[TeX_( "/yy0{{bind}{/the/yy(3)}{}{}{/the/yy(5)}}" );@]@=} + | BIND '(' exp ')' BLOCK '(' exp ')' atype ':' + {@>@[TeX_( "/yy0{{bind}{/the/yy(3)}{block}{/the/yy(7)}{/the/yy(9)}}" );@]@=} + ; + +opt_exp_without_type: + exp ':' {} + | ':' {} + ; + +opt_nocrossrefs: + {} + | NOCROSSREFS {} + ; + +memspec_opt: + '>' NAME {@>@[TeX_( "/yy0{/the/yy(2)}" );@]@=} + | {@>@[TeX_( "/yy0{}" );@]@=} + ; + +phdr_opt: + {@>@[TeX_( "/yy0{}" );@]@=} + | phdr_opt ':' NAME {@>@<Add another pheader@>@=} + ; + +overlay_section: + + | overlay_section + NAME {@>@[TeX_( "/ldlex@@script" );@]@=} + '{' statement_list_opt '}' {@>@[TeX_( "/ldlex@@popstate/ldlex@@expression" );@]@=} + phdr_opt fill_opt {@>@[TeX_( "/ldlex@@popstate" );@]@=} + opt_comma + ; +@g + +@ @<Add another pheader@>= + @[TeXb( "/yytoksempty{/yy(1)}{/yy0{{/the/yy(3)}}}" );@]@; + @[TeXao( "{/yy0{/the/yy(1)/noexpand/ldor{/the/yy(3)}}}" );@]@; + +@ @<Grammar rules@>= +@G +phdrs: + PHDRS '{' phdr_list '}' + ; + +phdr_list: + + | phdr_list phdr + ; + +phdr: + NAME {@>@[TeX_( "/ldlex@@expression" );@]@=} + phdr_type phdr_qualifiers {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ';' {} + ; + +phdr_type: + exp {} + ; + +phdr_qualifiers: + {} + | NAME phdr_val phdr_qualifiers {} + | AT '(' exp ')' phdr_qualifiers {} + ; + +phdr_val: + {} + | '(' exp ')' {} + ; +@g + +@*1 Other types of script files. At present time other script types +are ignored, although some of the rules are used in linker scripts +that are processed by the parser. +@<Dynamic list file rules@>= +@G +dynamic_list_file: + {@>@[TeX_( "/ldlex@@version@@file" );@]@=} + dynamic_list_nodes {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; + +dynamic_list_nodes: + dynamic_list_node + | dynamic_list_nodes dynamic_list_node + ; + +dynamic_list_node: + '{' dynamic_list_tag '}' ';' + ; + +dynamic_list_tag: + vers_defns ';' {} + ; +@g + +@ {\it This syntax is used within an external version script file.} +@<Version file rules@>= +@G +version_script_file: + {@>@[TeX_( "/ldlex@@version@@file" );@]@=} + vers_nodes {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; +@g + +@ {\it This is used within a normal linker script file.} +@<Grammar rules@>= +@G +version: + {@>@[TeX_( "/ldlex@@version@@script" );@]@=} + VERSIONK '{' vers_nodes '}' {@>@[TeX_( "/ldlex@@popstate" );@]@=} + ; + +vers_nodes: + vers_node + | vers_nodes vers_node + ; + +vers_node: + '{' vers_tag '}' ';' {} + | VERS_TAG '{' vers_tag '}' ';' {} + | VERS_TAG '{' vers_tag '}' verdep ';' {} + ; + +verdep: + VERS_TAG {} + | verdep VERS_TAG {} + ; + +vers_tag: + {} + | vers_defns ';' {} + | GLOBAL ':' vers_defns ';' {} + | LOCAL ':' vers_defns ';' {} + | GLOBAL ':' vers_defns ';' LOCAL ':' vers_defns ';' + {} + ; + +vers_defns: + VERS_IDENTIFIER {} + | NAME {} + | vers_defns ';' VERS_IDENTIFIER {} + | vers_defns ';' NAME {} + | vers_defns ';' EXTERN NAME '{' {} + vers_defns opt_semicolon '}' {} + | EXTERN NAME '{' {} + vers_defns opt_semicolon '}' {} + | GLOBAL {} + | vers_defns ';' GLOBAL {} + | LOCAL {} + | vers_defns ';' LOCAL {} + | EXTERN {} + | vers_defns ';' EXTERN {} + ; +@t}\vb{\inline\flatten}{@> +opt_semicolon: + + | ';' + ; +@g diff --git a/support/splint/examples/ld/ldgramo.w b/support/splint/examples/ld/ldgramo.w new file mode 100644 index 0000000000..3069bb71eb --- /dev/null +++ b/support/splint/examples/ld/ldgramo.w @@ -0,0 +1,1645 @@ +@q Copyright 2015 Alexander Shibakov@> +@q Copyright 2002-2015 Free Software Foundation, Inc.@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1 The original parser. {\it Here we present the full grammar of \ld, including some actions. +The grammar is split into sections but otherwise is reproduced exactly. +In addition to improving readability, such splitting allows \CWEB\ to process the code in +manageable increments. An observant reader will notice the difficulty +\CWEAVE\ is having with typesetting the structure tags that have the same +name as the structure variables of the appropriate type. This is a well-known +defect in \CWEAVE's design (see the requisite documentation) left +uncorrected to discourage the poor programming practice.} + +@s bfd_boolean int +@s bfd_vma int +@s fill_type int +@s lang_nocrossref int +@s lang_output_section_phdr_list int +@s bfd_elf_version_deps int +@s bfd_elf_version_expr int +@s bfd_elf_version_tree int +@s etree_union int +@s wildcard_spec int +@s section_type int +@s lang_memory_region_type int + +@<The original \ld\ parser@>= +@G +%{ + @>@<\Cee\ setup for \ld\ grammar@>@; +%} +%union { +@O + bfd_vma integer; + struct big_int + { + bfd_vma integer; + char *str; + } bigint; + fill_type *fill; + char *name; + const char *cname; + struct wildcard_spec wildcard; + struct wildcard_list *wildcard_list; + struct name_list *name_list; + struct flag_info_list *flag_info_list; + struct flag_info *flag_info; + int token; + union etree_union *etree; + struct phdr_info + { + bfd_boolean filehdr; + bfd_boolean phdrs; + union etree_union *at; + union etree_union *flags; + } phdr; + struct lang_nocrossref *nocrossref; + struct lang_output_section_phdr_list *section_phdr; + struct bfd_elf_version_deps *deflist; + struct bfd_elf_version_expr *versyms; + struct bfd_elf_version_tree *versnode; +@o + } + @>@<Token definitions for the \ld\ grammar@>@; +%% + @>@<Original \ld\ grammar rules@>@; +%% +@g + +@ {\it The \Cee\ code is left mostly intact (with the exception of a few +comments) although it does not show up in the final output. The parts +that {\it are\/} typeset represent the semantics that is reproduced in +the typesetting parser. This includes all the state switching, as well +as some other actions that affect the parser-lexer interaction (such +as opening a new input buffer). The only exception to this rule is the +code for the \MRI\ script section of the grammar. It is reproduced +mostly as an example of a pretty printed grammar, since otherwise, +\MRI\ scripts are completely ignored by the typeseting parser.} +\def\initbootstrap{\restorecslist{parser-prototypes}\yyunion}% + +@<\Cee\ setup for \ld\ grammar@>= +#define DONTDECLARE_MALLOC + +#include "sysdep.h" +#include "bfd.h" +#include "bfdlink.h" +#include "ld.h" +#include "ldexp.h" +#include "ldver.h" +#include "ldlang.h" +#include "ldfile.h" +#include "ldemul.h" +#include "ldmisc.h" +#include "ldmain.h" +#include "mri.h" +#include "ldctor.h" +#include "ldlex.h" + +#ifndef YYDEBUG +#define YYDEBUG 1 +#endif + +static enum section_type sectype; +static lang_memory_region_type *region; + +bfd_boolean ldgram_had_keep = FALSE; +char *ldgram_vers_current_lang = NULL; + +#define ERROR_NAME_MAX 20 +static char *error_names[ERROR_NAME_MAX]; +static int error_index; +#define PUSH_ERROR(x) if (error_index < ERROR_NAME_MAX) error_names[error_index] = x; error_index++; +#define POP_ERROR() @[error_index--;@] + +@ {\it The token definitions and the corresponding \prodstyle{\%union} +styles are intermixed, which makes sense in the traditional style of a +\bison\ script. When \CWEB\ is used, however, it helps to introduce the +code in small, manageable sections and take advantage of \CWEB's crossreferencing +facilities to provide cues on the relationships between various parts +of the code.} +@<Token definitions for the \ld\ grammar@>= +@G +%type <etree> exp opt_exp_with_type mustbe_exp opt_at phdr_type phdr_val +%type <etree> opt_exp_without_type opt_subalign opt_align +%type <fill> fill_opt fill_exp +%type <name_list> exclude_name_list +%type <wildcard_list> file_NAME_list +%type <flag_info_list> sect_flag_list +%type <flag_info> sect_flags +%type <name> memspec_opt casesymlist +%type <name> memspec_at_opt +%type <cname> wildcard_name +%type <wildcard> wildcard_spec +%token <bigint> INT +%token <name> NAME LNAME +%type <integer> length +%type <phdr> phdr_qualifiers +%type <nocrossref> nocrossref_list +%type <section_phdr> phdr_opt +%type <integer> opt_nocrossrefs + +%right <token> PLUSEQ MINUSEQ MULTEQ DIVEQ '=' LSHIFTEQ RSHIFTEQ ANDEQ OREQ +%right <token> '?' ':' +%left <token> OROR +%left <token> ANDAND +%left <token> '|' +%left <token> '^' +%left <token> '&' +%left <token> EQ NE +%left <token> '<' '>' LE GE +%left <token> LSHIFT RSHIFT + +%left <token> '+' '-' +%left <token> '*' '/' '%' + +%right UNARY +%token END +%left <token> '(' +%token <token> ALIGN_K BLOCK BIND QUAD SQUAD LONG SHORT BYTE +%token SECTIONS PHDRS INSERT_K AFTER BEFORE +%token DATA_SEGMENT_ALIGN DATA_SEGMENT_RELRO_END DATA_SEGMENT_END +%token SORT_BY_NAME SORT_BY_ALIGNMENT SORT_NONE +%token SORT_BY_INIT_PRIORITY +%token '{' '}' +%token SIZEOF_HEADERS OUTPUT_FORMAT FORCE_COMMON_ALLOCATION OUTPUT_ARCH +%token INHIBIT_COMMON_ALLOCATION +%token SEGMENT_START +%token INCLUDE +%token MEMORY +%token REGION_ALIAS +%token LD_FEATURE +%token NOLOAD DSECT COPY INFO OVERLAY +%token DEFINED TARGET_K SEARCH_DIR MAP ENTRY +%token <integer> NEXT +%token SIZEOF ALIGNOF ADDR LOADADDR MAX_K MIN_K +%token STARTUP HLL SYSLIB FLOAT NOFLOAT NOCROSSREFS +%token ORIGIN FILL +%token LENGTH CREATE_OBJECT_SYMBOLS INPUT GROUP OUTPUT CONSTRUCTORS +%token ALIGNMOD AT SUBALIGN HIDDEN PROVIDE PROVIDE_HIDDEN AS_NEEDED +%type <token> assign_op atype attributes_opt sect_constraint opt_align_with_input +%type <name> filename +%token CHIP LIST SECT ABSOLUTE LOAD NEWLINE ENDWORD ORDER NAMEWORD ASSERT_K +%token LOG2CEIL FORMAT PUBLIC DEFSYMEND BASE ALIAS TRUNCATE REL +%token INPUT_SCRIPT INPUT_MRI_SCRIPT INPUT_DEFSYM CASE EXTERN START +%token <name> VERS_TAG VERS_IDENTIFIER +%token GLOBAL LOCAL VERSIONK INPUT_VERSION_SCRIPT +%token KEEP ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS ALIGN_WITH_INPUT +%token EXCLUDE_FILE +%token CONSTANT +%type <versyms> vers_defns +%type <versnode> vers_tag +%type <deflist> verdep +%token INPUT_DYNAMIC_LIST +@g + +@ {\it The original \Cee\ code has been preserved and presented along with +the grammar rules in the next two sections (the \Cee\ code has not +been deleted in the subsequent sections either, it is just not +typeset).} +\saveparseoutputfalse +@<Original \ld\ grammar rules@>= +@G +file: + INPUT_SCRIPT script_file + | INPUT_MRI_SCRIPT mri_script_file + | INPUT_VERSION_SCRIPT version_script_file + | INPUT_DYNAMIC_LIST dynamic_list_file + | INPUT_DEFSYM defsym_expr + ; + + +filename: NAME; + + +defsym_expr: + { @> ldlex_defsym(); @=} + NAME '=' exp + { +@O + ldlex_popstate(); +@o + lang_add_assignment (exp_defsym ($2, $4)); + } + ; + +@g + +@ Syntax within an \MRI\ script file. +@<Original \ld\ grammar rules@>= +@G +mri_script_file: + { +@O + @[ldlex_mri_script ();@] +@o + PUSH_ERROR (_("MRI style script")); + } + mri_script_lines + { +@O + ldlex_popstate (); +@o + mri_draw_tree (); + POP_ERROR (); + } + ; + +mri_script_lines: + mri_script_lines mri_script_command NEWLINE + | + ; + +@g + +@ @<Original \ld\ grammar rules@>= +@G +mri_script_command: + CHIP exp + | CHIP exp ',' exp + | NAME { @> @<Flag an unrecognized keyword@> @=} + | LIST { @> config.map_filename = "-"; @=} + | ORDER ordernamelist + | ENDWORD + | PUBLIC NAME '=' exp + { @> mri_public($2, $4); @=} + | PUBLIC NAME ',' exp + { @> mri_public($2, $4); @=} + | PUBLIC NAME exp + { @> mri_public($2, $3); @=} + | FORMAT NAME + { @> mri_format($2); @=} + | SECT NAME ',' exp + { @> mri_output_section($2, $4); @=} + | SECT NAME exp + { @> mri_output_section($2, $3); @=} + | SECT NAME '=' exp + { @> mri_output_section($2, $4); @=} + | ALIGN_K NAME '=' exp + { @> mri_align($2,$4); @=} + | ALIGN_K NAME ',' exp + { @> mri_align($2,$4); @=} + | ALIGNMOD NAME '=' exp + { @> mri_alignmod($2,$4); @=} + | ALIGNMOD NAME ',' exp + { @> mri_alignmod($2,$4); @=} + | ABSOLUTE mri_abs_name_list + | LOAD mri_load_name_list + | NAMEWORD NAME + { @> mri_name($2); @=} + | ALIAS NAME ',' NAME + { @> mri_alias($2,$4,0); @=} + | ALIAS NAME ',' INT + { @> mri_alias ($2, 0, (int) $4.integer); @=} + | BASE exp + { @> mri_base($2); @=} + | TRUNCATE INT + { @> mri_truncate ((unsigned int) $2.integer); @=} + | CASE casesymlist + | EXTERN extern_name_list + | INCLUDE filename + { @> ldlex_script (); ldfile_open_command_file($2); @=} + mri_script_lines END + { @> ldlex_popstate (); @=} + | START NAME + { @> lang_add_entry ($2, FALSE); @=} + | + ; + +ordernamelist: + ordernamelist ',' NAME { @> mri_order($3); @=} + | ordernamelist NAME { @> mri_order($2); @=} + | + ; + +mri_load_name_list: + NAME { @> mri_load($1); @=} + | mri_load_name_list ',' NAME { @> mri_load($3); @=} + ; + +mri_abs_name_list: + NAME { @> mri_only_load($1); @=} + | mri_abs_name_list ',' NAME { @> mri_only_load($3); @=} + ; + +casesymlist: + { @> $$ = NULL; @=} + | NAME + | casesymlist ',' NAME + ; + +@g + +@ {\it Here is one way to deal with code that is too long to fit in an action.} +@<Flag an unrecognized keyword@>= + einfo(_("%P%F: unrecognised keyword in MRI style script '%s'\n"),$1); + +@ Parsed as expressions so that commas separate entries. +@<Original \ld\ grammar rules@>= +@G +extern_name_list: + {@> @[ldlex_expression ();@]@=} + extern_name_list_body + {@> @[ldlex_popstate ();@]@=} + +extern_name_list_body: + NAME + { ldlang_add_undef ($1, FALSE); } + | extern_name_list_body NAME + { ldlang_add_undef ($2, FALSE); } + | extern_name_list_body ',' NAME + { ldlang_add_undef ($3, FALSE); } + ; + +script_file: + {@> @[ldlex_both();@]@=} + ifile_list + {@> @[ldlex_popstate();@]@=} + ; + +ifile_list: + ifile_list ifile_p1 + | + ; + +@g + +@ {\it All the commands that can appear in a standard linker script.}% +@<Original \ld\ grammar rules@>= +@G +ifile_p1: + memory + | sections + | phdrs + | startup + | high_level_library + | low_level_library + | floating_point_support + | statement_anywhere + | version + | ';' + | TARGET_K '(' NAME ')' + { lang_add_target($3); } + | SEARCH_DIR '(' filename ')' + { ldfile_add_library_path ($3, FALSE); } + | OUTPUT '(' filename ')' + { lang_add_output($3, 1); } + | OUTPUT_FORMAT '(' NAME ')' + { lang_add_output_format ($3, (char *) NULL, + (char *) NULL, 1); } + | OUTPUT_FORMAT '(' NAME ',' NAME ',' NAME ')' + { lang_add_output_format ($3, $5, $7, 1); } + | OUTPUT_ARCH '(' NAME ')' + { ldfile_set_output_arch ($3, bfd_arch_unknown); } + | FORCE_COMMON_ALLOCATION + { command_line.force_common_definition = TRUE ; } + | INHIBIT_COMMON_ALLOCATION + { command_line.inhibit_common_definition = TRUE ; } + | INPUT '(' input_list ')' + | GROUP + { lang_enter_group (); } + '(' input_list ')' + { lang_leave_group (); } + | MAP '(' filename ')' + { lang_add_map($3); } + | INCLUDE filename + { +@O + ldlex_script (); ldfile_open_command_file($2); +@o + } + ifile_list END + {@> @[ldlex_popstate ();@]@=} + | NOCROSSREFS '(' nocrossref_list ')' + { + lang_add_nocrossref ($3); + } + | EXTERN '(' extern_name_list ')' + | INSERT_K AFTER NAME + { lang_add_insert ($3, 0); } + | INSERT_K BEFORE NAME + { lang_add_insert ($3, 1); } + | REGION_ALIAS '(' NAME ',' NAME ')' + { lang_memory_region_alias ($3, $5); } + | LD_FEATURE '(' NAME ')' + { lang_ld_feature ($3); } + ; + +@g + +@ @<Original \ld\ grammar rules@>= +@G +input_list: + NAME + { lang_add_input_file($1,lang_input_file_is_search_file_enum, + (char *)NULL); } + | input_list ',' NAME + { lang_add_input_file($3,lang_input_file_is_search_file_enum, + (char *)NULL); } + | input_list NAME + { lang_add_input_file($2,lang_input_file_is_search_file_enum, + (char *)NULL); } + | LNAME + { lang_add_input_file($1,lang_input_file_is_l_enum, + (char *)NULL); } + | input_list ',' LNAME + { lang_add_input_file($3,lang_input_file_is_l_enum, + (char *)NULL); } + | input_list LNAME + { lang_add_input_file($2,lang_input_file_is_l_enum, + (char *)NULL); } + | AS_NEEDED '(' + { $<integer>$ = input_flags.add_DT_NEEDED_for_regular; + input_flags.add_DT_NEEDED_for_regular = TRUE; } + input_list ')' + { input_flags.add_DT_NEEDED_for_regular = $<integer>3; } + | input_list ',' AS_NEEDED '(' + { $<integer>$ = input_flags.add_DT_NEEDED_for_regular; + input_flags.add_DT_NEEDED_for_regular = TRUE; } + input_list ')' + { input_flags.add_DT_NEEDED_for_regular = $<integer>5; } + | input_list AS_NEEDED '(' + { $<integer>$ = input_flags.add_DT_NEEDED_for_regular; + input_flags.add_DT_NEEDED_for_regular = TRUE; } + input_list ')' + { input_flags.add_DT_NEEDED_for_regular = $<integer>4; } + ; + +sections: + SECTIONS '{' sec_or_group_p1 '}' + ; + +sec_or_group_p1: + sec_or_group_p1 section + | sec_or_group_p1 statement_anywhere + | + ; + +statement_anywhere: + ENTRY '(' NAME ')' + { lang_add_entry ($3, FALSE); } + | assignment end + | ASSERT_K {@>@[ldlex_expression ();@>@=} '(' exp ',' NAME ')' + { +@O + ldlex_popstate (); +@o + lang_add_assignment (exp_assert ($4, $6)); + } + ; + +@g + +@ \tracebadcharstrue +The \prodstyle{'*'} and \prodstyle{'?'} cases are there because the lexer returns them as +separate tokens rather than as \prodstyle{NAME}. +\tracebadcharsfalse +@<Original \ld\ grammar rules@>= +@G +wildcard_name: + NAME + { + $$ = $1; + } + | '*' + { + $$ = "*"; + } + | '?' + { + $$ = "?"; + } + ; + +@g + +@ @<Original \ld\ grammar rules@>= +@G +wildcard_spec: + wildcard_name + { + $$.name = $1; + $$.sorted = none; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | EXCLUDE_FILE '(' exclude_name_list ')' wildcard_name + { + $$.name = $5; + $$.sorted = none; + $$.exclude_name_list = $3; + $$.section_flag_list = NULL; + } + | SORT_BY_NAME '(' wildcard_name ')' + { + $$.name = $3; + $$.sorted = by_name; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_ALIGNMENT '(' wildcard_name ')' + { + $$.name = $3; + $$.sorted = by_alignment; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_NONE '(' wildcard_name ')' + { + $$.name = $3; + $$.sorted = by_none; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_NAME '(' SORT_BY_ALIGNMENT '(' wildcard_name ')' ')' + { + $$.name = $5; + $$.sorted = by_name_alignment; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_NAME '(' SORT_BY_NAME '(' wildcard_name ')' ')' + { + $$.name = $5; + $$.sorted = by_name; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_ALIGNMENT '(' SORT_BY_NAME '(' wildcard_name ')' ')' + { + $$.name = $5; + $$.sorted = by_alignment_name; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_ALIGNMENT '(' SORT_BY_ALIGNMENT '(' wildcard_name ')' ')' + { + $$.name = $5; + $$.sorted = by_alignment; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + | SORT_BY_NAME '(' EXCLUDE_FILE '(' exclude_name_list ')' wildcard_name ')' + { + $$.name = $7; + $$.sorted = by_name; + $$.exclude_name_list = $5; + $$.section_flag_list = NULL; + } + | SORT_BY_INIT_PRIORITY '(' wildcard_name ')' + { + $$.name = $3; + $$.sorted = by_init_priority; + $$.exclude_name_list = NULL; + $$.section_flag_list = NULL; + } + ; + +sect_flag_list: NAME + { + struct flag_info_list *n; + n = ((struct flag_info_list *) xmalloc (sizeof *n)); + if ($1[0] == '!') + { + n->with = without_flags; + n->name = &$1[1]; + } + else + { + n->with = with_flags; + n->name = $1; + } + n->valid = FALSE; + n->next = NULL; + $$ = n; + } + | sect_flag_list '&' NAME + { + struct flag_info_list *n; + n = ((struct flag_info_list *) xmalloc (sizeof *n)); + if ($3[0] == '!') + { + n->with = without_flags; + n->name = &$3[1]; + } + else + { + n->with = with_flags; + n->name = $3; + } + n->valid = FALSE; + n->next = $1; + $$ = n; + } + ; + +sect_flags: + INPUT_SECTION_FLAGS '(' sect_flag_list ')' + { + struct flag_info *n; + n = ((struct flag_info *) xmalloc (sizeof *n)); + n->flag_list = $3; + n->flags_initialized = FALSE; + n->not_with_flags = 0; + n->only_with_flags = 0; + $$ = n; + } + ; + +exclude_name_list: + exclude_name_list wildcard_name + { + struct name_list *tmp; + tmp = (struct name_list *) xmalloc (sizeof *tmp); + tmp->name = $2; + tmp->next = $1; + $$ = tmp; + } + | + wildcard_name + { + struct name_list *tmp; + tmp = (struct name_list *) xmalloc (sizeof *tmp); + tmp->name = $1; + tmp->next = NULL; + $$ = tmp; + } + ; + +file_NAME_list: + file_NAME_list opt_comma wildcard_spec + { + struct wildcard_list *tmp; + tmp = (struct wildcard_list *) xmalloc (sizeof *tmp); + tmp->next = $1; + tmp->spec = $3; + $$ = tmp; + } + | + wildcard_spec + { + struct wildcard_list *tmp; + tmp = (struct wildcard_list *) xmalloc (sizeof *tmp); + tmp->next = NULL; + tmp->spec = $1; + $$ = tmp; + } + ; + +input_section_spec_no_keep: + NAME + { + struct wildcard_spec tmp; + tmp.name = $1; + tmp.exclude_name_list = NULL; + tmp.sorted = none; + tmp.section_flag_list = NULL; + lang_add_wild (&tmp, NULL, ldgram_had_keep); + } + | sect_flags NAME + { + struct wildcard_spec tmp; + tmp.name = $2; + tmp.exclude_name_list = NULL; + tmp.sorted = none; + tmp.section_flag_list = $1; + lang_add_wild (&tmp, NULL, ldgram_had_keep); + } + | '[' file_NAME_list ']' + { + lang_add_wild (NULL, $2, ldgram_had_keep); + } + | sect_flags '[' file_NAME_list ']' + { + struct wildcard_spec tmp; + tmp.name = NULL; + tmp.exclude_name_list = NULL; + tmp.sorted = none; + tmp.section_flag_list = $1; + lang_add_wild (&tmp, $3, ldgram_had_keep); + } + | wildcard_spec '(' file_NAME_list ')' + { + lang_add_wild (&$1, $3, ldgram_had_keep); + } + | sect_flags wildcard_spec '(' file_NAME_list ')' + { + $2.section_flag_list = $1; + lang_add_wild (&$2, $4, ldgram_had_keep); + } + ; +@g + +@ @<Original \ld\ grammar rules@>= +@G +input_section_spec: + input_section_spec_no_keep + | KEEP '(' + { ldgram_had_keep = TRUE; } + input_section_spec_no_keep ')' + { ldgram_had_keep = FALSE; } + ; + +statement: + assignment end + | CREATE_OBJECT_SYMBOLS + { + lang_add_attribute(lang_object_symbols_statement_enum); + } + | ';' + | CONSTRUCTORS + { + + lang_add_attribute(lang_constructors_statement_enum); + } + | SORT_BY_NAME '(' CONSTRUCTORS ')' + { + constructors_sorted = TRUE; + lang_add_attribute (lang_constructors_statement_enum); + } + | input_section_spec + | length '(' mustbe_exp ')' + { + lang_add_data ((int) $1, $3); + } + + | FILL '(' fill_exp ')' + { + lang_add_fill ($3); + } + | ASSERT_K {@>@[ldlex_expression ();@]@=} '(' exp ',' NAME ')' end + { +@O + ldlex_popstate (); +@o + lang_add_assignment (exp_assert ($4, $6)); + } + | INCLUDE filename + {@> @[ldlex_script (); ldfile_open_command_file($2);@]@=} + statement_list_opt END + {@> @[ldlex_popstate ();@]@=} + ; +@t}\vb{\flatten\inline\squashtermstrue}{@> +statement_list: + statement_list statement + | statement + ; + +statement_list_opt: + + | statement_list + ; + +length: + QUAD + { $$ = $1; } + | SQUAD + { $$ = $1; } + | LONG + { $$ = $1; } + | SHORT + { $$ = $1; } + | BYTE + { $$ = $1; } + ; + +fill_exp: + mustbe_exp + { + $$ = exp_get_fill ($1, 0, "fill value"); + } + ; + +fill_opt: + '=' fill_exp + { $$ = $2; } + | { $$ = (fill_type *) 0; } + ; + +assign_op: + PLUSEQ + { $$ = '+'; } + | MINUSEQ + { $$ = '-'; } + | MULTEQ + { $$ = '*'; } + | DIVEQ + { $$ = '/'; } + | LSHIFTEQ + { $$ = LSHIFT; } + | RSHIFTEQ + { $$ = RSHIFT; } + | ANDEQ + { $$ = '&'; } + | OREQ + { $$ = '|'; } + + ; +end: ';' | ',' + ; +@t}\vb{\resetf}{@> + +assignment: + NAME '=' mustbe_exp + { + lang_add_assignment (exp_assign ($1, $3, FALSE)); + } + | NAME assign_op mustbe_exp + { + lang_add_assignment (exp_assign ($1, + exp_binop ($2, + exp_nameop (NAME, + $1), + $3), FALSE)); + } + | HIDDEN '(' NAME '=' mustbe_exp ')' + { + lang_add_assignment (exp_assign ($3, $5, TRUE)); + } + | PROVIDE '(' NAME '=' mustbe_exp ')' + { + lang_add_assignment (exp_provide ($3, $5, FALSE)); + } + | PROVIDE_HIDDEN '(' NAME '=' mustbe_exp ')' + { + lang_add_assignment (exp_provide ($3, $5, TRUE)); + } + ; + +@t}\vb{\inline\flatten}{@> +opt_comma: + ',' | ; +@t}\vb{\resetf}{@> +memory: + MEMORY '{' memory_spec_list_opt '}' + ; + +memory_spec_list_opt: memory_spec_list | ; + +memory_spec_list: + memory_spec_list opt_comma memory_spec + | memory_spec + ; + + +memory_spec: NAME + { region = lang_memory_region_lookup ($1, TRUE); } + attributes_opt ':' + origin_spec opt_comma length_spec + {} + | INCLUDE filename + {@> @[ldlex_script (); ldfile_open_command_file($2);@]@=} + memory_spec_list_opt END + {@> @[ldlex_popstate ();@]@=} + ; +@g + +@ @<Original \ld\ grammar rules@>= +@G +origin_spec: + ORIGIN '=' mustbe_exp + { + region->origin = exp_get_vma ($3, 0, "origin"); + region->current = region->origin; + } + ; + +length_spec: + LENGTH '=' mustbe_exp + { + region->length = exp_get_vma ($3, -1, "length"); + } + ; + +attributes_opt: + + { /* dummy action to avoid bison 1.25 error message */ } + | '(' attributes_list ')' + ; + +attributes_list: + attributes_string + | attributes_list attributes_string + ; + +attributes_string: + NAME + { lang_set_flags (region, $1, 0); } + | '!' NAME + { lang_set_flags (region, $2, 1); } + ; + +startup: + STARTUP '(' filename ')' + { lang_startup($3); } + ; + +high_level_library: + HLL '(' high_level_library_NAME_list ')' + | HLL '(' ')' + { ldemul_hll((char *)NULL); } + ; + +high_level_library_NAME_list: + high_level_library_NAME_list opt_comma filename + { ldemul_hll($3); } + | filename + { ldemul_hll($1); } + + ; + +low_level_library: + SYSLIB '(' low_level_library_NAME_list ')' + ; low_level_library_NAME_list: + low_level_library_NAME_list opt_comma filename + { ldemul_syslib($3); } + | + ; + +floating_point_support: + FLOAT + { lang_float(TRUE); } + | NOFLOAT + { lang_float(FALSE); } + ; + +nocrossref_list: + + { + $$ = NULL; + } + | NAME nocrossref_list + { + struct lang_nocrossref *n; + + n = (struct lang_nocrossref *) xmalloc (sizeof *n); + n->name = $1; + n->next = $2; + $$ = n; + } + | NAME ',' nocrossref_list + { + struct lang_nocrossref *n; + + n = (struct lang_nocrossref *) xmalloc (sizeof *n); + n->name = $1; + n->next = $3; + $$ = n; + } + ; + +mustbe_exp: {@> @[ldlex_expression ();@]@=} + exp + {@> @[ldlex_popstate ();@]@= $$=$2;} + ; + +@g + +@ {\it Rich expression syntax reproducing the one in \Cee.} +@<Original \ld\ grammar rules@>= +@G +exp : + '-' exp %prec UNARY + { $$ = exp_unop ('-', $2); } + | '(' exp ')' + { $$ = $2; } + | NEXT '(' exp ')' %prec UNARY + { $$ = exp_unop ((int) $1,$3); } + | '!' exp %prec UNARY + { $$ = exp_unop ('!', $2); } + | '+' exp %prec UNARY + { $$ = $2; } + | '~' exp %prec UNARY + { $$ = exp_unop ('~', $2);} +@t}\vb{\flatten\inline}{@> + | exp '*' exp + { $$ = exp_binop ('*', $1, $3); } + | exp '/' exp + { $$ = exp_binop ('/', $1, $3); } + | exp '%' exp + { $$ = exp_binop ('%', $1, $3); } + | exp '+' exp + { $$ = exp_binop ('+', $1, $3); } + | exp '-' exp +@t}\vb{\resetf}{@> + { $$ = exp_binop ('-' , $1, $3); } +@t}\vb{\flatten\inline}{@> + | exp LSHIFT exp + { $$ = exp_binop (LSHIFT , $1, $3); } + | exp RSHIFT exp + { $$ = exp_binop (RSHIFT , $1, $3); } + | exp EQ exp + { $$ = exp_binop (EQ , $1, $3); } + | exp NE exp + { $$ = exp_binop (NE , $1, $3); } + | exp LE exp +@t}\vb{\resetf}{@> + { $$ = exp_binop (LE , $1, $3); } +@t}\vb{\flatten\inline}{@> + | exp GE exp + { $$ = exp_binop (GE , $1, $3); } + | exp '<' exp + { $$ = exp_binop ('<' , $1, $3); } + | exp '>' exp + { $$ = exp_binop ('>' , $1, $3); } + | exp '&' exp + { $$ = exp_binop ('&' , $1, $3); } + | exp '^' exp +@t}\vb{\resetf}{@> + { $$ = exp_binop ('^' , $1, $3); } +@t}\vb{\flatten\inline}{@> + | exp '|' exp + { $$ = exp_binop ('|' , $1, $3); } + | exp '?' exp ':' exp + { $$ = exp_trinop ('?' , $1, $3, $5); } + | exp ANDAND exp + { $$ = exp_binop (ANDAND , $1, $3); } + | exp OROR exp +@t}\vb{\resetf}{@> + { $$ = exp_binop (OROR , $1, $3); } +@t}\vb{\flatten\inline}{@> + | DEFINED '(' NAME ')' + { $$ = exp_nameop (DEFINED, $3); } + | INT + { $$ = exp_bigintop ($1.integer, $1.str); } + | SIZEOF_HEADERS +@t}\vb{\resetf}{@> + { $$ = exp_nameop (SIZEOF_HEADERS,0); } +@t}\vb{\flatten\inline}{@> + | ALIGNOF '(' NAME ')' + { $$ = exp_nameop (ALIGNOF,$3); } + | SIZEOF '(' NAME ')' +@t}\vb{\resetf}{@> + { $$ = exp_nameop (SIZEOF,$3); } +@t}\vb{\flatten\inline}{@> + | ADDR '(' NAME ')' + { $$ = exp_nameop (ADDR,$3); } + | LOADADDR '(' NAME ')' +@t}\vb{\resetf}{@> + { $$ = exp_nameop (LOADADDR,$3); } +@t}\vb{\flatten\inline}{@> + | CONSTANT '(' NAME ')' + { $$ = exp_nameop (CONSTANT,$3); } + | ABSOLUTE '(' exp ')' + { $$ = exp_unop (ABSOLUTE, $3); } + | ALIGN_K '(' exp ')' +@t}\vb{\resetf}{@> + { $$ = exp_unop (ALIGN_K,$3); } +@t}\vb{\flatten\inline}{@> + | ALIGN_K '(' exp ',' exp ')' + { $$ = exp_binop (ALIGN_K,$3,$5); } + | DATA_SEGMENT_ALIGN '(' exp ',' exp ')' +@t}\vb{\resetf}{@> + { $$ = exp_binop (DATA_SEGMENT_ALIGN, $3, $5); } + | DATA_SEGMENT_RELRO_END '(' exp ',' exp ')' + { $$ = exp_binop (DATA_SEGMENT_RELRO_END, $5, $3); } + | DATA_SEGMENT_END '(' exp ')' + { $$ = exp_unop (DATA_SEGMENT_END, $3); } + | SEGMENT_START '(' NAME ',' exp ')' + { $$ = exp_binop (SEGMENT_START, + $5, + exp_nameop (NAME, $3)); } + | BLOCK '(' exp ')' + { $$ = exp_unop (ALIGN_K,$3); } + | NAME + { $$ = exp_nameop (NAME,$1); } + | MAX_K '(' exp ',' exp ')' + { $$ = exp_binop (MAX_K, $3, $5 ); } + | MIN_K '(' exp ',' exp ')' + { $$ = exp_binop (MIN_K, $3, $5 ); } + | ASSERT_K '(' exp ',' NAME ')' + { $$ = exp_assert ($3, $5); } + | ORIGIN '(' NAME ')' + { $$ = exp_nameop (ORIGIN, $3); } + | LENGTH '(' NAME ')' + { $$ = exp_nameop (LENGTH, $3); } + | LOG2CEIL '(' exp ')' + { $$ = exp_unop (LOG2CEIL, $3); } + ; + +@g + +@ @<Original \ld\ grammar rules@>= +@G +memspec_at_opt: + AT '>' NAME { $$ = $3; } + | { $$ = 0; } + ; + +opt_at: + AT '(' exp ')' { $$ = $3; } + | { $$ = 0; } + ; + +opt_align: + ALIGN_K '(' exp ')' { $$ = $3; } + | { $$ = 0; } + ; + +opt_align_with_input: + ALIGN_WITH_INPUT { $$ = ALIGN_WITH_INPUT; } + | { $$ = 0; } + ; + +opt_subalign: + SUBALIGN '(' exp ')' { $$ = $3; } + | { $$ = 0; } + ; + +sect_constraint: + ONLY_IF_RO { $$ = ONLY_IF_RO; } + | ONLY_IF_RW { $$ = ONLY_IF_RW; } + | SPECIAL { $$ = SPECIAL; } + | { $$ = 0; } + ; + +@g + +@ The \prodstyle{GROUP} case is just enough to support the \gcc\ +|svr3.ifile| script. It is not intended to be full +support. I'm not even sure what \prodstyle{GROUP} is supposed +to mean. +@<Original \ld\ grammar rules@>= +@G +section: NAME {@> @[ldlex_expression();@]@=} + opt_exp_with_type + opt_at + opt_align +@t}\vb{\breakline}{@> + opt_align_with_input + opt_subalign {@> @[ldlex_popstate (); ldlex_script ();@]@=} + sect_constraint + '{' + { + lang_enter_output_section_statement($1, $3, + sectype, + $5, $7, $4, $9, $6); + } + statement_list_opt + '}' {@> @[ldlex_popstate (); ldlex_expression ();@]@=} + memspec_opt memspec_at_opt phdr_opt fill_opt + { +@O + ldlex_popstate (); +@o + lang_leave_output_section_statement ($18, $15, $17, $16); + } + opt_comma + {} + | OVERLAY + {@> @[ldlex_expression ();@]@=} + opt_exp_without_type opt_nocrossrefs +@t}\vb{\breakline}{@> + opt_at opt_subalign + {@> @[ldlex_popstate (); ldlex_script ();@]@=} + '{' + { + lang_enter_overlay ($3, $6); + } + overlay_section + '}' + {@> @[ldlex_popstate (); ldlex_expression ();@]@=} + memspec_opt memspec_at_opt phdr_opt fill_opt + { +@O + ldlex_popstate (); +@o + lang_leave_overlay ($5, (int) $4, + $16, $13, $15, $14); + } + opt_comma + | GROUP {@> @[ldlex_expression ();@]@=} + opt_exp_with_type + { +@O + ldlex_popstate (); +@o + lang_add_assignment (exp_assign (".", $3, FALSE)); + } + '{' sec_or_group_p1 '}' + | INCLUDE filename + {@> @[ldlex_script (); ldfile_open_command_file($2);@]@=} + sec_or_group_p1 END + {@> @[ldlex_popstate ();@]@=} + ; +@t}\vb{\flatten\inline}{@> +type: + NOLOAD { sectype = noload_section; } + | DSECT { sectype = noalloc_section; } + | COPY { sectype = noalloc_section; } + | INFO { sectype = noalloc_section; } + | OVERLAY { sectype = noalloc_section; } + ; + +atype: + '(' type ')' + | { sectype = normal_section; } + | '(' ')' { sectype = normal_section; } + ; + +@g + +@ The \prodstyle{BIND} cases are to support the \gcc\ |svr3.ifile| +script. They aren't intended to implement full +support for the \prodstyle{BIND} keyword. I'm not even sure +what \prodstyle{BIND} is supposed to mean. +@<Original \ld\ grammar rules@>= +@G + +opt_exp_with_type: + exp atype ':' { $$ = $1; } + | atype ':' { $$ = (etree_type *)NULL; } + | BIND '(' exp ')' atype ':' { $$ = $3; } + | BIND '(' exp ')' BLOCK '(' exp ')' atype ':' + { $$ = $3; } + ; + +opt_exp_without_type: + exp ':' { $$ = $1; } + | ':' { $$ = (etree_type *) NULL; } + ; + +opt_nocrossrefs: + + { $$ = 0; } + | NOCROSSREFS + { $$ = 1; } + ; + +memspec_opt: + '>' NAME + { $$ = $2; } + | { $$ = DEFAULT_MEMORY_REGION; } + ; + +phdr_opt: + + { + $$ = NULL; + } + | phdr_opt ':' NAME + { + struct lang_output_section_phdr_list *n; + + n = ((struct lang_output_section_phdr_list *) + xmalloc (sizeof *n)); + n->name = $3; + n->used = FALSE; + n->next = $1; + $$ = n; + } + ; + +overlay_section: + + | overlay_section + NAME + { +@O + ldlex_script (); +@o + lang_enter_overlay_section ($2); + } + '{' statement_list_opt '}' + {@> @[ldlex_popstate (); ldlex_expression ();@]@=} + phdr_opt fill_opt + { +@O + ldlex_popstate (); +@o + lang_leave_overlay_section ($9, $8); + } + opt_comma + ; +@g + +@ @<Original \ld\ grammar rules@>= +@G +phdrs: + PHDRS '{' phdr_list '}' + ; + +phdr_list: + + | phdr_list phdr + ; + +phdr: + NAME {@> @[ldlex_expression ();@]@=} + phdr_type phdr_qualifiers {@> @[ldlex_popstate ();@]@=} + ';' + { + lang_new_phdr ($1, $3, $4.filehdr, $4.phdrs, $4.at, + $4.flags); + } + ; + +phdr_type: + exp + { + $$ = $1; + + if ($1->type.node_class == etree_name + && $1->type.node_code == NAME) + { + const char *s; + unsigned int i; + static const char * const phdr_types[] = + { + "PT_NULL", "PT_LOAD", "PT_DYNAMIC", + "PT_INTERP", "PT_NOTE", "PT_SHLIB", + "PT_PHDR", "PT_TLS" + }; + + s = $1->name.name; + for (i = 0; + i < sizeof phdr_types / sizeof phdr_types[0]; + i++) + if (strcmp (s, phdr_types[i]) == 0) + { + $$ = exp_intop (i); + break; + } + if (i == sizeof phdr_types / sizeof phdr_types[0]) + { + if (strcmp (s, "PT_GNU_EH_FRAME") == 0) + $$ = exp_intop (0x6474e550); + else if (strcmp (s, "PT_GNU_STACK") == 0) + $$ = exp_intop (0x6474e551); + else + { + einfo (_("\ +%X%P:%S: unknown phdr type `%s' (try integer literal)\n"), + NULL, s); + $$ = exp_intop (0); + } + } + } + } + ; + +phdr_qualifiers: + + { + memset (&$$, 0, sizeof (struct phdr_info)); + } + | NAME phdr_val phdr_qualifiers + { + $$ = $3; + if (strcmp ($1, "FILEHDR") == 0 && $2 == NULL) + $$.filehdr = TRUE; + else if (strcmp ($1, "PHDRS") == 0 && $2 == NULL) + $$.phdrs = TRUE; + else if (strcmp ($1, "FLAGS") == 0 && $2 != NULL) + $$.flags = $2; + else + einfo (_("%X%P:%S: PHDRS syntax error at `%s'\n"), + NULL, $1); + } + | AT '(' exp ')' phdr_qualifiers + { + $$ = $5; + $$.at = $3; + } + ; + +phdr_val: + + { + $$ = NULL; + } + | '(' exp ')' + { + $$ = $2; + } + ; + +dynamic_list_file: + { +@O + ldlex_version_file (); +@o + PUSH_ERROR (_("dynamic list")); + } + dynamic_list_nodes + { +@O + ldlex_popstate (); +@o + POP_ERROR (); + } + ; + +dynamic_list_nodes: + dynamic_list_node + | dynamic_list_nodes dynamic_list_node + ; + +dynamic_list_node: + '{' dynamic_list_tag '}' ';' + ; + +dynamic_list_tag: + vers_defns ';' + { + lang_append_dynamic_list ($1); + } + ; + +@g + +@ This syntax is used within an external version script file. +@<Original \ld\ grammar rules@>= +@G + +version_script_file: + { +@O + ldlex_version_file (); +@o + PUSH_ERROR (_("VERSION script")); + } + vers_nodes + { +@O + ldlex_popstate (); +@o + POP_ERROR (); + } + ; +@g + +@ This is used within a normal linker script file. +@<Original \ld\ grammar rules@>= +@G +version: + { +@O + ldlex_version_script (); +@o + } + VERSIONK '{' vers_nodes '}' + { +@O + ldlex_popstate (); +@o + } + ; + +vers_nodes: + vers_node + | vers_nodes vers_node + ; + +vers_node: + '{' vers_tag '}' ';' + { + lang_register_vers_node (NULL, $2, NULL); + } + | VERS_TAG '{' vers_tag '}' ';' + { + lang_register_vers_node ($1, $3, NULL); + } + | VERS_TAG '{' vers_tag '}' verdep ';' + { + lang_register_vers_node ($1, $3, $5); + } + ; + +verdep: + VERS_TAG + { + $$ = lang_add_vers_depend (NULL, $1); + } + | verdep VERS_TAG + { + $$ = lang_add_vers_depend ($1, $2); + } + ; + +vers_tag: + + { + $$ = lang_new_vers_node (NULL, NULL); + } + | vers_defns ';' + { + $$ = lang_new_vers_node ($1, NULL); + } + | GLOBAL ':' vers_defns ';' + { + $$ = lang_new_vers_node ($3, NULL); + } + | LOCAL ':' vers_defns ';' + { + $$ = lang_new_vers_node (NULL, $3); + } + | GLOBAL ':' vers_defns ';' LOCAL ':' vers_defns ';' + { + $$ = lang_new_vers_node ($3, $7); + } + ; + +vers_defns: + VERS_IDENTIFIER + { + $$ = lang_new_vers_pattern (NULL, $1, ldgram_vers_current_lang, FALSE); + } + | NAME + { + $$ = lang_new_vers_pattern (NULL, $1, ldgram_vers_current_lang, TRUE); + } + | vers_defns ';' VERS_IDENTIFIER + { + $$ = lang_new_vers_pattern ($1, $3, ldgram_vers_current_lang, FALSE); + } + | vers_defns ';' NAME + { + $$ = lang_new_vers_pattern ($1, $3, ldgram_vers_current_lang, TRUE); + } + | vers_defns ';' EXTERN NAME '{' + { + $<name>$ = ldgram_vers_current_lang; + ldgram_vers_current_lang = $4; + } + vers_defns opt_semicolon '}' + { + struct bfd_elf_version_expr *pat; + for (pat = $7; pat->next != NULL; pat = pat->next); + pat->next = $1; + $$ = $7; + ldgram_vers_current_lang = $<name>6; + } + | EXTERN NAME '{' + { + $<name>$ = ldgram_vers_current_lang; + ldgram_vers_current_lang = $2; + } + vers_defns opt_semicolon '}' + { + $$ = $5; + ldgram_vers_current_lang = $<name>4; + } + | GLOBAL + { + $$ = lang_new_vers_pattern (NULL, "global", ldgram_vers_current_lang, FALSE); + } + | vers_defns ';' GLOBAL + { + $$ = lang_new_vers_pattern ($1, "global", ldgram_vers_current_lang, FALSE); + } + | LOCAL + { + $$ = lang_new_vers_pattern (NULL, "local", ldgram_vers_current_lang, FALSE); + } + | vers_defns ';' LOCAL + { + $$ = lang_new_vers_pattern ($1, "local", ldgram_vers_current_lang, FALSE); + } + | EXTERN + { + $$ = lang_new_vers_pattern (NULL, "extern", ldgram_vers_current_lang, FALSE); + } + | vers_defns ';' EXTERN + { + $$ = lang_new_vers_pattern ($1, "extern", ldgram_vers_current_lang, FALSE); + } + ; +@t}\vb{\inline\flatten}{@> +opt_semicolon: + + | ';' + ; + +@g + +@q%%@> + +@ \Cee\ sugar. +@c +void +yyerror(arg) + const char *arg; +{ + if (ldfile_assumed_script) + einfo (_("%P:%s: file format not recognized; treating as linker script\n"), + ldlex_filename ()); + if (error_index > 0 && error_index < ERROR_NAME_MAX) + einfo ("%P%F:%S: %s in %s\n", NULL, arg, error_names[error_index - 1]); + else + einfo ("%P%F:%S: %s\n", NULL, arg); +} + diff --git a/support/splint/examples/ld/ldint.sty b/support/splint/examples/ld/ldint.sty new file mode 100644 index 0000000000..478bdef17d --- /dev/null +++ b/support/splint/examples/ld/ldint.sty @@ -0,0 +1,259 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% Simple integer parsing macros to display various styles of integers +% used in ld scripts. Instead of creating a dedicated parser for this +% task, it was decided to implement it as a set of expandable +% macros. The degree of complexity of both implementations would +% probably be about the same, and a true parser approach will be more +% flexible. The choice was made in favor of a macro implementation to +% ensure expandability and speed. + +\def\ldintiterator#1#2#3#4#5#6#7#8{% + % #1 a control sequence to insert at the end + % #2 first divisor + % #3 second divisor + % #4 first remainder + % #5 second remainder + % #6 read digits + % #7 last read digit + % #8 current digit + \ifcat\noexpand#8\relax + \yybreak{#1{#2}{#3}{#4}{#5}{#6}{#7}}% + \else + \yybreak{\expandafter\lditeratormodone\expandafter{\number\incrementmod{#4}{#2}}{#5}{#3}{#1}{#2}{#3}{#6#7}{#8}}% + \yycontinue +} + +\def\incrementmod#1#2{% + \expandafter\incr@mentmod\expandafter{\number\xincrement{#1}}{#2}% +} + +\def\incr@mentmod#1#2{% + \ifnum#1=#2 + \yybreak{0}% + \else + \yybreak{#1}% + \yycontinue +} + +\def\decrementmod#1#2{% + \ifnum#1=\z@ + \yybreak{\xdecrement{#2}}% + \else + \yybreak{\xdecrement{#1}}% + \yycontinue +} + +\def\lditeratormodone#1#2#3{% + \expandafter\lditeratormodtwo\expandafter{\number\incrementmod{#2}{#3}}{#1}% +} + +\def\lditeratormodtwo#1#2#3#4#5{% + \ldintiterator{#3}{#4}{#5}{#2}{#1}% +} + +\def\ldintegerspacingmodone#1#2#3#4#5#6#7#8#9{% + % #1 control sequence to insert at the end + % #2 first divisor + % #3 second divisor + % #4 number of digits mod #2 + % #5 number of digits mod #3 + % #6 digits read + % #7 separator + % #8 last digit + % #9 current digit + \ifcat\noexpand#9\relax + \yybreak{#1{#2}{#3}{#4}{#5}{#6}{#8}}% + \else + \yybreak{\ldintegerspacingmodon@{#1}{#2}{#3}{#4}{#5}{#6}{#7}{#8}{#9}}% + \yycontinue +} + +\def\ldintegerspacingmodon@#1#2#3#4#5#6#7#8#9{% + \ifnum#4=\z@ + \yybreak{% + \yystringempty{#8}{% + \expandafter\ldintegerspacingmod@n@\expandafter{\number\decrementmod{#4}{#2}}{#1}{#2}{#3}{#5}{}{#7}{#9}% + }{% + \expandafter\ldintegerspacingmod@n@\expandafter{\number\decrementmod{#4}{#2}}{#1}{#2}{#3}{#5}{#6#8#7}{#7}{#9}% + }% + }% + \else + \yybreak{% + \expandafter\ldintegerspacingmod@n@\expandafter{\number\decrementmod{#4}{#2}}{#1}{#2}{#3}{#5}{#6#8}{#7}{#9}% + }% + \yycontinue +} + +\def\ldintegerspacingmod@n@#1#2#3#4#5#6#7#8{% + \ldintegerspacingmodone{#2}{#3}{#4}{#1}{#5}{#6}{#7}{#8}% +} + +\def\showiteratorresults#1#2#3#4#5#6{% + \errmessage{mod #1: #3, mod #2: #4, digits: #5, last digit: #6}% +} + +\def\showseparatorresults#1#2#3#4#5#6#7{% + \errmessage{digits after separation: \yystringempty{#1}{}{[#1]}#6.#7}% +} + +\def\ldseparatemodone#1#2#3#4#5#6#7{% + \expandafter\yystringempty\expandafter{\romannumeral-"0#7}{% + \ldintegerspacingmodone{#1{}}{#2}{#3}{#4}{#5}{}{\ldintsep}{}#6#7\end + }{% + \ldseparatewithsuffix{#1}{#2}{#3}{#4}{#5}{#6}{#7}% + }% +} + +\def\ldseparatewithsuffix#1#2#3#4#5#6#7{% + \ifcase\csname ldsuffix#7\endcsname\space + \expandafter\ldseparatewiths@ffix\expandafter{\number\decrementmod{#4}{#2}}{#1}{#2}{#3}{#5}{#6}{#7}% + \or + \expandafter\ldseparatewiths@ffix\expandafter{\number\decrementmod{#5}{#3}}{#1}{#3}{#2}{#4}{#6}{#7}% + \else + \fi +} + +\def\ldseparatewiths@ffix#1#2#3#4#5#6#7{% + \ldintegerspacingmodone{#2{#7}}{#3}{#4}{#1}{#5}{}{\ldintsep}{}#6\end +} + +\def\ldsuffixD{0} +\def\ldsuffixO{0} +\def\ldsuffixB{1} +\def\ldsuffixH{1} +\def\ldsuffixX{1} +% +\def\ldradixD{} +\def\ldradixO{8} +\def\ldradixB{01} +\def\ldradixH{16} +\def\ldradixX{16} +% +\def\ldsuffixK{0} +\def\ldsuffixM{0} +\def\ldsuffixG{0} + +\let\ldintsep\relax + +\def\intprefix#1{% analyzing the prefix of an integer + \intpr@fix#1..\end +} + +\def\intpr@fix#1#2#3\end{% + \if#10% + \if#2X% prefix 0X + 1% hex number + \else + \if#2.% + 4% zero + \else + 3% octal number + \fi + \fi + \else + \if#1$% + 2% hex number + \else + 0% decimal number (no prefix) + \fi + \fi +} + +\def\ldsciinteger#1{% + \ifcase\intprefix{#1} + % decimal number (no prefix) + \lddecsplitws{#1}{}% + \or % hex number (0X) + \expandafter\ldhexsplitws\expandafter{\eattwo#1}{16}% + \or % hex number ($) + \expandafter\ldhexsplitws\expandafter{\eatone#1}{16}% + \or % octal number + \lddecsplitws{#1}{8}% + \or % zero + 0% + \else + \fi +} + +\def\ldhexsplitws#1#2{% + \ldintiterator{\ldseparatemodone{\displayinteger{#2}}}{4}{4}{0}{0}{}{}#1\end +} + +\def\lddecsplitws#1#2{% + \ldintiterator{\ldseparatemodone{\displayinteger{#2}}}{3}{3}{0}{0}{}{}#1\end +} + +\def\ldbasedinteger#1{% + \ldintiterator{\ldseparatewithsuffix{\displayintegerws}}{3}{4}{0}{0}{}{}#1\end +} + +\def\displayinteger#1#2#3#4#5#6#7#8{% + {\def\ldintsep{$\,$}\hbox{\tt#7#8${}_{#1}\yystringempty{#2}{}{\,\hbox{\tt#2}}$}}% +} + +\def\displayintegerws#1#2#3#4#5#6#7{% + {\def\ldintsep{$\,$}\hbox{\tt#6#7${}_{\yystringempty{#1}{}{\csname ldradix#1\endcsname}}$}}% +} + +% typeseting examples in text + +\def\beginldprod{% +% \par + \begingroup + \b@ginldprod +} + +\long\def\b@ginldprod#1\endprod{% + \setldproduction{#1}% + \endgroup +% \par +} + +\long\def\setldproduction#1{% + \def\termidxrank{5}% + \def\headeridxrank{4}% + \def\defidxrank{3}% + \def\texcsidxrank{5}% + \ninepoint + \let\returnexplicitspace\splitexplicitspace + \let\acharswitch\texcharadjust + \let\onecharswitch\texcsadjust + \let\yyinputgroup\yyinputldgroup + \toldparser + \ldparserinit + \yyparse#1\yyeof\yyeof\endparseinput\endparse + \ifyyparsefail % revert to generic macros if parsing failed + \yybreak{\toks0{#1}\errmessage{failed to parse: \the\toks0}}% + \else % Stage three, process the parsed table + \yybreak{% + {% + \restorecslist{ld-display}\ldunion + \the\ldcmds +% \par + \the\lddisplay + }% + }% + \yycontinue +} + +% to make it possible to write {...} without changes + +\def\yyinputldgroup#1{% + \yyinput\{#1\}% +} + diff --git a/support/splint/examples/ld/ldlex.w b/support/splint/examples/ld/ldlex.w new file mode 100644 index 0000000000..9d1c256e62 --- /dev/null +++ b/support/splint/examples/ld/ldlex.w @@ -0,0 +1,550 @@ +@q Copyright 2012-2015 Alexander Shibakov@> +@q Copyright 2002-2014 Free Software Foundation, Inc.@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@** The lexer. +\ifx\parsernamespace\UNDEFINED + \input ldman.sty + \input limbo.sty + \input dcols.sty + \setupfootnotes + \def\MRI{} + \def\ld{} +\fi +The lexer used by \ld\ is almost straightforward. There are a few +facilities (\Cee\ header files, some output functions) needed by the +lexer that are conviniently coded into the \Cee\ code run by the +driver routines that make the lexer more complex than it should have +been but the function of each such facility can be easily clarified +using this documentation and occasionally referring to the manual for +the \bison\ parser which is part of this distribution. +@(ldl.ll@>= +@G + @> @<\ld\ lexer definitions@> @= +%{@> @<\ld\ lexer \Cee\ preamble@> @=%} + @> @<\ld\ lexer options@> @= +%% + @> @<\ld\ token regular expressions@> @= +%% +@O +void define_all_states( void ) { + @<Collect state definitions for the \ld\ lexer@>@; +} +@o +@g + +@ @<\ld\ lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="ldl.c" +@g + +@ @<\ld\ lexer \Cee\ preamble@>= +#include <stdint.h> +#include <stdbool.h> + +@ @<Collect state definitions for the \ld\ lexer@>= +#define _register_name( name ) @[Define_State( #name, name )@] +#include "ldl_states.h" +#undef _register_name + +@ \yyskipparsetrue The character classes used by the scanner as well as +lexer state declarations have been put in the definitions section of +the input file. No attempt has been made to clean up the definitions +of the character classes. +@<\ld\ lexer definitions@>= +@<\ld\ lexer states@>@; +@G +CMDFILENAMECHAR [_a-zA-Z0-9\/\.\\_\+\$\:\[\]\\\,\=\&\!\<\>\-\~] +CMDFILENAMECHAR1 [_a-zA-Z0-9\/\.\\_\+\$\:\[\]\\\,\=\&\!\<\>\~] +FILENAMECHAR1 [_a-zA-Z\/\.\\\$\_\~] +SYMBOLCHARN [_a-zA-Z\/\.\\\$\_\~0-9] +FILENAMECHAR [_a-zA-Z0-9\/\.\-\_\+\=\$\:\[\]\\\,\~] +WILDCHAR [_a-zA-Z0-9\/\.\-\_\+\=\$\:\[\]\\\,\~\?\*\^\!] +WHITE [ \t\n\r]+ +NOCFILENAMECHAR [_a-zA-Z0-9\/\.\-\_\+\$\:\[\]\\\~] +V_TAG [.$_a-zA-Z][._a-zA-Z0-9]* +V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)* +@g + +@ The lexer uses different sets of rules depending on the context and the current state. +These can be changed from the lexer itself or externally by the parser +(as is the case in \ld\ +implementation). \locallink{stateswitchers}Later\endlink, a number of +helper macros implement state switching so that the state names are +very rarely used explicitly. Keeping all the state declarations in the +same section simplifies the job of the +\locallink{bootstrapstates}bootstrap parser\endlink, as well. +\ifbootstrapmode\immediate\openout\stlist=ldl_states.h\fi +\yyskipparsefalse +@<\ld\ lexer states@>= +@G +%s SCRIPT +%s EXPRESSION +%s BOTH +%s DEFSYMEXP +%s MRI +%s VERS_START +%s VERS_SCRIPT +%s VERS_NODE +@g + +@*1 Macros for lexer functions. +The \locallink{pingpong}state switching\endlink\ `ping-pong' between the lexer and the parser aside, +the \ld\ lexer is very traditional. One implementation choice +deserving some attenion is the treatment of comments by the lexer. The +difficulty of implementing \Cee\ style comment lexing using regular +expressions is well-known so an often used alternative is a +special function that simply skips to the end of the comment. This is +exactly what the \ld\ lexer does with an aptly named |comment()| +function. The typesetting parser uses the \.{\\ldcomment} macro for +the same purpose. For the curious, here is a \flex\ style regular +expression defining \Cee\ comments\footnote{Taken from W.~McKeeman's site +at +\url{http://www.cs.dartmouth.edu/~mckeeman/cs118/assignments/comment.html} +and adopted to \flex\ syntax.}: +$$ +\hbox{\.{"/*" ("/"\yl[\^*/]\yl"*"+[\^*/])* "*"+ "/"}} +$$ +This expression does not handle {\it every\/} practical situation, +however, since it assumes that the end of line character can be +matched like any other. Neither does it detect some often made +mistakes such as attempting to nest comments. A few minor +modifications can fix this deficiency, as well as add some error +handling, however, for the sake of consistency, the approach taken +here mirrors the one in the original \ld. + +The top level of the \.{\\ldcomment} macro simply bypasses the state +setup of the lexer and enters a `|while| loop' in the input +routine. This macro is a reasonable approximation of the functionality +provided by |comment()|. +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldcomment{% + \let\oldyyreturn\yyreturn + \let\oldyylextail\yylextail + \let\yylextail\yymatch %/* start inputting characters until {\tt *}{\tt /} is seen */ + \let\yyreturn\ldcommentskipchars +} +@g + +@ The rest of the |while| loop merely waits for the \.{*/} combination. +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldcommentskipchars{% + \ifnum\yycp@@=`* + \yybreak{\let\yyreturn\ldcommentseekslash\yyinput}% + %/* {\tt *} found, look for {\tt /} */ + \else + \yybreak{\yyinput}% %/* keep skipping characters */ + \yycontinue +}% + +\def\ldcommentseekslash{% + \ifnum\yycp@@=`/ + \yybreak{\ldcommentfinish}%/* {\tt /} found, exit */ + \else + \ifnum\yycp@@=`* + \yybreak@@{\yyinput}% %/* keep skipping {\tt *}'s looking for a {\tt /} */ + \else + \yybreak@@{\let\yyreturn\ldcommentskipchars\yyinput}% + %/* found a character other than {\tt *} or {\tt /} */ + \fi + \yycontinue +}% +@g + +@ Once the end of the comment has been found, resume lexing the input +stream. +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldcommentfinish{% + \let\yyreturn\oldyyreturn + \let\yylextail\oldyylextail + \yylextail +} +@g + +@ The semantics of the macros defined above do not quite match that +of the |comment()| function. The most significant difference is that +the portion of the action following \.{\\ldcomment} expands {\it +before\/} the comment characters are skipped. In most applications, +|comment()| is the last function called so this would not limit the use +of \.{\\ldcomment} too dramatically. + +A more intuitive and easier to use version of \.{\\ldcomment} is +possible, however, if \.{\\yylextail} is not used inside actions (in the case of +an `optimized' lexer the restriction is even weaker, namely, +\.{\\yylextail} merely has to be absent in the portion of the action +following \.{\\ldcomment}). +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldcomment#1\yylextail{% + \let\oldyyreturn\yyreturn + \def\yylexcontinuation{#1\yylextail}% + \let\yyreturn\ldcommentskipchars %/* start inputting characters until {\tt *}{\tt /} is seen */ + \yymatch +} + +\def\ldcommentfinish{% + \let\yyreturn\oldyyreturn + \yylexcontinuation +} +@g + +@ \namedspot{pretendbufferswlex}The same idea can be applied to +`\locallink{pretendbuffersw}pretend buffer switching\endlink'. Whenever +the `real' \ld\ parser encounters an \prodstyle{INCLUDE} command, it +switches the input buffer for the lexer and waits for the lexer to +return the tokens from the file it just opened. When the lexer scans +the end of the included file, it returns a special token, \prodstyle{END} that +completes the appropriate production and lets the parser continue with +its job. + +We would like to simulate the file inclusion by inserting the +appropriate end of file marker for the lexer (a double +\.{\\yyeof}). After the relevant production completes, the marker +has to be cleaned up from the input stream (the lexer is designed to +leave it intact). The macros below are designed to handle this assignment. +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldcleanyyeof#1\yylextail{% + \let\oldyyinput\yyinput + \def\yyinput\yyeof\yyeof{\let\yyinput\oldyyinput#1\yylextail}% + \yymatch +} +@g + +@*1 Regular expressions. +The `heart' of any lexer is the collection of regular expressions that +describe the {\it tokens\/} of the appropriate language. The variey of +tokens recognized by \ld\ is quite extensive and is described in the +sections that follow. + +Variable names and algebraic operations come first. +@<\ld\ token regular expressions@>= +@G +<BOTH,SCRIPT,EXPRESSION,VERS_START,VERS_NODE,VERS_SCRIPT>"/*" {@> @[TeX_( "/ldcomment/yylexnext" );@]@=} +<DEFSYMEXP>"-" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<DEFSYMEXP>"+" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<DEFSYMEXP>{FILENAMECHAR1}{SYMBOLCHARN}* {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<DEFSYMEXP>"=" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<MRI,EXPRESSION>"$"([0-9A-Fa-f])+ {@> @<Return an absolute hex constant@> @=} +<MRI,EXPRESSION>([0-9A-Fa-f])+(H|h|X|x|B|b|O|o|D|d) {@> @<Return a constant in a specific radix@>@=} +<SCRIPT,DEFSYMEXP,MRI,BOTH,EXPRESSION>((("$"|0[xX])([0-9A-Fa-f])+)|(([0-9])+))(M|K|m|k)? { + @> @<Return a constant with a multiplier@>@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"]" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"[" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<<=" {@> @[TeX_( "/yylexreturnptr{LSHIFTEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">>=" {@> @[TeX_( "/yylexreturnptr{RSHIFTEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"||" {@> @[TeX_( "/yylexreturnptr{OROR}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"==" {@> @[TeX_( "/yylexreturnptr{EQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"!=" {@> @[TeX_( "/yylexreturnptr{NE}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">=" {@> @[TeX_( "/yylexreturnptr{GE}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<=" {@> @[TeX_( "/yylexreturnptr{LE}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<<" {@> @[TeX_( "/yylexreturnptr{LSHIFT}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">>" {@> @[TeX_( "/yylexreturnptr{RSHIFT}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"+=" {@> @[TeX_( "/yylexreturnptr{PLUSEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"-=" {@> @[TeX_( "/yylexreturnptr{MINUSEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"*=" {@> @[TeX_( "/yylexreturnptr{MULTEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"/=" {@> @[TeX_( "/yylexreturnptr{DIVEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&=" {@> @[TeX_( "/yylexreturnptr{ANDEQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"|=" {@> @[TeX_( "/yylexreturnptr{OREQ}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&&" {@> @[TeX_( "/yylexreturnptr{ANDAND}" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"," {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"|" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"~" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"!" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"?" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"*" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"+" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"-" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"/" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"%" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"=" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"}" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"{" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>")" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"(" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>":" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>";" {@> @[TeX_( "/yylexreturnchar" );@]@=} +@g + +@ The bulk of tokens produced by the lexer are the keywords used +inside script files. File name syntax is listed as well, along with +miscellanea such as whitespace and version symbols. +@<\ld\ token regular expressions@>= +@G +<BOTH,SCRIPT>"MEMORY" {@> @[TeX_( "/yylexreturnptr{MEMORY}" );@]@=} +<BOTH,SCRIPT>"REGION_ALIAS" {@> @[TeX_( "/yylexreturnptr{REGION_ALIAS}" );@]@=} +<BOTH,SCRIPT>"LD_FEATURE" {@> @[TeX_( "/yylexreturnptr{LD_FEATURE}" );@]@=} +<BOTH,SCRIPT,EXPRESSION>"ORIGIN" {@> @[TeX_( "/yylexreturnptr{ORIGIN}" );@]@=} +<BOTH,SCRIPT>"VERSION" {@> @[TeX_( "/yylexreturnptr{VERSIONK}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"BLOCK" {@> @[TeX_( "/yylexreturnptr{BLOCK}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"BIND" {@> @[TeX_( "/yylexreturnptr{BIND}" );@]@=} +<BOTH,SCRIPT,EXPRESSION>"LENGTH" {@> @[TeX_( "/yylexreturnptr{LENGTH}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGN" {@> @[TeX_( "/yylexreturnptr{ALIGN_K}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_ALIGN" {@> @[TeX_( "/yylexreturnptr{DATA_SEGMENT_ALIGN}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_RELRO_END" {@> @[TeX_( "/yylexreturnptr{DATA_SEGMENT_RELRO_END}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_END" {@> @[TeX_( "/yylexreturnptr{DATA_SEGMENT_END}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ADDR" {@> @[TeX_( "/yylexreturnptr{ADDR}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"LOADADDR" {@> @[TeX_( "/yylexreturnptr{LOADADDR}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGNOF" {@> @[TeX_( "/yylexreturnptr{ALIGNOF}" );@]@=} +<EXPRESSION,BOTH>"MAX" {@> @[TeX_( "/yylexreturnptr{MAX_K}" );@]@=} +<EXPRESSION,BOTH>"MIN" {@> @[TeX_( "/yylexreturnptr{MIN_K}" );@]@=} +<EXPRESSION,BOTH>"LOG2CEIL" {@> @[TeX_( "/yylexreturnptr{LOG2CEIL}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ASSERT" {@> @[TeX_( "/yylexreturnptr{ASSERT_K}" );@]@=} +<BOTH,SCRIPT>"ENTRY" {@> @[TeX_( "/yylexreturnptr{ENTRY}" );@]@=} +<BOTH,SCRIPT,MRI>"EXTERN" {@> @[TeX_( "/yylexreturnptr{EXTERN}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"NEXT" {@> @[TeX_( "/yylexreturnptr{NEXT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"sizeof_headers" {@> @[TeX_( "/yylexreturnptr{SIZEOF_HEADERS}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"SIZEOF_HEADERS" {@> @[TeX_( "/yylexreturnptr{SIZEOF_HEADERS}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"SEGMENT_START" {@> @[TeX_( "/yylexreturnptr{SEGMENT_START}" );@]@=} +<BOTH,SCRIPT>"MAP" {@> @[TeX_( "/yylexreturnptr{MAP}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"SIZEOF" {@> @[TeX_( "/yylexreturnptr{SIZEOF}" );@]@=} +<BOTH,SCRIPT>"TARGET" {@> @[TeX_( "/yylexreturnptr{TARGET_K}" );@]@=} +<BOTH,SCRIPT>"SEARCH_DIR" {@> @[TeX_( "/yylexreturnptr{SEARCH_DIR}" );@]@=} +<BOTH,SCRIPT>"OUTPUT" {@> @[TeX_( "/yylexreturnptr{OUTPUT}" );@]@=} +<BOTH,SCRIPT>"INPUT" {@> @[TeX_( "/yylexreturnptr{INPUT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"GROUP" {@> @[TeX_( "/yylexreturnptr{GROUP}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"AS_NEEDED" {@> @[TeX_( "/yylexreturnptr{AS_NEEDED}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"DEFINED" {@> @[TeX_( "/yylexreturnptr{DEFINED}" );@]@=} +<BOTH,SCRIPT>"CREATE_OBJECT_SYMBOLS" {@> @[TeX_( "/yylexreturnptr{CREATE_OBJECT_SYMBOLS}" );@]@=} +<BOTH,SCRIPT>"CONSTRUCTORS" {@> @[TeX_( "/yylexreturnptr{CONSTRUCTORS}" );@]@=} +<BOTH,SCRIPT>"FORCE_COMMON_ALLOCATION" {@> @[TeX_( "/yylexreturnptr{FORCE_COMMON_ALLOCATION}" );@]@=} +<BOTH,SCRIPT>"INHIBIT_COMMON_ALLOCATION" {@> @[TeX_( "/yylexreturnptr{INHIBIT_COMMON_ALLOCATION}" );@]@=} +<BOTH,SCRIPT>"SECTIONS" {@> @[TeX_( "/yylexreturnptr{SECTIONS}" );@]@=} +<BOTH,SCRIPT>"INSERT" {@> @[TeX_( "/yylexreturnptr{INSERT_K}" );@]@=} +<BOTH,SCRIPT>"AFTER" {@> @[TeX_( "/yylexreturnptr{AFTER}" );@]@=} +<BOTH,SCRIPT>"BEFORE" {@> @[TeX_( "/yylexreturnptr{BEFORE}" );@]@=} +<BOTH,SCRIPT>"FILL" {@> @[TeX_( "/yylexreturnptr{FILL}" );@]@=} +<BOTH,SCRIPT>"STARTUP" {@> @[TeX_( "/yylexreturnptr{STARTUP}" );@]@=} +<BOTH,SCRIPT>"OUTPUT_FORMAT" {@> @[TeX_( "/yylexreturnptr{OUTPUT_FORMAT}" );@]@=} +<BOTH,SCRIPT>"OUTPUT_ARCH" {@> @[TeX_( "/yylexreturnptr{OUTPUT_ARCH}" );@]@=} +<BOTH,SCRIPT>"HLL" {@> @[TeX_( "/yylexreturnptr{HLL}" );@]@=} +<BOTH,SCRIPT>"SYSLIB" {@> @[TeX_( "/yylexreturnptr{SYSLIB}" );@]@=} +<BOTH,SCRIPT>"FLOAT" {@> @[TeX_( "/yylexreturnptr{FLOAT}" );@]@=} +<BOTH,SCRIPT>"QUAD" {@> @[TeX_( "/yylexreturnptr{QUAD}" );@]@=} +<BOTH,SCRIPT>"SQUAD" {@> @[TeX_( "/yylexreturnptr{SQUAD}" );@]@=} +<BOTH,SCRIPT>"LONG" {@> @[TeX_( "/yylexreturnptr{LONG}" );@]@=} +<BOTH,SCRIPT>"SHORT" {@> @[TeX_( "/yylexreturnptr{SHORT}" );@]@=} +<BOTH,SCRIPT>"BYTE" {@> @[TeX_( "/yylexreturnptr{BYTE}" );@]@=} +<BOTH,SCRIPT>"NOFLOAT" {@> @[TeX_( "/yylexreturnptr{NOFLOAT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"NOCROSSREFS" {@> @[TeX_( "/yylexreturnptr{NOCROSSREFS}" );@]@=} +<BOTH,SCRIPT>"OVERLAY" {@> @[TeX_( "/yylexreturnptr{OVERLAY}" );@]@=} +<BOTH,SCRIPT>"SORT_BY_NAME" {@> @[TeX_( "/yylexreturnptr{SORT_BY_NAME}" );@]@=} +<BOTH,SCRIPT>"SORT_BY_ALIGNMENT" {@> @[TeX_( "/yylexreturnptr{SORT_BY_ALIGNMENT}" );@]@=} +<BOTH,SCRIPT>"SORT" {@> @[TeX_( "/yylexreturnptr{SORT_BY_NAME}" );@]@=} +<BOTH,SCRIPT>"SORT_BY_INIT_PRIORITY" {@> @[TeX_( "/yylexreturnptr{SORT_BY_INIT_PRIORITY}" );@]@=} +<BOTH,SCRIPT>"SORT_NONE" {@> @[TeX_( "/yylexreturnptr{SORT_NONE}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"NOLOAD" {@> @[TeX_( "/yylexreturnptr{NOLOAD}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"DSECT" {@> @[TeX_( "/yylexreturnptr{DSECT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"COPY" {@> @[TeX_( "/yylexreturnptr{COPY}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"INFO" {@> @[TeX_( "/yylexreturnptr{INFO}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"OVERLAY" {@> @[TeX_( "/yylexreturnptr{OVERLAY}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ONLY_IF_RO" {@> @[TeX_( "/yylexreturnptr{ONLY_IF_RO}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ONLY_IF_RW" {@> @[TeX_( "/yylexreturnptr{ONLY_IF_RW}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"SPECIAL" {@> @[TeX_( "/yylexreturnptr{SPECIAL}" );@]@=} +<BOTH,SCRIPT>"o" {@> @[TeX_( "/yylexreturnptr{ORIGIN}" );@]@=} +<BOTH,SCRIPT>"org" {@> @[TeX_( "/yylexreturnptr{ORIGIN}" );@]@=} +<BOTH,SCRIPT>"l" {@> @[TeX_( "/yylexreturnptr{LENGTH}" );@]@=} +<BOTH,SCRIPT>"len" {@> @[TeX_( "/yylexreturnptr{LENGTH}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"INPUT_SECTION_FLAGS" {@> @[TeX_( "/yylexreturnptr{INPUT_SECTION_FLAGS}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"INCLUDE" {@> @[TeX_( "/yylexreturnptr{INCLUDE}" );@]@=} +<BOTH,SCRIPT>"PHDRS" {@> @[TeX_( "/yylexreturnptr{PHDRS}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"AT" {@> @[TeX_( "/yylexreturnptr{AT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGN_WITH_INPUT" {@> @[TeX_( "/yylexreturnptr{ALIGN_WITH_INPUT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"SUBALIGN" {@> @[TeX_( "/yylexreturnptr{SUBALIGN}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"HIDDEN" {@> @[TeX_( "/yylexreturnptr{HIDDEN}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"PROVIDE" {@> @[TeX_( "/yylexreturnptr{PROVIDE}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"PROVIDE_HIDDEN" {@> @[TeX_( "/yylexreturnptr{PROVIDE_HIDDEN}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"KEEP" {@> @[TeX_( "/yylexreturnptr{KEEP}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"EXCLUDE_FILE" {@> @[TeX_( "/yylexreturnptr{EXCLUDE_FILE}" );@]@=} +<EXPRESSION,BOTH,SCRIPT>"CONSTANT" {@> @[TeX_( "/yylexreturnptr{CONSTANT}" );@]@=} +<MRI>"#".*\n? {@> @[TeX_( "/yylexnext" );@]@=} +<MRI>"\n" {@> @[TeX_( "/yylexreturnptr{NEWLINE}" );@]@=} +<MRI>"*".* {@> @[TeX_( "/yylexnext" );@]@=} +<MRI>";".* {@> @[TeX_( "/yylexnext" );@]@=} +<MRI>"END" {@> @[TeX_( "/yylexreturnptr{ENDWORD}" );@]@=} +<MRI>"ALIGNMOD" {@> @[TeX_( "/yylexreturnptr{ALIGNMOD}" );@]@=} +<MRI>"ALIGN" {@> @[TeX_( "/yylexreturnptr{ALIGN_K}" );@]@=} +<MRI>"CHIP" {@> @[TeX_( "/yylexreturnptr{CHIP}" );@]@=} +<MRI>"BASE" {@> @[TeX_( "/yylexreturnptr{BASE}" );@]@=} +<MRI>"ALIAS" {@> @[TeX_( "/yylexreturnptr{ALIAS}" );@]@=} +<MRI>"TRUNCATE" {@> @[TeX_( "/yylexreturnptr{TRUNCATE}" );@]@=} +<MRI>"LOAD" {@> @[TeX_( "/yylexreturnptr{LOAD}" );@]@=} +<MRI>"PUBLIC" {@> @[TeX_( "/yylexreturnptr{PUBLIC}" );@]@=} +<MRI>"ORDER" {@> @[TeX_( "/yylexreturnptr{ORDER}" );@]@=} +<MRI>"NAME" {@> @[TeX_( "/yylexreturnptr{NAMEWORD}" );@]@=} +<MRI>"FORMAT" {@> @[TeX_( "/yylexreturnptr{FORMAT}" );@]@=} +<MRI>"CASE" {@> @[TeX_( "/yylexreturnptr{CASE}" );@]@=} +<MRI>"START" {@> @[TeX_( "/yylexreturnptr{START}" );@]@=} +<MRI>"LIST".* {@> @[TeX_( "/yylexreturnptr{LIST}" );@]@=} +<MRI>"SECT" {@> @[TeX_( "/yylexreturnptr{SECT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT,MRI>"ABSOLUTE" {@> @[TeX_( "/yylexreturnptr{ABSOLUTE}" );@]@=} +<MRI>"end" {@> @[TeX_( "/yylexreturnptr{ENDWORD}" );@]@=} +<MRI>"alignmod" {@> @[TeX_( "/yylexreturnptr{ALIGNMOD}" );@]@=} +<MRI>"align" {@> @[TeX_( "/yylexreturnptr{ALIGN_K}" );@]@=} +<MRI>"chip" {@> @[TeX_( "/yylexreturnptr{CHIP}" );@]@=} +<MRI>"base" {@> @[TeX_( "/yylexreturnptr{BASE}" );@]@=} +<MRI>"alias" {@> @[TeX_( "/yylexreturnptr{ALIAS}" );@]@=} +<MRI>"truncate" {@> @[TeX_( "/yylexreturnptr{TRUNCATE}" );@]@=} +<MRI>"load" {@> @[TeX_( "/yylexreturnptr{LOAD}" );@]@=} +<MRI>"public" {@> @[TeX_( "/yylexreturnptr{PUBLIC}" );@]@=} +<MRI>"order" {@> @[TeX_( "/yylexreturnptr{ORDER}" );@]@=} +<MRI>"name" {@> @[TeX_( "/yylexreturnptr{NAMEWORD}" );@]@=} +<MRI>"format" {@> @[TeX_( "/yylexreturnptr{FORMAT}" );@]@=} +<MRI>"case" {@> @[TeX_( "/yylexreturnptr{CASE}" );@]@=} +<MRI>"extern" {@> @[TeX_( "/yylexreturnptr{EXTERN}" );@]@=} +<MRI>"start" {@> @[TeX_( "/yylexreturnptr{START}" );@]@=} +<MRI>"list".* {@> @[TeX_( "/yylexreturnptr{LIST}" );@]@=} +<MRI>"sect" {@> @[TeX_( "/yylexreturnptr{SECT}" );@]@=} +<EXPRESSION,BOTH,SCRIPT,MRI>"absolute" {@> @[TeX_( "/yylexreturnptr{ABSOLUTE}" );@]@=} +<MRI>{FILENAMECHAR1}{NOCFILENAMECHAR}* {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<BOTH>{FILENAMECHAR1}{FILENAMECHAR}* {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<BOTH>"-l"{FILENAMECHAR}+ {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<EXPRESSION>{FILENAMECHAR1}{NOCFILENAMECHAR}* {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<EXPRESSION>"-l"{NOCFILENAMECHAR}+ {@> @[TeX_( "/yylexreturnval{NAME}" );@]@=} +<SCRIPT>{WILDCHAR}* {@> @[@<Skip a possible comment and return a \prodstyle{NAME}@>@]@=} +<EXPRESSION,BOTH,SCRIPT,VERS_NODE>"\""[^\"]*"\"" {@> @[@<Return the \prodstyle{NAME} inside quotes@>@]@=} +<BOTH,SCRIPT,EXPRESSION>"\n" {@> @[TeX_( "/yylexnext" );@]@=} +<MRI,BOTH,SCRIPT,EXPRESSION>[ \t\r]+ {@> @[TeX_( "/yylexnext" );@]@=} +<VERS_NODE,VERS_SCRIPT>[:,;] {@> @[TeX_( "/yylexreturnchar" );@]@=} +<VERS_NODE>global {@> @[TeX_( "/yylexreturnptr{GLOBAL}" );@]@=} +<VERS_NODE>local {@> @[TeX_( "/yylexreturnptr{LOCAL}" );@]@=} +<VERS_NODE>extern {@> @[TeX_( "/yylexreturnptr{EXTERN}" );@]@=} +<VERS_NODE>{V_IDENTIFIER} {@> @[TeX_( "/yylexreturnval{VERS_IDENTIFIER}" );@]@=} +<VERS_SCRIPT>{V_TAG} {@> @[TeX_( "/yylexreturnval{VERS_TAG}" );@]@=} +<VERS_START>"{" {@> @[TeX_( "/yyBEGIN{VERS_SCRIPT}/yylexreturnchar" );@]@=} +@g + +@ There is a bit of a trick to returning an absolute hex value. The +macros are looking for a \.{\$} suffix while the contents of +\.{\\yytext} start with \.{\\\$}. +@<Return an absolute hex constant@>= +@[TeX_( "/edef/next{/yylval{/nx/hexint{$/expandafter/eatone/the/yytext}" );@]@; +@[TeX_( "{/the/yyfmark}{/the/yysmark}}}/next" );@]@; +@[TeX_( "/yylexreturn{INT}" );@]@; + +@ @<Return a constant in a specific radix@>= +@[TeX_( "/edef/next{/yylval{/nx/bint{/the/yytext}" );@]@; +@[TeX_( "{/the/yyfmark}{/the/yysmark}}}/next" );@]@; +@[TeX_( "/yylexreturn{INT}" );@]@; + +@ @<Return a constant with a multiplier@>= +@[TeX_( "/edef/next{/yylval{/nx/anint{/the/yytext}" );@]@; +@[TeX_( "{/the/yyfmark}{/the/yysmark}}}/next" );@]@; +@[TeX_( "/yylexreturn{INT}" );@]@; + +@ @<Additional macros for the \ld\ lexer/parser@>= +@G +@=\def\matchcomment@@#1/*#2\yyeof#3#4{%@>@; + \yystringempty{#1}{#3}{#4}% +} +\def\matchcomment#1{% +@= \expandafter\matchcomment@@\the#1/*\yyeof@>@; +} +\def\ldstripquotes@@"#1"\yyeof{#1} +\def\ldstripquotes#1{% + \yytext\expandafter\expandafter\expandafter + {\expandafter\ldstripquotes@@\the\yytext\yyeof}% + \yytextpure\expandafter\expandafter\expandafter + {\expandafter\ldstripquotes@@\the\yytextpure\yyeof}% +} +@g + +@ {\it Annoyingly, this pattern can match comments, and we have +longest match issues to consider. So if the first two +characters are a comment opening, put the input back and +try again.} +@<Skip a possible comment and return a \prodstyle{NAME}@>= + @[TeX_( "/matchcomment/yytextpure" );@]@; + @[TeX_( " {/yyless/tw@@/ldcomment}" );@]/*matched the beginning of a comment*/@; + @[TeX_( " {/yylexreturnval{NAME}}" );@]@; + +@ {\it No matter the state, quotes give what's inside.} +@<Return the \prodstyle{NAME} inside quotes@>= + @[TeX_( "/ldstripquotes/yylexreturnval{NAME}" );@]@; + +@ @<Additional macros for the \ld\ lexer/parser@>= +@G +\newcount\versnodenesting +\newcount\includestackptr +@g + +@ Some syntax specific to version scripts. +@<\ld\ token regular expressions@>= +@G +<VERS_SCRIPT>"{" {@> @[TeX_( "/yyBEGIN{VERS_NODE}/versnodenesting=/z@@/yylexreturnchar" );@]@=} +<VERS_SCRIPT>"}" {@> @[TeX_( "/yylexreturnchar" );@]@=} +<VERS_NODE>"{" {@> @[TeX_( "/advance/versnodenesting/@@ne /yylexreturnchar" );@]@=} +<VERS_NODE>"}" {@> @[TeX_( "/advance/versnodenesting/m@@ne" );@]@; + @> @[TeX_( "/ifnum/versnodenesting</z@@" );@]@; + @> @[TeX_( " /yyBEGIN{VERS_SCRIPT}" );@]@; + @> @[TeX_( "/fi" );@]@; + @> @[TeX_( "/yylexreturnchar" );@]@=} +<VERS_START,VERS_NODE,VERS_SCRIPT>[\n] {@> @[TeX_( "/yylexnext" );@]@=} +<VERS_START,VERS_NODE,VERS_SCRIPT>#.* {@> @[TeX_( "/yylexnext" );@]@=} +<VERS_START,VERS_NODE,VERS_SCRIPT>[ \t\r]+ {@> @[TeX_( "/yylexnext" );@]@=} + +<<EOF>> {@> @[@<Process the end of (possibly included) file@>@]@=} + +<SCRIPT,MRI,VERS_START,VERS_SCRIPT,VERS_NODE>. {@> @[TeX_( "/yycomplain{bad character `/the/yytext'" );@] + @> @[TeX_( " in script}" );@] + @> @[TeX_( "/yyerrterminate" );@]@=} +<EXPRESSION,DEFSYMEXP,BOTH>. {@> @[TeX_( "/yycomplain{bad character `/the/yytext'" );@] + @> @[TeX_( " in expression}" );@]@= + @> @[TeX_( "/yyerrterminate" );@]@=} +@g + +@ @<Process the end of (possibly included) file@>= + @[TeX_( "/advance/includestackptr/m@@ne" );@]@; + @[TeX_( "/ifnum/includestackptr=/z@@" );@]@; + @[TeX_( " /yybreak{/yyterminate}" );@]@; + @[TeX_( "/else" );@]@; + @[TeX_( " /yybreak{/ldcleanyyeof/yylexreturn{END}}" );@]@; + @[TeX_( "/yycontinue" );@]@; + +@*1 Parser-lexer interaction support. +\namedspot{stateswitchers}Here are the long promised auxiliary +macros for switching lexer states and handling file input. +@<Additional macros for the \ld\ lexer/parser@>= +@G +\def\ldlex@@script{\yypushstate{SCRIPT}} +\def\ldlex@@mri@@script{\yypushstate{MRI}} +\def\ldlex@@version@@script{\yypushstate{VERS_START}} +\def\ldlex@@version@@file{\yypushstate{VERS_SCRIPT}} +\def\ldlex@@defsym{\yypushstate{DEFSYMEXP}} +\def\ldlex@@expression{\yypushstate{EXPRESSION}} +\def\ldlex@@both{\yypushstate{BOTH}} +\let\ldlex@@popstate\yypopstate + +\def\ldfile@@open@@command@@file#1{% + \advance\includestackptr\@@ne + \appendl\yytextseen{\noexpand\yyeof\noexpand\yyeof}% + \yytextbackuptrue +} + +\def\ldlex@@filename{} +@g + diff --git a/support/splint/examples/ld/ldlexo.w b/support/splint/examples/ld/ldlexo.w new file mode 100644 index 0000000000..6b62d2fd77 --- /dev/null +++ b/support/splint/examples/ld/ldlexo.w @@ -0,0 +1,763 @@ +@q Copyright 2015 Alexander Shibakov@> +@q Copyright 2002-2015 Free Software Foundation, Inc.@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@*1 The original lexer. {\it Note that the \ld\ lexer was designed to accomodate +the syntax of various \flex\ flavors, such as the original \lex. The +options \prodstyle{\%a} and \prodstyle{\%o} are ignored by \flex\ and +are a leftover from the archaic days of the original scanner generator.} + +@s input_type int + +@<Original \ld\ lexer@>= +@G + @> @<Original \ld\ macros@> @= +%{@>@<Original \ld\ preamble@> @=%} +%option nounput + @> @<Ignored options@> @= +%% + @> @<Original \ld\ regular expressions@> @= +%% +@g + +@ @<Original \ld\ preamble@>= +#include "bfd.h" +#include "safe-ctype.h" +#include "bfdlink.h" +#include "ld.h" +#include "ldmisc.h" +#include "ldexp.h" +#include "ldlang.h" +#include <ldgram.h> +#include "ldfile.h" +#include "ldlex.h" +#include "ldmain.h" +#include "libiberty.h" + +input_type parser_input;/* The type of top-level parser input. + |yylex| and |yyparse| (indirectly) both check this. */ + +unsigned int lineno = 1;/* Line number in the current input file. */ + +const char *lex_string = NULL;/* The string we are currently lexing, or |NULL| if we are reading a + file. */ + +#undef YY_INPUT +#define YY_INPUT(buf,result,max_size) result = yy_input (buf, max_size) +/* Support for \flex\ reading from more than one input file (stream). + |include_stack| is \flex's input state for each open file; + |file_name_stack| is the file names. |lineno_stack| is the current + line numbers. + + If |include_stack_ptr| is 0, we haven't started reading anything yet. + Otherwise, stack elements 0 through |include_stack_ptr - 1| are valid. */ + + +#ifndef YY_NO_UNPUT +#define YY_NO_UNPUT +#endif + +#define MAX_INCLUDE_DEPTH 10 +static YY_BUFFER_STATE@, @[include_stack[MAX_INCLUDE_DEPTH]@]; +static const char *file_name_stack[MAX_INCLUDE_DEPTH]; +static unsigned int lineno_stack[MAX_INCLUDE_DEPTH]; +static unsigned int sysrooted_stack[MAX_INCLUDE_DEPTH]; +static unsigned int include_stack_ptr = 0; +static int vers_node_nesting = 0; + +static int yy_input (char *, int); +static void comment (void); +static void lex_warn_invalid (char *where, char *what); + +#define RTOKEN(x) { yylval.token = x; return x; } + +#ifndef yywrap +int yywrap (void) { return 1; }/* Some versions of \flex\ want this. */ +#endif + +@ @<Ignored options@>= +@G +%a 4000 +%o 5000 +@g + +@ {\it Some convenient abbreviations for regular expressions.}% +\ifbootstrapmode\else\yyskipparsetrue\fi@<Original \ld\ macros@>= +@G +CMDFILENAMECHAR [_a-zA-Z0-9\/\.\\_\+\$\:\[\]\\\,\=\&\!\<\>\-\~] +CMDFILENAMECHAR1 [_a-zA-Z0-9\/\.\\_\+\$\:\[\]\\\,\=\&\!\<\>\~] +FILENAMECHAR1 [_a-zA-Z\/\.\\\$\_\~] +SYMBOLCHARN [_a-zA-Z\/\.\\\$\_\~0-9] +FILENAMECHAR [_a-zA-Z0-9\/\.\-\_\+\=\$\:\[\]\\\,\~] +WILDCHAR [_a-zA-Z0-9\/\.\-\_\+\=\$\:\[\]\\\,\~\?\*\^\!] +WHITE [ \t\n\r]+ +NOCFILENAMECHAR [_a-zA-Z0-9\/\.\-\_\+\$\:\[\]\\\~] +V_TAG [.$_a-zA-Z][._a-zA-Z0-9]* +V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)* +@g + +@ {% +\def\aterm{\item{\sqbullet}\ignorespaces}% +\setbox0=\hbox{\sqbullet\enspace}% +\parindent=0pt +\advance\parindent by \wd0 +States: +\aterm |EXPRESSION| definitely in an expression + +\aterm |SCRIPT| definitely in a script + +\aterm |BOTH| either |EXPRESSION| or |SCRIPT| + +\aterm |DEFSYMEXP| in an argument to \.{--defsym} + +\aterm |MRI| in an \MRI\ script + +\aterm |VERS_START| starting a Sun style mapfile + +\aterm |VERS_SCRIPT| a Sun style mapfile + +\aterm |VERS_NODE| a node within a Sun style mapfile + +}% +\ifbootstrapmode\else\yyskipparsefalse\fi +@<\ld\ states@>= +@G +%s SCRIPT +%s EXPRESSION +%s BOTH +%s DEFSYMEXP +%s MRI +%s VERS_START +%s VERS_SCRIPT +%s VERS_NODE +@g + +@ @<\ld\ postamble@>= + if (parser_input != input_selected) + { + /* The first token of the input determines the initial parser state. */ + input_type t = parser_input; + parser_input = input_selected; + switch (t) + { + case input_script: return INPUT_SCRIPT; break; + case input_mri_script: return INPUT_MRI_SCRIPT; break; + case input_version_script: return INPUT_VERSION_SCRIPT; break; + case input_dynamic_list: return INPUT_DYNAMIC_LIST; break; + case input_defsym: return INPUT_DEFSYM; break; + default: abort (); + } +} + +@ @<Original \ld\ regular expressions@>= +@=<BOTH,SCRIPT,EXPRESSION,VERS_START,VERS_NODE,VERS_SCRIPT>"/*" {@> comment ();@+@=}@>@; +@G +<DEFSYMEXP>"-" {@>@[RTOKEN('-');@]@=} +<DEFSYMEXP>"+" {@>@[RTOKEN('+');@]@=} +<DEFSYMEXP>{FILENAMECHAR1}{SYMBOLCHARN}* {@>@[yylval.name = xstrdup (yytext);@+return NAME;@]@=} +<DEFSYMEXP>"=" {@>@[RTOKEN('=');@]@=} + +<MRI,EXPRESSION>"$"([0-9A-Fa-f])+ { +@O + yylval.integer = bfd_scan_vma (yytext + 1, 0, 16); + yylval.bigint.str = NULL; + return INT; +@o +} + +<MRI,EXPRESSION>([0-9A-Fa-f])+(H|h|X|x|B|b|O|o|D|d) { +@O + int ibase ; + switch (yytext[yyleng - 1]) { + case 'X': + case 'x': + case 'H': + case 'h': + ibase = 16; + break; + case 'O': + case 'o': + ibase = 8; + break; + case 'B': + case 'b': + ibase = 2; + break; + default: + ibase = 10; + } + yylval.integer = bfd_scan_vma (yytext, 0, + ibase); + yylval.bigint.str = NULL; + return INT; +@o +} + +<SCRIPT,DEFSYMEXP,MRI,BOTH,EXPRESSION>((("$"|0[xX])([0-9A-Fa-f])+)|(([0-9])+))(M|K|m|k)? { +@O + char *s = yytext; + int ibase = 0; + + if (*s == '$') + { + ++s; + ibase = 16; + } + yylval.integer = bfd_scan_vma (s, 0, ibase); + yylval.bigint.str = NULL; + if (yytext[yyleng - 1] == 'M' + || yytext[yyleng - 1] == 'm') + { + yylval.integer *= 1024 * 1024; + } + else if (yytext[yyleng - 1] == 'K' + || yytext[yyleng - 1]=='k') + { + yylval.integer *= 1024; + } + else if (yytext[0] == '0' + && (yytext[1] == 'x' + || yytext[1] == 'X')) + { + yylval.bigint.str = xstrdup (yytext + 2); + } + return INT; +@o +} + +<BOTH,SCRIPT,EXPRESSION,MRI>"]" {@>@[RTOKEN(']');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"[" {@>@[RTOKEN('[');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<<=" {@>@[RTOKEN(LSHIFTEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">>=" {@>@[RTOKEN(RSHIFTEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"||" {@>@[RTOKEN(OROR);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"==" {@>@[RTOKEN(EQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"!=" {@>@[RTOKEN(NE);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">=" {@>@[RTOKEN(GE);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<=" {@>@[RTOKEN(LE);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<<" {@>@[RTOKEN(LSHIFT);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">>" {@>@[RTOKEN(RSHIFT);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"+=" {@>@[RTOKEN(PLUSEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"-=" {@>@[RTOKEN(MINUSEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"*=" {@>@[RTOKEN(MULTEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"/=" {@>@[RTOKEN(DIVEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&=" {@>@[RTOKEN(ANDEQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"|=" {@>@[RTOKEN(OREQ);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&&" {@>@[RTOKEN(ANDAND);@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>">" {@>@[RTOKEN('>');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"," {@>@[RTOKEN(',');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"&" {@>@[RTOKEN('&');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"|" {@>@[RTOKEN('|');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"~" {@>@[RTOKEN('~');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"!" {@>@[RTOKEN('!');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"?" {@>@[RTOKEN('?');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"*" {@>@[RTOKEN('*');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"+" {@>@[RTOKEN('+');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"-" {@>@[RTOKEN('-');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"/" {@>@[RTOKEN('/');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"%" {@>@[RTOKEN('%');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"<" {@>@[RTOKEN('<');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"=" {@>@[RTOKEN('=');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"}" {@>@[RTOKEN('}') ; @]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"{" {@>@[RTOKEN('{'); @]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>")" {@>@[RTOKEN(')');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>"(" {@>@[RTOKEN('(');@]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>":" {@>@[RTOKEN(':'); @]@=} +<BOTH,SCRIPT,EXPRESSION,MRI>";" {@>@[RTOKEN(';');@]@=} +<BOTH,SCRIPT>"MEMORY" {@>@[RTOKEN(MEMORY);@]@=} +<BOTH,SCRIPT>"REGION_ALIAS" {@>@[RTOKEN(REGION_ALIAS);@]@=} +<BOTH,SCRIPT>"LD_FEATURE" {@>@[RTOKEN(LD_FEATURE);@]@=} +<BOTH,SCRIPT,EXPRESSION>"ORIGIN" {@>@[RTOKEN(ORIGIN);@]@=} +<BOTH,SCRIPT>"VERSION" {@>@[RTOKEN(VERSIONK);@]@=} +<EXPRESSION,BOTH,SCRIPT>"BLOCK" {@>@[RTOKEN(BLOCK);@]@=} +<EXPRESSION,BOTH,SCRIPT>"BIND" {@>@[RTOKEN(BIND);@]@=} +<BOTH,SCRIPT,EXPRESSION>"LENGTH" {@>@[RTOKEN(LENGTH);@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGN" {@>@[RTOKEN(ALIGN_K);@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_ALIGN" {@>@[RTOKEN(DATA_SEGMENT_ALIGN);@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_RELRO_END" {@>@[RTOKEN(DATA_SEGMENT_RELRO_END);@]@=} +<EXPRESSION,BOTH,SCRIPT>"DATA_SEGMENT_END" {@>@[RTOKEN(DATA_SEGMENT_END);@]@=} +<EXPRESSION,BOTH,SCRIPT>"ADDR" {@>@[RTOKEN(ADDR);@]@=} +<EXPRESSION,BOTH,SCRIPT>"LOADADDR" {@>@[RTOKEN(LOADADDR);@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGNOF" {@>@[RTOKEN(ALIGNOF); @]@=} +<EXPRESSION,BOTH>"MAX" {@>@[RTOKEN(MAX_K); @]@=} +<EXPRESSION,BOTH>"MIN" {@>@[RTOKEN(MIN_K); @]@=} +<EXPRESSION,BOTH>"LOG2CEIL" {@>@[RTOKEN(LOG2CEIL); @]@=} +<EXPRESSION,BOTH,SCRIPT>"ASSERT" {@>@[RTOKEN(ASSERT_K); @]@=} +<BOTH,SCRIPT>"ENTRY" {@>@[RTOKEN(ENTRY);@]@=} +<BOTH,SCRIPT,MRI>"EXTERN" {@>@[RTOKEN(EXTERN);@]@=} +<EXPRESSION,BOTH,SCRIPT>"NEXT" {@>@[RTOKEN(NEXT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"sizeof_headers" {@>@[RTOKEN(SIZEOF_HEADERS);@]@=} +<EXPRESSION,BOTH,SCRIPT>"SIZEOF_HEADERS" {@>@[RTOKEN(SIZEOF_HEADERS);@]@=} +<EXPRESSION,BOTH,SCRIPT>"SEGMENT_START" {@>@[RTOKEN(SEGMENT_START);@]@=} +<BOTH,SCRIPT>"MAP" {@>@[RTOKEN(MAP);@]@=} +<EXPRESSION,BOTH,SCRIPT>"SIZEOF" {@>@[RTOKEN(SIZEOF);@]@=} +<BOTH,SCRIPT>"TARGET" {@>@[RTOKEN(TARGET_K);@]@=} +<BOTH,SCRIPT>"SEARCH_DIR" {@>@[RTOKEN(SEARCH_DIR);@]@=} +<BOTH,SCRIPT>"OUTPUT" {@>@[RTOKEN(OUTPUT);@]@=} +<BOTH,SCRIPT>"INPUT" {@>@[RTOKEN(INPUT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"GROUP" {@>@[RTOKEN(GROUP);@]@=} +<EXPRESSION,BOTH,SCRIPT>"AS_NEEDED" {@>@[RTOKEN(AS_NEEDED);@]@=} +<EXPRESSION,BOTH,SCRIPT>"DEFINED" {@>@[RTOKEN(DEFINED);@]@=} +<BOTH,SCRIPT>"CREATE_OBJECT_SYMBOLS" {@>@[RTOKEN(CREATE_OBJECT_SYMBOLS);@]@=} +<BOTH,SCRIPT>"CONSTRUCTORS" {@>@[RTOKEN( CONSTRUCTORS);@]@=} +<BOTH,SCRIPT>"FORCE_COMMON_ALLOCATION" {@>@[RTOKEN(FORCE_COMMON_ALLOCATION);@]@=} +<BOTH,SCRIPT>"INHIBIT_COMMON_ALLOCATION" {@>@[RTOKEN(INHIBIT_COMMON_ALLOCATION);@]@=} +<BOTH,SCRIPT>"SECTIONS" {@>@[RTOKEN(SECTIONS);@]@=} +<BOTH,SCRIPT>"INSERT" {@>@[RTOKEN(INSERT_K);@]@=} +<BOTH,SCRIPT>"AFTER" {@>@[RTOKEN(AFTER);@]@=} +<BOTH,SCRIPT>"BEFORE" {@>@[RTOKEN(BEFORE);@]@=} +<BOTH,SCRIPT>"FILL" {@>@[RTOKEN(FILL);@]@=} +<BOTH,SCRIPT>"STARTUP" {@>@[RTOKEN(STARTUP);@]@=} +<BOTH,SCRIPT>"OUTPUT_FORMAT" {@>@[RTOKEN(OUTPUT_FORMAT);@]@=} +<BOTH,SCRIPT>"OUTPUT_ARCH" {@>@[RTOKEN( OUTPUT_ARCH);@]@=} +<BOTH,SCRIPT>"HLL" {@>@[RTOKEN(HLL);@]@=} +<BOTH,SCRIPT>"SYSLIB" {@>@[RTOKEN(SYSLIB);@]@=} +<BOTH,SCRIPT>"FLOAT" {@>@[RTOKEN(FLOAT);@]@=} +<BOTH,SCRIPT>"QUAD" {@>@[RTOKEN( QUAD);@]@=} +<BOTH,SCRIPT>"SQUAD" {@>@[RTOKEN( SQUAD);@]@=} +<BOTH,SCRIPT>"LONG" {@>@[RTOKEN( LONG);@]@=} +<BOTH,SCRIPT>"SHORT" {@>@[RTOKEN( SHORT);@]@=} +<BOTH,SCRIPT>"BYTE" {@>@[RTOKEN( BYTE);@]@=} +<BOTH,SCRIPT>"NOFLOAT" {@>@[RTOKEN(NOFLOAT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"NOCROSSREFS" {@>@[RTOKEN(NOCROSSREFS);@]@=} +<BOTH,SCRIPT>"OVERLAY" {@>@[RTOKEN(OVERLAY); @]@=} +<BOTH,SCRIPT>"SORT_BY_NAME" {@>@[RTOKEN(SORT_BY_NAME); @]@=} +<BOTH,SCRIPT>"SORT_BY_ALIGNMENT" {@>@[RTOKEN(SORT_BY_ALIGNMENT); @]@=} +<BOTH,SCRIPT>"SORT" {@>@[RTOKEN(SORT_BY_NAME); @]@=} +<BOTH,SCRIPT>"SORT_BY_INIT_PRIORITY" {@>@[RTOKEN(SORT_BY_INIT_PRIORITY); @]@=} +<BOTH,SCRIPT>"SORT_NONE" {@>@[RTOKEN(SORT_NONE); @]@=} +<EXPRESSION,BOTH,SCRIPT>"NOLOAD" {@>@[RTOKEN(NOLOAD);@]@=} +<EXPRESSION,BOTH,SCRIPT>"DSECT" {@>@[RTOKEN(DSECT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"COPY" {@>@[RTOKEN(COPY);@]@=} +<EXPRESSION,BOTH,SCRIPT>"INFO" {@>@[RTOKEN(INFO);@]@=} +<EXPRESSION,BOTH,SCRIPT>"OVERLAY" {@>@[RTOKEN(OVERLAY);@]@=} +<EXPRESSION,BOTH,SCRIPT>"ONLY_IF_RO" {@>@[RTOKEN(ONLY_IF_RO); @]@=} +<EXPRESSION,BOTH,SCRIPT>"ONLY_IF_RW" {@>@[RTOKEN(ONLY_IF_RW); @]@=} +<EXPRESSION,BOTH,SCRIPT>"SPECIAL" {@>@[RTOKEN(SPECIAL); @]@=} +<BOTH,SCRIPT>"o" {@>@[RTOKEN(ORIGIN);@]@=} +<BOTH,SCRIPT>"org" {@>@[RTOKEN(ORIGIN);@]@=} +<BOTH,SCRIPT>"l" {@>@[RTOKEN( LENGTH);@]@=} +<BOTH,SCRIPT>"len" {@>@[RTOKEN( LENGTH);@]@=} +<EXPRESSION,BOTH,SCRIPT>"INPUT_SECTION_FLAGS" {@>@[RTOKEN(INPUT_SECTION_FLAGS); @]@=} +<EXPRESSION,BOTH,SCRIPT>"INCLUDE" {@>@[RTOKEN(INCLUDE);@]@=} +<BOTH,SCRIPT>"PHDRS" {@>@[RTOKEN (PHDRS); @]@=} +<EXPRESSION,BOTH,SCRIPT>"AT" {@>@[RTOKEN(AT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"ALIGN_WITH_INPUT" {@>@[RTOKEN(ALIGN_WITH_INPUT);@]@=} +<EXPRESSION,BOTH,SCRIPT>"SUBALIGN" {@>@[RTOKEN(SUBALIGN);@]@=} +<EXPRESSION,BOTH,SCRIPT>"HIDDEN" {@>@[RTOKEN(HIDDEN); @]@=} +<EXPRESSION,BOTH,SCRIPT>"PROVIDE" {@>@[RTOKEN(PROVIDE); @]@=} +<EXPRESSION,BOTH,SCRIPT>"PROVIDE_HIDDEN" {@>@[RTOKEN(PROVIDE_HIDDEN); @]@=} +<EXPRESSION,BOTH,SCRIPT>"KEEP" {@>@[RTOKEN(KEEP); @]@=} +<EXPRESSION,BOTH,SCRIPT>"EXCLUDE_FILE" {@>@[RTOKEN(EXCLUDE_FILE); @]@=} +<EXPRESSION,BOTH,SCRIPT>"CONSTANT" {@>@[RTOKEN(CONSTANT);@]@=} +<MRI>"#".*\n? {@>@[++ lineno; @]@=} +<MRI>"\n" {@> @[++lineno;@+RTOKEN(NEWLINE);@]@=} +<MRI>"*".* {}/* \MRI\ comment line */ +<MRI>";".* {}/* \MRI\ comment line */ +<MRI>"END" {@>@[RTOKEN(ENDWORD); @]@=} +<MRI>"ALIGNMOD" {@>@[RTOKEN(ALIGNMOD);@]@=} +<MRI>"ALIGN" {@>@[RTOKEN(ALIGN_K);@]@=} +<MRI>"CHIP" {@>@[RTOKEN(CHIP); @]@=} +<MRI>"BASE" {@>@[RTOKEN(BASE); @]@=} +<MRI>"ALIAS" {@>@[RTOKEN(ALIAS); @]@=} +<MRI>"TRUNCATE" {@>@[RTOKEN(TRUNCATE); @]@=} +<MRI>"LOAD" {@>@[RTOKEN(LOAD); @]@=} +<MRI>"PUBLIC" {@>@[RTOKEN(PUBLIC); @]@=} +<MRI>"ORDER" {@>@[RTOKEN(ORDER); @]@=} +<MRI>"NAME" {@>@[RTOKEN(NAMEWORD); @]@=} +<MRI>"FORMAT" {@>@[RTOKEN(FORMAT); @]@=} +<MRI>"CASE" {@>@[RTOKEN(CASE); @]@=} +<MRI>"START" {@>@[RTOKEN(START); @]@=} +<MRI>"LIST".* {@>@[RTOKEN(LIST); @]@=}/* \prodstyle{LIST} and ignore to end of line */ +<MRI>"SECT" {@>@[RTOKEN(SECT); @]@=} +<EXPRESSION,BOTH,SCRIPT,MRI>"ABSOLUTE" {@>@[RTOKEN(ABSOLUTE); @]@=} +<MRI>"end" {@>@[RTOKEN(ENDWORD); @]@=} +<MRI>"alignmod" {@>@[RTOKEN(ALIGNMOD);@]@=} +<MRI>"align" {@>@[RTOKEN(ALIGN_K);@]@=} +<MRI>"chip" {@>@[RTOKEN(CHIP); @]@=} +<MRI>"base" {@>@[RTOKEN(BASE); @]@=} +<MRI>"alias" {@>@[RTOKEN(ALIAS); @]@=} +<MRI>"truncate" {@>@[RTOKEN(TRUNCATE); @]@=} +<MRI>"load" {@>@[RTOKEN(LOAD); @]@=} +<MRI>"public" {@>@[RTOKEN(PUBLIC); @]@=} +<MRI>"order" {@>@[RTOKEN(ORDER); @]@=} +<MRI>"name" {@>@[RTOKEN(NAMEWORD); @]@=} +<MRI>"format" {@>@[RTOKEN(FORMAT); @]@=} +<MRI>"case" {@>@[RTOKEN(CASE); @]@=} +<MRI>"extern" {@>@[RTOKEN(EXTERN); @]@=} +<MRI>"start" {@>@[RTOKEN(START); @]@=} +<MRI>"list".* {@>@[RTOKEN(LIST); @]@=}/* \prodstyle{LIST} and ignore to end of line */ +<MRI>"sect" {@>@[RTOKEN(SECT); @]@=} +<EXPRESSION,BOTH,SCRIPT,MRI>"absolute" {@>@[RTOKEN(ABSOLUTE); @]@=} + +<MRI>{FILENAMECHAR1}{NOCFILENAMECHAR}* { +@O +/* Filename without commas, needed to parse \MRI\ stuff */ + yylval.name = xstrdup (yytext); + return NAME; +@o +} +<BOTH>{FILENAMECHAR1}{FILENAMECHAR}* { +@O + yylval.name = xstrdup (yytext); + return NAME; +@o +} +<BOTH>"-l"{FILENAMECHAR}+ { +@O + yylval.name = xstrdup (yytext + 2); + return LNAME; +@o +} +<EXPRESSION>{FILENAMECHAR1}{NOCFILENAMECHAR}* { +@O + yylval.name = xstrdup (yytext); + return NAME; +@o +} +<EXPRESSION>"-l"{NOCFILENAMECHAR}+ { +@O + yylval.name = xstrdup (yytext + 2); + return LNAME; +@o +} +<SCRIPT>{WILDCHAR}* { +@O + /* Annoyingly, this pattern can match comments, and we have + longest match issues to consider. So if the first two + characters are a comment opening, put the input back and + try again. */ + if (yytext[0] == '/' && yytext[1] == '*') + { + yyless (2); + comment (); + } + else + { + yylval.name = xstrdup (yytext); + return NAME; + } +@o +} + +<EXPRESSION,BOTH,SCRIPT,VERS_NODE>"\""[^\"]*"\"" { +@O + /* No matter the state, quotes + give what's inside */ + yylval.name = xstrdup (yytext + 1); + yylval.name[yyleng - 2] = 0; + return NAME; +@o +} + +<BOTH,SCRIPT,EXPRESSION>"\n" {@>@[lineno++;@]@=} +<MRI,BOTH,SCRIPT,EXPRESSION>[ \t\r]+ {} + +<VERS_NODE,VERS_SCRIPT>[:,;] {@>@[return *yytext; @]@=} +<VERS_NODE>global {@>@[RTOKEN(GLOBAL); @]@=} +<VERS_NODE>local {@>@[RTOKEN(LOCAL); @]@=} +<VERS_NODE>extern {@>@[RTOKEN(EXTERN); @]@=} + +<VERS_NODE>{V_IDENTIFIER} { +@O + yylval.name = xstrdup (yytext); + return VERS_IDENTIFIER; +@o +} + +<VERS_SCRIPT>{V_TAG} { +@O + yylval.name = xstrdup (yytext); + return VERS_TAG; +@o +} + +<VERS_START>"{" {@>@[BEGIN(VERS_SCRIPT);@+return *yytext;@]@=} + +<VERS_SCRIPT>"{" { +@O + BEGIN(VERS_NODE); + vers_node_nesting = 0; + return *yytext; +@o +} + +<VERS_SCRIPT>"}" {@>@[return *yytext;@]@=} +<VERS_NODE>"{" {@>@[vers_node_nesting++;@+return *yytext;@]@=} + +<VERS_NODE>"}" { +@O + if (--vers_node_nesting < 0) + BEGIN(VERS_SCRIPT); + return *yytext; +@o +} + +<VERS_START,VERS_NODE,VERS_SCRIPT>[\n] {@>@[lineno++;@]@=} +<VERS_START,VERS_NODE,VERS_SCRIPT>#.* {}/* Eat up comments */ +<VERS_START,VERS_NODE,VERS_SCRIPT>[ \t\r]+ {}/* Eat up whitespace */ + +<<EOF>> { +@O + include_stack_ptr--; + if (include_stack_ptr == 0) + yyterminate (); + else + yy_switch_to_buffer (include_stack[include_stack_ptr]); + + lineno = lineno_stack[include_stack_ptr]; + input_flags.sysrooted = sysrooted_stack[include_stack_ptr]; + + return END; +@o +} + +<SCRIPT,MRI,VERS_START,VERS_SCRIPT,VERS_NODE>. @>@[lex_warn_invalid (" in script", yytext);@] +<EXPRESSION,DEFSYMEXP,BOTH>. @>@[lex_warn_invalid (" in expression", yytext);@] +@g + +@ Switch \flex\ to reading script file |name|, open on |file|, +saving the current input info on the include stack. +@<Supporting \Cee\ code@>= +void +lex_push_file (FILE *file, const char *name, unsigned int sysrooted) +{ + if (include_stack_ptr >= MAX_INCLUDE_DEPTH) + { + einfo ("%F:includes nested too deeply\n"); + } + file_name_stack[include_stack_ptr] = name; + lineno_stack[include_stack_ptr] = lineno; + sysrooted_stack[include_stack_ptr] = input_flags.sysrooted; + include_stack[include_stack_ptr] = YY_CURRENT_BUFFER; + + include_stack_ptr++; + lineno = 1; + input_flags.sysrooted = sysrooted; + yyin = file; + yy_switch_to_buffer (yy_create_buffer (yyin, YY_BUF_SIZE)); +} + +@ Return a newly created \flex\ input buffer containing |string|, +which is |size| bytes long. +@<Supporting \Cee\ code@>= +static YY_BUFFER_STATE +yy_create_string_buffer (const char *string, size_t size) +{ + YY_BUFFER_STATE b; + + b = malloc (sizeof (struct yy_buffer_state));/* Calls to |malloc| get turned by sed into |xmalloc|. */ + b->yy_input_file = 0; + b->yy_buf_size = size; + + b->yy_ch_buf = malloc ((unsigned) (b->yy_buf_size + 3)); + /* |yy_ch_buf| has to be 2 characters longer than the size given because + we need to put in 2 end-of-buffer characters. */ + + b->yy_ch_buf[0] = '\n'; + strcpy (b->yy_ch_buf+1, string); + b->yy_ch_buf[size+1] = YY_END_OF_BUFFER_CHAR; + b->yy_ch_buf[size+2] = YY_END_OF_BUFFER_CHAR; + b->yy_n_chars = size+1; + b->yy_buf_pos = &b->yy_ch_buf[1]; + + b->yy_is_our_buffer = 1; + b->yy_is_interactive = 0; + b->yy_at_bol = 1; + b->yy_fill_buffer = 0; + +#ifdef YY_BUFFER_NEW + b->yy_buffer_status = YY_BUFFER_NEW; +#else + b->yy_eof_status = EOF_NOT_SEEN; +#endif + + return b; +} + +@ Switch \flex\ to reading from |string|, saving the current input info +on the include stack. +@<Supporting \Cee\ code@>= +void +lex_redirect (const char *string, const char *fake_filename, unsigned int count) +{ + YY_BUFFER_STATE tmp; + + yy_init = 0; + if (include_stack_ptr >= MAX_INCLUDE_DEPTH) + { + einfo("%F: macros nested too deeply\n"); + } + file_name_stack[include_stack_ptr] = fake_filename; + lineno_stack[include_stack_ptr] = lineno; + include_stack[include_stack_ptr] = YY_CURRENT_BUFFER; + include_stack_ptr++; + lineno = count; + tmp = yy_create_string_buffer (string, strlen (string)); + yy_switch_to_buffer (tmp); +} + +@ Functions to switch to a different \flex\ start condition, +saving the current start condition on |state_stack|. +@<Supporting \Cee\ code@>= +static int state_stack[MAX_INCLUDE_DEPTH * 2]; +static int *state_stack_p = state_stack; + +void +ldlex_script (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (SCRIPT); +} + +void +ldlex_mri_script (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (MRI); +} + +void +ldlex_version_script (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (VERS_START); +} + +void +ldlex_version_file (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (VERS_SCRIPT); +} + +void +ldlex_defsym (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (DEFSYMEXP); +} + +void +ldlex_expression (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (EXPRESSION); +} + +void +ldlex_both (void) +{ + *(state_stack_p)++ = yy_start; + BEGIN (BOTH); +} + +void +ldlex_popstate (void) +{ + yy_start = *(--state_stack_p); +} + +@ Return the current file name, or the previous file if no file is +current. +@<Supporting \Cee\ code@>= +const char* +ldlex_filename (void) +{ + return file_name_stack[include_stack_ptr - (include_stack_ptr != 0)]; +} + +@ Place up to |max_size| characters in |buf| and return +either the number of characters read, or 0 to indicate |EOF|. +@<Supporting \Cee\ code@>= +static int +yy_input (char *buf, int max_size) +{ + int result = 0; + if (YY_CURRENT_BUFFER->yy_input_file) + { + if (yyin) + { + result = fread (buf, 1, max_size, yyin); + if (result < max_size && ferror (yyin)) + einfo ("%F%P: read in flex scanner failed\n"); + } + } + return result; +} + +@ Eat the rest of a \Cee-style comment. +@<Supporting \Cee\ code@>= +static void +comment (void) +{ + int c; + + while (1) + { + c = input(); + while (c != '*' && c != EOF) + { + if (c == '\n') + lineno++; + c = input(); + } + + if (c == '*') + { + c = input(); + while (c == '*') + c = input(); + if (c == '/') + break; /* found the end */ + } + + if (c == '\n') + lineno++; + + if (c == EOF) + { + einfo( "%F%P: EOF in comment\n"); + break; + } + } +} + +@ Warn the user about a garbage character |what| in the input +in context |where|. +@<Supporting \Cee\ code@>= +static void +lex_warn_invalid (char *where, char *what) +{ + char buf[5]; + + if (ldfile_assumed_script) + { /* If we have found an input file whose format we do not recognize, + and we are therefore treating it as a linker script, and we find + an invalid character, then most likely this is a real object file + of some different format. Treat it as such. */ + + bfd_set_error (bfd_error_file_not_recognized); + einfo ("%F%s: file not recognized: %E\n", ldlex_filename ()); + } + + if (! ISPRINT (*what)) + { + sprintf (buf, "\\%03o", *(unsigned char *) what); + what = buf; + } + + einfo ("%P:%S: ignoring invalid character `%s'%s\n", NULL, what, where); +} diff --git a/support/splint/examples/ld/ldman.w b/support/splint/examples/ld/ldman.w new file mode 100644 index 0000000000..c9551d1b3e --- /dev/null +++ b/support/splint/examples/ld/ldman.w @@ -0,0 +1,572 @@ +% Copyright 2012-2015, Alexander Shibakov +% Copyright 2002-2014 Free Software Foundation, Inc. +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. +@s TeX_ TeX +\input limbo.sty +\input ldfrontmatter.sty +\def\MRI{{\sc MRI}} +\def\optimization{5} +\input ldman.sty +% multi-column output +\input dcols.sty + +\let\oldN\N +\let\N\textN +\let\M\textM +\def\ld{\.{ld}} + +\showlastactiontrue +\immediate\openout\gindex=\jobname.gdx +@** Introduction. +\setupfootnotes +This is a manual documenting the development of a parser that can be +used to typeset \ld\ files (linker scripts) with or without the help +of \CWEB. An existing parser for \ld\ has been adopted as a base, with +appropriately designed actions specific to the task of +typesetting. The appendix to this manual contains the full source code +(including the parts written in \Cee) of both the scanner and the parser for \ld, +used in the original program. Some very minor modifications have been +made to make the programs more `presentable' in \CWEB\ (in particular, the +file had to be split into smaller chunks to satisfy \CWEAVE's +limitations). + +Nearly every aspect of the design is discussed, including the +supporting \TeX\ macros that make both the parser and this +documentation possible. The \TeX\ macros presented here are collected +in \.{ldman.sty} which is later included in the \TeX\ file produced by \CWEAVE. +@<Set up the generic parser machinery@>= +@G +\ifx\optimization\UNDEFINED %/* this trick is based on the premise that \.{\\UNDEFINED} */ + \def\optimization{0} %/* is never defined nor created with \.{\\csname$\ldots$\\endcsname} */ +\fi + +\let\nx\noexpand %/* convenient */ + +\input yycommon.sty %/* general routines for stack and array access */ +\input yymisc.sty %/* helper macros (stack manipulation, table processing, value stack pointers) */ +\input yyinput.sty %/* input functions */ +\input yyparse.sty %/* parser machinery */ +\input flex.sty %/* lexer functions */ +\input yyboth.sty %/* parser initialization, optimization */ + +\ifnum\optimization>\tw@ + \input yyfaststack.sty +\fi + +\input yystype.sty %/* scanner auxiliary types and functions */ +\input yyunion.sty %/* parser data structures */ +\input yxunion.sty %/* extended parser data structures */ +@=\input ldunion.sty %@>/* \ld\ parser data structures */@+ +@g + +@*1 Bootstrapping. +To produce a usable parser/scanner duo, several pieces of code must +be generated. The most important of these are the {\it table files\/} +(\.{ptab.tex} and \.{ltab.tex}) for the parser and the scanner. These +consist of the integer tables defining the operation of the parser and +scanner automata, the values of some constants, and the `action switch'. + +Just like in the case of `real' parsers and scanners, in order to +make the parser and the scanner interact seamlessly, some amount of `glue' +is required. As an example, a file containing the (numerical) +definitions of the token values is generated by \bison\ to be used by +a \flex\ generated scanner. Unfortunately, this file has too little +structure for our purposes (it contains definitions of token values +mixed in with other constants making it hard to distinguish one kind of +definition from another). Therefore, the `glue' is generated by +parsing our grammar once again, this time with a \bison\ grammar +designed for typesetting \bison\ files. A special {\it +bootstrapping\/} mode is used to extract the appropriate +information. The name `bootstrapping' notwithstanding, the parser and +lexer used in the bootstrapping phase are not the minimized versions +used in bootstrapping the \bison\ parser. + +The first component generated during the bootstrapping pass is a list +of `token equivalences' (or `aliases') to be used by the lexer. Every +token (to be precise, every {\it named token type}) used in +a \bison\ grammar is declared using one of the +\prodstyle{\%token}, \prodstyle{\%left}, \prodstyle{\%right}, +\prodstyle{\%precedence}, or \prodstyle{\%nonassoc} declarations. If +no {\it alias\/} (see below) has been declared using a +\prodstyle{\%token} declaration, this name ends up in the |yytname| +array output by \bison\ and can be used by the lexer after associating +the token names with their numerical values (accomplished by +\.{\\settokens}). If all tokens are named tokens, no token equivalence +list is necessary to set up the interaction between the lexer and the +parser. In this case (the present \ld\ parser is a typical +example), the token list serves a secondary role: it provides hints +for the macros that typeset the grammar terms, after the \.{\\tokeneq} +macro is redefined to serve this purpose. + +On the other hand, after a declaration such as `\prodstyle{\%token} +\.{CHAR} \.{"char"}' the string \.{"char"} becomes an alias +for the named token \.{CHAR}. Only the string version gets recorded in +the |yytname| array. Establishing the equivalence between the two +token forms can now only be accomplished by examining the grammar +source file and is delegated to the bootstrapping phase parser. + +\leavevmode\namedspot{bootstrapstates}The other responsibility of the bootstrapping parser is to extract +the information about \flex\ {\it states\/} used by the lexer from the +appropriate source file. As is the case with token names, this +information is output in a rather chaotic fashion by the scanner +generator and is all but useless for our purposes. The original +bootstrapping macros were designed to handle \flex's \prodstyle{\%x} +and \prodstyle{\%s} state declarations and produce a \Cee\ file with the +appropriate definitions. This file can later be included by the +`driver' routine to produce the appropriate table file for the lexer. +To round off the bootstrapping mode we only need to establish the +output streams for the tokens and the states, supply the appropriate file +names for the two lists, flag the bootstrapping mode for the +bootstrapping macros and inline typesetting (\.{\\prodstyle} macros) +and input the appropriate machinery. + +This is done by the macros +below. The bootstrap lexer setup (\.{\\bootstraplexersetup}) consists +of inputting the token equivalence table for the \bison\ parser +(i.e.~the parser that processes the \bison\ grammar file) and defining +a robust token output function which simply ignores the token values +the lexer is not aware of (it should not be necessary in our case +since we are using full featured lexer and parser). +@<Define the bootstrapping mode@>= +@G +\newwrite\tokendefs %/* token list */ +\newwrite\stlist %/* \flex\ state list */ +\newwrite\gindex %/* index entries */ + +\def\modebootstrap{% + \edef\bstrapparser{dyytab.tex}% + \bootstrapmodetrue + \def\bootstraplexersetup{% + \input bo.tok% + \let\yylexreturn\yylexreturnbootstrap /* only return tokens whose value is known */ + %\let\yylexreturn\yylexreturnregular /* should also work */ + }% + \input yybootstrap.sty% +} +@g + +@*1 Namespaces and modes. +Every parser/lexer pair (as well as some other macros) operates +within a dedicated {\it namespace\/}. This simply means that the macros +that output token values, switch lexer states and access various +tables `tack on' the string of characters representing the current +namespace to the `low level' control sequence name that performs the +actual output or access. Say, \.{\\yytname} becomes an alias of +\.{\\yytname[main]} while in the \.{[main]} namespace. When a parser +or lexer is initialized, the appropriate tables are aliased with a +generic name in the case of an `unoptimized' parser or lexer. The +optimized parser or lexer handles the namespace referencing internally. + +The mode setup macros for this manual define several separate +namespaces. The \.{[main]} namespace is established for the parser +that does the typesetting of the grammar. Every time a term name is +processed, the token names are looked up in the \.{[ld]} +namespace. The same namespace is used by the parser that typesets \ld\ +script examples in the manual (i.e.~the parser described here). This +is done to provide visual consistency between the description of the +parser and its output. The \.{[small]} namespace is used by the term +name parser itself. Since we use a customized version of the name parser, we +dedicate a separate namespace for this purpose, \.{[ldsmall]}. +The parser based on a subset of the full \bison\ +grammar describing prologue declarations uses the \.{[prologue]} +namespace. The \.{[index]} namespace is used for typesetting the +index entries and is not necessarily associated with any parser or +lexer. +@<Begin namespace setup@>= +@G +\def\indexpseudonamespace{[index]} +\let\parsernamespace\empty +@g + +@ After all the appropriate tables and `glue' have been generated, the +typesetting of this manual can be handled by the {\tt normal} mode. Note +that this requires the \ld\ parser, as well as the \bison\ parser, +including all the appropriate machinery. + +The normal mode is started by including the tables and lists and +initializing the \bison\ parser (accomplished by inputting +\.{yyinit.sty}), followed by handling the token typesetting for the +\ld\ grammar. +@<Define the normal mode@>= +@G +\newtoks\ldcmds + +\def\modenormal{% + \def\drvname{bo}% + \def\appendr##1##2{\edef\appnext{##1{\the##1##2}}\appnext}% + \def\appendl##1##2{\edef\appnext{##1{##2\the##1}}\appnext}% + \input yyinit.sty% + \let\hostparsernamespace\ldnamespace /* the namespace where tokens are looked up for typesetting purposes */ +} +@g + +@ The \ld\ parser initialization requires setting a few global +variables, as well as entering the \.{INITIAL} state for the \ld\ +lexer. The latter is somewhat counterintuitive and is necessitated by +the ability of the parser to switch lexer states. Thus, the parser can +switch the lexer state before the lexer is invoked for the first time +wreaking havoc on the lexer state stack. +@<Define the normal mode@>= +@G +\def\ldparserinit{% + \basicparserinit + \includestackptr=\@@ne + \versnodenesting=\z@@ + \ldcmds{}% + \yyBEGIN{INITIAL}% +} +@g + +@ This is the \ld\ parser invocation routine. It is coded according to +a straightforward sequence initialize-invoke-execute-or-fall back. +@<Define the normal mode@>= +@G +\expandafter\def\csname parserstack[l]\endcsname#1#2{% + \toldparser\ldparserinit\yyparse#1\yyeof\yyeof\endparseinput\endparse + \ifyyparsefail % /* revert to generic macros if parsing failed */ + \yybreak{\message{parsing failed ...}#2}% + \else % /* stage three, process the parsed table */ + \yybreak{% + \message{commands: \the\ldcmds}% + {% + \restorecslist{ld-display}\ldunion + \the\ldcmds + \par + \vskip-\baselineskip + \the\lddisplay + }% + }% + \yycontinue +} +@g + +@ @<Initialize the active mode@>= +@G +\ifx\modeactive\UNDEFINED + \def\modeactive{\modenormal} +\fi + +\modeactive + +\ifbootstrapmode\else + @>@[@<Initialize \ld\ parsers@>@]@; + @>@[@<Modified name parser for \ld\ grammar@>@]@; +\fi +@g + +@ Unless they are being bootstrapped, the \ld\ parser and its +term parser are initialized by the normal mode. The token typesetting +of \ld\ grammar tokens is adjusted at the same time (see the remarks +above about the mechanism that is responsible for this). Most nonterminals +(such as keywords, etc.) may be displayed unchanged (provided the +names used by the lexer agree with their appearance in the script file, +see below), while the typeseting of others is modified in +\.{ltokenset.sty}. + +In the original \bison-\flex\ interface, token names are +defined as straightforward macros (a poor choice as will be seen +shortly) which can sometimes clash with the standard \Cee\ macros. +This is why \ld\ lexer returns \prodstyle{ASSERT} as +\prodstyle{ASSERT_K}. The name parser treats \.{K} as a suffix to +supply a visual reminder of this flaw. Note that the `suffixless' part +of these tokens (such as \prodstyle{ASSERT}) is never declared and +thus has to be entered in \.{ltokenset.sty} by hand. + +The tokens that never appear as part of the input (such as +\prodstyle{END} and \prodstyle{UNARY}) or those that do but have no +fixed appearance (for example, \prodstyle{NAME}) are typeset in a +style that indicates their origin. The details can be found by +examining \.{ltokenset.sty}. +@<Initialize \ld\ parsers@>= +@G +\genericparser + name: ld, + ptables: ptab.tex, + ltables: ltab.tex, + tokens: {}, + asetup: {}, + dsetup: {}, + rsetup: {}, + optimization: {};% +\genericprettytokens + namespace: ld, + tokens: ldp.tok, + correction: ltokenset.sty, + host: ld;% +@g + +@ The macros are collected in a single file included at the beginning +of this documentation. +@(ldman.stx@>= +@<Set up the generic parser machinery@>@; +@<Begin namespace setup@>@; +@<Define the bootstrapping mode@>@; +@<Define the normal mode@>@; +@<Additional macros for the \ld\ lexer/parser@>@; +@<Initialize the active mode@>@; + +@i ldgram.x +@i ldlex.x + +@** Example output. Here is an example output of the \ld\ parser designed in this +document. The original linker script is presented in the section that +follows. The same parser can be used to present examples of \ld\ scripts +in text similar to the one below. +\beginldprod +MEMORY +{ + RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 20K + FLASH (rx) : ORIGIN = 0x8000000, LENGTH = 128K + ASH (rx) : ORIGIN = 8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000, LENGTH = 128K + ASH (rx) : ORIGIN = $8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000B, LENGTH = 128K + INCLUDE file.mem +} +\endprod +\noindent The syntax of \ld\ is modular enough so there does not seem to be a +need for a `parser stack' as in the case of the \bison\ +parser. If one must be able to display still smaller segments of \ld\ +code, using `hidden context' tricks (discussed elsewhere) seems to be +a better approach. +\iffalse +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +\tracestatestrue +\fi +% +@<Example \ld\ script@>= +@G(l) +INCLUDE file.ld + +MEMORY +{ + RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 20K + FLASH (rx) : ORIGIN = 0x8000000, LENGTH = 128K + ASH (rx) : ORIGIN = 8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000, LENGTH = 128K + ASH (rx) : ORIGIN = $8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000B, LENGTH = 128K + INCLUDE file.mem +} + +_estack = 0x20005000; +_bstack = a > 0 ? NEXT(11) : 0x19; + +SECTIONS +{ + .isr_vector ALIGN(8) (NOLOAD): AT(.) ALIGN(.) ALIGN_WITH_INPUT SUBALIGN(8) SPECIAL + { + . = ALIGN(4); + KEEP(*(.isr_vector)) + . = ALIGN(4); + } > FLASH AT > RAM : FLASH : RAM : OTHER = . + 8 + + .text : + { + /* skip this comment */; + . = ALIGN(4); + *(.text) + *(.text.*) + *(.rodata) + *(.rodata*) + *(.glue_7) + *(.glue_7t) + . = ALIGN(4); + _etext = . + 8; + _sidata = _etext; + } >FLASH AT > RAM + + .data : AT ( _sidata ) + { + . = ALIGN(4); + _sdata = . ; + *(.data) + *(.data.*) + . = ALIGN(4); + _edata = . ; + } >RAM + + .bss : + { + . = ALIGN(4); + _sbss = .; + *(.bss) + *(COMMON) + . = ALIGN(4); + _ebss = . ; + } >RAM +} +@g + +@ @<The same example of an \ld\ script@>= +@G(b) +INCLUDE file.ld + +MEMORY +{ + RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 20K + FLASH (rx) : ORIGIN = 0x8000000, LENGTH = 128K + ASH (rx) : ORIGIN = 8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000, LENGTH = 128K + ASH (rx) : ORIGIN = $8000000, LENGTH = 128K + CLASH (rx) : ORIGIN = 700000B, LENGTH = 128K + INCLUDE file.mem +} + +_estack = 0x20005000; +_bstack = a > 0 ? NEXT(11) : 0x19; + +SECTIONS +{ + .isr_vector ALIGN(8) (NOLOAD): AT(.) ALIGN(.) ALIGN_WITH_INPUT SUBALIGN(8) SPECIAL + { + . = ALIGN(4); + KEEP(*(.isr_vector)) + . = ALIGN(4); + } > FLASH AT > RAM : FLASH : RAM : OTHER = . + 8 + + .text : + { + /* skip this comment */; + . = ALIGN(4); + *(.text) + *(.text.*) + *(.rodata) + *(.rodata*) + *(.glue_7) + *(.glue_7t) + . = ALIGN(4); + _etext = . + 8; + _sidata = _etext; + } >FLASH AT > RAM + + .data : AT ( _sidata ) + { + . = ALIGN(4); + _sdata = . ; + *(.data) + *(.data.*) + . = ALIGN(4); + _edata = . ; + } >RAM + + .bss : + { + . = ALIGN(4); + _sbss = .; + *(.bss) + *(COMMON) + . = ALIGN(4); + _ebss = . ; + } >RAM +} +@g + +@ +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +\tracestatesfalse + +@i ldnp.x + +@** Appendix. The original code of the \ld\ parser and lexer is reproduced below. It +is mostly left intact and is typeset by the pretty printing parser for +\bison\ input. The lexer (\flex) input is reproduced verbatim and is +left mostly unformatted with the exception of spacing and the embedded +\Cee\ code. + +The treatment of comments is a bit more invasive. \CWEB\ silently +assumes that the comment refers to the preceding statement or a group +of statements which is reflected in the way the comment is +typeset. The comments in \ld\ source files use the +opposite convention. For the sake of consistency, such comments have +been moved so as to make them fit the \CWEB\ style. The comments meant to refer to a +sizable portion of the program (such as a whole function or a group +of functions) are put at the beginning of a \CWEB\ section containing +the appropriate part of the program. + +\CWEB\ treats comments as ordinary \TeX\ so the comments are changed +to take advantage of \TeX\ formatting and introduce some visual +cues. The convention of using {\it italics\/} for the original +comments has been reversed: the italicized comments are the ones +introduced by the author, {\it not\/} the original +creators of \ld. + +@i ldgramo.x +@i ldlexo.x + +@** Index. This section lists the variable names and (in some cases) +the keywords used inside the `language sections' of the \CWEB\ +source. It takes advantage of the built-in facility of \CWEB\ to supply +references for both definitions (set in {\it +italic}) as well as uses for each \Cee\ identifier in the text. + +Special facilities have been added to extend indexing to +\bison\ grammar terms, \TeX\ control sequences encountered in +\bison\ actions, and file and section names encountered in \ld\ +scripts. For a detailed description of the various conventions adhered +to by the index entries the reader is encouraged to consult the +remarks preceding the index of the document describing the core of +the \splint\ suite. We will only mention here that (consistent with +the way \bison\ references are treated) a script example: +$$ +\vbox{ +\beginldprod +MEMORY +{ + MEMORY1 (xrw) : ORIGIN = 0x20000000, LENGTH = 20K + MEMORY2 (rx) : ORIGIN = 0x8000000, LENGTH = 128K +} +_var_1 = 0x20005000; +\endprod +}% +$$ +\noindent inside the \TeX\ part of a \CWEB\ section will generate several +index entries, as well, mimicking \CWEB's behavior for the +{\it inline \Cee\/} (\.{\yl}$\ldots$\.{\yl}). Such entries are labeled +with $^\circ$, to provide a reminder of their origin. +\def\otherlangindexseparator{% + \par + \vskip.5\baselineskip + \centerline{B{\sc ISON}, LD, {\sc AND} \TeX\ {\sc INDICES}}% + \vskip.5\baselineskip + \par +} +\let\currentparsernamespace\parsernamespace + \let\parsernamespace\indexpseudonamespace + \prettywordpair{emptyrhs}{$\circ$ {\rm(empty rhs)}}% + \prettywordpair{inline_action}{$\diamond$ {\rm(inline action)}}% +\let\parsernamespace\currentparsernamespace +\closeout\gindex +\let\inx\inxmod +\let\fin\finmod +\let\oldMRL\MRL +\def\MRL#1{\smash{\oldMRL{#1}}} % a more sophisticated way to handle it woud be to add a \smash whenever we are + % in the [index] namespace but this is simpler and works as well +\def\topofcontents{\null\vskip-3\baselineskip\centerline{C{\sc ONTENTS} (\sc\uppercase\expandafter{\title})}\medskip} diff --git a/support/splint/examples/ld/ldnp.w b/support/splint/examples/ld/ldnp.w new file mode 100644 index 0000000000..f92d0dfc8c --- /dev/null +++ b/support/splint/examples/ld/ldnp.w @@ -0,0 +1,362 @@ +@q Copyright 2012-2015, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@** The name parser for \ld\ term names. +We take a lazy approach to the typeseting of term names for the \ld\ +grammar by creating a dedicated parser for name processing. This way +any pattern we notice can be quickly incorporated into our typesetting scheme. +% We include the macros here since this file is intended to be +% included by the documentation `aggregator' so putting bare \TeX\ +% at the beginning of the file runs the risk of producing an error +% of having \TeX\ material inside a \Cee\ section. +\genericprettytokens namespace: ldsmall, tokens: {}, correction: lstokenset.sty, host: ldsmall; +@(ld_small_parser.yy@>= +@G Switch to generic mode. +%{@> @<Name parser \Cee\ preamble@> @=%} + @> @<Bison options@> @= +%union {@> @<Union of parser types@> @=} +%{@> @<Name parser \Cee\ postamble@> @=%} + @> @<Token and types ...@> @= +%% + @> @<Parser productions@> @= +%% +@g + +@ To put the new name parser to work, we need to initialize it. The +initialization is done by the macros below. After the initialization +has been completed, the switch command is replaced by the one that +activates the new name parser. +@<Modified name parser for \ld\ grammar@>= +@G +\genericparser + name: ldsmall, + ptables: ld_small_tab.tex, + ltables: ld_small_dfa.tex, + tokens: {}, + asetup: {}, + dsetup: {}, + rsetup: {\noexpand\savefullstateextra}, + optimization: {};% +\let\otosmallparser\tosmallparser % /* save the old name parser */ +\let\tosmallparser\toldsmallparser +@g + +@ @<Bison options@>= +@G +%token-table +%debug +%start full_name +@g + +@ @<Token and types declarations@>= +@G +%token PERCENT_IDENTIFIER +%token IDENTIFIER +%token OPTIONAL +%token K_SUFFIX +%token INTEGER +%token EXTENDED +%token WILDCARD +@g + +@*1 The name parser productions. These macros do a bit more than we +need to typeset the term names. Their core is designed to treat +suffixes and prefixes of a certain form in a special way. In addition, +some productions were left in place from the original name parser in +order to be able to refer to, say, \flex\ options in text. The inline +action in one of the rules for \prodstyle{identifier\_string} was +added to adjust the number and the position of the terms so that the +appropriate action can be reused later for +\prodstyle{qualified\_identifier\_string}. +@<Parser productions@>= +@G +full_name: + identifier_string suffixes.opt {@> @<Compose the full name@> @=} +| qualifier '_' identifier_string suffixes.opt {@> @<Compose a qualified name@> @=} +; + +identifier_string: + PERCENT_IDENTIFIER {@> @<Attach option name@> @=} +| IDENTIFIER {@> @<Start with an identifier@> @=} +| '\'' WILDCARD '\'' {@> @<Start with a quoted string@> @=} +| '\'' '.' '\'' {@> @<Start with a \prodstyle{'.'} string@> @=} +| '\'' '_' '\'' {@> @<Start with an \prodstyle{'\_'} string@> @=} +@t}\vb{\flatten}{@> +| incomplete_identifier_string {} IDENTIFIER {@> @<Attach an identifier@> @=} +; +@t}\vb{\resetf}{@> +incomplete_identifier_string: + identifier_string '_' {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| qualified_identifier_string '_' {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +; + +qualified_identifier_string: + identifier_string '_' qualifier {@> @<Attach qualifier to a name@> @=} +| qualified_identifier_string '_' qualifier {@> @<Attach qualifier to a name@> @=} +; + +suffixes.opt: + {@> TeX_( "/yy0{}" ); @=} +| '.' {@> TeX_( "/yy0{/nx/dotsp/nx/sfxnone}" ); @=} +| '.' suffixes {@> @<Attach suffixes@> @=} +| '.' qualified_suffixes {@> @<Attach qualified suffixes@> @=} +| INTEGER {@> @<Attach an integer@> @=} +| '_' INTEGER {@> @<Attach a subscripted integer@> @=} +| '_' qualifier {@> @<Attach a subscripted qualifier@> @=} +; + +suffixes: + IDENTIFIER {@> @<Start with a named suffix@> @=} +| INTEGER {@> @<Start with a numeric suffix@> @=} +| suffixes '.' {@> @<Add a dot separator@> @=} +| suffixes IDENTIFIER {@> @<Attach a named suffix@> @=} +| suffixes INTEGER {@> @<Attach integer suffix@> @=} +| qualifier '.' {@> TeX_( "/yy0{/nx/sfxn/the/yy(1)/nx/dotsp}" ); @=} +| suffixes qualifier '.' {@> TeX_( "/yy0{/the/yy(1)/nx/sfxn/the/yy(2)/nx/dotsp}" ); @=} +; + +qualified_suffixes: + suffixes qualifier {@> @<Attach a qualifier@> @=} +| qualifier {@> @<Start suffixes with a qualifier@> @=} +; + +qualifier: + OPTIONAL {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| K_SUFFIX {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +| EXTENDED {@> TeX_( "/yy0{/the/yy(1)}" ); @=} +; +@g + +@ @<Compose the full name@>= + @[TeX_( "/yy0{/the/yy(1)/the/yy(2)}/namechars/yyval" );@]@; + +@ @<Compose a qualified name@>= + @[TeX_( "/yy0{/the/yy(3)/the/yy(4)/nx/dotsp/nx/qual/the/yy(1)}/namechars/yyval" );@]@; + +@ @<Attach option name@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/optstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with an identifier@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with a tag@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{</the/toksa>}{</the/toksb>}}" );@]@; + +@ @<Start with a quoted string@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/chstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with a \prodstyle{'.'} string@>= + @[TeX_( "/yy0{/nx/chstr{.}{.}}" );@]@; + +@ @<Start with an \prodstyle{'\_'} string@>= + @[TeX_( "/yy0{/nx/chstr{/uscoreletter}{/uscoreletter}}" );@]@; + +@ @<Turn a qualifier into an identifier@>= + @<Start with an identifier@>@; + +@ @<Attach an identifier@>= + @[TeX_( "/getsecond{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/appendr/toksa{/noexpand/_}" );@]@; + @[TeX_( "/getfirst{/yy(3)}/to/toksb" );@]@; + @[TeX_( "/concat/toksa/toksb" );@]@; + @[TeX_( "/getthird{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/appendr/toksb{/uscoreletter}" );@]@; + @[TeX_( "/getsecond{/yy(3)}/to/toksc" );@]@; + @[TeX_( "/concat/toksb/toksc" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Attach qualifier to a name@>= + + +@ @<Attach an integer@>= + @[TeX_( "/yy0{/nx/dotsp/nx/sfxi/the/yy(1)}" );@]@; + +@ @<Attach a subscripted integer@>= + @[TeX_( "/yy0{/nx/dotsp/nx/sfxi/the/yy(2)}" );@]@; + +@ @<Attach a subscripted qualifier@>= + @[TeX_( "/yy0{/nx/dotsp/nx/qual/the/yy(2)}" );@]@; + +@ @<Attach suffixes@>= + @[TeX_( "/yy0{/nx/dotsp/the/yy(2)}" );@]@; + +@ @<Attach qualified suffixes@>= + @<Attach suffixes@>@; + +@ @<Start with a named suffix@>= + @[TeX_( "/yy0{/nx/sfxn/the/yy(1)}" );@]@; + +@ @<Start with a numeric suffix@>= + @[TeX_( "/yy0{/nx/sfxi/the/yy(1)}" );@]@; + +@ @<Add a dot separator@>= + @[TeX_( "/yy0{/the/yy(1)/nx/dotsp}" );@]@; + +@ @<Attach integer suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxi/the/yy(2)}" );@]@; + +@ @<Attach a named suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxn/the/yy(2)}" );@]@; + +@ @<Attach a qualifier@>= + @[TeX_( "/yy0{/the/yy(1)/nx/qual/the/yy(2)}" );@]@; + +@ @<Start suffixes with a qualifier@>= + @[TeX_( "/yy0{/nx/qual/the/yy(1)}" );@]@; + +@ \Cee\ preamble. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<Name parser \Cee\ preamble@>= + +@ \Cee\ postamble. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<Name parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) @[yyprint (file, type, value)@] + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ Union of types. + +@<Union of parser types@>= + +@*1 The name scanner. +%\checktabletrue +@(ld_small_lexer.ll@>= +@G + @> @<Lexer definitions@> @= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% +@O +void define_all_states( void ) { + @<Collect all state definitions@>@; +} +@o +@g +@t}\ifbootstrapmode\yyskipparsetrue\fi{@> + +@ @<Lexer definitions@>= + @<Lexer states@>@; +@G +aletter [a-zA-Z] +wc ([^\\\']{-}[a-zA-Z0-9]|\\.) +id ({aletter}|{aletter}({aletter}|[0-9])*{aletter}) +int [0-9]+ +@g + +@ @<Collect all state definitions@>= +#define _register_name( name ) @[Define_State( #name, name )@] +/* nothing for now */ +#undef _register_name + +@ Strings and characters in directives/rules. +@<Lexer states@>= +@G +%x SC_ESCAPED_STRING SC_ESCAPED_CHARACTER +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + +@t}\yyskipparsefalse{@> +@ @<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="ld_small_lexer.c" +@g + +@ @<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +% +%\prodstyle{abc123} +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +% +\yyskipparsetrue +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ This collection of regular expressions might seem redundant, and in +its present state, it certainly is. However, if later on the +typesetting style for some of the keywords would need to be adjusted, +such changes would be easy to implement, since the template is already +here. +@<Scan identifiers@>= +@G +"%"({aletter}|[0-9]|[-_]|"%"|[<>])+ {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} + +"opt" {@> @[TeX_( "/yylexreturnval{OPTIONAL}" );@]@=} +"K" {@> @[TeX_( "/yylexreturnval{K_SUFFIX}" );@]@=} +"ext" {@> @[TeX_( "/yylexreturnval{EXTENDED}" );@]@=} + +[\'._] {@> @[TeX_( "/yylexreturnchar" );@]@=} +{wc} {@> @[TeX_( "/yylexreturnval{WILDCARD}" );@]@=} + +{id} {@> @[@<Prepare to process an identifier@>@]@=} +{int} {@> @[TeX_( "/yylexreturnval{INTEGER}" );@]@=} + +"\"" {@> @[TeX_( "/yylexnext" );@]@=} +. {@> @[@<React to a bad character@>@]@=} +@g + +@ @<Prepare to process an identifier@>= + @[TeX_( "/yylexreturnval{IDENTIFIER}" );@]@; + +@ \let\hostparsernamespace\ldnamespace\yyskipparsefalse A simple routine to detect +trivial scanning problems. +@<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; diff --git a/support/splint/examples/ld/ldnump.w b/support/splint/examples/ld/ldnump.w new file mode 100644 index 0000000000..5d4d45f724 --- /dev/null +++ b/support/splint/examples/ld/ldnump.w @@ -0,0 +1,306 @@ +@q Copyright 2012-2015, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +@** The name parser for \ld\ term names. +We take a lazy approach to the typeseting of term names for the \ld\ +grammar by creating a dedicated parser for name processing. This way +any pattern we notice can be quickly incorporated into our typesetting scheme. +% We include the macros here since this file is intended to be +% included by the documentation `aggregator' so putting bare \TeX\ +% at the beginning of the file runs the risk of producing an error +% of having \TeX\ material inside a \Cee\ section. +\genericprettytokens namespace: ldsmall, tokens: {}, correction: lstokenset.sty, host: ldsmall; +\iffalse +\let\currentparsernamespace\parsernamespace + \let\parsernamespace\ldsmallnamespace + \let\hostparsernamespace\ldsmallnamespace + \input lstokenset.sty +\let\parsernamespace\currentparsernamespace +\fi +@(ld_num_parser.yy@>= +@G Switch to generic mode. +%{@> @<Name parser \Cee\ preamble@> @=%} + @> @<Bison options@> @= +%union {@> @<Union of parser types@> @=} +%{@> @<Name parser \Cee\ postamble@> @=%} + @> @<Token and types ...@> @= +%% + @> @<Parser productions@> @= +%% +@g + +@ To put the new name parser to work, we need to initialize it. The +initialization is done by the macros below. After the initialization +has been completed, the switch command is replaced by the one that +activates the new name parser. +@<Modified name parser for \ld\ grammar@>= +@G +\genericparser name: ldnum, + ptables: ld_num_tab.tex, + ltables: ld_num_dfa.tex, + tokens: {}, + asetup: {}, + dsetup: {}, + rsetup: {\noexpand\savefullstateextra}, + optimization: {};% +@g + +@ @<Bison options@>= +@G +%token-table +%debug +%start hex_number +@g + +@ @<Token and types declarations@>= +@G +%token DIGIT NYBBLE +@g + +@*1 The name parser productions. These macros do a bit more than we +need to typeset the term names. Their core is designed to treat +suffixes and prefixes of a certain form in a special way. In addition, +some productions were left in place from the original name parser in +order to be able to refer to, say, \flex\ options in text. The inline +action in one of the rules for \prodstyle{identifier\_string} was +added to adjust the number and the position of the terms so that the +appropriate action can be reused later for +\prodstyle{qualified\_identifier\_string}. +@<Parser productions@>= +@G +hex_number: + DIGIT {} + | DIGIT DIGIT {} + | DIGIT DIGIT DIGIT {} + | hex_number nybble {} + ; + +nybble: + DIGIT DIGIT DIGIT DIGIT {} + ; +@g + +@ @<Compose the full name@>= + @[TeX_( "/yy0{/the/yy(1)/the/yy(2)}/namechars/yyval" );@]@; + +@ @<Compose a qualified name@>= + @[TeX_( "/yy0{/the/yy(3)/the/yy(4)/nx/dotsp/nx/qual/the/yy(1)}/namechars/yyval" );@]@; + +@ @<Attach option name@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/optstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with an identifier@>= + @[TeX_( "/getfirst{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Start with a tag@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/idstr{</the/toksa>}{</the/toksb>}}" );@]@; + +@ @<Start with a quoted string@>= + @[TeX_( "/getfirst{/yy(2)}/to/toksa" );@]@; + @[TeX_( "/getsecond{/yy(2)}/to/toksb" );@]@; + @[TeX_( "/yy0{/nx/chstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Turn a qualifier into an identifier@>= + @<Start with an identifier@>@; + +@ @<Attach an identifier@>= + @[TeX_( "/getsecond{/yy(1)}/to/toksa" );@]@; + @[TeX_( "/appendr/toksa{/noexpand/_}" );@]@; + @[TeX_( "/getfirst{/yy(3)}/to/toksb" );@]@; + @[TeX_( "/concat/toksa/toksb" );@]@; + @[TeX_( "/getthird{/yy(1)}/to/toksb" );@]@; + @[TeX_( "/appendr/toksb{/uscoreletter}" );@]@; + @[TeX_( "/getsecond{/yy(3)}/to/toksc" );@]@; + @[TeX_( "/concat/toksb/toksc" );@]@; + @[TeX_( "/yy0{/nx/idstr{/the/toksa}{/the/toksb}}" );@]@; + +@ @<Attach qualifier to a name@>= + + +@ @<Attach an integer@>= + @[TeX_( "/yy0{/nx/dotsp/nx/sfxi/the/yy(1)}" );@]@; + +@ @<Attach a subscripted integer@>= + @[TeX_( "/yy0{/nx/dotsp/nx/sfxi/the/yy(2)}" );@]@; + +@ @<Attach a subscripted qualifier@>= + @[TeX_( "/yy0{/nx/dotsp/nx/qual/the/yy(2)}" );@]@; + +@ @<Attach suffixes@>= + @[TeX_( "/yy0{/nx/dotsp/the/yy(2)}" );@]@; + +@ @<Attach qualified suffixes@>= + @<Attach suffixes@>@; + +@ @<Start with a named suffix@>= + @[TeX_( "/yy0{/nx/sfxn/the/yy(1)}" );@]@; + +@ @<Start with a numeric suffix@>= + @[TeX_( "/yy0{/nx/sfxi/the/yy(1)}" );@]@; + +@ @<Add a dot separator@>= + @[TeX_( "/yy0{/the/yy(1)/nx/dotsp}" );@]@; + +@ @<Attach integer suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxi/the/yy(2)}" );@]@; + +@ @<Attach a named suffix@>= + @[TeX_( "/yy0{/the/yy(1)/nx/sfxn/the/yy(2)}" );@]@; + +@ @<Attach a qualifier@>= + @[TeX_( "/yy0{/the/yy(1)/nx/qual/the/yy(2)}" );@]@; + +@ @<Start suffixes with a qualifier@>= + @[TeX_( "/yy0{/nx/qual/the/yy(1)}" );@]@; + +@ \Cee\ preamble. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<Name parser \Cee\ preamble@>= + +@ \Cee\ postamble. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<Name parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) @[yyprint (file, type, value)@] + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ Union of types. + +@<Union of parser types@>= + +@*1 The name scanner. +%\checktabletrue +@(ld_num_lexer.ll@>= +@G + @> @<Lexer definitions@> @= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% +@O +void define_all_states( void ) { + @<Collect all state definitions@>@; +} +@o +@g + +@ \ifbotstrapmode\yyskipparsetrue\fi@<Lexer definitions@>= + @<Lexer states@>@; +@G +aletter [a-zA-Z] +wc ([^\\\'\"-]{-}[_a-zA-Z0-9]|\\.) +id ({aletter}|{aletter}({aletter}|[0-9])*{aletter}) +int [0-9]+ +@g + +@ @<Collect all state definitions@>= +#define _register_name( name ) @[Define_State( #name, name )@] +/* nothing for now */ +#undef _register_name + +@ Strings and characters in directives/rules. +@<Lexer states@>= +@G +%x SC_ESCAPED_STRING SC_ESCAPED_CHARACTER +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + +@ \yyskipparsefalse@<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="ld_num_lexer.c" +@g + +@ @<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +% +%\prodstyle{abc123} +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +% +%\yyskipparsetrue +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ This collection of regular expressions might seem redundant, and in +its present state, it certainly is. However, if later on the +typesetting style for some of the keywords would need to be adjusted, +such changes would be easy to implement, since the template is already +here. +%\yyskipparsefalse % this is not necessary +@<Scan identifiers@>= +@G +"%"({aletter}|[0-9]|[-_]|"%"|[<>])+ {@> @[TeX_( "/yylexreturnval{PERCENT_IDENTIFIER}" );@]@=} + +"opt" {@> @[TeX_( "/yylexreturnval{OPTIONAL}" );@]@=} +"na" {@> @[TeX_( "/yylexreturnval{NO_ATTR}" );@]@=} +"ext" {@> @[TeX_( "/yylexreturnval{EXTENDED}" );@]@=} + +[<>._\'] {@> @[TeX_( "/yylexreturnchar" );@]@=} +{wc} {@> @[TeX_( "/yylexreturnval{WILDCARD}" );@]@=} + +{id} {@> @[@<Prepare to process an identifier@>@]@=} +{int} {@> @[TeX_( "/yylexreturnval{INTEGER}" );@]@=} + +"\"" {@> @[TeX_( "/yylexnext" );@]@=} +. {@> @[@<React to a bad character@>@]@=} +@g + +@ @<Prepare to process an identifier@>= + @[TeX_( "/yylexreturnval{IDENTIFIER}" );@]@; + +@ \let\hostparsernamespace\ldnamespace @<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; diff --git a/support/splint/examples/ld/ldunion.sty b/support/splint/examples/ld/ldunion.sty new file mode 100644 index 0000000000..b2c3837e57 --- /dev/null +++ b/support/splint/examples/ld/ldunion.sty @@ -0,0 +1,351 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% prototypes of all the macros produced by the ld parser +% we will follow the convention: 12string is a string of category 12 characters and spaces, tex_string: a string +% of TeX tokens; sptr is a pointer to the stash stream, fptr is a pointer to the format stream + +\def\yyuniontag{\ldunion} +\def\parserstrictnamespace{ld-parser-strict} +\def\parserprototypesnamespace{ld-parser-strict:headers} + +\def\ldunion{\currentyyunionnamespace} +\def\currentyyunionnamespace{ld-generic} + +% types returned by the lexer (* marks the types that get removed by the parser in some cases) + +\defp\anint#1#2#3{} % integer :: \anint{integer}{fptr}{sptr} +\defp\bint#1#2#3{} % integer in a specifix radix :: \bint{integer}{fptr}{sptr} +\defp\hexint#1#2#3{} % hex integer :: \hexint{integer}{fptr}{sptr} + +% ld commands + +\defp\ldregexp#1{} % name pattern :: \ldregexp{name} +\defp\ldspace{} % space separator :: \ldspace +\defp\ldattributes#1{} % attributes :: \ldattributes{name} +\defp\ldattributesneg#1{} % complemented attributes :: \ldattributesneg{name} +\defp\ldfilename#1{} % file name :: \ldfilename{name} +\defp\ldcommandseparator#1#2{} % command separator +\defp\ldassignment#1#2#3{} % assignment :: \ldassignment{lhs}{op}{rhs} +\defp\ldinclude#1{} % file inclusion :: \ldinclude{file name} +\defp\ldmemory#1{} % memory specification :: \ldmemory{memory spcification} +\defp\ldfill#1{} % fill expression :: \ldfill{expression} +\defp\ldmemoryspec#1#2#3#4{} % memory item :: \ldmemoryspec{name}{attributes}{origin}{length} +\defp\ldoriginspec#1{} % origin :: \ldoriginspec{expression} +\defp\ldlengthspec#1{} % length :: \ldlengthspec{expression} +\defp\ldsections#1{} % ld sections :: \ldsections{sections} +\defp\ldsectionseparator{} % section separator +\defp\ldtype#1{} % section type :: \ldtype{type} +\defp\ldstatement#1{} % stray statement :: \ldstatement{statement} +\defp\ldnamedsection#1#2#3#4#5#6#7{} % named section :: \ldnamedsection{name} + % {{expression}{type}}{at} + % {{}{}{}:alignment} + % {constraint}{statements} + % {{}{}{}{}:placement} + +\savecslist{ld-parser-prototypes}\ldunion + +\input ldint.sty + +\def\bigbracedel{\delimiter"4266308} + +\newtoks\lddisplay + +\defc\ldinclude{% + \toksa={&##\cr\ttl include\ &}#1% + \concat\toksa\toksc + \toksc{\cr} + \concat\toksa\toksc + \edef\next{\lddisplay{\the\lddisplay\halign{\the\toksa}}}\next +} + +\defc\ldmemory{% + \savecslist{local-namespace}{\ldunion}% + \restorecslist{ld-parser:memory-spec}\ldunion + \toksa{}#1% + \toksc{% + \hfil##\qquad&##\hfil&\qquad##\hfil\quad&\hfil##&\quad\hfil##\cr + \ttl memory&\hfil&\ttl attributes&\ttl origin&\ttl length\cr + \noalign{\smallskip}% + }% + \edef\next{\lddisplay{\the\lddisplay\halign{\the\toksc\the\toksa}}}\next + \restorecslist{local-namespace}{\ldunion}% +} + +\defc\ldsections{% + \savecslist{local-namespace}{\ldunion}% + \restorecslist{ld-parser:sections}\ldunion + \toksa{}\def\sections@header{\ttl sections}% + #1% + \toksc{% + &##\hfil\quad\cr + }% + \edef\next{\lddisplay{\the\lddisplay\halign{\the\toksc\the\toksa}}}\next + \restorecslist{local-namespace}{\ldunion}% +} + +\def\ldextractname#1#2{% + \edef\next{\toksc{\gidxentry{\termttstring}{#2}{\ntt #2}}}\next +} + +% the grammar of ld scripts is very uniform so the separator form +% below is more than adequate; if a more sophisticated spacing +% strategy is required, one may consult the design of +% \separatorswitcheq and \separatorswitchneq in yyunion.sty + +\defc\ldcommandseparator{\yyifsamestring{#1}{#2}{}{\appendr\lddisplay{\noexpand\medskip}}} + +\defc\ldstatement{\toksc{\hbox{$#1$}}\concat\lddisplay\toksc} + +\defc\ldassignment{% + #1#2#3% +} + +\defc\anint{% + \uppercase{\ldsciinteger{#1}}% +} + +\defc\hexint{% + \ldsciinteger{#1}% +} + +\defc\bint{% + \uppercase{\ldbasedinteger{#1}}% +} + +\defc\ldregexp{% + \ldreg@xp#1% +} + +\def\ldreg@xp#1#2{% + \hbox{\let\termindex\writeidxentry\edef\next{\toksc{\gidxentry{\termttstring}{#2}}}\next\the\toksc\ntt@#2}% +} + +\defc\ldfill{% + #1% +} + +\defc\ldfilename{\ldextractname#1} + +\savecslist{ld-display}\ldunion + +% memory specifications + +\restorecslist{ld-parser-strict}\ldunion + +\defc\ldmemoryspec{% + \toksb{\hfil&}% + \let\termindex\writeidxhentry + \ldextractname#1\concat\toksb\toksc + \let\termindex\eatone + \appendr\toksb{&}% + \toksc{}#2\concat\toksb\toksc + \toksc{}#3\concat\toksb\toksc + \toksc{}#4\concat\toksb\toksc + \concat\toksa\toksb + \toksb{\cr}\concat\toksa\toksb +} + +\defc\ldattributes{% + \ldextractname#1\appendr\toksc{&}% +} + +\defc\ldlengthspec{% + \toksc{#1}% +} + +\defc\ldoriginspec{% + \toksc{#1&}% +} + +\defc\ldinclude{% + \toksc={\hfil&\ttl include }% + \concat\toksa\toksc#1% + \concat\toksa\toksc + \toksc{&\hfil&\hfil&\hfil\cr}% + \concat\toksa\toksc +} + +\restorecs{ld-display}{\anint\ldfilename} +\toyyunion{ld-parser:memory-spec} + +% sections commands + +\restorecslist{ld-parser-strict}\ldunion + +\newif\ifplacementpushed +\newif\ifsectioncomplete +\newif\iffillextracted + +\defc\ldnamedsection{ % named section :: \ldnamedsection{name} + % {{bind?}{expression}{block?}{expression}{type}}{at} + % {{}{}{}:alignment} + % {constraint}{statements} + % {{}{}{}{}:placement} + \tempca=\z@ % line counter + \tempcb=\z@ % alignment line counter + \placementpushedfalse + \sectioncompletefalse + \fillextractedfalse + \bloop + \toksb{}% + \ifnum\tempca=\z@ + \toksb\expandafter{\sections@header&}% section header + \ldextractname#1% section name + \concat\toksb\toksc + \appendr\toksb{&}% + \ldexpwithtype#2% location and type + \concat\toksb\toksc + \appendr\toksb{&}% + \yystringempty{#3}{\ldpushalignment#4}{\toksc{{\ttl at }$#3$}}% alignment + \concat\toksb\toksc + \appendr\toksb{&}% + \yystringempty{#5}{\ldpushplacement#7}{\toksc{{\ttl #5}}}% constraint + \concat\toksb\toksc + \appendr\toksb{&}% + \ldstartpheaders#7% possible pheaders + \concat\toksb\toksc + \appendr\toksb{\cr}% + \def\sections@header{}% + \else + \toksb\expandafter{&}% section header + \advance\tempca\m@ne + \ldextractitem\tempca{#6}% expression + \advance\tempca\@ne + \yytoksempty\toksc{% + \ldextractfill#7% + \iffillextracted + \ifnum\tempcb<\tw@ + \else + \sectioncompletetrue + \fi + \fi + \fillextractedtrue + }{% + \toksc\expandafter{\expandafter\qquad\expandafter$\the\toksc{}$}% + }% + \concat\toksb\toksc + \appendr\toksb{&}% +% \ldexpwithtype#2% location and type +% \concat\toksa\toksc + \appendr\toksb{&}% + \ldpushalignment#4% alignment + \concat\toksb\toksc + \appendr\toksb{&}% + \ldpushplacement#7% placement + \concat\toksb\toksc + \appendr\toksb{&}% + \ldpushpheaders#7% possible pheaders + \concat\toksb\toksc + \appendr\toksb{\cr}% + \def\sections@header{}% + \fi + \ifsectioncomplete + \else + \concat\toksa\toksb + \advance\tempca\@ne + \repeat +} + +\def\ldextractitem#1#2{% + \yystringempty{#2}{% + \toksc{}% + }{% + \let\ldor\or + \toksc=\ifcase#1#2\else{}\fi + \let\ldor\relax + }% +} + +\def\ldpushalignment#1#2#3{% + \toksd=\ifcase\tempcb{#1}\or{#2}\or{#3}\else{}\fi + \yytoksempty\toksd{% + \advance\tempcb\@ne + \ifnum\tempcb<\tw@ + \yybreak{\ldpushalignment{#1}{#2}{#3}}% + \else + \yybreak{\toksc{}}% + \yycontinue + }{% + \toksc=\ifcase\tempcb{{\ttl align }}\concat\toksc\toksd\or + {{\ttl align\_with\_input}}\or + {{\ttl subalign }}\concat\toksc\toksd\fi + \advance\tempcb\@ne + }% +} + +\def\ldpushplacement#1#2#3#4{% + \ifplacementpushed + \toksc{}% + \else + \yystringempty{#1}{% any > ? + \toksc{}% + }{% + \ldextractname#1% + \toksd{{\ttl in }}% + \appendl\toksc{\the\toksd}% + }% + \yystringempty{#2}{% any AT > ? + }{% + \yytoksempty\toksc{\toksc{{\ttl as }}}{\appendr\toksc{ {\noexpand\ttl as }}}% + \toksd=\toksc + \ldextractname#2% + \concat\toksd\toksc + \toksc=\toksd + }% + \placementpushedtrue + \fi +} + +\def\ldstartpheaders#1#2#3#4{% + \tempcc=\z@ + \yystringempty{#3}{\toksc{}}{\toksc{{\ttl phdrs}}}% +} + +\def\ldextractfill#1#2#3#4{% + \yystringempty{#4}{% + \toksc{}% + \fillextractedtrue + }{% + \iffillextracted + \toksc{}% + \else + \toksc={\qquad{\ttl fill }$#4$}% + \fi + }% +} + +\def\ldpushpheaders#1#2#3#4{% + \ldextractitem\tempcc{#3}% + \advance\tempcc\@ne + \yytoksempty\toksc{}{\expandafter\ldextractname\the\toksc}% +} + +\def\ldexpwithtype#1#2#3#4#5{% TODO + \yystringempty{#2}{\toksc{$}}{\toksc{$#2}}%$ + \yystringempty{#5}{\toksd{{}$}}{#5\appendr\toksd{]$}\appendl\toksd{[}}% + \concat\toksc\toksd +} + +\defc\ldtype{% + \toksd{\hbox{\ttl #1}}% +} + +\defc\ldstatement{} +\defc\ldsectionseparator{\toksc{&\multispan5\cdotfill\quad\cr}\concat\toksa\toksc} + +\restorecs{ld-display}{\anint\ldregexp\ldassignment\ldfill} +\toyyunion{ld-parser:sections} diff --git a/support/splint/examples/ld/lstokenset.sty b/support/splint/examples/ld/lstokenset.sty new file mode 100644 index 0000000000..3ab3ae2d16 --- /dev/null +++ b/support/splint/examples/ld/lstokenset.sty @@ -0,0 +1,23 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\prettywordpair{OPTIONAL}{{\tt opt}}% +\prettywordpair{SUFFIX}{{\tt suffix}}% +\prettywordpair{EXTENDED}{{\tt ext}}% +\prettywordpair{INTEGER}{{$[\,0\ldots9\,]\ast$}}% +\prettywordpair{IDENTIFIER}{{$[\,\hbox{\tt a}\ldots\hbox{\tt Z}\,0\ldots9\,]\ast$}}% +\prettywordpair{WILDCARD}{{{\tt *} {\rm or} {\tt ?}}}% +\prettywordpair{PERCENT_IDENTIFIER}{{\tt\%$[\,\hbox{\tt a}\ldots\hbox{\tt Z}\,0\ldots9\,]\ast$}}% diff --git a/support/splint/examples/ld/ltokenset.sty b/support/splint/examples/ld/ltokenset.sty new file mode 100644 index 0000000000..d5d0d9b1ee --- /dev/null +++ b/support/splint/examples/ld/ltokenset.sty @@ -0,0 +1,91 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\prettywordpair{comma}{{\tt,}} +\prettywordpair{semicolon}{{\tt;}} +\prettywordpair@@{file\_NAME\_list}{\ifinheader\itbold\else\it\fi file\_{\rm name}\_list} +\prettywordpair@@{high\_level\_library\_NAME\_list}{\ifinheader\itbold\else\it\fi high\_level\_library\_{\rm name}\_list} +\prettywordpair@@{low\_level\_library\_NAME\_list}{\ifinheader\itbold\else\it\fi low\_level\_library\_{\rm name}\_list} +\prettywordpair@@{NAME}{{\rm name}} +\prettywordpair@@{LNAME}{{\rm name$_{\rm L}$}} +\prettywordpair@@{END}{{\rm end}} +\prettywordpair@@{UNARY}{{\rm unary}} +\prettywordpair@@{VERSIONK}{{\tt VERSION}$_{\rm K}$} +\prettytoken{ALIGN} +\prettytoken{MIN} +\prettytoken{MAX} +\prettytoken{TARGET} +\prettytoken{ASSERT} +\prettytoken{INSERT} +\prettywordpair@@{PLUSEQ}{{}$\MRL{+{\K}}${}} +\prettywordpair@@{MINUSEQ}{{}$\MRL{-{\K}}${}} +\prettywordpair@@{MULTEQ}{{}$\MRL{\times{\K}}${}} +\prettywordpair@@{DIVEQ}{{}$\MRL{{\div}{\K}}${}} +\prettywordpair@@{ANDEQ}{{}$\Xandxeq${}} +\prettywordpair@@{OREQ}{{}$\Xorxeq${}} +\prettywordpair@@{LSHIFTEQ}{{}$\MRL{\ll{\K}}${}} +\prettywordpair@@{RSHIFTEQ}{{}$\MRL{\gg{\K}}${}} +\prettywordpair@@{NE}{{}$\not=${}} +\prettywordpair@@{LE}{{}$\leq${}} +\prettywordpair@@{GE}{{}$\geq${}} +\prettywordpair@@{EQ}{{}$=${}} +\prettywordpair@@{OROR}{{}$\V${}} +\prettywordpair@@{ANDAND}{{}$\W${}} +\prettywordpair@@{LSHIFT}{{}$\ll${}} +\prettywordpair@@{RSHIFT}{{}$\gg${}} +\prettywordpair@@{=}{{}$\K${}} +\prettywordpair@@{+}{{}$+${}} +\prettywordpair@@{-}{{}$-${}} +\prettywordpair@@{!}{{}$\CM${}} +\prettywordpair@@{~}{{}$\R${}} +\prettywordpair@@{^}{{}$\XOR${}} +\prettywordpair@@{|}{{}$\OR${}} +\prettywordpair@@{&}{{}$\AND${}} + +\def\gtsymswitch{% + \raw exp *left *right\raw {% + {}$>${}% + } +} + +\def\gtsymswitchdefault{{\tt >}} + +\def\ltsymswitch{% + \raw exp *left *right\raw {% + {}$<${}% + } +} + +\def\ltsymswitchdefault{{\tt <}} + +\setspecialcharsfrom\gtsymswitch +\setspecialcharsfrom\ltsymswitch + +\prettywordpair@@{>}{{% + \let\default\gtsymswitchdefault + \switchon{#1}\in\gtsymswitch +}} + +\prettywordpair@@{<}{{% + \let\default\ltsymswitchdefault + \switchon{#1}\in\ltsymswitch +}} + +\prettywordpair@@{*}{\yyifsamestring{exp}{#1}{{}$\times${}}{{\tt *}}} + +\prettywordpair@@{/}{{}$/${}} + +\expandafter\prettywordpair@@\expandafter{\harmlesscomment}{{}$\div${}} diff --git a/support/splint/examples/symbols/Makefile b/support/splint/examples/symbols/Makefile new file mode 100644 index 0000000000..d22110baaf --- /dev/null +++ b/support/splint/examples/symbols/Makefile @@ -0,0 +1,37 @@ +# this Makefile uses a flat directory structure for demonstration +# purposes; the main directory (../..) follows a slightly more +# modular organization scheme. + +SPLINT_ROOT = $(shell pwd)/../.. +SYMBOLS_PREREQS = %.tex %.idx ${SPLINT_XTEXSTYLES} ${SPLINT_ROOT}/cweb/bo.tok symtoks.sty +SYMBOLS_PREREQS += ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} symmap.sty ${SPLINT_ROOT}/cweb/bg.y + + +include ${SPLINT_ROOT}/makefile.inc + +# rules specific to this example + +%.tex: %.w + +%.tex %.idx: %.w + ${CWEAVE} $< + +%.gdx: ${SYMBOLS_PREREQS} + ${TEX} $*.tex + +%.pdf %.sns: ${SYMBOLS_PREREQS} %.gdy + ${PDFTEX} $*.tex && touch $*.gdy && touch $*.pdf + +docs: symbols.pdf + +clean: clean_core + rm -f symbols.tex + +mostlyclean: clean_temp + +distclean: clean + cd ${SPLINT_ROOT} && ${MAKE} clean + +.PRECIOUS: symbols.gdy + +.INTERMEDIATE: symbols.gdx diff --git a/support/splint/examples/symbols/symbols.w b/support/splint/examples/symbols/symbols.w new file mode 100644 index 0000000000..47c210e4f8 --- /dev/null +++ b/support/splint/examples/symbols/symbols.w @@ -0,0 +1,193 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. +\newwrite\gindex + +\input limbo.sty +\def\optimization{5} +\input yy.sty +\input dcols.sty +\input symmap.sty + +\def\symnamespace{[symbols]} + +\let\currentnamespace\parsernamespace +\let\parsernamespace\symnamespace + \input symtoks.sty % +\let\parsernamespace\currentnamespace +\let\hostparsernamespace\symnamespace % the namespace where tokens are looked up + % for typesetting purposes + +\immediate\openout\exampletable=\jobname.exl + +\def\cite[#1]{% + \def\next{#1}\setbox0=\hbox{l} + [\ifx\next\empty$\,$\hbox{\vrule width\wd0 height\ht0 depth\dp0}$\,$\else #1\fi]% +} + +\let\oldN\N +\let\N\textN +\let\M\textM + +\defreserved{Y}{\.{Y}} + +@** Introduction. \setupfootnotes +The same parser and lexer (with a slightly different input routine) +are used to typeset examples of productions in text. +Some of the features of this parser +are collected below for future reference. One might keep this section +handy when typesetting his own examples (for example, it is probably +unintuitive that `\.{`}' produces `\.{\yl}' but there is simply no +way to use `\.{\yl}' as a character inside the \TeX\ section of \CWEB). +The first, rather eclectic and lengthy example demonstrates various +typesetting features of the \bison\ parser. The parsing output as well +as the resulting table are saved in {\tt \jobname.exl}. All the \Cee\ +typesetting is performed by \CWEB, using its \.{\yl}$\ldots$\.{\yl} +facility. +\saveparseoutputtrue +\expandafter\def\csname lexspecial[^^D^^D]\endcsname{} +\medskip +\beginprod +\inline +example: + term.1 term.2 \{\} term.3 \stashed{\relax} \{\stashed{\relax}\} +` term.more \{\} "nonterminal"[sym_name.1] term_other[sym_name.2] \{\} +` \ +` \%empty \ +` terms terms ',' terms \%? \{ \stashed{|a = b = c = d = e;|} \} \%dprec 7 +` terms terms ',' terms \%? \{ \stashed{|a = b = c;|} \} +; +% +another: + term.8 \%merge <some_tag> term.one \%dprec 3 term.two \{\stashed{\rm|int a, b, c;|}\} +` term.17 \%merge <other_tag> \{\stashed{|f(a,b)==c;|}\} +` term.78 \{\stashed{|h(b)=g(c);|}\} \%merge <tag.2> \%dprec 0x17 +` term.77 \{\stashed{|h(b)=g(c);|}\} \%prec new_term +; +% +\resetf +and_another: + term.8 \%merge <some_tag> term.one \%dprec 3 term.two \{\stashed{\rm|int a, b, c;|}\} +` term.17 \%merge <other_tag> \{\stashed{|f(a,b)==c;|}\} +` term.78 \{\stashed{|h(b)=g(c);|}\} \%merge <tag.2> \%dprec 0x17 +` term.77 \{\stashed{|h(b)=g(c);|}\} \%prec new_term +; +% +\%token ANOTHER NONEXISTENT GENERIC TOKEN 7 "token" ANOTHER 0x77 "more" TOKEN TOKEN ; +\%token bogey1 bogey2 ; +\%type <some> TOKEN ANOTHER ; +\%start inputer; +\stashed{\rm Example 1 of flushed code}\sflush{F}{flush this} +\%default-prec; +\%no-default-prec; +\stashed{\rm Example 2 of flushed code} +\%destructor \{ \stashed{\rm |func(int a, char b); a = b + c;|}\^^D\^^D % ignored because anything is accepted inside braces + \stashed{\6\rm |func2(int a, char b);|} \} \^^D\^^D + "token" TOKEN NONEXISTENT "none" BOGEY "more" A_TOKEN IDENTIFIER; +\%printer \{ \stashed{\rm |func(int a, char b); a = b + c;|} + \stashed{\6\rm |func2(int a, char b);|} \} \^^D\^^D + "token" TOKEN NAME NONEXISTENT "none" ANOTHER BOGEY "more" A_TOKEN identifier.1 identifier.2; +\%code token.3 \{ \stashed{\rm |func(int a, char b); a = b + c;|} + \stashed{\6\rm |func2(int a, char b);|} \}; +\%code \{ \stashed{\rm |new_function(int x, char y); |} \}; +\%left <whoops> "one" 1 "two" 2 three.137 0x7; +\%precedence five six seven; +\%nonassoc two;\^^D\^^D +\%code \{ \stashed{\rm |other_function(int x, char y); |} \}; +\endprod +\medskip +%\checktabletrue +\beginprod +\%expect 0x137; +\%expect-rr 17; +\%lex-param \{\stashed{|int number;|}\}; +\%define var.1 \{ \stashed{ |func3(8, "string"){n = m++;};| } \} +\%union var.2 \{ \stashed{\rm |int a, b, c;|\6 |char a_char;|} \} +\%\{ \stashed{\rm |int a, b, c;|\6 |char a_char;|} \%\} +\endprod +% +\medskip +\noindent The next example is a demonstration of the hidden context +added to an incomplete language fragment and local typesetting +variations enabled by such context. +\def\cset#1{% + \nx\colorset{darkwood}% + {#1}% + \nx\restorecolor +}% +\def\dset#1{% + $\nx\underbrace{\hbox{#1}}_{{\nx\rm identifier:\ \the\toksc}}$% +}% +\def\esets#1{% + \nx\beginub#1% +}% +\def\esete#1{% + #1\nx\endub +}% +\def\beginub#1\endub{% + $\underbrace{\hbox{#1}}_{\rm a\ group}$% +} +\smallskip +\beginprod +\skipheader +ghost: + headerless_term.1 \formatlocal{\let\termmetastyle\cset} headerless_term.2 \{\stashed{\colorset{link}\rm|color(x,y,z);|\restorecolor}\} +` \formatlocal{\let\termmetastyle\dset}more.of.the.same.0\formatlocal{\restorecs{table-render}\termmetastyle} but.not.here + \{\stashed{\rm|color(a, b, c);|}\} +` \formatlocal{\let\termmetastyle\esets}three.more.terms\formatlocal{\restorecs{table-render}\termmetastyle}follow + \formatlocal{\let\termmetastyle\esete}this\formatlocal{\restorecs{table-render}\termmetastyle}one\{\stashed{\rm|assign(x, y, z);|}\} +; +\endprod +\medskip +\noindent Finally, an incomplete listing of the characters that can be +typeset, as well as the way to typeset them. The use of `\.{`}' to +typeset `\.{\yl}' deserves a special note---\CWEB's rules make it +nearly impossible to use `\.{\yl}' in the \TeX\ portion of the program. +\medskip +\tomainparser +\prettywordpair{GENERIC}{\_Generic} +\prettywordpair{ss}{$^{\rm C99[}\,$\aftergroup\aftergroup\aftergroup\ignorespaces} +\prettywordpair{es}{\unskip$\,{}^{\rm ]C99}$}% there is still a problem when this appears in headers +\beginprod +line_breaking_and_symbols: + GENERIC '(' expression',' \ ss another es')' \ + ` inline_c \{ \stashed{\X{$\infty$}:See this example to deduce $\ldots$\X\6}\stashed{|b == a - c|} \} + ` more_inline_c \{ \stashed{|func(int a, char b);|} \} +% +\endprod +\medskip +\beginprod +\format{\inline\flatten} +symbol_tricks: + '\&' \ + ` '*' \ + ` '+' \ + ` '-' \ + ` '\~' \ + ` '!' \ + ` '\{' \ + ` '`' \ + ` '\`' \ + ` '\'' \ + ` '\\' \ + ` ' ' \ + ; +\endprod + +\closeout\gindex +@** Index. Totally superfluous in this case. +\let\inx\inxmod +\let\fin\finmod +\def\topofcontents{\null\vskip-3\baselineskip\centerline{C{\sc ONTENTS} (\sc\uppercase\expandafter{\title})}\medskip} diff --git a/support/splint/examples/symbols/symmap.sty b/support/splint/examples/symbols/symmap.sty new file mode 100644 index 0000000000..12261be656 --- /dev/null +++ b/support/splint/examples/symbols/symmap.sty @@ -0,0 +1,164 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% assigning symbolic names to production terms (this is only a demonstration) +% no attempt is made to make this code modular or namespace safe; +% the approach is not very elegant and can stand a lot of improvement; +% it was decided against such optimization to present a more explicit demonstration +% of the procedures + +\def\makesymbols{% + \ifsymdemo + \restorecslist{symbols}\yyunion + \toksa{}\toksb{}\toksc{}\toksd{}% + %\showthe\newsymswitch + \the\symstream + \global\newsymswitch\newsymswitch + \else + \newsymswitch{}% otherwise \yysymswitch is trying to expand the wrong names (\namepair is not present to do \csname ... ) + \fi +} + +% symbolic reference name parser + +\newtoks\newsymswitch +\newread\symnames + +\newwrite\symbolicswitch +\newif\ifsymdemo + +\symdemotrue + +\ifbootstrapmode + \symdemofalse +\fi + +\ifsymdemo + \def\inamespace{[intermediate]}% + \let\parsernamespace\inamespace + \let\hostparsernamespace\mainnamespace % purely for testing reasons + \pinittoks{}% + \input cweb/gyytab.tex % this should be the parser that will be used later + % in this case it is just an example + \edef\tointermediateparser{% + \noexpand\savefullstate % unnecessary + \noexpand\savefullstateextra % unnecessary + \let\noexpand\parsernamespace\noexpand\inamespace % switch to the new namespace + \the\pinittoks % restore all the tables, tokens and constants, and stacks + \let\noexpand\getcurrentparser\noexpand\tointermediateparser + }% + \settokens % this simply assign values to tokens where the name of each token is taken out of yytname ... + \input \drvname.tok % this will set up token equivalences in the namespace above ... + % those are the values gleaned during the bootstrapping stage. + % in the general case, one needs to run a bootsrapping (or similar) + % parser to extract the token information. + \optimizeall % this is necessary for correct rule listing in the output stage: + % otherwise \fgetelemof will use the current value of the \yy... token registers which + % will hold the values of the full parser that is loaded next + \newsymswitch{}% + \listrules % ... to be used while listing the rules + % note that we do not bother to set up a lexer for this parser (even though we already have one and the + % \optimizeall macro above will create a set of associative tables for it---this is merely an unwanted + % sideffect); after the rules have been listed, the intemediate parser in no longer needed. + %\showthe\newsymswitch + \def\fullnamespace{[full]}% this is the parser that parses the bison grammar from a raw + % bison file; it can play a role of the bootstrap parser for + % the grammar above, as well, since its input is a complete + % bison file; note that it cannot be a bootstrap parser for itself + % because it will reject any input that does not form a complete + % bison file; this is why a special grammar was created that includes + % only a small subset of the full set of productions (%token rules only) to + % serve as a bootstrap grammar. + %\tomainparser % this will set the value of \setflexstates, so that, if the lexer initialization + % below is omitted, the \setflexstates macro can still be used to correctly set the + % lexer states + \let\parsernamespace\fullnamespace + \pinittoks{}% %% + \input cweb/fyytab.tex + \input cweb/ltab.tex % we reinitialize the lexer out of necessity: + % \collecttokennames needs to know how to switch + % between two lexer/parser environments so both states + % have to be preserved; if one is sure that + % grammar_declarations will not used in the productions + % part of the file, this can simply be disabled + \settokens + \setflexstates % the main lexer can be reused in this case; the states still need to be set up + \input \drvname.tok % set up the tokens for the bison grammar parser + \newparserstate + \newlexerstate + \newlexerstateextra + % + \setnulstack{yyirulestack}% + % + \edef\tofullparser{% + \noexpand\savefullstate + \noexpand\savefullstateextra + \let\noexpand\parsernamespace\noexpand\fullnamespace % switch to the new namespace + \the\pinittoks % restore all the tables, tokens and constants, and stacks + \let\noexpand\getcurrentparser\noexpand\tofullparser + }% + \optimizeall + \toksa{\input bg.y}% start building the parsing command + % this has to be done carefully, since all the characters input + % from the parser file have to be `harmless', so their categories + % have to be reset; in order for the parser to be able to stop, + % appropriate command sequences would have to be inserted at the end + \toksb{\yyeof\yyeof\endparseinput\endparse + \let\yyinput\yyinputold + \undoascii + \ifyyparsefail + \errmessage{could not process symbols}% + \else + \symstream\table + \tointermediateparser % the parsing is finished, so we have two very long strings: + % o the rules of the hosted parser ([intermediate]) + % o the rules of the same parser just parsed + % these two strings are used to associate the symbolic names + % (which only exist in the second list) to the rule numbers and + % the relevant terms; + % switching the parse namespace above is done so that the term + % indices are looked up in the appropriate table + %\showthe\table + \makesymbols % list all the rules deriving explicit non-terminals + %\showthe\newsymswitch + \setexplicitinlinerules\newsymswitch % process implicit non-terminals resulting from inline actions + %\showthe\newsymswitch + \makesymrefs\newsymswitch % create the switch to be used by the parser (\yyparse) + \fi + }% + % build the command to create the symbolic name switch + \toksc{\tofullparser\parserinit + \let\yyinputold\yyinput + \let\yyinput\yyinputtrivial % a demo of a stripped down, slightly faster input routine + \doascii{11}\expandafter\yyparse}% + \edef\next{\the\toksc\the\toksa\relax\the\toksb}\next % \relax is to stop \TeX\ from trying to expand the file name further + % it is not flagged as a bad character because it is part of the epilogue + \immediate\openout\symbolicswitch=\jobname.sns + % + \edef\next{\setsncommands{\def\noexpand\symswitch\hashletter1{\harmlesscomment^^J\noexpand\ifcase\hashletter1\relax + \harmlesscomment^^J\the\setsncommands + \noexpand\else\harmlesscomment^^J% + \noexpand\fi\harmlesscomment^^J% + }\harmlesscomment^^J^^J}}\next + \edef\next{\unsetsncommands{\def\noexpand\symswitchoff\hashletter1{\harmlesscomment^^J\noexpand\ifcase\hashletter1\relax + \harmlesscomment^^J\the\unsetsncommands + \noexpand\else\harmlesscomment^^J% + \noexpand\fi\harmlesscomment^^J% + }\harmlesscomment^^J^^J}}\next + {\newlinechar=`\^^J \immediate\write\symbolicswitch{\the\setsncommands\the\unsetsncommands}}% + \immediate\closeout\symbolicswitch + \tomainparser % go back to the main parser +\fi diff --git a/support/splint/examples/symbols/symtoks.sty b/support/splint/examples/symbols/symtoks.sty new file mode 100644 index 0000000000..8d6cdc5596 --- /dev/null +++ b/support/splint/examples/symbols/symtoks.sty @@ -0,0 +1,8 @@ +\prettywordpair{ANOTHER}{{\rm(}another\rm{)}} +\prettywordpair{TOKEN}{$\langle${\it token}$\rangle$} +\prettywordpair{GENERIC}{{\tt \_Generic}} +\prettywordpair{IDENTIFIER}{{\tt Identifier}} +\prettywordpair{BOGEY}{{\tt \_Bogey}} +\prettywordpair{NAME}{{\tt NAME}} +\prettywordpair{A_TOKEN}{{a\_token.}} +\prettywordpair{NONEXISTENT}{$\hbox{\tt Non}^{\rm existent}$} diff --git a/support/splint/examples/types/basic.sty b/support/splint/examples/types/basic.sty new file mode 100644 index 0000000000..856faf2374 --- /dev/null +++ b/support/splint/examples/types/basic.sty @@ -0,0 +1,464 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% various macros that implement `expandable arithmetic' that can be +% used with the tree evaluator macros; +% these are here as an example only; a much more complete version of +% big integer arithmetic (including division and exponentiation) is +% implemented in the `bigintcalc' package by Heiko Oberdiek; I was +% unaware of the existence of `bigintcalc' when these macros were written; +% note that the tree evaluator macros perform recursive expansion of +% their arguments; `bigintcalc' uses \number for the same purpose +% (since \number will keep expanding tokens until a non digit is +% encountered) + +% macros that expand into a sequence of given length + +% #1 is the 1-radix of the number read so far +% #2 is the `digit' +% #3 is the next 10-radix digit to be processed or `S' + +\def\unroll#1#2#3{\csname unroll#3\endcsname{#1}{#2}} +\expandafter\def\csname unroll0\endcsname#1#2{\unroll{#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll1\endcsname#1#2{\unroll{#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll2\endcsname#1#2{\unroll{#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll3\endcsname#1#2{\unroll{#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll4\endcsname#1#2{\unroll{#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll5\endcsname#1#2{\unroll{#2#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll6\endcsname#1#2{\unroll{#2#2#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll7\endcsname#1#2{\unroll{#2#2#2#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll8\endcsname#1#2{\unroll{#2#2#2#2#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} +\expandafter\def\csname unroll9\endcsname#1#2{\unroll{#2#2#2#2#2#2#2#2#2#1#1#1#1#1#1#1#1#1#1}{#2}} + +\def\unrollS#1#2#3{% + \ifx#3F% + \xskiptofi{#1S}% + \else + \xskiptofi{G{#1}}% + \fi +} + +\def\sequence#1#2{\expandafter\s@quence\expandafter{\number#1}{#2}} +\def\s@quence#1#2{\unrollbegin{}{#2}{}#1} + +\def\unrollbegin#1#2#3#4#5{% + \ifx#5S% + \ifx#40% + \yybreak@{S}% + \else + \yybreak@{\unroll{#1}{#2}{#3}#4#5}% + \fi + \else + \yybreak{\unroll{#1}{#2}{#3}#4#5}% + \yycontinue +} + +% macros that count the number of non-S in a sequence + +\def\startconversion#1{% + \ifx#1S% + \xskiptofi{0}% + \else + \xskiptofi{\convertone{}{}}% + \fi +} + +\def\convertzer@#1#2#3{% + \ifx#3S% + \xskiptofi{#1}% + \else + \xskiptofi{\convertone{0#1}{#2}}% + \fi +} + +\def\convertzero#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzer@{#1}{}#2S}% + \else + \xskiptofi{\convertone{#1}{#2}}% + \fi +} + +\def\convertone#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{1#1}{}#2S}% + \else + \xskiptofi{\converttwo{#1}{#2}}% + \fi +} + +\def\converttwo#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{2#1}{}#2S}% + \else + \xskiptofi{\convertthree{#1}{#2}}% + \fi +} + +\def\convertthree#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{3#1}{}#2S}% + \else + \xskiptofi{\convertfour{#1}{#2}}% + \fi +} + +\def\convertfour#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{4#1}{}#2S}% + \else + \xskiptofi{\convertfive{#1}{#2}}% + \fi +} + +\def\convertfive#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{5#1}{}#2S}% + \else + \xskiptofi{\convertsix{#1}{#2}}% + \fi +} + +\def\convertsix#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{6#1}{}#2S}% + \else + \xskiptofi{\convertseven{#1}{#2}}% + \fi +} + +\def\convertseven#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{7#1}{}#2S}% + \else + \xskiptofi{\converteight{#1}{#2}}% + \fi +} + +\def\converteight#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{8#1}{}#2S}% + \else + \xskiptofi{\convertnine{#1}{#2}}% + \fi +} + +\def\convertnine#1#2#3{% + \ifx#3S% + \xskiptofi{\convertzero{9#1}{}#2S}% + \else + \xskiptofi{\convertzero{#1}{#2#3}}% + \fi +} + +% list evaluator + +\def\expander#1#2{% + \ifcat\noexpand#2\relax + \xskiptofi{\@xp@nder#1}% + \else + \ifx#2G% + \grabexpanderpostfix{#1}% + \else + \xskiptofifi{#1}% + \fi + \fi + #2% +} + +\def\@xpander#1#2{% + \ifcat\noexpand#2\relax + \xskiptofi{\expandafter\expander\expandafter{\expandafter#1\expandafter}#2}% + \else + \xskiptofi{\@xpander{#1\expandafter#2}}% + \fi +} + +\def\@xp@nder#1{% + \ifcat\noexpand#1\relax + \xskiptofi{\expandafter\expander\expandafter{\expandafter}}% + \else + \xskiptofi + \@xpander% + \fi + #1% +} + + +\def\grabexpanderpostfix#1\else#2\fi\fi G#3{% + \fi\fi\expander{#1#3}% +} + +% the next two macros set up the multiplication and addition tables in the form of +% \xdigit a b c which expands to the two digits of a\times b + c +% and +% \sdigit a b c which expands to the two digits of a + b + c +% where c \in \{0, 1\}. +% altogether there are 1200 sequences in use. this number can be +% reduced to under 500 at the expence of more sophisticated +% conditionals: +% o in the case of \xdigit, a == 1 or b == 1 reduces \xdigit a b c to +% \sdigit a c 0 (for b == 1), whereas a == 0 or b == 0 makes +% \xdigit a b c {c}{0}, finally for a == 2 or b == 2, \xdigit a b c is +% the same as \sdigit a a c (if b == 2); in addition, \xdigit a b 0 +% and \xdigit a b 1 are the same for all pairs (a,b) other than (3,3) +% and (7,7); thus, the number of \xdigit sequences can be reduced to +% 261 (by also requiring that a < b) +% o similar techniques can be used to reduce the number of \sdigit +% sequences necessary to under 60 + +\def\setxdigitmachine{% + \tempca=0 + \loop + \ifnum10>\tempca + \expandafter\setxdcs\expandafter0\expandafter0\the\tempca + \else + \ifnum100>\tempca + \expandafter\setxdcs\expandafter0\the\tempca + \else + \expandafter\setxdcs\the\tempca + \fi + \fi + \advance\tempca\@ne + \ifnum1000>\tempca + \repeat +} + +\def\setsdigitmachine{% + \tempca=0 + \loop + \ifnum10>\tempca + \expandafter\setsdcs\expandafter0\expandafter0\the\tempca + \else + \ifnum100>\tempca + \expandafter\setsdcs\expandafter0\the\tempca + \else + \expandafter\setsdcs\the\tempca + \fi + \fi + \advance\tempca\@ne + \ifnum200>\tempca + \repeat +} + +% #1: carry +% #2: the first summand +% #3: the second summand + +% the result: +% {sum digit}{new carry} + +\def\setsdcs#1#2#3{% + \tempcb=#2 + \advance\tempcb#3 + \advance\tempcb#1 + \ifnum10>\tempcb + \expandafter\edef\csname sdigit#2#3#1\endcsname{\expandafter\parenthesize\expandafter0\the\tempcb}% + \else + \expandafter\edef\csname sdigit#2#3#1\endcsname{\expandafter\parenthesize\the\tempcb}% + \fi +} + +% #1: the first multiplier +% #2: the second multiplier +% #3: carry + +% the result: +% {product digit}{new carry} + +\def\setxdcs#1#2#3{% + \tempcb=#1 + \multiply\tempcb#2 + \advance\tempcb#3 + \ifnum10>\tempcb + \expandafter\edef\csname xdigit#1#2#3\endcsname{\expandafter\parenthesize\expandafter0\the\tempcb}% + \else + \expandafter\edef\csname xdigit#1#2#3\endcsname{\expandafter\parenthesize\the\tempcb}% + \fi +} + +\def\parenthesize#1#2{{#2}{#1}} + +\setxdigitmachine +\setsdigitmachine + +% #1: carryover +% #2: shift +% #3: multiplier +% #4: carry +% #5: digits so far +% #6: next digit + +\def\smallmultiply#1#2#3#4#5#6{% + \ifx#6F% + \ifx#40% + \yybreak@{\attachnxtnumber#1{#2#5}}% + \else + \yybreak@{\attachnxtnumber#1{#2#5#4}}% + \fi + \else + \yybreak{\expandafter\expandafter\expandafter + \sm@llmultiply\csname xdigit#3#6#4\endcsname{#1}{#2}{#3}{#5}}% + \yycontinue +} + +\def\sm@llmultiply#1#2#3#4#5#6{% + \smallmultiply{#3}{#4}{#5}{#2}{#6#1}% +} + +% carryover: +% {number1}{number2}{shift}{product so far} + +% #1: number1 +% #2: number2 +% #3: shift +% #4: product so far +% #5; new number + +\def\attachnxtnumber#1#2#3#4#5{% + \startnxtproduct{#1}{#2}{#30}{#4#5F}#1% +} + +% #1: number1 +% #2: number2 +% #3: shift +% #4: product so far +% #5; next digit + +\def\startnxtproduct#1#2#3#4#5{% + \ifx#5F% + \xskiptofi{\addmultiples#4G}% + \else + \xskiptofi{\expandafter\sm@llm@ltiply\expandafter{\expandafter{\eatone#1}{#2}{#3}{#4}}{#3}{#5}{0}{#2}}% + \fi +} + +% #1: carryover +% #2: shift +% #3: multiplier +% #4: carry +% #5: number2 +% #6: remainder of the number + +\def\sm@llm@ltiply#1#2#3#4#5#6F{% + \smallmultiply{#1}{#2}{#3}{#4}{}#5% +} + +\def\addmultiples#1F#2{% + \ifx#2G% + \xskiptofi{\postprocesssum{#1}}% + \else + \xskiptofi{\startnxtsum{#1}#2}% + \fi +} + +\def\postprocesssum#1#2{% + #2{#1}% +} + +% the summation macro below can be used for subtraction, as well, +% using a well known 9-complement + 1 technique (compare two numbers, +% take the complement of the smaller, add 1, add the results, remove +% the first nonzero digit of the sum; the result of the comparison +% determines the sign of the sum; + +\def\startnxtsum#1#2F{% + \startnxts@m{}{0}{#1}{#2}% +} + +% #1: sum so far +% #2: carry +% #3: first number +% #4: second number + +\def\startnxts@m#1#2#3#4{% + \ifx F#3F% the first number is exhausted + \ifx F#4F% both numbers are exhausted: finished + \yybreak@{\addmultiples#1#2F}% + \else % the second number is still non empty + \yybreak@{\makesimples@m{#1}{#2}0F#4F}% + \fi + \else + \ifx F#4F% the second number is exhausted + \yybreak@{\makesimples@m{#1}{#2}#3F0F}% + \else % both numbers are non empty + \yybreak@{\makesimples@m{#1}{#2}#3F#4F}% + \fi + \yycontinue +} + +% #1: sum so far +% #2: carry +% #3: first digit +% #4: rest of first number +% #5: second digit +% #6: rest of second number + +\def\makesimples@m#1#2#3#4F#5#6F{% + \expandafter\expandafter\expandafter\makesimpl@s@m\csname sdigit#3#5#2\endcsname{#1}{#4}{#6}% +} + +% #1: digit +% #2: carry +% #3: sum so far +% #4: first number +% #5: second mumber + +\def\makesimpl@s@m#1#2#3#4#5{% + \startnxts@m{#3#1}{#2}{#4}{#5}% +} + +% #1: prefix +% #2: postfix +% #3: digits + +\def\reversepp#1#2#3{% + \r@versepp{#1}{#2}{}#3R% +} + +\def\r@versepp#1#2#3#4{% + \ifx#4R% + \xskiptofi{#1#3#2}% + \else + \xskiptofi{\r@versepp{#1}{#2}{#4#3}}% + \fi +} + +\def\xmul#1#2{% + \reversepp{\xm@l}{F}{#2F#1}% +} + +\def\xm@l#1F#2F{% + \startnxtproduct{#1F}{#2F}{}{}#1F{\reversepp{\eatzeros}{F}}% +} + +\def\xsum#1#2{% + \reversepp{\addmultiples}{FG{\reversepp{\eatzeros}{F}}}{#2F#1}% +} + +\def\eatzeros#1{% to remove zero carry digits + \ifx#10% + \yybreak\eatzeros + \else + \ifx#1F% + \yybreak@0% + \else + \yybreak@{\removepostfix#1}% + \fi + \yycontinue +} + +\def\removepostfix#1F{#1} diff --git a/support/splint/examples/types/test.sty b/support/splint/examples/types/test.sty new file mode 100644 index 0000000000..8a7859d3d6 --- /dev/null +++ b/support/splint/examples/types/test.sty @@ -0,0 +1,63 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% some examples of the `tree evaluator' and the `expandable arithmetic' macros + +\input ../../yycommon.sty +\input ../../yymisc.sty +\input basic.sty +\input tree.sty + +% defining the node arities for the tree evaluator + +\nodesetup{xmul}{2} +\nodesetup{xsum}{2} +\nodesetup{xxttop}{0} +\nodesetup{the}{0} +\nodesetup{number}{0} + +%\tracingmacros=3 +%\tracingonline=3 + +\tempca=77 + +\edef\asum{\xsum{0}{21809808}} +\edef\aproduct{\xmul{118768770987098709870987098787979}{69876987698769824320980}} + +\message{product: \aproduct, sum: \asum} + +\def\anexpression{\treeeval{\xmul{\xsum{\xsum{1}{\xmul{2764567}{74674}}}{3}}{\xsum{\the\tempca}{\xsum{\number\tempcb}{\xmul{5764567}{7}}}}}} + +\message{value of the expression: \anexpression} + +% `symbolic definitions' of the nodes for the tree evaluator + +\def\xsum#1#2{(#1 + #2)} +\def\xmul#1#2{#1 * #2} + +\message{expression: \anexpression} + +%\tracingmacros=0 +%\tracingonline=0 + +%\tempca=17 + +%\edef\seq{\noexpand\startconversion\expander{}\sequence\tempca{*}SS\sequence\tempca{.}SS\sequence\tempca{.}SF} +%\show\seq +%\edef\cseq{\seq} +%\show\cseq + +\end diff --git a/support/splint/examples/types/tree.sty b/support/splint/examples/types/tree.sty new file mode 100644 index 0000000000..46a4c94a97 --- /dev/null +++ b/support/splint/examples/types/tree.sty @@ -0,0 +1,190 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% tree evaluator macros. + +% #1: first token of a leaf + +\def\leafexpand#1{% + \ifcat\noexpand#1\relax % the leaf starts with an expandable token or control sequence + \xskiptofi{\expandafter\leafexpand}% + \else + \xskiptofi{\leafattach}% + \fi + #1% +} + +% #1: the leaf +% #2: {{pruned tree}{last pruned branch}} + +\def\leafattach#1\endleaf#2{% + \leafatt@ch{#1}#2% +} + +% #1: the leaf +% #2: pruned tree +% #3: last pruned branch + +\def\leafatt@ch#1#2#3{% + \leaf@tt@ch{#1}{#2}#3% +} + +\def\xname{\expandafter\eatone\string} + +% #1: the leaf +% #2: pruned tree +% #3: treenode type + +\def\leaf@tt@ch#1#2#3{% + \csname\xname#3.ifargdone\endcsname + % all leaves of the last pruned node have been evaluated, evaluate the node + \xskiptofi{\csname leafprune\csname\xname#3.arity\endcsname\endcsname{#2}#3{#1}}% + \else + % look at the next argument + \xskiptofi{\csname nextargument\csname\xname#3.arity\endcsname\endcsname{#2}#3{#1}}% rest of the arguments follow + \fi +} + +% #1: pruned tree +% #2: treenode type +% #3: argument + +\expandafter\def\csname leafprune0\endcsname#1#2#3{% + #3% +} + +% #1: pruned tree +% #2: treenode type +% #3: argument + +\expandafter\def\csname leafprune1\endcsname#1#2#3{% + \leafexpand#2{#3}\endleaf{#1}% +} + +% #1: pruned tree +% #2: treenode type +% #3: argument1 +% #4: argument2 + +\expandafter\def\csname leafprune2\endcsname#1#2#3#4{% + \leafexpand#2{#3}{#4}\endleaf{#1}% +} + +% #1: pruned tree +% #2: treenode type +% #3: argument1 + +\expandafter\def\csname nextargument1\endcsname#1#2#3{% + \expandafter\expandafter\expandafter + \treedesc@nd#3\endleaf{{#1}{#2}}% +} + +% #1: pruned tree +% #2: treenode type +% #3: argument1 +% #4: argument2 + +\expandafter\def\csname nextargument2\endcsname#1#2#3#4{% + \expandafter\expandafter\expandafter + \treedescend\csname\xname#2.argshift\endcsname{#4}{{#3}}{#1}% +} + +% #1: treenode type +% #2: last argument +% #3: rest of arguments +% #4: pruned tree + +\def\treedescend#1#2#3#4{% + \treedesc@nd#2\endleaf{{#4}{#1#3}}% +} + +% #1: treenode type or leaf + +\def\treedesc@nd#1{% + \ifcat\noexpand#1\noexpand\relax % in case someone redefined \relax + % this is an argument that has to be expanded + \xskiptofi{\csname treed@sc@nd\csname\xname#1.arity\endcsname\endcsname}% + \else + % this is a leaf + \xskiptofi{\leafattach}% + \fi + #1% +} + +% data type, expand it + +\expandafter\def\csname treed@sc@nd0\endcsname{% + \expandafter\leafattach % \leafattach will consume \endleaf +} + +% unary operation +% #1: treenode type +% #2: argument1 +% #3: pruned tree + +\expandafter\def\csname treed@sc@nd1\endcsname#1#2\endleaf#3{% + \csname nextargument1\endcsname{#3}#1{#2}% +} + +% binary operation +% #1: treenode type +% #2: argument1 +% #3: argument2 +% #4: pruned tree + +\expandafter\def\csname treed@sc@nd2\endcsname#1#2#3\endleaf#4{% + \csname nextargument2\endcsname{#4}#1{#2}{#3}% +} + +\def\treeeval#1{% + \treeev@l#1% +} + +\def\treeev@l#1{% + \csname nextargument\csname\xname#1.arity\endcsname\endcsname{{}\xxttop}#1% +} + +% #1: node name +% #2: arity + +\def\nodesetup#1#2{% + \expandafter\let\csname #1.ifargdone\endcsname\iftrue + \ifnum#2>0 + \expandafter\edef\csname #1.argshift\endcsname{% + \expandafter\noexpand\csname #11\endcsname + }% + \fi + \expandafter\edef\csname #1.arity\endcsname{#2}% + \tempca\@ne + \loop + \ifnum\tempca<#2 + \expandafter\def\csname #1\the\tempca.arity\endcsname{#2}% + \makeargdonefalse{#1}% + \tempcb\tempca + \advance\tempca\@ne + \ifnum\tempca<#2 + \expandafter\edef\csname #1\the\tempcb.argshift\endcsname{% + \expandafter\noexpand\csname #1\the\tempca\endcsname + }% + \else + \expandafter\edef\csname #1\the\tempcb.argshift\endcsname{% + \expandafter\noexpand\csname #1\endcsname + }% + \fi + \repeat +} + +\def\makeargdonefalse#1{\expandafter\let\csname #1\the\tempca.ifargdone\endcsname\iffalse} diff --git a/support/splint/examples/xxpression/Makefile b/support/splint/examples/xxpression/Makefile new file mode 100644 index 0000000000..4450e3126d --- /dev/null +++ b/support/splint/examples/xxpression/Makefile @@ -0,0 +1,74 @@ +# this Makefile uses a flat directory structure for demonstration +# purposes; the main directory (../..) contains a slightly more +# modular organization. + +SPLINT_ROOT = $(shell pwd)/../.. +XXPRESSION_PREREQS = %.tex %.scn %.idx %.tok ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} ${SPLINT_ROOT}/cweb/bo.tok xymmap.sty xtoks.sty + +include ${SPLINT_ROOT}/makefile.inc +include ${SPLINT_ROOT}/makefile.loc + +# rules specific to this example + +PARSER = xxpp +LEXER = xxpl + +all: xxpression.pdf ptab.tex ltab.tex + +step1: ${PARSER}.y ${LEXER}.l +step2: ptabout ltabout +step3: ltab.tex ptab.tex +step4: xymbols.sns + +ptabout: ${SPLINT_ROOT}/cweb/mkeparser.c ${PARSER}.c + ${CC} -DPARSER_FILE=\"../examples/xxpression/$(lastword $^)\" -DYYPARSE_PARAMETERS= -o $@ $< + +ptab.tex: ptabout + $< --optimize-actions $@ + +ltabout: ${SPLINT_ROOT}/cweb/mkscanner.c ${LEXER}.c + ${CC} -DLEXER_FILE=\"../examples/xxpression/$(lastword $^)\" -o $@ $< + +ltab.tex: ltabout + $< --optimize-actions $@ + +${LEXER}.c: ${LEXER}.l + ${FLEX} -o $@ $< + +test.tex xymbols.tex \ +${PARSER}.yy ${LEXER}.ll: xxpression.x + @${CTANGLE} $< + ${UNLINE} test.txx test.tex + ${UNLINE} xymbols.txx xymbols.tex + +xymbols.sns: xymbols.tex xxpression.tok ${SPLINT_ROOT}/cweb/bo.tok ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} ptab.tex + ${PDFTEX} $< + +xxpression.tex: xxpression.x + ${CWEAVE} $< + +xxpression.gdx: %.gdx: ${XXPRESSION_PREREQS} + ${TEX} $*.tex + +xxpression.pdf: %.pdf: ${XXPRESSION_PREREQS} %.gdy + ${PDFTEX} $* && touch $*.gdy && touch $*.pdf +# if [ -f $*.gdx ] ; then ${BINDX} $*.gdx $*.gdy; fi + +xxpression.tok: xxpression.tex ${SPLINT_XPTABLES} ${SPLINT_XLTABLES} ${SPLINT_ROOT}/cweb/bo.tok + ${PDFTEX} $< && touch xxpression.tok + +test: test.tex xymbols.sns xxpression.tok ptab.tex ltab.tex + tex test.tex + +docs: xxpression.pdf + +clean: clean_core + -rm -f ptabout ltabout *.txx + +mostlyclean: clean_temp + -rm -f ptabout ltabout xxpression.c *.txx + +distclean: clean + cd ${SPLINT_ROOT} && ${MAKE} clean + +.INTERMEDIATE: xxpression.gdx diff --git a/support/splint/examples/xxpression/xtoks.sty b/support/splint/examples/xxpression/xtoks.sty new file mode 100644 index 0000000000..2dc5abafb4 --- /dev/null +++ b/support/splint/examples/xxpression/xtoks.sty @@ -0,0 +1,2 @@ +\prettywordpair{IDENTIFIER}{{$\langle$\rm identifier$\rangle$}} +\prettywordpair{INTEGER}{{\bf int}} diff --git a/support/splint/examples/xxpression/xxpression.sty b/support/splint/examples/xxpression/xxpression.sty new file mode 100644 index 0000000000..422901f6da --- /dev/null +++ b/support/splint/examples/xxpression/xxpression.sty @@ -0,0 +1,27 @@ +\def\optimization{5} + +\input yycommon.sty % general routines for stack and array access +\input yymisc.sty % helper macros (stack manipulation, table processing, value stack pointers) +\input yyinput.sty % input functions +\input yyparse.sty % parser machinery +\input flex.sty % lexer functions +\input yyfaststack.sty % sped up stack access functions +\input yyboth.sty % parser initializatio, optimization + +\def\yycomplain#1{\immediate\write16{#1}} % lexer errors + +\let\yylexreturn\yylexreturnregular +\let\setflexstates\relax +\let\parsernamespace\empty + +\genericparser + name: main, + ptables: ptab.tex, + ltables: ltab.tex, + tokens: xxpression.tok, + asetup: {}, + dsetup: {}, + rsetup: {}, + optimization: \optimizeall;% + +\tomainparser diff --git a/support/splint/examples/xxpression/xxpression.w b/support/splint/examples/xxpression/xxpression.w new file mode 100644 index 0000000000..00834b4436 --- /dev/null +++ b/support/splint/examples/xxpression/xxpression.w @@ -0,0 +1,365 @@ +@q Copyright 2012-2014, Alexander Shibakov@> +@q This file is part of SPLinT@> + +@q SPLinT is free software: you can redistribute it and/or modify@> +@q it under the terms of the GNU General Public License as published by@> +@q the Free Software Foundation, either version 3 of the License, or@> +@q (at your option) any later version.@> + +@q SPLinT is distributed in the hope that it will be useful,@> +@q but WITHOUT ANY WARRANTY; without even the implied warranty of@> +@q MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the@> +@q GNU General Public License for more details.@> + +@q You should have received a copy of the GNU General Public License@> +@q along with SPLinT. If not, see <http://www.gnu.org/licenses/>.@> + +% The scheme for extracting token equivalences below does not use a +% bootstrap parser, which would be easier. +% To use a different parser (the `prologue' parser, \.{dyytab.tex} in +% this case), some extra steps have to be inserted in +% \.{yybootstrap.sty}. First, the token equivalence table for the `main' +% parser (rather, for the `main' scanner) had to be loaded +% (\.{yybootstrap.sty} usually relies on the tokens that are +% `hard-coded' with the bootstrap parser). Second, it was necessary to +% define \.{\\let\\yylexreturn\\yylexregular} to use the scanner. One +% advantage of using a different parser is the ability to intermix token +% definitions with grammar productions (the bootstrap mode macros in +% \.{\\yyunion} will simply ignore the extra definitions). +% Note also, that the `grammar rule' parser cannot be used in this +% case since the token definitions as they are used in this file fit +% the `prologue' parser syntax only (there are no semicolons at the +% end of the definitions). A more elaborate scheme (similar to how the +% typesetting of rules is set up) using several parsers can be used +% instead. +\newwrite\gindex +\input limbo.sty +\input dcols.sty +\def\optimization{5} +\newread\testeof +\immediate\openin\testeof=\jobname.tok +\ifeof\testeof % make the local token equivalence table + \def\drvname{bo} + \csname newwrite\endcsname\tokendefs + \let\nx\noexpand + \edef\tokendeffile{\jobname.tok} % where to put the token equivalence table + \edef\bstrapparser{dyytab.tex} + \def\bootstraplexersetup{% + \let\yylexreturn\yylexreturnregular + \bootstrapmodetrue + \input \drvname.tok % use token equivalence table to set the values of non-string tokens + % this has to be added if a non-bootstrap parser is used to + % extract token information (see the comments above) + } + \toks0{% + \input yycommon.sty % general routines for stack and array access + \input yymisc.sty % helper macros (stack manipulation, table processing, value stack pointers) + \input yyinput.sty % input functions + \input yyparse.sty % parser machinery + \input flex.sty % lexer functions + \input yyboth.sty % parser initialization, optimization + \input yyfaststack.sty + \input yystype.sty % scanner auxiliary types and functions + \input yyunion.sty % parser data structures + % the main parser + \let\parsernamespace\empty + % create token equivalence table (making, say, \tokenID the same as \csname token"identifier"\endcsname) + \input yybootstrap.sty + } +\else + \toks0{% + \input yy.sty + \let\currentparsernamespace\parsernamespace + \let\parsernamespace\smallnamespace + \let\hostparsernamespace\smallnamespace + \input xtoks.sty + \let\parsernamespace\currentparsernamespace + } +\fi +\immediate\closein\testeof +\the\toks0 + +@**Parser file. \setupfootnotes +This an enhanced parser for expressions. It takes +advantage of the `symbolic term name' mechanism and extends the basic +expression syntax. + +The top-level structure of the input file is an exact copy of the one +for the expression parser. +@s TeX_ TeX + +@(xxpp.yy@>= +@G Switch to generic mode. +%{@> @<Extended \.{expression} parser \Cee\ preamble@> @=%} + @> @<Bison options@> @= +%union {@> @<Union of parser types@> @=} +%{@> @<Extended \.{expression} parser \Cee\ postamble@> @=%} + @> @<Token and types ...@> @= +%% + @> @<Parser productions@> @= +%% +@g + +@ The following is reproduced from the simple expression example. + +The \prodstyle{\%token-table} option is not merely a debugging help, +as it is in the case of the `real' \bison\ parsers and cannot be +omitted . The name table it is responsible for setting up is used as +a set of keys for various associative arrays. Token declarations are +parsed by a bootstrap parser during the \TeX\ processing stage to +establish equivalences between the names kept in |yytname| and the +macro names used internally by the parsers built by \bison. The reason +this is necessary is not very complicated: either version of the token +name can be used in the grammar while the `driver' program +(\.{mkeparser.c}) only has access to the names in |yytname|. In +general, this is important whenever the grammar uses a different set of +token names from the lexer or when diagnostics messages are output. An +important case is the symbolic name switch: before the rules can be +listed to create the switch, the token numerical values must be +known. If the parser is only aware of the |yytname| listed names and the +grammar being parsed uses the `internal' names, the listing macros +will fail. The array, |yytname| is used in a few functions inside the +`driver', as well, so omitting this option would make building the +parser impossible. +@<Bison options@>= +@G +%token-table +%debug +%start value +@g + +@ To continue the token name discussion, this parser uses internal +names only but the |yytname| array contains a string equivalent of +\prodstyle{IDENTIFIER}. Thus, bootstrapping is necessary. The beginning +of this file contains a simple scheme for producing a token +equivalence table. +The typesetting of the tokens can be adjusted using \.{\\prettywordpair} +macros (see the included \.{xtoks.sty} file for examples and the way +\prodstyle{IDENTIFIER} is typeset). +@<Token and types declarations@>= +@G +%token IDENTIFIER "identifier" +%token INTEGER +@g + +@ Here is the whole grammar, simply additive expressions with two +levels of precedence. We have added `divide' and `subtract' operations. +The use of \prodstyle{IDENTIFIER} instead of \.{"identifier"} below +necessitates `harvesting' of token equivalences in \.{xxpression.tok} +at the beginning of this file. +\showlastactiontrue +\input yynested.sty +@<Parser productions@>= +@G +value: + expression[exp] {@> TeX_( "/yy0{/the/yy]exp[}" ); @=} +; + +expression: + term {@> TeX_( "/yy0{/the/yy]term[}" ); @=} +| expression[exp] add_op term {@> @<Add a term@> @=} +; + +term: + atom {@> TeX_( "/yy0{/the/yy]atom[}" ); @=} +| term mult_op atom {@> @<Make a term@> @=} +; +@t}\vb{\inline\flatten}{@> +mult_op: + '*' {@> TeX_( "/yy0{/multiply}" ); @=} +| '/' {@> TeX_( "/yy0{/divide}" ); @=} +; + +add_op: + '+' {@> TeX_( "/yy0{}" ); @=} +| '-' {@> TeX_( "/yy0{-}" ); @=} +; +@t}\vb{\resetf}{@> +atom: +@t}\vb{\inputboundary{\boundarylower}}{@> + IDENTIFIER[id] {@> @<Assign variable value to an atom@> @=} +| INTEGER[int] {@> @<Assign value to an atom@> @=} +| '(' expression[exp] ')' {@> TeX_( "/yy0{/the/yy]exp[}" ); @=} +; +@t}\vb{\inputboundary{\boundaryupper}}{@> +@g + +@ @<Add a term@>= + @[TeX_( "/tempca/the/yy]exp[/relax" );@]@; + @[TeX_( "/tempcb/the/yy]term[/relax" );@]@; + @[TeX_( "/advance/tempca by /the/yy]add_op[/tempcb" );@]@; + @[TeX_( "/yy0{/the/tempca}" );@]@; + +@ @<Make a term@>= + @[TeX_( "/tempca/the/yy]term[/relax" );@]@; + @[TeX_( "/tempcb/the/yy]atom[/relax" );@]@; + @[TeX_( "/the/yy]mult_op[/tempca by /tempcb" );@]@; + @[TeX_( "/yy0{/the/tempca}" );@]@; + +@ @<Assign variable value to an atom@>= + @[TeX_( "/getsecond{/yy]id[}/to/toksa" );@]@; + @[TeX_( "/toksb/expandafter/expandafter/expandafter{/expandafter" );@]@; + @[TeX_( " /number/csname/the/toksa/endcsname}" );@]@; + @[TeX_( "/yy0{/the/toksb}" );@]@; + +@ @<Assign value to an atom@>= + @[TeX_( "/getfirst{/yy]int[}/to/toksa" );@]@; + @[TeX_( "/yy0{/the/toksa}" );@]@; + +@ \Cee\ preamble. In this case, there are no `real' actions that our +grammar performs, only \TeX\ output, so this section is empty. + +@<Extended \.{expression} parser \Cee\ preamble@>= + +@ \Cee\ postamble. It is tricky to insert function definitions that use \bison's internal types, +as they have to be inserted in a place that is aware of the internal definitions but before said +definitions are used. + +@<Extended \.{expression} parser \Cee\ postamble@>= +#define YYPRINT(file, type, value) @[yyprint (file, type, value)@] + static void yyprint (FILE *file, int type, YYSTYPE value){} + +@ Union of types. Empty as well. + +@<Union of parser types@>= + +@**The lexer file. The scanner for the grammar above is the same as +for a regular expression parser. Identifiers are interpreted as +variable names that expand to appropriate values. +%\checktabletrue +@(xxpl.ll@>= +@G + @> @<Lexer definitions@>@= +%{@> @<Lexer \Cee\ preamble@> @=%} + @> @<Lexer options@> @= +%% + @> @<Regular expressions@> @= +%% +@g + +@ @<Lexer definitions@>= +@G +letter [_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] +id {letter}({letter}|[-0-9])* +int [0-9]+ +@g + +@ @<Lexer \Cee\ preamble@>= + +#include <stdint.h> +#include <stdbool.h> + + void define_all_states( void ){} + +@ @<Lexer options@>= +@G +%option bison-bridge +%option noyywrap nounput noinput reentrant +%option noyy_top_state +%option debug +%option stack +%option outfile="xxpl.c" +@g + +@ @<Regular expressions@>= + @<Scan white space@>@; + @<Scan identifiers@>@; + +@ White space skipping. +\traceparserstatestrue +\tracestackstrue +\tracerulestrue +\traceactionstrue +\tracelookaheadtrue +\traceparseresultstrue +\tracebadcharstrue +\yyflexdebugtrue +% +\traceparserstatesfalse +\tracestacksfalse +\tracerulesfalse +\traceactionsfalse +\tracelookaheadfalse +\traceparseresultsfalse +\tracebadcharsfalse +\yyflexdebugfalse +@<Scan white space@>= +@G +[ \f\n\t\v] {@> @[TeX_( "/yylexnext" );@]@=} +@g + +@ @<Scan identifiers@>= +@G +{id} {@> @[TeX_( "/yylexreturnval{IDENTIFIER}" );@]@=} +{int} {@> @[TeX_( "/yylexreturnval{INTEGER}" );@]@=} +[-+*/()] {@> @[TeX_( "/yylexreturnchar" );@]@=} +. {@> @[@<React to a bad character@>@]@=} +@g + +@ @<React to a bad character@>= + @[TeX_( "/iftracebadchars" );@]@; + @[TeX_( " /yycomplain{invalid character(s): /the/yytext}" );@]@; + @[TeX_( "/fi" );@]@; + @[TeX_( "/yylexreturn{$undefined}" );@]@; + +@**Generating symbols. This is the routine that creates symbolic name +assignments for the grammar. The internal mechanics of creating such +assignments is inside \.{xymmap.sty} which should be consulted if +any adjustments are needed. +@(xymbols.txx@>= +@G +\def\optimization{5} % this can be omitted +\input yy.sty +\input xymmap.sty +\end +@g + +@**Test file. The test file includes a handy list of debugging options +that can be activated to see the inner workings of the parser and +scanner routines. +@(test.txx@>= +@G +\input xxpression.sty + +\iftrue + \tracedfatrue + \traceparserstatestrue + \tracestackstrue + \tracerulestrue + \traceactionstrue + \tracelookaheadtrue + \traceparseresultstrue + \tracebadcharstrue + \yyflexdebugtrue + \yyinputdebugtrue +\fi + +\newread\ssw +\immediate\openin\ssw = xymbols.sns +\ifeof\ssw +\else + \immediate\closein\ssw + \input xymbols.sns + \let\yysymswitch\symswitch + \let\yysymcleanup\symswitchoff +\fi + +\def\varone{10} +\def\expression{1 + 3 * ( 5 + 7 ) + varone - 10} + +\basicparserinit\expandafter\yyparse \expression \yyeof\yyeof\endparseinput\endparse + +{ + \newlinechar`^^J + \immediate\write16{^^Jexpression: \expression^^Jthe value: \the\yyval^^J^^J} +} + +\bye +@g + +@**Index. +\let\inx\inxmod +\let\fin\finmod +\immediate\closeout\exampletable +\def\Tex{\TeX\ output} +\def\TeXx{\TeX\ output} diff --git a/support/splint/examples/xxpression/xymmap.sty b/support/splint/examples/xxpression/xymmap.sty new file mode 100644 index 0000000000..3d36bae920 --- /dev/null +++ b/support/splint/examples/xxpression/xymmap.sty @@ -0,0 +1,155 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% assigning symbolic names to production terms (this is only a demonstration) + +\def\makesymbols{% + \restorecslist{symbols}\yyunion + \toksa{}\toksb{}\toksc{}\toksd{}% + \the\symstream + \global\newsymswitch\newsymswitch +} + +% symbolic reference name parser + +\newtoks\newsymswitch +\newread\symnames + +\newwrite\symbolicswitch + +\def\inamespace{[intermediate]}% +\let\parsernamespace\inamespace +\let\hostparsernamespace\mainnamespace % purely for testing reasons +\pinittoks{}% +\input ptab.tex % this is the enhanced expression parser +\edef\tointermediateparser{% + \noexpand\savefullstate % unnecessary + \noexpand\savefullstateextra % unnecessary + \let\noexpand\parsernamespace\noexpand\inamespace % switch to the new namespace + \the\pinittoks % restore all the tables, tokens and constants, and stacks + \let\noexpand\getcurrentparser\noexpand\tointermediateparser +}% +\settokens % this simply assigns values to tokens where the name of each token is taken out of yytname ... +\input xxpression.tok % this will set up token equivalences in the namespace above ... + % those are the values gleaned during the bootstrapping stage. + % in the general case, one needs to run a bootsrapping (or similar) + % parser to extract the token information. + % this is only needed if the bison file contains a definition that + % lists both the idenitifier and a string as names for a token but the grammar + % uses the identifier version at least once: + % (say, %token IDENTIFIER "identifier"; is used as well as + % ... + % token: + % IDENTIFIER { ... } + % ... + % see the included xxpression.w) +\optimizeall % this is necessary for correct rule listing in the output stage: + % otherwise \fgetelemof will use the current value of the \yy... token registers which + % will hold the values of the full parser that is loaded next +\newsymswitch{}% +\listrules % ... to be used while listing the rules +% note that we do not bother to set up a lexer for this parser (even though we already have one and the +% \optimizeall macro above will create a set of associative tables for it---this is merely an unwanted +% sideffect); after the rules have been listed, the intemediate parser in no longer needed. +%\showthe\newsymswitch +\def\fullnamespace{[full]}% this is the parser that parses the bison grammar from a raw + % bison file; it can play a role of the bootstrap parser for + % the grammar above, as well, since its input is a complete + % bison file; note that it cannot be a bootstrap parser for itself + % because it will reject any input that does not form a complete + % bison file; this is why a special grammar was created that includes + % only a small subset of the full set of productions (%token rules only) to + % serve as a bootstrap grammar. +%\tomainparser % this will set the value of \setflexstates, so that, if the lexer initialization + % below is omitted, the \setflexstates macro can still be used to correctly set the + % lexer states +\let\parsernamespace\fullnamespace +\pinittoks{}% %% +\input cweb/fyytab.tex +\input cweb/ltab.tex % we reinitialize the lexer out of necessity: + % \collecttokennames needs to know how to switch + % between two lexer/parser environments so both states + % have to be preserved; if one is sure that + % grammar_declarations will not used in the productions + % part of the file, this can simply be disabled +\settokens +\setflexstates % the main lexer can be reused in this case; the states still need to be set up +\input \drvname.tok % set up the tokens for the bison grammar parser +\newparserstate +\newlexerstate +\newlexerstateextra +% +\setnulstack{yyirulestack}% +% +\edef\tofullparser{% + \noexpand\savefullstate + \noexpand\savefullstateextra + \let\noexpand\parsernamespace\noexpand\fullnamespace % switch to the new namespace + \the\pinittoks % restore all the tables, tokens and constants, and stacks + \let\noexpand\getcurrentparser\noexpand\tofullparser +}% +\optimizeall +\toksa{\input xxpp.y}% start building the parsing command + % this has to be done carefully, since all the characters input + % from the parser file have to be `harmless', so their categories + % have to be reset; in order for the parser to be able to stop, + % appropriate command sequences would have to be inserted at the end +\toksb{\yyeof\yyeof\endparseinput\endparse + \let\yyinput\yyinputold + \undoascii + \ifyyparsefail + \errmessage{could not process symbols}% + \else + \symstream\table + \tointermediateparser % the parsing is finished, so we have two very long strings: + % o the rules of the hosted parser ([intermediate]) + % o the rules of the same parser just parsed + % these two strings are used to associate the symbolic names + % (which only exist in the second list) to the rule numbers and + % the relevant terms; + % switching the parse namespace above is done so that the term + % indices are looked up in the appropriate table + %\showthe\table + \makesymbols % list all the rules deriving explicit non-terminals + %\showthe\newsymswitch + \setexplicitinlinerules\newsymswitch % process implicit non-terminals resulting from inline actions + %\showthe\newsymswitch + \makesymrefs\newsymswitch % create the switch to be used by the parser (\yyparse) + \fi +}% +% build the command to create the symbolic name switch +\toksc{\tofullparser\parserinit + \let\yyinputold\yyinput + \let\yyinput\yyinputtrivial % a demo of a stripped down, slightly faster input routine + \doascii{11}\expandafter\yyparse}% +\edef\next{\the\toksc\the\toksa\relax\the\toksb}\next % \relax is to stop \TeX\ from trying to expand the file name further + % it is not flagged as a bad character because it is part of the epilogue +\immediate\openout\symbolicswitch=\jobname.sns +% +\edef\next{\setsncommands{\def\noexpand\symswitch\hashletter1{\harmlesscomment^^J\noexpand\ifcase\hashletter1\relax + \harmlesscomment^^J\the\setsncommands + \noexpand\else\harmlesscomment^^J% + \noexpand\fi\harmlesscomment^^J% + }\harmlesscomment^^J^^J}}\next +\edef\next{\unsetsncommands{\def\noexpand\symswitchoff\hashletter1{\harmlesscomment^^J\noexpand\ifcase\hashletter1\relax + \harmlesscomment^^J\the\unsetsncommands + \noexpand\else\harmlesscomment^^J% + \noexpand\fi\harmlesscomment^^J% + }\harmlesscomment^^J^^J}}\next +{\newlinechar=`\^^J \immediate\write\symbolicswitch{\the\setsncommands\the\unsetsncommands}}% +\immediate\closeout\symbolicswitch +\tomainparser % go back to the main parser + diff --git a/support/splint/makefile.inc b/support/splint/makefile.inc new file mode 100644 index 0000000000..d48478bb1a --- /dev/null +++ b/support/splint/makefile.inc @@ -0,0 +1,76 @@ +TEXINPUTS := .:${SPLINT_ROOT}/cweb/:${SPLINT_ROOT}/tex/:${SPLINT_ROOT}/:${TEXINPUTS} + +SPLINT_DRIVER_DIR = ${SPLINT_ROOT}/cweb + +SPLINT_TEXSTYLES = flex.sty yyfaststack.sty yyparse.sty yyunion.sty yyboth.sty yyinput.sty yystype.sty yy.sty +SPLINT_TEXSTYLES += yyinit.sty yybootstrap.sty dcols.sty limbo.sty yycommon.sty yymisc.sty yytexlex.sty +SPLINT_BOOTSTRAP_STYLES = grabstates.sty +SPLINT_PTABLES = byytab.tex dyytab.tex fyytab.tex gyytab.tex small_tab.tex +SPLINT_LTABLES = ltab.tex small_dfa.tex + +SPLINT_XTEXSTYLES = $(patsubst %, ${SPLINT_ROOT}/tex/%, ${SPLINT_TEXSTYLES}) +SPLINT_XBOOTSTRAP_STYLES = $(patsubst %, ${SPLINT_ROOT}/tex/%, ${SPLINT_BOOTSTRAP_STYLES}) +SPLINT_XPTABLES = $(patsubst %, ${SPLINT_ROOT}/cweb/%, ${SPLINT_PTABLES}) +SPLINT_XLTABLES = $(patsubst %, ${SPLINT_ROOT}/cweb/%, ${SPLINT_LTABLES}) + +SPLINT_DOC_PREREQS = %.tex ${SPLINT_PTABLES} ${SPLINT_LTABLES} ${SPLINT_XTEXSTYLES} ${SPLINT_ROOT}/tex/btokenset.sty bo.tok +SPLINT_DOC_PREREQS_XREF = ${SPLINT_DOC_PREREQS} %.scn %.idx + +PDFTEX = export TEXINPUTS=${TEXINPUTS} && pdftex +TEX = export TEXINPUTS=${TEXINPUTS} && tex +DVIPS = dvips +PDFVIEW = evince +CWEAVE = cweave -bhp +CTANGLE = ctangle -bhp +BISON = ${BISON_ROOT}bison -v +FLEX = ${FLEX_ROOT}flex + +MODEBOOTSTRAP = \\def\\modeactive\{\\modebootstrap\} + +BRACK = ${SPLINT_ROOT}/scripts/brack.pl +UNLINE = ${SPLINT_ROOT}/scripts/unline.pl +BINDX = ${SPLINT_ROOT}/scripts/bindx.pl + +%.c: %.w +%.c: %.y +%.pdf: %.tex +%.ps: %.dvi +%.o: %.c +%.c: %.l + +%.x: %.w + ${BRACK} $< $@ + +%.c: %.y + ${BISON} $^ -o $@ + +%.c: %.l + ${FLEX} -o $@ $^ + +%.l: %.ll + ${UNLINE} $^ $@ + +%.y: %.yy + cp -f $^ $@ + +%.ps: %.dvi + ${DVIPS} $^ -o $@ + +%.gdy: %.gdx + ${BINDX} $^ $@ + +all: docs + +${SPLINT_ROOT}/cweb/%: + cd ${SPLINT_ROOT}/cweb && ${MAKE} $(notdir $@) + +clean_temp: + -rm -f *.o *.tab.* *.dvi *.pdf *.ps *~ *.log \ + *.rli *.rls *.lpg *.output *.lst *.exl *.ftn *.*pk *.gdx + +clean_core: clean_temp + -rm -f *.c *.h *.tex *.idx *.scn *.toc *.tok *.sns *.gdy *.x *.yy *.ll *.y *.l + +# to ensure that interrupted and erroneous builds are cleaned up + +.DELETE_ON_ERROR: diff --git a/support/splint/makefile.loc b/support/splint/makefile.loc new file mode 100644 index 0000000000..58d7b0abaf --- /dev/null +++ b/support/splint/makefile.loc @@ -0,0 +1,5 @@ +# uncomment the lines below to use your local version of bison or flex +# BISON_ROOT ?= ${HOME}/bin/ +# FLEX_ROOT ?= ${HOME}/bin/ +# uncomment the line below if you have bison version 3.0 or newer +# BISON_STATE := -DBISON_IS_CRIPPLED=yes diff --git a/support/splint/scripts/bindx.pl b/support/splint/scripts/bindx.pl new file mode 100755 index 0000000000..ace62429ba --- /dev/null +++ b/support/splint/scripts/bindx.pl @@ -0,0 +1,101 @@ +#!/usr/bin/perl + +open FILE, "$ARGV[0]" or die "Cannot open input file $ARGV[0]\n"; +open FILE_OUT, ">$ARGV[1]" or die "Cannot open input file $ARGV[1]\n"; + +sub lex_order (\@\@) { # lexicographic ordering + + my (@string1) = @{shift @_}; + my (@string2) = @{shift @_}; + + my ($i); + + $i = 0; + + while ( $string1[$i] == $string2[$i] && $i <= $#string1 && $i <= $#string2 ) { $i++ } + + if ( $i > $#string1 || $i > $#string2 ) { + + return $#string1 <=> $#string2; + + } + + return ( $string1[$i] <=> $string2[$i] ); + +} + +sub numerically { $b <=> $a; } + +sub alphabetically { # lexicographic ordering based on the ASCII order defined in @main_order + + my (@chars1) = map { $main_order{$_} } split //, $a; + my (@chars2) = map { $main_order{$_} } split //, $b; + + return lex_order @chars1, @chars2; + +} + +sub lexicographically { # lexicographic ordering for numeric sequences separated by spaces + + my (@chars1) = split / /, $a; + my (@chars2) = split / /, $b; + + return lex_order @chars1, @chars2; +} + +$alphabet = "\#\$\%^&*<>[]{}()+-=_|\\,:;~`.?!\'\"\@0123456789AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz"; +@main_set = split //, $alphabet; +map { $main_order{$_} = index $alphabet, $_ } @main_set; # inefficient ... + +$ldelim[1] = "\\["; $rdelim[1] = "]"; +$ldelim[0] = "\\("; $rdelim[0] = ")"; +$ldelim[5] = "\\e"; $rdelim[5] = "e"; +$ldelim[4] = "\\f"; $rdelim[4] = "f"; +$ldelim[3] = "\\g"; $rdelim[3] = "g"; + +while (<FILE>) { + + $input = $_; + + if ( $input =~ /\\(.)TI\s*([0-9]+)\s*\{(\\[a-z]+)\s*\{(.+)\}\}\{(.+)\}\{(.+)\}\{(.+)\}\n/ ) { + + ($domain, $rank, $type, $key, $nspace, $pageno, $ref) = ($1, $2, $3, $4, $5, $6, $7); + $key =~ s/\{([0-9]+)\}/pack "c1", $1/eg; + + push @{$index{$domain}{$key}{refs}}, "$ref $rank"; + $index{$domain}{$key}{nspace} = $nspace; + $index{$domain}{$key}{type} = $type; + + } + +} + +$i = 0; + +foreach $domain (sort keys %index ) { + + if ( $i > 0) { + + print FILE_OUT "\\indexseparator{$domain}{$i}\n"; + + } + + $i++; + + foreach $key ( sort alphabetically keys %{$index{$domain}} ) { + + %ref_list = (); + map { exists $ref_list{$_} ? ($ref_list{$_}++) : ($ref_list{$_} = 0) } @{$index{$domain}{$key}{refs}}; + + @ref_list = sort lexicographically keys %ref_list; + @ref_list = map { @r = split / /, $_; "$ldelim[$r[1]]$r[0]$rdelim[$r[1]]" } @ref_list; + + $ukey = $key; + $ukey =~ s/(.)/"\{".(unpack "c1", $1)."\}"/eg; + + print FILE_OUT "\\GI{$index{$domain}{$key}{nspace}}{$index{$domain}{$key}{type}}{$ukey}, ", + (join ', ', @ref_list), ". \% $key, sec nos. ", (join ', ', @ref_list ), "\n"; + + } +} + diff --git a/support/splint/scripts/brack.pl b/support/splint/scripts/brack.pl new file mode 100755 index 0000000000..6e110e5cf3 --- /dev/null +++ b/support/splint/scripts/brack.pl @@ -0,0 +1,105 @@ +#!/usr/bin/perl + +# a simple script to replace all @G ... @`other' regions with +# @= ... @>@; lines +# comments are allowed at the end of the lines. +# only one style of comments is accepted: /* ... */. note that these are not +# output + +open FILE, "$ARGV[0]" or die "Cannot open input file $ARGV[0]\n"; +open FILEOUT, ">$ARGV[1]" or die "Cannot open output file $ARGV[1]\n"; + +$state = 0; +while (<FILE>) { + + $inline = $_; + + + if ( $inline =~ m/^\@G(.*)$/ ) { # @G detected, this line is part of the `other language' region + + $inline = $1; $state = 1; + if ( $inline =~ m/^\(([^)]*)\).*/ ) { + printf FILEOUT "\@q Start generic language section\@>\n\@t}\\begingsec{%s}{\@>\n", "$1"; # a parser switcher + } else { + $inline = " Start \@\@G (generic) language section"; + printf FILEOUT "\@q%s\@>\n\@t}\\begingsec{b}{\@>\n", "$inline"; # a parser switcher + } + + } elsif ( $inline =~ m/^\@[\scp].*$/ ) { # @`other' detected, so `other language' region is over + + if ($state == 1) { + printf FILEOUT "\@q%s\@>\n\@t}\\endgsec{\@>\n", + "End of generic language section"; # a parser switcher + } + $state = 0; + printf FILEOUT "%s", "$inline"; + + } elsif ( $inline =~ m/^\s*\@[=t].*$/ ) { # @= detected, just copy the line + + printf FILEOUT "%s", "$inline"; + + } elsif ( $inline =~ m/^\@g(.*)$/ ) { # @`other' detected, so `other language' region is over + + if ($state == 1) { + printf FILEOUT "\@q%s\@>\n\@t}\\endgsec{\@>\n", + "End of generic language section"; # a parser switcher + } + + $inline = $1; $state = 0; + + if ( not $inline ) { $inline = "End generic language section"; } + printf FILEOUT "\@q%s\@>\n", "$inline"; + + } elsif ( $inline =~ m/^\@O(.*)$/ ) { # @O detected, so `other language' region is paused + + $inline = $1; $state = 0; + + if ( not $inline ) { $inline = "End generic language section"; } + printf FILEOUT "\@q%s\@>\n", "$inline"; + + } elsif ( $inline =~ m/^\@o(.*)$/ ) { # @o detected, so `other language' region is resumed + + $inline = $1; $state = 1; + + if ( not $inline ) { $inline = "End generic language section"; } + printf FILEOUT "\@q%s\@>\n", "$inline"; + + } elsif ( $state == 1 ) { + + if ( $inline =~ m/\/\*.*\*\/\s*$/ ) { + + $inline =~ m/^(.*\S|)\s*(\/\*.*\*\/)\s*$/; + $string = $1; $comment = $2; + + } else { + + $inline =~ m/^(.*)$/; + $string = $1; $comment = ""; + + } + + if ( $string ) { + + printf FILEOUT "\@=%s\@>\@;", "$string"; + + } else { + + printf FILEOUT "\@=%s\@>\@;", " "; + + } + + if ( $comment ) { + + printf FILEOUT "%s", "$comment"; + + } + + printf FILEOUT "%s", "\n"; + + } else { + + printf FILEOUT "%s", "$inline"; + + } + +} diff --git a/support/splint/scripts/cslist.pl b/support/splint/scripts/cslist.pl new file mode 100755 index 0000000000..d1df62bc07 --- /dev/null +++ b/support/splint/scripts/cslist.pl @@ -0,0 +1,40 @@ +for ($i = 0; $i <= $#ARGV; $i++) { + + open FILE, "<$ARGV[$i]" or die "Cannot open file $ARGV[$i]!\n"; + $filename = $ARGV[$i]; + + while(<FILE>) { + + $text = $_; + + while( $text =~ /(\\newcount|\\newtoks|\\newdimen|\\newif|\\let|\\([exg]|char|toks|count)*def)\w*(\\[a-zA-Z@]+)/g ) { + + $cs = $3; + push @all, $cs; + + if ( defined $secse{$cs}->{$filename} ) { $secse{$cs}->{$filename}++; } + else { $secse{$cs}->{$filename} = 1; }; + + } + + } +} +#print keys %secse; +@alphsecs = sort keys %secse; + +foreach $ocs (@alphsecs) { + + print "$ocs %"; + @fnames = sort keys $secse{$ocs}; + foreach $fname (@fnames) { + print " $fname ($secse{$ocs}->{$fname} occurence"; + if ( $secse{$ocs}->{$fname} > 1 ) {print "s";} + print ")"; + } + print "\n"; +} + +print "\n"; +$i = $#alphsecs; +$i++; +print "total sequences: $i\n"; diff --git a/support/splint/scripts/unline.pl b/support/splint/scripts/unline.pl new file mode 100755 index 0000000000..5349156cb4 --- /dev/null +++ b/support/splint/scripts/unline.pl @@ -0,0 +1,31 @@ +#!/usr/bin/perl + +# a simple script to remove comments and #line directives left by CTANGLE +# this allows one to build Makefiles and linker scripts from inside CWEB +# by making a few simple changes to the macros (to facilitate typeseting) +# and using CWEB's @= ... @> facility. + +open FILE, "$ARGV[0]" or die "Cannot open input file $ARGV[0]\n"; +open FILEOUT, ">$ARGV[1]" or die "Cannot open output file $ARGV[1]\n"; + +while (<FILE>) { + + $inline = $_; + + s/^(\#line.*)\n$//g; + + if ( $ARGV[2] != 3 ) { + + s/\/\*.*\*\///g; + + } + + printf FILEOUT "%s", "$_"; + +# if ( m/^.*\S.*\n$/ ) { + +# printf FILEOUT "%s", "$_"; + +# } + +} diff --git a/support/splint/tex/btokenset.sty b/support/splint/tex/btokenset.sty new file mode 100644 index 0000000000..07affb717e --- /dev/null +++ b/support/splint/tex/btokenset.sty @@ -0,0 +1,31 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\prettywordpair{\%<flag>}{{$\langle\star\rangle$}} +\prettywordpair{\%nondeterministic-parser}{{$\langle$non...ic-parser$\rangle$}} +\prettywordpair{PERCENT_NONDETERMINISTIC_PARSER}{PER...NON...IC\_PARSER} +\prettywordpair{semi}{{\tt;}} +\prettywordpair{TAG}{{{\tt<}{\it tag\/}{\tt>}}} +\prettywordpair{<tag>}{{{\tt<}{\it tag\/}{\tt>}}} +\prettywordpair{FLEX_OPTION}{{$\langle$\bf option$\rangle_{\rm f}$}} +\prettywordpair{FLEX_STATE_X}{{$\langle$\bf state-x$\rangle_{\rm f}$}} +\prettywordpair{FLEX_STATE_S}{{$\langle$\bf state-s$\rangle_{\rm f}$}} +\prettywordpair{ID}{{\cyr\lqq{\rm identifier}\rqq}} +\prettywordpair{ID_COLON}{{\cyr\lqq{\rm identifier: }\rqq}} +\prettywordpair{NUL}{{$\Lambda$}} +\prettywordpair{INT}{{\bf int}} +\prettywordpair{CHAR}{{\bf char}} +\prettywordpair{STRING}{{\cyr\lqq{\tt string}\rqq}} diff --git a/support/splint/tex/dcols.sty b/support/splint/tex/dcols.sty new file mode 100644 index 0000000000..2560495d45 --- /dev/null +++ b/support/splint/tex/dcols.sty @@ -0,0 +1,817 @@ +% Copyright 2004-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% multicolumn output: +% the general layout of the page is similar to the following, where +% the number of columns and the location and the number of `footnote' +% columns can be adjusted. + +%%%%%%%%%%% %%%%%%%%%%% %%%%%%%%%%% +%%column1%% %%column2%% %%column3%% +%%%%%%%%%%% ----------- %%%%%%%%%%% +%%%%%%%%%%% %footnotes% %%%%%%%%%%% + +\newskip\realfootins +\newskip\footglue +\newskip\adjskip % skip to improve \vboxes +\newcount\linecount +\newcount\noofcolumns +\newcount\fnotesstart +\newcount\fnotesspan +\newdimen\columnwidth +\newdimen\icgap + +\newdimen\pagewidth +\newdimen\pageheight +\newdimen\longvsize % the height of all the columns combined +\newdimen\fnoteboxwidth +\newdimen\footvboxht % holds the previous value of the height of the + % box of footnotes + +\newbox\partialpage +\newbox\currentcolumn +\newbox\foothbox % holds the footnotes `in one line' +\newbox\foothhbox % holds every footnote in a separate \hbox +\newbox\migrantbox +{\setbox\migrantbox=\box\migrantbox} + +\newtoks\fromword +\newtoks\toword + +\newcount\remaindergaps % the number of gaps between columns after the + % first `footnote column' +\newcount\remaindernonfoots % the number of columns without footnotes + % after the columns with footnotes + +\newif\ifoutputnow +\newif\iflastpageup + +\maxdeadcycles=600 % this should be approximately half the \pageheight in points + % to handle the case when \enddoublecolumns is called before the + % last doublecolumn page is full + +% setup the page length and the column count + +\linecount=53 % number of lines in a page +\fnotesstart=1 % start footnotes under the first column +\fnotesspan=2 % footnotes span two columns +\noofcolumns=2 + +\realfootins=12pt plus 1pt minus 1pt +\footglue=1em plus.2em minus.1em + +\adjskip=0pt plus 6pt +\topskip=9pt + +% + +\def\setmcparams{% + \columnwidth=\hsize + \advance\columnwidth-\noofcolumns\icgap + \advance\columnwidth\icgap + \divide\columnwidth by\noofcolumns +% \advance\linecount\m@ne +% \vsize=\baselineskip +% \multiply\vsize by \linecount +% \advance\vsize by \topskip +% \advance\linecount\@ne + % + \longvsize=\vsize + \advance\longvsize by-\topskip + \advance\longvsize by\baselineskip + \divide\longvsize\baselineskip + \multiply\longvsize\baselineskip + \count\@cclv\noofcolumns + \multiply\longvsize by \count\@cclv + \advance\longvsize-\baselineskip + \advance\longvsize\topskip +} + +\setmcparams +\skip\footins=\realfootins +\count\footins=\@m % 1000 + +\pagewidth=\hsize +\pageheight=\vsize + +% + +\def\footnote#1{% + \edef\@sf{\spacefactor\the\spacefactor}\setfnmark + \@sf + \insert\footins{\floatingpenalty\@MM % 20000 + \setbox\z@=\hbox{\fnfsize\strut\fnmark\localfn\nobreak + \enskip{#1}\penalty-10\hskip\footglue}% + \setfootnoteheight + \box\z@ + }% +} + +\def\setfnoteparams{% + \fnfsize +% \emergencystretch=10pt + \hbadness\@M % 10000 + \hyphenpenalty\z@ \doublehyphendemerits\z@ + \exhyphenpenalty\z@ \finalhyphendemerits\z@ + \righthyphenmin\z@ \lefthyphenmin=\tw@ + \leftskip\z@\rightskip\z@ + \adjdemerits\z@ \uchyph=\@ne \pretolerance=-1 +} + +\let\fnfsize\eightpoint + +\def\setfootnoteheight{ + \global\setbox\foothbox=\hbox{\unhbox\foothbox\unhcopy\z@}% + \setbox\@ne=\vbox{ + \hsize=\fnoteboxwidth + \setfnoteparams + \noindent\unhcopy\foothbox + \par + }% + \dimen\z@=\ht\@ne \advance\dimen\z@ by -\footvboxht + \global\footvboxht=\ht\@ne + \ht\z@=\dimen\z@ \dp\z@=\z@ + \global\setbox\foothhbox=\hbox{\copy\z@\unhbox\foothhbox}} + +\def\makefootnoteparagraph{ + \count\@cclv=\z@ + \unvbox\footins + \makehboxofhboxes + \cleanupfootbuffer\readjustfoots + \setbox\z@=\hbox{\unhbox\z@ \removehboxes} + \setfnoteparams + \noindent\unhbox\z@\par +} + +\def\makehboxofhboxes{% + \setbox\z@=\hbox{}% + \loop + \setbox\tw@=\lastbox + \ifhbox\tw@ + \advance\count\@cclv\@ne + \setbox\z@=\hbox{\box\tw@\unhbox\z@}% + \repeat +} + +% the macro below admittedly stretches \TeX's stack capacity but it is rather efficient +% at removing the \hbox packaging in the correct order. +\def\removehboxes{\setbox\z@=\lastbox + \ifhbox\z@{\removehboxes}\unhbox\z@ \fi} + +\def\cleanupfootbuffer{% remove the footnotes that made it to the page + % form the buffer + \global\setbox\foothhbox=\hbox{\unhbox\foothhbox\cleanupf@otbuffer}} + +\def\cleanupf@otbuffer{% + \ifnum\count\@cclv>\z@ + \setbox\z@=\lastbox + \advance\count\@cclv\m@ne + \let\next=\cleanupf@otbuffer + \else + \let\next=\relax + \fi + \next} + +\def\readjustfoots{% + \setbox\@ne=\copy\foothhbox % keep the footnotes that did not make it + % in the buffer + \setbox3=\vbox{}% + \global\setbox\foothbox=\hbox{}% + \setbox\@ne=\hbox{\unhbox\@ne\readjustf@ots}% +} + +\def\readjustf@ots{% + \setbox\tw@=\lastbox + \ifvoid\tw@ + \let\next=\readjustf@@ts + \else + \global\setbox\foothbox=\hbox{\unhbox\foothbox\unhcopy\tw@}% + \setbox3=\vbox{\box2\unvbox3}% + \let\next=\readjustf@ots + \fi + \next +} + +\def\readjustf@@ts{% + \setbox\tw@=\vbox{ + \hsize=\fnoteboxwidth + \setfnoteparams + \noindent\unhcopy\foothbox\par + } + \global\footvboxht=\ht\tw@ + \dimen\z@=\ht\tw@ \advance\dimen\tw@ by -\ht3 + \ifnum\dimen\z@=\z@ + \else + \global\setbox\footins=\vbox{}% + \advance\dimen\z@ by \m@ne sp + \ht\footins=\dimen\z@ \dp\footins=\@ne sp % to indicate that this is an + % adjustment box for inner (i.e. the ones inserted by \footblock rathar than + % as part of the main output routine) footnotes (looked at by the output routine); + % if the height of the already contributed footnotes needs to be + % adjusted, adjust the height of the \box\footins by that amount + \fi +} + +\fnoteboxwidth=\pagewidth % footnotes will spread across the whole page + +\newskip\dsskip + +\dsskip\smallskipamount + +\def\begindoublecols{\begingroup\r@ggedbottomfalse\parindent=0pt + \count\footins=\@m \multiply\count\footins by\fnotesspan + \skip\footins=\fnotesspan\realfootins + \remaindergaps=\noofcolumns + \advance\remaindergaps by -\fnotesstart + \remaindernonfoots=\remaindergaps + \advance\remaindernonfoots by -\fnotesspan + \advance\remaindernonfoots\@ne + \ifnum\noofcolumns>\@ne % typesetting in one column is an exception, + % otherwise ... + % first, calculate the intercolumn skip ... + \dimen\z@=\pagewidth \dimen\@ne=\columnwidth + \multiply\dimen\@ne by\noofcolumns \advance\dimen\z@ by-\dimen\@ne + \count\@cclv=\noofcolumns \advance\count\@cclv\m@ne + \divide\dimen\z@ by\count\@cclv + \count\@cclv=\fnotesspan + \advance\count\@cclv\m@ne + \dimen\@ne=\columnwidth + \multiply\dimen\@ne by\fnotesspan + \advance\dimen\@ne by \count\@cclv\dimen\z@ + % ... to get the width of the footnote box + \fnoteboxwidth=\dimen\@ne + \fi + % this final modification is needed due to the fact that the whole + % page is put in \hbox to\hsize{...; without the modification the + % width of the footnote box is not going to match the width of the + % page, and when the footnotes span all the columns there will be no + % `\hfill' to compensate for arithmetic imprecision + \ifnum\fnotesspan=\noofcolumns + \fnoteboxwidth=\pagewidth + \fi + % + \output={ + \ifvoid\partialpage + \else % this is a border case: too much material has been contributed, this routine will be called twice + % so output the full page now + \normaloutput{\unvbox\partialpage}\lheader\rheader + \fi + \global\setbox\partialpage=\vbox{\unvbox\@cclv\vskip\dsskip}% + \dimen\z@=\longvsize + \advance\dimen\z@ by -\noofcolumns\ht\partialpage + \ifnum\dimen\z@<\z@ % the amount of material contributed to the current page is more than the + % required \vsize (so that \pageshrink would have to be involved); + % output the page normally + \normaloutput{\unvbox\partialpage}\lheader\rheader + \else % move the footnotes into the buffer, recalculate their size + \migratefootnotes + \fi + }% + \eject + \def\footnoterule{\kern -3\p@ \hrule width \fnoteboxwidth \kern 2.6\p@}% + \outputnowfalse % not ready for immediate output, play with dimensions first + \lastpageupfalse % not the final page yet + \holdinginserts=\@ne + \let\thepage\pagesofar + \output={\multcolumnout} + \hsize=\columnwidth + \global\vsize=\pageheight% + \global\advance\vsize by-\ht\partialpage% + \global\divide\vsize\baselineskip + \global\multiply\vsize\baselineskip + \global\multiply\vsize\noofcolumns% +%\message{starting with vsize at \the\vsize, with pp at \the\ht\partialpage} + \global\advance\vsize by-\noofcolumns\topskip + \global\advance\vsize by\topskip + \global\advance\vsize by\noofcolumns\baselineskip + \global\advance\vsize by-\baselineskip +%\message{vsize set at \the\vsize} + \ifvoid\migrantbox + \else + \insert\footins{\box\migrantbox}% + \fi +} + +\newif\ifpenultimatepage +\newif\ifp@nultimatepage + +\def\enddoublecols{\par % fire the page buidling routine + \ifnum\pagetotal>\pagegoal + \global\penultimatepagetrue + \eject % if the multicolumn output ended just barely after accumulating a full page + \fi % but before the page is shipped out, output the page with the standard multicolumn output + \dimen7 =\pagegoal + \vsize=\longvsize + % now recalculate all the dimensions assuming that the footnotes stretch across the whole page + \ifnum\footvboxht>\z@ + \advance\vsize by \fnotesspan\footvboxht + \advance\dimen7 by \fnotesspan\footvboxht + \advance\vsize by -\noofcolumns\realfootins + \advance\vsize by \fnotesspan\realfootins + \advance\dimen7 by -\noofcolumns\realfootins + \advance\dimen7 by \fnotesspan\realfootins + \fi + \fnotesspan=\noofcolumns + \fnotesstart=\@ne + \remaindergaps=\noofcolumns + \advance\remaindergaps by -\fnotesstart + \remaindernonfoots=\z@ + \fnoteboxwidth=\pagewidth + \setbox\z@=\hbox{}\setfootnoteheight + % calculate and adjust the height of the footnotes; this introduces an extra \hbox to \foothhbox + % remember this when in the output routine for the last page! + \advance\vsize by -\noofcolumns\footvboxht + \advance\dimen7 by -\noofcolumns\footvboxht + \pagegoal=\dimen7 + % + \lastpageuptrue + \ifp@nultimatepage % everything has been output already + \global\penultimatepagefalse + \global\p@nultimatepagefalse + \else + \global\penultimatepagefalse + \eject + \fi + \endgroup + \holdinginserts=\z@ + \global\pagegoal=\pageheight + \global\vsize=\pageheight} + +\def\migratefootnotes{% + \ifvoid\footins + \else + \ifdim\dp\footins=\z@ % it is not a correction + \setbox\@ne=\vbox{\unvbox\footins + % similar to \makehboxofhboxes + \setbox\z@=\hbox{}% + \loop + \setbox\tw@=\lastbox + \ifhbox\tw@ + \setbox\z@=\hbox{\unhbox\tw@\unhbox\z@}% + \repeat + % ... + \setbox\foothbox=\hbox{}% + \setbox\foothhbox=\hbox{}% + \footvboxht=\z@ + \setfootnoteheight\global\setbox\migrantbox=\box\z@ + } + \else + {\global\setbox\footins=\box\footins}% + \fi + \fi +} + +\def\multcolumnout{% + \splittopskip=\topskip + \splitmaxdepth=\maxdepth + \dimen\z@=\pageheight + \advance\dimen\z@ by-\ht\partialpage + \advance\dimen\z@ by-\dp\partialpage + \ifnum\dimen\z@<\baselineskip % another extreme case: at most one line will fit, so + % the adjustments to \vsize will not change the page layout + % quickly enough resulting in an infinite \deadcycles loop + \normaloutput{\unvbox\partialpage}\lheader\rheader + \unvbox\@cclv \penalty\lastpenalty + \fi + \divide\dimen\z@\baselineskip \multiply\dimen\z@\baselineskip +%\message{aiming for d0: \the\dimen0} + \dimen\@ne=\pagegoal % we claculate the maximum allowable height of the + % middle column next + \advance\dimen\@ne by-\vsize + \advance\dimen\@ne by \fnotesspan\dimen\z@ + \divide\dimen\@ne by\fnotesspan +%\message{d1: \the\dimen1, d0: \the\dimen0, pageheight: \the\pageheight }% +%\message{(vsize: \the\vsize, pagegoal: \the\pagegoal\space ppage: \the\ht\partialpage)^^J^^J}% + % + \balanceallcolumns + % + \ifoutputnow + \global\fromword=\expandafter{\firstmark} + \global\toword=\expandafter{\botmark} + \iflastpageup + \ifnum\outputpenalty<-9999 % this break was forced (most likely by the \break in \enddouble...) + \ifvoid\footins + \else + \ifdim\dp\footins=\z@ \else + {\setbox\footins=\box\footins}% this was a correcion box after which no new real footnotes + \fi % were cotributed + \fi + % a correction (\hbox{}) was added by the \enddouble... command; + % this correction will affect \box\footins (since it will appear + % as a box in \foothhbox, a box that is not a footnote); + % this correction will be translated into an appropriate + % adjustment to \box\footins (since the output routine thinks + % it is a footnote); however, it does throw off the footnote count in \foothhbox + % off by one; kill it here; the footnotes will be inserted from the footins box + \setbox\z@=\vbox{\pagesofar}% + \unvbox\z@ \penalty\lastpenalty + % since we could not touch \foothhbox above (to keep the footnote height calculations correct), + % the \footins box is messed up; fix it + {\setbox\footins=\box\footins}% + \global\setbox\foothhbox=\hbox{}% to keep the footnote counting + \global\setbox\foothbox=\hbox{}% consistent + \global\footvboxht=\z@ + \else + \normaloutput\page\lheader\rheader + % + \global\vsize=\longvsize + \global\outputnowfalse + \global\holdinginserts=\@ne + \fi + \else % this is not the last page, preform normal multicolumn output + \normaloutput\page\lheader\rheader + % + \global\vsize=\longvsize + \global\outputnowfalse + \global\holdinginserts=\@ne + \ifpenultimatepage % page broken at a penalty inserted by \enddoublecolumns + \global\p@nultimatepagetrue % the next penalty should be removed + \fi + \fi + {\setbox\@cclv=\box\@cclv}% use up \box255 ! + \else % do not output yet, see if adjustments worked + \ifnum\dimen\tw@>\dimen\@ne + \global\advance\vsize by -\fnotesspan pt + \else + \global\outputnowtrue + \global\holdinginserts=\z@ + \fi + \unvbox\@cclv + \ifnum\outputpenalty=\@M % 10000 the break was not at a penalty item + \else + \ifp@nultimatepage % we already split the page at this penalty + \global\p@nultimatepagefalse % let \enddoublecolumns insert the penalty + \else + \penalty\outputpenalty + \fi + \fi + \fi +} + +% we will change some of the plain TeX macros used by cwebmac + +\def\pagebody{% + \vbox to \pageheight{ + \boxmaxdepth=\maxdepth \pagecontents + \the\footline + \ifr@ggedbottom + \vskip\adjskip + \fi + } +} + +\def\normaloutput#1#2#3{\ifodd\pageno\hoffset=\pageshift\fi + \shipout\vbox{ + \vbox to\fullpageheight{ + \iftitle\global\titlefalse + \else\hbox to\pagewidth{\vbox to10pt{}\ifodd\pageno #3\else#2\fi}\fi + \vfill#1}} % parameter #1 is the page itself + \global\advance\pageno by1} + +\let\page\pagebody % this is a cwebmac command but it has to be executed here, after the \pagebody macro is changed + +\def\pagecontents{ + \ifvoid\topins + \else + \unvbox\topins + \fi + \setbox\z@=\vbox{\thepage}\dimen\z@=\dp\z@ % for \ruggedbottom + \unvbox\z@ + \ifr@ggedbottom + \kern-\dimen\z@\vfil + \fi + \ifvoid\footins + \else + \ifnum\dp\footins=\z@ % this is not an adjustment + \vskip\realfootins\footnoterule + \vbox{\hsize=\pagewidth\makefootnoteparagraph}% + \fi + \fi +} + +\def\thepage{\unvbox\@cclv} + +\def\balanceallcolumns{ + \tcthr=\vbadness + \vbadness=1000000 % set \vbadness to the badness of a overfull box + \count\@cclv\z@ % (does not really prevent TeX from reporting one) + \setbox\z@=\vbox{\unvcopy\@cclv}% + \setbox\tw@=\vbox{}%%% this box is changed globally + \b@lanceallcolumns +} + +\def\b@lanceallcolumns{ + \advance\count\@cclv\@ne + \whichcolumn{\count\@cclv}% + \ifinfoots % if the footnotes go under these columns ... + \dimen\tw@=\ht\z@ +%\message{set d2(=ht0): \the\dimen2, d0: \the\dimen0; vs. ph: \the\pageheight}% + \advance\dimen\tw@ by -\remaindernonfoots\dimen\z@ + \advance\dimen\tw@ by -\remaindergaps\baselineskip + \advance\dimen\tw@ by \remaindergaps\topskip + \divide\dimen\tw@ by \fnotesspan +%\message{set d2: \the\dimen2}% + \let\next=\b@l@nceallcolumns + \else % otherwise, split off the columns ... + \ifnum\dimen\z@<\topskip + \setbox\@ne=\vsplit\z@ to \topskip + \else + \setbox\@ne=\vsplit\z@ to \dimen\z@ + % see a remark below on why this was necessary + \fi + \wd\@ne=\hsize + \setbox\tw@=\vbox{\box\@ne \unvbox\tw@}%%% this box is changed globally + \let\next=\b@lanceallcolumns + \fi + \next +} + +\newcount\tcthr + +\def\b@l@nceallcolumns{% splitting off the columns after the footnotes start + \count\@cclv=\fnotesstart + \setbox3=\copy\z@ + \setbox5=\vbox{}% + \b@l@nc@allcolumns +} + +\def\b@l@nc@allcolumns{ + \whichcolumn{\count\@cclv}% + \iflastcolumn +%\message{box3 height: \the\ht3, box3 depth: \the\dp3, d0: \the\dimen0, d2: \the\dimen2}% + \ifnum\ht3>\dimen\ifinfoots \tw@ \else \z@ \fi + \advance\dimen\tw@ by \@ne pt + \let\next=\b@l@nceallcolumns + \else + \ifinfoots + \else + \ifnum\dimen\z@<\topskip + \setbox3=\vsplit3 to \topskip + \else + \setbox3=\vsplit3 to \dimen\z@ + \fi + \fi +%\message{**box3 height: \the\ht3, d0: \the\dimen0, bs: \the\baselineskip, ts: \the\topskip}% + \wd3=\hsize + \setbox\tw@=\vbox{\box3 \unvbox5 \unvbox\tw@}% + %%% \box2 is laterchanged globally + \vbadness=\tcthr + \let\next=\relax + \fi + \else + \ifnum\dimen\ifinfoots \tw@ \else \z@ \fi<\topskip + \setbox4=\vsplit3 to \topskip + \else + \setbox4=\vsplit3 to \dimen\ifinfoots \tw@ \else \z@ \fi + \fi + % the code above is a replacement for: + % \setbox4=\vsplit3 to \dimen\ifinfoots \tw@ \else \z@ \fi + % \ifnum\badness>\@MM \errmessage{\the\badness:\the\vbadness}\fi + % the replacement was necessary to avoid `Overfull \vbox ... ' warnings + % that (unlike those for underfull boxes) cannot be silenced by raising + % \vbadness + \wd4=\hsize + \setbox5=\vbox{\box4 \unvbox5}% + \advance\count\@cclv\@ne + \let\next=\b@l@nc@allcolumns + \fi + \next +} + +\def\pagesofar{% + \unvbox\partialpage + \count\@cclv\z@ + \hbox to\pagewidth{\p@gesofar}% + \kern-\prevdepth + \vskip\adjskip +} + +\def\p@gesofar{\advance\count\@cclv\@ne + \whichcolumn{\count\@cclv}% + \ifinfoots + \let\next=\footblock + \else + \iflastcolumn + \getnextcolumn\box\currentcolumn + \let\next=\relax + \else + \getnextcolumn\box\currentcolumn\hfil + \let\next=\p@gesofar + \fi + \fi + \next +} + +\def\footblock{% + \hbox to \fnoteboxwidth{\f@otblock}% + \count\@cclv=\fnotesstart + \advance\count\@cclv by\fnotesspan + \advance\count\@cclv\m@ne + \whichcolumn{\count\@cclv}% + \iflastcolumn + \let\next=\relax + \else + \hfil + \let\next=\p@gesofar + \fi + \next +} + +\def\f@otblock{% + \whichcolumn{\count\@cclv}% + \ifinfoots + \iffirstfootcol + \setbox\z@=\rlap{\vbox{ + \ifvoid\footins + \else + \ifdim\dp\footins=\z@ + % footnote info is present + \vskip\realfootins\footnoterule + \vbox{\hsize=\fnoteboxwidth\makefootnoteparagraph} + \fi + \fi + }}% + \dimen5=\ht\z@ + \dimen6=\dimen5 + \advance\dimen6 by\dimen\tw@ + \setbox\z@=\vbox{ + \getnextcolumn\unvbox\currentcolumn\kern-\dimen3 + \vskip\adjskip + \box\z@ + }% + \ifnum\ht\z@>\baselineskip + \setbox\z@=\vbox to \dimen\iflastpageup 6 \else \z@ \fi{\unvbox\z@}% + \fi + \wd\z@=\hsize + \box\z@ + \else + \hfil\setbox\z@=\vbox{ + \getnextcolumn\unvbox\currentcolumn\kern-\dimen3 + \vskip\adjskip + \hrule width\z@ height \dimen5 depth\z@ + }% + \ifnum\ht\z@>\baselineskip + \setbox\z@=\vbox to \dimen\iflastpageup 6 \else \z@ \fi{\unvbox\z@}% + \fi + \wd\z@=\hsize + \box\z@ + \fi + \iflastcolumn + \let\next=\relax + \else + \let\next=\f@otblock + \fi + \else + \let\next=\relax + \fi + \advance\count\@cclv\@ne + \next +} + +\def\getnextcolumn{% + \global\setbox\tw@=\vbox{\unvcopy\tw@ \global\setbox\currentcolumn=\lastbox}% + \dimen3=\dp\currentcolumn +} + +\newif\ifinfoots +\newif\iflastcolumn +\newif\ifbeforefoots +\newif\iffirstfootcol + +\def\whichcolumn#1{% + \infootsfalse + \lastcolumnfalse + \beforefootsfalse + \firstfootcolfalse + \ifnum#1<\fnotesstart + \beforefootstrue + \else + \ifnum#1=\fnotesstart + \firstfootcoltrue + \infootstrue + \else + \advance\fnotesstart by\fnotesspan + \ifnum#1<\fnotesstart + \infootstrue + \fi + \advance\fnotesstart by-\fnotesspan + \fi + \ifnum#1=\noofcolumns + \lastcolumntrue + \fi + \fi +} + +% footnote counting macros + +\let\footthenotes\empty +\footline{\footthenotes}% so the footnotes know where to reset + +\def\fnmark#1{{% + \ifnum#1<\@M + \setbox\z@=\hbox{\rm)}\vbox to \ht\z@{\hbox{$\scriptstyle\the#1$}\vss}\box\z@ + \else + \setbox\z@=\hbox{\rm)}\setbox2=\hbox{$\scriptstyle0$}% + \setbox\@ne=\hbox{\vrule width\wd2 height\ht2}% + \vbox to \ht\z@{\box\@ne\vss}\box\z@ + \fi}} + +\def\addfnlabel{\expandafter\fnstream\expandafter{\the\fnstream*}} +\def\addpgendlabel{\expandafter\fnstream\expandafter{\the\fnstream|}} + +% the commands above will be exectuted when the .ftn file is loaded below + +\newtoks\fnstream % locations of footnotes +\fnstream={} + +% before processing the footnotes one by one we have to open an +% auxiliary file (we have to read it in whole because it will be +% rewritten as the document is being typeset); this opening has to be +% done carefully for the file might not be there yet; check for it first. + +\newread\fnlabels +\newwrite\fnlabels +\newcount\localfn % footnote count +\newcount\fniteration + +\def\setupfootnotes{% this macro should be used at the beginning of the file, after a box (preferably) + \def\footthenotes{\write\fnlabels{\noexpand\addpgendlabel}}% + \openin\fnlabels=\jobname.ftn\relax + \ifeof\fnlabels + \closein\fnlabels\relax + \fniteration\z@ + \else + \closein\fnlabels\relax + \input \jobname.ftn\relax + \fi + \edef\savedfnstream{\the\fnstream}% + % now we open the same file for writing ... + \openout\fnlabels=\jobname.ftn\relax + \advance\fniteration\@ne + \write\fnlabels{\noexpand\fniteration\the\fniteration\space}% +} + +\def\unsetfootnotes{% a clean(er) way to terminate footnote accounting +% \let\footthenotes\footthenoteslast + \def\footthenotes{\closeout\fnlabels\let\footthenotes\empty}% +} + +% the control sequence below is not used, although the idea is clear; +% unfortunately, this does not work, since \closeout\fnlabels is delayed +% so \input \jobname.ftn is likely to import an empty file; +% a more elaborate strategy is possible whereby all the macros inside +% \jobname.ftn are made expandable and the test below (along with \input \jobname.ftn) +% is put inside \write\m@ne{ ... } (so that the expansion of \input \jobname.ftn is +% delayed) using string comparison macros defined elsewhere +% but at the moment it seems excessive + +\def\footthenoteslast{% unused, see above + \closeout\fnlabels + \fnstream{}% + \input \jobname.ftn + \edef\next{\the\fnstream}% + \ifx\next\savedfnstream + \else + \message{Footnotes may be set incorrectly ...}% + \fi + \let\footthenotes\relax +} + +\def\setfnmark{{% + \ifnum\localfn<\@M + \getfntok + \if*\nexttok + \global\advance\localfn\@ne + \else + \global\localfn=\@M + \fi + \fi + \removewhitespace\kern2pt\fnmark\localfn + \write\fnlabels{\noexpand\addfnlabel}% +}} + +\def\getfntok{\loop\splitfnstream\if|\nexttok\global\localfn=\z@ \repeat} + +\def\splitfnstream{\edef\tempdefone{\the\fnstream}% + \ifx\tempdefone\empty + \def\nexttok{?}\message{You need to recount the footnotes ...}% + \else + \expandafter\splitfnstre@m\tempdefone\end + \fi +} + +\def\splitfnstre@m#1#2\end{\def\nexttok{#1}\global\fnstream={#2}} + diff --git a/support/splint/tex/flex.sty b/support/splint/tex/flex.sty new file mode 100644 index 0000000000..da3aa7b0b7 --- /dev/null +++ b/support/splint/tex/flex.sty @@ -0,0 +1,547 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\newtoks\tayybyte % throwaway token register +\newcount\tayymarker % throwaway count register + +\chardef\EOL=`\^^J% + +\def\yylex{% + \ifnum\yyg@yyinit=\z@ % if ( !yyg->yy_init ) { + \yyg@yyinit=\@ne % yyg->yy_init = 1; + \ifnum\yyg@yystart=\z@ % if ( ! yyg->yy_start ) { + \yyg@yystart=\@ne % yyg->yy_start = 1; + \fi % } + \fi % } + \yylwhile +} + +\def\yylwhile{% + % yy_cp = yyg->yy_c_buf_p; + % *yy_cp = yyg->yy_hold_char; + % yy_bp = yy_cp; + % + \yycurrentstate=\yyg@yystart % yy_current_state = yyg->yy_start; + \advance\yycurrentstate\YYATBOL % yy_current_state += YY_AT_BOL(); + \yymatch +} + +\newif\ifyytextbackup +\newif\iftraceflexbuffers + +\def\yymatch{% + \ifyytextbackup + \xskiptofi\yyreinput + \else + \xskiptofi\yyinput + \fi +} + +\def\yyreinput{\expandafter\yyresetstreams\expandafter\yyinput\the\yytextseen} + +\def\yyresetstreams{% + \iftraceflexbuffers\ferrmessage{will rescan: \the\yytextseen}\fi + \yytextseen{}% + \yytextseenpure\yytextseen + \yyformatseen\yytextseen + \yystashseen\yytextseen + \yytextbackupfalse +} + +\def\yym@tch{% do { + \yyc=\fgetelemof{yyec}\at\yycp@\relax % yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)]; + \yyact=\fgetelemof{yyaccept}\at\yycurrentstate\relax + % ... ... reuse \yyact ( yy_act = yy_accept[yy_current_state] ) + \ifnum\yyact=\z@ % if ( yy_accept[yy_current_state] ) { + \else % + \yyg@yylastacceptingstate=\yycurrentstate % yyg->yy_last_accepting_state = yy_current_state; + \concat\yytext\yytextseen % yyg->yy_last_accepting_cpos = yy_cp; + \concat\yytextpure\yytextseenpure % ... ... + \concat\yyformat\yyformatseen % ... ... + \concat\yystash\yystashseen % ... ... + \yytextlastchar=\yytextseenlastchar % + \yyfmark=\yyfmarklast % ... ... + \yysmark=\yysmarklast % ... ... + \yytextseen={}% + \yytextseenpure\yytextseen + \yyformatseen\yytextseen + \yystashseen\yytextseen + \fi % } + \yyllwhile +} + +\def\yyllwhile{% ... yyllwhile: + \yyact=\fgetelemof{yybase}\at\yycurrentstate\relax % ... ... reusing \yyact ( yy_act = yy_base[yy_current_state] ) + \advance\yyact\yyc % ... ... ( yy_act = yy_act + yy_c ) + \yyact=\fgetelemof{yychk}\at\yyact\relax % ... ... ( yy_act = yy_chk[yy_act] ( == yy_chk[yy_base[yy_current_state] + yy_c] ) ) + \ifnum\yyact=\yycurrentstate % while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { + \yybreak\yymatchtail % ... if ( yy_chk[yy_base[yy_current_state] + yy_c] == yy_current_state ) + \else % ... goto yy_match_tail; + % ... else + \yycurrentstate=\fgetelemof{yydef}\at\yycurrentstate\relax + % yy_current_state = (int) yy_def[yy_current_state]; + \ifnum\yycurrentstate>\YYMAXREALCHAR\relax % if ( yy_current_state >= sizeof( yy_accept ) ) { + \yyc=\fgetelemof{yymeta}\at\yyc\relax % yy_c = yy_meta[(unsigned int) yy_c]; + \fi % } + \yybreak\yyllwhile % ... goto yyllwhile; + \yycontinue % } +} + +\def\yymatchtail{% + \yycurrentstate=\fgetelemof{yybase}\at\yycurrentstate\relax + \advance\yycurrentstate\yyc % + \yycurrentstate=\fgetelemof{yynxt}\at\yycurrentstate\relax + % yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; + \concat\yytextseen\yybyte % ++yy_cp; + \concat\yytextseenpure\yybytepure % ... + \concat\yyformatseen\yyfbyte\yyfbyte{}% % ... + \concat\yystashseen\yysbyte\yysbyte{}% % ... + \yytextseenlastchar=\yycp@ % + \yyfmarklast=\formatmarker % ... + \yysmarklast=\stashmarker % ... + \yyact=\fgetelemof{yybase}\at\yycurrentstate\relax % ... reusing \yyact ( yy_act = yy_base[yy_current_state] ) + \ifnum\yyact=\YYBASEMAXENTRY\relax % } while ( yy_base[yy_current_state] != max( yy_base ) ); + \xskiptofi\yyfindaction % ... if ( yy_base[yy_current_state] == max( yy_base ) ) goto yy_find_action; + \else % ... else goto yy_match; + \xskiptofi\yymatch + \fi +} + +\newif\ifyyflexdebug +\newif\iftracebadchars + +\def\yyfindaction{% + \yyact=\fgetelemof{yyaccept}\at\yycurrentstate\relax % yy_act = yy_accept[yy_current_state]; + \ifnum\yyact=\z@ % if ( yy_act == 0 ) { + \yytextbackuptrue % yy_cp = yyg->yy_last_accepting_cpos; + \yycurrentstate=\yyg@yylastacceptingstate % yy_current_state = yyg->yy_last_accepting_state; + \yyact=\fgetelemof{yyaccept}\at\yycurrentstate\relax% yy_act = yy_accept[yy_current_state]; + \ifyyflexdebug\ferrmessage{backup: \the\yytext:\the\yytextseen}\fi + \else + % + \concat\yytext\yytextseen + \concat\yytextpure\yytextseenpure + \concat\yyformat\yyformatseen + \concat\yystash\yystashseen + \yytextlastchar=\yytextseenlastchar + \yyfmark=\yyfmarklast + \yysmark=\yysmarklast + \yytextseen{}% + \yytextseenpure{}% + \yyformatseen{}% + \yystashseen{}% + % + \fi % } +% \YYDOBEFOREACTION % YY_DO_BEFORE_ACTION; + \ifyyflexdebug + \ferrmessage{action: \the\yyact, state: \the\yycurrentstate\space(\the\yytext)}% + \fi + \doaction +} + +\def\YYRULESETUP{% %#define YY_RULE_SETUP \ + \yytoksempty{\yytext}{}{% % if ( yyleng > 0 ) \ + \ifnum\EOL=\yytextlastchar % YY_CURRENT_BUFFER_LVALUE->yy_at_bol = (yytext[yyleng - 1] == '\n'); \ + \YYATBOL=\@ne % YY_USER_ACTION + \else %#define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yy_at_bol) + \YYATBOL=\z@ + \fi + }% +} + +%#define BEGIN yyg->yy_start = 1 + 2 * +% should be: +%#define BEGIN(state) yyg->yy_start = 1 + 2 * ( state ) + +\newif\iftracestates + +\def\yylexstate#1{\csname flexstate\parsernamespace #1\endcsname} + +\def\yyBEGIN#1{% + \yyg@yystart + \expandafter\ifx\csname flexstate\parsernamespace #1\endcsname\relax + #1 \iftracestates\ferrmessage{now in state: #1}\fi + \else + \csname flexstate\parsernamespace #1\endcsname\relax + \iftracestates\ferrmessage{now in state: #1 (\number\csname flexstate\parsernamespace #1\endcsname)}\fi + \fi + \multiply\yyg@yystart\tw@ + \advance\yyg@yystart\@ne +} + +\def\yyBEGINr#1{% + \yyg@yystart#1% + \iftracestates\ferrmessage{new state: \the\yyg@yystart}\fi + \multiply\yyg@yystart\tw@ + \advance\yyg@yystart\@ne +} + +%#define YY_START ((yyg->yy_start - 1) / 2) + +\def\YYSTART{% + \expandafter\xdivbytwo\expandafter{\number + \expandafter\xdecrement\expandafter{\number\yyg@yystart}} +} + +\def\yypushstate#1{% + \expandafter\yypush\expandafter{\number\YYSTART}\on\yystatestack + \yyBEGIN{#1}% +} + +\def\yypopstate{% + \yypop\yystatestack\into{\expandafter\yyBEGIN\romannumeral0}% + \iftracestates\message{new state (* 2 + 1 scrambled): \the\yyg@yystart}\fi +} + +\def\yytopstate#1{% + \yyreadstack\yystatestack\at\z@\to#1% +} + +\let\yylexcontinue\yylwhile % complete the while loop (requires \yylexnext) +\def\yylexcontinue{\yytext{}\yytextpure\yytext\yylwhile} % complete the while loop + +% \yylextail is the return point from the lexer + +\let\yyreturn\yym@tch % the return point from the low-level input + +\def\yylexreturnbootstrap#1{% + \yytext{}\yytextpure\yytext + \expandafter\ifx\csname token\parsernamespace #1\endcsname\relax + % this token value is undefined + \let\yylextail\yylex % so lex the next token + \else + \yychar\csname token\parsernamespace #1\endcsname\relax + \let\yylextail\yyparsetail + \fi +} + +\def\yylexreturnregular#1{\yychar\csname token\parsernamespace #1\endcsname\relax\yytext{}\yytextpure{}\let\yylextail\yyparsetail} + +\def\yylexreturnval#1{% return value (yytext) only + \yychar\csname token\parsernamespace #1\endcsname\relax + \yylval\expandafter{\expandafter{\the\yytext}}% + \yylval\expandafter\expandafter\expandafter{\expandafter\the\expandafter\yylval\expandafter{\the\yytextpure}}% + % the above is an equivalent of + % \edef\next{\yylval{{\the\yytext}{\the\yytextpure}}}\next + % but does not `pollute' the definition of \next + \yytext{}\yytextpure\yytext\let\yylextail\yyparsetail +} + +\def\yylexreturnptr#1{% return stream pointers only + \yylval\expandafter{\expandafter{\the\yyfmark}}% + \yylval\expandafter\expandafter\expandafter{\expandafter\the\expandafter\yylval\expandafter{\the\yysmark}}% + % the above is an equivalent of + % \edef\next{\yylval{{\the\yyfmark}{\the\yysmark}}}\next + % but does not `pollute' the definition of \next + \yylexreturn{#1}% +} + +\def\yylexreturntext{\yylexreturnptr{\the\yytextpure}} + +\def\yylexreturnraw#1{% return the character value + \yychar`#1\relax + \yytext{}\yytextpure{}\let\yylextail\yyparsetail +} + +\def\yylexreturnchar{% + \expandafter\yylexreturnraw\the\yytextpure +} + +\def\yylexnext{\yytext{}\yytextpure{}} % use this with a trivial \yylexcontinue +\let\yylexnext\empty + +\def\yyterminate{\yychar\YYEOF\yylval{}\yytext{}\yytextpure{}\let\yylextail\yyparsetail} + +\def\yyerrterminate{\yylexreturn{$undefined}} %$ + +\def\ferrmessage#1{\immediate\write16{#1}} + +% \yyless is slow and the macros below have not been tested in their current +% form: avoid them + +\def\yyless#1{% + \ifnum#1=\z@ + \ROLLBACKCURRENTTOKEN + \else + \let\oldyyreturn\yyreturn + \edef\yyreturn{\noexpand\yyskipnchars{\number#1}}% + \expandafter\yyinput\the\yytext\yyeof + \fi +} + +\def\yyskipnchars#1{% + \ifnum\yycp@=\YYENDOFBUFFERCHAR % read \yyeof + \yycomplain{yyless buffer overflow: #1 characters too many}% + \else + \ifnum#1=\@ne % skipped the required number of tokens + \yybreak@\yyl@ss + \else + \edef\yyreturn{\noexpand\yyskipnchars{\xdecrement{#1}}}% + \yybreak@\yyinput + \fi + \yycontinue +} + +\def\yyl@ss#1\yyeof{% + \yytext{#1}% + \concatl\yytext\yytextseen + \yytext{}\yytextpure{}% + \yytextbackuptrue + \let\yyreturn\oldyyreturn +} + +%#define ROLLBACK_CURRENT_TOKEN \ +% do { \ +% scanner_cursor.column -= mbsnwidth (yytext, yyleng, 0); \ +% yyless (0); \ +% } while (0) + +\def\ROLLBACKCURRENTTOKEN{% + \concatl\yytext\yytextseen + \yytext{}\yytextpure{}% + \yytextbackuptrue +} + +% \yytext---current yytext +% \yytextseen---last accepted yytext +% |--\yytext--|--\yytextseen--|--remaining buffer--| +% after accepting state: +% [---------\yytext-----------] +% after backing up: +% |--\yytext--|--\yytextseen-----remaining buffer--| +% \yytextseen={} + +%#define YY_DO_BEFORE_ACTION \ +% yyg->yytext_ptr = yy_bp; \ +% yyleng = (size_t) (yy_cp - yy_bp); \ +% yyg->yy_hold_char = *yy_cp; \ +% *yy_cp = '\0'; \ +% yyg->yy_c_buf_p = yy_cp; + +\def\doaction{% + \yydoactionswitch\yyact +} + +%#define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) + +% as the macros above and below make a lot of assignments, +% expandability was not a +% priority, however, keeping the macro namespace clean was; +% expandability is, of course a guaranteed way to do so, grouping is +% another; + +\def\yylexeofaction{% + \yygetnextbuffer + \ifYYEOBLASTMATCH + \yygetpreviousstate + \ifyyflexdebug\ferrmessage{found previous state: \the\yycurrentstatelocal\space matched text: \the\yytext>>\the\yytextseen(\the\yybyte))}\fi + \yycurrentstate\yycurrentstatelocal + \let\yylextail\yyfind@ction + \YYEOBLASTMATCHfalse + \else + \yyact\YYENDOFBUFFER\relax + \advance\yyact\YYSTART + \advance\yyact\@ne % yy_act = YY_STATE_EOF(YY_START); + \let\yylextail\doaction % goto do_action; + \appendl\yytextseen{\the\yytext}\yytextbackuptrue % in case the scanner gets called again + \ifyyflexdebug + \yytoksempty{\yytext}% + {\errmessage{internal error: matched nothing in eob state}}% + {\yyisthiscsr\yytext\yyeof{}{\errmessage{internal error: matched more that a single eob}}}% + \ferrmessage{eob in user state: \the\yyact\space matched text: \the\yytext(\the\yytextpure)\space + to read: \the\yytextseen}% + \fi + \yytext{}% + \fi +} + +\def\yygetnextbuffer{% + \yytoksempty{\yytext}{\errmessage{internal error: empty matched text in eob state}}% + {% + \expandafter\expandafter\expandafter + \yystringempty % we have matched exactly one token + \expandafter\expandafter\expandafter{\expandafter\eatone\the\yytext}% + {\YYEOBLASTMATCHfalse}{\YYEOBLASTMATCHtrue}% + }% +} + +\def\yyeofcontainer{\yyeof} + +\newif\ifYYEOBLASTMATCH + +\def\yygetpreviousstate{% + \yycurrentstatelocal=\yyg@yystart + \advance\yycurrentstatelocal\YYATBOL + % + \yytoksempty{\yytext}{\errmessage{internal error: empty matched text in eob state}}{}% + \expandafter\yysplitoff\expandafter.\the\yytext\yyeof\end + \appendl\yytextseen{\noexpand\yyeof}\yytextbackuptrue + \ifyyflexdebug\ferrmessage{searching for last match in text: \the\yytext}\fi % TODO: delete + \yytoksempty{\yytext}{}{% yyg->yytext_ptr >= yyg->yy_c_buf_p /* do not enter the for loop */ + \yytextpure{}% + \yyfutureyytext\yytextseen % to be reinserted in yyfind@action + \yytextseen{}\yytextseenpure{}% + \expandafter\yytext\expandafter + {\expandafter}\expandafter + \yygetpreviousstatefor\the\yytext\splitoffend % at least one iteration of the for loop + }% +} + +\def\yysplitoff#1\yyeof#2\end{% + \yystringempty{#2}{% + \errmessage{internal error: something other than \noexpand\yyeof\space triggered eob action}% + }{% + \yytext\expandafter{\eatone#1}% to remove the `.' at the beginning + }% +} + +\def\yygetpreviousstatefor{% + % save all the lexer variables affected by \yyinput + % to be restored on exit from the `for' loop + % i.e.~when the \splitoff token is read + \let\currentyyreturn\yyreturn +% + \let\currentyyfbyte\yyfbyte + \let\currentyysbyte\yysbyte + \let\currentstashmarker\stashmarker + \let\currentformatmarker\formatmarker +% + \let\yyfbyte\tayybyte + \let\yysbyte\tayybyte + \let\stashmarker\tayymarker + \let\formatmarker\tayymarker +% + \let\yyreturn\yygetpreviousstatef@r + \yyinput +} + +\def\yygetpreviousstatef@r{% + \let\default\yygetpreviousstatedefault + \ifyyflexdebug\ferrmessage{just read: \the\yybyte}\fi % TODO: delete + \switchon{\expandafter\getfirsttoken\expandafter{\the\yybyte}}\in\yygetpreviousstateswitch +} + +\def\yygetpreviousstateswitch{% + \yyeof {% scanned a NUL + \yyclocal\YYECMAGIC\relax % the constant extracted from yy_get_next_state + \yygetpreviousstat@f@r + } + \splitoffend {% yyg->yytext_ptr >= yyg->yy_c_buf_p /* exit the for loop */ + % this is not an actual character but a marker for the end of the `for' loop + \let\yyfbyte\currentyyfbyte + \let\yysbyte\currentyysbyte + \let\stashmarker\currentstashmarker + \let\formatmarker\currentformatmarker + \yycp@=\YYENDOFBUFFERCHAR\relax % this is not necessary + \yybyte{}\yybytepure{}% +% + \let\yyreturn\currentyyreturn + } + \endparse \endparseinput {% + \errmessage{internal error.: reading past the end of the input buffer}% + } +} + +\def\yygetpreviousstatedefault{% + \yyclocal=\fgetelemof{yyec}\at\yycp@\relax % yy_c = yy_ec[YY_SC_TO_UI(*yy_cp)]; + \yytextseenlastchar=\yycp@ + \yygetpreviousstat@f@r +} + +\def\yygetpreviousstat@f@r{% + \ifnum\fgetelemof{yyaccept}\at\yycurrentstatelocal=\z@ + % if ( yy_accept[yy_current_state] ) { + \concat\yytextseen\yybyte + \concat\yytextseenpure\yybytepure + \yybyte{}\yybytepure\yybyte + \else % + \ifyyflexdebug\ferrmessage{possible accepting state: \the\yycurrentstatelocal:% + \fgetelemof{yyaccept}\at\yycurrentstatelocal}\fi % TODO: delete + \yyg@yylastacceptingstate=\yycurrentstatelocal + % yyg->yy_last_accepting_state = yy_current_state; + \concat\yytext\yytextseen % yyg->yy_last_accepting_cpos = yy_cp; TODO:????????? + \concat\yytextpure\yytextseenpure % ... ... + \yytextseen=\yybyte + \yytextseenpure=\yybytepure + \yytextbackuptrue + \yybyte{}\yybytepure\yybyte + \yytextlastchar=\yytextseenlastchar % + \fi + \yygpswhile +} + + +\def\yygpswhile{% ... yyllwhile: + \yyact=\fgetelemof{yybase}\at\yycurrentstatelocal\relax% ... ... reusing \yyact ( yy_act = yy_base[yy_current_state] ) + \advance\yyact\yyclocal % ... ... ( yy_act = yy_act + yy_c ) + \ifnum\fgetelemof{yychk}\at\yyact=\yycurrentstatelocal % while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { + \yybreak\yygpsfornext % ... if ( yy_chk[yy_base[yy_current_state] + yy_c] == yy_current_state ) + % ... ... exit the while loop + \else % ... return yy_current_state; + % ... else + \yycurrentstatelocal=\fgetelemof{yydef}\at\yycurrentstatelocal\relax + % yy_current_state = (int) yy_def[yy_current_state]; + \ifnum\yycurrentstatelocal>\YYMAXREALCHAR\relax % if ( yy_current_state >= sizeof( yy_accept ) ) { + \yyclocal=\fgetelemof{yymeta}\at\yyclocal\relax% yy_c = yy_meta[(unsigned int) yy_c]; + \fi % } + \yybreak\yygpswhile % ... goto yyllwhile; + \yycontinue % } +} + +\def\yygpsfornext{% + \yycurrentstatelocal=\fgetelemof{yybase}\at\yycurrentstatelocal\relax + \advance\yycurrentstatelocal\yyclocal + \yycurrentstatelocal=\fgetelemof{yynxt}\at\yycurrentstatelocal\relax + \ifyyflexdebug\ferrmessage{switching to state: \the\yycurrentstatelocal}\fi % TODO: delete + \yyinput +} + +\def\yyfind@ction{% we have to use this macro instead of the + % regular \yyfindaction because we use a stop token (\splitoffend) to terminate + % the `for' loop and are thus `one token behind' by the time this action is taken + % therefore we have to forego the `yy_cp actions' in the original \yyfindaction + \yyact=\fgetelemof{yyaccept}\at\yycurrentstate\relax % yy_act = yy_accept[yy_current_state]; + \ifnum\yyact=\z@ % if ( yy_act == 0 ) { + % yy_cp = yyg->yy_last_accepting_cpos; + \yycurrentstate=\yyg@yylastacceptingstate % yy_current_state = yyg->yy_last_accepting_state; + \yyact=\fgetelemof{yyaccept}\at\yycurrentstate\relax% yy_act = yy_accept[yy_current_state]; + \ifyyflexdebug\ferrmessage{backup.: \the\yytext:\the\yytextseen}\fi + \else + % + \concat\yytext\yytextseen + \concat\yytext\yybyte + \yybyte{}% + \yytextseen{}% + \concat\yytextpure\yytextseenpure + \concat\yyformat\yyformatseen + \concat\yystash\yystashseen + \yytextlastchar=\yytextseenlastchar + \yyfmark=\yyfmarklast + \yysmark=\yysmarklast + \yyformatseen{}% + \yystashseen{}% + % + \fi % } + \concat\yytextseen\yyfutureyytext % ... reinsert the tokens seen by the lexer before + % ... \yygetpreviousaction got rolling + \yyfutureyytext{}% + \yytextbackuptrue +% \YYDOBEFOREACTION % YY_DO_BEFORE_ACTION; + \ifyyflexdebug + \ferrmessage{action.: \the\yyact, state: \the\yycurrentstate\space(\the\yytext) \parsernamespace}% + \fi + \doaction +} diff --git a/support/splint/tex/frontmatter.sty b/support/splint/tex/frontmatter.sty new file mode 100644 index 0000000000..b0071a4bef --- /dev/null +++ b/support/splint/tex/frontmatter.sty @@ -0,0 +1,2 @@ +% this file is left blank on purpose to avoid referencing non-free fonts +% it would contain the \TeX\ macros that generate the cover page for \splint\ manual diff --git a/support/splint/tex/grabstates.sty b/support/splint/tex/grabstates.sty new file mode 100644 index 0000000000..6fa805168a --- /dev/null +++ b/support/splint/tex/grabstates.sty @@ -0,0 +1,69 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\def\optimization{5} + +\input yycommon.sty % general routines for stack and array access +\input yymisc.sty % helper macros (stack manipulation, table processing, value stack pointers) +\input yyinput.sty % input functions +\input yyparse.sty % parser machinery +\input flex.sty % lexer functions +\input yyfaststack.sty % sped up stack access functions +\input yyboth.sty % parser initialization, optimization +\input yyunion.sty + +\genericparser + name: main, + ptables: cweb/byytab.tex, + ltables: cweb/lstab.tex, + tokens: {}, + asetup: \let\setflexstates\relax, + dsetup: {}, + rsetup: {}, + optimization: \optimizeall;% + +\let\parsernamespace\mainnamespace +\let\yylexreturn\yylexreturnregular + +% main section macro + +\let\nx\noexpand + +\expandafter\def\csname parserstack[b]\endcsname#1#2{% + \basicparserinit\yyparse#1\yyeof\yyeof\endparseinput\endparse + \ifyyparsefail % do nothing if parsing failed + \yybreak{}% + \else % Stage three, process the parsed table + \yybreak{% + \restorecslist{bootstrap}\yyunion + \the\table\relax + }% + \yycontinue +} + +\expandafter\def\csname parserstack[]\endcsname#1#2{% + \basicparserinit\yyparse#1\yyeof\yyeof\endparseinput\endparse + \ifyyparsefail % do nothing if parsing failed + \yybreak{}% + \else % Stage three, process the parsed table + \yybreak{% + \restorecslist{bootstrap}\yyunion + \the\table\relax + }% + \yycontinue +} + +\newwrite\stlist diff --git a/support/splint/tex/limbo.sty b/support/splint/tex/limbo.sty new file mode 100644 index 0000000000..1ab76629a4 --- /dev/null +++ b/support/splint/tex/limbo.sty @@ -0,0 +1,1372 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% limbo.sty version 2.3 + +\input epsf.tex +\input amssym.def + +\catcode`\@=11 % a name space protection measure (weak) + +% fonts +\font\tenboldi=cmbxti10 +\font\nineboldi=cmbxti9 +\font\eightboldi=cmbxti8 +\font\sevenboldi=cmbxti7 +\font\sixboldi=cmbxti6 +\font\fiveboldi=cmbxti5 + +\font\twelvebi=cmmib10 at 12pt +\font\tenbi=cmmib10 +\font\tencyr=lhr10 +\font\ninebi=cmmib9 +\font\sevenbi=cmmib7 +\font\sixbi=cmmib6 +\font\twelvebsy=cmbsy10 at 12pt +\font\ninebsy=cmbsy9 +\font\sevenbsy=cmbsy7 + +\font\twelvei=cmmi10 at 12pt +\font\ninei=cmmi9 +\font\twelvesy=cmsy10 at 12pt +\font\ninesy=cmsy9 + +\font\twelvetitle=cmssbx12 +\font\tentitle=cmssbx10 +\font\tentitlei=cmssbxo10 +\font\ninetitle=cmssbx9 +\font\eighttitle=cmssbx8 +\font\seventitle=cmssbx7 +\font\titlefrak=eufb10 at 12pt + +\font\tenss=cmss10 +\font\niness=cmss9 +\font\eightss=cmss8 + +\font\twelvetitleit=cmssbxo10 at 12pt +\font\ninetitleit=cmssbxo10 at 9pt +\font\seventitleit=cmssbxo10 at 7pt + +\font\authorfont=cmssqi8 +\font\affiliationfont=cmssq8 +\font\timefont=cmss7 +\font\keywordsfn=cmssbx8 + +\font\tentex=cmtex10 +\font\ninetex=cmtex9 +\font\eighttex=cmtex8 + +\font\inchhigh=cminch +\font\titlefont=cmssdc10 at 40pt + +\font\ninerm=cmr9 +\font\eightrm=cmr8 +\font\sevenrm=cmr7 +\font\ninecyr=lhr9 +\font\eightcyr=lhr8 +\font\sevencyr=lhr7 +\font\sixrm=cmr6 + +\font\ninei=cmmi9 +\font\eighti=cmmi8 +\font\sixi=cmmi6 +\skewchar\ninei='177 \skewchar\eighti='177 \skewchar\sixi='177 + +\font\ninesy=cmsy9 +\font\eightsy=cmsy8 +\font\sixsy=cmsy6 +\skewchar\ninesy='60 \skewchar\eightsy='60 \skewchar\sixsy='60 + +\font\eightss=cmssq8 + +\font\eightssi=cmssqi8 + +\font\twelvebf=cmbx12 +\font\ninebf=cmbx9 +\font\eightbf=cmbx8 +\font\sixbf=cmbx6 + +\font\ninett=cmtt9 +\font\eighttt=cmtt8 +\font\seventt=cmtt7 + +\hyphenchar\tentt=-1 % inhibit hyphenation in typewriter type +\hyphenchar\ninett=-1 +\hyphenchar\eighttt=-1 + +\font\ninesl=cmsl9 +\font\eightsl=cmsl8 +\font\sevensl=cmsl7 + +\font\nineit=cmti9 +\font\eightit=cmti8 +\font\sevenit=cmti7 +\font\sixit=cmti6 +\font\fiveit=cmti5 + +\font\twelvemsam=msam10 at 12pt +\font\tenmsam=msam10 +\font\ninemsam=msam9 +\font\eightmsam=msam8 +\font\sevenmsam=msam7 +\font\sixmsam=msam6 +\font\fivemsam=msam5 + +\font\twelvemsbm=msbm10 at 12pt +\font\tenmsbm=msbm10 +\font\ninemsbm=msbm9 +\font\eightmsbm=msbm8 +\font\sevenmsbm=msbm7 +\font\sixmsbm=msbm6 +\font\fivemsbm=msbm5 + +\font\tenfrak=eufm10 +\font\ninefrak=eufm9 +\font\eightfrak=eufm8 +\font\sevenfrak=eufm7 + +\font\hv=phvr + +\newfam\itbfam +\newfam\frakfam +\newfam\msbmfam +\newfam\msamfam + +% take `<=' and `>=' from Euler ... +% but this will be changed back in the title due to +% the absence of the bold version + +\mathchardef\leq="3\hexnumber@\msamfam36 \let\le=\leq +\mathchardef\geq="3\hexnumber@\msamfam3E \let\ge=\geq +\mathchardef\emptyset="0\hexnumber@\msbfam3F + +% left and right `french quotes' (guillemots); the characters reside in lhr fonts + +\chardef\lqq='375 +\chardef\rqq='376 + +\def\cdotfill{\cleaders\hbox {$\m@th \mkern .5mu\cdot\mkern .5mu$}\hfill} +\def\dinkus{$\ast\ast\ast$} +\def\asterism{\setbox0\hbox{$\ast$}\copy0 \kern-.3\wd0 \raise\ht0 \copy0 \kern-.3\wd0 \copy0 } + +\newskip\ttglue +\def\tenpoint{\def\rm{\fam0\tenrm}% + \textfont0=\tenrm \scriptfont0=\sevenrm \scriptscriptfont0=\fiverm + \textfont1=\teni \scriptfont1=\seveni \scriptscriptfont1=\fivei + \textfont2=\tensy \scriptfont2=\sevensy \scriptscriptfont2=\fivesy + \textfont3=\tenex \scriptfont3=\tenex \scriptscriptfont3=\tenex + \def\it{\fam\itfam\tenit}% + \textfont\itfam=\tenit\scriptfont\itfam=\sevenit + \scriptscriptfont\itfam=\fiveit + \def\itbold{\fam\itbfam\tenboldi}% + \textfont\itbfam=\tenboldi\scriptfont\itfam=\sevenboldi + \scriptscriptfont\itbfam=\fiveboldi + \def\sl{\fam\slfam\tensl}% + \textfont\slfam=\tensl + \def\bf{\fam\bffam\tenbf}% + \textfont\bffam=\tenbf \scriptfont\bffam=\sevenbf + \scriptscriptfont\bffam=\fivebf + \def\tt{\fam\ttfam\tentt + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \def\ttex{\tentex + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \def\ttl{\let\it\tentitlei\let\sl\tentitlei\tentitle}% + \def\ssf{\tenss}% + \textfont\ttfam=\tentt + \tt \ttglue=.5em plus.25em minus.15em + \def\Bbb{\fam\msbmfam\tenmsbm}% + \textfont\msbmfam=\tenmsbm\scriptfont\msbmfam=\sevenmsbm \scriptscriptfont\msbmfam=\fivemsbm + \textfont\msamfam=\tenmsam\scriptfont\msamfam=\sevenmsam + \scriptscriptfont\msamfam=\fivemsam + \textfont\frakfam=\tenfrak + \def\frak##1{{\fam\frakfam\tenfrak ##1}}% + \normalbaselineskip=12pt + \let\sc=\eightrm + \let\mc=\ninerm + \let\cyr=\tencyr + \let\big=\tenbig + \setbox\strutbox=\hbox{\vrule height8.5pt depth3.5pt width0pt}% + \normalbaselines\rm} + +\def\ninepoint{\def\rm{\fam0\ninerm}% + \textfont0=\ninerm \scriptfont0=\sixrm \scriptscriptfont0=\fiverm + \textfont1=\ninei \scriptfont1=\sixi \scriptscriptfont1=\fivei + \textfont2=\ninesy \scriptfont2=\sixsy \scriptscriptfont2=\fivesy + \textfont3=\tenex \scriptfont3=\tenex \scriptscriptfont3=\tenex + \def\it{\fam\itfam\nineit}% + \textfont\itfam=\nineit\scriptfont\itfam=\sixit + \def\itbold{\fam\itbfam\nineboldi}% + \textfont\itbfam=\nineboldi\scriptfont\itfam=\sixboldi + \def\sl{\fam\slfam\ninesl}% + \textfont\slfam=\ninesl + \def\bf{\fam\bffam\ninebf}% + \textfont\bffam=\ninebf \scriptfont\bffam=\sixbf + \scriptscriptfont\bffam=\fivebf + \def\tt{\fam\ttfam\ninett + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \def\ttex{\ninetex + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \def\ttl{\let\it\ninetitleit\let\sl\ninetitleit\ninetitle}% + \def\ssf{\niness}% + \textfont\ttfam=\ninett + \tt \ttglue=.5em plus.25em minus.15em + \def\Bbb{\fam\msbmfam\ninemsbm}% + \textfont\msbmfam=\ninemsbm\scriptfont\msbmfam=\sixmsbm \scriptscriptfont\msbmfam=\fivemsbm + \textfont\msamfam=\ninemsam\scriptfont\msamfam=\sixmsam + \scriptscriptfont\msamfam=\fivemsam + \textfont\frakfam=\ninefrak + \def\frak##1{{\fam\frakfam\ninefrak ##1}}% + \normalbaselineskip=11pt + \let\sc=\sevenrm + \let\mc=\eightrm + \let\cyr=\ninecyr + \let\big=\ninebig + \setbox\strutbox=\hbox{\vrule height8pt depth3pt width0pt}% + \normalbaselines\rm} + +\def\eightpoint{\def\rm{\fam0\eightrm}% + \textfont0=\eightrm \scriptfont0=\sixrm \scriptscriptfont0=\fiverm + \textfont1=\eighti \scriptfont1=\sixi \scriptscriptfont1=\fivei + \textfont2=\eightsy \scriptfont2=\sixsy \scriptscriptfont2=\fivesy + \textfont3=\tenex \scriptfont3=\tenex \scriptscriptfont3=\tenex + \def\it{\fam\itfam\eightit}% + \def\itbold{\fam\itbfam\eightboldi}% + \textfont\itfam=\eightit + \def\sl{\fam\slfam\eightsl}% + \textfont\slfam=\eightsl + \def\bf{\fam\bffam\eightbf}% + \textfont\bffam=\eightbf \scriptfont\bffam=\sixbf + \scriptscriptfont\bffam=\fivebf + \def\tt{\fam\ttfam\eighttt + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \def\ttex{\eighttex + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \textfont\ttfam=\eighttt + \tt \ttglue=.5em plus.25em minus.15em + \def\Bbb{\fam\msbmfam\eightmsbm}% + \textfont\msbmfam=\eightmsbm\scriptfont\msbmfam=\sixmsbm \scriptscriptfont\msbmfam=\fivemsbm + \textfont\msamfam=\eightmsam\scriptfont\msamfam=\sixmsam + \scriptscriptfont\msamfam=\fivemsam + \textfont\frakfam=\eightfrak + \def\frak##1{{\fam\frakfam\eightfrak ##1}}% + \normalbaselineskip=9pt + \let\sc=\sixrm + \let\mc=\sevenrm + \let\cyr=\eightcyr + \let\big=\eightbig + \setbox\strutbox=\hbox{\vrule height7pt depth2pt width0pt}% + \normalbaselines\rm} + +\def\sevenpoint{\def\rm{\fam0\sevenrm}% + \textfont0=\sevenrm \scriptfont0=\sixrm \scriptscriptfont0=\fiverm + \textfont1=\seveni \scriptfont1=\sixi \scriptscriptfont1=\fivei + \textfont2=\sevensy \scriptfont2=\sixsy \scriptscriptfont2=\fivesy + \textfont3=\tenex \scriptfont3=\tenex \scriptscriptfont3=\tenex + \def\it{\fam\itfam\sevenit}% + \def\itbold{\fam\itbfam\sevenboldi}% + \textfont\itfam=\sevenit + \def\sl{\fam\slfam\sevensl}% + \textfont\slfam=\sevensl + \def\bf{\fam\bffam\sevenbf}% + \textfont\bffam=\sevenbf \scriptfont\bffam=\sixbf\scriptscriptfont\bffam=\fivebf + \def\tt{\fam\ttfam\seventt + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + }% + \textfont\ttfam=\seventt + \tt \ttglue=.5em plus.25em minus.15em + \def\Bbb{\fam\msbmfam\sevenmsbm}% + \textfont\msbmfam=\sevenmsbm\scriptfont\msbmfam=\sixmsbm \scriptscriptfont\msbmfam=\fivemsbm + \textfont\msamfam=\sevenmsam\scriptfont\msamfam=\sixmsam + \scriptscriptfont\msamfam=\fivemsam + \textfont\frakfam=\sevenfrak + \def\frak##1{{\fam\frakfam\sevenfrak ##1}}% + \normalbaselineskip=8pt + \let\sc=\sixrm + \let\mc=\sixrm + \let\big=\sevenbig + \setbox\strutbox=\hbox{\vrule height6pt depth2pt width0pt}% + \normalbaselines\rm} + +% temporary registers + +\input trt1.sty + +% we can prevent acrobat from defaulting to +% the "shrink to printable area" setting, which causes an 8.5"\times11" page to shrink +% by about 6\% to fit onto the printable region of the physical printer; +% this macro was contributed by Jeff Norden. + +\ifx\pdfcatalog\UNDEFINED\else + \pdfcatalog{/ViewerPreferences<</PrintScaling/None>>}% +\fi + +% a slightly involved way of determining whether we are making a pdf file is +% due to the fact that pdftex can output dvi files as well. + +\newif\ifmakepdf + +\ifx\pdfoutput\undefined + \makepdffalse +\else + \ifnum\pdfoutput>\z@ + \makepdftrue + \else + \makepdffalse + \fi +\fi + +% cleaning up whitespace + +\newif\ifspaceleft + +\def\removewhitespace{% a macro that removes glue, kerns, and + % penalties before the current point + \loop\spaceleftfalse\unwhitespaceonce\ifspaceleft\repeat +} + +\def\unwhitespaceonce{% +\ifhmode + \ifdim\lastskip=\z@\else\unskip\spacelefttrue\fi + \ifdim\lastkern=\z@\else\unkern\spacelefttrue\fi + \ifnum\lastpenalty=\z@\else\unpenalty\spacelefttrue\fi +\fi} + +% color management + +% remember the cmyk color in an impossible control sequence + +\def\colordef#1#2{\expandafter\def\csname cmykc@l@r #1\endcsname{#2}} + +% color switch: we maintain our own color stack for pdf output; +% newer versions of pdf\TeX\ have `proper' color stack primitives +% which provide a more reliable implementation (across page breaks +% and inside insertions, for example) + +\def\colorset#1{\begingroup + \ifmakepdf + \expandafter\let\expandafter\currentc@l@r\csname cmykc@l@r #1\endcsname + \edef\next{\noexpand\pdfliteral{\csname cmykc@l@r #1\endcsname\space k}}\next + \else + \special{ps:gsave}% + \edef\next{\special{ps:\csname cmykc@l@r #1\endcsname\space setcmykcolor}}\next + \fi +} + +\def\currentc@l@r{1 1 1 1} % start out black + +\def\quickcolordef#1#2{\colordef{#1}{#2}% + \expandafter\def\csname color#1\endcsname##1{\colorset{#1}##1\restorecolor}} + +\def\grayset#1{\begingroup + \ifmakepdf + \def\currentc@l@r##1k{#1 g}% so we can use \restoreco... + \pdfliteral{#1 g}% + \else + \special{ps:gsave}\special{ps:#1 setgray}% + \fi +} + +\def\restorecolor{\endgroup + \ifmakepdf + \expandafter\pdfliteral\expandafter{\currentc@l@r\space k}% + \else + \special{ps:grestore}% + \fi +} + +% macros to provide pdf references to outside documents as well as inside the code + +\def\locallink#1{% + \ifmakepdf\colorset{link}\pdfstartlink attr {/Border [0 0 0]} goto name {#1}\fi} + +\def\exlinkpage#1#2#3{\colorset{link}% + \ifmakepdf\pdfstartlink attr {/Border [0 0 0]} goto file {#1} page #2 {#3}\fi} + +\def\endlink{\ifmakepdf\pdfendlink\restorecolor\fi} + +\def\namedspot#1{\ifmakepdf + % raise the reference to make sure that the viewer window shows the + % whole line in which the reference occurs + \raise\ht\strutbox\vbox to 0pt{\vss\hbox to 0pt{\pdfdest name {#1} xyz}}% +\fi} + +% macros to typeset hex values in customized form (useful when typesetting embedded code) + +% the line translator: take a sequence of tokens and translate it into another sequence +% #1 is the sequence of tokens (will be expanded) to insert at the beginning +% #2 is the sequence of tokens (will be expanded) to insert before every token in the sequence +% #3 is the first sequence of tokens (will be expanded) to insert between tokens in the sequence +% #4 is the second sequence of tokens (will be expanded) to insert between tokens in the sequence +% #5 is the sequence of tokens (will be expanded) to insert at the end +% #6 is the command to execute after each token +% #7 is original sequence +% #8 is the resulting sequence + +\def\linetranslator#1#2#3#4#5#6:#7>#8{% + \def\ins@rt{#2}\def\insert@{#3}\def\@insert{#4}% + \def\@nd{#5}\def\c@mmand{#6}\let\t@kens#8% + \edef\next{\t@kens{\the\t@kens#1}}\next + \expandafter\linetr@nslator\the#7\end} + +\def\linetr@nslator#1#2{% + \ifx#1\end + \let\next\relax % no more tokens left + \else + \ifx#2\end % #1 is the last token + \edef\next{\t@kens{\the\t@kens\ins@rt#1\@nd}}\next + \else + \edef\next{\t@kens{\the\t@kens\ins@rt#1\insert@\@insert}}\next + \c@mmand + \fi + \let\next\linetr@nslator + \fi\next#2 +} + +% the mask translator: take a sequence of pairs {length:number}{contents} and translate it into another sequence +% #1 is the sequence of tokens (will be expanded) to insert at the beginning +% #2 is the sequence of tokens (will be expanded) to insert before every pair in the sequence +% #3 is the first sequence of tokens (will be expanded) to insert between pairs in the sequence +% #4 is the second sequence of tokens (will be expanded) to insert between pairs in the sequence +% #5 is the sequence of tokens (will be expanded) to insert at the end +% #6 is the command to execute after each pair +% #7 is original sequence +% #8 is the resulting sequence + +\def\masktranslator#1#2#3#4#5#6:#7>#8{% + \def\ins@rt{#2}\def\insert@{#3}\def\@insert{#4}% + \def\@nd{#5}\def\c@mmand{#6}\let\t@kens#8% + \edef\next{\t@kens{\the\t@kens#1}}\next + \m@sktranslator#7{"40000000}\relax\relax} + +\def\m@sktranslator#1#2#3{% + \ifnum#1<"40000000 % only happens when there are no pairs + \ifnum#3<"40000000 % not the last pair + \edef\next{\t@kens{\the\t@kens\ins@rt{#1}{#2}\insert@\@insert}}\next + \c@mmand + \let\next\m@sktranslator + \else + \edef\next{\t@kens{\the\t@kens\ins@rt{#1}{#2}\@nd}}\next + \let\next\m@sktr@nslator + \fi + \else + \let\next\m@sktr@nslator + \fi + \next{#3}% +} + +\def\m@sktr@nslator#1{} + +% binary converter + +\def\binary#1{% + \ifx#1\end + \let\next\relax + \else + \bin@ry#1\let\next\binary + \fi\next} + +\let\bits\toksa % storage for binary conversion bits +\newtoks\bytes % storage for the binary representation +\let\quotient\tempca +\let\oldquotient\tempcb + +\def\bin@ry#1{\tempcc=4 \quotient="#1 \oldquotient=\quotient +\loop \divide\quotient by2 \multiply\quotient by2 \advance\oldquotient by-\quotient + \advance\tempcc by\m@ne + \edef\next{\bits{\the\oldquotient\the\bits}}\next + \divide\quotient by2 \oldquotient=\quotient +\ifnum\tempcc > \z@ +\repeat +\edef\next{\bytes{\the\bytes\the\bits}}\next +\bits{}} + +% display macros: +% +% o registers + +\newtoks\digittable % the table representing the register +\newcount\columncount % counter for the current column + +% o fonts + +\font\fivessb=phvb at4pt + +% o generic control sequences + +\def\incrementcc{\advance\columncount by\@ne} +\def\decrementcc{\advance\columncount by\m@ne} + +% o mask highlighting + +% the strange looking definition of \m@skspan originates from \multispan in plain.tex: +% \newcount\mscount +% \def\multispan#1{\omit \mscount#1\relax +% \loop\ifnum\mscount>\@ne \sp@n\repeat} +% \def\sp@n{\span\omit\advance\mscount\m@ne} +% +% the number of `physical' columns to span is (2 * columns - 1) due to the fact that we +% have to span the gaps between columns as well. + +\def\m@skspan#1#2{% + \multispan{#1\advance\mscount by\mscount\advance\mscount by-1}{\vrule\highltcl{\leaders\hrule height 4pt\hfil}\vrule}% +} + +% o mask underlining + +\def\m@sksp@n#1#2{% + \multispan{#1\advance\mscount by\mscount\advance\mscount by-1}{\hrulefill}% +} + +% o mask legend typesetting + +\def\m@sklegend#1#2{% + \multispan{#1\advance\mscount by\mscount\advance\mscount by-1}{\hfil\lgndcl{\fivessb#2}\hfil}% +} + +% o displaying each digit + +\def\dgtdispl@y#1#2{% + \ifx#20% + \let\colorbg\lightcl\let\colord\darkcl + \else + \let\colorbg\darkcl\let\colord\lightcl + \fi + \vrule\colorbg{\vrule height4pt width 12.5pt}% + \llap{\raise.6pt\hbox{\colord{\fivessb#1}$\,$}}\vrule +} + +% o typesetting space between digits + +\def\c@lumnspacer#1{\tempca#1\tempcb\tempca\divide\tempca by8\multiply\tempca by8 + \ifnum\tempca=\tempcb + \middlediamond + \else + \tempca#1\tempcb\tempca\divide\tempca by4\multiply\tempca by4 + \ifnum\tempca=\tempcb + \middledot + \else + \hfil + \fi + \fi} + +% oo dimond to separate bytes + +\def\middlediamond{\setbox0=\hbox{$\scriptscriptstyle\diamond$}\tempda=4pt \advance\tempda by-\ht0 + \divide\tempda by2 \hss\raise\tempda\box0\hss}% + +% oo dot to separate nybbles + +\def\middledot{\setbox0=\hbox{.}\tempda=4pt \advance\tempda by-\ht0 + \divide\tempda by2 \hss\raise\tempda\box0\hss}% + +% o put it all together + +\let\stcr\cr % so that this can be used inside \tabalign which redef's \cr + +\def\regdisplay#1#2{{% + \binary#1\end + \linetranslator{}{\hrulefill\noexpand\eatone}{&$\,\,$}{&}{\stcr}{\incrementcc}:\bytes>\digittable + \linetranslator{}{\noexpand\dgtdispl@y{\the\columncount}}{&\noexpand\c@lumnspacer{\the\columncount}}{&}% + {\stcr}{\decrementcc}:\bytes>\digittable + \masktranslator{}{\noexpand\m@sksp@n}{&\hfil}{&}{\stcr}{\relax}:{#2}>\digittable + \masktranslator{}{\noexpand\m@skspan}{&\hfil}{&}{\stcr}{\relax}:{#2}>\digittable + \masktranslator{}{\noexpand\m@sksp@n}{&\hfil}{&}{\stcr}{\relax}:{#2}>\digittable + \expandafter\digittable\expandafter{\the\digittable\noalign{\vskip-3.8pt}}% + \masktranslator{}{\noexpand\m@sklegend}{&\hfil}{&}{\stcr}{\relax}:{#2}>\digittable + \vbox{\offinterlineskip + \halign{&##\stcr + \the\digittable + } + }% +}} + +% an alternate display + +\def\altmiddledot{\setbox0=\hbox{.}% + \tempda=\ht\thezero \advance\tempda by-\ht0 + \divide\tempda by2 \advance\tempda by -\dp\thezero + $\,$\raise\tempda\box0$\,$% +} + +\def\altmiddlediamond{\setbox0=\hbox{$\scriptscriptstyle\diamond$}% + \tempda=\ht\thezero \advance\tempda by-\ht0 + \divide\tempda by2 \advance\tempda by -\dp\thezero + \hss\raise\tempda\box0\hss +} + +\def\altdgtdispl@y#1#2{% + \ifx#20% + \let\colorbg\lightcl\let\colord\darkcl + \else + \let\colorbg\darkcl\let\colord\lightcl + \fi + \setbox0=\hbox{$\,$0$\,$}\tempda\ht0 + \advance\tempda by 2pt \ht0\tempda + \vrule\colorbg{\lower1pt\hbox{\vrule height\ht0 depth\dp0 width \wd0}}% + \llap{\colord{\raise.5pt\hbox{\fivessb#1$\,$}}}\vrule +} + +\def\altc@lumnspacer#1{\tempca#1\tempcb\tempca + \divide\tempca by4\multiply\tempca by4 + \ifnum\tempca=\tempcb + \ % + \else + \kern1pt% + \fi +} + +\def\bytedisplay#1{{% + \binary#1\end + \linetranslator{}{\hrulefill\noexpand\eatone}{&\hfil}{&}{\stcr}{\incrementcc}:\bytes>\digittable + \linetranslator{}{\noexpand\altdgtdispl@y{\the\columncount}}{&\noexpand\altc@lumnspacer{\the\columncount}}{&}% + {\stcr}{\decrementcc}:\bytes>\digittable + \linetranslator{}{\hrulefill\noexpand\eatone}{&\hfil}{&}{\stcr}{\incrementcc}:\bytes>\digittable + \lower\dp\thezero\vbox{\offinterlineskip + \halign{&##\stcr + \the\digittable + } + }% +}} + +\quickcolordef{peach}{0 0.06 0.14 0} +\quickcolordef{corn}{0 0.06 0.63 0.02} +\quickcolordef{sandybrown}{0 0.33 0.61 0.04} +\quickcolordef{darkwood}{0 0.29 0.5 0.48} +\quickcolordef{link}{0 0 1 0.30} + +\def\colorblack#1{\grayset{0}#1\restorecolor} +\def\colorwhite#1{\grayset{1}#1\restorecolor} +\def\colorgray#1{\grayset{.5}#1\restorecolor} + +\newif\iffullcolorpalette +\fullcolorpalettetrue + +\iffullcolorpalette + \let\lightcl=\colorpeach + \let\shadedcl=\colorcorn + \let\highltcl=\colorgray + \let\lgndcl=\colorwhite +\else + \let\lightcl=\colorwhite + \let\shadedcl=\colorgray + \let\highltcl=\colorwhite + \let\lgndcl=\colorblack +% \let\colorlinkstart=\begingroup +% \let\colorlinkend=\endgroup +\fi + \let\darkcl=\colorblack + +% macros to typeset the hex numbers in `graphic' format + +\newtoks\registerlegend + +\def\registerdisplay#1\end{\edef\next{\the\registerlegend}% + \ifx\next\empty + \edef\next{\noexpand\bytedisplay{#1}}\next + \else + \edef\next{\noexpand\regdisplay{#1}{\the\registerlegend}}\next + \fi +} + +\newbox\thezero + +\setbox\thezero=\hbox{\lower1pt\hbox{\vbox{\offinterlineskip +\halign{#&$\,$#$\,$&#\cr +\noalign{\hrule} +\omit\vrule height1pt&&\omit\vrule height1pt\cr +\vrule&\phantom0&\vrule\cr +\omit\vrule height1pt&&\omit\vrule height1pt\cr +\noalign{\hrule} +}}}} + +\def\texbinary{\global\let\oldT\T + \global\def\T##1{{\let\end\relax\let\^\registerdisplay##1\end{\tt :}\let\^\hexify##1\end}}} + +\def\endtexbinary{\global\let\T\oldT} + +\def\hexify#1\end{\hbox{{\tt #1}$_{\scriptscriptstyle1\kern-.7pt6}$}} + +% verbatim listing macros borrowed (with changes) from The TeXbook + +\newcount\democodelc + +\chardef\other=12 +\def\ttverbatim{\begingroup + \catcode`\\=\other + \catcode`\{=\other + \catcode`\}=\other + \catcode`\$=\other + \catcode`\&=\other + \catcode`\#=\other + \catcode`\%=\other + \catcode`\~=\other + \catcode`\_=\other + \catcode`\|=\other + \obeyspaces \obeylines \tt \ttglue=.5em plus.25em minus.15em } + +{\catcode`\^=\active \catcode`\*=7 \obeylines + \gdef^{\leavevmode\ttverbatim \spaceskip\ttglue \def**M{\endgroup\cr}\let^=\endgroup}} + +\def\beginlines{\begingroup\parindent0pt \obeylines \everypar{\strut}\catcode`\^=\active} +\def\endlines{\endgroup} + +\def\paddednumber#1{\ifnum #1 < 10 0\number#1\relax\else \number#1\fi} + +% demo code display +% **H will produce a `|', **K can be used as an escape character, **L and **N +% can be used i place of left and right braces to provide some control + +{\catcode`\ =\active\gdef {\phantom{\char"20\relax}}\catcode`\^^H=\active\gdef^^H{\yl}}% + +\newif\ifnumberlines + +\def\begindemo{% + \medskip + \bgroup + \def\linenoup{% + \global\advance\democodelc by1 + \ifnumberlines{\sevenrm\paddednumber\democodelc}\fi} + \def\nooutput{\omit\hfil{\sevenrm\strut[more code $\ldots$]}\hfil\cr\noalign{\kern3pt}} + \catcode`\^=\active\democodelc=0 \catcode`\*=7 + \chardef\*=`\*% + \catcode`\^^H=\active % ^^H + \catcode`\^^K=0 \catcode`\^^L=1 \catcode`\^^N=2 + \tabskip=0pt plus 1 fill + \halign \demoastyle\bgroup\linenoup\hskip\demomargin ##\hfil\hskip\demomargin% + \ifnumberlines{\sevenrm\paddednumber\democodelc}\fi\cr% +} + +\def\enddemo{\egroup\egroup\medskip\noindent} + +\newskip\demomargin +\demomargin=3em + +\def\demoastyle{to\hsize} + +% macros to provide some primitive alignment + +\def\texalignment{\setbox0\lastbox\setbox0\lastbox\unkern\begingroup\settabs 3 \columns +\let\C\tabC\let\8\tabalign\def\5{\relax\ifmmode\oopsamp\else\clearamp\fi}\def\6{\strut\cr}} + +\def\oopsamp{$&$} +\def\clearamp{&} + +\def\texalignmentshort{\setbox0\lastbox\setbox0\lastbox\unkern\begingroup\settabs 2 \columns + \let\C\tabC\let\8\tabalign\def\5{\ifmmode\oopsamp\else\clearamp\fi}\def\6{\strut\cr}} + +\def\endtexalignment{\vskip-\baselineskip\strut\endgroup} + +\def\eatcr#1\6{\ignorespaces} + +\newtoks\postlimbotoks + +\def\postlimbostuff{% +} + +\def\picture#1{\ifmakepdf\pdfximage{#1.pdf}\pdfrefximage\pdflastximage + \else + \epsfbox{#1.eps}\fi +} + +\let\yl| % to use in \TeX\ mode instead of `|' +\def\greaterthan{>} % CTANGLE always leaves a space after `>' in the output which interferes with some macros + +% redefinitions of various CWEB macros; + +% macros that provide a way to customize printing of reserved words + +\newtoks\tempseq +\tempseq={RVD} +\let\oldamp\& +\let\oldoneletterid\| +\def\replaceunderscore#1{\let\next\replaceunderscore + \ifx#1\_\tempseq=\expandafter{\the\tempseq x}\else + \ifx#1\#\tempseq=\expandafter{\the\tempseq H}\else + \ifx#1\end\let\next\relax\else\tempseq=\expandafter{\the\tempseq #1}\fi + \fi + \fi\next} +\def\reservedid#1{{\replaceunderscore#1\end + \expandafter\let\expandafter\newseq\csname\the\tempseq\endcsname + \ifx\relax\newseq\oldamp{#1}\else\newseq\fi}} + +\def\reservedoneletterid#1{{% + \expandafter\let\expandafter\newseq\csname\the\tempseq\string#1\endcsname + \ifx\relax\newseq\oldoneletterid{#1}\else\newseq\fi}} + +\def\defreserved#1{% + \expandafter\def\csname\the\tempseq #1\endcsname +} + +\let\&\reservedid +\let\|\reservedoneletterid + +% cweb macros adopted to making typesetting languages other than C possible: Makefile and linker scripts. + +\tempca=\catcode`\^^I% remember the old catcode +\catcode`\^^I=\active +\def\vb{\begingroup\catcode`\^^I=\active\vvb} % give the tab a chance to change its catcode before the rest is scanned +\def\vvb#1{{% + \hbox{\vbox{\ttex%\tt + \let\\=\BS % backslash in a string + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + \let\~=\TL % tilde in a string + %\let\ =\space % space in a string + \def\ {\phantom{0}}% + \let\_=\UL % underline in a string + \let\&=\AM % ampersand in a string + \let\^=\CF % circumflex in a string + \let\$=\oldmathS % dollar sign + \def^^I{$\llcorner$&$\lrcorner$}% a tab is a tab + \tabalign{}#1\cr}}}\endgroup} % verbatim string +\catcode`\^^I=\tempca + +\def\hex{\hbox{\tt\aftergroup}\aftergroup\hexsubscript} +\def\hexsubscript{\hbox{$_{\scriptscriptstyle1\kern-.7pt6}$}} +\let\K=\Leftarrow +\setbox\MGbox=\hbox{\hbox{\ninei\char'053}} +\def\R{\hbox{$^{\rm not}$}} +\let\CM\lnot +\def\MRL#1{\KK#1}\def\KK#1#2{\buildrel\;{\let\OR\lor\scriptscriptstyle#1}\over#2} +\def\C#1{\5\5\quad$\triangleright\,${\rm #1}$\,\triangleleft$} +\let\Z=\leq \let\G=\geq +\let\E== + +\def\tabC#1{\quad$\triangleright\,${\cmntfont#1}$\,\triangleleft$} + +% better `hash' + +\let\oldhash\# +\def\#{\lower.5pt\hbox{\.{\oldhash}}} + +\def\.#1{\leavevmode\hbox{\tt % typewriter type for strings + \let\\=\BS % backslash in a string + \let\{=\LB % left brace in a string + \let\}=\RB % right brace in a string + \let\~=\TL % tilde in a string + \let\ =\SP % space in a string + %\let\_=\UL % underline in a string + \let\&=\AM % ampersand in a string + \let\^=\CF % circumflex in a string + \let\uscore\_\chardef\_=`\_% + #1\kern.05em}} + +% macros to change the appearance of section headers + +\outer\def\N#1#2#3.{% beginning of starred section + \ifacro{\toksF={}\makeoutlinetoks#3\outlinedone\outlinedone}\fi + \gdepth=#1\gtitle={#3}\MN{#2}% + \ifon\ifnum#1<\secpagedepth \vfil\eject % force page break if depth is small + \else\vfil\penalty-100\vfilneg\vskip\intersecskip\fi\fi + \message{*\secno} % progress report + \def\stripprefix##1>{}\def\gtitletoks{#3}% + \edef\gtitletoks{\expandafter\stripprefix\meaning\gtitletoks}% + \edef\next{\write\cont{%\noexpand\noexpand\noexpand\eatone{\noexpand\meaning\noexpand\ZZ}% + \noexpand\noexpand\noexpand % AS this is a (sort of) bug in cwebmac.tex + % as long as the index is output (i.e. ...\eject) + % before all the sections this is unnecessary + % uncomment the line after the brace to see what \ZZ + % expands to in case of weird errors + \ZZ{\gtitletoks}{#1}{\secno}% write to contents file + {\noexpand\the\pageno}{\the\toksE}}}\next % \ZZ{title}{depth}{sec}{page}{ss} + \ifpdftex\expandafter\xdef\csname curr#1\endcsname{\secno}% + \ifnum#1>0\countB=#1 \advance\countB by-1 + \advancenumber{chunk\the\countB.\expnumber{curr\the\countB}}\fi\fi + \ifpdf\special{pdf: outline #1 << /Title (\the\toksE) /Dest + [ @thispage /FitH @ypos ] >>}\fi + \ifon\startsection{\ttl#3.\ }\ignorespaces} % changed the font; AS + +\let\oldN\N + +\outer\def\textN#1#2#3.{% beginning of starred section in `text' mode + \ifacro{\toksF={}\makeoutlinetoks#3\outlinedone\outlinedone}\fi + \gdepth=#1\gtitle={#3}\MN{#2}% + \ifon + \ifnum#1<2 + \vfil\eject % force page break for chapters + \else + \vskip0pt plus 3.5\baselineskip\penalty-100\vskip0pt plus -3.5\baselineskip\vskip\intersecskip % no page break + \fi + \fi + \message{*\secno} % progress report + \def\stripprefix##1>{}\def\gtitletoks{#3}% + \edef\gtitletoks{\expandafter\stripprefix\meaning\gtitletoks}% + \edef\next{\write\cont{%\noexpand\noexpand\noexpand\eatone{\noexpand\meaning\noexpand\ZZ}% + \noexpand\noexpand\noexpand % AS this is a (sort of) bug in cwebmac.tex + % as long as the index is output (i.e. ...\eject) + % before all the sections this is unnecessary + % uncomment the line after the brace to see what \ZZ + % expands to in case of weird errors + \ZZ{\gtitletoks}{#1}{\secno}% write to contents file + {\noexpand\the\pageno}{\the\toksE}}}\next % \ZZ{title}{depth}{sec}{page}{ss} + \ifpdftex\expandafter\xdef\csname curr#1\endcsname{\secno}% + \ifnum#1>0\countB=#1 \advance\countB by-1 + \advancenumber{chunk\the\countB.\expnumber{curr\the\countB}}\fi\fi + \ifpdf\special{pdf: outline #1 << /Title (\the\toksE) /Dest + [ @thispage /FitH @ypos ] >>}\fi + \ifon\startsection{\ttl#3}\smallskip\noindent\ignorespaces} + +\outer\def\textM#1{\MN{#1}\ifon\vskip0pt plus 3\baselineskip\penalty-100\vskip0pt plus -3\baselineskip % no page break + \vskip\intersecskip\startsection\ignorespaces} + +\def\MN#1{\par % common code for \M, \N + {\xdef\secstar{#1}\let\*=\empty\xdef\secno{#1}}% remove \* from section name + \ifx\secno\secstar \onmaybe \else\ontrue \fi + \mark{{\secno}{\the\gdepth}{\the\gtitle}}} % remove the annoying + % `paragraph' sign; AS + +% a new start section macro: fix the extra space bug and push the setion number to the margins + +\def\stseclap{\rightskip=0pt % get out of C mode (cf. \B) + \sfcode`;=1500 \pretolerance 200 \hyphenpenalty 50 \exhyphenpenalty 50 + \noindent{\let\*=\lapstar\llap{\tentitle\secstar\quad}}% push it to the margins + \ifpdftex\smash{\raise\baselineskip\hbox to0pt{% + \let\*=\empty\ifmakepdf\pdfdest num \secstar fith\fi}}% this space is a bug in the original cwebmac.tex; AS + \else\ifpdf\smash{\raise\baselineskip\hbox to0pt{% + \let\*=\empty\special{% + pdf: dest (\romannumeral\secstar) [ @thispage /FitH @ypos ]}}}\fi\fi} + +\let\startsection\stseclap + +% make page and sectoin number hang in the margins; \quad is chosen to +% be consistent with section appearance; the page numbers are printed +% in old-style numerals; + +\def\lheader{\headertrue\llap{\mainfont\oldstyle\the\pageno\tentitle\quad}\eightpoint\rm\grouptitle + \hfill\title\rlap{\tentitle\quad\tenpoint$^{\hbox{\sevenrm\topsecno}}_{\hbox{\sevenrm\botsecno}}$}} % top line on left-hand pages +\def\rheader{\headertrue +\llap{\tenpoint$^{\hbox{\sevenrm\topsecno}}_{\hbox{\sevenrm\botsecno}}$\tentitle\quad}% +\eightpoint\rm\title\hfill + \grouptitle\rlap{\tentitle\quad\mainfont\oldstyle\the\pageno}} % top line on right-hand pages + +\def\botsecno{\expandafter\takeone\botmark} % the first section on the + % next page + +% from time to time, entries must be omitted from the index; +% the macros below implement a mechanism to do that + +\newif\ifisindexable + +\def\setindexable#1{% + \isindexabletrue + \def\next{#1}% + \expandafter\s@t@ndexable\unindexable.\end +} + +\def\s@t@ndexable#1#2\end{% + {% + \def\ifisindexable{#1}% + \ifx\ifisindexable\next + \yybreak{\aftergroup\isindexablefalse\aftergroup\eattoend}% + \else + \def\ifisindexable{#2}% + \ifx\ifisindexable\dotcontainer + \yybreak@{\aftergroup\eattoend}% + \else + \yybreak@{\aftergroup\s@t@ndexable}% + \fi + \yycontinue + }% + #2\end +} + +\def\inxmod{% + \write\cont{} % ensure that the contents file isn't empty + \write\cont{\catcode `\noexpand\@=12\relax} % \makeatother + \closeout\cont % the contents information has been fully gathered + \message{Index:} + \medskip + \eightpoint\raggedright + \fnotesstart=2 + \fnotesspan=1 + \noofcolumns=3 + \icgap=5pt% + \linecount=3 + \setmcparams + \dsskip=0pt% + \adjskip=0pt plus 9pt% + \ifx\unindexable\UNDEFINED + \def\unindexable{{$\TeXx$}{$\TeXa$}{$\TeXb$}{$\TeXf$}{$\TeXao$}{$\TeXfo$}}% + \else + \expandafter\def\expandafter\unindexable\expandafter{\unindexable + {$\TeXx$}{$\TeXa$}{$\TeXb$}{$\TeXf$}{$\TeXao$}{$\TeXfo$}% + }% + \fi + \def\I##1, ##2.{% + {% + \setindexable{##1}% + \ifisindexable + \else + \aftergroup\eatone + \fi + }% + {% + \hangindent1em\noindent##1:\kern.5em + \ifacro\pdfnote##2.\else##2\fi.% + \par + }% + }% + % indexing macros for grammar terms + \def\termidstring##1{% processed name in italics + \numberstocharsandspaces##1\end + \let\optstrextra\optstrextraesc + \expandafter\nameproc\expandafter{\the\toksa}% + \def\idxentry{{\it\the\toksa}}% + }% + \def\termvstring##1{% processed name in typewriter style + \numberstocharsandspaces##1\end + \let\optstrextra\optstrextraesc + \expandafter\nameproc\expandafter{\the\toksa}% + \def\idxentry{{\tt\def\_{\char`\_}\the\toksa}}% + }% + \def\termttstring##1{% straightforward typewriter text + \numberstocharsandspaces##1\end + \def\idxentry{{\tt\the\toksa}}% + }% + \def\termostring##1{% options (e.g. \flex\ and \bison\) + \numberstocharsandspaces##1\end + \def\idxentry{{$\langle$\bf\the\toksa$\rangle$}}% + }% + \let\termhdrstring\termidstring + \def\termstring##1{% + \numberstocharsandspaces##1\end + \let\optstrextra\optstrextraesc + \expandafter\nameproc\expandafter{\the\toksa}% + \def\idxentry{{\tt"\the\toksa"}}% + }% + \def\texcsstring##1{% + \numberstocharsandspaces##1\end + \def\idxentry{% + \def\texnspace{index}% + \def\getcescape{% an \ seen is really an \, and will not go through C string processing + \def\next{escape}% + \action\next\in\currentstate + }% + \termindexfalse\expandafter\inlineTeXx\expandafter{\expandafter/\the\toksa}% + }% + }% + \expandafter\def\csname acharswitch:index\endcsname{% correct reserved \TeX\ characters a la CWEB verbatim + %$\%\\ % unaffected + %\#\ % these never appear + _{% + \yybyte\expandafter{\csname \the\yybyte\endcsname}% + \expandafter\yycp@\expandafter`\the\yybyte\relax + \mkpurebyte + \yyreturn + } + }% + \expandafter\setspecialcharsfrom\csname acharswitch:index\endcsname + \expandafter\let\expandafter\acharswitch\csname acharswitch:index\endcsname + \def\texlexer{% now that all character codes are 12 + \let\default\yygetchar + \let\next\yycp@ + \ifnum\yycp@>"3F % + \ifnum\yycp@<"5B % an uppercase letter or @ + \def\next{letter}% + \fi + \fi + \ifnum\yycp@>"60 % + \ifnum\yycp@<"7B % + \def\next{letter}% + \fi + \fi + \action\next\in\currentstate + }% + \def\indexseparator##1##2{% + \vskip.5\baselineskip + \centerline{\dinkus}% + \vskip.5\baselineskip + } + \def\GI##1##2##3##4.{% + {% + \def\hostparsernamespace{##1}% + \toksa{}##2{##3}% + \I{\idxentry}##4.% + }% + }% + % end indexing macros for grammar terms + \let\*=\lapstar + \def\[##1]{{\it##1}} % definitions + \def\(##1){$\underline{##1}$} % underlined index item + \def\(##1){{\bf ##1}} % an alternative to the above + \def\e##1e{##1{\sevenpoint$^\circ$}} % terms in examples + \def\f##1f{{\it##1\/\kern.2ex}${}^\circ$} % lhs in examples (italic correction does not seem to be enough) + \def\g##1g{$\underline{##1}^\circ$} % definitions in examples + \def\g##1g{{\bf ##1}$^\circ$} % an alternative to the above + \begindoublecols + \readindex + \otherlangindexseparator + \readgindex +} + +\def\otherlangindexseparator{% + \par + \vskip.5\baselineskip + \centerline{B{\sc ISON AND} \TeX\ {\sc INDEX}}% + \vskip.5\baselineskip + \par +} + +\def\finmod{% + \enddoublecols + \closeout\exampletable + \unsetfootnotes + \parfillskip 0pt plus 1fil + \def\grouptitle{NAMES OF THE SECTIONS} + \let\topsecno=\nullsec + \message{Section names:} + \setpage + \def\note##1##2.{\quad{\eightrm##1~\ifacro{\pdfnote##2.}\else{##2}\fi.}} + \def\Q{\note{Cited in section}} % crossref for mention of a section + \def\Qs{\note{Cited in sections}} % crossref for mentions of a section + \def\U{\note{Used in section}} % crossref for use of a section + \def\Us{\note{Used in sections}} % crossref for uses of a section + \def\I{\par\hangindent 2em}\let\*=* + \ifacro + \def\outsecname{Names of the sections} + \let\Xpdf\X + % + \ifpdftex + \makebookmarks + \pdfdest name {NOS} fitb + \pdfoutline goto name {NOS} count -\secno {\outsecname} + \def\X##1:##2\X{\Xpdf##1:##2\X \firstsecno##1.% + {\toksF={}\makeoutlinetoks##2\outlinedone\outlinedone}% + \pdfoutline goto num \the\toksA \expandafter{\the\toksE}} + \else + \ifpdf + \special{pdf: outline -1 << /Title (\outsecname) + /Dest [ @thispage /FitH @ypos ] >>} + \def\X##1:##2\X{\Xpdf##1:##2\X \firstsecno##1.% + {\toksF={}\makeoutlinetoks##2\outlinedone\outlinedone}% + \special{pdf: outline 0 << /Title (\the\toksE) + /A << /S /GoTo /D (\romannumeral\the\toksA) >> >>}} + \fi + \fi + \fi + % + \bigskip + \vskip0pt plus 5pt + \penalty-10 + \tenpoint + \centerline{\sc A LIST OF ALL SECTIONS} + \penalty300 + \medskip + \readsections +} + +\newread\trygindex + +\def\readgindex{% + \openin\trygindex=\jobname.gdy + \ifeof\trygindex + \else + \closein\trygindex + \input \jobname.gdy + \fi +} + +% main section macro + +\let\oldB\B + +% stage one macros for `\B' sections: collecting tokens + +% \B is always paired with \par (see CWEAVE section 213) + +\long\def\Bvbpp#1\par{% + \Bvbp@#1\X\X${}\E{}$\par +} + +\long\def\Bvbp@#1\X#2\X$#3\E#4$#5\par{% + \yystringempty{#5}% + {% + \Bvb@@#1\X#2\X$#3\E#4$\par + }{% + \Bvb@@#1\X#2\X${#3}\E{#4}$\begingroup + \ninepoint + #5\par\endgroup + }% +} + +\long\def\Bvb@@#1\X\X$#2\E#3$\par{% + \oldB#1\par +} + +\def\begingsec#1#2\endgsec{% + \ifmmode + \yybreak{{}${}\vbpp{}$#2\vb\endgsec{#1}{$#2}}%$ + \else + \yybreak{\vbpp{}#2\vb\endgsec{#1}{#2}}% + \yycontinue +} + +\long\def\vbpp#1#2\vb#3#{% verbatim section preprocessor + \yystringempty{#3}{% + \vbp@{#1\stashed{#2}}% + }{% + #3{#1\stashed{#2}}% + }% +} + +\long\def\vbp@#1#2{% + \vbpp{#1#2}% +} + +\let\B\Bvbpp + +% URL typesetting + +\def\url#1{% + \hskip0pt plus .3\hsize\penalty100 + \hskip0pt plus -.3\hsize + {% + \def~{\string~}% + \ifmakepdf + \pdfURL{{\tt #1}}{#1}% + \else + {\tt #1}% + \fi + }% + \hskip0pt plus .2\hsize\penalty100 + \hskip0pt plus -.2\hsize +} + +% commonly used names + +\def\CWEB{\.{CWEB}}\def\CTANGLE{\.{CTANGLE}}\def\CWEAVE{\.{CWEAVE}} +\def\splint{\.{SPLinT}} +\def\POSIX{{\sc POSIX}} +\def\ISO{{\mc ISO}} +\def\bison{\.{bison}} +\def\flex{\.{flex}} +\def\lex{\.{lex}} +\def\gcc{\.{gcc}} +\def\GNU{{\sc GNU}} +% from tugboat.cmn +\newbox\TestBox +\def\La{\tempca=\the\fam \leavevmode L% + \setbox\TestBox=\hbox{$\fam\tempca\scriptstyle A$}% + \kern-.5\wd\TestBox\raise.42ex\box\TestBox} +\def\LaTeX{\La\kern-.15em\TeX} + +\def\sqbullet{\kern.3ex\vrule height .9ex width .8ex depth -.1ex }% square bullet (from manmac.tex) + +\def\sqebullet{% empty square bullet + \kern.3ex\vbox{\hsize=.8ex + \hrule height .4pt + \kern-.4pt + \hbox to\hsize{\vrule height .9ex \hfill\vrule height .9ex} + \kern-.4pt + \hrule height .4pt + \kern.1ex + }% +} + +% debugging + +\def\shownethe#1{% + \edef\next{\the#1}% + \ifx\next\empty + \else + \showthe#1% + \fi +} + +\def\showem#1#2#3{\toksa{#1}\toksb{#2}\toksc{#3}{\newlinechar=`^^J% + \errmessage{% + arg. 1: \the\toksa^^J% + arg. 2: \the\toksb^^J% + arg. 3: \the\toksc% +}}} + +% sugar + +\def\FOREVER{{\bf forever}} + +% delay functions prettyprinting and other miscellanea; + +\def\delayhalfsec{\hbox{wait \kern-.5pt\raise1pt\hbox{$\scriptstyle1$}% +\kern-1pt/\kern-1pt\lower1pt\hbox{$\scriptstyle2$} sec, }} +\def\delayquartersec{\hbox{wait \kern-.5pt\raise1pt\hbox{$\scriptstyle1$}% +\kern-1pt/\kern-1pt\lower1pt\hbox{$\scriptstyle4$} sec, }} + +\def\delay#1sec{% +\hbox{\def\secmult{}wait +\getsecmult#1\end +\if\secmult u\def\secmodifier{ $\mu$\kern.05pt }\else +\if\secmult m\def\secmodifier{ m}\else +\if\secmult n\def\secmodifier{ n}\else + \secmult\def\secmodifier{ }% +\fi\fi\fi +\secmodifier sec$\rm\scriptstyle s$, } +} + +\def\getsecmult#1{% +\ifx#1\end\let\next\relax\else + \secmult\def\secmult{#1}\let\next\getsecmult +\fi\next +} + +% attribute prettyprinting; + +\long\def\parseattrib#1#2\end{\csname attrib#1\endcsname} + +\def\xxattributexx#1{\ifx(#1\let\next\xxattributex\else\let\next\xxattribute\fi\next#1} +\def\xxattributex((\\#1{% + \expandafter\let\expandafter\currentattrib\csname attrib#1\endcsname + \ifx\currentattrib\attribpacked + \strut\colorblack{% + \lower2pt\hbox{\vrule width \wd\thezero height + \ht\thezero depth + \dp\thezero}}\llap{\colorwhite{\hbox{$\scriptscriptstyle\bf 01\kern.5pt$}}}% + \let\next\eatparentheses + \fi + \ifx\currentattrib\attribnoinline + {\ \bf noinline\rm_a}% + \let\next\eatparentheses + \fi + \ifx\currentattrib\attribformat + {\ \bf format\rm_a}% + \let\next\eatarguments + \fi + \next +} + +\def\eatparentheses#1)){} +\def\eatarguments(\\#1,#2,#3))){(\hbox{\rm as }\.{#1})} + +\def\xxattribute#1{\.{\_\_attribute\_\_} (\sc GNU) #1} + +\let\attribnoinline=1 +\let\attribpacked=2 +\let\attribformat=3 + +\tenpoint +%\registerlegend{{8}{\noexpand\phantom{B}}} diff --git a/support/splint/tex/stokenset.sty b/support/splint/tex/stokenset.sty new file mode 100644 index 0000000000..7146fcf0c1 --- /dev/null +++ b/support/splint/tex/stokenset.sty @@ -0,0 +1,23 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\prettywordpair{OPTIONAL}{{\tt opt}}% +\prettywordpair{NO_ATTR}{{\tt na}}% +\prettywordpair{EXTENDED}{{\tt ext}}% +\prettywordpair{INTEGER}{{$[\,0\ldots9\,]\ast$}}% +\prettywordpair{IDENTIFIER}{{$[\,\hbox{\tt a}\ldots\hbox{\tt Z}\,0\ldots9\,]\ast$}}% +\prettywordpair{WILDCARD}{{{\tt *} {\rm or} {\tt ?}}}% +\prettywordpair{PERCENT_IDENTIFIER}{{\tt\%$[\,\hbox{\tt a}\ldots\hbox{\tt Z}\,0\ldots9\,]\ast$}}% diff --git a/support/splint/tex/trt1.sty b/support/splint/tex/trt1.sty new file mode 100644 index 0000000000..e3e722c91f --- /dev/null +++ b/support/splint/tex/trt1.sty @@ -0,0 +1,37 @@ +% Copyright 2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\csname loadtexruntime1\endcsname + +\message{Temporary register definitions ...} +\newcount\tempca +\newcount\tempcb +\newcount\tempcc +\newcount\tempcd +\newcount\tempce + +\newdimen\tempda +\newdimen\tempdb +\newdimen\tempdc + +\newtoks\toksa +\newtoks\toksb +\newtoks\toksc +\newtoks\toksd +\newtoks\tokse +\newtoks\toksf + +\expandafter\def\csname loadtexruntime1\endcsname{\endinput} diff --git a/support/splint/tex/xarithm.sty b/support/splint/tex/xarithm.sty new file mode 100644 index 0000000000..4d424a863c --- /dev/null +++ b/support/splint/tex/xarithm.sty @@ -0,0 +1,317 @@ +% Copyright 2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% a minimum of expandable arithmetic to implement a few macros in yyparse.sty and flex.sty +% the goal of these macros was not implementing a complete +% integer arithmetic but rather providing the minimal functionality +% where such arithmetic is required. +% there are a few instances in which the utility of such arithmetic is +% beneficial: +% o whenever an `expession macro' should behave like a number (i.e. be +% useful in a \number context); see \YYSTART implementation for an +% example +% o whenever the use of a register is to be avoided in iterative +% macros and replaced by a `tail parameter' (see \yyless for a hint +% of such a use) +% o whenever \TeX's number size limitations are inadequate +% +% the last situation has not been a concern for this package while the +% other two were. +% +% `expandable' may have different meanings. in the context of +% arithmetic, expansion within \edef context, `prefix expansion' or +% \number or \romannumeral0 context expansion may all be valid +% targets. the preferred way seems to be the \romannumeral0 +% interface. the disadvantage of this method is that each macro needs +% to produce a space at the end of its `expansion cycle', the +% advantage is the ability to carry the extension for arbitrarily +% large numbers (unlike, say, the \number interface). the macros below +% are implemented so that the \expandafter\expandafter `prefix' +% expansion and \number both work, even though internally, the +% \romannumeral0 approach is taken. this is done to present a cleaner +% use pattern to the consumer of the macros. +% +% for a complete implementation of expandable arithmetic (including +% floating point), see the corresponding parts of the +% \LaTeX\ project. it was decided against using that implementation to +% avoid creating dependencies. + +% temporary definition, to avoid using reserved counters + +\edef\xxcarry{\count3=\the\count3\count\z@\the\count\z@\count\@ne\the\count\@ne} + +% digit extractor, used as part of several macros below. + +\count\z@\z@ +\bloop + \edef\next{\def\csname digit\number\count\z@\endcsname##1\number\count\z@##2##3.{##2}}\next % +\ifnum9>\count\z@ + \advance\count\z@\@ne +\repeat + +% multiplication tables: +% \xmultby?#1 expands into a pair ab which is a product of ? and #1 + +\count\z@\z@ +\bloop{% + \count\@ne\z@ + \let\next\empty + \bloop + \count3\count\z@ + \multiply\count3\count\@ne + \edef\next{\next\number\count\@ne{\ifnum9<\count3 \else0\fi\number\count3}}% + \ifnum9>\count\@ne + \advance\count\@ne\@ne + \repeat + \edef\next{\def\csname xmultby\number\count\z@\endcsname##1{\noexpand\csname digit##1\noexpand\endcsname\next.}}\next +\ifnum9>\count\z@ + \advance\count\z@\@ne +}\repeat + +% addition tables: +% \xaddto?#1 expands into a pair ab which is a sum of ? and #1 + +\count\z@\z@ +\bloop{% + \count\@ne\z@ + \let\next\empty + \bloop + \count3\count\z@ + \advance\count3\count\@ne + \edef\next{\next\number\count\@ne{\ifnum9<\count3 \else0\fi\number\count3}}% + \ifnum9>\count\@ne + \advance\count\@ne\@ne + \repeat + \edef\next{\def\csname xaddto\number\count\z@\endcsname##1{\noexpand\csname digit##1\noexpand\endcsname\next.}}\next +\ifnum9>\count\z@ + \advance\count\z@\@ne +}\repeat + +% division tables: +% \xdivby?#1 expands into #1 div ? + +\count\z@\@ne +\bloop{% + \count\@ne\z@ + \let\next\empty + \bloop + \count3\count\@ne + \divide\count3\count\z@ + \edef\next{\next\number\count\@ne{\number\count3}}% + \ifnum9>\count\@ne + \advance\count\@ne\@ne + \repeat + \edef\next{\def\csname xdivby\number\count\z@\endcsname##1{\noexpand\csname digit##1\noexpand\endcsname\next.}}\next +\ifnum9>\count\z@ + \advance\count\z@\@ne +}\repeat + +\xxcarry % restore registers + +% #1\times #2 + #3 + +\def\xxcarry#1#2#3{% + \expandafter\x@carry\number1\csname xmultby#1\endcsname#2#3% +} + +% #1 is a bogus parameter, so that a leading 0 is not dropped by \number + +\def\x@carry#1#2#3#4{% + \expandafter\x@@arry\number1\csname xaddto#3\endcsname#4#2% +} + +\def\x@@arry#1#2#3#4{% + \ifnum#2>\z@\number\csname xaddto#2\endcsname#4\else#4\fi#3% +} + +% most macros internally consume the digits from left to right. + +\def\xreverse#1{% + \romannumeral0\xr@verse{}#1.% +} + +\def\xr@verse#1#2{% + \if#2.% + \yybreak{ #1}% + \else + \yybreak{\xr@verse{#2#1}}% + \yycontinue +} + +% multiplication by 0-9 +% #1 is the number +% #2 is the digit +% expands to #1 * #2 in reverse digid order + +\def\xmultiplybydigit#1#2{% + \romannumeral0\xbydigit{#2}{0}{}#1.% +} + +% #1 is the digit +% #2 is the carry +% #3 is the result +% #4 is the next digit of the big factor + +\def\xbydigit#1#2#3#4{% + \if#4.% + \yybreak{\ifnum#2=\z@\xskiptofi{ #3}\else\xskiptofi{ #3#2}\fi}% + \else + \yybreak{\expandafter\xb@digit\number1#1\xxcarry#1#4#2 {#3}}% + \yycontinue +} + +% #1 is a bogus parameter (see above) +% #2 is the digit +% #3 is the carry +% #4 is the next digit of the result +% #5 is the result up to now + +\def\xb@digit#1#2#3#4#5{% + \xbydigit#2#3{#5#4}% +} + +% a version of the macros above that perform the reversion to save an +% extra step + +% #1 is the digit +% #2 is the carry +% #3 is the result +% #4 is the next digit of the big factor + +\def\xbytigid#1#2#3#4{% + \if#4.% + \yybreak{\ifnum#2=\z@\xskiptofi{ #3}\else\xskiptofi{ #2#3}\fi}% + \else + \yybreak{\expandafter\xb@tigid\number1#1\xxcarry#1#4#2 {#3}}% + \yycontinue +} + +% #1 is a bogus parameter (see above) +% #2 is the digit +% #3 is the carry +% #4 is the next digit of the result +% #5 is the result up to now + +\def\xb@tigid#1#2#3#4#5{% + \xbytigid#2#3{#4#5}% +} + +% similar to \xmultiply by digit above by produces the result in the +% usual digit order + +\def\xsmallmultiple#1#2{% + \romannumeral0\expandafter\xsm@llmultiple\expandafter{\romannumeral0\xr@verse{}#1.}#2% +} + +\def\xsm@llmultiple#1#2{% + \xbytigid{#2}{0}{}#1.% +} + +% division by 2 + +\def\xdivbytwo#1{% + \romannumeral0\expandafter\xdivbytw@\romannumeral0\xr@verse{}#1..% +} + +\def\xdivbytw@#1#2{% + \if#2.% + \yybreak{\expandafter\space\number\csname xdivby2\endcsname#1}% + \else + \yybreak{\expandafter\xdivb@tw@\expandafter{\number\csname xdivby2\endcsname#1}#2}% + \yycontinue +} + +\def\xdivb@tw@#1{% + \xbytigid{5}{#1}{}% +} + +% increment by 1 + +\def\xincrement#1{% + \romannumeral0\expandafter\x@ncrement\expandafter{\romannumeral0\xr@verse{}#1.}% +} + +\def\x@ncrement#1{% + \x@ncr@ment{1}{}#1.% +} + +% #1 is the carry +% #2 is the result +% #3 is the next digit of the operand + +\def\x@ncr@ment#1#2#3{% + \if#3.% + \yybreak{\ifnum#1=\z@\xskiptofi{ #2}\else\xskiptofi{ #1#2}\fi}% + \else + \yybreak{\ifnum#1=\z@\xskiptofi{\x@@cr@m@nt{#3#2}}\else + \xskiptofi{\expandafter\x@ncr@m@nt\number1\csname xaddto#1\endcsname#3{#2}}\fi}% + \yycontinue +} + +\def\x@ncr@m@nt#1#2#3#4{% + \x@ncr@ment#2{#3#4}% +} + +\def\x@@cr@m@nt#1#2.{% + \xr@verse{}#2.#1% +} + +% decimal complements table + +\def\xcplof#1{\ifcase#1 \or9\or8\or7\or6\or5\or4\or3\or2\or1\fi} + +% single digit decrement by 1 table + +\def\xdecof#1{\ifcase#1 \or0\or1\or2\or3\or4\or5\or6\or7\or8\fi} + +% decrement by 1 + +\def\xdecrement#1{% + \romannumeral0\expandafter\xd@crement\expandafter{\romannumeral0\xr@verse{}#1.}% +} + +\def\xd@crement#1{% + \xd@cr@ment1{}#1.% +} + +% #1 is the carry +% #2 is the result +% #3 is the next digit of the operand + +\def\xd@cr@ment#1#2#3{% + \if#3.% + \yybreak{ -1}% + \else + \yybreak{\ifnum#1=\z@ + \xskiptofi{\xde@@@m@nt{#3#2}}% + \else + \xskiptofi{% + \ifnum#3=\z@ + \xskiptofi{\xd@cr@ment1{9#2}}% + \else + \xskiptofi{\expandafter\xde@@@m@nt\expandafter{\number\xdecof#3 #2}}% + \fi}% + \fi}% + \yycontinue +} + +\def\xd@cr@m@nt#1#2#3{% + \xd@cr@ment#1{#2#3}% +} + +\def\xde@@@m@nt#1#2.{% + \xr@verse{}#2.#1% +} diff --git a/support/splint/tex/yxunion.sty b/support/splint/tex/yxunion.sty new file mode 100644 index 0000000000..fb0d2460b3 --- /dev/null +++ b/support/splint/tex/yxunion.sty @@ -0,0 +1,33 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% the original, minimal bootstrapping macros were designed to process +% \prodstyle{\%token} declarations only and are enough to establish the +% interface between the \bison\ parser and the \bison\ lexer; to serve +% the secondary task of providing typesetting information to +% the \bison\ parser, all forms of token declarations must be processed. + +\restorecslist{bootstrap}\yyunion % get the original bootstrap macros + +\def\precdecls#1#2#3#4#5{#3} + +\def\symbolprec#1#2{% + \toksa{}\toksb{}% + #1% + \yytoksempty\toksb{}{\immediate\write\tokendefs{\noexpand\tokenpp{\the\toksb}}}% +} + +\savecslist{bootstrap}\yyunion diff --git a/support/splint/tex/yy.sty b/support/splint/tex/yy.sty new file mode 100644 index 0000000000..22a7948e9d --- /dev/null +++ b/support/splint/tex/yy.sty @@ -0,0 +1,76 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% macros for typeseting productions + +% these macros carefully orchestrate the startup of the parsing machinery, bootstrapping the parser if +% necessary. they can be used as an example of such macros when different use of the parsing mechanisms +% is desired. + +\ifx\optimization\UNDEFINED % this trick is based on the premise that \UNDEFINED is never defined nor + \def\optimization{0}% created with \csname ... \endcsname +\fi + +\newwrite\tokendefs +\let\nx\noexpand + +\def\drvname{bo} + +\input yycommon.sty % general routines for stack and array access +\input yymisc.sty % helper macros (stack manipulation, table processing, value stack pointers) +\input yyinput.sty % input functions +\input yyparse.sty % parser machinery +\input flex.sty % lexer functions +\input yyboth.sty % parser intitialization, optimization + +\ifnum\optimization>\tw@ + \input yyfaststack.sty +\fi + +\input yystype.sty % scanner auxilary types and functions +\input yyunion.sty % parser data structures + +\def\indexpseudonamespace{[index]} + +% the main parser + +\let\parsernamespace\empty + +\def\modebootstrap{% + \edef\tokendeffile{\drvname.tok}% + \edef\bstrapparser{byytab.tex}% sets \bootstrapmodetrue + \def\bootstraplexersetup{\let\yylexreturn\yylexreturnbootstrap}% only return tokens whose value is known at bootstrap + \input yybootstrap.sty% +} + +\def\modenormal{% + \def\appendr##1##2{% faster but dirtier + \edef\appnext{##1{\the##1##2}}\appnext + }% + \def\appendl##1##2{% faster but dirtier + \edef\appnext{##1{##2\the##1}}\appnext + }% + %\def\fgetelemof##1\at##2{% speeds up the lookup + % \csname ##1\parsernamespace\the##2\endcsname + %}% + \input yyinit.sty% +} + +\ifx\modeactive\UNDEFINED + \def\modeactive{\modenormal} +\fi + +\modeactive diff --git a/support/splint/tex/yybootstrap.sty b/support/splint/tex/yybootstrap.sty new file mode 100644 index 0000000000..185ad3b0a6 --- /dev/null +++ b/support/splint/tex/yybootstrap.sty @@ -0,0 +1,70 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% token equivalence table does not exist, switch to a bootstrap parser + +\ifx\tokendeffile\UNDEFINED +\else + \immediate\openout\tokendefs=\tokendeffile +\fi +\genericparser + name: main, + ptables: \bstrapparser, + ltables: cweb/ltab.tex, + tokens: {}, + asetup: \bootstraplexersetup, + dsetup: \newlexerstateextra, + rsetup: \noexpand\savefullstateextra, + optimization: \optimizeall;% + +\tomainparser % this saves the current state + +\input yytexlex.sty + +\let\unparse\eatone + +\expandafter\def\csname parserstack[b]\endcsname#1#2{% + \parsevb{#1}% Stage two, start the parsing + \ifyyparsefail % do nothing if parsing failed + \yybreak{}% + \else % Stage three, process the parsed table + \yybreak{\initbootstrap\the\table}% + \yycontinue +} + +\expandafter\def\csname parserstack[]\endcsname#1#2{% + \parsevb{#1}% Stage two, start the parsing + \ifyyparsefail % do nothing if parsing failed + \yybreak{}% + \else % Stage three, process the parsed table + \yybreak{\initbootstrap\the\table}% + \yycontinue +} + +% stage two macros: parsing + +\def\parsevb#1{% + %\ifchecktable {\toks0{#1}\errmessage{table before parsing: \the\toks0}}\fi + \tomainparser % this is not currently required since there are no namespace switching macros + % in the \TeX\ portion of the text; it is left here in case such macros are used in the future + \doparse{#1}% +} + +\long\def\beginprod#1\endprod{% + \endgroup +} + +\let\begincprod\beginprod diff --git a/support/splint/tex/yyboth.sty b/support/splint/tex/yyboth.sty new file mode 100644 index 0000000000..f15fe006c2 --- /dev/null +++ b/support/splint/tex/yyboth.sty @@ -0,0 +1,181 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% parser environment + +\newif\ifyyskipparse +\newif\ifsaveparseoutput +\newif\ifchecktable +\newif\ifchecktrim + +\newwrite\exampletable + +\def\basicparserinit{% + \yytext{}% + \yytextpure{}% + \yytextseenpure{}% + \yytextseen{}% + \yybyte{}% + \yyfbyte{}\yysbyte{}% + \yystash{}% + \yystashseen{}% + \yyformat{}% + \yyformatseen{}% + \yyfutureyytext{}% + \yyinitstack\yystatestack + \yyfmark=\z@ + \yysmark=\z@ + \yyfmarklast=\z@ + \formatmarker=\z@ + \yysmarklast=\z@ + \stashmarker=\z@ + \yytextbackupfalse + \yyg@yyinit=\z@ + \yyg@yystart=\z@ + \YYATBOL=\@ne + \yyparsefailfalse + \YYEOBLASTMATCHtrue +} + +\def\parserinit{% + \basicparserinit + \yyinitstack\obstackforstring + \yyinitstack\obstackforstringraw + \percentpercentcount=\z@ + \lonesting=\z@ + \laststring{}\laststringraw{}% + \currentlaststring{}\currentlaststringraw{}% + \parserdatainit +} + +\def\parserdatainit{% + % make control sequences inserted by the parser non expandable to facilitate + % token list manipulation; currently nothing is done +} + +% optimization options + +\def\optimizeall{% + % lexer + \ifnum\optimization>\z@ + \optimize{yynxt}% + \optimize{yyaccept}% + \optimize{yydef}% + \optimize{yychk}% + \optimize{yybase}% + \optimize{yyec}% + \optimize{yymeta}% + \tracingstats=\@ne + \fi + % parser + \ifnum\optimization>\@ne + \optimize{yytranslate}% + \optimize{yyrone}% + \optimize{yyrtwo}% + \optimize{yyrthree}% + \optimize{yydefact}% + \optimize{yydefgoto}% + \optimize{yypact}% + \optimize{yypgoto}% + \optimize{yytable}% + \optimize{yycheck}% + \optimize{yyprhs}% + \optimize{yyrhs}% + \optimize{yytoknum}% + \optimize{yystos}% + \optimizetext{yytname}% + \fi +} + +\long\def\endgsec#1#2#3{% + \ifyyskipparse + \yybreak{\unparse{#3}}% + \else + \yybreak{\csname parserstack[#2]\endcsname{#1}{#3}}% + \yycontinue +} + +\let\endcprod\endgroup +\let\endmprod\endgroup +\let\endprod\endgroup + +\def\genericparser name: #1, ptables: #2, ltables: #3, tokens: #4, asetup: #5, dsetup: #6, rsetup: #7, optimization: #8;{% + % parser initialization + % + \expandafter\def\csname #1namespace\endcsname{[#1]}% + \savecs{local-namespace}\parsernamespace + \expandafter\let\expandafter\parsernamespace\csname #1namespace\endcsname + \pinittoks{}% + \input #2 % load main parser table + \settokens % set the values of all tokens + \yystringempty{#4}{}{% + \input #4 % use token equivalence table to set the values of non-string tokens + }% + #5% additional setups + % + \input #3 % load lexer tables + % + % at this point the macros inside the table files (\newtable, \constset, + % \yybigswitch, \stashswitch, \addname, \yydoactionswitch, \setflexstates, + % \stateset, \tokeneq) have set up the corresponding stuctures in + % the `parser namespace' (e.g. if the parser namespace is `main', + % \newtable{yyaccept} created a token register \yyaccept[main]), + % assigned the `generic' names to them (to continue + % the example above, \newtable does \let\yyaccept\yyaccept[main]) and + % recorded the corresponding commands in \pinittoks for future use. + % + % lexer state macros are namespace specific (just like token names) + % so they have to be set in each namespace. + % + \setflexstates + #8% possible optimization + % + % finally, we add the definitions for the variables used in running + % the lexer and the parser. + \newparserstate + \newlexerstate + #6% additional data setup (say, \newlexerstateextra) + % + % we record all the commands necessary to switch to the desired namespace + % in a convenient macro + \expandafter\edef\csname to#1parser\endcsname{% + \noexpand\savefullstate % save the state of the current parser + #7% any data that needs to be saved + % switch to the new namespace + \let\noexpand\parsernamespace\expandafter\noexpand\csname #1namespace\endcsname + \the\pinittoks % restore all the tables, tokens and constants, and stacks + \let\noexpand\getcurrentparser\expandafter\noexpand\csname to#1parser\endcsname + }% + \restorecs{local-namespace}\parsernamespace +} + +\def\genericprettytokens namespace: #1, tokens: #2, correction: #3, host: #4;{% + \savecs{local-namespace}{\parsernamespace\tokeneq}% + \yystringempty{#2}{}{% + \expandafter\let\expandafter\parsernamespace\csname #1namespace\endcsname + \def\tokeneq##1##2{\prettytoken{##1}}% + \let\tokenpp\prettytoken + \input #2 % /* re-use token equivalence table to set the typesetting of tokens */ + }% + \yystringempty{#3}{}{% + \expandafter\let\expandafter\parsernamespace\csname #1namespace\endcsname + \input #3 % input customized typesetting rules for tokens + }% + \yystringempty{#4}{}{% + \expandafter\let\expandafter\hostparsernamespace\csname #4namespace\endcsname + }% + \restorecs{local-namespace}{\parsernamespace\tokeneq}% +} diff --git a/support/splint/tex/yycommon.sty b/support/splint/tex/yycommon.sty new file mode 100644 index 0000000000..5058fafe8a --- /dev/null +++ b/support/splint/tex/yycommon.sty @@ -0,0 +1,588 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\catcode`\@=11 + +% stacks will be defined as `list' macros, consisting of \sts{...}\sts{...}... type lists +% in \yypopstack, the second parameter must be a positive number +% +% note: perhaps, replacing \sts with the name of the stack would allow for more economical +% use of the user namespace, however, this will somewhat complicate the macros below, as well +% as make it impossible to assign a different control sequence to the +% stack (which may be considered a feature by itself) + +% note: a somewhat clumsy way in which the code below is written is due to the goal of making it +% independent of general use registers (\tempc?); the result is extremely slow code + +\def\yyinitstack#1{% to provide consistency with the accelerated macros + \let#1\empty +} + +\long\def\scoopupstack#1#2\stackend#3{\def#3{\sts{#1}#2}} + +\def\stackend#1{\def#1{}} % if we got here, the stack is empty + +% the following macro is a mild example of expansion tricks + +\def\yypopstack#1\by#2{% + \ifnum#2>\z@ + \yyp@pst@ck{#1}{#2}% + \fi +} + +\def\yyp@pstack#1{% + \expandafter\space#1% +} + +% the definition below is here purely for clarity + +\catcode`\.=\active +\let.\expandafter + +\def\yyp@pst@ck#1#2{% + \let\sts\or + \iffalse{\fi...\def...#1...{...\sts.\ifcase\number\number.\xincrement.{\number#2} \yyp@pstack#1}\else}\fi +} + +\catcode`\.=12 % other character + +% #1 is the name of the stack, #2 is a token register + +\def\yypop#1\into#2{\def\sts{\consumeone{#2}}#1\stackend#1} + +\long\def\consumeone#1#2{% + #1{#2}\let\sts\scoopupstack +} + +% pushing stuff on a stack: \yypush{t o k e n s}\on\yyvs or \expandafter\yypush\number\yystate\on\yyss + +\long\def\yypush#1\on#2{\toksa={#1}\expandafter\toksb\expandafter{#2}\edef#2{\noexpand\sts{\the\toksa}\the\toksb}} + +% push register contents on a stack: #1 is a register, #2 is a stack (a control +% sequence that expands to a `\sts{v a l u e}\sts...' list) + +\def\yypushr#1\on#2{\expandafter\toksa\expandafter{#2}\edef#2{\noexpand\sts{\the#1}\the\toksa}} + +% the first parameter is the stack, the second is the location from the top (a nonnegative number), the third is +% the control sequence that will hold the value; + +\def\yyreadstack#1\at#2\to#3{\edef\sts{\noexpand\skipandcount{\number#2}{\noexpand#3}}#1\stackfinish#1} + +\def\skipandcount#1#2#3{% + \ifnum#1=\z@ %we have got to the element we need + \def#2{#3}% + \yybreak\ignorestack + \else + \yybreak{\edef\sts{\noexpand\skipandcount{\xdecrement{#1}}{\noexpand#2}}}% + \yycontinue +} + +% same as above except read the value into a register + +\def\yyreadstackr#1\at#2\to#3{\edef\sts{\noexpand\skipandcountr{\number#2}{#3}}#1\stackfinish#1} + +\def\skipandcountr#1#2#3{% + \ifnum#1=\z@ %we have got to the element we need + #2#3% + \yybreak\ignorestack + \else + \yybreak{\edef\sts{\noexpand\skipandcountr{\xdecrement{#1}}{\noexpand#2}}}% + \yycontinue +} + +\long\def\ignorestack#1\stackfinish#2{} + +\def\stackfinish#1{\def#1{0\message{:stack empty:}}} + +\def\yyreadvstack#1\upto#2{% assume that #2 > 0 + \edef\sts{\noexpand\splitstack{\number#2}{\expandafter\xincrement\expandafter{\number\toptoks}}}#1\stackend#1% +} + +\long\def\splitstack#1#2#3{% + \expandafter\def\csname$'#1\endcsname{#3}% $ + \ifnum#2<\@cclvi % we have not reached the maximum allocated number of token registers + \expandafter\toksdef\csname$$'#1\endcsname=#2 + \toks#2{#3}% + \fi + \ifnum#1=\@ne %we have read the values + \let\sts\scoopupstack + \else + \edef\sts{\noexpand\splitstack{\xdecrement{#1}}{\xincrement{#2}}}% + \fi +} + +\def\yypeekvstack#1\upto#2{% assume #2 > 0 + \edef\sts{\noexpand\peelstack{\number#2}{\expandafter\xincrement\expandafter{\number\toptoks}}}#1\relax% +} + +\long\def\peelstack#1#2#3{% + \expandafter\def\csname$'#1\endcsname{#3}% $ + \ifnum#2<\@cclvi % we have not reached the maximum allocated number of token registers + \expandafter\toksdef\csname$$'#1\endcsname=#2 + \toks#2{#3}% + \fi + \ifnum#1=\@ne %we have read the values + \let\sts\eatone + \else + \edef\sts{\noexpand\peelstack{\xdecrement{#1}}{\xincrement{#2}}}% + \fi +} + +% macros to support new printing routines + +\def\yypeeksstack#1\upto#2\withprefix#3{% assume #2 > 0 + \edef\sts{\noexpand\peelsstack{\number#2}}% + \expandafter\def\expandafter\sts\expandafter{\sts{#3}{}}#1\relax% +} + +\long\def\peelsstack#1#2#3#4{% + \ifnum#1=\@ne %we have read the values + #3\let\sts\eatone + \else + \edef\sts{\noexpand\peelsstack{\xdecrement{#1}}}% + \expandafter\def\expandafter\sts\expandafter{\sts{#2}{#2{#4}#3}}% + \fi +} + +% token register access + +\def\concat#1#2{% store the concatenation result in the first sequence + #1\expandafter\expandafter\expandafter{\expandafter\the\expandafter#1\the#2}% +} + +\def\concatl#1#2{% store the concatenation result in the second sequence + #2\expandafter\expandafter\expandafter{\expandafter\the\expandafter#1\the#2}% +} + +\def\appendr#1#2{% + \begingroup + \edef\next{#1{\the#1#2}}\next + \tokreturn{}{}{#1{\the#1}}% +} + +\def\appendl#1#2{% + \begingroup + \edef\next{#1{#2\the#1}}\next + \tokreturn{}{}{#1{\the#1}}% +} + +% the following macros are an expandable way to determine if a token register is empty; +% while a number of different conditionals can be used, including plain \iffalse, +% this choice seems to result in a shortest macro and the fewest number of \expandafter's; +% an idea from +% http://tex.stackexchange.com/questions/2936/test-whether-token-list-is-empty +% where it is attributed to Ulrich Diez can be generalized to make multiple tests inside braces +% in a row; the macros from that discussion are quoted below; note, however, that these macros +% lead to unbalanced braces inside alignments (see The \TeX book, Appendix~D, p.~385 for the +% discussion of the `master counter' and the `balance counter' and their behavior when +% \TeX\ evaluates the constants `{ and `}); in addition, the first `1' is superfluous; + +%\newcommand\@ifempty@toks[1]{% +% \ifcase\iffalse{{{\fi\expandafter\@ifempty@@toks\the#1}1}1}0 +% \expandafter\@firstoftwo +% \else +% \expandafter\@secondoftwo +% \fi} +%\newcommand{\@ifempty@@toks} +% {\expandafter\@gobble\expandafter{\expandafter{% +% \ifcase`}\expandafter}\expandafter\fi\string} + +% as a note of explanation, the reason this works relies on the fact +% that \string will turn a `{', a `}', or any other token into a +% non-brace while the parameter scanning mechanism of \TeX\ will try +% to collect the smallest possible balanced input; the `excessive' +% braces will disappear in the expansion of the `\if...' construct; +% the reason \if or other macros that expand their argumens are so well suited for this +% `chain expansion' mechanism is in the fact that the expansions for \string and \if... are launched +% from the same point. + +\long\def\yytoksempty#1{% + \iffalse{{\fi + \if{\expandafter\yytoks@mpty\the#1}}}% + \yybreak\yyfirstoftwo + \else + \yybreak\yysecondoftwo + \yycontinue +} + +% when the token list is empty, \TeX\ will try to expand \yybreak premaurely; +% in this case \yybreak forces a \fi to be expanded while skipping the rest; +% note that a simple \expandafter would not work in this case as \TeX would +% insert a \relax when trying to expand a premature \else (this can be only +% gleaned from `\TeX\ the program') + +\long\def\yystringempty#1{% + \iffalse{{\fi + \if{\yytoks@mpty#1}}}% + \yybreak\yyfirstoftwo + \else + \yybreak\yysecondoftwo + \yycontinue +} + +\catcode`\>=2 + +\def\yytoks@mpty{% + \expandafter\eatone\expandafter{\expandafter{% + \if}\expandafter>\expandafter\fi\string +} + +\catcode`\>=12 + +\long\def\yystartsinspace#1{% is the first token a \charcode 32, \catcode 10 token? + \iffalse{\fi\yystartsinspac@#1 }% +} + +\long\def\yystartsinspac@#1 {% + \yystringempty{#1}% + {\expandafter\yysecondofthree\expandafter{\string}}% + {\expandafter\yythirdofthree\expandafter{\string}}% +} + +% the macros below are a derivation of David Kastrup's magnificent string comparison +% macros below: +% \def\strequal#1{\number\strequalstart{}{}#1\relax} +% \def\strequalstart#1#2#3{\if#3\relax\strequalstop\fi +% \strequalstart{\if#3#1}{#2\fi}} +% \def\strequalstop\fi\strequalstart#1#2#3{\fi#1#3\relax'#213 } +% +% use: \if\strequal{string}{string}... +% +% they were adjusted to handle spaces in the strings and conform to a different +% syntax, namely \yyifsamestring{string1}{string2}{true}{false} +% the original macros use the fact that, say \if1\fi will expand to nothing and +% that \number'13 expands to 11 whereas \number13 expands to 13; the elegance of +% the second test was lost due to a different syntax; + +\edef\yyifsamestring#1{\noexpand\yyifsamestr@ng{}{}#1 \noexpand\yyifsam@str@ng\space} +\def\yyifsamestr@ng#1#2#3 {\ifx\yyifsam@str@ng#3\yyifsam@str@ng\fi + \yyifs@m@str@ng{#1}{#2}#3\space} + +\def\yyifs@m@str@ng#1#2#3{% + \if#3\space + \expandafter\yyifsamestr@ng + \else + \expandafter\yyifs@m@str@ng + \fi + {\if#3#1}{#2\fi}% +} + +\def\yyifsam@str@ng\fi\yyifs@m@str@ng#1#2\yyifsam@str@ng\space#3{\fi + \if\noexpand\yyifsam@str@ng#1#3 \noexpand\yyifsam@str@ng\yystrcleanup#2\fi + \yysecondoftwo +} + +\def\yystrcleanup#1\yysecondoftwo{#1\yyfirstoftwo} + +% a `self-propagating \expandafter'; allows building lists like \yysx a\yysx b ... +% so that a \romannumeral-1 at the beginning of the list would cary the expansion +% to the last token but leave the list intact; note that #1 should be a single token + +\def\yysx#1#2{% + \expandafter\space\expandafter\yysx\expandafter#1\romannumeral-1#2% +} + +% the macro below can be simplified by reducing the number of braces +% but then \yytoks@mpty could not be reused + +\long\def\yystartsinbrace#1{% + \iffalse{\fi + \if{\yytoks@mpty#1}}% + \yybreak\yysecondoftwo + \else + \yybreak\yyfirstoftwo + \yycontinue +} + +% a test to determine whether the argument is a given control sequence + +\long\def\yyisthiscs#1#2{% + \yystringempty{#1}{\yysecondoftwo}{% + \yystartsinspace{#1}{\yysecondoftwo}{% + \yystartsinbrace{#1}{\yysecondoftwo}{% + \expandafter\yystringempty\expandafter{\eatone#1}{% + \expandafter\yyisth@scs\expandafter{\string#1}{#2}% + }{\yysecondoftwo}% + } + } + }% +} + +\long\def\yyisth@scs#1#2{% + \expandafter\yyifsamestring\expandafter{\string#2}{#1}% +} + +% same as above but the argument is a token register + +\def\yyisthiscsr#1{% + \expandafter\yyisthiscs\expandafter{\the#1}% +} + +\long\def\yyfirstoftwo#1#2{#1} +\long\def\yysecondoftwo#1#2{#2} +\long\def\yysecondofthree#1#2#3{#2} +\long\def\yythirdofthree#1#2#3{#3} + +% arrays of integers are going to be represented by a string of tokens `element0 \or element1 \or ...' +% #2 is a register (otherwise the case and the integer from the array `coalesce'); +% the following macro was designed to make something like +% \vara=\getelemof\yytable\at\yyn\relax possible so it has to expand to a number; +% incidentally, some care should be taken using the above asignment to make sure that +% it is not followed by an expandable token (such as \if) as in this case the token might be +% expanded prematurely, as the assignment is looking for the first non-expandable token which +% is not part of the number; this is the reason for the \relax + +\def\getelemof#1\at#2{% the original meaning of this macro + \ifcase\expandafter#2\the#1\else\fi +} + +\def\getelemof#1\at#2{% no longer limited to registers for #2 + \expandafter\get@lemof\expandafter{\the#1}{#2}% +} + +\def\get@lemof#1#2{% + \ifcase#2 #1\else\fi +} + +\def\fastgetelemof#1\at#2{% + \csname #1\parsernamespace\number#2\endcsname +} + +\def\fgetelemof#1\at#2{% + \expandafter\ifx\csname optopt[#1]\parsernamespace\endcsname\relax + \expandafter\getelemof\csname #1\endcsname\at{#2}% + \else + \csname #1\parsernamespace\number#2\endcsname + \fi +} + +% a nestable loop + +\def\bloop#1\repeat{#1\bloop{#1}\repeat\fi} + +% optimization macros: currently, the level of optimization has to be consistent throughout the +% document, i.e. \optimize macros have to be called on the same arrays after loading. +% the reason is the yyfaststack.sty file that modifies the \newtable macro once for all the tables + +\def\optimize#1{% + \setoptopt{#1}% + \tempca\z@ + \bloop + \tempcb=\expandafter\ifcase\expandafter\tempca\the\csname#1\endcsname\else\@MM\fi\relax + \ifnum\tempcb<\@MM % + \expandafter\edef\csname #1\parsernamespace\the\tempca\endcsname{\the\tempcb}% + \advance\tempca\@ne + \repeat +} + +\def\optimizetext#1{% optimizing text arrays + \setoptopt{#1}% + \tempca\z@ + \@ptimizetext{#1} +} + +\def\@ptimizetext#1{% + \edef\next{\expandafter\ifcase\expandafter\tempca\the\csname#1\endcsname\else\end\fi}% + \ifx\next\endcontainer + \let\next\eatone + \else + \expandafter\edef\csname #1\parsernamespace\the\tempca\endcsname{\next}% + \advance\tempca\@ne + \let\next\@ptimizetext + \fi + \next{#1}% +} + +\def\uoptimize#1{% same as the macro above but produces nonnegative constants as \mathchardef's + \setoptopt{#1}% + \tempca\z@ + \bloop + \tempcb=\expandafter\ifcase\expandafter\tempca\the\csname#1\endcsname\else\@MM\fi\relax + \ifnum\tempcb<\@MM % + \toksa\expandafter{\csname #1\parsernamespace\the\tempca\endcsname}% + \edef\next{\mathchardef\the\toksa=\the\tempcb\relax}\next + \advance\tempca\@ne + \repeat +} + +\def\setoptopt#1{% + \expandafter\let\csname optopt[#1]\parsernamespace\endcsname\end +} + +\countdef\toptoks=15 % register responsible for token allocations + +% returning token register values from a group +% in such a way that no other register values are affected + +\def\tokreturn#1#2#3{% #1 is the code to be prepended (will be expanded) + % #2 is a list of token registers + % #3 is the code to be appended + \t@kreturn{#1}{#3}#2\end +} + +\def\t@kreturn#1#2#3{% first step: see if the list is non-empty and pick the first token register + \ifx#3\end % there are no registers to return so \toksa can be used as temporary storage + % (on exiting the current \begingroup its value will be restored to what it was + % before the group was entered) + \edef\next{\toksa{#1#2}}\next % return prepended and appended code + \def\t@kreturn{\expandafter\endgroup\the\toksa}% + \else + \edef\tokreturn{#3{{#2}#1#3{\the#3}}}\tokreturn + \let\tokreturn#3% + \let\t@kreturn\t@kr@turn + \fi + \t@kreturn % this sequence will be restored to its original value when the group is exited +} + +\def\t@kr@turn#1{% + \ifx#1\end + \def\t@kreturn##1##2\end{\tokreturn{##2##1}}% + \expandafter\t@kreturn\the\tokreturn\end + \def\t@kreturn{\expandafter\endgroup\the\tokreturn}% + \else + \edef\t@kreturn{\tokreturn{\the\tokreturn#1{\the#1}}}\t@kreturn + \let\t@kreturn\t@kr@turn + \fi + \t@kreturn +} + +% switch macros, also used to implement state machines +% a lot of care has been taken to ensure that no control sequence is changed +% as well as all the register values are preserved. + +\newif\iftracedfa + +\def\taction#1\in#2{% + \begingroup + \edef\acstring{#1}% in case #1 is, say, \the\toksa, so we no longer have to keep track of it + \iftracedfa\derrmessage{acting on <\meaning\acstring>\space in (\string#2) \getstatename#2 }\fi + \toksa\expandafter{#2}\toksb\expandafter{\acstring}% + \edef\next{\toksa{\the\toksa\the\toksb{% + \iftracedfa\noexpand\derrmessage{default action: \noexpand\meaning\noexpand\default}\fi + \noexpand\default}}% + \def\noexpand\next####1\the\toksb####2####{\noexpand\grabaction}}\next + \expandafter\next\the\toksa\grabaction + \tokreturn{}{}{\the\toksa}% +} + +\def\tactionx#1\in#2{% exclusive version of the macro above (i.e. match the last action before the brace) + \begingroup + \edef\acstring{#1}% in case #1 is, say, \the\toksa, so we no longer have to keep track of it + \iftracedfa\errmessage{acting on <\meaning\acstring>\space in (\string#2) \getstatename#2 }\fi + \toksa\expandafter{#2}\toksb\expandafter{\acstring}% + \edef\next{\toksa{\the\toksa\the\toksb{% + \iftracedfa\noexpand\derrmessage{default action: \noexpand\meaning\noexpand\default}\fi + \noexpand\default}}% + \def\noexpand\next####1\the\toksb####{\noexpand\grabaction}}\next + \expandafter\next\the\toksa\grabaction + \tokreturn{}{}{\the\toksa}% +} + +\def\getstatename#1{\expandafter\g@tstatename#1.\raw} + +\def\g@tstatename#1#2\raw{\expandafter\eatone\string#1} + +\def\caction#1\in#2{% + \begingroup + \uccode`.=#1\relax + \uppercase{\toksa{\taction{.}\in}}% + \toksb{#2}\concat\toksa\toksb + \tokreturn{}{}{\the\toksa}% +} + +\def\checkforcount#1{% a rough implementation of `type checking' for a parameter + \expandafter\expandafter\expandafter + \ch@ckforcount\expandafter\meaning\expandafter#1\meaning\count\end +} + +\expandafter\def\expandafter\ch@ckforcount\expandafter#\expandafter1\meaning\count#2\end{% + \yystringempty{#2}{\toksa{\taction}}{\toksa{\caction}}% +} + +\def\action#1\in#2{% + \begingroup + \checkforcount#1% + \toksb{{#1}\in{#2}}\concat\toksa\toksb + \tokreturn{}{}{\the\toksa}% +}% + +\let\switchon\taction +\let\default\relax + +\def\grabaction#1#2\grabaction{\toksa{#1}} + +% grab the first token unless it is a space or a brace + +\def\getfirsttoken#1{% + \yystartsinbrace{#1}{ }{\yystartsinspace{#1}{ }{% + \expandafter\g@tfirsttoken\string{#1} % terminate \romannumeral + }}% +} + +\def\g@tfirsttoken#1#2{% + \expandafter\noexpand\expandafter#2\romannumeral0\expandafter\eatone\expandafter{\string}% +} + +% macros for `breaking out of' conditionals: +% the idea is probably folklore; +% \yybreak ... \yycontinue are the most efficient as they read everything exactly once +% and expand only what is necessary; the next preferred way is the \xskip ... series +% the \yyfinish macro is here `to plug a hole' when it is stylistically preferable +% to keep the existing conditional structure and efficiency is not that important + +\long\def\xskiptofi#1#2\fi{\fi#1} +\long\def\xskiptofifi#1#2\fi\fi{\fi\fi#1} +\long\def\xskiptofififi#1#2\fi\fi\fi{\fi\fi\fi#1} + +\long\def\yyfinish#1#2\yycontinue{#2#1}% here just for completeness, use the ones below instead +\long\def\yybreak#1#2\yycontinue{\fi#1} +\long\def\yybreak@#1#2\yycontinue{\fi\fi#1} +\long\def\yybreak@@#1#2\yycontinue{\fi\fi\fi#1} +\long\def\yybreak@@@#1#2\yycontinue{\fi\fi\fi\fi#1} +\long\def\yybreak@@@@#1#2\yycontinue{\fi\fi\fi\fi\fi#1} + +% we intentionally leave \yycontinue undefined since it should not be expanded normally +% every conditional that uses \yybreak?{...} ... \yycontinue construct +% must have an \else clause, i.e.\ a conditional such as +% \if ab +% \yybreak{}% +% \yycontinue +% is a bad idea as it will result in an incomplete \iffalse +%\let\yycontinue\fi + +% macros for taking care of extra tokens + +\long\def\yyid#1{#1} +\long\def\yypione#1#2{#1} +\long\def\yypitwo#1#2{#2} +\long\def\yyswap#1#2{#2#1} +\long\def\eatone#1{} +\long\def\eattwo#1#2{} +\long\def\eattoend#1\end{} + +\input xarithm.sty + +% temporaries + +\input trt1.sty + +% \tempcd used by \printrule and implicit rule name macros in yymisc.sty +% \tempce used by implicit rule name macros in yymisc.sty + +% \tokse and \toksf so far only used in the bison action for +% \codepropstype + +\newif\ifbootstrapmode diff --git a/support/splint/tex/yyfaststack.sty b/support/splint/tex/yyfaststack.sty new file mode 100644 index 0000000000..cc43c33115 --- /dev/null +++ b/support/splint/tex/yyfaststack.sty @@ -0,0 +1,352 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% fast versions of stack access functions + +\catcode`\@=11 + +\def\yyinitstack#1{% #1 is the counter + \csname \expandafter\eatone\string#1<count>\endcsname\m@ne +} + +% stacks will be defined as pairs ( \s t a c k n a m e [n u m b e r], \s t a c k n a m e <count> ) +% where \s t a c k n a m e [n u m b e r] is a control sequence and \s t a c k n a m e <count> +% is a \count register +% +% note that in the implementaion below the stack grows `the wrong +% way'; this is done for convenience, since there is no `memory read +% with increment' operation, the `usual' (say, hardware) stack +% implementations take advantage of, and our stack can grow +% unrestricted this way + +\def\getstackpointer#1{% expands to the value of the current top of the stack position + \expandafter\the\csname \expandafter\eatone\string#1<count>\endcsname +} + +\def\gettopofstackcs#1{% expands to the control sequence (a member of the associative array) + % that holds the value of the current element at the top of the stack + \csname \expandafter\eatone\string#1[\getstackpointer#1]\parsernamespace\endcsname +} + +\def\getmidstackcs#1#2{% expands to the control sequence (a member of the associative array) + % that holds the value of the current element at the specified (#2) position + \csname \expandafter\eatone\string#1[#2]\parsernamespace\endcsname +} + +\def\movestackpointer#1\by#2{% + \expandafter\advance\csname \expandafter\eatone\string#1<count>\endcsname#2% +} + +\def\yypopstack#1\by#2{% removes the top #2 elements from the top of the stack #1 + \movestackpointer#1\by{-#2}\relax +} + +% + +\def\yypop#1\into#2{% + #2\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter{\gettopofstackcs#1}% + \movestackpointer#1\by\m@ne +} + +% pushing stuff on a stack: \yypush{t o k e n s}\on\yyvs or \expandafter\yypush\number\yystate\on\yyss + +\long\def\yypush#1\on#2{% + \movestackpointer#2\by\@ne + \expandafter\expandafter\expandafter\def\gettopofstackcs#2{#1}% +} + +% push register contents on a stack: #1 is a register, #2 is a stack + +\def\yypushr#1\on#2{% + \movestackpointer#2\by\@ne + \expandafter\expandafter\expandafter\edef\gettopofstackcs#2{\the#1}% +} + +% the first parameter is the stack, the second is the location from the top (a nonnegative number), the third is +% the control sequence that will hold the value; + +\def\yyreadstack#1\at#2\to#3{% + \expandafter\advance\csname \expandafter\eatone\string#1<count>\endcsname-#2\relax + % + \expandafter\let\expandafter#3\csname \expandafter\eatone\string#1[\expandafter\the + \csname \expandafter\eatone\string#1<count>\endcsname]\parsernamespace\endcsname + % + \expandafter\advance\csname \expandafter\eatone\string#1<count>\endcsname#2\relax +} + +% same as above but reads the value into a register + +\def\yyreadstackr#1\at#2\to#3{% + \expandafter\advance\csname \expandafter\eatone\string#1<count>\endcsname-#2\relax + % + #3\csname \expandafter\eatone\string#1[\expandafter\the + \csname \expandafter\eatone\string#1<count>\endcsname]\parsernamespace\endcsname\relax + % + \expandafter\advance\csname \expandafter\eatone\string#1<count>\endcsname#2\relax +} + +% the macros below use general purpose registers and have a chance to interfere with +% the user namespace; they have been replaced by the macros that only use expandable +% arithmetic which leads to about 3 percent slowdown. thus the original macros were left +% in the package in case such a slowdown is undesirable + +\def\yyreadvstack#1\upto#2{% assume that #2 > 0 + \tempca=#2\relax + \tempcb\toptoks + \advance\tempcb\@ne + \yyreadvst@ck{#1}% +} + +\def\yyreadvst@ck#1{% + \expandafter\toksdef\csname$$'\the\tempca\endcsname=\tempcb% + \expandafter\yypop\expandafter#1\expandafter\into\csname$$'\the\tempca\endcsname%$ + \expandafter\expandafter\expandafter\def + \expandafter\expandafter\csname$'\the\tempca\endcsname\expandafter{\the\toks\tempcb}%$ + \ifnum\tempca=\@ne %we have read the values + \yybreak\relax + \else + \advance\tempcb\@ne + \advance\tempca\m@ne + \yybreak{\yyreadvst@ck{#1}}% + \yycontinue +} + +% the macros below replace the two macros above to only use expandable arithmetic + +\def\yyreadvstack#1\upto#2{% assume that #2 > 0 + \expandafter\yyr@@dvst@ck\expandafter{\number#2}{#1}% +} + +\def\yyr@@dvst@ck#1#2{% + \expandafter\yyr@@@vst@@k\expandafter{\number\toptoks}{#1}{#2}% +} + +\def\yyr@@@vst@@k#1#2#3{% + \yyr@@dvst@@k{#2}{#1}{#3}% +} + +\def\yyreadvst@ck#1#2#3{% + \expandafter\toksdef\csname$$'#2\endcsname=#1\relax % + \expandafter\yypop\expandafter#3\expandafter\into\csname$$'#2\endcsname%$ + % the portion below is redundant if the \yn macro below is used and no symbolic switch + % is implemented + \expandafter\expandafter\expandafter\def + \expandafter\expandafter\csname$'#2\endcsname\expandafter{\the\toks#1}%$ + \ifnum#2=\@ne %we have read the values + \yybreak\relax + \else + \yybreak{\expandafter\yyr@@dvst@@k\expandafter{\number\xdecrement{#2}}{#1}{#3}}% + \yycontinue +} + +\def\yyr@@dvst@@k#1#2#3{% + \expandafter\yyreadvst@ck\expandafter{\number\xincrement{#2}}{#1}{#3}% +} + +% end of replacement macros + +\def\yypeekvstack#1\upto#2{% assume that #2 > 0 + \yyreadvstack#1\upto{#2}\relax + \movestackpointer#1\by#2\relax +} + +% macros for reading the state stack (needed if using bison version 3.0 and above) + +\def\yyreadsstack#1\upto#2\withprefix#3{% assume that #2 > 0 + \expandafter\yyr@@dsst@ck\expandafter{\number#2}{#1}{#3}% +} + +\def\yyr@@dsst@ck#1#2#3{% + \expandafter\yyr@@@sst@@k\expandafter{\number\toptoks}{#1}{#2}{#3}% +} + +\def\yyr@@@sst@@k#1#2#3#4{% + \yyr@@dsst@@k{#2}{#1}{#3}{#4}{}% +} + +\def\yyr@@dsst@@k#1#2#3#4#5{% + \expandafter\yyreadsst@ck\expandafter{\number\xincrement{#2}}{#1}{#3}{#4}{#5}% +} + +\def\yyreadsst@ck#1#2#3#4#5{% + \ifnum#2=\z@ %we have read the values + \yybreak{#5}% + \else + \yybreak{\expandafter\yyr@@dsst@@@\expandafter{\number\gettopofstackcs#3}{#1}{#2}{#3}{#4}{#5}}% + \yycontinue +} + +\def\yyr@@dsst@@@#1#2#3#4#5#6{% + \movestackpointer#4\by\m@ne + \expandafter\yyr@@dsst@@k\expandafter{\number\xdecrement{#3}}{#2}{#4}{#5}{#5{#1}#6}% +} + +\def\yypeeksstack#1\upto#2\withprefix#3{% assume that #2 > 0 + \yyreadsstack#1\upto{#2}\withprefix{#3}\relax + \movestackpointer#1\by#2\relax +} + +% new stack access macros that read the stack directly (to reduce namespace pollution) +% these are here more as an example + +\def\yn#1#{% + \ifnum#1<\@ne + \yybreak{\putyyval}% + \else + \yybreak{\p@tyyassignment{#1}}% + \yycontinue +} + +\def\p@tyyassignment#1{% + \expandafter\advance\csname yyvsa<count>\endcsname-\yylen + \expandafter\advance\csname yyvsa<count>\endcsname#1\relax + % + \expandafter\expandafter\expandafter + \p@@yyassignment + \expandafter\expandafter\expandafter{% + \csname yyvsa[\expandafter\the + \csname yyvsa<count>\endcsname]\parsernamespace\endcsname + }{% + \expandafter\advance\csname yyvsa<count>\endcsname-#1\relax + \expandafter\advance\csname yyvsa<count>\endcsname\yylen + }% +} + +% \bb macro implementation + +\def\p@twwassignment#1{% + \expandafter\advance\csname yyvsa<count>\endcsname-#1\relax + \expandafter\advance\csname yyvsa<count>\endcsname\@ne + % + \expandafter\expandafter\expandafter + \p@@yyassignment + \expandafter\expandafter\expandafter{% + \csname yyvsa[\expandafter\the + \csname yyvsa<count>\endcsname]\parsernamespace\endcsname + }{% + \expandafter\advance\csname yyvsa<count>\endcsname#1\relax + \expandafter\advance\csname yyvsa<count>\endcsname\m@ne + }% +} + +\def\p@@yyassignment#1#2#3{% + #2\yystringempty{#3}{#1}{#3{#1}}% +} + +\def\putyyval#1{\edef\next{\yyval{#1}}\next} + +% dysplaying stack contents (\yyparse specific) + +\def\showstack#1{% + \begingroup + \let\stackcs\empty + \count\z@\csname \expandafter\eatone\string#1<count>\endcsname + \ifnum\count\z@<\z@ + \else + \bloop + \count\@ne\csname \expandafter\eatone\string#1[\the\count\z@]\parsernamespace\endcsname + \count\tw@=\fgetelemof{yystos}\at{\count\@ne}% + \edef\stackcs{\stackcs^^J \space\space\fgetelemof{yytname}\at{\count\tw@}}% + \ifnum\count\z@>\z@ + \advance\count\z@\m@ne + \repeat + \fi + \tokreturn{}{}{\def\noexpand\stackcs{\stackcs}}% +} + +% saving token registers + +\def\newtable@generic#1{% + \expandafter\ifx\csname #1\endcsname\relax + \toksa{\csname newtoks\endcsname}% + \expandafter\the\expandafter\toksa\csname #1\endcsname + \fi + \csname #1\endcsname=% +} + +\let\newtable\newtable@generic % TODO: right now this is done either for all tables or none of them + +% macros to support `hardwired' optimized tables + +\def\appendtoyy@generic#1{% + \toksa{\or}% + \expandafter\ifx\csname optopt[#1]\parsernamespace\endcsname\relax + \expandafter\concat\csname#1\endcsname\toksb + \expandafter\concat\csname#1\endcsname\toksa + \else + \expandafter\edef\csname #1\parsernamespace\the\tempca\endcsname{\the\toksb}% + \fi +} + +\def\appendtoyy@genericlast#1{% + \expandafter\ifx\csname optopt[#1]\parsernamespace\endcsname\relax + \expandafter\concat\csname#1\endcsname\toksb + \else + \expandafter\edef\csname #1\parsernamespace\the\tempca\endcsname{\the\toksb}% + \fi +} + +\def\beginoptimizednonnumeric#1{% + \tempca\z@ + \def\appendtoyytname{\appendtoyy@generic{#1}}% + \def\appendtoyytnamelast{\appendtoyy@genericlast{#1}}% +} + +% list macros + +\def\listx#1{% #1 is the list name + \expandafter\ifx\csname #1[\the\csname#1<count>\endcsname]\endcsname\relax + \csname#1<count>\endcsname\z@ + \yybreak\relax + \else + \yybreak{\csname #1[\the\csname#1<count>\endcsname]\endcsname + \expandafter\advance\csname#1<count>\endcsname\@ne + \listx{#1}}% + \yycontinue +} + +\def\newlist#1{% + \toksa{\csname newcount\endcsname}% + \expandafter\the\expandafter\toksa\csname#1<count>\endcsname +} + +\def\initlistx#1{% + \csname#1<count>\endcsname\z@ + \expandafter\let\csname #10\endcsname\relax +} + +\def\resetlistx#1{% + \csname#1<count>\endcsname\z@ +} + +\def\listxlastindex#1{% + \the\csname#1<count>\endcsname +} + +\def\addtolist#1#2{% + \expandafter\def\csname#1[\the\csname#1<count>\endcsname]\endcsname{#2}% + \expandafter\advance\csname#1<count>\endcsname\@ne +} + +\def\addxtolist#1#2{% + \expandafter\edef\csname#1[\the\csname#1<count>\endcsname]\endcsname{#2}% + \expandafter\advance\csname#1<count>\endcsname\@ne +} + +\def\finishlist#1{% + \expandafter\advance\csname#1<count>\endcsname\@ne + \expandafter\let\csname#1[\the\csname#1<count>\endcsname]\endcsname\relax +} diff --git a/support/splint/tex/yyinit.sty b/support/splint/tex/yyinit.sty new file mode 100644 index 0000000000..8d9b7d33c3 --- /dev/null +++ b/support/splint/tex/yyinit.sty @@ -0,0 +1,682 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% finish parser initialization + +\let\yylexreturn\yylexreturnregular + +\genericparser + name: main, + ptables: cweb/gyytab.tex, + ltables: cweb/ltab.tex, + tokens: \drvname.tok, + asetup: {}, + dsetup: \newlexerstateextra, + rsetup: \noexpand\savefullstateextra, + optimization: \optimizeall;% + +% prologue parser + +\genericparser + name: prologue, + ptables: cweb/dyytab.tex, + ltables: cweb/ltab.tex, + tokens: \drvname.tok, + asetup: {}, + dsetup: \newlexerstateextra, + rsetup: \noexpand\savefullstateextra, + optimization: \optimizeall;% + +% parser for term names: this is not really a great idea in itself but rather an +% illustration of what is possible + +\genericparser + name: small, + ptables: cweb/small_tab.tex, + ltables: cweb/small_dfa.tex, + tokens: {}, + asetup: {}, + dsetup: {}, + rsetup: \noexpand\savefullstateextra, + optimization: \optimizeall;% + +\tomainparser + +\input yytexlex.sty + +\let\unparse\yyid + +\expandafter\def\csname parserstack[b]\endcsname#1#2{% + \parsevb{#1}% Stage two, start the parsing + \ifyyparsefail % revert to generic macros if parsing failed + \yybreak{\message{parsing failed ...}#2}% + \else % Stage three, process the parsed table + \yybreak{\typesetalltables}% + \yycontinue +} + +\expandafter\def\csname parserstack[]\endcsname#1#2{% + \parsevb{#1}% Stage two, start the parsing + \ifyyparsefail % revert to generic macros if parsing failed + \yybreak{\message{parsing failed ...}#2}% + \else % Stage three, process the parsed table + \yybreak{\typesetalltables}% + \yycontinue +} + +% stage two macros: parsing + +\newtoks\symstream + +\def\parsevb#1{% + \ifchecktable + \ifsaveparseoutput + {\toks0{#1}\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment + table before parsing:^^J\the\toks0}}% + \else + {\toks0{#1}\errmessage{table before parsing: \the\toks0}}% + \fi + \fi + \tomainparser + \doparse{#1}% + \ifyyparsefail + \toprologueparser + \doparse{#1}% + \ifyyparsefail + \else + \ifsaveparseoutput{\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment parsed table (prologue): + ^^J\the\table}}% + \fi + \fi + \else + \ifsaveparseoutput{\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment parsed table (grammar): + ^^J\the\table}}% + \fi + \fi +} + +% stage three macros: typesetting + +\def\tlskip{\z@} +\def\tfskip{\parindent} + +\def\typesetalltables{% + \begingroup + \ifchecktable + \ifsaveparseoutput + {\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment + stashed stream:^^J\the\yystash}}% + \else + \errmessage{parsed table: \the\table^^J^^Jstashed stream: \the\yystash}% + \fi + \fi + \symstream\table + \table{}% + \setprodtable + \the\symstream\relax + \postoks{}\pushothertables + \ifchecktable + \ifsaveparseoutput + \else + \errmessage{table after processing: \the\table}% + \fi + \fi + \ifsaveparseoutput + {\newlinechar=`^^J\immediate\write\exampletable{^^J\harmlesscomment + processed table:^^J\the\table}}% + \fi + \parindent1em + \checkforpropertable\table + \tabskip\tfskip + \ruletableset + \ifchecktable + \message{remaining stash: \the\yystash}% + \fi + \toksa{}\the\yystash + \cleanstash\stripstash\checkforccode + \ifchecktable + \message{stash after cleaning: \the\toksa}% + \fi + \ifnum\wd0>\z@ + %\ifchecktable + % \showboxdepth=1000 + % \showboxbreadth=1000 + % \showbox0 + %\fi + % currently testing for nontrivial leftover stash involves packaging the stash material + % into a \vbox; as a result, the stash containing ${}{}$\hbox{} will have a nonzero length + % which is why the test below is necessary + \ifnum\ht0>\z@ + \indent\boxstash + \fi + \fi + \endgroup +} + +% setting the rule table: cross-section alignment and other effects are applied here; +% in order to produce the proper line skips before and after \unvbox, the rules followed by +% \TeX\ while adding an \halign to a vertical list have to be reproduced explicitly + +\def\ruletableset{% + \par + \vskip-\baselineskip + \ifx\gatoks\relax + \else + \table\expandafter{\the\table\gatoks}% + \fi + \setbox0\vbox\expandafter{\expandafter + \null\expandafter\prevdepth\the\prevdepth + \halign \ifpropertable to \hsize \fi + {\hbox to 2em{##\/$\,${\rm:}\hss}\hfil\tabskip\z@&\setallterms{##}&##\hfil\tabskip0pt plus1fil&% + \toksa{}##\makestashbox\hfil\tabskip\tlskip\cr + \the\table + } + \expandafter + }\expandafter + \unvbox\expandafter0\expandafter + \prevdepth\the\prevdepth\relax +} + +\def\setallterms#1{\setbox\z@=\hbox{\it#1}\ifsquashterms\hbox to0pt{\unhbox\z@\hss}\else\unhbox\z@\fi\hfil} + +% quick and dirty global alignment: the size of the last box (and those in between) +% can be chosen automatically after one pass and read in for the final pass; +% in the future this will be the default implementation; for now, the inelegant +% solution below works as well. + +\def\setglobalalignrules#1{% + \def\gatoks{% + \omit\hfil&\omit\hfil&\omit\hfil&\omit\hfil\hbox to #1{\hfil}\cr + \noalign{\vskip-\baselineskip}% + %\noalign{\centerline{$\diamond$}} + }% +} + +\setglobalalignrules{3 in}% + +% to align all the actions across the document, comment out the next line + +\let\gatoks\relax + +\def\checkforccode{% + \setbox0=\vbox{\setlazyc\the\toksa}% +} + +\def\setlazyc{% + \hidecs{\1\4\5\6\8}% +} + +\newif\ifdisplaytokenraw + +\def\toksdefline#1#2#3#4#5{% #1 is the parsed version of the internal bison name + % #2 is the index entry + % #3 is the type + % #4 is the explicit value + % #5 is the parsed version of the string value + \ifnum\tempca=\z@ + \tokdectoks\expandafter{\the\tokdectoks&{#2}#1\hfil\cr}% + \tempca\tempcb + \else + \ifnum\tempcb=\tempca + \tokdectoks\expandafter{\the\tokdectoks{#2}#1\hfil}% + \else + \tokdectoks\expandafter{\the\tokdectoks&{#2}#1\hfil}% + \fi + \advance\tempca\m@ne + \fi +} + +\def\attachtokentable{% typesetting the token declarations as part of the grammar + \tempcb3 \tempca\tempcb + \expandafter\tokdectoks\expandafter{\expandafter}\the\tokdectoks + \ifnum\tempca=\tempcb + \else + \bloop + \tokdectoks\expandafter{\the\tokdectoks&\omit\hfil}% + \ifnum\tempca=\z@ + \else + \advance\tempca\m@ne + \repeat + \tokdectoks\expandafter{\the\tokdectoks\cr}% + \fi + \toksa{\tt\strut##\hfil\tabskip=0pt plus 1fil}% + \bloop + \toksa\expandafter{\the\toksa&\tt\strut##\hfil}% + \ifnum\tempcb=\tw@ + \else + \advance\tempcb\m@ne + \repeat + \toksa\expandafter{\the\toksa&\tt\strut##\hfil\tabskip\z@\cr}% + \concat\toksa\tokdectoks + \tokdectoks\toksa + \edef\next{\table{\the\table\noalign{% + \tabskip\parindent + \nx\displaytokenrawtrue + \halign to\hsize{\the\tokdectoks}% + }% + }}\next + \tokdectoks{}% +} + +% typesetting token lists, precedence declaratins, etc +% these belong to yyunion.sty but they use `global' settings that are +% too specific to put there + +\def\attachtypestable{% typesetting the type declarations as part of the grammar + \toksa{}\the\typestable + \edef\next{\table{\the\table\noalign{% + {% + \the\toksa + }% + }% + }}\next\typestable{}% +} + +\long\def\onetype#1#2{% + \toksb{{% + \hbadness\@M + \rightskip=\z@ + \tempda\hsize \advance\tempda-1in + \indent{$\langle$\tt union$\rangle.#1$:}\ \hfill\nobreak\hfill\nobreak\null\penalty1\relax\null\nobreak\hfill + \hbox to\tempda{$\vtop{\noindent\hsize\tempda\raggedright #2\strut}$}\par + }}% + \concat\toksa\toksb +} + +\long\def\flexsstatelist#1{% + \toksb{{% + \hbadness\@M + \rightskip=\z@ + \tempda\hsize \advance\tempda-1in + \indent{$\langle$\tt states-s$\rangle_{\rm f}$:}\ \hfill\nobreak\hfill\nobreak\null\penalty1\relax\null\nobreak\hfill + \hbox to\tempda{$\vtop{\noindent\hsize\tempda\raggedright #1\strut}$}\par + }}% + \concat\toksa\toksb +} + +\long\def\flexxstatelist#1{% + \toksb{{% + \hbadness\@M + \rightskip=\z@ + \tempda\hsize \advance\tempda-1in + \indent{$\langle$\tt states-x$\rangle_{\rm f}$:}\ \hfill\nobreak\hfill\nobreak\null\penalty1\relax\null\nobreak\hfill + \hbox to\tempda{$\vtop{\noindent\hsize\tempda\raggedright #1\strut}$}\par + }}% + \concat\toksa\toksb +} + +\def\attachprectable{% typesetting the precedence declarations as part of the grammar + \toksa{}\the\prectable + \edef\next{\table{\the\table\noalign{% + {% + \the\toksa + }% + }% + }}\next\prectable{}% +} + +\long\def\oneprec#1#2#3{% + \toksc{% + \hbadness\@M + \rightskip=\z@ + \tempda\hsize \advance\tempda-1in + \indent + }% + \toksd{% + \ \hfill\nobreak\hfill\nobreak\null\penalty1\relax\null\nobreak\hfill + \hbox to\tempda{$\vtop{\noindent\hsize\tempda\raggedright #3\strut}$}\par + } + \def\next{#2}% + \ifx\next\empty + \toksb{}% + \else + \toksb{${}:{}$#2}% + \fi + \edef\next{\toksb{{$\nx\langle$\nx\bf #1\the\toksb$\nx\rangle$}}}\next + \appendr\toksa{{\the\toksc\the\toksb\the\toksd}}% +} + +\def\attachoptionstable{% typesetting the options as part of the grammar + \edef\next{\table{\the\table\noalign{% + \tabskip\parindent + \halign{####\nx\hfil\tabskip\z@&\nx\qquad\nx\it####\nx\hfil\cr + \the\opttable + }% + }% + }}\next\opttable{}% +} + +\newif\ifpropertable + +\def\checkforpropertable#1{{% checking if there is any table material +% this is a hack to avoid underfull boxes when there is no actual alignment material in +% \halign to \hsize + \hbadness\@M + \let\noalign\trivialnoalign + \let\omit\relax + \setbox\z@=\vbox{ + \halign + {\eatone{##}&\eatone{##}.&\eatone{##}&\eatone{##}\cr + \the#1% + }% + }% + \ifnum\wd\z@>\z@ + \aftergroup\propertabletrue + \else + \aftergroup\propertablefalse + \fi +}} + +\long\def\trivialnoalign#1{} + +% macros for processing \Cee\ mode material + +%\long\def\buildstash#1{\toksa\expandafter{\the\toksa#1}} % = stashed + +\def\cleanstash{% + \ifchecktrim\errmessage{collected stash: \the\toksa}\fi + \expandafter\cleanst@sh\the\toksa\packagebox} + +\def\cleanst@sh{\let\6\testsbox\setbox0=\vbox\bgroup} + +\def\testsbox{% + \ifmmode + \let\next\relax + \else + \egroup + \ifnum\wd\z@>\z@ + \ifnum\ht\z@>\z@ + \let\next\scoopupstash + \else + \let\next\rebuildstash + \fi + \else + \let\next\rebuildstash + \fi + \fi + \next +} + +\long\def\rebuildstash#1\packagebox{\toksa{#1}\cleanstash} + +\long\def\scoopupstash#1\packagebox{} + +\def\packagebox{\egroup\ifnum\wd0>\z@\else\toksa{}\fi} + +\def\stripstash{% + \ifchecktrim\errmessage{before trimming: \the\toksa}\fi + \def\6{}\expandafter\stripst@sh\expandafter\ignorespaces\the\toksa\6\str@pst@sh} + +\def\stripst@sh{\toksa{}\stripst@shi} + +\long\def\stripst@shi#1\6{% + \toksb{#1}\futurelet\next\str@pst@sh +} + +\def\str@pst@sh{% + \ifx\next\str@pst@sh + \iftrailingreturn + \striptrim + \fi + \concat\toksa\toksb + \iftrailingreturn % not done yet + \let\next\stripagain + \else + \let\next\eatone + \fi + \trailingreturnfalse + \else + \edef\next{\toksa{\the\toksa\the\toksb\noexpand\6}}\next + \def\next{\stripst@shi\relax\ignorespaces}% + \trailingreturntrue + \fi + \next +} + +\def\stripagain#1{\stripstash} + +\newif\iftrailingreturn + +\def\striptrim{% + \ifchecktable\errmessage{trimming: \the\toksb}\fi + \edef\next{\the\toksb}% + \expandafter\striptr@m\the\toksb\relax\end +} + +\def\striptr@m + #1% \relax (from last \stripst@shi) + #2% \ignorespaces (from last \stripst@shi) + % intervening spaces + #3% \relax + #4% ? + \end{% +% \def\next{#4}% +% \ifx\next\empty +% \toksc{#3#4}\showthe\toksc + \setbox\z@\vbox{#3#4}% + \ifnum\wd\z@=\z@ + \expandafter\trimreturn\the\toksa\end + \toksb{}% + \else + \ifnum\ht\z@=\z@ + \expandafter\trimreturn\the\toksa\end + \toksb{}% + \else + \trailingreturnfalse + \fi + \fi +} + +\def\trimreturn#1\6\end{% + \toksa{#1}% +} + +\def\boxstash{% + \ifchecktrim\errmessage{stash contents: \the\toksa}\fi + $\vtop{\activateinlinec\tabskip\z@\halign{\strut\ignorespaces##\hfil\cr\the\toksa\crcr}}$} + +\def\makestashbox{\cleanstash\stripstash\boxstash} + +\newbox\indentbox + +\def\activateinlinec{% + \setbox\indentbox=\hbox{}% + \def\1{\setbox\indentbox\hbox{\box\indentbox\hbox to 3em{\hfil}}}% + \def\2{\setbox\indentbox\hbox{\box\indentbox\hbox to -3em{}}}% + \def\4{\hbox to -3em{}}% + \def\5{}% +% \let\6\cr + \def\6{% + \edef\setindentbox{% + \setbox\indentbox\hbox to\the\wd\indentbox{\noexpand\hfil}% + \copy\indentbox\ignorespaces + }% + \expandafter\crcontainer\setindentbox + }% + \def\7{% + \edef\setindentbox{% + \setbox\indentbox\hbox to\the\wd\indentbox{\noexpand\hfil}% + \copy\indentbox\ignorespaces + }% + \expandafter\crcontainerspread\setindentbox + }% + \def\8{\hskip-1em}% + \let\$\prodterm +} + +\def\crcontainer{\cr} +\def\crcontainerspread{\cr\noalign{\yskip}} + +% rough version of the term typesetting + +\def\prodterm{% + \futurelet\next\analyzepterm +} + +\def\analyzepterm{% + \ifx\next\$% + \let\next\pr@dterm + \else + \ifcat\noexpand\next0% + \let\next\pr@dterm + \else + \let\next\oldmathS + \fi + \fi + \next +} + +\def\pr@dterm#1{% + \ifx#1\$% + \def\next{\hbox{$\Upsilon$}}% + \else + \if\noexpand#1[% + \let\next\seeksym + \else + \ifnum`#1<"3A\relax + \ifnum`#1>`0\relax + \def\next{\seekno#1}% + \else + \def\next{\hbox{$\Upsilon$}#1}% + \fi + \else + \def\next{\hbox{$\Upsilon$}#1}% + \fi + \fi + \fi + \next +}% + +\let\oldmathS\$ +\let\$\prodterm + +\def\seekno{\afterassignment\printterm\tempca}% + +\def\seeksym#1]{% + \hbox{$\Upsilon\kern-1pt{}_{\rm#1}$}} + +\def\printterm{\hbox{$\Upsilon\kern-1pt{}_{\number\tempca}$}}% + +\long\def\setproduction#1{% + \def\termidxrank{5}% + \def\headeridxrank{4}% + \def\defidxrank{3}% + \def\texcsidxrank{5}% + \ninepoint + \let\returnexplicitspace\splitexplicitspace + \let\acharswitch\texcharadjust + \let\onecharswitch\texcsadjust + \showlastactionfalse + \let\actionfiller\empty + \parsevb{#1} % Stage two, start the parsing + \ifyyparsefail + {\toks0{#1}\errmessage{failed to parse: \the\toks0}}% + \else % Stage three, process the parsed table + \par\hbox{\strut}% + \typesetalltables + \fi +} + +\def\splitexplicitspace{% + \yyinput\{\}% +} + +\def\texcharadjust{ + ` {% + \yybyte{|}% + \expandafter\yycp@\expandafter`\the\yybyte\relax + \mkpurebyte + \yyreturn + } + _ {% + \yybyte{\_}% + \expandafter\yycp@\expandafter`\the\yybyte\relax + \mkpurebyte + \yyreturn + } +} + +\def\texcsadjust{ + \n {% + \yycp@=\n + \mkpurebyte + \yyreturn + } + \^^M {% + \yyinput\{\}% + } + \' {% + \expandafter\yyinput\benignescape'% + } + \\ {% + \expandafter\expandafter\expandafter\yyinput\expandafter\benignescape\benignescape% + } +}% + +% production typesetting + +\def\beginprod{% + \par + \begingroup + \b@ginprod +} + +\long\def\b@ginprod#1\endprod{% + \setproduction{#1}% + \endgroup + \par +} + +\def\beginmprod{% + $$ + \vbox\bgroup + \def\checkforpropertable##1{\propertablefalse} % so that the table is set to its natural width + \b@ginmprod +} + +\long\def\b@ginmprod#1\endmprod{% + \setproduction{#1}% + \egroup + $$% +} + +% centering productions: works for rule listing only (no token declarations, etc.) + +\def\begincprod#1\endcprod{{\def\tlskip{0 pt plus1fill}\let\tfskip\tlskip\beginprod#1\endprod}} + +% the next macro assumes that \gindex being defined implies that all the bookkeeping required +% for maintaining the index of grammar terms has been taken care of + +\ifx\gindex\UNDEFINED +\else + \termindextrue + \immediate\openout\gindex=\jobname.gdx +\fi + +% stringing all the manuals together (disabled for now) + +%\newwrite\lastpageinfo +%\newread\testeof + +%\toksa\expandafter{\fin} +%\edef\fin{\the\toksa\noexpand\savelastpagenumber} +%\def\savelastpagenumber{% +% \openout\lastpageinfo=\jobname.lpg% +% \write\lastpageinfo{\def\noexpand\contentspagenumber{\number\pageno}\pageno=\noexpand\contentspagenumber \advance\pageno by 1}% +% \write\lastpageinfo{\def\noexpand\secno{\number\secno}}% +% \closeout\lastpageinfo +%} + diff --git a/support/splint/tex/yyinput.sty b/support/splint/tex/yyinput.sty new file mode 100644 index 0000000000..d424d299d8 --- /dev/null +++ b/support/splint/tex/yyinput.sty @@ -0,0 +1,406 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +\def\mkpurebyte{\uccode`\@=\yycp@\uppercase{\yybytepure{@}}\uccode`\@=`\@} + +\def\yyinput{\futurelet\next\yyinp@t} % get the code of the next character ... + +\def\yyinp@t{% test the code and decide whether to continue lexing the + % token or return to the parser + \ifcat\noexpand\next\bgroup + \yybreak\yyinputgroup + \else + \if\noexpand\next\space % code 32 character token is assumed to +% be an ordinary space (while category 10 tokens are not necessarily); +% the reason for this choice is the way code 32 characters are treated +% by \string (turned into `real' spaces); `funny' spaces of both kinds +% (a character that differs from a `real' space (\catcode`\ ==10 +% \number`\ ==32) in either category or character code) can be +% created, however, category 10 characters of character code other +% than 32 are very rare (and take some effort to produce), while +% category 12, character code 32 tokens are poduced as a result of +% `sanitizing' input all the time; note that the spaces are processed +% at the lowest level of the input routine and thus cannot be +% substituted by the switches below; this choice is not bullet proof +% but should suffice for most uses; in the case of more polluted +% input, an extra `sanitation' step can be performed first; if +% desired, category 10 charactes can be assumed to be spaces at this +% point, as well, although this will introduce an extra test, affecting +% the (already poor) efficiency of the input macros. + \yycp@=`\ % + \yybyte={ }% + \mkpurebyte + \yybreak@\yyskipspace % return the space + \else + \if\noexpand\next\eolletter % cannot use ^^M here since TeX will simply drop the rest of the line + \yycp@=\n + \yybyte={\n}% + \mkpurebyte + \yybreak@@\yyskipspace % return the end of line character + \else + \yybreak@@\yy@np@t + \fi + \fi + \yycontinue +} + +\chardef\n=`\^^J +\chardef\charseq1 +\chardef\charac2 +\chardef\chargroup3 + +\def\yyskipspace{\afterassignment\yyreturn\let\next= } + +\def\yyinputgroup#1{% + \yycp@=\chargroup + \yybyte={{#1}}% + \mkpurebyte + \yyreturn +} + +\newif\ifyyinputdebug + +\def\yy@np@t#1{% start lexing: + % whitespace (category 10) tokens are skipped + % automatically + \yybyte{#1}% + \ifyyinputdebug + \immediate\write16{read: \the\yybyte}% + \fi + \ifx#1\ % a space token + \yybreak\returnexplicitspace + \else + \expandafter\ischar\string#100\end + \ifchar % a single character (not a control sequence) + \ifcat\noexpand#1\noexpand~% is it an active character? + \yycp@=\charac % yes, return it + \mkpurebyte + \yybreak@@\yyreturn + \else % it is a non-active character ... + % ... or a control sequence with an empty name (obtained by \csname\endcsname or + % \enlinechar=-1 \toks0={\ + % } this case can be handled by the switch below but is ignored for the moment) + \let\default\achardefault + \yybreak@@{\switchon{\the\yybyte}\in\acharswitch}% + \fi + \else % it is a control sequence, return it + \expandafter\ischar\string#10\end + \ifchar % it is a one-char control sequence + \let\default\onechardefault + \yybreak@@{\switchon{\the\yybyte}\in\onecharswitch}% + \else + \let\default\multichardefault + \yybreak@@{\switchon{\the\yybyte}\in\multicharswitch}% + \fi + \fi + \yycontinue +} + +\chardef\explicitspacecode=`\ % + +\def\returnexplicitspace{% + \yycp@=\explicitspacecode + \mkpurebyte + \yyreturn % keep looking for a non-whitespace token +} + +\def\onecharswitch{ + \raw \n \raw {% + \yycp@=\n + \mkpurebyte + \yyreturn + } + \raw \^^M\raw {% + \returnexplicitspace + } +} + +\def\onechardefault{% + \expandafter\yycp@\expandafter`\the\yybyte\relax + \mkpurebyte + \yyreturn +}% + +\def\multicharswitch{ + \raw\stashed\raw {% + \stashnext + } + \raw\format \formatlocal\raw {% + \formatnext + } + \raw\sflush\raw {% + \sflushnext + } + \raw\yyeof\raw {% + \yycp@=\YYENDOFBUFFERCHAR\relax + \yybytepure={}% + \yyreturn + } + \raw\inputboundary\raw {% + \inputboundarynext + } + \raw\flatten \resetf \inline \skipheader\raw {% + \expandafter\yyinput\expandafter\format\expandafter{\the\yybyte}% + } + \raw\fold \breakline\raw {% + \expandafter\yyinput\expandafter\formatlocal\expandafter{\the\yybyte}% + } + \raw\break\raw {% for testing purposes + \yycp@=`\ % + \yybytepure={ }% + \yyreturn + } + \raw\squashtermstrue\raw {% + \yycp@=`\ % + \yybytepure={ }% + \squashtermstrue + \yyreturn + } + \raw\endparse \endparseinput\raw {% + \errmessage{internal error: reading past the end of the input buffer}% + } +} + +\def\multichardefault{% + \yycp@\charseq + \mkpurebyte + \yyreturn +}% + +\def\achardefault{% + \expandafter\yycp@\expandafter`\the\yybyte\relax + \mkpurebyte + \yyreturn +} + +\def\acharswitch{} + +% the following commands are only here for debugging purposes +% they slow down the input + +\setspecialcharsfrom\onecharswitch +\setspecialcharsfrom\multicharswitch +\setspecialcharsfrom\acharswitch + +\chardef\stashchar=`\ % +\chardef\formatchar=`\ % + +\newcount\stashmarker +\newcount\formatmarker + +\def\stashnext#1{% + \yybytepure{ }\yycp@\stashchar + \toksa{#1}% + \advance\stashmarker\@ne + \edef\next{\yysbyte{\noexpand\strm[\the\stashmarker]{\the\yybyte{\the\toksa}}.[\the\stashmarker]}}\next + % the extra `.' is to avoid the stripping of braces by the parameter scanning mechanism of TeX + \yybyte\expandafter{\the\yybyte{#1}}% + \yyreturn +} + +\def\formatnext#1{% + \yybytepure{ }\yycp@\formatchar + \toksa{#1}% + \advance\formatmarker\@ne + \edef\next{\yyfbyte{\noexpand\strm[\the\formatmarker]{\the\yybyte{\the\toksa}}.[\the\formatmarker]}}\next + % the extra `.' is to avoid the stripping of braces by the parameter scanning mechanism of TeX + \yybyte\expandafter{\the\yybyte{#1}}% + \yyreturn +} + +\chardef\boundarychar=`\ % + +% the following is a minimal setup of a parsing boundary + +\def\inputboundarynext#1{% `l' for left boundary + \yybytepure{ }\yycp@\boundarychar + \yybyte\expandafter{\the\yybyte{#1}}% + \yyreturn % inserting something here will effectively insert it into the input stream +} + +\def\sflushnext#1#2{% #1 is the marker + % #2 is the contents + \yybytepure{ }\yycp@\stashchar + \edef\next{\toksc{\the\yystash\the\yystashseen}}\next + \yyfifolastidx\toksc\in\toksc + \toksa{#1}\toksb{#2}% + \edef\next{\yysbyte{\noexpand\strm[\the\toksa]{\the\yybyte{\the\toksb}{\the\toksc}}.[\the\toksa]}}\next + % the extra `.' is to avoid the stripping of braces by the parameter scanning mechanism of TeX + \yybyte\expandafter{\the\yybyte{#1}{#2}}% + \yyreturn +} + +\def\yyreadfifo#1\to#2\in#3{% + \def\yyr@adfifo##1\strm[#2]##2[#2]##3\end{% + \def\yyr@@dfifo{##3}% + \ifx\yyr@@dfifo\empty + #3{}% there is no such marker in the fifo + \else + #3{##1\strm[#2]##2[#2]}% + \def\yyr@@dfifo####1\strm[#2]####2[#2]\end{#1{####1}}% strip off the inserted string + \yyr@@dfifo##3\end + \fi + }% + \expandafter\yyr@adfifo\the#1\strm[#2].[#2]\end +} + +\def\yyreadfifox#1\to#2\in#3{% read the fifo to a marker in #2 + \expandafter\yyreadfifo\expandafter#1\expandafter\to\number#2\in#3% +} + +\def\yyreadfifoxx#1\to#2\in#3{% read the fifo to a marker in #2 + \expandafter\yyreadfifo\expandafter#1\expandafter\to#2\in#3% +} + +\def\yyreadfifot#1\to#2\in#3\with#4{% read fifo with a test: + \def\yyr@adfifo##1\strm[#2]##2[#2]##3\end{% #4 is a \macro[##1]##2.[##3] which + \def\yyr@@dfifo{##3}% looks at its parameters and fills up + \ifx\yyr@@dfifo\empty % \othertoks (to be attached at the beginning of #1 + % after the initial segment has been removed) + % and \fifotoks (to be used by another macro) + #3{}% there is no such marker in the fifo + \else + \let\fifotoks#3% + \let\othertoks#1 + \let\@@strm\strm + \let\strm#4% + \fifotoks{}% + \othertoks{}% + ##1\strm[#2]##2[#2]% run all the tests + \let\strm\@@strm + \def\yyr@@dfifo####1\strm[#2]####2[#2]\end{#1\expandafter{\the\othertoks####1}}% + \yyr@@dfifo##3\end + \fi + }% + \expandafter\yyr@adfifo\the#1\strm[#2].[#2]\end +} + +\def\yyreadfifotx#1\to#2\in#3\with#4{% read the fifo to a marker in #2 + \expandafter\yyreadfifot\expandafter#1\expandafter\to\number#2\in#3\with{#4}% +} + +% an example of a sorting macro: extract all the local formatting + +\def\formatsort[#1]#2.[#3]{% clobbers \toksb, \toksc + \formatgrab#2\end + \toksc{\strm[#1]{#2}.[#3]}% + \let\default\formatdefaultact + \switchon{\the\toksb}\in\formatswitch +} + +\def\formatgrab#1#2\end{\toksb{#1}} + +\def\formatswitch{% + \formatlocal {% + \edef\next{\fifotoks{\the\fifotoks\the\toksc}}\next + } +} + +\def\formatdefaultact{% + \edef\next{\othertoks{\the\othertoks\the\toksc}}\next +} + +\def\yygrabfifoelem#1\at#2\in#3{% + \def\yyr@adfifo##1\strm[#2]##2[#2]##3\end{% + \def\yyr@@dfifo{##3}% + \ifx\yyr@@dfifo\empty + #3{}% there is no such marker in the fifo + \else + #3{\strm[#2]##2[#2]}% + \def\yyr@@dfifo####1\strm[#2]####2[#2]\end{#1{####1}}% strip off the inserted string + \yyr@@dfifo##1##3\end + \fi + }% + \expandafter\yyr@adfifo\the#1\strm[#2].[#2]\end +} + +\def\yygrabfifoelemx#1\at#2\in#3\with#4{% read the fifo to a marker in #2 + \expandafter\yygrabfifoelem\expandafter#1\expandafter\at\number#2\in#3\with{#4}% +} + +\def\yypeekfifoelem#1\at#2\in#3{% just read the first element that matches the marker + \def\yyr@adfifo##1\strm[#2]##2[#2]##3\end{% + \def\yyr@@dfifo{##3}% + \ifx\yyr@@dfifo\empty + #3{}% there is no such marker in the fifo + \else + #3{##2}% just get the contents + \fi + }% + \expandafter\yyr@adfifo\the#1\strm[#2].[#2]\end +} + +\def\yypeekfifoelemx#1\at#2\in#3\with#4{% read the fifo to a marker in #2 + \expandafter\yypeekfifoelem\expandafter#1\expandafter\at\number#2\in#3\with{#4}% +} + +\def\strmgetfirstidx[#1]#2.[#3]{\toksa{#1}} + +\def\yyfifolastidx#1\in#2{% + \let\next\strm + \let\strm\strmgetfirstidx + \toksa{}\the#1% + #2\toksa + \let\strm\next +} + +\def\z@rotest{0} +\newif\ifchar + +\def\ischar#1#2#3\end{% three parameters because #1 can be an + % \escapechar + \def\lastnamechar{#3}% + \ifx\lastnamechar\z@rotest\chartrue\else\charfalse\fi +} + +% trivial input routine + +\def\yyinputtrivial{\futurelet\next\yyinp@ttrivial} % get the code of the next character ... + +\def\yyinp@ttrivial{ + \ifcat\noexpand\next\space % category 10 token + \yycp@=`\ % + \yybyte={ }% + \mkpurebyte + \yybreak\yyskipspace % return the space + \else + \yybreak\yy@np@ttrivial + \yycontinue +} + +\def\yy@np@ttrivial#1{% + \ifcat\noexpand#1a% + \yycp@`#1% + \yybyte{#1}% + \yybytepure{#1}% + \else + \if\noexpand#1\eolletter + \yycp@=\n + \yybyte\expandafter{\eolletter}% + \mkpurebyte + \else + \ifx#1\yyeof + \yycp@=\YYENDOFBUFFERCHAR\relax + \yybyte{#1}% + \yybytepure={}% + \else + \yycp@\charseq + \mkpurebyte + \fi + \fi + \fi + \yyreturn +} diff --git a/support/splint/tex/yymisc.sty b/support/splint/tex/yymisc.sty new file mode 100644 index 0000000000..fcc38da96d --- /dev/null +++ b/support/splint/tex/yymisc.sty @@ -0,0 +1,1422 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% `data structure access' macros': picking the n-th undelimited parameter +% in a parameter list inside a token register +% it is assumed that none of the arguments is \end, and that there are enough +% parameters to pick the desired one + +\def\getfirst#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tfirst\the#1\end}% +} + +\def\g@tfirst#1#2\end{#1} + +\def\getsecond#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tsecond\the#1\end}% +} + +\def\g@tsecond#1#2#3\end{#2} + +\def\getthird#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tthird\the#1\end}% +} + +\def\g@tthird#1#2#3#4\end{#3} + +\def\getfourth#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tfourth\the#1\end}% +} + +\def\g@tfourth#1#2#3#4#5\end{#4} + +\def\getfifth#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tfifth\the#1\end}% +} + +\def\g@tfifth#1#2#3#4#5#6\end{#5} + +\def\getsixth#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tsixth\the#1\end}% +} + +\def\g@tsixth#1#2#3#4#5#6#7\end{#6} + +\def\getseventh#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@tseventh\the#1\end}% +} + +\def\g@tseventh#1#2#3#4#5#6#7#8\end{#7} + +\def\geteightth#1\to#2{% + #2\expandafter\expandafter\expandafter{\expandafter\g@teightth\the#1\end}% +} + +\def\g@teightth#1#2#3#4#5#6#7#8#9\end{#8} + +\def\getninth#1\to#2{% the `.' is necessary since \TeX's scanning mechanism will + % strip any potential braces surrounding the last parameter + % (assuming there are exactly nine) twice: + % o first, expanding \g@tninth, + % o second, expanding \g@tfirst + #2\expandafter\expandafter\expandafter{\expandafter\g@tninth\the#1.\end}% + #2\expandafter\expandafter\expandafter{\the#2}% +} + +\def\g@tninth#1#2#3#4#5#6#7#8#9\end{\g@tfirst#9\end} + +\def\gettenth#1\to#2{% no need for `.' (or any other placeholder) here, + % since the #9-th parameter to \g@ttenth is a list of + % at least two parameters itself, thus any existing braces + % have survived intact + #2\expandafter\expandafter\expandafter{\expandafter\g@ttenth\the#1\end}% + #2\expandafter\expandafter\expandafter{\the#2}% +} + +\def\g@ttenth#1#2#3#4#5#6#7#8#9\end{\g@tsecond#9\end} + +% string replacement: all arguments are registers, nothing is expanded, no \next is defined +% note that this is not a greedy replacement: this could be arranged with a more sophisticated macro +% also note that the string being replaced cannot have any braces in it + +\newif\ifyytracereplacements +\newif\ifyyreplaced + +\yytracereplacementstrue + +\def\yyreplacestring#1\in#2\with#3{% + \expandafter\def\expandafter\r@placestring\expandafter##\expandafter1\the#1##2\end{% + \def\r@placestring{##2}% is this the string at the very end? + \ifx\r@placestring\empty % then it is the one we inserted, + % report + \yyreplacedfalse + \ifyytracereplacements + \errmessage{string <\the#1> not present in \the#2}% do not change the register if the string is not there + \fi + \else % remove the extra copy of #1\end at the end + \yyreplacedtrue + \expandafter#2\expandafter\expandafter\expandafter + {\expandafter\r@plac@string\expandafter{\the#3}{##1}##2\end}% + \fi}% end of \r@placestring definition + \expandafter\def\expandafter\r@plac@string + \expandafter##\expandafter1% + \expandafter##\expandafter2% + \expandafter##\expandafter3% + \the#1\end{##2##1##3}% + \expandafter\expandafter\expandafter\r@placestring\expandafter\the\expandafter#2\the#1\end +} + +% creating a sequence containing all pairs from the two given sequences; +% the (long) string produced by the \diagprod macro +% lists all the elements so that each ordered pair ab where a and b are +% different elements from each of the two sets appears exactly once; +% a simple strategy for creating such strings is to build them recursively so that if +% S is a string that lists n values in this manner and s is a new item type, +% start with sSs and then add each symbol from S on either side +% in such a way that among any two consecutive symbols exactly one is s + +% these macros are used to create switch statements that use such ordered pairs +% and are not particularly general or robust; +% they are supposed to be used once in the setup stage; +% the assumption made by these mactos is that `.' or `;' never appear as elements +% of the two sets + +\def\gnxtelem#1\to#2\and#3{% + \expandafter\gnxtel@m\expandafter#2\expandafter#3\the#1.;% +} + +\def\gnxtel@m#1#2#3#4;{% + \def\next{#4}% + \ifx\next\empty + #1{}% + \else + #1{#3}% + \gnxt@l@m#2#4% + \fi +} + +\def\gnxt@l@m#1#2.{% + #1{#2}% +} + +\def\pairup{% + \gnxtelem\toksa\to\toksc\and\toksa % get the first remaining element of set A (\toksa) + \edef\elemofA{\the\toksc}% + \ifx\elemofA\empty % no more elements in A + \let\next\relax + \else + \ifx\elemofA\lastelemofB % the current element of A is the same as the last element of B + \edef\next{\the\toksa}% + \ifx\next\empty % it is the last remaining element of A + \expandafter\p@ir@p\the\toksb.% form all pair of \the\toksc with elements in B except the first one + \else + \concat\toksa\toksc % move it to the end of A + \fi + \let\next\pairup + \else + \expandafter\pair@p\the\toksb.% form all pairs of \the\toksc with elements in B + \let\next\pairup + \fi + \fi + \next +} + +\def\dotcontainer{.} + +\def\pair@p#1{% + \def\next{#1}% + \ifx\next\elemofA % the next element of B is the current element of A + \ifx\next\lastelemofB % it is the last element of B + % we can arrive here only if 1) B has more than one element and + % 2) \elemofA is the last remaining element of A + \edef\next{\toksd{\the\toksd\the\tokse}}\next + \let\next\eatone % eat the remaining dot + \else + \removeelem\elemofA\from\toksb % remove it from B + \let\next\pair@p + \fi + \else + \ifx\next\dotcontainer % no more elements in B + \let\next\relax + \else + \toksf{#1}% + \edef\next{\toksd{\the\toksd\the\toksc\the\toksf}}\next + %\showthe\toksd + \let\next\pair@p + \fi + \fi + \next +} + +\def\p@ir@p#1{% + \def\next{#1}% + \ifx\next\elemofA % the next element of B is the current element of A + % this can only happen if B consists of a single element + \let\next\eatone + \else + \toksf{#1}% + \edef\next{\toksd{\the\toksd\the\toksf}}\next + \let\next\pair@p + \fi + \next +} + +\def\removeelem#1\from#2{% #1 should be a sequence containing the element + % #2 is the token register + \expandafter\def\expandafter\r@moveelem\expandafter##\expandafter1#1##2.{% + \def\next{##2}% + \ifx\next\empty % there was no such element + \errmessage{Could not find \expandafter\string#1 in \the#2}% + \else + \expandafter\def\expandafter\r@mov@elem\expandafter####\expandafter1#1{% + #2{##1####1}% + }\r@mov@elem##2% + \fi + }% + \expandafter\expandafter\expandafter\r@moveelem\expandafter\the\expandafter#2#1.% +} + +% the intersection of A and B should be at the end of A: + +\def\pushintersect{% + \gnxtelem\toksf\to\toksc\and\toksf % get the first remaining element of set A (\toksa) + \edef\next{\the\toksc}% + \ifx\next\empty % there are no elements left in A + \let\next\relax + \else + \expandafter\def\expandafter\p@shintersect\expandafter##\expandafter1\the\toksc##2.{% + \def\next{##2}% + \ifx\next\empty % there was no such element in the other set + \edef\next{\toksa{\the\toksc\the\toksa}}\next % append it to the front + \else + \edef\next{\toksa{\the\toksa\the\toksc}}\next % append it to the back + \fi + }% + \expandafter\expandafter\expandafter\p@shintersect\expandafter\the\expandafter\toksb\the\toksc.% + \let\next\pushintersect + \fi + \next +} + +\def\diagprod#1#2\in#3{% + \toksa\expandafter{#1}% + \toksb\expandafter{#2}% + \gnxtelem\toksb\to\tokse\and\toksb % \tokse is a selected element in set B (\toksb) + \concat\toksb\tokse % now \tokse is the last element of B + \toksf\toksa\toksa{}% + \pushintersect % prepare the sequence representing set A so that the intersection elements are in the tail + \edef\lastelemofB{\the\tokse}% the last element of B + \toksd\expandafter{\the\tokse}% future sequence pairs + \pairup + \edef#3{\the\toksd}% +} + +% namespace management for macros +% note that if one of the sequences were made \let\name. the macros would break; +% this can be fixed by using a more sophisticated comparison but was decided against +% in the interest of efficiency; the old version (that used a control sequence +% as a `stop marker') was more prone to this bug. + +\def\savecs#1#2{\s@vecs{#1}#2.} + +\def\s@vecs#1#2{% + \ifx#2.% + \yybreak{}% + \else + \yybreak{% + \expandafter\let\csname '#1'[\expandafter\eatone\string#2]\endcsname#2% + \s@vecs{#1}% + }% + \yycontinue +} + +\def\restorecs#1#2{\r@storecs{#1}#2.} + +\def\r@storecs#1#2{% + \ifx#2.% + \yybreak{}% + \else + \yybreak{% + \expandafter\let\expandafter#2\csname '#1'[\expandafter\eatone\string#2]\endcsname + \r@storecs{#1}% + }% + \yycontinue +} + +\def\hidecs#1{\h@decs#1.} + +\def\h@decs#1{% + \ifx#1.% + \yybreak{}% + \else + \yybreak{% + \let#1\relax + \h@decs + }% + \yycontinue +} + +\def\savehcs#1#2{\savecs{#1}{#2}\hidecs{#2}} + +\def\savecslist#1#2{% + \expandafter\s@vecslist\expandafter{#2}{#1}% +} + +\def\s@vecslist#1#2{% + \savecs{#2}{#1}% +} + +\def\restorecslist#1#2{% + \expandafter\r@storecslist\expandafter{#2}{#1}% +} + +\def\r@storecslist#1#2{% + \restorecs{#2}{#1}% +} + +% a twist on the macros above: save control sequences with a postfix + +\def\savecsx#1#2{\s@vecsx{#1}#2.} + +\def\s@vecsx#1#2{% + \ifx#2.% + \yybreak{}% + \else + \yybreak{% + \expandafter\s@vecsxlet\expandafter#2\csname\expandafter\defprefix\expandafter\eatone\string#2\defpostfix\endcsname{#1}% + \s@vecsx{#1}% + }% + \yycontinue +} + +\def\s@vecsxlet#1#2{% + \expandafter\let\csname '#2'[\expandafter\defprefix\expandafter\eatone\string#1\defpostfix]\endcsname +} + +\def\restorecsx#1#2{\r@storecsx{#1}#2.} + +\def\r@storecsx#1#2{% + \ifx#2.% + \yybreak{}% + \else + \yybreak{% + \expandafter\r@storecsxlet\expandafter#2\csname '#1'[\expandafter\defprefix\expandafter\eatone\string#2\defpostfix]\endcsname + \r@storecsx{#1}% + }% + \yycontinue +} + +\def\restorecsxlist#1#2{% + \expandafter\r@storecsxlist\expandafter{#2}{#1}% +} + +\def\r@storecsxlist#1#2{% + \restorecsx{#2}{#1}% +} + +\def\r@storecsxlet#1{% + \expandafter\let\csname\expandafter\defprefix\expandafter\eatone\string#1\defpostfix\endcsname +} + +\def\hidecsx#1{\h@decsx#1.} + +\def\h@decsx#1{% + \ifx#1.% + \yybreak{}% + \else + \yybreak{% + \expandafter\let\csname\expandafter\defprefix\expandafter\eatone\string#1\defpostfix\endcsname\relax + \h@decsx + }% + \yycontinue +} + +\def\savehcsx#1#2{\savecsx{#1}{#2}\hidecsx{#2}} + +\def\defx#1{% defining sequences as above + \toksa\expandafter{% + \csname\expandafter\defprefix\expandafter\eatone\string#1\defpostfix\endcsname}% + \afterassignment\d@fx + \expandafter\def\the\toksa +} + +\def\d@fx#1{% + \toksb{#1}\edef\next{\noexpand\savehcs{\the\toksb}\the\toksa}\next +} + +\def\defy#1#2#{% in addition to the definition of the contorl sequence + % in the appropriate namespace, this macro adds a + % preamble, a postamble and a `this' type macro; this + % will mostly be used with indexing \TeX\ control sequences + \tokse{\def\thisname{#1}\edef\thisnamex{\expandafter\eatone\string#1}}% + \toksa\expandafter{% + \csname\expandafter\defprefix\expandafter\eatone\string#1\defpostfix\endcsname}% + \toksc\expandafter{\expandafter\def\the\toksa#2}% + \afterassignment\d@f@ + \toksd=% +} + +\def\d@f@{% + \appendl\toksd{\the\tokse}% + \tokse\expandafter{\defypreamble}% + \appendl\toksd{\the\tokse}% + \tokse\expandafter{\defypostamble}% + \appendr\toksd{\the\tokse} + \toksd\expandafter{\expandafter{\the\toksd}}% + \concat\toksc\toksd + \afterassignment\d@fy + \the\toksc +} + +\def\d@fy#1{% + \toksb{#1}\edef\next{\noexpand\savehcs{\the\toksb}\the\toksa}\next +} + +% dynamic typing macros + +\def\defp#1#2#{% flexible dynamic type checking + \toksa\expandafter\expandafter\expandafter{\yyuniontag#1}% + \expandafter\edef\yyuniontag{\the\toksa}% + \def#1#2{\errmessage{unexpected type: \string#1 in namespace <\currentyyunionnamespace>}}% + \savecs\parserstrictnamespace#1% + \toksa{#2}% + \edef#1{\the\toksa}% + \savecs\parserprototypesnamespace#1% + \def#1#2% +} + +% define the macro with checking of the prototype + +\def\defu#1#2#{% + \restorecs\parserprototypesnamespace#1% + \toksa{#2}% + \edef\next{\the\toksa}% + \ifx#1\next + \yybreak{\def#1#2}% + \else + \toksb\expandafter{#1}% + \yybreak{% + {\newlinechar=`^^J% + \errhelp{the prototype of #1is from <parser-strict:headers>^^J% + you might want to look for a \defp#1line... somewhere}% + \errmessage{macro definition of \noexpand#1 + does not match its prototype:^^J + \the\toksa\space (should be \the\toksb)}% + }% + }% + \yycontinue +} + +% define the macro with an automatic prototype + +\def\defc#1{% + \restorecs\parserprototypesnamespace#1% + \expandafter\def\expandafter#1#1% +} + +\def\t@yyunion#1#2{\def\currentyyunionnamespace{#2}\savecslist{#2}#1} +\def\toyyunion#1{\expandafter\t@yyunion\yyuniontag{#1}} + +{\catcode`\ =13 \aftergroup\def\aftergroup\activespace\aftergroup{\aftergroup \aftergroup}}% active space +{\catcode`\% =12 \aftergroup\def\aftergroup\harmlesscomment\aftergroup{\aftergroup%\aftergroup}}% not really a comment +{\catcode`\{ =12 \catcode`\[=1 \aftergroup\def\aftergroup\lbchar\aftergroup[\aftergroup{\aftergroup}}% not really a brace +{\catcode`\} =12 \catcode`\]=2 \aftergroup\def\aftergroup\rbchar\aftergroup{\aftergroup}\aftergroup]]% not really a brace +{\catcode`\_=12 \aftergroup\def\aftergroup\uscoreletter\aftergroup{\aftergroup_\aftergroup}} +{\catcode`\^^M=12 \aftergroup\def\aftergroup\eolletter\aftergroup{\aftergroup^^M\aftergroup}}% end of line, ... not really +{\catcode`\|=0\catcode`\\=12 |aftergroup|def|aftergroup|benignescape|aftergroup{|aftergroup\|aftergroup}}% not an escape +{\catcode`\#=12 \aftergroup\def\aftergroup\hashletter\aftergroup{\aftergroup#\aftergroup}}% not a parameter token + +% token name input + +\def\doascii#1{% + \tempca=\z@ \tempcb=\@cclvi + \let\yyasciicc\empty + \let\sts\relax + \loop + \edef\yyasciicc{\sts{\the\catcode\tempca}\yyasciicc}% + \catcode\tempca=#1 + \advance\tempca\@ne + \ifnum\tempca<\tempcb\relax + \repeat + \catcode`\ =10 % keep normal spaces (otherwise \string will convert the category to 10) + %\catcode`\^^M=5 % if newline is to be preserved + \catcode`\^^M=12 % to match \eolletter +} + +\def\undoascii{% + \tempca=\@cclv + \let\sts\undoacatcode + \yyasciicc\def\yyasciicc{}% +} + +\def\undoacatcode#1{\catcode\tempca=#1\advance\tempca\m@ne} + +% yytname macros + +\def\endcontainer{\end}% +\def\aaddname{\addname} + +\def\addname{% + \toksb{}% + \@ddname +} + +\def\@ddname#1{% + \def\next{#1}% + \ifx\next\aaddname + \expandafter\edef\csname term\parsernamespace\the\toksb \endcsname{\the\tempca}% + \appendtoyytname + \advance\tempca1\relax + \let\next\addname + \else + \ifx\next\endcontainer + \expandafter\edef\csname term\parsernamespace\the\toksb \endcsname{\the\tempca}% + \appendtoyytnamelast + \let\next\relax + \else + \ifnum#1=`\_\relax + \uccode`\.=#1\relax + \uppercase{\toksa{.}}% + \uccode`\.=`\.% + \else + \ifnum#1=`\-\relax + \toksa{-}% + \else + \ifnum#1=`\ \relax + \toksa{ }% + \else + \uccode`\@=#1\relax + \uppercase{\toksa{@}}% + \uccode`\@=`\@\relax + \fi + \fi + \fi + \concat\toksb\toksa + \let\next\@ddname + \fi + \fi + \next +} + +\def\appendtoyytname{% + \concat\yytname\toksb + \toksa{\or}% + \concat\yytname\toksa +} + +\def\appendtoyytnamelast{% + \concat\yytname\toksb +} + +\def\print#1{{\endlinechar=`\^^J\immediate\write16{#1}}} + +% token and state setup +% these are defined for a specific (though dynamic) namespace only + +\def\tokenset#1#2{% + \expandafter\edef\csname token\parsernamespace \fgetelemof{yytname}\at{#1}\endcsname{#2}% +} + +\def\settokens{% + \tempca=1\relax + \loop + \tempcb=\fgetelemof{yytranslate}\at\tempca\relax % important \relax! + \tokenset\tempcb{\the\tempca}% + \advance\tempca\@ne + \ifnum\YYTRANSLATESIZE>\tempca + \repeat + \relax +} + +\def\stateset#1#2{% + \expandafter\def\csname flexstate\parsernamespace #1\endcsname{#2}% +} + +% token equivalence for bootstrap + +\def\tokeneq#1#2{% + \toksa{}% + \numberstochars#2\end + \toksa\expandafter{\csname token\parsernamespace\the\toksa\endcsname}% + \toksb\expandafter{\csname token\parsernamespace#1\endcsname}% + \edef\next{\let\the\toksb\the\toksa}\next +}% + +\def\numberstochars#1{% + \ifx\end#1% + \yybreak{}% + \else + \uccode`.=#1\relax + \uppercase{\toksa\expandafter{\the\toksa.}}% + \uccode`.=`\.% + \yybreak{\numberstochars}% + \yycontinue +} + +\def\numberstocharsandspaces#1{% same as above but turn \number`\ into a real space token + \ifx\end#1% + \yybreak{}% + \else + \ifnum#1=` \relax + \yybreak@{% + \expandafter\expandafter\expandafter + \toksa\expandafter\expandafter\expandafter{\expandafter\the\expandafter\toksa\space}\numberstocharsandspaces}% + \else + \uccode`.=#1\relax + \uppercase{\toksa\expandafter{\the\toksa.}}% + \uccode`.=`\.% + \fi + \yybreak{\numberstocharsandspaces}% + \yycontinue +} + + +% other parser and lexer variables + +\newtoks\pinittoks % a token register to hold the code that switches the parser to a different namespace + +% tables + +\def\newtable@full#1{% + \toksa{\csname newtoks\endcsname}% + \expandafter\the\expandafter\toksa\csname #1\parsernamespace\endcsname + \edef\next{\let\csname #1\endcsname\csname #1\parsernamespace\endcsname}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa + \csname #1\parsernamespace\endcsname=% +} + +\let\newtable\newtable@full % optimization can change this + +% constants + +\def\constset#1#2{% + % a \mathchardef would be nicer but it cannot handle negative numbers + \expandafter\def\csname #1\parsernamespace\endcsname{#2}% + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksa\the\toksb}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +\def\uconstset#1#2{% + % a \mathchardef for positive constants + \expandafter\mathchardef\csname #1\parsernamespace\endcsname=#2 % + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksa\the\toksb}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +\def\charset#1#2{% + \expandafter\chardef\csname #1\parsernamespace\endcsname=#2\relax% + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksa\the\toksb}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +% switch macro + +\def\stashswitch#1{% + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksb\the\toksa}\next + \edef\next{\let\the\toksa\the\toksb}% + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +% parser and lexer state control + +\def\settokreg#1{% + \toksa{\csname newtoks\endcsname}% + \expandafter\the\expandafter\toksa\csname #1\parsernamespace\endcsname + \edef\next{\let\csname #1\endcsname\csname #1\parsernamespace\endcsname}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +\def\setcntreg#1{% + \toksa{\csname newcount\endcsname}% + \expandafter\the\expandafter\toksa\csname #1\parsernamespace\endcsname + \edef\next{\let\csname #1\endcsname\csname #1\parsernamespace\endcsname}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa +} + +\def\setnulstack#1{% + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksb\noexpand\empty}\next + \edef\next{\let\the\toksa\the\toksb}\next + \toksa\expandafter{\next}\concat\pinittoks\toksa + \setcntreg{#1<count>}% this is only needed for the accelerated stack +} + +\def\setcurrentcs#1{% + \toksa\expandafter{\csname #1\endcsname}% + \toksb\expandafter{\csname #1\parsernamespace\endcsname}% + \edef\next{\let\the\toksb\the\toksa}\next + \edef\next{\toksa{\let\the\toksa\the\toksb}}\next + \concat\pinittoks\toksa +} + +\def\newparserstate{% + \setcntreg{yytoken}% + \setcntreg{yystate}% + \setcntreg{yyn}% + \setcntreg{yylen}% + % + \setcntreg{yyilen}% depth of stack before an inline action + % \yyval and \yylval will be token registers + \settokreg{yyval}% + \settokreg{yylval}% + \setnulstack{yyssa}% + \setnulstack{yyvsa}% +} + +\def\newlexerstate{% + \settokreg{yytext}% + \settokreg{yytextseen}% + \settokreg{yybyte}% + \settokreg{yyfutureyytext}% token register used to save read text in eob macros + % + \settokreg{yytextpure}% % the registers that serve the same role + \settokreg{yytextseenpure}% % as \yytext et al except they collect characters + \settokreg{yybytepure}% % with category code 11 (letter) and the same character code + % + \settokreg{yyformat}% % the registers that serve the same role + \settokreg{yyformatseen}% % as \yytext et al except they collect format + \settokreg{yyfbyte}% % commands + % + \settokreg{yystash}% % the registers that serve the same role + \settokreg{yystashseen}% % as \yytext et al except they collect stashed + \settokreg{yysbyte}% % tokens + % + \setcntreg{yycurrentstate}% + \setcntreg{yycurrentstatelocal}% a state variable used in eob macros + \setcntreg{yyc}% + \setcntreg{yyclocal}% another eob macro variable + \setcntreg{yyact}% + \setcntreg{yyg@yyinit}% + \setcntreg{yyg@yystart}% + \setcntreg{yyg@yylastacceptingstate}% + \setcntreg{yycp@}% + \setcntreg{YYATBOL}% + \setcntreg{yychar}% + % + \setcntreg{yytextlastchar}% + \setcntreg{yytextseenlastchar}% + % + \setcntreg{yyfmark}% % the last marker in the current token + \setcntreg{yyfmarklast}% + \setcntreg{yysmark}% + \setcntreg{yysmarklast}% + \setnulstack{yystatestack}% + \setcurrentcs{setflexstates}% + \setcurrentcs{ifyytextbackup}% this is purely to record the recovery command in \pinittoks +} + +\def\savestatelist#1{% this elaborate definition is needed to ensure that `throwaway token registers' like \toksa + % survive the namespace switch + \edef\next{\expandafter\expandafter\expandafter + \let\expandafter\expandafter\expandafter\noexpand\expandafter\expandafter\csname #1\parsernamespace\endcsname + \expandafter\noexpand\csname #1\endcsname}\next +} + +\def\savefullstate{% + \savestatelist{yyssa}% + \savestatelist{yyvsa}% + \savestatelist{yystatestack}% + \savestatelist{ifyytextbackup}% buffer +} + +% use \pinittoks to compose a `parser restore' macro along with +% \let\yyvsa\yyvsa[parser namespace] and \let\yyssa\yyssa[parser namespace]; +% `parser save' macro only has to set up \yy?sa[parser namespace]'s; +% if a fully reentrant parser is required, use the macros above to save the contents of +% all the variables by redefining \settokreg and \setcntreg and saving \yy?sa stacks + +% switch and dfa macros + +%% character class checking: #1 is the character, #2 is the character class control sequence + +\newif\ifinclass + +\def\ifclassof#1\is#2{% + \tempca#1\relax + \expandafter\checkclass#2\end +} + +\def\checkclass#1{% + \ifx#1\end + \let\next\relax + \inclassfalse + \else + \ifnum`#1=\tempca + \let\next\classend + \inclasstrue + \else + \let\next\checkclass + \fi + \fi + \next +} + +\tempca=\catcode`\^^A +\catcode`\^^A=12 % so that the characters inside sequences get the appropriate catcodes + +\def\classend#1\end{} + +\def\setspecialchars#1{% + \toksa{}% + \tempca=`\ % + \loop + \advance\tempca\@ne + \tempcb=\fgetelemof{yytranslate}\at\tempca\relax + \ifnum\tempcb>\tw@ + \tempcb=\uccode`^^A\uccode`^^A=\tempca + \uppercase{\edef\next{\toksa{\the\toksa^^A}}}\next + \uccode`^^A=\tempcb + \fi + \ifnum\tempca<\@cclv + \repeat + \edef#1{\the\toksa}% +} + +\newif\iftraceswitchlabels +\traceswitchlabelstrue + +\def\setspecialcharsfrom#1{% + \toksb\expandafter{#1}% + \iftraceswitchlabels + \edef\next{\toksa{\expandafter\noexpand\csname <state: \expandafter\eatone\string#1>\endcsname}}\next + \expandafter\let\the\toksa\empty + \else + \toksa{}% + \fi + \tempcb=\uccode`\^^A\relax + \expandafter\s@tspecialcharsfrom\the\toksb\end + \edef#1{\the\toksa}% + \uccode`\^^A=\tempcb +} + +\def\s@tspecialcharsfrom{% + \futurelet\next\s@tsp@cialcharsfrom +}% + +\def\s@tsp@cialcharsfrom{% + \ifcat\space\noexpand\next + \expandafter + \s@tsp@cialch@rsfr@m + \else + \expandafter + \s@tsp@cialch@rsfrom + \fi +} + +\def\s@tsp@cialch@rsfr@m{% + \afterassignment\s@tspecialcharsfrom\let\next= % this is the optional space +} + +\def\s@tsp@cialch@rsfrom#1{% + \toksb{#1}% + \ifx\next\bgroup % this is an action group + \edef\next{\toksa{\the\toksa{\the\toksb}}}\next + \let\next\s@tspecialcharsfrom + \else + \let\default\specchardefault + \iftracedfa % to avoid conflicts while tracing actions + \tracedfafalse + \switchon{\the\toksb}\in\speccharswitch + \tracedfatrue + \else + \switchon{\the\toksb}\in\speccharswitch + \fi + \fi + \next +}% + +\def\speccharswitch{% + \end {% + \let\next\relax + } + \raw \rawcode \classexpand \meanit \statecomment {% + % + } +} + +\def\specchardefault{% + \uccode`\^^A\expandafter`\the\toksb + \uppercase{\edef\next{\toksa{\the\toksa^^A}}}\next + \let\next\s@tspecialcharsfrom +} + +\def\raw#1\raw{% + \toksb{#1}% + \concat\toksa\toksb + \s@tspecialcharsfrom +} + +\def\rawcode#1{% + \ifx#1\rawcode + \let\next\s@tspecialcharsfrom + \else + \uccode`\^^A=#1\relax + \uppercase{\edef\next{\toksa{\the\toksa^^A}}}\next + \let\next\rawcode + \fi + \next +} + +\def\classexpand#1{% + \toksb\expandafter{#1}% + \concat\toksa\toksb + \s@tspecialcharsfrom +} + +\def\meanit#1{% + \toksb\expandafter{\meaning#1}% + \concat\toksa\toksb + \s@tspecialcharsfrom +} + +\def\statecomment#1\statecomment{\edef\next{\toksa{\the\toksa + \expandafter\noexpand\csname (#1)\endcsname}}\next\s@tspecialcharsfrom} + +\def\putother#1\in#2{% + \tempca=\catcode`\^^A\relax + \tempcb=\uccode`\^^A\relax + \uccode`\^^A=#1\relax + \uppercase{\edef\next{#2{^^A}}}\next + \uccode`\^^A=\tempcb +} + +% listing all the rules in a macro + +\def\listrules{% + \tempcb=\tw@ % start with a user-set rule + \loop + \listrule + \ifnum\tempcb<\YYNRULES\relax + \advance\tempcb\@ne + \repeat +} + +\catcode`\$=11 + +\def\listrule{% + \tempcc=\fgetelemof{yyrone}\at\tempcb\relax % get the symbol this rule derives + \edef\next{\toksa{\fgetelemof{yytname}\at\tempcc}}\next + \expandafter\checkforimplicit\the\toksa $@\end%$ + \ifimplicit + \edef\next{\toksc{\noexpand\ih{\the\toksa}}}\next + \edef\next{\toksa{\noexpand\implicitrule[\the\toksa]}}\next + \else + \edef\next{\toksa{\the\tempcc}}\next + \toksc{}% + \fi + \appendr\toksa{:}% + \toksb{}% + \tempcc=\fgetelemof{yyprhs}\at\tempcb\relax + \tempcd=\fgetelemof{yyrhs}\at\tempcc\relax + \listr@le + \edef\next{\newsymswitch{\the\newsymswitch\the\toksa\def{\the\toksc:\the\tempcb}\noexpand\ruleor}}\next +} + +\def\implicitsymbol#1{}% + +\newif\ifimplicit + +\def\listr@le{% + \ifnum\tempcd>\m@ne + \ifnum\tempcd<\YYNTOKENS\relax % it is a terminal + \edef\next{\toksb{\the\tempcd}}\next + \else + \edef\next{\toksb{\fgetelemof{yytname}\at\tempcd}}\next + \expandafter\checkforimplicit\the\toksb $@\end%$ + \ifimplicit + \appendl\toksc{\noexpand\imn{\the\toksb}} + \toksb{@implicit@}% + \else + \edef\next{\toksb{\the\tempcd}}\next + \fi + \fi + \appendr\toksa{ }\concat\toksa\toksb + \advance\tempcc\@ne + \tempcd=\fgetelemof{yyrhs}\at\tempcc\relax + \let\next\listr@le + \else % this is the end of the rhs + \let\next\relax + \fi + \next +} + +\def\checkforimplicit#1$@#2\end{% $ + \def\next{#1}% + \ifx\next\empty + \implicittrue + \else + \implicitfalse + \fi +} +\catcode`\$=3 + +% symbol switch macros + +\newtoks\oneimplicitrule + +% inline explicit symbolic names + +\def\itermstack#1#2{% + \toksa=\oneimplicitrule + \appendl\toksa{\noexpand\lhs{#1}{#2}}% + \expandafter + \yypush + \expandafter + {\the\toksa}\on\yyirulestack + \appendr\oneimplicitrule{\noexpand\implicitterm{#1}{#2}}% +} + +\def\makeisymnames#1:#2\end{% + \tempcc=#2\relax + \def\next{#1}% + \ifx\next\empty + \relax + \else + \let\imn\assigninames + \fi + #1% +} + +\def\assigninames#1{% + \yypop\yyirulestack\into\toksc + \def\sts##1{##1}% + \savehcs{symn}{\term\lhs\implicitterm\onerule}% + \edef\next{\noexpand\onerule{\the\toksc}}% + \edef\next{\toksc{\next}}\next + \toksb{\implicitrule[#1]:\def}% + \yyreplacestring\toksb\in\newsymswitch\with\toksc + \restorecs{symn}{\term\lhs\implicitterm\onerule}% +} + +\def\oneruleid#1#2{% + \oneimplicitrule{}% + \yyinitstack\yyirulestack + \toksb{#1}% + #1% + \makeisymnames#2\end +} + +\def\termexsymname#1#2{% + \appendr\oneimplicitrule{\noexpand\term{#1}{#2}}% +} + +\def\iruleplaceholder#1:\def#2{} + +\def\setexplicitinlinerules#1{% + \let\onerule\oneruleid + \let\term\termexsymname + \let\lhs\eattwo + \let\implicitterm\itermstack + \let\ruleor\relax + \let\ih\eatone + \let\imn\eatone + \let\implicitrule\iruleplaceholder + \the#1% set explicit names +} + +\catcode`\^^A=\tempca + +\def\makesymrefs#1{% #1 is the symbolic rule switch + \let\ruleor\or + \nameflagtoks{}% + \edef\next{\setsncommands{\noexpand\or\space\harmlesscomment\space (rule 0)^^J% + \noexpand\or\space\harmlesscomment\space (rule 1)^^J}}\next + \edef\next{\unsetsncommands{\noexpand\or\space\harmlesscomment\space (rule 0)^^J% + \noexpand\or\space\harmlesscomment\space (rule 1)^^J}}\next + \yyn=\tw@ + \loop + \yylen=\fgetelemof{yyrtwo}\at\yyn\relax + \ifnum\yylen>\z@ + \else + \yylen=\fgetelemof{yyrthree}\at\yyn\relax + \fi + \xm@kesymrefs#1% + \xm@k@symrefs#1% + \addseparator + \ifnum\yyn<\YYNRULES + \advance\yyn\@ne + \repeat +} + +% these macros are here to hide \else and \fi tokens + +\def\xm@kesymrefs#1{\expandafter\m@kesymrefs\expandafter\yyn\the#1\else\fi} % explicit names +\def\xm@k@symrefs#1{\expandafter\m@k@symrefs\expandafter\yyn\the#1\else\fi} % implicit names + +\def\m@kesymrefs#1{% + \let\onerule\rulesymsetup + \ifcase#1\or\or % skip the first two rules +} + +\def\m@k@symrefs#1{% + \let\onerule\r@lesymsetup + \ifcase#1\or\or % skip the first two rules +} + +\def\addseparator{% + \sprintrule\yyn\to\toksa + \edef\next{\setsncommands{\the\setsncommands\noexpand\or\space\harmlesscomment\the\toksa^^J}}\next + \edef\next{\unsetsncommands{\the\unsetsncommands\noexpand\or\space\harmlesscomment\the\toksa^^J}}\next + \the\nameflagtoks + \nameflagtoks{}% +} + +\def\rulesymsetup#1#2{% + \let\term\termsymname + \let\lhs\lhssymname + \let\implicitterm\term + \tempca\@ne + #1% set explicit names +} + +\newtoks\setsncommands +\newtoks\unsetsncommands + +\def\termsymname#1#2{% set the symbolic name, if it is nonempty + \def\next{#2}% + \ifx\next\empty + \else % there is a symolic name + \expandafter\ifx\csname$[#2]\endcsname\relax% i$ this name unassigned? + \setnameflag{#2}\end % ... this name has been explicitly set + \setuprefs{#2}{\the\tempca}% + \else + \errmessage{ambiguous symbolic name: #2}% + \fi + \fi + \advance\tempca\@ne +} + +\newtoks\nameflagtoks + +\def\setnameflag#1#2{% + \toksa\expandafter{\csname$[#1]sym\endcsname}% $ + \edef\next{\let\the\toksa\noexpand#2}\next + \appendr\nameflagtoks{\noexpand\unsetnameflag{#1}}% +} + +\def\unsetnameflag#1{% + \toksa\expandafter{\csname$[#1]sym\endcsname}% $ + \edef\next{\let\the\toksa\relax}\next +} + +\def\unsetsymname#1{% + \toksa\expandafter{\csname$[#1]\endcsname}% $ + \edef\next{\let\the\toksa\relax}\next +} + +\def\unsetsym#1{% + \toksa\expandafter{\csname$[#1]\endcsname}% $ + \toksb\expandafter{\csname$$[#1]\endcsname}% $ + \edef\next{\let\the\toksa\relax\let\the\toksb\relax}\next +} + +\def\lhssymname#1#2{% set an explicit symbolic name for the left hand side + \setuplhsref{#2}% + \setnameflag{#2}\end % signal that this name has ben explicitly set +} + +% the code below is just an example. One has to figure out how to record percent symbols and other +% dangerous \TeX\ symbols; it is rather slow, a faster two stage scheme can be envisioned +% where the first stage builds the command while iterating over the rule numbers; this is left as an exercise. + +\let\hc\harmlesscomment +\let\uu\space +\def\setuprefs#1#2{% + \expandafter\let\csname$[#1]\endcsname.% $o the \ifx ... \relax test makes sense ... + \appendr\nameflagtoks{\noexpand\unsetsymname{#1}}% clean it up later + \toksa{}\expandafter\charstonumbers#1\end + \edef\next{\toksb{\space\space\noexpand\setsym{\the\toksa}{#2}\hc^^J\uu\uu\hc\hc\hc\uu#1 --> #2^^J}}\next + \edef\next{\toksc{\space\space\noexpand\unsetsym{\the\toksa}\hc^^J\uu\uu\hc\hc\hc\uu#1 --> \relax^^J}}\next + \concat\setsncommands\toksb + \concat\unsetsncommands\toksc +}% + +\def\setuplhsref#1{% + \def\next{#1}% + \ifx\next\empty + \else + \expandafter\let\csname$[#1]\endcsname.% $o the \ifx ... \relax test makes sense ... + \appendr\nameflagtoks{\noexpand\unsetsymname{#1}}% clean it up later + \toksa{}\expandafter\charstonumbers#1\end + \edef\next{\toksb{\uu\uu\noexpand\setlhs{\the\toksa}\hc^^J\uu\uu\hc\hc\hc\uu#1 --> $$^^J}}\next + \edef\next{\toksc{\uu\uu\noexpand\unsetsym{\the\toksa}\hc^^J\uu\uu\hc\hc\hc\uu#1 --> \relax^^J}}\next + \concat\setsncommands\toksb + \concat\unsetsncommands\toksc + \fi +}% + +\def\sprintrule#1\to#2{% + \tempcc=#1\relax + \tempcd=\fgetelemof{yyrone}\at\tempcc\relax + \edef\ruleline{\space(rule \number#1)\space\fgetelemof{yytname}\at\tempcd:}% + \tempcd=\fgetelemof{yyprhs}\at\tempcc\relax + \tempcc=\fgetelemof{yyrhs}\at\tempcd\relax + \ifnum\tempcc>\m@ne + \fillruleline + \else + \edef\ruleline{\ruleline\space<empty>}% + \fi + #2\expandafter{\ruleline}% +} + +\def\fillruleline{% + \edef\ruleline{\ruleline\space\fgetelemof{yytname}\at\tempcc}% + \advance\tempcd\@ne + \tempcc=\fgetelemof{yyrhs}\at\tempcd\relax + \ifnum\tempcc>\m@ne + \let\next\fillruleline + \else + \let\next\relax + \fi + \next +} + +\def\r@lesymsetup#1#2{% + \let\term\termsymextra + \let\lhs\lhssymextra + \let\implicitterm\term + \tempca=\@ne + #1% set implicit names +} + +\def\termsymextra#1#2{% set an implicit symbolic name: + % 1) if it has not been defined yet, define it + % 2) if it is defined implicitly, make it \def + \expandafter\let\expandafter\next\csname$[#1]\endcsname% $ this is the name we are trying to define + \ifx\next\relax % it is not defined yet + \setuprefs{#1}{\the\tempca}% + \else % already defined + \expandafter\ifx\csname$[#1]sym\endcsname\end % it i$ an existing explicit symbolic name, done + \else + \expandafter\ifx\csname$[#1]sym\endcsname\noindent % it i$ an existing implicit symbolic name of lhs, overwrite it + \setuprefs{#1}{\the\tempca}% + \else + \edef\next{\toksa{\space\space\noexpand\locksymname{#1}\harmlesscomment^^J}}\next + \edef\next{\toksb{\space\space\noexpand\unlocksymname{#1}\harmlesscomment^^J}}\next + \concat\setsncommands\toksa % symbolic name defined implicitly, disallow it + \concat\unsetsncommands\toksb + \fi + \fi + \fi + \advance\tempca\@ne +} + +\def\locksymname#1{\expandafter\let\csname$[#1]\endcsname\def} %$ +\def\unlocksymname#1{\expandafter\let\csname$[#1]\endcsname\relax} %$ + +\def\lhssymextra#1#2{% set the name of the left hand side implicitly + \expandafter\let\expandafter\next\csname$[#1]\endcsname%$ + \ifx\next\relax % not defined yet + \setuplhsref{#1}% + \setnameflag{#1}\noindent % this lhs name has been implicitly set + \fi % it is already defined +} + +\catcode`\^^A=\tempca + +% stack variable access + +\def\ym#1){% + \ifnum#1<\@ne + \putyyvalt + \else + \ifnum#1>\yyilen + \errmessage{parameter out of range (#1\space out of\space \the\yyilen)}% + \else + \expandafter\putyytoksx\csname$$'#1\endcsname + \fi + \fi +} + +\def\yn#1#{% + \ifnum#1<\@ne + \putyyval + \else + \ifnum#1>\yyilen + \errmessage{parameter out of range}% + \else + \expandafter\putyyassignment\csname$'\number#1\endcsname%$ + \fi + \fi +} + +\def\yx#1[{% + \expandafter\ifx\csname$[#1]\endcsname\yyval%$ + \putyyvalt + \else + \expandafter\ifx\csname$[#1]\endcsname\def%$ + \errmessage{reference to ambiguous symbolic name: #1}% + \else + \expandafter\putyytoksx\csname$$[#1]\endcsname + \fi + \fi +} + +\def\yz#1]{% + \expandafter\ifx\csname$[#1]\endcsname\yyval%$ + \putyyval + \else + \expandafter\ifx\csname$[#1]\endcsname\def%$ + \errmessage{reference to ambiguous symbolic name: #1}% + \else + \expandafter\putyyassignment\csname$[#1]\endcsname%$ + \fi + \fi +} + +\def\putyyval\else#1\fi\fi{% + \fi\yyvalx +} + +\def\yyvalx#1{% + \edef\next{\yyval{#1}}\next +} + +\def\putyyvalt\else#1\fi\fi{% + \fi\yyval +} + +\def\putyyassignment#1\fi\fi{% + \fi\fi\p@tyyassignment#1% +} + +\def\putyytoksx#1\fi\fi{% + \fi\fi#1% +} + +\def\p@tyyassignment#1#2{% + \yystringempty{#2}{#1}{#2\expandafter{#1}}% +} + +% natural order stack access macros + +\def\p@twwassignment#1#2{% + \edef\sts{\noexpand\p@@wwassignment{\expandafter\xdecrement\expandafter{\number#1}}}% + \expandafter\def\expandafter\sts\expandafter{\sts{#2}}% + \yyvsa\empty +} + +\def\p@@wwassignment#1#2#3{% + \ifnum#1=\z@ %we have got to the element we need + \yybreak{\yystringempty{#2}{#3}{#2{#3}}\let\sts\eatone}% + \else + \yybreak{\edef\sts{\noexpand\p@@wwassignment{\xdecrement{#1}}}% + \expandafter\def\expandafter\sts\expandafter{\sts{#2}}}% + \yycontinue +} + +\def\yy#1{% + \ifx#1[% + \yybreak{\yz}% + \else + \ifx#1]% + \yybreak@{\yx}% + \else + \ifx#1(% + \yybreak@@{\ym}% + \else + \yybreak@@{\yn#1}% + \fi + \fi + \yycontinue +} + +% a macro to access the value stack in direct order, i.e.\ from right to left +% no \yylen is necessary + +\def\bb#1#{% + \ifnum#1<\@ne + \yybreak{\putyyval}% + \else + \yybreak{\p@twwassignment{#1}}% + \yycontinue +} + +% symbolic name macros that prepare the environment for the use of the stack access macros above + +\def\setsymcs#1#2{% + \toksa{}\numberstochars#1\end + \toksb\expandafter{\csname$[\the\toksa]\endcsname}% $ + \toksc\expandafter{\csname$'#2\endcsname}% $ + \edef\next{\let\the\toksb\the\toksc}\next +} + +\def\setsymtr#1#2{% + \toksa{}\numberstochars#1\end + \toksb\expandafter{\csname$$[\the\toksa]\endcsname}% + \toksc\expandafter{\csname$$'#2\endcsname}% + \edef\next{\let\the\toksb\the\toksc}\next +} + +\def\setlhs#1{% + \toksa{}\numberstochars#1\end + \toksb\expandafter{\csname$[\the\toksa]\endcsname}% $ + \edef\next{\let\the\toksb\yyval}\next +} + +\def\setsym#1#2{% + \setsymcs{#1}{#2}% + \setsymtr{#1}{#2}% +} diff --git a/support/splint/tex/yynested.sty b/support/splint/tex/yynested.sty new file mode 100644 index 0000000000..db725abfc5 --- /dev/null +++ b/support/splint/tex/yynested.sty @@ -0,0 +1,85 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% nested and boundary parsing macros + +\newcount\boundarylower +\newcount\boundaryupper + +% for parsers that use short setup + +\boundaryupper-\@M +\boundarylower-\@M + +\expandafter\def\expandafter\parserdatainit\expandafter{% + \parserdatainit + \boundaryupper-\@M + \boundarylower-\@M +} + +% the next sequence is just a reminder of what would have to be done if +% reentrancy is required; the current state saving mechanism is intended for +% bare unoptimized parsers exclusively and does not handle any constant saving + +\expandafter\def\expandafter\savefullstateextra\expandafter{% + \savefullstateextra + % save \boundaryupper and \boundarylower +} + +\def\inputboundarynext#1{% + \yybytepure{ }\yycp@\boundarychar + #1\getstackpointer\yyvsa\relax + \yybyte\expandafter{\the\yybyte{#1}}% + \yyreturn % inserting something here will effectively insert it into the input stream +} + +\newcount\vspointer + +% to use while debugging, the sequence below displays the contents of the value stack + +\def\yyactione{% + \message{value stack at: \getstackpointer\yyvsa}% + \toksa\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter{\gettopofstackcs\yyvsa}% + \toksb\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter\expandafter{\getmidstackcs\yyvsa{0}}% + \message{on the stack: \the\toksa, bottom: \the\toksb}% + \message{the value: \the\yyval}% + \message{rule: \the\yyn, length: \the\yylen}% +} + +% this is an example of boundary processing + +\def\yyaction{% + \vspointer=\getstackpointer\yyvsa\relax + \advance\vspointer-\yylen + \ifnum\vspointer<\boundarylower + \ifnum\boundaryupper>-\@M % all nested tokens will be reduced + \boundaryupper=-\@M + \boundarylower=-\@M + \message{reduction...(\the\vspointer: \the\yy(1).\the\yy(2).)}% + \printrule\yyn + \else % pre boundary tokens are getting reduced + \boundarylower=\vspointer + \advance\boundarylower\@ne + \fi + \else + \ifnum\vspointer<\boundaryupper % top nested tokens will be reduced + \boundaryupper=\vspointer + \fi + \fi + \yysymswitch{\yyn}% setting symbol names + \yybigswitch{\yyn}% + \yysymcleanup{\yyn}% removing symbol names from the namespace +} diff --git a/support/splint/tex/yyparse.sty b/support/splint/tex/yyparse.sty new file mode 100644 index 0000000000..ce5ab8b5e3 --- /dev/null +++ b/support/splint/tex/yyparse.sty @@ -0,0 +1,319 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% global variables: current token, state, variable to store various table indices + +\def\yyparse{% + \yylen=\z@ % + \yyinitstack\yyssa % state stack + \yyinitstack\yyvsa % data stack + \yystate=\z@ % + \yychar=\YYEMPTY\relax + \yylval{}% bogus value + \yynewstate +} + +\newif\iftracestacks +\newif\iftraceparserstates +\newif\iftracerules + +\def\yynewstate{% + \ifnum\yystate=\YYFINAL\relax + \yybreak\yypaccept + \else + \yypushr\yystate\on\yyssa + {% + \iftraceparserstates + \derrmessage{^^Jstate \the\yystate\iftracestacks, \fi}% + \fi + \iftracestacks + \iftraceparserstates + \else + \derrmessage{^^J}% + \fi + \showstack\yyssa\derrmessage{state stack: \stackcs^^J}% + \fi + }% + \yybreak\yybackup + \yycontinue +} + +\def\yybackup{% + \yyn=\fgetelemof{yypact}\at\yystate\relax + % yyn = yypact[yystate] + \ifnum\yyn=\YYPACTNINF\relax % if (yyn == YYPACT_NINF) (if(yypact_value_is_default(yyn))) + \yybreak\yydefault % goto yydefault + \else % else + \ifnum\yychar=\YYEMPTY\relax % if (yychar == YYEMPTY) + % ... ... if desired, this is where `hidden context' may be inserted + % ... ... for example: + % ... ... \gethidden\yychar + % ... ... \ifnum\yychar=\YYEMPTY\relax + % ... ... \yybreak@@\yylex + % ... ... \else + % ... ... \yybreak@@\yyparsetail + % ... ... \fi + % ... ... here \gethidden#1 is a hidden context-extracting macro that + % ... ... takes a token value from a special stream and puts it in #1 + % ... ... filling this special stream should be done by the lexer, + % ... ... respecting any state changes required. + \yybreak@\yylex % yychar = YYLEX + \else + \yybreak@\yyparsetail + \fi + \yycontinue +} + +\newif\iftracelookahead + +\def\yyparsetail{% this is where the lexer returns + \ifnum\yychar>\YYEOF\relax % if (yychar > YYEOF) + \yytoken=\fgetelemof{yytranslate}\at\yychar\relax + \iftracelookahead + \derrmessage{lookahead: \tokname, value: \the\yylval^^J}% + \fi + % yytoken = yytranslate[yychar] + \else % else + \yychar=\YYEOF\relax % yychar = yytoken = YYEOF + \yytoken=\YYEOF\relax + \fi% valid lookahead token + \advance\yyn by\yytoken % yyn = yypact[yystate] + yytoken + \ifnum\yyn<\z@ + \yybreak\yydefault + \else + \ifnum\yyn>\YYLAST\relax + \yybreak@\yydefault + \else + \ifnum\fgetelemof{yycheck}\at\yyn=\yytoken + % if (yyn >= 0 && yyn <= YYLAST && + % yytoken == yycheck[yyn]) + \yyn=\fgetelemof{yytable}\at\yyn\relax + % yyn = yytable[ yypact[yystate] + yytoken ] + \ifnum\yyn>\z@ % if (yyn > 0) ...shift + \iftraceactions + \derrmessage{shifting \tokname^^J}% + \fi + \yystate=\yyn % yystate = yytable[ yypact[yystate] + yytoken ] + \yychar=\YYEMPTY\relax + % yychar = YYEMPTY ...drop + % ...the + % ...shifted token + \yypushr\yylval\on\yyvsa + % ...shift the value + % *++yyvsa = yylval + \yybreak@@@\yynewstate % new state will be shifted + \else % ...if (yyn < 0) + \iffalse % if (yytable_value_is_error(yyn)) + % \ifnum\yyn=\YYTABLEERROR\relax + \yybreak@@@@\yyerr + \else + \yyn=-\yyn + \yybreak@@@@\yyreduce + \fi + \fi + \else + \yybreak@@\yydefault% goto yydefault + \fi + \fi + \yycontinue +} + +\newif\iftraceactions + +\def\yydefault{% + \yyn=\fgetelemof{yydefact}\at\yystate\relax + % yyn = yydefact[yystate] + \iftraceactions + \derrmessage{default action^^J}% + \fi + \ifnum\yyn=\z@ + \xskiptofi\yyerr + \else + \xskiptofi\yyreduce % if (yyn != 0) goto yyreduce + \fi +} + +\def\yyreduce{% + \yylen=\fgetelemof{yyrtwo}\at\yyn\relax + \iftracerules + \derrmessage{^^Jreducing (rule \the\yyn)}\printrule{\yyn}% + \fi + {% + \iftracestacks + \iftracerules + \else + \derrmessage{^^J}% + \fi + \showstack\yyssa + \derrmessage{stack: \stackcs^^Jpopping \the\yylen^^J}% + \fi + }% + \yyilen=\fgetelemof{yyrthree}\at\yyn\relax + \ifnum\yyilen>\z@ % this is an inline rule + \ifnum\yylen>\z@ + \errmessage{an inline rule with a nontrivial left hand side: rule \the\yyn}% + \fi + \else + \yyilen\yylen + \fi + \ifnum\yyilen>\z@ + \yypeekvstack\yyvsa\upto\yyilen + \yyval\csname $$'1\endcsname % $$ = $1 + \fi + % + \yyaction % value + % + \yypopstack\yyvsa\by\yylen + \yypopstack\yyssa\by\yylen % YYPOPSTACK(yylen) + \yypushr\yyval\on\yyvsa + % ...push the computed + % ...value on the value stack + % + \yyn=\fgetelemof{yyrone}\at\yyn\relax + % yyn = yyr1[yyn] + % + \yylen\yyn % + \advance\yylen-\YYNTOKENS\relax % ...reuse yylen + \yystate=\fgetelemof{yypgoto}\at\yylen\relax + % yystate = yypgoto[ yyn - YYNTOKENS ] + \yyreadstackr\yyssa\at\z@\to\yyilen % + % ...reuse yyilen + \advance\yystate\yyilen % yystate = yypgoto[ yyn - YYNTOKENS ] + yyssa[0] + % + \ifnum\yystate<\z@ % if (yystate < 0) + \yystate=\fgetelemof{yydefgoto}\at\yylen\relax + % yystate = yydefgoto[ yyn - YYNTOKENS ] + \else % else + \ifnum\yystate>\YYLAST\relax % ... + \yystate=\fgetelemof{yydefgoto}\at\yylen\relax + \else + \ifnum\fgetelemof{yycheck}\at\yystate=\yyilen + % if (yystate >= 0 && yystate <= YYLAST && yyssa[0] == yycheck[yystate]) + % + \yystate=\fgetelemof{yytable}\at\yystate\relax + % yystate = yytable[yystate] + \else + \yystate=\fgetelemof{yydefgoto}\at\yylen\relax + \fi + \fi + \fi + \yylen=\z@ + \yynewstate +} + +\newif\iftraceparseresults + +\def\yypaccept{% + \iftraceparseresults + \derrmessage{accepted^^J}% + \fi + \finishparse +} + +\def\yyerr{% + \iftraceparseresults + \derrmessage{(parse) error^^J}% + \fi + \cleanupparse +} + +% no symbolic name support by default + +\let\yysymswitch\eatone +\let\yysymcleanup\eatone + +\def\derrmessage#1{{% + \newlinechar=`\^^J% + \edef\tokname{\fgetelemof{yytname}\at\yytoken}% + \message{#1}% +}} + +% these macros reuse dedicated counters ... locally + +\def\printrule#1{{% some values are computed several times to save on register assignments + % this has more to do with not polluting the namespace than speed + \edef\ruleline{\space\space\space\fgetelemof{yytname}\at{\fgetelemof{yyrone}\at{#1}}:}% + \yyn=\fgetelemof{yyprhs}\at{#1} + \ifnum\fgetelemof{yyrhs}\at\yyn>\m@ne + \loop + \edef\ruleline{\ruleline\space\fgetelemof{yytname}\at{\fgetelemof{yyrhs}\at\yyn}}% + \advance\yyn\@ne + \ifnum\fgetelemof{yyrhs}\at\yyn>\m@ne + \repeat + \else + \edef\ruleline{\ruleline\space<empty>}% + \fi + \message{ -->\ruleline}% +}} + +% print the rule without using yyprhs and yyrhs (necessary if using +% bison version 3.0 or above) + +\def\printrule#1{{% + \edef\ruleline{\space\space\space\fgetelemof{yytname}\at{\fgetelemof{yyrone}\at{#1}}:}% + \ifnum\yylen>\z@ + \yypeeksstack\yyssa\upto\yylen\withprefix{\appendtoruleline}% + \else + \edef\ruleline{\ruleline\space<empty>}% + \fi + \message{ -->\ruleline}% +}} + +\def\appendtoruleline#1{% + \edef\ruleline{\ruleline\space\fgetelemof{yytname}\at{\fgetelemof{yystos}\at{#1}}}% +} + +% the two control sequences below are replaced in yyfaststack.sty +% to match the appropriate stack implementation + +\def\showstacksym#1{% + \edef\stackcs{\stackcs^^J \space\space\fgetelemof{yytname}\at{\fgetelemof{yystos}\at{#1}}}% +} + +\def\showstack#1{% + \def\stackcs{}% + \let\sts\showstacksym + #1% +} + +\newif\iftracediscardedinput + +\long\def\cleanupparse#1\endparse{% + \iftracediscardedinput + \immediate\write16{discarding the rest of the input}% + \fi + \yyerror +} + +\newif\ifyyparsefail + +\def\yyerror{\yyparsefailtrue} + +\long\def\finishparse#1\endparseinput{% + \yystringempty{#1}{}{\yylval{#1}\immediate\write16{skipping: \the\yylval}}% +} +\let\endparse\relax +\let\endparseinput\relax + +% action code + +\def\yyaction{% + \yysymswitch{\yyn}% setting symbol names + % the big switch, set \yyval + \yybigswitch{\yyn}% + % + \yysymcleanup{\yyn}% removing symbol names from the namespace +} diff --git a/support/splint/tex/yystype.sty b/support/splint/tex/yystype.sty new file mode 100644 index 0000000000..e37b298590 --- /dev/null +++ b/support/splint/tex/yystype.sty @@ -0,0 +1,65 @@ +% Copyright 2012-2014, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% extra types for the bison scanner + +\def\lexspecialchar#1#2#3{\immediate\write16{encountered: \noexpand#1 fmark: #2, smark: #3}} + +\def\newlexerstateextra{% + \setnulstack{obstackforstring}% + \setnulstack{obstackforstringraw}% +} + +\def\savefullstateextra{% + \savestatelist{obstackforstring}% + \savestatelist{obstackforstringraw}% +} + +% lexer environment + +\def\yycomplain#1{\immediate\write16{#1}} % lexer errors +\def\yypdeprecated#1{\errmessage{option: #1 is ignored}} % outdated options + +\def\STRINGGROW{% + \concat\currentlaststring\yytextpure + \concat\currentlaststringraw\yytext} + +\def\STRINGFINISH{% + \laststring\currentlaststring + \laststringraw\currentlaststringraw + \yypushr\currentlaststring\on\obstackforstring + \yypushr\currentlaststringraw\on\obstackforstringraw + \currentlaststring{}% + \currentlaststringraw{}% +} +\def\STRINGFREE{% + \yypopstack\obstackforstring\by\@ne + \currentlaststring{}% + \yypopstack\obstackforstringraw\by\@ne + \currentlaststringraw{}% +} + +\def\bracketedidstr{} + +\newcount\contextstate % keeping track of the previous lexer state +\newcount\bracketedidcontextstate % similar but in the context of bracketed identifiers +\newcount\lonesting % nesting level for tags +\newcount\percentpercentcount + +\newtoks\laststring +\newtoks\laststringraw +\newtoks\currentlaststring +\newtoks\currentlaststringraw diff --git a/support/splint/tex/yytexlex.sty b/support/splint/tex/yytexlex.sty new file mode 100644 index 0000000000..a7e7dc3250 --- /dev/null +++ b/support/splint/tex/yytexlex.sty @@ -0,0 +1,1145 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% parsing \TeX\ input for pretty-printing; +% scanning and parsing are both done by using a low-level +% finite automaton; + +\def\yygetchar{% single buffered input + \ifyytextbackup + \yybyte\yytextseen + \yycp@\yytextseenlastchar + \yytextbackupfalse + \let\next\yyreturn + \else + \let\next\yyinput + \fi + \next +} + +% \TeX\ input + +\def\texdefaultstate{% +<>+-()^~!?*[]:.,`'\%\$\_ +\raw digit letter \raw +{% + \putother\yycp@\in\yybyte + \appendyybyte} += +{% + \appendr\texlinetoks{{}${}={}${}}% + \yygetchar +} +\{ +{% + \putother\yycp@\in\yybyte + \appendl\yybyte{\noexpand\bbal[\the\bbalance]}% + \advance\bbalance1\relax + \appendyybyte} +\} +{% + \putother\yycp@\in\yybyte + \advance\bbalance-1\relax + \appendl\yybyte{\noexpand\bbal[\the\bbalance]}% + \appendyybyte} +0123456789 +{% + \expandafter\ifcat\expandafter\noexpand\the\yybyte1% + \def\next{digit}% + \else + \def\next{^^A}% + \fi + \action\next\in\texdefaultstate +} +^^A % \yycp@==1 +{% + \edef\next{\noexpand\noexpand\the\yybyte}% + \action\next\in\texdefaultstate} +\raw\hbox\raw +{% + \appendnext} +\raw\end\raw +{% + \relax} +\raw\\ \_ \raw +{% + \appendyybyte} +\ \ +{% + \appendyybyte} +\\ +{% + \getcescape +} +/ +\raw escape \raw +{% + \let\currentstate\texcsname + \yygetchar} +}% + +\def\texcsname{% +<>+-()^~!?*\{\}[]:.,`'=\%\$\_/ +0123456789 +^^A +\ \ +\raw \end \\ \_ \ \raw +\raw escape \raw +{% + \let\currentstate\texdefaultstate + \putother\yycp@\in\yytext + \outputtexcs + \yygetchar} +% +\\ +{% + \getcescape +} +\raw letter \raw +{% + \let\currentstate\texcsnamelong + \yytext\yybyte + \yygetchar} +}% + +\def\texcsnamelong{% +<>+-()^~!?*\{\}[]:.,`'=\%\$\_\& +0123456789 +^^A +\ \ +\raw \end \\ \_ \raw +{% + \let\currentstate\texdefaultstate + \yytextseenlastchar=\yycp@ + \yytextseen=\yybyte + \yytextbackuptrue + \outputtexcs + \yygetchar} +\\ +{% + \getcescape +} +/ +\raw escape \raw +{% + \let\currentstate\texcsname + \outputtexcs + \yygetchar} +\raw letter \raw +{% + \concat\yytext\yybyte + \yygetchar} +}% + +\def\texcescape{% +\\ +{% + \let\currentstate\esccurrentstate + \yycp@=`\\% + \putother\yycp@\in\yybyte + \def\next{escape}% + \action\next\in\currentstate} +} + +\def\texescdefault{% + \let\currentstate\esccurrentstate + \yycp@=`\ % + \yybyte{\ }% + \action\yycp@\in\currentstate +} + +\def\getcescape{% + \let\esccurrentstate\currentstate + \let\currentstate\texcescape + \let\default\texescdefault + \yygetchar +} + +\setspecialcharsfrom\texdefaultstate +\setspecialcharsfrom\texcsname +\setspecialcharsfrom\texcsnamelong +\setspecialcharsfrom\texcescape + +\newtoks\texlinetoks % parsed \TeX\ tokens +\newtoks\texidxtoks % indexing commands + +\def\texlexer{% + \let\default\yygetchar + \let\next\yycp@ + \ifnum\yycp@>"20 % + \ifnum\yycp@<"7F % a character + \expandafter\ifcat\expandafter\noexpand\the\yybyte a% + % this test allows one some control over what constitutes a control + % sequence name (by saying, for example, \let\$a or manipulating \catcodes) + % this is bordering on a bug + \def\next{letter}% + \fi + \fi + \fi + \action\next\in\currentstate +} + +\def\outputtexcs{% + \expandafter\ifx\csname\the\yytext\defTpostfix\endcsname\relax + \putother{`\\}\in\toksa + \concat\texlinetoks\toksa + \concat\texlinetoks\yytext + \appendr\texlinetoks{\noexpand\hbox{$\noexpand\,$}}% + \else + \appendr\texlinetoks{\expandafter\expandafter\expandafter\noexpand\expandafter\csname\the\yytext xTeXmode\endcsname}% + \fi + \appendr\texidxtoks{% + \termindex{{\nx\nx\nx\texcsstring{\expandafter\charstonumberse\the\yytext\end}}{\texnspace}{\nx\the\pageno}{\secno}}% + }% +} + +% the `calling conventions' for indexing \TeX\ control sequences are too different to use canned indexing macros +% (the second parameter is a register so it has to be expanded before the \charstonumberse processes it) +% we therefore invoke the macros by hand + +\def\texidxdomain{T} + +\def\texcsidxrank{2} + +\def\writetexidxentry#1{\iftermindex\write\gindex{\nx\nx\expandafter\nx\csname\texidxdomain TI\endcsname\texcsidxrank#1}\fi} + +\def\appendyybyte{\concat\texlinetoks\yybyte\yygetchar} +\def\appendnext#1{\toksa{#1}\concat\texlinetoks\toksa\yygetchar} + +\def\defTpostfix{xTeXmode} + +\def\stripyybyte{% + \expandafter\stripyyb@te\the\yybyte +} + +\def\stripyyb@te#1#2{\noexpand#2} + +% TODO: +% the macros below are in their present shape due to the forces of `historical evolution': +% \TeX\ pretty printing needs to be completely rewritten, however, the existing macros +% do a passable job so they remain in this rather random shape. +% the items below need more urgent attention: +% +% o introduce TeXn_ that take a number and a string argument + +\newcount\bbalance +\newif\iftracetexpp +\newtoks\textoks + +\def\TeXx(#1)#2;{% TODO + \let\oldttdot\.\relaxcweb + \textoks{}\let\.\dotcollect + #1% + \restorecweb\let\.\oldttdot + \expandafter\T@Xx\the\textoks} + +\def\mypar{\par} + +\def\TeXb(#1)#2;{% TeX material begin + {}$% + \let\oldsix\6% + \let\6\ignorespaces + \let\oldC\C + \let\C\saveCcomments + \let\oldttdot\.\relaxcweb + \textoks{}\toksa{}\let\.\dotcollectb + #1% + \restorecweb\let\.\oldttdot + \dotcollectstripquotes + ${}} + +\def\TeXa(#1)#2;{% TeX material add + {}$% + \let\oldttdot\.\relaxcweb + \toksa{}\let\.\dotcollectb + #1% + \restorecweb\let\.\oldttdot + \dotcollectstripquotes + ${}} + +\def\TeXf(#1)#2;{% TeX material add + {}$% + \let\oldttdot\.\relaxcweb + \toksa{}\let\.\dotcollectb + \textoks\expandafter{\the\textoks\hbox{\6}}% + #1% + \restorecweb\let\.\oldttdot + \dotcollectstripquotes + ${}} + +\def\TeXo(#1)#2;{% TeX material output + \let\oldttdot\.\relaxcweb + \let\.\dotcollect + \let\6\oldsix + \let\C\oldC + #1% + \restorecweb\let\.\oldttdot + $\expandafter\T@Xx\expandafter"\the\textoks"$} + +\def\TeXao(#1)#2;{% TeX material output + {}$\let\oldttdot\.\relaxcweb + \toksa{}\let\.\dotcollectb + \let\6\oldsix + \let\C\oldC + #1% + \restorecweb\let\.\oldttdot + \dotcollectstripquotes + $\expandafter\T@Xx\expandafter"\the\textoks"${}${}} + +\def\TeXfo(#1)#2;{% TeX material output + {}$\let\oldttdot\.\relaxcweb + \toksa{}\let\.\dotcollectb + \textoks\expandafter{\the\textoks\hbox{\6}}% + \let\6\oldsix + \let\C\oldC + #1% + \restorecweb\let\.\oldttdot + \dotcollectstripquotes + $\expandafter\T@Xx\expandafter"\the\textoks"${}${}} + +\let\TeXxi\TeXx +\def\inlineTeXx#1{$\TeXxi(\.{"#1"});$} % for indexing macros + +\def\dotcollect#1{\toksa{#1}\concat\textoks\toksa} +\def\dotcollectb#1{\toksb{#1}\concat\toksa\toksb} +\def\dotcollectstripquotes{\expandafter\d@tcollectstripquotes\the\toksa} +\def\d@tcollectstripquotes"#1"{\toksa{#1}\concat\textoks\toksa} +\def\saveCcomments#1{\toksc{\hbox{\oldC{#1}}}\concat\textoks\toksc} +\def\relaxcweb{\savehcs{local-namespace}{\)}} +\def\restorecweb{\restorecs{local-namespace}{\)}} + +\def\T@Xx"#1"{% + {}${}%$ + \iffalse{\fi % alignment! + \begingroup + % tune up the standard input routines + \let\yyreturn\texlexer + \let\multicharswitch\empty + \let\onecharswitch\empty + \setspecialcharsfrom\multicharswitch % debugging + \setspecialcharsfrom\onecharswitch + % + \let\currentstate\texdefaultstate + \texlinetoks{}\texidxtoks{}\bbalance\z@ + \let\bbal\bbalempty + \yytextbackupfalse + \restorecsxlist\texnspace\alltexsymbols + \let\termindex\writetexidxentry + \yygetchar#1\end + \toksc{#1}% + \iftracetexpp{\newlinechar=`^^J% + \errmessage{TeX_ input: \the\toksc^^JTeX_ first pass: \the\texlinetoks}}\fi + \ifnum\bbalance=\z@ + \else + \bbbalance + \fi + \iftracetexpp\errmessage{TeX_ final pass: \the\texlinetoks}\fi + \concat\texlinetoks\texidxtoks + \expandafter + \endgroup + \expandafter + \begingroup +% the actual typesetting must happen outside the group +% otherwise there is a risk that an output routine is called before +% the group is complete and \yyreturn definition is wrong so \yyparse +% will not be able to function + \expandafter\texlinetoks\expandafter{\the\texlinetoks}% + \let\bbal\bbalempty\bbalance\z@ + \restorecsxlist\texnspace\alltexsymbols + \let\termindex\writetexidxentry\tt\chardef\_=`\_\the\texlinetoks + \endgroup + \iffalse}\fi + {}${}%$ +} + +\def\alltexsymbols{% + \space\toksa\toksb\toksc\tokse\toksf\the\ifx\ifnum\fi\else + \def\edef\let\empty\next\switchon\in\concat\appendr\default + \noexpand\emptyterm\print\relax\yy\inmath\omit\hfil\getfirst + \getsecond\getthird\getfourth\getfifth\nx\to\hspace\toksd\rhscont + \rhscnct\rhsbool\table\ifrhsfull\rhsfulltrue\rhsfullfalse + \yyval\tempca\tempcb\z@\@ne\tw@\m@ne\advance\ifcat\iftracebadchars + \bb\yylexreturnptr\yylexreturn\yylexreturnval\yylexreturnchar + \yylexreturntext\yylexnext +} + +\def\collectspaces#1{% + \def\next{#1}% + \ifx#1\endcontainer + \let\next\relax + \else + \ifx\next\spacecontainer + \appendr\toksa\next + \let\next\collectspaces + \else + \let\next\eattoend + \fi + \fi + \next +} + +\def\spacecontainer{\ } + +\def\texnspace{texline} + +\let\defpostfix\defTpostfix +\let\defprefix\empty + +\let\defypreamble\empty % for indexing macros +\def\defypostamble{ {\rm(\.{\\\thisnamex})}} + +\let\settgroup\relax + +\defx\space{\hbox{$\,$\char`\ $\,$}}{texline} + +\defx\hspace{\hbox{$\,$\char`\ $\,$}}{texline} + +\defy\hspace{\hbox{$\,$\char`\ $\,$}}{index} + +\defx\advance{% + $\mathop{\hbox{\bf add}}{}$% +}{texline} + +\defy\advance{% + {\bf add}% +}{index} + +\defx\z@{% + $\,0_{\rm R}\,$% +}{texline} + +\defy\z@{% + $0_{\rm R}$% +}{index} + +\defx\@ne{% + $\,1_{\rm R}\,$% +}{texline} + +\defy\@ne{% + $1_{\rm R}$% +}{index} + +\defx\tw@{% + $\,2_{\rm R}\,$% +}{texline} + +\defy\tw@{% + $2_{\rm R}$% +}{index} + +\defx\m@ne{% + $\,-1_{\rm R}\,$% +}{texline} + +\defy\m@ne{% + $-1_{\rm R}$% +}{index} + +\defx\tempca{% + ${}t_a{}$% +}{texline} + +\defy\tempca{% + $t_a$% +}{index} + +\defx\tempcb{% + ${}t_b{}$% +}{texline} + +\defy\tempcb{% + $t_b$% +}{index} + +\defx\toksa{% + {}${}v_a{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\toksa{% + ${}v_a{}$% +}{index} + +\defx\toksb{% + {}${}v_b{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\toksb{% + ${}v_b{}$% +}{index} + +\defx\toksc{% + {}${}v_c{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\toksc{% + ${}v_c{}$% +}{index} + +\defx\toksd{% + {}${}v_d{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\toksd{% + ${}v_d{}$% +}{index} + +\defx\tokse{% + {}${}v_e{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\tokse{% + ${}v_e{}$% +}{index} + +\defx\toksf{% + {}${}v_f{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\hbox{\the\toksa}% + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\toksf{$v_f$}{index} + +\defx\yyval{% + {}${}\Upsilon{}${}% + \def\setegroup{}% + \def\setpgroup{% + $\leftarrow\langle\,\the\toksa + \if\next]% + \else + \rangle% + \fi + $}\grabbalanced +}{texline} + +\defy\yyval{% + ${}\Upsilon{}$% +}{index} + +\def\setflexreturn#1{% + \def\setegroup{{\bf return}#1}% + \def\setpgroup{% + {% + \let\texidxdomain\bisonidxdomain + \tokse\toksa + \expandafter\nameproc\expandafter{\the\toksa}% + \ifyyparsefail + \edef\next{\toksa{\termmetastyle{% + \expandafter\gidxentry\expandafter{\expandafter\termvstring\expandafter}\expandafter{\the\toksa}% + \let\nx\idxfont\nx\empty\nx\tt\the\toksa\nx\/% + }}}\next + \else + \expandafter\settermstyle\expandafter{\the\tokse}% + \fi + {}$\mathop{\hbox{{\bf return}#1}}\hbox{\the\toksa}${}}% + }\grabbalanced +} + +\defx\yylexreturnptr{% + \setflexreturn{$_p${}}% +}{texline} + +\defy\yylexreturnptr{% + {\bf return$_p$}% +}{index} + +\defx\yylexreturnchar{% + {\bf return$_c$}% +}{texline} + +\defy\yylexreturnchar{% + {\bf return$_c$}% +}{index} + +\defx\yylexnext{% + {\bf continue}% +}{texline} + +\defy\yylexnext{% + {\bf continue}% +}{index} + +\defx\yylexreturn{% + \setflexreturn{$_l${}}% +}{texline} + +\defy\yylexreturn{% + {\bf return$_l$}% +}{index} + +\defx\yylexreturnval{% + \setflexreturn{$_v${}}% +}{texline} + +\defy\yylexreturnval{% + {\bf return$_v$}% +}{index} + +\defx\yylexreturntext{% + {\bf return$_t$}% +}{texline} + +\defy\yylexreturntext{% + {\bf return$_t$}% +}{index} + +\defx\table{{}${}\Omega{}${}}{texline} + +\defy\table{$\Omega$}{index} + +\defx\relax{\hbox{$\circ$}}{texline} + +\defy\relax{\hbox{$\circ$}}{index} + +\defx\the#1{% + \toksa{#1}% + \let\prevdefault\default + \def\default{\let\default\prevdefault{}$\mathop{\rm val}\,${}#1}% + \taction{\the\toksa}\in\thecomaction +}{texline} + +\defy\the{% + $\mathop{\rm val}\cdot$ {\rm or }% + \lcenclose{$\cdot$}% +}{index} + +\def\thecomaction{% +\raw \toksaxTeXmode\toksbxTeXmode\tokscxTeXmode\toksdxTeXmode\toksexTeXmode\toksfxTeXmode \raw +{% + \let\default\prevdefault + \lcenclose{\the\toksa}% +}% +} + +\setspecialcharsfrom\thecomaction + +\def\lcenclose#1{% + {}$\llcorner\hbox{#1}\lrcorner${}% +} + +\defx\inmath{% + {}$^\bullet${}% + \def\setegroup{}% + \def\setpgroup{% + $(\,\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\inmath{% + $^\bullet(\,\cdot\,)$% +}{index} + +\defx\ifx{% + {\bf if$_{\rm x}\;$}% +}{texline} + +\defy\ifx{% + {\bf if$_{\rm x}$}% +}{index} + +\defx\ifnum{% + {\bf if$_\omega\;$}% +}{texline} + +\defy\ifnum{% + {\bf if$_\omega$}% +}{index} + +\defx\ifcat{% + {\bf if$_{\rm cat}\;$}% +}{texline} + +\defy\ifcat{% + {\bf if$_{\rm cat}$}% +}{index} + +\defx\iftracebadchars{% + {\bf if$_t$ {\rm[{\tt bad char}]}$\;$}% +}{texline} + +\defy\iftracebadchars{% + {\bf if$_t$ {\rm[{\tt bad char}]}}% +}{index} + +\defx\else{% + {\bf else$\;$}% +}{texline} + +\defx\fi{% + {\bf fi$\;$}% +}{texline} + +\defx\def{% + {\bf def$\;$}% +}{texline} + +\defy\def{% + {\bf def}% +}{index} + +\defx\edef{% + {\bf def$_{\rm x}\;$}% +}{texline} + +\defy\edef{% + {\bf def$_{\rm x}$}% +}{index} + +\defx\let{% + {\bf let$\;$}% +}{texline} + +\defx\next{% + {\bf next$\;$}% +}{texline} + +\defx\empty{% + $\emptyset$% +}{texline} + +\defy\empty{% + $\emptyset$% +}{index} + +\defx\ifrhsfull{% + {\bf if {\rm ($\,$rhs${}={}$full$\,$)}$\;$}% +}{texline} + +\defy\ifrhsfull{% + {\bf if {\rm ($\,$rhs${}={}$full$\,$)}}% +}{index} + +\defx\rhsfulltrue{% + {\rm rhs${}={}$full$\,$}% +}{texline} + +\defy\rhsfulltrue{% + {\rm rhs${}={}$full}% +}{index} + +\defx\rhsfullfalse{% + {\rm rhs${}={}$not full$\,$}% +}{texline} + +\defy\rhsfullfalse{% + {\rm rhs${}={}$not full}% +}{index} + +\defx\noexpand{% + $^{\rm nox}$% +}{texline} + +\defx\nx{% + $^{\rm nx}$% +}{texline} + +\defy\noexpand{% + $^{\rm nox}$% +}{index} + +\defy\nx{% + $^{\rm nx}$% +}{index} + +\defx\to{% + ${}\mapsto{}$% +}{texline} + +\defx\emptyterm{% + \hbox{$\ulcorner\ldots\urcorner$}% +}{texline} + +\defy\emptyterm{% + \hbox{$\ulcorner\ldots\urcorner$}% +}{index} + +\defx\hfil{\hbox{$\,\Leftrightarrow\,$}}{texline} +\defx\omit{\hbox{$\,{\times\atop\times}$}}{texline} + +\defx\yy#1{% + \if\noexpand#1]% + \let\next\seeksymtt + \else + \if\noexpand#1[% + \let\next\seeksymts + \else + \if\noexpand#1(% + \let\next\seeknott + \else + \def\next{\seeknots#1}% + \fi + \fi + \fi + \next +}{texline} + +\defy\yy{% + $\Upsilon\kern-1pt{}_{\rm?}$% +}{index} + +\def\seeknots#1\bbal{% + \let\setegroup\relax + \ifnum#1>0\relax + \def\setpgroup{% + \edef\next{\the\toksa}% + \ifx\next\empty + $\Upsilon\kern-1pt{}_{\number\tempca}$% + \else + $\Upsilon\kern-1pt{}_{\number\tempca}\rightarrow[{}${\tt\the\toksa}]$\;$% + \fi}% + \else + \def\setpgroup{% + $\Upsilon\leftarrow\langle{}${\tt\the\toksa}%${}% + \if\next]% + \else + $\rangle\;$% + \fi}% + \fi + \tempca#1\relax\printtermt +} + +\def\seeksymts#1]{% + \let\setegroup\relax + \def\setpgroup{$\Upsilon\kern-1pt{}_{\rm#1}\langle{}${\tt\the\toksa}${}\rangle\;$}\grabbalanced +} + +\def\seeksymtt#1[{{$\Upsilon\kern-1pt{}_{\rm#1}$}} + +\def\seeknott#1){{$\Upsilon\kern-1pt{}_{\rm#1}$}} + +\def\printtermt{\grabbalanced\bbal} + +\defx\print{% + \hbox{\it print$\;$}% + \def\setpgroup{{\tt"}\the\toksa{\tt"}$\,$}% + \def\setegroup{}% + \grabbalanced +}{texline} + +\defx\bb#1\bbal{% + \let\setegroup\relax + \ifnum#1>0\relax + \def\setpgroup{% + \edef\next{\the\toksa}% + \ifx\next\empty + ${}_{\number\tempca}\kern-1.5pt\Upsilon$% + \else + ${}_{\number\tempca}\kern-1.5pt\Upsilon\rightarrow[{}${\tt\the\toksa}$]\;$% + \fi}% + \else + \def\setpgroup{% + $\Upsilon\leftarrow\langle{}${\tt\the\toksa}%${}% + \if\next]% + \else + $\rangle\;$% + \fi}% + \fi + \tempca#1\relax\printtermt +}{texline} + +\defy\bb{% + ${}_{\rm?}\kern-2pt\Upsilon$% +}{index} + +\defx\switchon{% + \hbox{\bf switch$\;$}% + \def\setpgroup{{$($}\the\toksa{$)$}$\,$}% + \def\setegroup{}% + \grabbalanced +}{texline} + +\defy\switchon{% + \hbox{\bf switch$\;$}% +}{index} + +\defx\in{% + $\,\varepsilon\,$% +}{texline} + +\defy\in{% + $\,\varepsilon\,$% +}{index} + +\defx\concat#1#2{% + \hbox{#1${}\leftarrow{}$#1${}+_{\rm s}{}$#2$\;$}% +}{texline} + +\defy\concat{% + \hbox{$A\leftarrow A+_{\rm s}B$}% +}{index} + +\defx\appendr#1{% + \hbox{#1${}\leftarrow{}$#1${}+_{\rm sx}{}$}% + \def\setpgroup{{$[\,$}\the\toksa{$\,]$}$\,$}% + \def\setegroup{}% + \grabbalanced +}{texline} + +\defy\appendr{% + \hbox{$A\leftarrow A+_{\rm sx}B$}% +}{index} + +\defx\default{% + {\tt default}$\;$% +}{texline} + + +\defx\getfirst{% + {}${}\pi_1${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\getfirst{% + ${}\pi_1$% +}{index} + +\defx\getsecond{% + {}${}\pi_2${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\getsecond{% + ${}\pi_2$% +}{index} + +\defx\getthird{% + {}${}\pi_3${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\getthird{% + ${}\pi_3$% +}{index} + +\defx\getfourth{% + {}${}\pi_4${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\getfourth{% + ${}\pi_4$% +}{index} + +\defx\getfifth{% + {}${}\pi_5${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\getfifth{% + ${}\pi_5$% +}{index} + +\defx\rhscont{% + {}${}\pi_{\{\}}${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\rhscont{% + ${}\pi_{\{\}}$% +}{index} + +\defx\rhscnct{% + {}${}\pi_{\leftrightarrow}${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\rhscnct{% + ${}\pi_{\leftrightarrow}$% +}{index} + +\defx\rhsbool{% + {}${}\pi_{\vdash}${}% + \def\setegroup{\relax}% + \def\setpgroup{% + ${}(\hbox{\the\toksa}% + \if\next]% + \else + )% + \fi + $}\grabbalanced +}{texline} + +\defy\rhsbool{% + ${}\pi_{\vdash}$% +}{index} + +\def\bbalempty[#1]#2{% + \ifx#2]\relax + \else + \ifx#2[\relax + \else + \ifnum`#2=`\{% + #2\hbox{$\,$}% + \else + \unskip\hbox{$\,$}#2% + \fi + \fi + \fi +}% + +\def\bbbalance{% + \ifnum\bbalance>0 + \loop + \advance\bbalance-1\relax + \appendr\texlinetoks{\noexpand\bbal[\the\bbalance]]}% + \ifnum\bbalance>0 + \repeat + \else + \tempca-\bbalance + \toksd{}% + \loop + \advance\tempca-1\relax + \appendl\toksd{\noexpand\bbal[\the\tempca][}% + \ifnum\tempca>0 + \repeat + \tempca-\bbalance + \bbalance\tempca + \texlinetoks{}\texidxtoks{}% + \expandafter\yygetchar\the\toksc\end +% \errmessage{\the\texlinetoks...\the\toksd...\the\bbalance}% + \appendl\texlinetoks{\the\toksd}% + \fi +} + +\def\grabbalanced{% + \futurelet\next\gr@bbalanced +} + +\def\gr@bbalanced{% + \ifx\next\bbal + \let\next\gr@bb@lanced + \else + \let\next\settgroup + \fi + \next +} + +\def\gr@bb@lanced\bbal[#1]#2{% + \ifnum`#2=`\}% this is a closing parenthesis we are inside a group + \def\next{\setegroup\bbal[#1]#2}% + \toksa{#2}% + \else + \ifnum`#2=`]% + \def\next{\setegroup\bbal[#1]#2}% + \toksa{#2}% + \else + \def\next##1\bbal[#1]##2{% + \toksa{##1}\let\next##2% + \setpgroup + }% + \fi + \fi + \next +} diff --git a/support/splint/tex/yyunion.sty b/support/splint/tex/yyunion.sty new file mode 100644 index 0000000000..d9749a86b9 --- /dev/null +++ b/support/splint/tex/yyunion.sty @@ -0,0 +1,1523 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% prototypes of all the macros produced by the parser +% we will follow the convention: 12string is a string of category 12 characters and spaces, tex_string: a string +% of TeX tokens; sptr is a pointer to the stash stream, fptr is a pointer to the format stream + +\def\yyuniontag{\yyunion} +\def\parserstrictnamespace{parser-strict} +\def\parserprototypesnamespace{parser-strict:headers} + +\def\yyunion{\currentyyunionnamespace} +\def\currentyyunionnamespace{generic} + +% types inserted by the input routines: these should not be tampered with randomly +% therefore they are not included in the yyunion list and only defined once below and redefined later +% permanently + +% insidemost + +\long\def\stashed#1{} % stashed material (usually \Cee\ code) :: \stashed{tex_string} +\long\def\sflush#1#2{} % stash flushing marker :: \sflush{tex_string} +\def\format#1{} % format material triggered by braces and the start of production :: \format{tex_string} +\def\formatlocal#1{} % format material triggered by every term :: \formatlocal{tex_string} + +% level 1 + +\long\def\strm[#1]#2.[#3]{} % a stream element :: \strm[digits]{\stashed|\format|\formatlocal}.[digits] + +% types returned by the lexer (* marks the types that get removed by the parser in some cases) + +\def\midf#1#2{} % rule separator :: \midf{fptr}{sptr} +\defp\stringify#1#2#3#4{} % STRING :: \stringify{12string}{tex_string}{fptr}{sptr} +\defp\idit#1#2#3#4{} % ID :: \idit{12string}{tex_string}{fptr}{sptr} +\defp\tagit#1#2#3#4{} % TAG :: \tagit{12string}{tex_string}{fptr}{sptr} +\defp\charit#1#2#3#4{} % CHAR :: \charit{12string}{tex_string}{fptr}{sptr} +\defp\anint#1#2#3{} % INT :: \anint{digits}{fptr}{sptr} +\defp\hexint#1#2#3{} % INT :: \hexint{0[xX][0-9A-F]+}{fptr}{sptr} +\defp\braceit#1#2#3{} % *BRACED_CODE :: \braceit{tex_string}{fptr}{sptr} +\defp\preckind#1#2#3{} % *precedence operator :: \preckind{12string}{fptr}{sptr} +\defp\flexoptionpair#1#2{} % a flex option :: \flexoptoinpair{\idit}{\idit|\stringify} + +% types inserted by the parser + +% insidemost + +\defp\termname#1#2{} % a production term :: \termname{\idit|\stringify|\charit}{\idit} +\defp\mergeop#1#2#3{} % merge directive :: \mergeop{\tagit}{fptr}{sptr} +\defp\dprecop#1#2#3{} % dprec directive :: \dprecop{\anint|\hexint}{fptr}{sptr} +\defp\sprecop#1#2#3{} % prec directive :: \sprecop{\idit|\stringify|\charit}{fptr}{sptr} +\defp\rrhssep#1#2{} % :: \rrhssep{fptr}{sptr} +\defp\arhssep#1#2{} % :: \arhssep{fptr}{sptr} +\defp\rarhssep#1#2{} % :: \rarhssep{fptr}{sptr} +\defp\hspace#1#2{} % a space has to take two parameters just like any connector :: \hspace{fptr}{sptr} +\defp\emptyterm{} +\defp\onesymbol#1#2#3{} % a symbol definition :: \onesymbol{\idit}{\anint|\hexint}{\stringify} +\defp\bracedvalue#1#2#3{} % a braced variable value :: \bracedvalue{tex_string}{fptr}{sptr} + +%level 1 + +\defp\rhs#1#2#3{} % a `right-hand side' :: \rhs{[\termname\hspace\rarhssep\arhssep\actbraces]+}{\arhssep|<nothing>}{\ifrhsfull} +\defp\actbraces#1#2#3#4#5\bdend{} % action braces :: \actbraces{tex_string}{\idit|<nothing>}{fptr}{sptr}directive*\bdend +\defp\bpredicate#1#2#3#4#5\bdend{} % predicate :: \bpredicate{tex_string}{<nothing>}{fptr}{sptr}directive*\bdend + +% level 2 + +\defp\prodheader#1#2#3#4{} % a `left-hand side' of a production :: \prodheader{\idit}{\idit}{fptr}{sptr} +\defp\rules#1#2#3{} % a complete right-hand side :: \rules{[\rhs|\rrhssep]}{max fptr}{max sptr} +\defp\symbolprec#1#2{} % symbol precedence :: \symbolprec{\idit|\stringify|\charit}{\anint|\hexint} +%\def\pcluster#1#2{} % a production cluster :: \pcluster{\prodheader}{\termname|...} should never appear in the final output + +% level 3 + +\defp\oneproduction#1#2#3{} % a complete production :: \oneproduction{\prodheader\rules}{fptr}{sptr} +\defp\prodprodsepsmall{} % production separator +\defp\prodprodsep{} % production separator +\defp\prodprodseplarge{} % production separator +\defp\optoptsepsmall{} % separator between options +\defp\posmark#1#2{} % the position marker :: \posmark{fptr}{sptr} +\defp\tokendecls#1#2#3{} % token declarations :: \tokendecls{[\onesymbol]+}{fptr}{sptr} +\defp\ntermdecls#1#2#3{} % nterm declarations :: \ntermdecls{[\onesymbol]+}{fptr}{sptr} +\defp\typedecls#1#2#3#4{} % type declarations :: \typedecls{\tagit}{[\idit\stringify\charit]+}{fptr}{sptr} +\defp\precdecls#1#2#3#4#5{} % precedence declarations :: \precdecls{left|right|nonassoc|precedence}{\tagit}{[\idit\stringify\charit]+}{fptr}{sptr} +\defp\flexsstatedecls#1#2#3{} % flex state declarations (nonexlusive) :: \flexsstatedecls{[\idit\stringify]+}{fptr}{sptr} +\defp\flexxstatedecls#1#2#3{} % flex state declarations (exclusive) :: \flexxstatedecls{[\idit\stringify]+}{fptr}{sptr} +\defp\flexoptiondecls#1#2#3{} % flex state declarations :: \flexoptiondecls{[\flexoptionpair]+}{fptr}{sptr} +\defp\oneparametricoption#1#2#3#4{} % option :: \oneparametricoption{option-name}{\stringify|\idit}{fptr}{sptr} +\defp\optionflag#1#2#3#4{} % option flag :: \optionflag{flag name}{abbreviated name}{fptr}{sptr} +\defp\paramdef#1#2#3#4{} % parameters :: \paramdef{\braceit}{parameter domain}{fptr}{sptr} +\defp\codepropstype#1#2#3#4#5#6#7{} % term method :: \codepropstype{printer|destructor}{tex_string}{\stringit|\idit}{fptr}{sptr}{fptr}{sptr} +\defp\initaction#1#2#3{} % initial action :: \initaction{tex_string}{fptr}{sptr} +\defp\codeassoc#1#2#3#4#5#6#7{} % code association :: \codeassoc{code kind}{|\idit}{tex_string}{fptr}{sptr}{fptr}{sptr} +\defp\vardef#1#2#3#4{} % variable definition :: \vardef{\idit|\stringify}{|\idit|\stringify|\bracedvalue}{fptr}{sptr} +\defp\prologuecode#1#2#3#4#5{} % prologue code :: \prologuecode{tex_string}{fptr}{sptr}{fptr}{sptr} + +% level 4 + +%\def\grammar#1#2{} % a grammar: should never appear in the final output + +\savecslist{parser-prototypes}\yyunion + +% table construction: the macros run to build the contents of the table of productions to be +% typeset. + +\newtoks\table +\newtoks\tokdectoks +\newtoks\opttable +\newtoks\typestable +\newtoks\prectable + +\newtoks\tokstoks +\newtoks\typesset + +\let\rhscont\getsecond +\let\rhscnct\getthird +\let\rhsbool\getfourth + +\newif\ifrhsfull + +\newtoks\yystashlocal +\newtoks\yyformatlocal + +\newif\iftermindex + +\def\doparse#1{% + \table{}\typestable{}\prectable{}\opttable{}% + \parserinit\yyparse#1\yyeof\yyeof\endparseinput\endparse +} + +\long\def\strm[#1]#2.[#3]{#2} +\long\def\inmath#1{$#1$} + +\long\def\stashed#1{\toksa\expandafter{\the\toksa#1}} +\long\def\sflush#1#2{} + +\def\ntt{\noexpand\ntt@} % a convenient shortcut +\def\ntt@{\tt\let\_\UL} + +\def\prodheader#1#2#3#4{% + \yyreadfifo\yyformat\to#3\in\yyformatlocal + \restorecs{table-render}{\strm}\the\yyformatlocal + \let\termindex\writeidxhentry + \prodhead@r{#1}{#2}% + \let\termindex\eatone +} + +\def\headeridxrank{1} % index rank of lhs + +\def\writeidxhentry#1{\write\gindex{% + \nx\nx + \expandafter + \nx + \csname\bisonidxdomain TI\endcsname\headeridxrank#1}% +} + +\def\prodhead@r#1#2{% + \let\idit\prodhdridit + #1\relax\concat\table\toksa\hrhss@p +} + +\def\hrhss@p{\toksa{&\omit\hfil&\omit\hfil&\omit\hfil\cr\noalign{\penalty100}\omit\hfil&\termspostformat}\concat\table\toksa} + +\def\rrhssep#1#2{% + \yyreadfifot\yyformat\to#1\in\yyformatlocal\with\formatsort\relax + \the\yyformatlocal % this will have a chance to change the definition of \rrhss@p + \rrhss@p +} + +\def\rrhss@p{\toksa{\omit\hfil&}\concat\table\toksa} + +\def\arhssep#1#2{% + \yyreadfifo\yyformat\to#1\in\yyformatlocal + \the\yyformatlocal % this will have a chance to change the definition of \arhss@p + \arhss@p +} + +%% between the inline action and next term: + +\def\arhss@p{\toksa{\omit\hfil&\hbox to 2em{\hfil}}\concat\table\toksa} + +\def\rarhssep#1#2{% + \yyreadfifo\yyformat\to#1\in\yyformatlocal + \the\yyformatlocal % this will have a chance to change the definition of \rarhss@p + \rarhss@p +} + +%% between a term and the following action + +\def\rarhss@p{\appendr\table&} + +\newtoks\hstoks + +\def\hspace#1#2{% + \hstoks\yyformat + \yyreadfifot\yyformat\to#1\in\yyformatlocal\with\formatsort\relax + \the\yyformatlocal % may change \hspac@ + \hspac@ +} + +\def\hspac@{% + \appendr\table{ }% +} + +\def\rhs#1#2#3{#1} + +\def\supplybdirective#1#2{% + \edef\next{\nx\s@pplybdirective#1{\the#2}\the#1\nx\bdend}\next +} + +\def\s@pplybdirective#1#2#3\bdend\bdend{% + #1{#3#2\bdend}% +} + +\def\termname#1#2{% + \let\idit\termnameidit + \let\stringify\termnamestringify + \let\optstrextra\optstrextraesc + \let\termindex\writeidxentry + #1\relax\concat\table\toksa + \let\termindex\eatone +} + +\def\bisonidxdomain{B} % marker fo the domain (section of the index) +\def\termidxrank{2} % rank (affects the order in which the entries are + % listed in the index and the listing style) + +\def\writeidxentry#1{\write\gindex{% + \nx\nx + \expandafter + \nx + \csname\bisonidxdomain TI\endcsname\termidxrank#1}% +} + +\let\termindex\eatone + +% index implementation +% note that the rank and domain parameters are handed down by the +% `outer' macro while the `qualifier' (\term...string) is decided +% at the point of invocation + +\def\gidxentry#1#2{% a generic index entry + \iftermindex + \termindex{% + {\nx\nx\nx#1{\charstonumberse#2\end}}% + {\hostparsernamespace}{\nx\the\pageno}{\secno}% + }% + \fi +} + +\def\tidxentry#1#2{% a text index entry + \iftermindex + \write\gindex{% + \nx\nx\expandafter\nx\csname\bisonidxdomain TI\endcsname\termidxrank + {\nx\nx\nx#1{\charstonumberse#2\end}}% + {\indexpseudonamespace}{\nx\the\pageno}{\secno}% + }% + \fi +} + +% + +\newtoks\bdirects + +\def\actbraces#1#2#3#4#5\bdend{% + \yyreadfifo\yyformat\to#3\in\yyformatlocal + \restorecs{table-render}{\strm}% + #5% accumulate all the directives + \edef\next{\the\bdirects}% + \ifx\next\empty + \else + \edef\next{\bdirects{% + $\vtop{\halign{\nx\quad####\nx\hfil&####\nx\hfil\cr\the\bdirects}}$}}\next + \fi + \the\yyformatlocal + \actbrac@s{#1}{#2}{#4}% +} + +\def\actbrac@s#1#2#3{% + \yyreadfifo\yystash\to#3\in\yystashlocal + \unmathstash{#3}% + \edef\next{\toksa{\the\bdirects&\the\yystashlocal\cr}}\next + \bdirects{}% + \concat\table\toksa +} + +\def\unmathstash#1{% + \setbox\z@=\vbox{ + \hsize\maxdimen\emergencystretch\maxdimen + \rightskip\z@ plus 1fill + \let\TeXx\relax\let\stashed\copystashed + \the\yystashlocal\relax + \ifmmode + {}$\aftergroup\unm@thstash % $o that the box can be finished + \else + \aftergroup\eatone + \fi}{#1}% +} + +\def\copystashed#1{#1} + +\def\unm@thstash#1{% + \toksa{\strm[NaN]{\stashed{{}$}}.[NaN]}% $o it cannot be matched if \yystashlocal is read again + \concat\yystashlocal\toksa + \toksa{\strm[#1]{\stashed{${}}}.[#1]}%$ + \appendl\yystash{\the\toksa}% +} + +\def\bpredicate#1#2#3#4#5\bdend{% + \yyreadfifo\yyformat\to#3\in\yyformatlocal + \restorecs{table-render}{\strm}% + #5% accumulate all the directives + \edef\next{\the\bdirects}% + \ifx\next\empty + \bdirects{\quad$\dashv\ \nx\vdash$}% + \else + \edef\next{\bdirects{% + $\vtop{\halign{\nx\quad####\nx\hfil&####\nx\hfil\cr$\dashv\ $&\omit\nx\hfil$\nx\vdash$\cr + \the\bdirects}}$}}\next + \fi + \the\yyformatlocal + \bpredicat@{#1}{#2}{#4}% +} + +\def\bpredicat@#1#2#3{% + \yyreadfifo\yystash\to#3\in\yystashlocal + \unmathstash{#3}% + \edef\next{\toksa{\the\bdirects&\the\yystashlocal\cr}}\next + \bdirects{}% + \concat\table\toksa +} + +\def\bdirective#1#2#3#4{% + \savecslist{local-namespace}\yyunion + \restorecslist{table-render:directives}\yyunion + \toksa{}% + #2% make a directive list + \edef\next{\toksa{$\nx\langle${\ntt #1} &\the\toksa$\nx\rangle$\cr}}\next + \concat\bdirects\toksa + \restorecslist{local-namespace}\yyunion +} + +\def\mergeop#1#2#3{% + \bdirective{merge}{#1}{#2}{#3}% +} + +\def\dprecop#1#2#3{% + \bdirective{dprec}{#1}{#2}{#3}% +} + +\def\sprecop#1#2#3{% + \bdirective{prec}{#1}{#2}{#3}% +} + +\def\rules#1#2#3{#1} + +\def\oneproduction#1#2#3{#1\setrulecontext{}\stopproduction} + +% to make it possible to typeset tokens depending on their context +% the context name is passed to the token typeseting macros; the most +% common context name is the left hand side of a production; special +% context names (such as *left, *right, etc) are defined for terms that are +% not part of a rule; all such context names start with `*' since this +% character cannot be part of a term name + +\def\setrulecontext#1{\def\currentrulecontext{#1}} + +\let\currentrulecontext\empty + +\def\emptyterm{% + \edef\next{\toksa{$\nx\circ$\tidxentry{\termidstring}{emptyrhs}}}\next + \concat\table\toksa +} + +\def\termnameidit#1#2#3#4{% + \yyreadfifot\yyformat\to#3\in\yyformatlocal\with\formatsort\relax + \the\yyformatlocal + \nameproc{#2}% + \peekstash{#4}% + \settermstyle{#1}% + } + +\def\settermstyle#1{% + \edef\next{\toksa{\termmetastyle{% + \gidxentry{\termidstring}{#1}% + \let\nx\idxfont\nx\empty\nx\it\the\toksa\nx\/% + }}}\next +} + +\def\idxinline#1{% index the term in text + \edef\next{\tidxentry{\termidstring}{#1}}\next +} + +\def\termnamestringify#1#2#3#4{% + \nameproc{#2}% + \edef\next{\toksa{{% + \gidxentry{\termstring}{#1}% + \ntt\the\toksa + }}}\next +} + +\def\termmetastyle#1{{#1}} + +\let\peekstash\eatone + +\newif\ifinheader + +\def\prodhdridit#1#2#3#4{% + \setrulecontext{}% + \nameproc{#2}% + \expandafter\setrulecontext\expandafter{\the\toksc}% + \edef\next{\toksa{{% + \gidxentry{\termhdrstring}{#1}% + \nx\inheadertrue\let\nx\idxfont\nx\bf\nx\itbold\the\toksa\nx\/% + }}}\next +} + +\def\charit#1#2#3#4{% + \expandafter\ifx\csname'parser'\hostparsernamespace#1\endcsname\relax + \edef\next{\toksa{{\ntt #1}\gidxentry{\termvstring}{'#1'}}}% + \else + \edef\next{\expandafter\toksa\expandafter{\expandafter\noexpand + \csname'parser'\hostparsernamespace#1\endcsname{\currentrulecontext}\gidxentry{\termvstring}{'#1'}}}% + \fi + \next +} + +\def\anint#1#2#3{% + \toksa{#1}\concat\table\toksa +} + +\def\hexint#1#2#3{% + \toksa\expandafter{\eattwo#1$_{16}$}\concat\table\toksa +} + +\def\optoptsepnone{% + \concat\opttable\postoks +} + +\def\optoptsepsmall{% keep the alignment going but insert a small space + \concat\opttable\postoks + \toksa{\noalign{\smallskip}}\concat\opttable\toksa +} + +\def\prodprodsepsmall{% + \pushothertables + \toksa{\noalign{\penalty-150 \smallskip}}\concat\table\toksa +} + +\def\prodprodsep{% + \pushothertables + \toksa{\noalign{\medskip}}\concat\table\toksa +} + +\def\prodprodseplarge{% + \pushothertables + \toksa{\noalign{\bigskip}}\concat\table\toksa +} + +\def\separatorswitcheq{% + \oneproduction {% + \toksd{\prodprodsepsmall}% + } + \codepropstype\codeassoc {% + \toksd{\optoptsepsmall}% + } +} + +\def\separatorswitchdefaulteq{% + \toksd{\optoptsepnone}% +} + +% a switch for inserting position information into the parsed table +% clobbers \toksd + +\newtoks\postoks + +\def\positionswitch{ + \empty + {% + \postoks{}% semicolons should be invisible + } + \paramdef\oneparametricoption\optionflag\typedecls\vardef + {% + \getfourth{\yy(2)}\to\postoks % the format marker + \getfifth{\yy(2)}\to\toksd % the stash marker + \edef\next{\postoks{\nx\posmark{\the\postoks}{\the\toksd}}}\next + } + \prologuecode\ntermdecls\tokendecls + \flexsstatedecls\flexxstatedecls\flexoptiondecls + {% + \getthird{\yy(2)}\to\postoks % the format marker + \getfourth{\yy(2)}\to\toksd % the stash marker + \edef\next{\postoks{\nx\posmark{\the\postoks}{\the\toksd}}}\next + } + \precdecls + {% + \getfifth{\yy(2)}\to\postoks % the format marker + \getsixth{\yy(2)}\to\toksd % the stash marker + \edef\next{\postoks{\nx\posmark{\the\postoks}{\the\toksd}}}\next + } + \oneproduction + {% + \getsecond{\yy(2)}\to\postoks % the \prodheader + \getfourth{\postoks}\to\toksd % the format marker + \getfifth{\postoks}\to\postoks % the stash marker + \edef\next{\postoks{\nx\posmark{\the\toksd}{\the\postoks}}}\next + } + \codeassoc\codepropstype + {% + \getseventh{\yy(2)}\to\postoks % the format marker + \geteightth{\yy(2)}\to\toksd % the stash marker + \edef\next{\postoks{\nx\posmark{\the\postoks}{\the\toksd}}}\next + } +} + +\def\positionswitchdefault{% + \errmessage{unknown chunk type in: \the\yy(2)}% +} + +\def\posmark#1#2{% +% \yyreadfifo\yyformat\to#1\in\yyformatlocal +% no formattig for locations currently + \restorecs{table-render}{\strm}% +% \the\yyformatlocal + \posm@rk{#2}% +} + +\def\posm@rk#1{% + \yyreadfifo\yystash\to#1\in\yystashlocal + \edef\next{\the\yystashlocal}% + \ifx\next\empty + \postoks{}% + \else + \unmathstash{#1}% + % TODO: move this out + \toksa{}\the\yystashlocal\cleanstash\stripstash + \setbox\z@=\vbox{\setlazyc\the\toksa}% + \ifnum\ht\z@=\z@ + \postoks{}% + \else + \edef\next{\postoks{\noalign{\indent\toksa{}\the\yystashlocal\nx\makestashbox}}}\next + \fi + \fi +} + +\def\codeoptionlist{% + \codepropstype\codeassoc\prologuecode +} + +\def\simpleoptionsplit{% + \oneparametricoption\vardef\paramdef\optionflag +} + +\def\noncodeoptionlist{% + \oneparametricoption\vardef\paramdef\optionflag + \codepropstype\codeassoc\prologuecode +} + +\def\alloptionlist{% + \oneparametricoption\vardef\paramdef\optionflag + \codepropstype\codeassoc\prologuecode + \typedecls\tokendecls\precdecls\oneproduction + \flexsstatedecls\flexxstatedecls\flexoptiondecls +} + +\def\emptyoption{\empty} + +\diagprod\codeoptionlist\noncodeoptionlist\in\codenoncodeseplist +\diagprod\simpleoptionsplit\simpleoptionsplit\in\noncodenoncodeseplist +\diagprod\alloptionlist\emptyoption\in\emptyotherseplist + +% this is a switch for inserting separators between dissimilar items + +\def\separatorswitchneq{% + \classexpand\codenoncodeseplist + {% separate options with code (\codepropstype, \codeassoc) from other options by a small space + \toksd{\optoptsepsmall}% + } + \classexpand\noncodenoncodeseplist + {% + \toksd{\optoptsepnone}% + } + % a separator after the first rule or before or after a semicolon + \classexpand\emptyotherseplist + {% + \toksd{\optoptsepnone}% + } +} + +\setspecialcharsfrom\separatorswitchneq + +\def\separatorswitchdefaultneq{% + \toksd{\prodprodsep}% +} + +\def\pushothertables{% + %token's + \edef\next{\the\tokdectoks}% + \ifx\next\empty + \else + \attachtokentable + \fi + %type's + \edef\next{\the\typestable}% + \ifx\next\empty + \else + \attachtypestable + \fi + %precedence's + \edef\next{\the\prectable}% + \ifx\next\empty + \else + \attachprectable + \fi + % options + \edef\next{\the\opttable}% + \ifx\next\empty + \else + \attachoptionstable + \fi + \concat\table\postoks +} % + +% the various \attach... macros above are defined elsewhere and are the macros that +% determine the actual typesetting of the corresponding tables. + +\let\stopproduction\relax + +\def\tokendecls#1#2#3{% + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:token-declarations}\yyunion + \let\termindex\writetokenidxentry + \displaytokenrawtrue + \let\currenttokentype\empty#1% + \restorecslist{local-namespace}{\yyunion\termindex}% +} + +\def\typedecls#1#2#3#4{% + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:type-declarations}\yyunion + \let\termindex\writetokenidxentry + \typesset{}% + #2\relax#1% + \edef\next{\typestable{\the\typestable\nx\onetype{\the\toksa}{\the\typesset}}}\next + \restorecslist{local-namespace}{\yyunion\termindex}% +} + +% \onetype is a typeseting macro defined elsewhere + +\def\flexsstatedecls#1#2#3{% + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:flex-state-declarations}\yyunion + \let\termindex\writetokenidxentry + \typesset{}% + #1\relax + \edef\next{\typestable{\the\typestable\nx\flexsstatelist{\the\typesset}}}\next + \restorecslist{local-namespace}{\yyunion\termindex}% +} + +\def\flexxstatedecls#1#2#3{% + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:flex-state-declarations}\yyunion + \let\termindex\writetokenidxentry + \typesset{}% + #1\relax + \edef\next{\typestable{\the\typestable\nx\flexxstatelist{\the\typesset}}}\next + \restorecslist{local-namespace}{\yyunion\termindex}% +} + +% \flexsstatelist and \flexxstatelist are typesetting macros defined elsewhere + +\def\flexoptiondecls#1#2#3{% + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:flex-option-definitions}\yyunion + \let\termindex\writetokenidxentry + \toksa{}% + #1\relax + \restorecslist{local-namespace}{\yyunion\termindex}% +} + +\def\precdecls#1#2#3#4#5{% #1 type + % #2 tag + % #3 list of symbols + % #4 format pointer + % #5 stash pointer + \savecslist{local-namespace}{\yyunion\termindex}% + \restorecslist{table-render:prec-declarations}\yyunion + \setrulecontext{*#1}% + \let\termindex\writetokenidxentry + \typesset{}#3\toksa{}#2% + \edef\next{\prectable{\the\prectable\noexpand\oneprec{#1}{\the\toksa}{\the\typesset}}}\next + \restorecslist{local-namespace}{\yyunion\termindex}% + \setrulecontext{}% +} + +% \oneprec is a typesetting macro defined elsewhere + +% options typesetting + +\def\oneparametricoption#1#2#3#4{% + \restorecs{table-render:token-declarations}{\braceit\stringify\idit\anint\hexint}% + \toksa{}\toksb{}\toksc{}% + #2% + \edef\next{\noexpand\oneparametricopti@n{#1}{\the\toksa}{\the\toksb}{\the\toksc}}\next +} + +\def\oneparametricopti@n#1#2#3#4{% + \edef\next{#2}% + \ifx\next\empty % there is no identifier + \def\next{#4}% + \ifx\next\empty % there is no digit + \edef\next{\toksa{{\ntt"\the\toksb"}}}\next + \else + \edef\next{\toksa{{\nx\rm\the\toksc}}}\next + \fi + \else + \nameproc{#2}% + \fi % now \toksa contains the value of the parameter + \toksb{#1}% + \let\termindex\writetokenidxentry + \edef\next{\toksa{\gidxentry{\termostring}{#1}$\nx\langle${\nx\bf\the\toksb}$\nx\rangle$&\the\toksa\nx\/\nx\cr}}\next + \let\termindex\eatone + \concat\opttable\toksa +} + +\def\paramdef#1#2#3#4{% + \restorecs{table-render:token-declarations}\braceit + \let\termindex\writetokenidxentry + \edef\next{\toksa{\gidxentry{\termostring}{#2}}}\next + \toksb\expandafter{\the\toksa$\langle${\bf #2}$\rangle$}\toksa{}% + #1% + \let\termindex\eatone + \concat\toksb\toksa + \concat\opttable\toksb +} + +\def\optionflag#1#2#3#4{% + \def\next{#2}% + \ifx\next\empty + \toksb{}% + \else + \toksb{ {\rm(set as $\langle${\bf#2}$\rangle$)}}% + \fi + \let\termindex\writetokenidxentry + \edef\next{\toksa{\gidxentry{\termostring}{#1}$\nx\langle${\nx\bf#1}$\nx\rangle$\nx\rlap{$\nx\,\star$}&\the\toksb\cr}}\next + \let\termindex\eatone + \concat\opttable\toksa +} + +\def\codepropstype#1#2#3#4#5#6#7{% + \yyreadfifo\yyformat\to#4\in\yyformatlocal + \restorecs{table-render}{\strm}% + \the\yyformatlocal % this can change \codepropstyp@ + \codepropstyp@{#2}{#3}{#1}{#5}% +} + +\def\codepropstyp@#1#2#3#4{% + \yyreadfifo\yystash\to#4\in\yystashlocal + \unmathstash{#4}% + \restorecs{table-render:type-declarations}{\idit\stringify\hspace}% + \let\termindex\writetokenidxentry + \typesset{}% + #2% collect all the symbols in \typesset + \edef\next{\toksa{\gidxentry{\termostring}{#3}$\nx\langle${\nx\bf #3}$\nx\rangle$&% + \toksa{}\the\yystashlocal\nx\makestashbox\nx\hfil\cr + \noalign{\nx\smallskip}% + \noalign{\indent$\nx\rlap{\hbox to2em{\nx\hfil$\nx\star$\nx\hfil}}\vcenter{ + \advance\hsize by-\parindent + \emergencystretch10pt\nx\raggedright\noindent\hangafter\z@\hangindent2em\nx\strut\the\typesset\nx\strut}$}% + }}\next + \let\termindex\eatone + \concat\opttable\toksa +} + +\def\codeassoc#1#2#3#4#5#6#7{% + \yyreadfifo\yyformat\to#4\in\yyformatlocal + \restorecs{table-render}{\strm}% + \the\yyformatlocal % this can change \codeass@c + \codeass@c{#1}{#2}{#3}{#5}% +} + +\def\codeass@c#1#2#3#4{% + \yyreadfifo\yystash\to#4\in\yystashlocal + \unmathstash{#4}% + \let\idit\termnameidit + \let\termindex\writetokenidxentry + \toksa{}% in case #1 is empty + #2% put the typeset form of the identifie in \toksa + \toksb\toksa + \edef\next{\toksa{\yystringempty{#1}{}{\gidxentry{\termostring}{#1}}% + $\nx\langle${\nx\bf #1}$\nx\rangle$\nx\quad{\the\toksb}&% + \toksa{}\the\yystashlocal + \nx\makestashbox + \nx\hfil\cr + }}\next + \concat\opttable\toksa + \let\termindex\eatone +} + +\def\vardef#1#2#3#4{% + \yyreadfifo\yyformat\to#3\in\yyformatlocal + \restorecs{table-render}{\strm}% + \the\yyformatlocal % this can change \vard@f + \vard@f{#1}{#2}{#4}% +} + +\def\vard@f#1#2#3{% + \let\idit\termnameidit + \let\stringify\termnamestringify + \let\termindex\writetokenidxentry + #1% put the typeset form of the identifier or string in \toksa + \toksb\toksa + \savecslist{local-namespace}\yyunion + \restorecslist{table-render:variable-definitions}\yyunion + \toksa{}% in case #2 is empty + #2% put the value in toksa + \toksc\toksa + \restorecslist{local-namespace}\yyunion + \let\termindex\eatone + \edef\next{\toksa{$\nx\langle${\nx\bf define}$\nx\rangle$\nx\quad{\the\toksb}&% + \the\toksc\cr + }}\next + \concat\opttable\toksa +} + +\def\prologuecode#1#2#3#4#5{% similar to \actbrac@s + \yyreadfifo\yystash\to#5\in\yystashlocal + \unmathstash{#5}% + \restorecs{table-render}{\strm\stashed}% + \toksa{}\the\yystashlocal + \edef\next{\toksa{\omit\span\omit\nx\cdotfill\cr + \omit\span\omit\toksa{\the\toksa}\nx\makestashbox\nx\cr + \omit\span\omit\nx\cdotfill\cr}}\next + \concat\opttable\toksa +} + +\def\yyunion@tablerender{% + \strm\stashed\rarhss@p\rrhss@p\prodhead@r\hrhss@p\arhss@p\actbrac@s\hspac@\codepropstyp@\vard@f\stopproduction + \termmetastyle\peekstash +} + +\savecslist{table-render}\yyunion +\savecslist{table-render}\yyunion@tablerender + +\def\setprodtable{% + \restorecslist{table-render}\yyunion + \restorecslist{table-render}\yyunion@tablerender +} + +% formatting macros + +\def\midf#1#2{} + +\def\format#1{% + #1% +} +\let\formatlocal\format + +\newtoks\yystashlastcontent % contents of the last action +\newif\ifshowlastaction % if true, show the last action of a flattened set of rules + +\def\flatten{% + \let\stopproduction\stopproduction@flatten + \let\actbrac@s\actbrac@s@flatten + \let\rrhss@p\rrhss@p@flatten + \let\arhss@p\relax + \let\rarhss@p\relax +} + +\def\actbrac@s@flatten#1#2#3{% do everything but add the stash contents to the table + \yyreadfifo\yystash\to#3\in\yystashlocal + \unmathstash{#3}% + \yystashlastcontent\yystashlocal + \futurelet\actsep\actbrac@s@fl@tten +} + +\def\actbrac@s@fl@tten{% + \ifx\actsep\arhssep + \edef\next{% + \toksa{$\nx\ \nx\diamond\nx\ $\tidxentry{\termidstring}{inline_action}}}\next + \concat\table\toksa + \fi +} + +\def\rrhss@p@flatten{% + \toksa{$\ \vert\ $}\concat\table\toksa +}% + +\def\stopproduction@flatten{% + \ifshowlastaction + \toksa\yystashlastcontent + \else + \toksa{\actionfiller}% + \fi + \toksa\expandafter{\expandafter&\expandafter&\the\toksa\cr}\concat\table\toksa +} + +\def\fold{% + \restorecs{table-render}{\actbrac@s\rrhss@p\arhss@p\rarhss@p\stopproduction}% + \ifshowlastaction + \toksa\yystashlastcontent + \else + \toksa{\actionfiller}% + \fi + \toksa\expandafter{\expandafter&\expandafter&\the\toksa\cr}\concat\table\toksa +} + +\def\actionfiller{\omit\quad$\ldots$\hfil} + +\def\inline{% + \let\prodhead@r\prodhead@r@inline + \let\hrhss@p\hrhss@p@inline +} + +\def\prodhead@r@inline#1#2{% + \let\idit\prodhdridit + #1\relax + \setbox\z@=\hbox{\strut\the\toksa\/$\,$\rm:\quad}% + \tempda=\wd\z@\relax + \advance\tempda-2em + \edef\next{\table{\the\table{\the\toksa}}}\next + \hrhss@p +} + +\newif\ifsquashterms + +\def\termspostformat{\ifsquashterms\aftergroup\noexpand\squashtermstrue\fi} + +\def\hrhss@p@inline{% + \edef\next{\toksa{&\termspostformat\hbox to \the\tempda{\hss}}}\next\concat\table\toksa +} + +\def\breakline{% + \let\hspac@\hspac@@breakline +} + +\def\hspac@@breakline{% + \toksa{{$\,\hookleftarrow$}&\omit\hfil&\omit\hfil\cr\omit\hfil&\hbox to 2em{\hfil}}\concat\table\toksa + \restorecs{table-render}\hspac@ +} + +\def\skipheader{% + \let\prodhead@r\prodhead@r@skipheader + \let\hrhss@p\hrhss@p@skipheader +} + +\def\prodhead@r@skipheader#1#2{% + \hrhss@p +} + +\def\hrhss@p@skipheader{\toksa{\omit\hbox to 2em{\hfil}&\termspostformat}\concat\table\toksa\restorecs{table-render}{\hrhss@p}} + +\def\stashtoterm#1{% in case one needs to insert previous stash as a term + % usage: \let\peekstash\stashtoterm + \yyreadfifo\yystash\to#1\in\yystashlocal + \unmathstash{#1}% + \edef\next{\toksb{{% + \nx\restorecs{table-render}{\nx\stashed\nx\strm}% + \toksa{}\the\yystashlocal\nx\makestashbox}}}\next + \def\termmetastyle##1{\the\toksb}% + \restorecs{table-render}\peekstash +} + +\def\resetf{% + \restorecs{table-render}{\prodheader\prodhead@r\hrhss@p\rrhss@p\arhss@p\rhs\termname\actbraces\actbrac@s\rarhss@p\hspac@\rules + \oneproduction\emptyterm\stringify\idit\charit\prodprodsep\onesymbol\strm\stopproduction}% + \squashtermsfalse +} + +\def\prodstyle#1{% to typeset token names in text + {% + \let\optstrextra\optstrextraesc + \nameproc{#1}% + \settermstyle{#1}% + \the\toksa\nobreak + }% +} + +% macros used in typesetting the result of the prologue parsing + +%type declarations + +\restorecslist{parser-strict}\yyunion + + +\def\idit#1#2#3#4{% + \termnameidit{#1}{#2}{#3}{#4}% + \concat\typesset\toksa +} + +\def\stringify#1#2#3#4{% + \termnamestringify{#1}{#2}{#3}{#4}% + \concat\typesset\toksa +} + +\def\hspace#1#2{% + \appendr\typesset{ }% +} + +\def\charit#1#2#3#4{% + \toksa{\.{#2}}% + \concat\typesset\toksa +} + +\def\tagit#1#2#3#4{% + \toksa{\\{#2}}% +} + +\toyyunion{table-render:type-declarations} + +%token declarations + +\newif\iftracetokennames + +\restorecslist{parser-strict}\yyunion + +\def\hspace#1#2{} + +\def\idit#1#2#3#4{\toksa{#2}} + +\def\charit#1#2#3#4{\toksa{'#1'}} + +\def\stringify#1#2#3#4{\toksb{#2}} + +\def\anint#1#2#3{\toksc{#1}} + +\def\hexint#1#2#3{\toksc\expandafter{\eattwo#1$_{16}$}} + +\def\tagit#1#2#3#4{% + \def\currenttokentype{#2}% +} + +\def\braceit#1#2#3{% similar to \actbrac@s + \yyreadfifo\yystash\to#3\in\yystashlocal + \unmathstash{#3}% + \restorecs{table-render}{\strm\stashed}% + \toksa{}\the\yystashlocal + \edef\next{\toksa{&\toksa{\the\toksa}\nx\makestashbox\nx\cr}}\next +} + +\def\onesymbol#1#2#3{% #1 is the macro name used internally by bison (\idit{}{}{}{}) + % #2 is the integer value (\anint{}{}{}) + % #3 is the string value (\stringify{}{}{}{}) + \toksa{}\toksb{}\toksc{}% + #1#2#3\relax + \edef\next{\noexpand\onesymb@l{\the\toksa}{\the\toksc}{\the\toksb}}\next +} + +\def\onesymb@l#1#2#3{% #1 is the macro name used internally by bison + % #2 is the integer value + % #3 is the string value + \let\optstrextra\optstrextraesc + \yystringempty{#3}{}{\nameproc{#3}\tokstoks\toksa}% + \yystringempty{#1}{}{\nameproc{#1}}% + \iftracetokennames\derrmessage{token: \the\toksa, string: \the\tokstoks}\fi + \toksb\expandafter{\currenttokentype}% + \edef\next{% + \tokdectoks{\the\tokdectoks + \noexpand\toksdefline + {\the\toksa}% + {% + \yystringempty{#1}{}{\gidxentry{\termidstring}{#1}}% + \yystringempty{#3}{}{\gidxentry{\termstring}{#3}}% + }% + {\yytoksempty\toksb{auto}{\the\toksb}}% + {#2}% + {\the\tokstoks}% + }% + }\next + \restorecs{nameparser}{\optstrextra}% +} + +\def\defidxrank{0} % index rank of definitions + +\def\writetokenidxentry#1{\write\gindex{% + \nx\nx + \expandafter + \nx + \csname\bisonidxdomain TI\endcsname\defidxrank#1}% +} + +\toyyunion{table-render:token-declarations} + +%precedence declarations + +\restorecslist{parser-strict}\yyunion + +\def\idit#1#2#3#4{% + \termnameidit{#1}{#2}{#3}{#4}% + \concat\typesset\toksa +} + +\def\stringify#1#2#3#4{% + \termnamestringify{#1}{#2}{#3}{#4}% + \concat\typesset\toksa +} + +\def\charit#1#2#3#4{% + \expandafter\ifx\csname'parser'\hostparsernamespace#1\endcsname\relax + \edef\next{\toksa{{\ntt #1}\gidxentry{\termvstring}{'#1'}}}% + \else + \edef\next{\expandafter\toksa\expandafter{\expandafter\noexpand + \csname'parser'\hostparsernamespace#1\endcsname{\currentrulecontext}\gidxentry{\termvstring}{'#1'}}}% + \fi + \next + \concat\typesset\toksa +} + +\def\anint#1#2#3{% + \edef\next{\toksa{${}=\hbox{\ntt#1}$}}\next + \concat\typesset\toksa +} + +\def\hexint#1#2#3{% + \toksa\expandafter{\eattwo#1}% + \edef\next{\toksa{${}=\hbox{\ntt\the\toksa}_{16}$}}\next + \concat\typesset\toksa +} + +\def\hspace#1#2{% + \appendr\typesset{ }% +} + +\def\tagit#1#2#3#4{% + \toksa{\\{#2}}% +} + +\def\symbolprec#1#2{% + #1#2% +} + +\toyyunion{table-render:prec-declarations} + +% variable definitions + +\restorecslist{parser-strict}\yyunion + +\def\idit#1#2#3#4{\toksa{#2}} + +\def\stringify#1#2#3#4{\toksa{#2}} + +\def\bracedvalue#1#2#3{% similar to \actbrac@s + \yyreadfifo\yystash\to#3\in\yystashlocal + \unmathstash{#3}% + \restorecs{table-render}{\strm\stashed}% + \toksa{}\the\yystashlocal + \edef\next{\toksa{\toksa{\the\toksa}\nx\makestashbox}}\next +} + +\toyyunion{table-render:variable-definitions} + +% directives + +\restorecslist{parser-strict}\yyunion + +\def\idit#1#2#3#4{% + \termnameidit{#1}{#2}{#3}{#4}% +} + +\def\stringify#1#2#3#4{% + \termnamestringify{#1}{#2}{#3}{#4}% +} + +\def\tagit#1#2#3#4{% + \toksa{\\{#2}}% +} + +\def\anint#1#2#3{% + \toksa{#1}% +} + +\def\hexint#1#2#3{% + \toksa\expandafter{\eattwo#1$_{16}$}% +} + +\toyyunion{table-render:directives} + +% flex state declarations + +\restorecslist{parser-strict}\yyunion + +\def\idit#1#2#3#4{% + \nameproc{#2}% + \edef\next{\toksa{{% + \gidxentry{\termttstring}{#1}% + \ntt\the\toksa + }}}\next + \concat\typesset\toksa +} + +\def\hspace#1#2{% + \appendr\typesset{ }% +} + +\toyyunion{table-render:flex-state-declarations} + +% flex option definitions + +\restorecslist{parser-strict}\yyunion + +\def\flexoptionpair#1#2{% + #1% + \edef\next{#2}% + \ifx\next\empty % there is no rihgt hand side + \edef\next{\toksa{$\nx\langle${\ntt\the\toksa}$\nx\rangle_{\rm f}\,\star$&\cr}}\next + \else + \toksb\toksa\toksa{}#2% + \edef\next{\toksa{$\nx\langle${\ntt\the\toksb}$\nx\rangle_{\rm f}$&\nx\it\the\toksa\nx\/\nx\cr}}\next + \fi + \concat\opttable\toksa +} + +\def\idit#1#2#3#4{\toksa{#2}} + +\def\stringify#1#2#3#4{\toksa{{\ntt@"#2"}}} + +\def\hspace#1#2{} + +\toyyunion{table-render:flex-option-definitions} + +% rule listing macros + +\restorecslist{parser-prototypes}\yyunion % we just want to ignore the non-rule related code + +\def\prodheader#1#2#3#4{% + #1\relax + \lastlastname\lastname + \setnumname\numname + \lastname{}% + #2\relax + \edef\next{\toksc{\numname:}\toksd{\noexpand\lhs{\the\lastlastname}{\the\lastname}}}\next +} + +\def\rrhssep#1#2{} +\def\arhssep#1#2{} +\def\rarhssep#1#2{} +\def\hspace#1#2{} + +\def\rhs#1#2#3{% + \concat\toksa\toksc + \concat\toksb\toksd + #1\relax + \edef\next{\toksb{\noexpand\onerule{\the\toksb}}}\next + \appendr\toksa\def + \possibleimplicitfalse + \yyreplacestring\toksa\in\newsymswitch\with\toksb + \toksa{}\toksb{}% +} + +\newtoks\lastname +\newtoks\lastlastname + +\def\termname#1#2{% + \flushact + #1\relax + \lastlastname\lastname + \setnumname\numname + \lastname{}% + #2\relax + \appendr\toksa{ \numname}\appendr\toksb{\noexpand\term{\the\lastlastname}{\the\lastname}}% +} + +\def\setnumname#1{% + \expandafter\let\expandafter\next\csname term\parsernamespace\the\lastname \endcsname + \ifx\next\relax % this is a terminal that was renamed + \expandafter\ifx\csname token\parsernamespace\the\lastname \endcsname\relax % this is probably a symbol from another grammar + \def#1{-1}% + \errmessage{token \csname token\parsernamespace\the\lastname \endcsname\space is undefined}% + \else + \expandafter\tempca\csname token\parsernamespace\the\lastname \endcsname + \edef#1{\fgetelemof{yytranslate}\at\tempca}% + \fi + \else + \let#1\next + \fi +} + +\def\actbraces#1#2#3#4#5\bdend{% + \flushact + \lastname{}% + #2\relax + \edef\next{\lastactsym{\the\lastname}}\next + \possibleimplicittrue +} + +\def\rules#1#2#3{#1} + +\def\oneproduction#1#2#3{#1} + +\def\emptyterm{% + \flushact + \lastname{}% +} + +\def\stringify#1#2#3#4{% + \lastname{"#1"}% +} + +\def\idit#1#2#3#4{% + \lastname{#1}% +} + +\def\charit#1#2#3#4{% + \lastname{'#1'}% +} + +\def\prodprodsep{} + +\def\onesymbol#1#2#3{} + +\def\anint#1#2#3{% + \lastname{#1}% +} + +\def\hexint#1#2#3{% + \lastname\expandafter{\expandafter\number\eattwo#1$_{16}$}% +} + +\newtoks\lastactsym +\newif\ifpossibleimplicit + +\def\flushact{% + \ifpossibleimplicit + \appendr\toksa{ @implicit@}\appendr\toksb{\noexpand\implicitterm{@implicit@}{\the\lastactsym}}% + \lastactsym{}% + \possibleimplicitfalse + \fi +} + +\savecslist{symbols}\yyunion + +% name parser macros + +\def\idstr#1#2{% + \toksb{#1}% + \concat\toksa\toksb + \toksb{#2}% + \concat\toksc\toksb +} + +\def\chstr#1#2{% + \toksb{{\ntt@#1}}% + \concat\toksa\toksb + \toksb{#2}% + \concat\toksc\toksb +} + +\def\sfxi#1#2{\toksb{\/${}_{\idxfont#1}$}\concat\toksa\toksb} + +\def\sfxn#1#2{\toksb{#1}\concat\toksa\toksb} + +\def\dotsp{\futurelet\next\d@tsp} + +\def\d@tsp{% + \ifx\next\sfxi + \else + \ifx\next\qual + \else + \toksb{.}\concat\toksa\toksb + \fi + \fi +} + +\def\optstr#1#2{% + \toksb\expandafter{\eatone#1}% + \optstrextra + \concat\toksa\toksb + \toksb{#2}% + \concat\toksc\toksb +} + +\def\optstrextra{% + \edef\next{\toksb{{$\nx\ulcorner$\ntt\the\toksb$\nx\urcorner$}}}\next +} + +\def\optstrextraesc{% + \edef\next{\toksb{{$\nx\langle$\ntt\the\toksb$\nx\rangle$}}}\next +} + +\def\qual#1#2{\toksb{\/${}_{\rm#2}$}\concat\toksd\toksb} + +\let\idxfont\relax + +\savecs{nameparser}{\idstr\chstr\sfxi\sfxn\qual\optstr\dotsp\optstringextra\idxfont} + +\newtoks\namechars + +\newif\iftracebadnames +\newif\iftracenames + +\def\nameproc#1{% + \ifbootstrapmode + \toksa{}% + \else + \savecs{local-namespace}\getcurrentparser + \tosmallparser + \basicparserinit + \yyparse#1\yyeof\yyeof\endparseinput\endparse + \ifyyparsefail + \iftracebadnames + \ferrmessage{bad name: #1}% + \fi + \toksa{#1}\toksc{}% + \else + \toksa{}\toksb{}\toksc{}\toksd{}% + \iftracenames + \ferrmessage{parsed name: \the\namechars}% + \fi + \the\namechars\relax + \iftracenames + \ferrmessage{processed name: \the\toksa<identifier>, \the\toksb<suffices>, \the\toksc<hash name>, \the\toksd<options>}% + \fi + \expandafter\ifx\csname'parser'\hostparsernamespace\the\toksc \endcsname\relax + \else + \toksa\expandafter{\csname'parser'\hostparsernamespace\the\toksc \endcsname}% + \appendr\toksa{{\currentrulecontext}}% + \fi + \concat\toksa\toksd + \fi + \restorecs{local-namespace}\getcurrentparser + \getcurrentparser + \fi +} + +%% token typesetting macros + +\def\setsafe#1{\def\saferword{#1}\setspecialcharsfrom\saferword} +\def\prettyword#1{\setsafe{#1}\expandafter\edef\csname'parser'\parsernamespace\saferword\endcsname##1{{\ntt\lowercase{\saferword}}}} +\def\prettytoken#1{\setsafe{#1}\expandafter\edef\csname'parser'\parsernamespace\saferword\endcsname##1{{\ntt\saferword}}} +\def\prettytoken@#1{\toksa{}\numberstocharsandspaces#1\end\expandafter\edef + \csname'parser'\parsernamespace\the\toksa\endcsname##1{{\ntt\the\toksa}}% +} +\def\prettywordpair#1#2{\setsafe{#1}\expandafter\def\csname'parser'\parsernamespace\saferword\endcsname##1{{\ntt@#2}}} +\def\prettywordpair@#1#2{% + \setsafe{#1}\toksa{}\numberstocharsandspaces#2\end\expandafter\nameproc\expandafter{\the\toksa}% + \expandafter\edef\csname'parser'\parsernamespace\saferword\endcsname##1{% + \noexpand\ifdisplaytokenraw + {\ntt\saferword}% + \noexpand\else + {\ntt\def\noexpand\_{\char`\noexpand\_}\the\toksa}% + \noexpand\fi + }% +} +\def\prettywordpair@@#1#2{\setsafe{#1}\expandafter\def\csname'parser'\parsernamespace\saferword\endcsname##1{{#2}}} + +% the bootstrap macros + +\def\initbootstrap{% + \restorecslist{bootstrap}\yyunion +} + +\restorecslist{parser-prototypes}\yyunion + +\def\tokendecls#1#2#3{#1} + +\def\stringify#1#2#3#4{\toksa{"#1"}} + +\def\idit#1#2#3#4{\toksb{#1}} + +%\def\anint#1#2#3{\toksc{#1}}% this definition is not needed as all + %information is extracted from yytname + +% note that the definitions below do not handle cases such as +%token '{' "left brace" since the lexer knows how to return '{' + +\def\onesymbol#1#2#3{% + \toksa{}\toksb{}%\toksc{}% + #1#2#3% + \yytoksempty\toksa{\expandafter\charstonumbers\the\toksb\end}% + {\expandafter\toksa\expandafter{\expandafter}\expandafter\charstonumbers\the\toksa\end}% + \yytoksempty\toksb{% + \yytoksempty\toksa{}{% + \immediate\write\tokendefs{\noexpand\tokeneq{\the\toksb}{\the\toksa}}% + }% + }{% + \immediate\write\tokendefs{\noexpand\tokeneq{\the\toksb}{\the\toksa}}% + }% +} + +\def\flexsstatedecls#1#2#3{% bootstrap parser does not produce state declarations so it is fine to share states + \savecslist{local-namespace}\yyunion + \restorecslist{bootstrap:flex-state-declarations}\yyunion + #1\relax + \restorecslist{local-namespace}\yyunion +} + +\let\flexxstatedecls\flexsstatedecls + +\savecslist{bootstrap}\yyunion + +\restorecslist{parser-strict}\yyunion + +\def\idit#1#2#3#4{{\immediate\write\stlist{_register_name(#1);}}} + +\def\hspace#1#2{} + +\savecslist{bootstrap:flex-state-declarations}\yyunion + +\def\charstonumberse#1\end{% + \yystringempty{#1}{}% + {\yystartsinspace{#1}{{\number`\ }\charstonumber@@#1\end}{\charstonumbers@#1\end}}% +} + +\def\charstonumbers@#1{% + {\number`#1}\charstonumberse +} + +\def\charstonumber@@#1 {\charstonumberse} + +\def\charstonumbers#1\end{\edef\next{\toksa{\charstonumberse#1\end}}\next} diff --git a/support/splint/tex/yyxunion.sty b/support/splint/tex/yyxunion.sty new file mode 100644 index 0000000000..fb0d2460b3 --- /dev/null +++ b/support/splint/tex/yyxunion.sty @@ -0,0 +1,33 @@ +% Copyright 2012-2015, Alexander Shibakov +% This file is part of SPLinT +% +% SPLinT is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published by +% the Free Software Foundation, either version 3 of the License, or +% (at your option) any later version. +% +% SPLinT is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty of +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with SPLinT. If not, see <http://www.gnu.org/licenses/>. + +% the original, minimal bootstrapping macros were designed to process +% \prodstyle{\%token} declarations only and are enough to establish the +% interface between the \bison\ parser and the \bison\ lexer; to serve +% the secondary task of providing typesetting information to +% the \bison\ parser, all forms of token declarations must be processed. + +\restorecslist{bootstrap}\yyunion % get the original bootstrap macros + +\def\precdecls#1#2#3#4#5{#3} + +\def\symbolprec#1#2{% + \toksa{}\toksb{}% + #1% + \yytoksempty\toksb{}{\immediate\write\tokendefs{\noexpand\tokenpp{\the\toksb}}}% +} + +\savecslist{bootstrap}\yyunion |