summaryrefslogtreecommitdiff
path: root/support/dktools/conceptions.dox
diff options
context:
space:
mode:
authorNorbert Preining <norbert@preining.info>2020-10-12 03:04:00 +0000
committerNorbert Preining <norbert@preining.info>2020-10-12 03:04:00 +0000
commit0ce40abb18ec02ec6fd6bcc5f21612c88daa7578 (patch)
tree416289fe1448873fd8ca33051f50ad85bffa8aaa /support/dktools/conceptions.dox
parentfdb18507cd80dc17f5a5256153d34668b4f4e61c (diff)
CTAN sync 202010120303
Diffstat (limited to 'support/dktools/conceptions.dox')
-rw-r--r--support/dktools/conceptions.dox179
1 files changed, 0 insertions, 179 deletions
diff --git a/support/dktools/conceptions.dox b/support/dktools/conceptions.dox
deleted file mode 100644
index a2e5ebada6..0000000000
--- a/support/dktools/conceptions.dox
+++ /dev/null
@@ -1,179 +0,0 @@
-/** @page conceptions Conceptions
-
-@section secdkchar dkChar
-
-POSIX systems use the char data type for both non-localized and localized
-texts. An encoding specification configured either in configuration
-files or as appendix in the LANG environment variable chooses a character
-encoding. Most Linux systems today use UTF-8 encoding, the LANG variable
-ends in "utf8" or "UTF-8".
-
-Windows systems use the wchar_t data type (16 bit wide on Windows) for
-localized texts, the encoding is UTF-16. When saving texts to file the
-least significant byte of a wchar_t is written first.
-
-For portability the dkChar data type is mapped to char on non-Windows
-systems and to wchar_t on Windows.
-
-@section secerrorrep Error reporting
-
-A called function should be able to report errors to the caller.
-The DK libraries use the dk4_er_t (DK4 error report type) data type.
-It consists of error code (type int), optionally there are details like
-an error position when processing text or a byte number for a failed
-memory allocation.
-
-If a function detects an error, it can use the functions from
-dk4error.h to fill the report. If the report already contains data for a
-previous error, the report is not overwritten. Sometimes one error results
-in further errors, information about the first error is of most interest
-in these situations.
-
-You can pass a NULL pointer to functions expecting a dk4_er_t pointer argument
-if you are not interested in error details.
-
-If you write functions expecting a dk4_er_t pointer argument you do not need
-special preparations for NULL pointers, as the dk4error.h functions check
-their dk4_er_t pointer argument.
-
-So you can simply write:
-
-~~~~~~~~~~~~~~~{.c}
-dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS);
-~~~~~~~~~~~~~~~
-
-There is no need to bloat the code size for checks like:
-
-~~~~~~~~~~~~~~~{.c}
-if (NULL != erp) {
- dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS);
-}
-~~~~~~~~~~~~~~~
-
-@section secintmax intmax_t, uintmax_t, dk4_im_t, dk4_um_t
-
-Recent C standards have the data types intmax_t/uintmax_t available.
-You can cast every signed/unsigned integer value to these data types without
-loss.
-Unfortunately there are compilers without these data types.
-The dk4_im_t/dk4_um_t types from the DK libraries are for the same purpose.
-They are typedefed to intmax_t/uintmax_t if these types are available,
-otherwise long long/unsigned long long or long/unsigned long are used.
-
-@section sectextproc Text processing
-
-Most DK libraries functions use the dkChar data type to store text in the
-computers memory while the program is running.
-
-Depending on the text origin text in files can be in different encodings,
-i.e.:
-
-- ASCII
-- ANSI
-- UTF-8
-- UTF-16, least significant byte first
-- UTF-16, most significant byte first
-- 32 bit, least significant byte first (rarely used)
-- 32 bit, most significant byte first (rarely used)
-
-So when reading text from files or other data sources into dkChar strings
-we first need to know the file encoding. Text files can have a byte order
-mark (BOM) - Unicode U+FEFF (zero width no-break space) - at start of text.
-For UTF-16/LSB the file starts with 0xFF, 0xFE. For UTF-16/MSB it starts
-with 0xFE, 0xFF. For UTF-8 it starts with 0xEF, 0xBB, 0xBF.
-
-So when processing text you need to know the dkChar size and encoding
-to use in memory and the expected file encoding used by the input.
-After inspecting the start of input one can decide about the input encoding
-really used and recode the bytes to the desired dkChar size and encoding.
-
-Text processing is done using state machines in the DK libraries.
-You provide a callback function handling either a single dkChar character
-or a dkChar line. For line processing you also have to provide buffers
-to store the line.
-After setup you feed the bytes into the state machine, it inspects input
-for BOM, recodes if necessary and calls the callback functions...
-
-Look at the dk-cat program for a working example of characterwise processing.
-
-For linewise processing see the dk-sort program.
-
-@section secgenio Generic I/O using dk4_stream_t
-
-The dk4strm module and related modules provide a generic interface
-to read from or write to a data stream. A data stream can be connected
-to a plain file, a gzip or bzip2 compressed file or any other data source
-or sink.
-
-To construct a stream one has to provide the address of a low level
-data object to handle I/O requests (i.e. a file) and a handler function.
-
-Convenience functions are available in the dk4strmr and dk4strmw modules
-to open plain files, gzip or bzip2 compressed files for reading and writing.
-
-Different strategies are available to use the streams:
-
-- Read and write arbitrary data.<br>
- The functions from the dk4strm module can be used to read and
- write buffers by specifying buffer start address and buffer size
- in bytes.
-- Serialize data to and unserialize data from stream.<br>
- The functions from the dk4strms and dk4strmu modules can be used
- to read/write int16_t, uint16_t, int32_t, uint32_t, double values
- and char strings.<br>
- Integer numbers are stored LSB first.
- For strings the uint16_t string length is written first, text follows.
- The finalizing 0x00 byte is both counted in the string length and
- included in the data. String length is limited to 65534 characters.
- No specific encoding is required for char strings, but UTF-8 is
- recommended.<br>
- For double values the value is converted to string and written as string.
- Routines unserializing data from stream must know which data types
- need to be read in which order.
-- Write text, adjust encoding automatically.<br>
- When writing pure text to stream, one can use the
- dk4stream_set_output_encoding() function from the dk4strmo32 module
- to set the desired output encoding. Now you can use the
- dk4stream_c08_puts() and equivalent functions from the
- dk4strmo08, dk4strmo16, dk4strmo32 and dk4strmodk modules to pass
- texts in a given encoding to the stream, they are converted to the
- desired output encoding automatically.
-
-@section secwincrt Windows and the C runtime
-
-The C runtime libraries should not be used on Windows for
-- drivers,
-- services or
-- programs using threads.
-
-The C runtime libraries are not included in a default Windows installation,
-they are available as separated downloads. So they are probably not present
-when the driver/service is installed (reason for 1, 2).
-Using the C runtime libraries in threads may result in memory leaks
-(reason for 3).
-
-May be the information in the paragraph above is out of date now,
-but I did not hear about all-clear.
-
-The build process for DK tools and libraries on Windows typically uses
-C runtime functions and links static multithreaded release
-runtime libraries.
-
-For most of the modules use in a Windows driver or service is unlikely,
-except:
-- dk4error (required by the modules below)
-- dk4mem
-- dk4sto
-- dk4strw
-- dk4maa*
-
-If you want to use these modules in a project for a Windows driver
-or service, you should copy the modules into your project and use the
-
-~~~~~~~~~~~~~~~
--DDK4_WIN_DENY_CRT=1
-~~~~~~~~~~~~~~~
-
-C compiler option.
-
-*/