diff options
author | Norbert Preining <norbert@preining.info> | 2020-10-12 03:04:00 +0000 |
---|---|---|
committer | Norbert Preining <norbert@preining.info> | 2020-10-12 03:04:00 +0000 |
commit | 0ce40abb18ec02ec6fd6bcc5f21612c88daa7578 (patch) | |
tree | 416289fe1448873fd8ca33051f50ad85bffa8aaa /support/dktools/conceptions.dox | |
parent | fdb18507cd80dc17f5a5256153d34668b4f4e61c (diff) |
CTAN sync 202010120303
Diffstat (limited to 'support/dktools/conceptions.dox')
-rw-r--r-- | support/dktools/conceptions.dox | 179 |
1 files changed, 0 insertions, 179 deletions
diff --git a/support/dktools/conceptions.dox b/support/dktools/conceptions.dox deleted file mode 100644 index a2e5ebada6..0000000000 --- a/support/dktools/conceptions.dox +++ /dev/null @@ -1,179 +0,0 @@ -/** @page conceptions Conceptions - -@section secdkchar dkChar - -POSIX systems use the char data type for both non-localized and localized -texts. An encoding specification configured either in configuration -files or as appendix in the LANG environment variable chooses a character -encoding. Most Linux systems today use UTF-8 encoding, the LANG variable -ends in "utf8" or "UTF-8". - -Windows systems use the wchar_t data type (16 bit wide on Windows) for -localized texts, the encoding is UTF-16. When saving texts to file the -least significant byte of a wchar_t is written first. - -For portability the dkChar data type is mapped to char on non-Windows -systems and to wchar_t on Windows. - -@section secerrorrep Error reporting - -A called function should be able to report errors to the caller. -The DK libraries use the dk4_er_t (DK4 error report type) data type. -It consists of error code (type int), optionally there are details like -an error position when processing text or a byte number for a failed -memory allocation. - -If a function detects an error, it can use the functions from -dk4error.h to fill the report. If the report already contains data for a -previous error, the report is not overwritten. Sometimes one error results -in further errors, information about the first error is of most interest -in these situations. - -You can pass a NULL pointer to functions expecting a dk4_er_t pointer argument -if you are not interested in error details. - -If you write functions expecting a dk4_er_t pointer argument you do not need -special preparations for NULL pointers, as the dk4error.h functions check -their dk4_er_t pointer argument. - -So you can simply write: - -~~~~~~~~~~~~~~~{.c} -dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS); -~~~~~~~~~~~~~~~ - -There is no need to bloat the code size for checks like: - -~~~~~~~~~~~~~~~{.c} -if (NULL != erp) { - dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS); -} -~~~~~~~~~~~~~~~ - -@section secintmax intmax_t, uintmax_t, dk4_im_t, dk4_um_t - -Recent C standards have the data types intmax_t/uintmax_t available. -You can cast every signed/unsigned integer value to these data types without -loss. -Unfortunately there are compilers without these data types. -The dk4_im_t/dk4_um_t types from the DK libraries are for the same purpose. -They are typedefed to intmax_t/uintmax_t if these types are available, -otherwise long long/unsigned long long or long/unsigned long are used. - -@section sectextproc Text processing - -Most DK libraries functions use the dkChar data type to store text in the -computers memory while the program is running. - -Depending on the text origin text in files can be in different encodings, -i.e.: - -- ASCII -- ANSI -- UTF-8 -- UTF-16, least significant byte first -- UTF-16, most significant byte first -- 32 bit, least significant byte first (rarely used) -- 32 bit, most significant byte first (rarely used) - -So when reading text from files or other data sources into dkChar strings -we first need to know the file encoding. Text files can have a byte order -mark (BOM) - Unicode U+FEFF (zero width no-break space) - at start of text. -For UTF-16/LSB the file starts with 0xFF, 0xFE. For UTF-16/MSB it starts -with 0xFE, 0xFF. For UTF-8 it starts with 0xEF, 0xBB, 0xBF. - -So when processing text you need to know the dkChar size and encoding -to use in memory and the expected file encoding used by the input. -After inspecting the start of input one can decide about the input encoding -really used and recode the bytes to the desired dkChar size and encoding. - -Text processing is done using state machines in the DK libraries. -You provide a callback function handling either a single dkChar character -or a dkChar line. For line processing you also have to provide buffers -to store the line. -After setup you feed the bytes into the state machine, it inspects input -for BOM, recodes if necessary and calls the callback functions... - -Look at the dk-cat program for a working example of characterwise processing. - -For linewise processing see the dk-sort program. - -@section secgenio Generic I/O using dk4_stream_t - -The dk4strm module and related modules provide a generic interface -to read from or write to a data stream. A data stream can be connected -to a plain file, a gzip or bzip2 compressed file or any other data source -or sink. - -To construct a stream one has to provide the address of a low level -data object to handle I/O requests (i.e. a file) and a handler function. - -Convenience functions are available in the dk4strmr and dk4strmw modules -to open plain files, gzip or bzip2 compressed files for reading and writing. - -Different strategies are available to use the streams: - -- Read and write arbitrary data.<br> - The functions from the dk4strm module can be used to read and - write buffers by specifying buffer start address and buffer size - in bytes. -- Serialize data to and unserialize data from stream.<br> - The functions from the dk4strms and dk4strmu modules can be used - to read/write int16_t, uint16_t, int32_t, uint32_t, double values - and char strings.<br> - Integer numbers are stored LSB first. - For strings the uint16_t string length is written first, text follows. - The finalizing 0x00 byte is both counted in the string length and - included in the data. String length is limited to 65534 characters. - No specific encoding is required for char strings, but UTF-8 is - recommended.<br> - For double values the value is converted to string and written as string. - Routines unserializing data from stream must know which data types - need to be read in which order. -- Write text, adjust encoding automatically.<br> - When writing pure text to stream, one can use the - dk4stream_set_output_encoding() function from the dk4strmo32 module - to set the desired output encoding. Now you can use the - dk4stream_c08_puts() and equivalent functions from the - dk4strmo08, dk4strmo16, dk4strmo32 and dk4strmodk modules to pass - texts in a given encoding to the stream, they are converted to the - desired output encoding automatically. - -@section secwincrt Windows and the C runtime - -The C runtime libraries should not be used on Windows for -- drivers, -- services or -- programs using threads. - -The C runtime libraries are not included in a default Windows installation, -they are available as separated downloads. So they are probably not present -when the driver/service is installed (reason for 1, 2). -Using the C runtime libraries in threads may result in memory leaks -(reason for 3). - -May be the information in the paragraph above is out of date now, -but I did not hear about all-clear. - -The build process for DK tools and libraries on Windows typically uses -C runtime functions and links static multithreaded release -runtime libraries. - -For most of the modules use in a Windows driver or service is unlikely, -except: -- dk4error (required by the modules below) -- dk4mem -- dk4sto -- dk4strw -- dk4maa* - -If you want to use these modules in a project for a Windows driver -or service, you should copy the modules into your project and use the - -~~~~~~~~~~~~~~~ --DDK4_WIN_DENY_CRT=1 -~~~~~~~~~~~~~~~ - -C compiler option. - -*/ |