/** @page conceptions Conceptions @section secdkchar dkChar POSIX systems use the char data type for both non-localized and localized texts. An encoding specification configured either in configuration files or as appendix in the LANG environment variable chooses a character encoding. Most Linux systems today use UTF-8 encoding, the LANG variable ends in "utf8" or "UTF-8". Windows systems use the wchar_t data type (16 bit wide on Windows) for localized texts, the encoding is UTF-16. When saving texts to file the least significant byte of a wchar_t is written first. For portability the dkChar data type is mapped to char on non-Windows systems and to wchar_t on Windows. @section secerrorrep Error reporting A called function should be able to report errors to the caller. The DK libraries use the dk4_er_t (DK4 error report type) data type. It consists of error code (type int), optionally there are details like an error position when processing text or a byte number for a failed memory allocation. If a function detects an error, it can use the functions from dk4error.h to fill the report. If the report already contains data for a previous error, the report is not overwritten. Sometimes one error results in further errors, information about the first error is of most interest in these situations. You can pass a NULL pointer to functions expecting a dk4_er_t pointer argument if you are not interested in error details. If you write functions expecting a dk4_er_t pointer argument you do not need special preparations for NULL pointers, as the dk4error.h functions check their dk4_er_t pointer argument. So you can simply write: ~~~~~~~~~~~~~~~{.c} dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS); ~~~~~~~~~~~~~~~ There is no need to bloat the code size for checks like: ~~~~~~~~~~~~~~~{.c} if (NULL != erp) { dk4error_set_simple_error_code(erp, DK4_E_INVALID_ARGUMENTS); } ~~~~~~~~~~~~~~~ @section secintmax intmax_t, uintmax_t, dk4_im_t, dk4_um_t Recent C standards have the data types intmax_t/uintmax_t available. You can cast every signed/unsigned integer value to these data types without loss. Unfortunately there are compilers without these data types. The dk4_im_t/dk4_um_t types from the DK libraries are for the same purpose. They are typedefed to intmax_t/uintmax_t if these types are available, otherwise long long/unsigned long long or long/unsigned long are used. @section sectextproc Text processing Most DK libraries functions use the dkChar data type to store text in the computers memory while the program is running. Depending on the text origin text in files can be in different encodings, i.e.: - ASCII - ANSI - UTF-8 - UTF-16, least significant byte first - UTF-16, most significant byte first - 32 bit, least significant byte first (rarely used) - 32 bit, most significant byte first (rarely used) So when reading text from files or other data sources into dkChar strings we first need to know the file encoding. Text files can have a byte order mark (BOM) - Unicode U+FEFF (zero width no-break space) - at start of text. For UTF-16/LSB the file starts with 0xFF, 0xFE. For UTF-16/MSB it starts with 0xFE, 0xFF. For UTF-8 it starts with 0xEF, 0xBB, 0xBF. So when processing text you need to know the dkChar size and encoding to use in memory and the expected file encoding used by the input. After inspecting the start of input one can decide about the input encoding really used and recode the bytes to the desired dkChar size and encoding. Text processing is done using state machines in the DK libraries. You provide a callback function handling either a single dkChar character or a dkChar line. For line processing you also have to provide buffers to store the line. After setup you feed the bytes into the state machine, it inspects input for BOM, recodes if necessary and calls the callback functions... Look at the dk-cat program for a working example of characterwise processing. For linewise processing see the dk-sort program. @section secgenio Generic I/O using dk4_stream_t The dk4strm module and related modules provide a generic interface to read from or write to a data stream. A data stream can be connected to a plain file, a gzip or bzip2 compressed file or any other data source or sink. To construct a stream one has to provide the address of a low level data object to handle I/O requests (i.e. a file) and a handler function. Convenience functions are available in the dk4strmr and dk4strmw modules to open plain files, gzip or bzip2 compressed files for reading and writing. Different strategies are available to use the streams: - Read and write arbitrary data.
The functions from the dk4strm module can be used to read and write buffers by specifying buffer start address and buffer size in bytes. - Serialize data to and unserialize data from stream.
The functions from the dk4strms and dk4strmu modules can be used to read/write int16_t, uint16_t, int32_t, uint32_t, double values and char strings.
Integer numbers are stored LSB first. For strings the uint16_t string length is written first, text follows. The finalizing 0x00 byte is both counted in the string length and included in the data. String length is limited to 65534 characters. No specific encoding is required for char strings, but UTF-8 is recommended.
For double values the value is converted to string and written as string. Routines unserializing data from stream must know which data types need to be read in which order. - Write text, adjust encoding automatically.
When writing pure text to stream, one can use the dk4stream_set_output_encoding() function from the dk4strmo32 module to set the desired output encoding. Now you can use the dk4stream_c08_puts() and equivalent functions from the dk4strmo08, dk4strmo16, dk4strmo32 and dk4strmodk modules to pass texts in a given encoding to the stream, they are converted to the desired output encoding automatically. @section secwincrt Windows and the C runtime The C runtime libraries should not be used on Windows for - drivers, - services or - programs using threads. The C runtime libraries are not included in a default Windows installation, they are available as separated downloads. So they are probably not present when the driver/service is installed (reason for 1, 2). Using the C runtime libraries in threads may result in memory leaks (reason for 3). May be the information in the paragraph above is out of date now, but I did not hear about all-clear. The build process for DK tools and libraries on Windows typically uses C runtime functions and links static multithreaded release runtime libraries. For most of the modules use in a Windows driver or service is unlikely, except: - dk4error (required by the modules below) - dk4mem - dk4sto - dk4strw - dk4maa* If you want to use these modules in a project for a Windows driver or service, you should copy the modules into your project and use the ~~~~~~~~~~~~~~~ -DDK4_WIN_DENY_CRT=1 ~~~~~~~~~~~~~~~ C compiler option. */