diff options
Diffstat (limited to 'README.details')
| -rw-r--r-- | README.details | 560 |
1 files changed, 560 insertions, 0 deletions
diff --git a/README.details b/README.details new file mode 100644 index 00000000..560ea55e --- /dev/null +++ b/README.details @@ -0,0 +1,560 @@ + README.details for sg3_utils + ============================ +Introduction +============ +This package contains low level command line utilities for devices that use +the SCSI command set. Originally the SCSI command set was associated +exclusively with the SCSI Parallel Interface (SPI) transport. SPI has now +almost been completely replaced by the Serial Attached SCSI (SAS) transport +which also accepts the SCSI command set. Additionally many other storage +related transports use the SCSI command set (amongst others); examples are +ATAPI devices (CD/DVDs and tapes), USB mass storage devices (including those +using the newer UAS[P]), Fibre Channel disks, IEEE 1394 storage devices (SBP +protocol), iSCSI, FCoE and SOP devices. Even NVMe which has its own command +set accepts SCSI commands in some contexts; one example is for enclosure +management where NVME-MI has SES Send and SES Receive commands. SES refers +to the SCSI Enclosure Services command set. + +This package originally targeted the Linux SCSI subsystem. Since most +operating systems contain a SCSI command pass-through mechanism, many +utilities within this package have been ported. This README mainly +concentrates on Linux: see the README.freebsd file for the FreeBSD port, +README.solaris for the Solaris port, the README.tru64 file for the Tru64 +(OSF) port and README.win32 for the Windows ports (of which there are two +variants). + +Most utilities within the sg3_utils package work at the SCSI command level. +For example the sg_inq utility issues a SCSI INQUIRY command and decodes the +response. The COVERAGE file has a table containing a row for each SCSI +command issued by this package; to the right of each row is the utility +(sometimes more than one) that issue that SCSI command. The COVERAGE file +has a second table for ATA commands usage. + +Some utilities interface at a slightly higher level, for example: sg_dd, +sgm_dd and sgp_dd. These are closely related to the Unix dd command and +typically issue a sequence of SCSI READ and WRITE commands to copy data. +These utilities are relatively tightly bound to Linux and are not ported to +other Operating Systems. A new utility called ddpt (in a package of the same +name) is more generic while still allowing a copy to be done in terms of +SCSI READ and WRITE commands. ddpt has been ported to other OSes. + +License +======= +All utilities and libraries have either a "2 clause" BSD license or are +"GPL-2ed". The "2 clause" BSD license is taken from the FreeBSD project but +drops the last paragraph that directly refers to the "FreeBSD project". +That BSD license was updated from the "3 clause" to the newer "2 clause" +version on 20180119. To save space various source code files refer to a +file called "BSD_LICENSE" in the main, src and lib directories. The author's +intention is that users may incorporate all or part of the code in their work +as they please. Attribution is encouraged. Please check the code as other +contributors (apart from the author) may also have copyright notices. For a +list of contributors see the CREDITS file. + + +Description +=========== +A web site supporting the sg3_utils package can be found at +https://sg.danny.cz/sg/sg3_utils.html . That page has a table of released +versions for download. The most recent release or beta of sg3_utils may +be found on this page: https://sg.danny.cz/sg in the News section. + +The predecessor to this package was called sg_utils. It is described in +https://sg.danny.cz/sg/uu_index.html and old versions can be downloaded +from the Downloads section of https://sg.danny.cz/sg . + +In the Linux 2.4 kernel series these utilities need to use the SCSI generic +(sg) driver to access SCSI devices. The name of this package (i.e. sg3_utils) +refers to version 3 of the SCSI generic (sg) driver which was introduced at +the beginning of the 2.4 Linux kernel series. Significantly this added a new +SCSI command interface structure (i.e. struct sg_io_hdr) that is more +flexible than the older "sg_header" structure found in the sg driver in the +2.2 and earlier Linux kernel series. The sg_io_hdr structure is also more +flexible than the awkward (and limiting) interface to the +SCSI_IOCTL_SEND_COMMAND ioctl supported by the Linux SCSI mid level. The +version 3 sg driver also added the SG_IO ioctl that is synchronous (i.e. it +issues the requested SCSI command and waits for the response (or a timeout) +before the ioctl returns to the user space program that invoked it). The +SG_IO ioctl is now supported in other parts of the Linux kernel in the 2.6 +series. + +In sg3_utils version 1.27 support has been added for the Linux bsg driver +which use the sg version 4 interface. There seems no point in renaming +this package sg4_utils. The existing utilities just silently support either. +Currently the source build must be able to see the /usr/include/linux/bsg.h +file. Then at run time the /proc/devices pseudo file needs to have an entry +for the bsg driver (appeared around lk 2.6.28). With this in place each +utility at run time checks the device it has been given and if it is a char +device whose major number matches the bsg entry in /proc/devices then the +sg v4 interface is used. Otherwise the sg v3 interface is used. + +Utilities that wish to use the asynchronous SCSI command interface (i.e. via +a write() read() sequence) or issue special "commands" (e.g. bus and device +resets) still need to use the Linux sg driver. Note that various +drivers (e.g. cdrom/sr) have different open() flag and permissions policies +that the user may need to take into account. + +If users have problems or questions about them please contact the author. +Documentation for the Linux sg device driver can be found at: +https://sg.danny.cz/sg/p/sg_v3_ho.html . This is written in DocBook and the +original xml can be found in the same directory with the ".xml" extension. +Postscript and pdf renderings are also in that directory. Older documentation +for the sg version 3 driver can be found at: +https://sg.danny.cz/sg/p/scsi_generic_v3.txt . + +To save the repetition of common code (e.g. SCSI error processing) and +reduce the size of the executable files, a shared library called +libsgutils<num>.so (its Linux name) is created during the build process. +That library is built from the contents of the include and lib +subdirectories. The header files in the include subdirectory can be seen +as the API of libsgutils and are commented with that in mind. The SCSI +pass-through code for the supported operating systems is found in the lib +subdirectory with names like sg_pt_linux.c and sg_pt_win32.c . + +Various distributions (of Linux mainly) distribute sg3_utils as 3 +installable packages. One is a package containing the shared library +discussed above (e.g. libsgutils2-2_1.33-0.1_i386.deb). A second package +contains the utilities (e.g. sg3-utils_1.33-0.1_i386.deb) and depends on the +first package). Finally there is an optional package that contains header +files and a static library (e.g. libsgutils2-dev_1.33-0.1_i386.deb). This +final package is only needed to build other packages (e.g. sdparm) that +wish to use the sg3_utils shared library. + +All the utilities in the src subdirectory have "man" pages that are +placed in the doc subdirectory. There is also a sg3_utils (8) man page that +summarizes common facilities including exit statuses. Additional +information (including each utility's version number) can be found towards +the top of each ".c" file corresponding to the utility name. + +The sg driver in Linux can be seen as having 3 distinct versions: + + v1 lk < 2.2.6 sg_header based relatively unchanged since 1992 + v2 lk >= 2.2.6 enhanced sg_header interface structure [1999/4/16] + v3 lk >= 2.4 additional sg_io_hdr interface structure [2001/1/4] + v3 lk >= 2.6 same interface as found in lk 2.4 [2.6.0: 2003/12/18] + +and the bsg driver supports the sg v4 interface and was added around +lk 2.6.28 . This package is targeted at "v3" and "v4". Another package called +"sg_utils" is targeted at "v2" and to a lesser extent "v1". The "sg_utils" +package has a subset of the utilities found in this package. + +In Linux some sg driver ioctls (notably SG_IO) are defined for many block +devices in lk 2.6 series. In practice this means all SCSI block devices, +ATAPI block devices (mainly CD, DVD and BD optical devices) but _not_ ATA +disks, depending on which kernel configuration options, can be accessed by +the utilities in this package. SATA disks that use the libata kernel library +(or some other SCSI to ATA Translation (SAT) Layer (SATL)) accept SCSI +commands and thus are supported. Support for the SG_IO as been added to the +scsi tape driver (st) in lk 2.6.6 . + +In the src directory the bulk of the utilities are written in relatively +clean POSIX compliant C code with Linux specific system calls and structures +removed and placed in Linux specific files in the lib directory. A small +number of utilities in the src directory do contain Linux specific logic +and are not ported to other OSes (e.g. sg_dd). One utility, sg_scan, has +two separate implementations, one for Linux (sg_scan_linux.c) and one for +Windows (sg_scan_win32.c). The src-lib directory split approach allows +FreeBSD, Solaris, Tru64 and Windows specific code to be isolated to a few +files in the lib directory whose interfaces match those of the Linux +specific code. + +Darwin is not supported because the Apple folks do not want to give their +users a pass-through SCSI interface. The author has read about creative +hackers using a VM containing a real OS to circumvent the Apple restriction. + +C standard is C11 +================== +The C code in this package is written for portability rather than speed. +It assumes a level of C99 compliance (the C standard prior to C11) and +favours POSIX system and library calls over OS specific calls. + +The C code is written in a C++ friendly way and is checked from time to +time that it compiles clean with C++. To accommodate C++ certain C99 +constructs such as designated initializers cannot be used. To build +with C++, C++11 (i.e. the C++ standard from 2011) or later is required. +Finding a common C and C++ syntax for zeroing stack variables (including +aggregates) may need to wait until C23 allows this syntax: + struct example_t ex1 {}; +which C++ introduced in C++11. In the meantime the SG_C_CPP_ZERO_INIT +define (hack) does this. + +The author has not seriously attempted to build this code on MSVC (aka +Visual Studio). There are a few roadblocks (that may be overcome in the +future) that include MSVC being basically a C++ compiler, not a C/C++ +compiler. For some reason MSVC only claims C89 compliance (i.e. the first +C standard from 1989). MSVC 2013 and 2015 are moving closer to C99 +compliance and may be sufficient to compile this package. Another problem +is the assumption of the availability of basic Unix system calls such as +open(). Nearly 20 years ago Microsoft indicated (promised ?) that it +would move in the direction of POSIX compliance, but very little ever +happened. "Talk is cheap, there should be a tax on it." + +Building +======== +This package is designed to be built with the usual: + "./configure ; make ; make install" +sequence. In some situations that may need to be prefixed by a call to +the "./autogen.sh" script which invokes autoconf and automake. That in turn +may require packages containing those utilities to be installed. The +libtool utility is also required. Naturally a C compiler is required +and due to the vagaries of libtool a C++ compiler also. + +The "./configure" takes many command line options with the defaults +being usually sufficient to start with. One quirk is that the location +of the installation is under the /usr/local directory. So the sg_inq +utility will be installed at /usr/local/bin/sg_inq . This is controlled +by the "--prefix=<directory>" option which defaults to +"--prefix=/usr/local". As an example to install the executables in /usr/bin +and disable the creation of the shared library (libsgutils<num>.so) this +invocation could be used: "./configure --prefix=/usr --disable-shared". +To reduce the size of an executable as well try this: +"./configure --prefix=/usr --disable-shared --disable-scsistrings". +Also --disable-shared will produce (relatively) "static" executables in +the src directory that are easier to debug. And +"./configure --enable-debug" will compile with more debug type options, +including more compiler checks and defining "DEBUG" within the src and +lib source files. Most utilities in the src directory set '-vv' (i.e. +equivalent to calling "--verbose" twice) when "DEBUG" is set. + +In Linux there are package build files for "rpm" based and for "deb" based +systems. The 'sg3_utils.spec' file in the main directory can be used like +this: 'rpmbuild -ba sg3_utils.spec' in a rpmbuild tree SPECS directory. +To cross build or make a more widely distributable package then the --target +option may be useful: 'rpmbuild --target=i386 -ba sg3_utils.spec' or +'rpmbuild --target=x86_64 -ba sg3_utils.spec' . The sg3_utils.spec file +in the main directory targets Red Hat systems, an alternative "spec" file +for Suse systems has been placed under the 'suse' directory. + +The 'build_debian.sh' script should build several "deb" packages and place +them in the parent directory. In debian based systems doing +a 'apt-get install build-essential' is one way to get most of build +environment needed if it has not already been loaded. There are now some +problems with this script and the superseded Debian 4.0 ("etch"). See +debian/README.debian4 for a workaround. Amongst other things debian +builds are sensitive to the value in the debian/compat file. If it +contains "7" then it works on lenny and gives warning on squeeze (but +fails on the earlier etch). + +Warning +======= +Many devices use SCSI command sets over transport protocols not normally +associated with SCSI (as defined at https://www.t10.org ). Some of these +devices react poorly (e.g. lock up) when sent SCSI commands that they don't +support. Even sending a supported SCSI command with a field set to an +unexpected value can cause problems. [The author is talking about billions +of USB devices with horrible SCSI implementations.] + +For example, all "SCSI" devices must support the INQUIRY command which the +SCSI-2 standard says should request a 36 byte response. However later SCSI +standards (e.g. SPC-2) have increased that length but some SCSI devices lock +up when they receive a request for anything other than a 36 byte response. + +Any well implemented "SCSI" device should react sensibly when a utility in +sg3_utils sends a SCSI command that it doesn't support. Unfortunately this +cannot be guaranteed. + +Prior to lk 2.6.29 USB mass storage limited sense data to 18 bytes which +caused problems for certain types of descriptor based sense data. An +example of this is the SCSI ATA PASS-THROUGH command with the CK_COND bit +set. + + +Utilities +========= +Here is list in alphabetical order of utilities found in the 'src' +subdirectory of the sg3_utils package: + sginfo, sg_bt_ctl, sg_compare_and_write, sg_copy_results, sgm_dd, sgp_dd, + sg_dd, sg_decode_sense, sg_emc_trespass, sg_format, sg_get_config, + sg_get_elem_status, sg_get_lba_status, sg_ident, sg_inq, sg_logs, + sg_luns, sg_map, sg_map26, sg_modes, sg_opcodes, sg_persist, sg_prevent, + sg_raw, sg_rbuf, sg_rdac, sg_read, sg_read_attr, sg_readcap, + sg_read_block_limits, sg_read_buffer, sg_read_long, sg_reassign, + sg_referrals, sg_rem_rest_elem, sg_rep_density, sg_rep_pip, sg_rep_zones, + sg_request, sg_reset, sg_rmsn, sg_rtpg, sg_safte, sg_sanitize, + sg_sat_identify, sg_sat_phy_event, sg_sat_read_gplog, sg_sat_set_features, + sg_scan, sg_seek, sg_senddiag, sg_ses, sg_ses_microcode, sg_start, + sg_stpg, sg_stream_ctl, sg_sync, sg_test_rwbuff, sg_timestamp, sg_turs, + sg_unmap, sg_verify, sg_vpd, sg_write_buffer, sg_write_long, + sg_write_same, sg_write_verify, sg_write_x, sg_wr_mode, sg_xcopy, sg_zone, + sg_z_act_query + +Each of the above utilities depends on header files found in the 'include' +subdirectory and library code found in the 'lib' subdirectory. Associated +man pages are found in the 'doc' subdirectory. Additional programs found +in the 'archive', 'examples' and 'utils' subdirectories in not build by the +top level build infrastructure. Linux binary distributions of the sg3_utils +package (e.g. "rpm" and debian packages) typically contain the shared +library, the utilities found in the 'src' subdirectory, their associated man +pages and some documentation files (e.g. README, INSTALL, CREDITS, COPYING +and COVERAGE). See the INSTALL file for generic instructions about building +with autotools (e.g. ./configure ). + +Man pages can be read (without building and installing the package) by +going to the 'doc' subdirectory and executing something like this: + $ man ./sg_dd.8 + +To see which SCSI commands (and ATA commands) are used by these utilities +refer to the COVERAGE file. + +Here is a list in alphabetical order of utilities found in the 'examples' +subdirectory: + - sg_excl, scsi_inquiry, sg_sat_chk_power, sg__sat_identify, + sg__sat_phy_event, sg__sat_set_features, sg_sat_smart_rd_data, + sg_simple1, sg_simple2, sg_simple3, sg_simple4, sg_simple5, + sg_simple16 + +Also in that subdirectory is a script to test sg_persist, an example data +file for sg_persist (called "transport_ids.txt") and an example data file for +sg_reassign (called "reassign_addr.txt"). There are several scripts +for 'sg_senddiag -pf -raw=-' that will put some SAS disk phys into +a "compliant jitter tolerance pattern" (CJTPAT). + +The 'testing' subdirectory contains source and a Makefiles to test +kernel pass-through and associated drivers, mainly for Linux. There is +both C code (with the extension ".c") and C++ code (with the extension +".cpp"). There is a "Makefile" to build the C + C++ code. The Makefile +depends on some object files from the "lib" subdirectory. So a sequence +like this may be required prior to invoking make: "cd <top_of_package> ; +./configure ; cd lib ; make ; cd ../testing". + +Here is a list in alphabetical order of utilities found in the 'testing' +subdirectory: + - bsg_queue_tst, sgh_dd (C++), sg_iovec_tst, sg_queue_tst, sg_sense_tst, + sg_tst_async (C++), sg_tst_context (C++), sg_tst_excl (C++), + sg_tst_excl2 (C++), sg_tst_excl3 (C++) + +The 'utils' subdirectory contains source and a Makefile to build "hxascdmp" +which accepts binary data from stdin (or a file on the command line) and +outputs an ASCII-HEX and ASCII representation of it. It is similar to the +Unix od command. There is also code to sg_chk_asc.c which checks a given +text file (typically a copy of https://www.t10.org/lists/asc-num.txt ) and +checks it against the asc/ascq text strings held in sg_lib_data.c . + +The 'doc' subdirectory contains a README file containing the urls of +various related documents. + +The 'scripts' subdirectory contains some Bourne (bash) shell scripts that +rely on utilities in the main directory. One script uses the sdparm utility. +These scripts are described in the scripts/README file and have usage +messages. + + +Notes for utilities without man pages +===================================== +These utils are found in the 'examples' subdirectory. + +The "scsi_inquiry" program shows the use of the SCSI_IOCTL_SEND_COMMAND +ioctl to send a SCSI INQUIRY command. That ioctl() is supported by the +SCSI sub system mid level and so is common to all sd, sr, st and sg devices. +That ioctl is deprecated in the lk 2.6 series. This program has been placed +in the "examples" subdirectory. + +"sg_simple1" and "sg_simple2" are example programs demonstrating calls +to the SCSI INQUIRY and TEST UNIT READY commands. They only differ in their +error processing: sg_simple1 uses sg_lib.[hc] for error processing while +sg_simple2 does its own more primitive checks. + +"sg_simple3" tests out user space scatter gather added to the version 3 +sg driver. + +"sg_simple4" shows the INQUIRY command using mmap-ed IO to obtain its +response buffer. + +"sg_simple5" also sends and INQUIRY and TEST UNIT READY commands. It +uses the generic pass through mechanism based on sg_pt.h . It will +currently build in Linux and FreeBSD (with "make -f Makefile.freebsd"). +It has extensive error checking code. + +"sg_simple16" attempts to send a 16 byte SCSI command, READ_16, to the +scsi device. This is only supported for lk >= 2.4.15 and for adapter +drivers that indicate that they have 16 byte CDB capability (otherwise +DID_ABORT will appear in the host_status). + +"sg_sat_chk_power" attempts to push an ATA CHECK POWER MODE command +through the SAT-defined ATA PASS_THROUGH (16) SCSI command. That +ATA command needs to read the "FIS" registers after the command is +completed which involves using the ATA Status Return (sense data) +descriptor (as defined in SAT). + +"sg_sat_smart_rd_data" attempts to push an ATA SMART/READ DATA command +through the SAT-defined ATA PASS_THROUGH (16) SCSI command. If +successful, the 256 word (512 byte) response is output. + +"sg_tst_excl" and "sg_tst_excl2" use multiple threads to bombard the +given device with O_EXCL open flags, so only one should succeed at a +time. While holding O_EXCL control a thread attempts a double increment +on an integer in the given LBA. If the integer starts even (after the +first read) then it should remain even if the O_EXCL flag is doing its job. +The "sg_tst_excl" variant uses the Linux SG_IO v3 interface while the +"sg_tst_excl2" uses the more generic sg_pt infrastructure. + +"sg_tst_excl3" is a variant of "sg_tst_excl2". "sg_tst_excl3" only does +the double increment from the first thread, each time using O_EXCL on +open. The remaining threads check the value is even, each time doing +an open without the O_EXCL flag. + +"bsg_queue_tst" sends an INQUIRY command via the Linux SG_IO v4 interface +which is used by the bsg driver. So it will take device names like +"/dev/bsg/6:0:0:0". It tests if sending repeated INQUIRYs with +the BSG_FLAG_Q_AT_HEAD or BSG_FLAG_Q_AT_TAIL flag makes any difference. + +"sg_tst_async" is a test harness for the Linux sg driver. It is multi +threaded, submitting either TEST UNIT READY, READ(16) or WRITE(16) SCSI +commands asynchronously. Each thread opens a file descriptor and submits +those commands up to the queue limit (sg driver has a per file descriptor +queue limit of 16). Multiple threads doing the same thing act as a +multiplier to that queue limit. + + +NVME Support +============ +Firstly the author has no intention of extending this package to contain +general purpose NVMe utilities. That leaves the areas where SCSI overlaps +with NVMe. There was a SCSI to NVMe Translation Layer (SNTL) driver in the +Linux kernel based on a white paper from NVM Express. Intel has withdrawn +that driver and T10 (SCSI) and NVM Express have made no further attempts +to standardize a SNTL. Given the SCSI to ATA Translation Layer (SATL) which +is standardized by T10, it is pretty clear what a SNTL should do. + +The NVMe Management Interface (NVME-MI) committee have decided to use SES-3 +standard from T10 via the newly added SES Send and SES Receive MI commands. +So the sg_ses utility and this package's library have been extended to use +these commands when a NVMe device (typically a disk enclosure) is detected. +This has been tested by a disk vendor who is happy with the results. Other +user reports are welcome as the author does not have equipment to test +this. + +Other utilities in this package that use the SES Send and Receive commands, +or the SNTL in the library are sg_senddiag, sg_inq, sg_raw and sg_readcap. + + +Command line processing +======================= +These utilities can be divided into 3 groups when their handling of command +line arguments is considered: + - ad hoc, typically in a short form only, sometimes longer (e.g. + "sg_logs -pcb /dev/sdc") + - inspired by the dd Unix command (e.g. sg_dd, sgm_dd, sgp_dd, sg_read) + - recent utilities use "getopt_long" (see "man getopt_long") + type command lines. These have short form (starting with "-") + and corresponding longer form (starting with "--") options. + +The older utilities that use ad hoc options, in alphabetical order: + - sg_emc_trespass, sginfo(1/2), sg_inq, sg_logs, sg_map, sg_modes, + sg_opcodes, sg_rbuf, sg_rdac, sg_readcap, sg_reset, sg_scan (Linux), + sg_senddiag, sg_start, sg_test_rwbuf, sg_turs +In sg3_utils version 1.23 the following utilities from this group were +converted to have a dual getopt_long/ad_hoc interface, defaulting to +the getop_long interface: + - sg_inq, sg_logs, sg_modes, sg_opcodes, sg_rbuf, sg_readcap, + sg_senddiag, sg_start, sg_turs +These can be switched back to the older (backward compatible) ad hoc +interface by defining the SG3_UTILS_OLD_OPTS environment variable +or using '-O' as the first command line option. + +The more recent utilities that use "getopt_long" only are: + - sg_bt_ctl, sg_compare_and_write, sg_decode_sense, sg_format, + sg_get_config, sg_get_lba_status, sg_ident, sg_luns, sg_map26, + sg_persist, sg_prevent, sg_raw, sg_read_attr, sg_read_block_limits, + sg_read_buffer, sg_read_long, sg_reassign, sg_referrals, sg_rep_pip, + sg_rep_zones, sg_requests, sg_rmsn, sg_rtpg, sg_safte, sg_sanitize, + sg_sat_identify, sg_sat_phy_event, sg_sat_read_gplog, + sg_sat_set_features, sg_scan(w), sg_seek, sg_ses, sg_ses_microcode, + sg_stpg, sg_stream_ctl, sg_sync, sg_test_rwbuf, sg_timestamp, sg_unmap, + sg_verify, sg_vpd, sg_write_buffer, sg_write_long, sg_write_same, + sg_write_verify, sg_write_x, sg_wr_mode, sg_zone, sg_z_act_query + + +Dangerous code +============== +This C code snippet: + unsigned char uc = 0x80; + uint64_t ull; + ull = (uc << 24); +Somewhat surprisingly sets ull to: + ull: 0xffffffff80000000 +This result is due to the 'unary conversion' of uc to a (32 bit signed) +'int' before the shift. The resultant type from the shift is also an int +and it has its top bit set so there is sign extension when it is assigned +into a 64 bit unsigned integer. Making sure there is no conversion to 'int' +solves the problem. In this case if uc is declared as unsigned int the +result will be as expected (i.e. 0x80000000). + + +Bypassing the somewhat dangerous shift operators +================================================ +The shift operators in C are "<<" and ">>". They can be dangerous (as shown +in the above section) or tedious and hence error prone to use. However they +are often needed to cope with the translation of integers on the host OS to +the corresponding representation within a SCSI command or parameter data +moved to or from a SCSI device. The Logical Block Address (LBA) is a good +example; it is either 32 or 64 bits long typically (i.e. 4 or 8 bytes +respectively). The host machine representation may be big or little endian +and may prefer or require alignment to a particular memory address boundary +(e.g. module 4 (or in 'C' code: "(lba % 4) == 0")). For SCSI commands and +the parameter data moved to or from a SCSI device, the integer +representation is big endian and it is unaligned. + +Recent versions of this package have replaced the explicit use of the C +shift operators with a group of functions modelled on those found in the +Linux kernel. These functions contain either "get_unaligned" or +"put_unaligned" in their names and are found in the asm/unaligned.h +header. This package contains the sg_unaligned.h header that implements +a similar set of functions. The current implementation favours correctness +over speed. The functions in the package use a "sg_" prefix but otherwise +use the same function name as the Linux kernel for the same action. + +An example of the change made to a snippet of sg_write_buffer.c may +clarify this change. The old code was: + + wbufCmdBlk[3] = (unsigned char)((buffer_offset >> 16) & 0xff); + wbufCmdBlk[4] = (unsigned char)((buffer_offset >> 8) & 0xff); + wbufCmdBlk[5] = (unsigned char)(buffer_offset & 0xff); + +and it has been replaced by: + + sg_put_unaligned_be24(buffer_offset, wbufCmdBlk + 3); + +The Linux kernel only supplies "unaligned" functions for 16, 32 and 64 +bit quantities. SCSI commands also have cases of 24 and 48 bit numbers +so sg_unaligned.h contains support for those plus a variant where the +byte length is passed as an argument. + +The unaligned functions are inlined for speed (at the possible expense of +space) and now have specializations depending whether the host is big or +(more likely) little endian. These functions can be broken down to a +memcpy() and optionally a byte-swap for 16, 32 and 64 bit operations. +The memcpy() takes care of alignment while the byte-swap (bswap_16(), +bswap_32() and bswap_64() ) addresses integer endianness. If the host is +little endian and a little endian variant of the unaligned functions is +requested, then no byte-swap is required. These specializations can be +"compiled out" with this configure option: './configure +--disable-fast-lebe' in which case the classic "C shifting" technique is +used to implement all the unaligned functions. + +Associated with the above change, fixed length integer types seem a better +fit for SCSI command and parameter integers than the traditional integer +types in the C language. Fixed length integer types were standardized in +C99 and require the inclusion of <stdint.h>. For example this means for +an integer that will represent a 64 bit LBA, to favour using "uint64_t" +over the "unsigned long long" type. Also "unsigned char" has mostly been +replaced by "uint8_t" as the 8 bit (unsigned) byte type; "char" is still +used for ASCII text. + + +Coding Style +============ +Everyone has their own C/C++ coding style and the author is no different. +In terms of the GNU indent command: + indent -i4 -il0 -nut -br -npcs -ncs -ce +is pretty close. That is similar to the Linux kernel coding style but +with 4 space indentations and no tabs. + + +Other SCSI and storage tools +============================ +See https://sg.danny.cz/sg/tools.html + + +Douglas Gilbert dgilbert@interlog.com +26th August 2022 |