iorate(1) General Commands Manual iorate(1) NAME iorate - tests I/O subsystems under various loads SYNOPSIS iorate [ -a ] [ -l ] [ -n ] [ -s ] [ -u ] [ -v ] [ -d <debug_level> ] [ -c skew_shift ] [ -r target_rate ] [ -f device_file ] [ -p pattern_file ] [ -t test_file ] [ -i iops_file ] [ -o output_base ] DESCRIPTION The iorate utility conducts a series of I/O tests. Each test is defined in test_file using a collection of patterns. Each pattern is defined in pat- tern_file. The tests are run simultaneously against all of the devices in device_file. A useful user guide can be downloaded from the web page at: http://iorate.org. Each pattern defines a specific I/O pattern that will be run. Each pattern can be either read or write, either random or sequential. For sequential patterns, a maximum number of contiguous sequential tests can be specified, after which the next random sequence will start at a new, random location. All I/O for a pattern is of a fixed size. Each test uses one or more patterns, specified as a percentage of the I/Os to be performed. The tests specify how long they should run, where they start in the file, and the size of the device that they should use for testing. The can also specify how much of the test period should be ig- nored for performance characteristic reasons to allow time for the test to achieve a 'steady state,' which can be useful when testing disk arrays with large amounts of cache. Iorate generates a log file of the test being run. The log file contains a copy of the configuration files being used (device_file, pattern_file and test_file). These will not have any comments, and will be printed directly from the information that will be used to drive the actual tests. These copies, if split out into the separate input files, will reproduce the same test. They are included in the log to show exactly what was run, and to allow that test to be reproduced at any later time. The log also records the starting and ending times of each test, by device. Any errors encoun- tered are also included in the log. Note that if an iops_file was used, or if the target rate was scaled using a target_rate, then the iops target for each test in the test_file section of the log will have a value reflecting these target rate adjustments. Spliting these sections out to run a new test will run the exact same test again, at the adjusted target rate, without the need for target iops or rate adjustments. If the adjustments (iops file and/or target rate per- centage) are applied again, the tests may be altered. In addition to the log file, iorate generates a performance file for the test group. The performance file contains detailed statistics on each test run on each device. Each line in the file represents the data for a single test on a single device. The first line of the file is a header that de- scribes the contents of each test line. The file uses tab separated val- ues, that are easy to read with scripts or spreadsheet programs (though the alignments are a challenge if read with a standard editor or simply printed). The fields are: test_number -- Number of the test being run test_name -- Name of the test being run device_num -- Number of the device for this line device_name -- Name of the device for this line total_sec -- Total seconds the test was run measured_sec -- Time that results are measured over reads -- Number of reads from this device KB_read -- KB of data reads from the device read_resp -- Avg. response time for all device reads writes -- Number of writes to this device KB_written -- KB of data written to the device write_resp -- Avg. response time for all device writes total_I/Os -- Number of I/Os to this device total_KB -- KB of data transfered to the device total_resp -- Avg. response time for all device I/Os I/Os_per_sec -- Avg. number of device I/Os per second KB_per_sec -- Avg. KB of data transferred per second dev_copy -- Which copy of the device this is for (for devices with multiples) Each line is a set of performance numbers for a single device. To get the total for a given test, the numbers for each of the devices needs to be summarized for each test. The gen_sums AWK program provided will generate such test summaries, which are often more useful for comparisons. If the pair of output files (performance and log) are available from any test, the numbers can be examined, and the test can be reproduced. USAGE The log file is usually named iorate.log, and the performance data is in iorate.perf. If the -o option is used, log file will be output_base.log and the performance data will be output_base.perf. If large numbers of test passes will be run, it is suggested that sub-directories be built for each group of test. The output_base parameter can include sub-directory names as well, but those sub-directories must exist before iorate begins (iorate will not create sub-directories). The -l limits option causes iorate to display file limits and exit (no testing). This is used in the makefile to verify support for large file sizes. The -n no testing option turns off test execution. The program will still do all setup checks, including a seek to test the size of each file. The program will still fork off a child process to do the testing. However, the child will exit quickly. The -s silent option stops all but the error and warning output to standard out. The other information normally reported, such as test configurations and progress, is still recorded in the log file for later reference. The -u use direct I/O option sets the O_DIRECT (or appropriate) flag when opening the data files for testing. This should cause file system buffers to be bypassed and provide a more accurate measure of the true array per- formance. The -a allow causes the checks on read sizes to be lenient. This option is helpful on some platforms where the reads from files on file systems may not return as many bytes as the read I/O is requesting. The -v verbose option turns on verbose mode. Since this will generate log information for every I/O of every device, it is not recommended for use in a lengthy test. It will have a dramatic effect on the I/O rates of many tests, and should not be turned on while real test numbers are to be gener- ated. Note that multiple processes (one per active test process) will be writing to the same log file, so the output lines may get a bit messy with more than one active device. The log file will be very busy, and may grow to be quite large. The -d <debug_level> debug option tells iorate to turn on a number of de- bugging outputs. This will provide a large number of messages that may not be understandable to someone not reading the program source code. The op- tional debug_level allows the user to shift from some information (10) to an excrutiatingly detailed level of information (1000+). The -c skew_shift option tells iorate to alter the sub-LUN skew data. This shifts skew_shift percent of the data between the target levels, with hot regions becoming cold, cold regions becoming warm, and warm regions ecoming hot. This is limited to be 0 and 99. Note that the 99% shift hsd 100% of the data on a new performance level, and thus is not close to the 0% level. The -r target_rate option tells iorate to run at a reduced target rate. This will cause each test to run at this percentage of the maximum iops given for that test (this has no effect if an iops has not been specified for that test). The target (base) iops rate for each test may be specified in the test_file, or in the iops_file. Valid target rates are from 0 (no target rate used - leave iops values alone) to 100% (full test). PARAMETER FILES Iorate requires that all three configuration files be properly formatted. Any errors in the files will be reported as the program starts, and may cause testing to be aborted. In general, anything following a hash mark (#) is considered a comment to the end of the physical line (marked by a carriage return). Each logical line describes a single item (device, pat- tern, test, or iops rating) and ends in a semicolon (;). Logical lines (single items) can span many input lines. In general, each item will have an item type identifier (device, pattern, test, or target) and then information about that item. The details of the various options for each item type are noted in the file descriptions be- low. When providing sizes, a number can be followed by a sizing factor (KB, MB, GB, TB). Each step will increase the size by a multiplier of 1024 (not 1000). When providing time values, a number must be followed by a time type (sec, min, hour) which will scale the number to seconds. When provid- ing numbers, general math functions also work, so 4 * 4 GB is the same as 16 GB. DEVICE FILE Iorate will assume a device_file name of devices.ior if none is specified. Each device entry describes a single device, such as: Device 1 = "/mnt/datafile.tst" capacity 3GB; Any valid file name can be given, whether a raw device file or the name of a file system file that exists and has the stated size. Device numbers must be unique, but are also optional (if not provided, device numbers will be assigned in order automatically - a record of which is provided in the log file). The capacity is the usable space in the device for testing - the actual size can be much larger. There is a minumum offset of 8 KB, which is also the default, to make sure that any device header such as a label is not overwritten. Alternate val- ues can be set with offset <size>. The capacity of the device is the us- able capacity starting at the offset point. I/O can be targetted at the last 2 GB of a 36 GB device using offset 34 GB capacity 2 GB. The minumum block size, which is also the default, is 512 B for most oper- ating systems (1024 B on HP-UX). Alternate values can be set with block size <size>. This block size is the minimum I/O size for the device, and all I/O will be aligned to this size, as well as being a multiple of it. The patterns in use from any test must be a multiple of this size. Rather than placing the same file information in the device file repeti- tively, a single definition can have multiple tasks set to access it using count <count>. Each process will open the file separately, choosing a ran- dom start position and performing seeks and I/O separately from the others. The file can be flagged as being read-only by specifiying read only. This will cause iorate to open the file for reads but not writes - and any tests attempting to do writes will cause iorate to fail. Unless the file is set to read only, iorate will attempt to open the file for reads and writes - even if none of the active patterns in the tests will do writes. There is also an option to lock the file by specifiying lock. This will use the fcntl() system call to lock the area of the device specified by the offset and capacity. Note that these are exclusive (for writes), non- blocking locks - if they cannot be obtained, the entire test will be aborted. Note that multiple read locks on the same area of a file are per- mitted, so setting the devices to read only will allow them to be shared even if locking is requested. If the device is not set to read only, then iorate will attempt to aquire a write lock (exclusive) on the defined re- gion of the file - even if none of the active patterns in the tests will do writes. PATTERN FILE Iorate will assume a pattern_file name of patterns.ior if none is speci- fied. Each pattern entry describes a single pattern, such as: Pattern 1 = "2K Seq Read" io size 2KB sequential read; The pattern number must be given, since that is how the patterns are chosen from the tests. They need not be sequential. The io size <size> parameter specifies the size of all I/Os that will be driven from this pattern. To mix I/O sizes, tests can be configured that use multiple patterns. The area to run the pattern against can be specified with from <size> or from <count>% to set the starting location, and size <size> or size <count>% to set the size to test against (starting at the from location). Any percentage sizes are scaled from the active size of the test using this pattern. By default, the pattern will do all sequential read. Setting random will change the pattern to random address selections. For a mix, use max se- quential <count> to limit the number of sequential I/Os that will be done in a row before a new, random location is chosen. Note that setting <count> to 10 will allways to 10 sequential I/Os before selecting a new lo- cation. To mix the I/O types, select write for 100% write activities, or read <count>% to have the pattern do a mix of reads and writes. To more closely model a real application, I/Os to the same address can be repeated. Setting reuse <count>% will cause that percentage of I/Os to be done to the same address as another recent I/O. Setting history <count> will determine how long of a history of I/Os is kept to reuse. Further modeling is available by creating 'locality of reference' zones. By setting zone <size> as a fixed size, or zone <count>% as a percentage of the pattern test area, iorate will create count <count> localities to do I/Os against. To model 'moving hot spots,' specify limit <count> to deter- mine how many I/Os will be done to a given locality before a new one is chosen. Defining any of zone, count, or limit will cause default values to be set for the others if none are specified. TEST FILE Iorate will assume a test_file name of tests.ior if none is specified. WARNING: the test-fx.ior tests are destructive to all files/devices used for testing. Files/devices that have data contained in them should be in- cluded in the device file only if extreme care has been taken to ensure that all test patterns are read-only. Because of this danger, the default test file uses read-only patterns. Even so, great care should be taken whenever any entry in the devices file has data that is valued. Failure to do so has resulted not only in lost data, but in one case caused such dam- age to the root file system (random writes of garbage data) that the server had to be reinstalled from tape. There is no attempt made to check for other users of any device listed, the tests are simply run. Each test entry describes a single test, such as: Test 1 = "Mixed 2K Seq" for 120 sec ignore 20 sec 50% pat 1, 50% pat 2; Test numbers must be unique, but are also optional (if not provided, device numbers will be assigned in order automatically - a record of which is pro- vided in the log file). The duration of the test is set with for <time>. To make sure that the performance during any start-up period is not mea- sured, but rather the steady-state numbers, setting ignore <time> causes the performance statistics to be reset that far into the test time. In the event that one test may leave work in for the test storage, such as writes sitting in the cache of a disk array, pause <time> will introduce that length of delay before the test is activated. The area to run the test against can be specified with from <size> or from <count>% to set the starting location, and size <size> or size <count>% to set the size to test against (starting at the from location). Sizing that are based on percentages are based on the capacity of each device being tested - so if there are devices with different capacities, the tests will resize themselves for each one. To limit the level of I/O generated by this test, set <count> iops. This sets an I/Os per second target for the entire test - and since there are separate processes run for each active device defined, they will each tar- get their share independently. This means that if there are 10 active de- vices, and 100 iops is specified, then the program will attempt to drive 10 iops to each device. In the event that one device cannot do its share, the other devices will not be tasked will additional work, as the test process working on each devie are not aware of each other. Finally, a list of patterns to run is included. Each pattern is specified as <count>% pat <pattern>, where each percentage is a whole number (deci- mals ignored), and the total percentages of all patterns must equal exactly 100%. IOPS FILE Iorate assumes that no iops_file is available. Unless an iops_file is given on the command line or through the environment variable, the only iops limits will be set in the test file itself. If there is no iops limit specified for a test, then I/O will be generated as fast as possible on every device. When a file is given, each entry sets a new iops limit for a given test, such as: Target test 5 at 2000 iops; This is most useful for times when a given set of tests is to be run on differing configurations, and it is not known ahead of time what the upper limit of each test will be (maximum test rate). From there, it may be in- teresting to run the same tests at some percentage of the maximum rate. If a full speed pass of a test has already been run, then the gen_iops AWK program provided will generate such an iops_file that matches the perfor- mance seen it that run. Together with a new target_rate, a pattern can be tested for response times at various percentages of the maximum attained throughput for each test. ENVIRONMENT The command line file names can be provided for in the environment, with: IOR_DEV_FILE The name of the device_file to use. IOR_PAT_FILE The name of the pattern_file to use. IOR_TEST_FILE The name of the test_file to use. IOR_IOPS_FILE The name of the iops_file to use. IOR_OUTPUT_BASE The base name of the output files. AUTHOR Written by Vince Westin (vince.westin@emc.com), with a lot of assistance from the EMC Engineering Team. Copyright by EMC Corporation, 1997-2011. All rights reserved. The latest version is available from the web page at http://iorate.org. Use of the latest version is strongly encouraged. Check the README and README.versions files from the latest version for new features and bugs from earlier revisions that have been fixed. This code is the property of EMC Corporation. However, it may be used, re- produced, and passed on to others as long as the contents, including all copyright notices, remain intact. Modifications to, or modified versions of these files, may also be distributed, provided that they are clearly la- beled as having been modified from the original. In the event that modi- fied files are created, the original files are to be included with every distribution of those modified files. Inclusion of this code into a com- mercial product by any company other than EMC is prohibited without prior written consent. Having said the legal stuff, this code is designed to provide a good, generic tool for testing I/O subsystems under various kinds of loads. If you have suggestions for improvements in this tool, please send them along to the above address. SEE ALSO read(2), write(2), seek(2), open(2), close(2), fcntl(2) October 2011 iorate(1)
NAME | SYNOPSIS | DESCRIPTION | USAGE | PARAMETER FILES | DEVICE FILE | PATTERN FILE | TEST FILE | IOPS FILE | ENVIRONMENT | AUTHOR | SEE ALSO
Want to link to this manual page? Use this URL:
<https://man.freebsd.org/cgi/man.cgi?query=iorate&sektion=1&manpath=FreeBSD+Ports+15.1.quarterly>