-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy pathstatistics.html
154 lines (154 loc) · 16.5 KB
/
statistics.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<title>WiredTiger: Statistics</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="wiredtiger.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectlogo"><a href="http://wiredtiger.com/"><img alt="Logo" src="LogoFinal-header.png" alt="WiredTiger" /></a></td>
<td style="padding-left: 0.5em;">
<div id="projectname">
 <span id="projectnumber">Version 2.8.0</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<div class="banner">
<a href="https://github.com/wiredtiger/wiredtiger">Fork me on GitHub</a>
<a class="last" href="http://groups.google.com/group/wiredtiger-users">Join my user group</a>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.9.1 -->
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main Page</span></a></li>
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
<li><a href="modules.html"><span>Modules</span></a></li>
<li><a href="examples.html"><span>Examples</span></a></li>
<li><a href="community.html"><span>Community</span></a></li>
<li><a href="license.html"><span>License</span></a></li>
</ul>
</div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('statistics.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Statistics </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>WiredTiger can be configured to maintain a variety of run-time and data-source statistics. As maintaining statistics may involve updating shared-memory data structures and traversing disk-based data structures, configuring statistics may decrease application performance.</p>
<p>The <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> <code>statistics</code> configuration must be set in order for statistics to be maintained. When configured to <code>none</code> (the default), no statistics are maintained and attempting to read the statistics will result in an error. Alternatively, the <code>fast</code> configuration maintains a subset of the statistics which are relatively inexpensive, and the <code>all</code> configuration maintains all statistics regardless of cost.</p>
<p>The following example configures WiredTiger to maintain all statistics, regardless of cost:</p>
<div class="fragment"><div class="line"> ret = <a class="code" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed">wiredtiger_open</a>(home, NULL, <span class="stringliteral">"create,statistics=(all)"</span>, &conn);</div>
</div><!-- fragment --><p> Statistics are gathered and returned to the application using a statistics cursor, which returns key/value pairs to the application. See <a class="el" href="data_sources.html#data_statistics">Statistics Data</a> for information about accessing the gathered statistics.</p>
<p>When the <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d" title="Open a new cursor on a data source or duplicate an existing cursor. ">WT_SESSION::open_cursor</a> method is called to open a statistics cursor, the statistics configuration specifies the statistics to be gathered, similarly to the <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> function. The <code>fast</code> configuration gathers the subset of the statistics that are relatively inexpensive, and the <code>all</code> configuration gathers all statistics, regardless of cost. If no configuration is specified, the current database statistics configuration is assumed.</p>
<p>The configuration of the <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d" title="Open a new cursor on a data source or duplicate an existing cursor. ">WT_SESSION::open_cursor</a> method must agree with the database configuration: if the database is configured to maintain <code>fast</code> statistics, attempts to open a statistics cursor in <code>all</code> mode will fail. If the database has been configured to maintain <code>all</code> statistics, the statistics cursor can be configured in either <code>fast</code> or <code>all</code> modes. For example, an application might configure the database to maintain all available statistics, but the application might gather expensive statistics less frequently than inexpensive ones.</p>
<p>The following example opens a statistics cursor on the database:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(</div>
<div class="line"> session, <span class="stringliteral">"statistics:"</span>, NULL, NULL, &cursor);</div>
</div><!-- fragment --><p> The following example opens a statistics cursor on a table:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(</div>
<div class="line"> session, <span class="stringliteral">"statistics:table:mytable"</span>, NULL, NULL, &cursor);</div>
</div><!-- fragment --><p> The following example opens a statistics cursor on a table, but gathering only the relatively inexpensive statistics:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session,</div>
<div class="line"> <span class="stringliteral">"statistics:table:mytable"</span>, NULL, <span class="stringliteral">"statistics=(fast)"</span>, &cursor);</div>
</div><!-- fragment --><p> The <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d" title="Open a new cursor on a data source or duplicate an existing cursor. ">WT_SESSION::open_cursor</a> method and the <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> function also support the statistics configuration value <code>clear</code>.</p>
<p>When <code>clear</code> is specified to the <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d" title="Open a new cursor on a data source or duplicate an existing cursor. ">WT_SESSION::open_cursor</a> method, gathered statistics will be reset, where appropriate, after they are gathered. For example, a cache size statistic is not cleared because it's not expected to change rapidly over time, while the count of cursor insert operations will be cleared. This allows applications to easily monitor changes in the system over time.</p>
<p>The following example gathers all statistics for a table, regardless of cost, and then clears them:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session,</div>
<div class="line"> <span class="stringliteral">"statistics:table:mytable"</span>,</div>
<div class="line"> NULL, <span class="stringliteral">"statistics=(all,clear)"</span>, &cursor);</div>
</div><!-- fragment --><p> When <code>clear</code> is specified to the <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> function, gathered statistics will be reset, where appropriate, after they are gathered. This applies to all statistics cursors (as if <code>clear</code> was configured when the cursor was opened), as well as statistics logging, when it is configured.</p>
<p>The following example configures WiredTiger to maintain only relatively inexpensive statistics, and to clears statistics after they are gathered or logged:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session,</div>
<div class="line"> <span class="stringliteral">"statistics:"</span>, NULL, <span class="stringliteral">"statistics=(fast,clear)"</span>, &cursor);</div>
</div><!-- fragment --><p> The following example opens a statistics cursor on an open join cursor:</p>
<div class="fragment"><div class="line"> ret = session-><a class="code" href="struct_w_t___s_e_s_s_i_o_n.html#afb5b4a69c2c5cafe411b2b04fdc1c75d">open_cursor</a>(session,</div>
<div class="line"> <span class="stringliteral">"statistics:join"</span>,</div>
<div class="line"> join_cursor, NULL, &stat_cursor);</div>
</div><!-- fragment --><p> The statistics gathered will be organized by reference cursors participating in the join (see <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#ae0ab118df83d173c6a20eb1ea3f3fd84" title="Join a join cursor with a reference cursor. ">WT_SESSION::join</a>); the uri of each reference cursor appears as a prefix in the description field returned as a value by the statistics cursor.</p>
<h1><a class="anchor" id="statistics_log"></a>
Statistics logging</h1>
<p>WiredTiger will optionally log database statistics into a file when the the <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> <code>statistics_log</code> configuration is set.</p>
<p>The resulting statistics can be displayed using the <code>wtstats</code> visualization tool. For more information, see <a class="el" href="wtstats.html">Visualizing performance with wtstats</a>.</p>
<p>The following example logs statistics every 30 seconds:</p>
<div class="fragment"><div class="line"> ret = <a class="code" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed">wiredtiger_open</a>(</div>
<div class="line"> home, NULL, <span class="stringliteral">"create,statistics_log=(wait=30)"</span>, &conn);</div>
</div><!-- fragment --><p> Each record is formatted as a space-separated timestamp, unsigned 64-bit value and a variable length string which describes the statistic.</p>
<p>The timestamp format may be changed with the <code>statistics_log.timestamp</code> configuration string. The <code>timestamp</code> value may contain ISO C90 standard strftime conversion specifications.</p>
<p>The statistics the database is configured to maintain are logged, that is, configuring the database to maintain <code>fast</code> or <code>all</code> statistics will modify the statistics that will be logged.</p>
<p>Statistics for specific underlying data sources may be included by adding a list of data source URIs to the <code>statistics_log</code> configuration string:</p>
<div class="fragment"><div class="line"> ret = <a class="code" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed">wiredtiger_open</a>(home, NULL,</div>
<div class="line"> <span class="stringliteral">"create, statistics_log=("</span></div>
<div class="line"> <span class="stringliteral">"sources=(\"lsm:table1\",\"lsm:table2\"), wait=5)"</span>,</div>
<div class="line"> &conn);</div>
</div><!-- fragment --><p> Statistics for all underlying data sources of a particular type may be included by adding a partial data source URI to the <code>statistics_log</code> configuration string:</p>
<div class="fragment"><div class="line"> ret = <a class="code" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed">wiredtiger_open</a>(home, NULL,</div>
<div class="line"> <span class="stringliteral">"create, statistics_log=(sources=(\"lsm:\"), wait=5)"</span>,</div>
<div class="line"> &conn);</div>
</div><!-- fragment --><p> When database statistics are logged, the database home will be the first space-separated entry for each record in the log file. For example:</p>
<div class="fragment"><div class="line">Mar 08 11:38:23 463 /database/home pthread mutex condition wait calls</div>
<div class="line">Mar 08 11:38:23 0 /database/home files currently open</div>
<div class="line">Mar 08 11:38:23 1855437 /database/home total heap memory allocations</div>
<div class="line">Mar 08 11:38:23 1856622 /database/home total heap memory frees</div>
<div class="line">Mar 08 11:38:23 1 /database/home total heap memory re-allocations</div>
<div class="line">Mar 08 11:38:23 472 /database/home total read I/Os</div>
</div><!-- fragment --><p>When data source statistics are logged, the data source's URI will be the first space-separated entry for each record in the log file. For example:</p>
<div class="fragment"><div class="line">Mar 20 10:42:36 21 table:mytable compressed pages written</div>
<div class="line">Mar 20 10:42:36 0 table:mytable page written failed to compress</div>
<div class="line">Mar 20 10:42:36 5 table:mytable page written was too small to compress</div>
<div class="line">Mar 20 10:42:36 586 table:mytable cursor insert calls</div>
<div class="line">Mar 20 10:42:36 0 table:mytable bulk-loaded cursor-insert calls</div>
</div><!-- fragment --><p>No statistics are logged for any data source for which a handle is not currently open in the database, nor will any statistics requiring the traversal of a tree (as if the <code>statistics_fast</code> configuration string were set).</p>
<p>The location of the log files may be changed with the <code>statistics_log.path</code> configuration string. The <code>path</code> value value may contain ISO C90 standard strftime conversion specifications. WiredTiger will not create non-existent directories in the path, they must exist before <a class="el" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed" title="Open a connection to a database. ">wiredtiger_open</a> is called.</p>
<p>The following example logs statistics into files named with the month, day and year:</p>
<div class="fragment"><div class="line"> ret = <a class="code" href="group__wt.html#ga9e6adae3fc6964ef837a62795c7840ed">wiredtiger_open</a>(home, NULL,</div>
<div class="line"> <span class="stringliteral">"create,"</span></div>
<div class="line"> <span class="stringliteral">"statistics_log=(wait=120,path=/log/log.%m.%d.%y)"</span>, &conn);</div>
</div><!-- fragment --><p> A Python script that parses the default logging output and uses the <a href="http://www.gnuplot.info/">gnuplot</a>, utility to generate Portable Network Graphics (PNG) format graphs is included in the WiredTiger distribution in the file <code>tools/statlog.py</code>.</p>
<p>To interactively examine statistics results, see <a class="el" href="wtstats.html">Visualizing performance with wtstats</a>. </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="index.html">Reference Guide</a></li><li class="navelem"><a class="el" href="programming.html">Writing WiredTiger applications</a></li>
<li class="footer">Copyright (c) 2008-2016 MongoDB, Inc. All rights reserved. Contact <a href="mailto:[email protected]">[email protected]</a> for more information.</li>
</ul>
</div>
</body>
</html>