-
Notifications
You must be signed in to change notification settings - Fork 5
/
Copy pathcustom_extractors.html
101 lines (101 loc) · 8.47 KB
/
custom_extractors.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<title>WiredTiger: Custom Extractors</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
$(window).load(resizeHeight);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="wiredtiger.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectlogo"><a href="http://wiredtiger.com/"><img alt="Logo" src="LogoFinal-header.png" alt="WiredTiger" /></a></td>
<td style="padding-left: 0.5em;">
<div id="projectname">
 <span id="projectnumber">Version 2.8.0</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<div class="banner">
<a href="https://github.com/wiredtiger/wiredtiger">Fork me on GitHub</a>
<a class="last" href="http://groups.google.com/group/wiredtiger-users">Join my user group</a>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.9.1 -->
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main Page</span></a></li>
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
<li><a href="modules.html"><span>Modules</span></a></li>
<li><a href="examples.html"><span>Examples</span></a></li>
<li><a href="community.html"><span>Community</span></a></li>
<li><a href="license.html"><span>License</span></a></li>
</ul>
</div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('custom_extractors.html','');});
</script>
<div id="doc-content">
<div class="header">
<div class="headertitle">
<div class="title">Custom Extractors </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h1><a class="anchor" id="custom_extractors_intro"></a>
Introduction to Custom Extractors</h1>
<p>A WiredTiger table can have zero or more associated indices. An index uses a different key to locate records than the table, and usually only stores a short key for each record, with the (larger) value in the table.</p>
<p>WiredTiger tables must be created with column names in order to create an index. This is required so that index cursors can support projections, and because WiredTiger optimizes some cases of "simple" tables without column names.</p>
<p>When the full schema of your records can be described in WiredTiger's packing format, you can create an index by specifying which columns from the record should appear in the index. However, for more complex records, or to associate multiple index keys to each record, applications can instead specify a custom extractor by implementing the <a class="el" href="struct_w_t___e_x_t_r_a_c_t_o_r.html" title="The interface implemented by applications to provide custom extraction of index keys or column group ...">WT_EXTRACTOR</a> interface.</p>
<p>The main method in the interface is <a class="el" href="struct_w_t___e_x_t_r_a_c_t_o_r.html#a4db101f00fce80af9ebd4f29f56f7b6a" title="Callback to extract a value for an index or column group. ">WT_EXTRACTOR::extract</a>. This is called by WiredTiger each time a record is updated in a table. The <code>extract</code> method should determine the index key(s) and call <a class="el" href="struct_w_t___c_u_r_s_o_r.html#ad1088d719df40babc1f57d086691ebdc" title="Set the key for the next operation. ">WT_CURSOR::set_key</a> followed by <a class="el" href="struct_w_t___c_u_r_s_o_r.html#aac90d9fbcc031570f924db55f8a1cee3" title="Insert a record and optionally update an existing record. ">WT_CURSOR::insert</a> on the supplied <code>result_cursor</code> for each index key.</p>
<p>If any operation fails, <a class="el" href="struct_w_t___e_x_t_r_a_c_t_o_r.html#a4db101f00fce80af9ebd4f29f56f7b6a" title="Callback to extract a value for an index or column group. ">WT_EXTRACTOR::extract</a> must return the failure to WiredTiger, or the index could become out of sync with the table.</p>
<p>Note that the extract callback is called for all operations that update the table, not just inserts. The callback sets the key and uses the <a class="el" href="struct_w_t___c_u_r_s_o_r.html#aac90d9fbcc031570f924db55f8a1cee3" title="Insert a record and optionally update an existing record. ">WT_CURSOR::insert</a> method to return the index key(s). WiredTiger will perform the required operation to keep the index in sync with the table.</p>
<p>Applications must register their <a class="el" href="struct_w_t___e_x_t_r_a_c_t_o_r.html" title="The interface implemented by applications to provide custom extraction of index keys or column group ...">WT_EXTRACTOR</a> implementations using <a class="el" href="struct_w_t___c_o_n_n_e_c_t_i_o_n.html#a2d65a70a305838e2a2a728fe5cb54903" title="Add a custom extractor for index keys or column groups. ">WT_CONNECTION::add_extractor</a>. This is often done by creating a <a class="el" href="extensions.html">WiredTiger extension</a>. They are then configured by passing <code>"extractor=..."</code> to <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#a358ca4141d59c345f401c58501276bbb" title="Create a table, column group, index or file. ">WT_SESSION::create</a> when creating an index.</p>
<p>See <a class="el" href="ex_extractor_8c-example.html">ex_extractor.c</a> for an example of how to implement custom extractors.</p>
<h1><a class="anchor" id="custom_extractors_notes"></a>
Implementation notes</h1>
<p>A WiredTiger index is a row store where the key columns contain all of the secondary and primary key columns, but only the secondary key columns are visible to applications. The value is empty, and WiredTiger's on-disk format optimizes for this case (empty values take up no space on disk).</p>
<p>Custom extractors only need to calculate the public index key columns. The <code>result_cursor</code> will be configured with a <code>key_format</code> corresponding to what was supplied to <a class="el" href="struct_w_t___s_e_s_s_i_o_n.html#a358ca4141d59c345f401c58501276bbb" title="Create a table, column group, index or file. ">WT_SESSION::create</a> when the index was created. WiredTiger will append the (hidden) primary key when populating the index.</p>
<p>If column names are specified for an index with a custom extractor, it is not permitted to use any column names from the table key. Custom index keys can include columns from the table value, but the extracted value must be equal to the value from that column of the record or the results of using a projection cursor on the index will be undefined.</p>
<h1><a class="anchor" id="custom_extractors_raw"></a>
Custom Collators in raw mode</h1>
<p>If a custom extractor needs to operate in raw mode on the <code>result_cursor</code>, it must take into account an implementation detail. To avoid rewriting the extracted key, WiredTiger appends a padding byte to the raw key using a <code>'x'</code> format. See <a class="el" href="schema.html#schema_format_types">Format types</a> for more information. If the callback operates in raw mode, it must also append this padding byte. </p>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="index.html">Reference Guide</a></li><li class="navelem"><a class="el" href="programming.html">Writing WiredTiger applications</a></li>
<li class="footer">Copyright (c) 2008-2016 MongoDB, Inc. All rights reserved. Contact <a href="mailto:[email protected]">[email protected]</a> for more information.</li>
</ul>
</div>
</body>
</html>