<h4>4.1 <span class="Heading">The idea of hashing</span></h4>
<p>If one wants to store a certain set of similar objects and wants to quickly access a given one (or come back with the result that it is unknown), the first idea would be to store them in a list, possibly sorted for faster access. This however still would need <span class="SimpleMath">\(\log(n)\)</span> comparisons to find a given element or to decide that it is not yet stored.</p>
<p>Therefore one uses a much bigger array and uses a function on the space of possible objects with integer values to decide, where in the array to store a certain object. If this so called hash function distributes the actually stored objects well enough over the array, the access time is constant in average. Of course, a hash function will usually not be injective, so one needs a strategy what to do in case of a so-called <q>collision</q>, that is, if more than one object with the same hash value has to be stored. This package provides two ways to deal with collisions, one is implemented in the so called <q>HashTabs</q> and another in the <q>TreeHashTabs</q>. The former simply uses other parts of the array to store the data involved in the collisions and the latter uses an AVL tree (see Chapter <a href="chap8_mj.html#X7F2630FB7AA3C73D"><span class="RefLink">8</span></a>) to store all data objects with the same hash value. Both are used basically in the same way but sometimes behave a bit differently.</p>
<p>The basic functions to work with hash tables are <code class="func">HTCreate</code> (<a href="chap4_mj.html#X79F46D9982BB0E12"><span class="RefLink">4.3-1</span></a>), <code class="func">HTAdd</code> (<a href="chap4_mj.html#X833166757E58B83B"><span class="RefLink">4.3-2</span></a>), <code class="func">HTValue</code> (<a href="chap4_mj.html#X8585E1687AE9E409"><span class="RefLink">4.3-3</span></a>), <code class="func">HTDelete</code> (<a href="chap4_mj.html#X82A8245E875B0B8E"><span class="RefLink">4.3-5</span></a>) and <code class="func">HTUpdate</code> (<a href="chap4_mj.html#X788B50C17A472DFD"><span class="RefLink">4.3-4</span></a>). They are described in Section <a href="chap4_mj.html#X8424E70E78FAA203"><span class="RefLink">4.3</span></a>.</p>
<p>In the <strong class="pkg">orb</strong> package hash functions are chosen automatically by giving a sample object together with the length of the hash table. This is done with the following operation:</p>
<p>The first argument <var class="Arg">ob</var> must be a sample object, that is, an object like those we want to store in the hash table later on. The argument <var class="Arg">len</var> is an integer that gives the length of the hash table. Note that this might be called later on automatically, when a hash table is increased in size. The operation returns a record with two components. The component <code class="code">func</code> is a <strong class="pkg">GAP</strong> function taking two arguments, see below. The component <code class="code">data</code> is some <strong class="pkg">GAP</strong> object. Later on, the hash function will be called with two arguments, the first is the object for which it should call the hash value and the second argument must be the data stored in the <code class="code">data</code> component.</p>
<p>The hash function has to return values between <span class="SimpleMath">\(1\)</span> and the hash length <var class="Arg">len</var> inclusively.</p>
<p>This setup is chosen such that the hash functions can be global objects that are not created during the execution of <code class="func">ChooseHashFunction</code> but still can change their behaviour depending on the data.</p>
<p>In the following we just document, for which types of objects there are hash functions that can be found using <code class="func">ChooseHashFunction</code>.</p>
<p>This method is for compressed vectors over the field <code class="code">GF(2)</code> of two elements. Note that there is no hash function for non-compressed vectors over <code class="code">GF(2)</code> because those objects cannot efficiently be recognised from their type.</p>
<p>Note that you can only use the resulting hash functions for vectors of the same length.</p>
<p>This method is for compressed vectors over a finite field with up to <span class="SimpleMath">\(256\)</span> elements. Note that there is no hash function for non-compressed such vectors because those objects cannot efficiently be recognised from their type.</p>
<p>Note that you can only use the resulting hash functions for vectors of the same length.</p>
<p>This method is for compressed matrices over the field <code class="code">GF(2)</code> of two elements. Note that there is no hash function for non-compressed matrices over <code class="code">GF(2)</code> because those objects cannot efficiently be recognised from their type.</p>
<p>Note that you can only use the resulting hash functions for matrices of the same size.</p>
<p>This method is for compressed matrices over a finite field with up to <span class="SimpleMath">\(256\)</span> elements. Note that there is no hash function for non-compressed such vectors because those objects cannot efficiently be recognised from their type.</p>
<p>Note that you can only use the resulting hash functions for matrices of the same size.</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">‣ HTCreate</code>( <var class="Arg">sample</var>[, <var class="Arg">opt</var>] )</td><td class="tdright">( operation )</td></tr></table></div>
<p>Returns: a new hash tableobject</p>
<p>A new hash table for objects like <var class="Arg">sample</var> is created. The second argument <var class="Arg">opt</var> is an optional options record, which will supplied in most cases, if only to specify the length and type of the hash table to be used. The following components in this record can be bound:</p>
<dl>
<dt><strong class="Mark"><code class="code">treehashsize</code></strong></dt>
<dd><p>If this component is bound the type of the hash table is a TreeHashTab. The value must be a positive integer and will be the size of the hash table. Note that for this type of hash table the keys to be stored in the hash must be comparable using <span class="SimpleMath">\(<\)</span>. A three-way comparison function can be supplied using the component <code class="code">cmpfunc</code> (see below).</p>
</dd>
<dt><strong class="Mark"><code class="code">treehashtab</code></strong></dt>
<dd><p>If this component is bound the type of the hash table is a TreeHashTab. This option is superfluous if <code class="code">treehashsize</code> is used.</p>
</dd>
<dt><strong class="Mark"><code class="code">forflatplainlists</code></strong></dt>
<dd><p>If this component is set to <code class="keyw">true</code> then the user guarantees that all the elements in the hash will be flat plain lists, that is, plain lists with no subobjects. For example lists of immediate integers will fulfill this requirement, but ranges don't. In this case, a particularly good and efficient hash function will automatically be taken and the components hashfunc, hfbig and hfdbig are ignored. Note that this cannot be automatically detected because it depends not only on the sample point but also potentially on all the other points to be stored in the hash table.
</dd>
<dt><strong class="Mark"><code class="code">hf</code> and <code class="code">hfd</code></strong></dt>
<dd><p>If these components are bound, they are used as the hash function. The value of <code class="code">hf</code> must be a function taking two arguments, the first being the object for which the hash function shall be computed and the second being the value of <code class="code">hfd</code>. The returned value must be an integer in the range from <span class="SimpleMath">\(1\)</span> to the length of the hash. If either of these components is not bound, an automatic choice for the hash function is done using <code class="func">ChooseHashFunction</code> (<a href="chap4_mj.html#X7ACED4FB7C971A5A"><span class="RefLink">4.2-1</span></a>) and the supplied sample object <var class="Arg">sample</var>.</p>
<p>Note that if you specify these two components and are using a HashTab table then this table cannot grow unless you also bind the components <code class="code">hfbig</code>, <code class="code">hfdbig</code> and <code class="code">cangrow</code>.</p>
</dd>
<dt><strong class="Mark"><code class="code">cmpfunc</code></strong></dt>
<dd><p>This component can be bound to a three-way comparison function taking two arguments <var class="Arg">a</var> and <var class="Arg">b</var> (which will be keys for the TreeHashTab) and returns <span class="SimpleMath">\(-1\)</span> if <span class="SimpleMath">\(\textit{a}<\textit{b}\)</span>, <span class="SimpleMath">\(0\)</span> if <span class="SimpleMath">\(\textit{a} = \textit{b}\)</span> and <span class="SimpleMath">\(1\)</span> if <span class="SimpleMath">\(\textit{a} > \textit{b}\)</span>. If this component is not bound the function <code class="func">AVLCmp</code> (<a href="chap8_mj.html#X83EC43DA871C7F60"><span class="RefLink">8.2-2</span></a>) is taken, which simply calls the generic operations <code class="code"><</code> and <code class="code">=</code> to do the job.</p>
</dd>
<dt><strong class="Mark"><code class="code">hashlen</code></strong></dt>
<dd><p>If this component is bound the type of the hash table is a standard HashTab table. That is, collisions are dealt with by storing additional entries in other slots. This is the traditional way to implement a hash table. Note that currently deleting entries in such a hash table is not implemented, since it could only be done by leaving a <q>deleted</q> mark which could pollute that hash table. Use TreeHashTabs instead if you need deletion. The value bound to <code class="code">hashlen</code> must be a positive integer and will be the initial length of the hash table.</p>
<p>Note that it is a good idea to choose a prime number as the hash length due to the algorithm for collision handling which works particularly well in that case. The hash function is chosen automatically.</p>
</dd>
<dt><strong class="Mark"><code class="code">hashtab</code></strong></dt>
<dd><p>If this component is bound the type of the hash table is a standard HashTab table. This component is superfluous if <code class="code">hashlen</code> is bound.</p>
</dd>
<dt><strong class="Mark"><code class="code">eqf</code></strong></dt>
<dd><p>For HashTab tables the function taking two arguments bound to this component is used to compare keys in the hash table. If this component is not bound the usual <code class="code">=</code> operation is taken.</p>
</dd>
<dt><strong class="Mark"><code class="code">hfbig</code> and <code class="code">hfdbig</code> and <code class="code">cangrow</code></strong></dt>
<dd><p>If you have used the components <code class="code">hf</code> and <code class="code">hfd</code> then your hash table cannot automatically grow when it fills up. This is because the length of the table is built into the hash function. If you still want your hash table to be able to grow automatically, then bind a hash function returning arbitrary integers to <code class="code">hfbig</code>, the corresponding data for the second argument to <code class="code">hfdbig</code> and bind <code class="code">cangrow</code> to <code class="keyw">true</code>. Then the hash table will automatically grow and take this new hash function modulo the new length of the hash table as hash function.</p>
<p>Stores the object <var class="Arg">key</var> into the hash table <var class="Arg">ht</var> and stores the value <var class="Arg">val</var> together with <var class="Arg">ob</var>. The result is <code class="keyw">fail</code> if an error occurred, which can be that an object equal to <var class="Arg">key</var> is already stored in the hash table or that the hash table is already full. The latter can only happen, if the hash table is no TreeHashTab and cannot grow automatically.</p>
<p>If no error occurs, the result is an integer indicating the place in the hash table where the object is stored. Note that once the hash table grows automatically this number is no longer the same!</p>
<p>If the value <var class="Arg">val</var> is <code class="keyw">true</code> for all objects in the hash, no extra memory is used for the values. All other values are stored in the hash. The value <code class="keyw">fail</code> cannot be stored as it indicates that the object is not found in the hash.</p>
<p>See Section <a href="chap4_mj.html#X8069137484662072"><span class="RefLink">4.4</span></a> for details on the data structures and especially about memory requirements.</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">‣ HTValue</code>( <var class="Arg">ht</var>, <var class="Arg">key</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<p>Returns: <code class="keyw">fail</code> or <code class="keyw">true</code> or a value</p>
<p>Looks up the object <var class="Arg">key</var> in the hash table <var class="Arg">ht</var>. If the object is not found, <code class="keyw">fail</code> is returned. Otherwise, the value stored with the object is returned. Note that if this value was <code class="keyw">true</code> no extra memory is used for this.</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">‣ HTUpdate</code>( <var class="Arg">ht</var>, <var class="Arg">key</var>, <var class="Arg">value</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<p>Returns: <code class="keyw">fail</code> or <code class="keyw">true</code> or a value</p>
<p>The object <var class="Arg">key</var> must already be stored in the hash table <var class="Arg">ht</var>, otherwise this operation returns <code class="keyw">fail</code>. The value stored with <varclass="Arg">key</var> in the hash is replaced by <var class="Arg">value</var> and the previously stored value is returned.</p>
<div class="func"><table class="func" width="100%"><tr><td class="tdleft"><code class="func">‣ HTDelete</code>( <var class="Arg">ht</var>, <var class="Arg">key</var> )</td><td class="tdright">( operation )</td></tr></table></div>
<p>Returns: <code class="keyw">fail</code> or <code class="keyw">true</code> or a value</p>
<p>The object <var class="Arg">key</var> along with its stored value is removed from the hash table <var class="Arg">ht</var>. Note that this currently only works for TreeHashTabs and not for HashTab tables. It is an error if <var class="Arg">key</var> is not found in the hash table and <code class="keyw">fail</code> is returned in this case.</p>
<p>This is a more or less internal operation. It is called when the space in a hash table becomes scarce. The first argument <var class="Arg">ht</var> must be a hash tableobject, the second a sample point. The function increases the hash size by a factor of 2. This makes it necessary to choose a new hash function. Usually this is done with the usual <code class="code">ChooseHashFunction</code> method. However, one can bind the two components <code class="code">hfbig</code> and <code class="code">hfdbig</code> in the options record of <code class="func">HTCreate</code> (<a href="chap4_mj.html#X79F46D9982BB0E12"><span class="RefLink">4.3-1</span></a>) to a function and a record respectively and bind <code class="code">cangrow</code> to <code class="keyw">true</code>. In that case, upon growing the hash, a new hash function is created by taking the function <code class="code">hfbig</code> together with <code class="code">hfdbig</code> as second data argument and reducing the resulting integer modulo the hash length. In this way one can specify a hash function suitable for all hash sizes by simply producing big enough hash values.</p>
<h4>4.4 <span class="Heading"> The data structures for hash tables </span></h4>
<p>A legacy hash tableobject is just a record with the following components:</p>
<dl>
<dt><strong class="Mark"><code class="code">els</code></strong></dt>
<dd><p>A <strong class="pkg">GAP</strong> list storing the elements. Its length can be as long as the component <code class="code">len</code> indicates but will only grow as necessary when elements are stored in the hash.</p>
</dd>
<dt><strong class="Mark"><code class="code">vals</code></strong></dt>
<dd><p>A <strong class="pkg">GAP</strong> list storing the corresponding values. If a value is <code class="keyw">true</code> nothing is stored here to save memory.</p>
</dd>
<dt><strong class="Mark"><code class="code">len</code></strong></dt>
<dd><p>Length of the hash table.</p>
</dd>
<dt><strong class="Mark"><code class="code">nr</code></strong></dt>
<dd><p>Number of elements stored in the hash table.</p>
</dd>
<dt><strong class="Mark"><code class="code">hf</code></strong></dt>
<dd><p>The hash function (value of the <code class="code">func</code> component in the record returned by <code class="func">ChooseHashFunction</code> (<a href="chap4_mj.html#X7ACED4FB7C971A5A"><span class="RefLink">4.2-1</span></a>)).</p>
</dd>
<dt><strong class="Mark"><code class="code">hfd</code></strong></dt>
<dd><p>The data for the second argument of the hash function (value of the <code class="code">data</code> component in the record returned by <code class="func">ChooseHashFunction</code> (<a href="chap4_mj.html#X7ACED4FB7C971A5A"><span class="RefLink">4.2-1</span></a>)).</p>
</dd>
<dt><strong class="Mark"><code class="code">eqf</code></strong></dt>
<dd><p>A comparison function taking two arguments and returning <code class="keyw">true</code> for equality or <code class="keyw">false</code> otherwise.</p>
</dd>
<dt><strong class="Mark"><code class="code">collisions</code></strong></dt>
<dd><p>Number of collisions (see below).</p>
</dd>
<dt><strong class="Mark"><code class="code">accesses</code></strong></dt>
<dd><p>Number of lookup or store accesses to the hash.</p>
</dd>
<dt><strong class="Mark"><code class="code">cangrow</code></strong></dt>
<dd><p>A boolean value indicating whether the hash can grow automatically or not.</p>
</dd>
<dt><strong class="Mark"><code class="code">ishash</code></strong></dt>
<dd><p>Is <code class="keyw">true</code> to indicate that this is a hash table record.</p>
</dd>
<dt><strong class="Mark"><code class="code">hfbig</code> and <code class="code">hfdbig</code></strong></dt>
<dd><p>Used for hash tables which need to be able to grow but where the user supplied the hash function. See Section <code class="func">HTCreate</code> (<a href="chap4_mj.html#X79F46D9982BB0E12"><span class="RefLink">4.3-1</span></a>) for more details.</p>
</dd>
</dl>
<p>A new style HashTab objects are component objects with the same components except that there is no component <code class="code">ishash</code> since these objects are recognised by their type.</p>
<p>A TreeHashTab is very similar. It is a positional object with basically the same components, except that <code class="code">eqf</code> is replaced by the three-way comparison function <code class="code">cmpfunc</code>. Since TreeHashTabs do not grow, the components <code class="code">hfbig</code>, <code class="code">hfdbig</code> and <code class="code">cangrow</code> are never bound. Each slot in the <code class="code">els</code> component is either unbound (empty), or bound to the only key stored in the hash which has this hash value or, if there is more than one key for that hash value, the slot is bound to an AVL tree containing all such keys (and values).</p>
<p>Due to the data structure defined above the hash table will need one machine word (<span class="SimpleMath">\(4\)</span> bytes on 32bit machines and <span class="SimpleMath">\(8\)</span> bytes on 64bit machines) per possible entry in the hash if all values corresponding to objects in the hash are <code class="keyw">true</code> and two machine words otherwise. This means that the memory requirement for the hash itself is proportional to the hash table length and not to the number of objects actually stored in the hash!</p>
<p>In addition one of course needs the memory to store the objects themselves.</p>
<p>For TreeHashTabs there are additional memory requirements. As soon as there are more than one key hashing to the same value, the memory for an AVL tree object is needed in addition. An AVL tree objects needs about 10 machine words for the tree object and then another 4 machine words for each entry stored in the tree. Note that for many collisions this can be significantly more than for HashTab tables. However, the advantage of TreeHashTabs is that even for a bad hash function the performance is never worse than <span class="SimpleMath">\(log(n)\)</span> for each operation where <span class="SimpleMath">\(n\)</span> is the number of keys in the hash with the same hash value.</p>
<h5>4.4-2 <span class="Heading"> Handling of collisions </span></h5>
<p>This section is only relevant for HashTab objects.</p>
<p>If two or more objects have the same hash value, the following is done: If the hash value is coprime to the hash length, the hash value is taken as <q>the increment</q>, otherwise <span class="SimpleMath">\(1\)</span> is taken. The code to find the proper place for an object just repeatedly adds the increment to the current position modulo the hash length. Due to the choice of the increment this will eventually try all places in the hash table. Every such increment step is counted as a collision in the <code class="code">collisions</code> component in the hash table. This algorithm explains why it is sensible to choose a prime number as the length of a hash table.</p>
<p>Hashing is efficient as long as there are not too many collisions. It is not a problem if the number of collisions (counted in the <code class="code">collisions</code> component) is smaller than the number of accesses (counted in the <code class="code">accesses</code> component).</p>
<p>A high number of collisions can be caused by a bad hash function, because the hash table is too small (do not fill a hash table to more than about 80%), or because the objects to store are just not well enough distributed. Hash tables will grow automatically if too many collisions are detected or if they are filled to 80%.</p>
<p>The advantage of TreeHashTabs is that even for a bad hash function the performance is never worse than <span class="SimpleMath">\(log(n)\)</span> for each operation where <span class="SimpleMath">\(n\)</span> is the number of keys in the hash with the same hash value. However, they need a bit more memory.</p>
Die Informationen auf dieser Webseite wurden
nach bestem Wissen sorgfältig zusammengestellt. Es wird jedoch weder Vollständigkeit, noch Richtigkeit,
noch Qualität der bereit gestellten Informationen zugesichert.
Bemerkung:
Die farbliche Syntaxdarstellung ist noch experimentell.
