|
| 1 | +# mongodb-hack-middleware |
| 2 | + |
| 3 | +mongodb-hack-middleware provides a comprehensive solution for enhancing MongoDB collections with full-text indexing capabilities and offers a convenient way to retrieve random documents. Whether you're building a search-intensive application or need a way to fetch diverse data for testing locally or on-premise, this streamlines these processes for MongoDB users using the [community version](https://www.mongodb.com/try/download/community) |
| 4 | + |
| 5 | +## Key Advantages |
| 6 | + |
| 7 | +- MongoDB [community version](https://www.mongodb.com/try/download/community) provides a text search feature but it operates on a word-by-word basis using the "or operator" and doesn't support searching word by prefixes. It also can't perform text search on a particular field. This limitation inspired the creation of this repository, offering an enhanced and versatile solution for text-based queries. |
| 8 | + |
| 9 | +- Elevate your random document retrieval experience with our enhanced solution, addressing limitations found in MongoDB's $sample stage. Enjoy precise counts, elimination of duplicates documents, and providing a truly random and distinct set of documents. |
| 10 | + |
| 11 | +## Installation |
| 12 | + |
| 13 | +```sh |
| 14 | +npm install mongodb-hack-middleware |
| 15 | +``` |
| 16 | + |
| 17 | +or using yarn |
| 18 | + |
| 19 | +```sh |
| 20 | +yarn add mongodb-hack-middleware |
| 21 | +``` |
| 22 | + |
| 23 | +## Usage |
| 24 | + |
| 25 | +### initialize MongoClientHack or proxy MongoClient |
| 26 | + |
| 27 | +```js |
| 28 | +import { proxyClient, MongoClientHack } from "mongodb-hack-middleware"; |
| 29 | + |
| 30 | +// using MongoClientHack instance |
| 31 | +const mongoServer = new MongoClientHack({ |
| 32 | + map: { |
| 33 | + 'examjoint::hack_test': { // your databaseName::collectionName |
| 34 | + random: true, |
| 35 | + fulltext: ['name', 'des'] |
| 36 | + } |
| 37 | + }, |
| 38 | + url: 'mongodb://127.0.0.1:27017', |
| 39 | + options: { |
| 40 | + useUnifiedTopology: true, |
| 41 | + ...otherProps |
| 42 | + } |
| 43 | +}); |
| 44 | + |
| 45 | +// using MongoClient |
| 46 | +const mongoServer = new MongoClient('mongodb://127.0.0.1:27017'); |
| 47 | + |
| 48 | +proxyClient({ |
| 49 | + 'examjoint::hack_test': { // your databaseName::collectionName |
| 50 | + random: true, |
| 51 | + fulltext: ['name', 'des'] |
| 52 | + } |
| 53 | +})(mongoServer); |
| 54 | + |
| 55 | +// connect |
| 56 | +mongoServer.connect(); |
| 57 | + |
| 58 | +const db = mongoServer.db('examjoint'); |
| 59 | + |
| 60 | +// insert document |
| 61 | +await db.collection('hack_test').insertOne({ |
| 62 | + _id: 'doc1', |
| 63 | + name: 'ademola onabanjo', |
| 64 | + date: Date.now(), |
| 65 | + des: 'Lorem ipsum dolor sit amet consectetur' |
| 66 | +}); |
| 67 | + |
| 68 | +``` |
| 69 | + |
| 70 | +## searching text |
| 71 | + |
| 72 | +```js |
| 73 | +const nameOrDesFieldSearch = await db.collection().find({ $text: { $search: 'sit amet consec' } }).toArray(); |
| 74 | + |
| 75 | +// outputs: |
| 76 | +// [{ |
| 77 | +// _id: 'doc1', |
| 78 | +// name: 'ademola onabanjo', |
| 79 | +// date: Date.now(), |
| 80 | +// des: 'Lorem ipsum dolor sit amet consectetur' |
| 81 | +// }] |
| 82 | +console.log('searchResult: ', nameOrDesFieldSearch); |
| 83 | + |
| 84 | +// search single field |
| 85 | + |
| 86 | +const nameFieldSearch = await db.collection().find({ $text: { $search: 'onaba', $field: 'name' } }).toArray(); |
| 87 | + |
| 88 | +// outputs: |
| 89 | +// [{ |
| 90 | +// _id: 'doc1', |
| 91 | +// name: 'ademola onabanjo', |
| 92 | +// date: Date.now(), |
| 93 | +// des: 'Lorem ipsum dolor sit amet consectetur' |
| 94 | +// }] |
| 95 | +console.log('searchResult: ', nameFieldSearch); |
| 96 | +``` |
| 97 | + |
| 98 | +## random query |
| 99 | + |
| 100 | +```js |
| 101 | +// make sure you followed this process when querying random document ($sample at the first pipeline and $match at the second one) |
| 102 | +// or else it fallback to using the native caller |
| 103 | +const searchRes = await db.collection('hack_test').aggregate([ |
| 104 | + { $sample: { $size: 5 } } |
| 105 | + { $match: { } } |
| 106 | +]).limit(1).toArray(); |
| 107 | +``` |
| 108 | + |
| 109 | +## Document transformation |
| 110 | + |
| 111 | +### fulltext index |
| 112 | + |
| 113 | +Some write operations are proxied to add an extra field value to the document for querying random document and searching text. |
| 114 | +Some read operations are also proxied to remove redundant field value. |
| 115 | +The following methods are proxied on the collection instance to transform the document |
| 116 | + |
| 117 | +- `insertOne` |
| 118 | +- `insertMany` |
| 119 | +- `updateOne` ($set, $unset) |
| 120 | +- `replaceOne` |
| 121 | +- `bulkWrite` |
| 122 | +- `find` |
| 123 | +- `findOne` |
| 124 | +- `watch` |
| 125 | +- `aggregate` |
| 126 | + |
| 127 | +### $text transformation |
| 128 | + |
| 129 | +`{ $text: { $search: 'onaba', $field: 'des' } }` is transformed to `{ $or: [...your-$Or-Props, { __fta_des: { $in: ['onaba'] } }] }` |
| 130 | +while `{ $text: { $search: 'onaba', $field: ['name', 'des'] } }` is transformed to `{ $or: [...your-$Or-Props, { __fta_name: { $in: ['onaba'] } }, { __fta_des: { $in: ['onaba'] } }] }` |
| 131 | + |
| 132 | +## Limitations |
| 133 | + |
| 134 | +- As the maximum size of a document in mongodb is 16mb, Avoid saving more than 37,000 bytes of characters for fulltext field. this 37,000 chars can produce approximately 301,391 strings in array with 11mb in size. |
| 135 | + |
| 136 | +## Cons |
0 commit comments