Meilisearch: How to Install Open Source Search Engine on MicroK8s

Introduction

In today's data-driven world, the ability to sift through vast amounts of information and extract meaningful insights has become crucial for businesses and individuals alike. Efficient search engines are the fulcrum upon which this capability rests. While there are a number of popular search solutions available, catering to various needs and complexities, the recent emergence of MeiliSearch has garnered attention in tech circles. Positioned as an open-source, user-friendly alternative, MeiliSearch promises to be a formidable competitor in the world of search solutions.

What is MeiliSearch?

MeiliSearch is an open-source search engine designed to provide developers with a tool that is not only powerful but also easy to implement and maintain. Unlike some of its counterparts which can be cumbersome or overly complex, MeiliSearch offers a breath of fresh air with its straightforward setup and intuitive features.

At its core, MeiliSearch focuses on delivering relevant search results at lightning speed. Built for both small-scale projects and expansive databases, it offers instant, typo-tolerant search results, ensuring users find what they're looking for even when they make typographical errors.

Furthermore, its open-source nature stands as a testament to its adaptability and community-driven ethos. This means developers can delve deep into its codebase to understand its mechanics or tweak it to their unique requirements, providing a level of customization and transparency that proprietary solutions often lack.

In essence, MeiliSearch amalgamates the best of both worlds – the robustness of large, established search engines with the simplicity and flexibility desired by developers in today's fast-paced tech landscape.

Meilisearch Behind The Scene

The technological foundation of MeiliSearch is as intriguing as its history. Designed to be efficient, user-friendly, and versatile, MeiliSearch leverages cutting-edge technologies and algorithms to provide optimal search results.

1. Developed in Rust: One of the standout features of MeiliSearch's technological backbone is its development in Rust, a systems programming language renowned for its performance and safety. Rust ensures MeiliSearch's operations are not only fast but also memory-efficient and free from common bugs, which can plague other software. By choosing Rust, the MeiliSearch team ensured that the engine could handle vast amounts of data with minimal overhead and maximum speed.
2. Ranking Rules and Relevance: At its heart, a search engine's primary job is to deliver relevant results. MeiliSearch employs a unique set of customizable ranking rules that allow it to determine the order of search results based on their relevance. This ensures users always get the most pertinent information first, enhancing the overall search experience.
3. Typo-Tolerant Algorithm: Recognizing the inevitability of human errors, especially in typing, MeiliSearch uses a typo-tolerant algorithm. This ensures that even if a user makes a mistake while inputting their search term, the engine will still provide accurate and relevant results. It achieves this by calculating the 'distance' between the query and potential matches, allowing for small deviations.
4. Asynchronous Task Handling: Another significant aspect of MeiliSearch's technology is its ability to handle tasks asynchronously. This means that when new data is added or existing data is updated, MeiliSearch can index it in real-time without slowing down the search queries. This feature ensures a seamless experience for users, even when the underlying data is constantly changing.
5. Instant Search Capabilities: Complementing its asynchronous task handling, MeiliSearch's architecture allows for instant search results. As users type their queries, the engine provides real-time feedback, narrowing down the results with each keystroke. This creates a dynamic and responsive search experience.

By integrating these advanced technological features, MeiliSearch offers a compelling blend of speed, accuracy, and adaptability, catering to the diverse needs of developers and end-users alike.

Architecture of MeiliSearch

A deeper dive into MeiliSearch's architecture provides a fascinating glimpse into how its various components harmoniously interact to deliver impeccable search results. Designed with efficiency and scalability in mind, the MeiliSearch architecture is both streamlined and robust.

1. Tokenizer: At the forefront of the MeiliSearch engine is its tokenizer. Whenever a new document is added to the database, the tokenizer breaks it down into individual tokens (essentially, words or terms). This process is vital as it aids in understanding and categorizing the content, enabling the engine to match user queries with relevant documents effectively.
2. Indexer: Once tokenization is complete, the indexer takes over. It's responsible for creating an index of all tokens, making it possible to rapidly locate specific terms within the dataset. By maintaining an efficient index, MeiliSearch ensures that search queries are executed with incredible speed, offering near-instant results.
3. Database: Underpinning the entire operation is MeiliSearch's database. This is where all indexed data is stored and managed. Designed for rapid data retrieval, the database's architecture plays a crucial role in supporting the engine's high-speed search capabilities. The choice of LMDB (Lightning Memory-Mapped Database) as its storage engine further enhances its performance and reliability.
4. Query Engine: When a user initiates a search, it's the query engine that swings into action. It parses the search query, consults the index to find matching tokens, and retrieves relevant documents from the database. All of this happens in a matter of milliseconds, thanks to the streamlined architecture and the efficient interplay between the tokenizer, indexer, and database.
5. Real-Time Update Mechanism: One of MeiliSearch's standout features is its ability to handle real-time updates. As new data is added or modified, it's instantly indexed and made available for search. This dynamic update mechanism is facilitated by the tight integration between the database and the indexer.
6. HTTP Server: Facilitating the interaction between MeiliSearch and external applications is its embedded HTTP server. It allows developers to communicate with MeiliSearch, send data for indexing, initiate search queries, and retrieve results—all via standard HTTP protocols. This design choice ensures ease of integration with various platforms and programming languages.

The architectural choices made in designing MeiliSearch mirror its overarching philosophy: simplicity without compromising on efficiency. By ensuring each component works in tandem with the others, MeiliSearch provides a seamless, responsive, and powerful search experience, irrespective of the dataset's size or complexity.

Elasticsearch vs Solr vs MeiliSearch

In the realm of search engines, several tools have carved their niche, providing robust solutions for various needs. Elasticsearch and Solr are two such dominant players that have been around for a while, offering powerful search functionalities. MeiliSearch, though a newer entrant, has been making waves with its unique approach. Let's delve into a comparative analysis of these three tools:

1. Elasticsearch: Born out of the Lucene library, Elasticsearch is a real-time distributed search and analytics engine. It's well-suited for big data applications and is widely used in enterprise settings due to its scalability and comprehensive features.
2. Solr: Also built on the Lucene library, Solr is a popular open-source search platform known for its performance, scalability, and flexibility. It offers powerful text search capabilities and is often used in enterprise search applications.
3. MeiliSearch: A newer contender, MeiliSearch is an open-source search engine focusing on simplicity and developer-friendliness. It's designed to provide fast, relevant results even for users who aren't search experts.

Install Meilisearch on MicroK8s

Installing MeiliSearch on a Kubernetes Cluster is quite straightforward. We will install two components: the MeiliSearch Server and the MeiliSearch Dashboard. The MeiliSearch Server is the core component that serves search requests, while the Dashboard is an additional component that provides a user-friendly UI to manage the search server.

Preparation

We will install MeiliSearch using the unofficial Helm Chart template provided by 8grams. Ensure you have Helm installed on your local machine. Additionally, we'll install the Cert Manager to supply a TLS Certificate for the dashboard. Please refer to this article for instructions.

How to get Free SSL Certificate for Kubernetes Cluster using Let’s Encrypt

Installation

First, clone Helm Chart Template by 8grams

~$ git clone git@github.com:8grams/microk8s-helm-chart.git charts/general

Override Helm Chart Template by creating values-server.yaml file

Install it with Helm

~$ helm install meilisearch charts/general -n meilisearch -f values-server.yaml --create-namespace

To install Meilisearch Dashboard, create a file values-dashboard.yaml

And install it using Helm as well

~$ helm install meilisearch-dashboard charts/general -n meilisearch -f values-dashboard.yaml

Check the installation

~$ kubectl -n meilisearch get deployment
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
meilisearch-dashboard-general   1/1     1            1           3m
meilisearch-general             1/1     1            1           1m

Voilà! You now have a powerful open-source search engine running on a single VM within a Kubernetes cluster.