标签：#liorandb

A normal secondary index can answer: status = "active" It cannot efficiently answer: documents containing "distributed database" LioranDB therefore has a dedicated text-segment architecture. Tokenization Text is split on non-alphanumeric characters. Depending on index options, tokens can be normalized to lowercase and filtered through stopwords. "Building Distributed Databases" becomes ["building", "distributed", "databases"] Segment contents A LioranDB text segment can contain several files and structures: Term dictionary Posting lists Document map Document-length norms Optional term positions Bloom filter Segment metadata A posting connects a term to the local documents containing it. "database" → [doc 2, doc 8, doc 19] Positions can record where the term appears inside each document. That enables more advanced query behaviour and phrase-aware features. Global and local document IDs Each segment assigns compact local IDs to its documents. A separate document map translates them back to global document IDs. This keeps postings smaller while preserving the external identity of the record. Bloom filters Each segment also maintains a Bloom filter for terms. Before reading a segment's postings, the query path can test whether the term might exist there. A negative answer is definitive. A positive answer means the segment may contain the term and should be checked. Query modes The text query layer supports modes such as: AND OR It also emits scored documents and metrics including: Query time Postings read Candidate documents Segments searched Full-text search is essentially a specialized database living beside the document database. Its data structures, compaction behaviour, scoring, and caching needs are different enough that treating it as a plain secondary index would be a mistake. Built by Swaraj Puppalwar under Lioran Group . Learn more: LioranDB Lioran Developer Solutions Lioran Group

Disk structures are durable, but updating them for every write is expensive. LioranDB uses memtables to absorb writes before flushing them to the on-disk B+ tree. What is a memtable? A memtable is an ordered in-memory map. In LioranDB, each entry contains either: Value ( bytes ) or: Tombstone A tombstone represents a deletion. The memtable also tracks: Approximate memory usage Minimum LSN Maximum LSN Entry count Put count Delete count The write path A simplified write path looks like this: Application write ↓ WAL durability ↓ Mutable memtable ↓ Immutable memtable queue ↓ Background flush ↓ Disk B+ tree The active mutable memtable accepts new writes. When it crosses a size limit, the engine rotates it into an immutable memtable. That immutable table is no longer modified and can safely be flushed in the background. Why ordered maps? LioranDB uses an ordered map for memtable entries. This helps because the flush process can emit keys in sorted order, which is friendly to the B+ tree and bulk-write paths. It also simplifies range merging between: Mutable data Immutable data On-disk pages Backpressure Background flushing cannot be allowed to fall behind forever. LioranDB therefore tracks limits such as: Maximum immutable memtables Maximum immutable bytes Partition-wide queue limits Maximum writer stall duration If the disk cannot drain the backlog quickly enough, the foreground write path slows down. That may sound undesirable, but controlled backpressure is much safer than consuming memory until the process dies. A memtable is not merely a cache. It is a pressure valve between CPU-speed writes and disk-speed persistence. Without that valve, the engine would either become slow on every commit or dangerously accumulate unbounded work. Built by Swaraj Puppalwar under Lioran Group . Links: LioranDB Lioran Developer Solutions Lioran Group

Most developers think of LioranDB as a document database. Internally, however, its storage engine does not understand documents, objects, fields, or JSON. It understands only: table + key bytes + value bytes That separation is intentional. The architecture LioranDB is split into two major layers: Application ↓ Document DBMS ↓ Transactional key-value engine ↓ WAL, memtables, B+ tree, pager and disk The engine exposes operations such as: get ( table , key ) put ( table , key , value ) delete ( table , key ) scan ( table , range ) The DBMS layer then adds document-oriented features: Collections JSON encoding Queries Updates Secondary indexes Text indexes Transactions For example, a secondary index can be represented as: idx:status:active → document_id A text index can be represented as: inv:database → posting_list The storage engine does not need to know what status , active , or database means. It only stores ordered bytes. Why this matters This architecture keeps the core engine small and reusable. The engine focuses on difficult low-level concerns: Durability Page management Transactions Recovery Ordering Concurrency Range scans The DBMS focuses on application-level semantics. This also makes it possible to build different data models over the same engine in the future. A document database is therefore not one giant component. It is a collection of carefully separated layers. That separation is one of the most important architectural decisions inside LioranDB. LioranDB is being developed by Swaraj Puppalwar under Lioran Group . Learn more: LioranDB Lioran Developer Solutions Lioran Group