Today we are excited to announce HorizonDB, our next generation geospatial database. HorizonDB has been in development for over 1 year, recently reached the milestone of 1.0 release and is now serving all Radar geocoding and search traffic in production.

Prior to the development of HorizonDB, Radar’s /track endpoint was powered by MongoDB and S2 while /geocoding/* and /search/autocomplete endpoints were powered by Pelias, which uses ElasticSearch and SQLite. Under our next generation infrastructure this has all been unified into a single custom database technology we call HorizonDB.

HorizonDB supports reverse geocoding queries, forwarding geocoding and partial input text queries all from a unified single process Rust binary. Traditionally a separate database and text search engine would be used to power these query patterns, but HorizonDB powers all of this from a single unified runtime.

The HorizonDB kernel is RocksDB, FSTs and Tantivy.

Like MySQL, CockroachDB and other popular databases, HorizonDB interfaces with the filesystem using RocksDB, which is optimized for SSDs. A Rust data structure called FST (similar in concept to String Tries or Finite State Machines) is a powerful abstraction for fast text search since it is stored as string prefixes and has the property of “compacting” the target dataset. Tantivy, a Rust analogue of Lucene, is also used to provide robust text search capabilities.

HorizonDB is incredibly efficient. It can process 2000 requests per second per core with a p50 latency of 1ms. This, from commodity hardware using memory mapped files of SSDs. HorizonDB reduced our monthly cloud costs by 35%, improved recall and lowered latency.

How did we get here? We knew for a long time that running two sets of infrastructure was not going to be our long term plan due to cost and consistency reasons. Still, picking the right time to take on creating new fundamental infrastructure, especially a new database runtime from scratch was not an easy decision to make. Like many innovations at Radar, it started with our Engineering team talking to prospects and customers. Engineers at Radar regularly speak directly with customers and incorporate their feedback into their work. In this case, we had been getting requests from customers for address validation APIs, which are built on a unique data format of address ranges, as opposed to discrete records. None of our data storage technologies supported ranges of addresses, so we knew we were going to need to build new infrastructure. What started with humble beginnings, powering address validation via range files, now handles tens of thousands of queries per second, powering all Radar production workloads for /track, /geocode/forward, /geocode/reverse, /search/autocomplete and /address/validate.

HorizonDB was created by Jeff Kao, with major contributions from Bradley Schoeneweis, Jacky Wang, and Greg Sadetsky. Interested in working on HorizonDB? We're hiring.