Show HN: NNext.net – A Firebase-like managed vector storage for ML applications
10 by wakahiu | 5 comments on Hacker News.
Hi HN. Peter here. As a machine learning engineer, I mostly think in terms of feature vectors, embeddings, and matrices. One of the most useful byproducts of deep neural networks is embeddings because they allow us to represent high-dimensional data in terms of lower-dimensional latent vectors. These feature vectors can be used for downstream applications like similarly search, recommendation systems and near duplicate detection. As an ML engineer, I was frustrated by the lack of a datastore in which vectors are first-class citizens. As a result, most ML engineers, including myself, end up using awkward workarounds to store vectors such as arrays in SQL/NoSQL databases, stringifying vectors and storing them as text in in-memory-based caching systems such as Redis ETC. Furthermore, these systems don't allow for vector-based query operations such as nearest neighbor search. Consequently, engineers have to deploy additional approximate nearest neighbor search systems such as Facebook's FAISS or Spotify's ANNOY. These systems, while nifty and fast, are difficult to install and are costly to maintain. To address these issues, I built NNext, a managed vector datastore in which vectors are first class citizens. NNext, allows you to store vectors along with any json-blob metadata. Furthermore, NNext comes with a fast approximate nearest-neighbor (ANN) search capability. I would love to get feedback on your experience as Data Scientist or ML engineer storing feature vectors and ANN systems. Please shoot me an email at p@nnext.net. https://nnext.net
No comments:
Post a Comment