When Pinecone announced a vector database at the beginning of last year, it was creating something tailored to machine learning and aimed at data scientists.
The idea was that you could query this data in a machine-readable format, making it much faster.
MORE FROM RAVZGADGET: Amazon To Delist Top Seller Appario On India Marketplace Amid Regulatory Heat
Initially, this entailed semantic searches, in which users could search based on meaning rather than specific words.
However, as people put Pinecone to use, it became clear that there were use cases where specific keywords mattered, and today the company announced that it is now possible to conduct searches that combine both semantic and keyword searches, a process that company founder and CEO Edo Liberty refers to as hybrid search.
“We’ve conducted a lot of research on this topic and we found that, in fact, hybrid search ends up being better [in many cases].
It’s better in the sense that if you can combine both semantic search, this is the deep NLP encoding of sentences that gets the context and the meaning and so on, but you can also infuse that with specific keywords…the combination of those two ends up being significantly better,” Liberty told TechCrunch.
In fact, he believes the two complement each other well, particularly when it comes to industry-specific terms.
This could be a doctor looking for keywords related to a specific disease. In those cases, combining a question and some specific keywords related to a given disease may yield better results in the medical context.
He claims that the keywords never take precedence over the semantic question asked by the user, but they do provide some additional information to help return more meaningful results.
“You might know exactly what you’re looking for, and you might be able to provide extra oomph when you make your semantic search keyword-aware – and that actually helps a lot.
So I don’t want to throw away the good parts of keyword search [by relying completely on semantic search]. I don’t want the keywords to be in the driver’s seat, but I don’t to ignore them completely either,” he said.
As Liberty told us during the company’s $28 million Series A last year, search has become a major use case:
“The predominant use of the vector databases is for search, and search in the broad sense of the word.
It’s searching through documents, but you can think about search as information retrieval in general, discovery, recommendation, anomaly detection and so on,” he said at the time.
Pinecone launched in 2019 and has raised $38 million, per Crunchbase.
Your point of view caught my eye and was very interesting. Thanks. I have a question for you.