Yandex has migrated its Search product to a transformer neural network based text processing platform, which has been in the making (and in development) for the past decade.
Yandex’s search engine is now better at determining and understanding the semantic interaction between requests, and the quality of the webpages which it crawls, and ultimately chooses to index/rank.
As claimed by Tigran Khudaverdyan at YaC 2020:
This is the biggest change in Yandex search in the last 10 years, and it laid the foundation for the next years.
Tigran Khudaverdyan said the company has been using artificial intelligence technology for over 20 years. Machine learning is the foundation of all services (search, ads, navigator, recommendations). The major breakthrough was the search technology portion, which is not visible to the eye. “The search for “Yandex” switched to new text analysis technologies based on huge neural networks, on transformer architecture,” he explained.
The transformer within is a sequential multiplication of matrices, says Yekaterina Serazhim, a specialist in ranking efficiency in Yandex quest. There is a single GPU card, but with it, you can’t read anything. A lot of GPU cards are required for you. The challenge of transmitting data across the network occurs as soon as there are more of them. “She said You need to physically bind these GPU accelerators together physically.
Now the corporation uses committees to achieve this. Eight GPU accelerators are packaged together on the host and are installed. In a rack, the servers are then lined up tight and wrapped in a net.
What Is YATI?
YATI stands for Yet Another Transformer with Improvements.
This is a new text analysis technology in which algorithms learn to recognize the relationship between a query and the content of documents (webpages).
