Loading...

Large-scale image search system

Non-fungible tokens (NFTs) are cryptographic assets on a blockchain with unique identification codes and metadata that distinguish them from each other.

Time: 8 months

Key idea

There is a vast amount of NFTs traded every day. The majority of NFTs today contain a vivid image, such as an avatar or football card. And it would be cool if it was possible to search for NFTs or for collections of NFTs in different ways:

  • find NFT by a similar image;
  • free-text search by images (using AI, without any text descriptions);
  • using their names;
  • using traits (building blocks, such as a hat or glasses);
  • by contract;
  • by other different filters.

Steps

1. Data collection.
The data is collected from existing APIs by separate scripts, according to the following algorithm:

  • the primary data was loaded at the start of the project;
  • in the process of work, scripts regularly fetch new data;
  • after collection, the data is indexed for search.

2. Scripts request NFT data from a public API and NFT images from an URL in IPFS distributed data storage.
Two types of search are currently supported:

  • full text, using ElasticSearch/OpenSearch;
  • AI-based, using the neural network.

How it works

The full-text search allows us to include single or multiple words or phrases in a search query that returns NFTs that match the search condition.

The second type of search uses a neural network, namely the CLIP model developed by OpenAI research and development company.

CLIP is one of a class of deep neural networks that can process both images and text and, for example, find a suitable caption (or tag) for an image or find similar images. Such neural networks can be used to find the text snippet that best represents a given image and vice versa. In other words, modern AI provides a very powerful bridge between computer vision and natural language understanding.

The use of this technology gives a lot of freedom to search against NFTs. It allows to search for the necessary or similar NFTs by image, search for NFT images by description, and so on. In other words, it allows us to link NFT names and descriptions with their images.

Features:

  • searching;
  • real-time rarity calculation;
  • dynamic metadata updates;
  • all NFTs in one spot.
Let’s work on your next project together!

Challenges

The client approached our company with a request to develop a system for NFT images that would solve three main functions:

  • search for similar images;
  • search by description;
  • search by collections.

Solution

It is worth noting that the system can be used both for searching for NFTs and for regular pictures. The system's scalability was tested on at least 6 million indexed NFTs. In addition to image search based on ML, you can check how our team has developed an AI-assisted text search and try its demo.

Have a project in mind?
Reach out to us. We’ll make something awesome together.
Have a project in mind?
...