This article shows how to use the image feature extraction model VGG and the vector search engine Milvus to build an video search system.

The entire workflow of the video search system can be represented by the following picture:


When importing video, first use the OpenCV algorithm library to cut a frame of a video in the incoming system, then use the feature extraction model VGG to extract the vectors of these key frame pictures, and then import the extracted vectors into Milvus. For the original video, Minio is used for storage, and then Redis is used to store the correspondence between video and vector.

When searching for video, first use the same VGG model to convert the uploaded image into a feature vector, then take this vector to Milvus to perform similar vector search, find the most similar vectors, and then use the vectors stored in Redis. The corresponding relationship with the video is to take the video from Minio and return it to the front-end interface.

This article uses Tumblr's approximately 100,000 animated gifs as an example to build an end-to-end solution that uses video search video. Readers can use their own video files to build the system.

This article builds the code of the video search video has been uploaded to the GitHub warehouse, the warehouse address is:

Step1 image build

The entire video search system needs to use Milvus0.10.0 docker, Redis docker, Minio docker, front-end interface docker and background API docker. The front-end interface docker and background API docker need to be built by the reader, and the remaining three dockers can be directly pulled from the docker hub.

# Get the video search code
$ git clone

# Build front-end interface docker and api docker images
$ cd search-video-demo & make all

Step2 environment configuration

This article uses docker-compose to manage the five containers mentioned earlier. The configuration of the docker-compose.yml file can refer to the following table:


Can't use or localhost



Upload tmp file


Can't use or localhost


Can't use or localhost





The larger the amount of data, the more buckets are needed

The ip address in the above table is the server address used in this article to build the video search system of the map. The user needs to modify it according to his actual situation.

Milvus, Redis and Minio require users to manually create storage directories and then perform the corresponding path mapping in docker-compose.yml. For example, the storage directories created in this article are:

/mnt/redis/data /mnt/minio/data /mnt/milvus/db

So the configuration part of Milvus, Redis and Minio in docker-compose.yml can be configured according to the following figure: img

Step3 system startup

Use the docker-compose.yml modified in Step 2 to start the five docker containers needed to search the video system:

$ docker-compose up -d

After the startup is complete, you can use the docker-compose ps command to check whether the five docker containers have started successfully. The result interface after normal startup is as shown below: img

Up to now, the entire Yisou video system has been built, but there is no video in the system's base library.

Step4 video import

Under the deploy directory of the system code repository, there is a video import script named Readers only need to modify the path of the video file in the script and the video import time interval to run the script for video import. img

data_path :The path of the video to be imported.

time.sleep(0.5) :indicates the time interval for importing video. In this paper, the server built by the video search system has 96 CPU cores. The time interval for importing video is set to 0.5 seconds. If there are fewer cpu cores, the time interval for importing video should be properly extended, otherwise it will cause the cpu to take up too much and generate a zombie process.

The startup command is as follows:

$ cd deploy
$ python3

The import process is shown below:


After waiting for the video to be imported, the entire Yisou video system is all set up!

Open the browser and enter to see the interface of searching video with pictures, as shown below: img

Click the settings icon in the upper right corner, you can see the video in the bottom library:


Click the upload box on the left, you can upload a picture you want to search, and then search for videos containing similar shots on the right interface:


Next, let's enjoy the fun of searching video with pictures!

To learn more about Milvus applications and learning materials, please visit: