Neural network programming for increasing the profitability of stores

Life without shops is already impossible to imagine; they save the most expensive resource – time. Instead of doing everything necessary for the household on our own, we prefer specialization and efficiency. Cheap oil allows you to make purchases without leaving your house; a specially trained courier will deliver your order by car. In the wake of the hype of the pandemic, the demand for the development of online stores has soared. Competition across the network has increased, therefore, owners need to find new ways of winning and retaining customers. Now, that's what we're looking to offer.

We've programmed a residual neural network, sewn into a mobile application, which increases the profitability of online stores. By sending a photo to the robot, the user formulates his preferences in terms of contours, shapes, textures and colors. Next, a neural network is activated, which is pre-trained in ‘beautiful images’ using a 100-GB database consisting of 2 million selected photos of goods. The photos are accumulated through the analysis and parsing of open sources, including 600 of the largest online stores. This idea is based on the search for a product that is as similar as possible to the model one shown in the photo. Instead of letting potential customers fly away to competitors, they are asked to show their preferences and are offered a link to purchase a similar product here and now.

A neural network is a mathematical model that functions on the principle of a biological neural network. That is, it acts in the same way as the nerve cells of a living organism. Neural networks simulate the activity of the human brain and are able not only to perform a once-programmed sequence of actions on predefined data, but also are able to analyze newly received information independently.

In our case, the neural network works the same way as a live sales consultant in a jewelry store or a stylist when selecting models of clothing and shoes.

A GAN (Generative adversarial network) has also been created, capable of generating completely new jewelry designs based on the manufacturer's catalog.

The generator and the discriminator

GANs are not programmed by the usual commands, but are independently trained in a specially created environment. After training, the grid is able to create products similar to those on which it was trained.

Indeed, GAN consists of two neural networks.

• The ‘generator’ is a network capable of generating images much like an exemplary one does.
• The ‘discriminator’ is a neural network that tries to distinguish real pictures from fake ones that are constructed by the generator.

Metaphorically, the process resembles learning to draw at art school. The student tries to repeatedly reproduce Vincent Van Gogh's ‘Starry Night’, each time showing the result to the teacher to acquire advice on how to improve.

While the master issues remarks, the padawan eliminates them, again and again checking with the teacher. Step by step, the images of the student differ less and less from the original. The teacher is extremely persistent, he practically exhausts the student with endless criticism. A man of flesh and blood would not be able to bear such a tyrant. The neuroscientist is completely cleansed of feelings and any other ‘human factor’, so the robot is able to perform 100,000 cycles of training-correction in 16 hours at the click of the teacher's fingers.

The discriminator evaluates what the generator did, determining the differences. Two neural networks are trained together: it becomes increasingly difficult for the discriminator to find differences, and the generator is constantly trying to improve the result.

GAN training works in the following way:

• We get a set of real and generated images;
• We train the discriminator to better distinguish between the original and the fake;
• We train a combined model consisting of both networks to improve the operation of the generator.

GAN Architecture

The discriminator uses the classical convolutional network architecture (CNN for short) to distinguish a real image from a fake one. The architecture is aimed at effective pattern recognition using convolution operations, the essence of which requires each fragment of the image to be multiplied by the matrix (core) of the convolution element by element. The result is summed and written to the output matrix. At each stage, its size is halved. At the end, the discriminator returns the number 0 if fake, and 1 if original.

The generator is also a convolutional network, but solves the inverse problem using a process called reverse convolution. The generator receives random noise as its input, and then generates images based on it, performing operations that are the reverse of the discriminator's actions due to transposed convolutional layers. The core passes through the matrix, increasing its dimension by 2 times in the process.

ResNet Architecture

The residual neural network (ResNet) is a neural network with connections for quick access. Such a neural network can be represented as an ensemble of shallower grids that are united by common outputs.

Client and Server

The management server accepts images from the mobile application and quickly searches for similar products among the database of one product category, which is about 15,000 images in size. There is an orchestration panel on the server to manage requests from clients. All photos are queued for processing and are waiting for the queue to be released. The stack is organized according to the FIFO (First In First Out) principle.

The client is an application that has the right to send tasks for calculating similar images to the server. The neural network decomposes the incoming image into 100–150 vectors (small images) and does the same with each of the base 15,000 images (among which we are looking for). Next, the neural network tries to find the most similar images – in which the maximum number of vectors coincided.

Surprisingly, the neural network will try to find at least something. That's why even if it doesn't find photos of jewelry, it's likely that abstract geometric images will be produced instead.

What does the buyer get?

One of the neural network implementations took place for the brand ‘Svetlov’ of the Belgorod jewelry factory ‘Art-Karat’. With the help of a neural network, the buyer of an offline or online store can quickly pick up analog products among a wide range of jewelry from the Karat factory, even if they don't know exactly what they want. You simply need to take a picture, upload an image, or draw a sketch of a suitable product and upload it to the application. The search for similar images takes 7–10 seconds, the results are then immediately displayed on the screen.

You can search for products with a complex insert that cannot be detected in filters. For the same product, you can choose something from other categories – for example, according to the photo of a ring with a complex insert, you can choose a harmoniously matching watch with a similar dial. By uploading a photo of an existing item, you can find a similar one. If you know what your loved one likes (for example, certain patterns or colors), you can pick up themed products as gifts.

When searching for similar products, they are analyzed in terms of:

Neural network algorithms can compare objects of complex shapes and will show what we ourselves would consider as alternatives.

The parameters are analyzed in 10 seconds, and as a result, more than 50 suggestions are issued. The neural network does not order the results in descending order of similarity since each buyer has their own determining criterion of choice.

New designs

The neural network is capable of generating previously non-existent jewelry and clothing designs. Let's look at the example of the latter and at how this happens

After the GAN has learned how to draw sweaters, we can generate an unlimited number of new designs. We decided to train GAN only on photos of sweaters that are popular with customers. Gradually adding popular sweaters to the training sample, we get reference designs that indirectly contain signs of successful goods. Once again, we emphasize that the designs were created by a machine: such designs did not exist before and were not drawn. The neural network remembers all the images that are ‘fed’ to it and generates images similar to what it saw, but not the same in any way.

Recommendations

Experiment and be sure to look through all the options that the neural network offers based on the search results.

• Take photos of the search query from the same angle as the products in the target store.
• Exclude unnecessary and non-essential elements from the frame.
• Use a white background.
• Avoid using other people's photos with watermarks.
• Draw sketches in maximum detail.

‘The Wise Grandfather’

You may be familiar with simple product recommendation services that work with a receipt database. The proposed technology is fundamentally different from the correlation recommendations, which examine only the statistics of previous customer checks.

Imitators are not able to come up with new things, but willingly borrow someone else's speech or ‘import’ behaviors. Neural network recommendations are effectively a ‘wise grandfather’ who knows in advance and has ready-made solutions, which is different from reproducing statistical events that have already happened.

If a company sells mono-branded goods, it is possible to instantly find similar products among the nomenclature of competitors and:

• Offer alternative products in return;
• Conduct a reconciliation of the price offering for tens of thousands of products;
• Avoid copying product designs by tracking the appearance of similar designs;
• Add a JS function to the site, for instance ‘upload a photo of the product, and we’ll find it among our assortment, or the next best alternative’.

A user can provide a picture or a link to a product from someone else's store, and your online store will instantly select high-quality alternatives. Instantly – because it's been completed in advance. Indeed, the neural network works like human logic.

A catalog of jewelry generated by the neural network

A Comparative Experiment in Classifying Images of Jewelry Using Convolutional Neural Networks