Logistic regression is a straightforward however standard machine studying algorithm for binary classification that makes use of the logistic, or sigmoid, perform at its core. It additionally comes carried out within the OpenCV library.

On this tutorial, you’ll learn to apply OpenCV’s logistic regression algorithm, beginning with a customized two-class dataset that we’ll generate ourselves. We are going to then apply these expertise for the precise picture classification software in a subsequent tutorial.

After finishing this tutorial, you’ll know:

A number of of a very powerful traits of the logistic regression algorithm.

Find out how to use the logistic regression algorithm on a customized dataset in OpenCV.

Kick-start your challenge with my guide Machine Studying in OpenCV. It gives self-study tutorials with working code.

Let’s get began.

## Tutorial Overview

This tutorial is split into two components; they’re:

Reminder of What Logistic Regression Is

Discovering Logistic Regression in OpenCV

## Reminder of What Logistic Regression Is

The subject surrounding logistic regression has already been defined effectively in these tutorials by Jason Brownlee [1, 2, 3], however let’s first begin with brushing up on a number of the most essential factors:

Logistic regression takes its title from the perform used at its core, the logistic perform (also called the sigmoid perform).

Regardless of using the phrase regression in its title, logistic regression is a technique for binary classification or, in less complicated phrases, issues with two-class values.

Logistic regression might be thought to be an extension of linear regression as a result of it maps (or squashes) the real-valued output of a linear mixture of options right into a likelihood worth inside the vary [0, 1] by means of using the logistic perform.

Inside a two-class state of affairs, the logistic regression methodology fashions the likelihood of the default class. As a easy instance, let’s say that we are attempting to differentiate between lessons of flowers A and B from their petal depend, and we’re taking the default class to be A. Then, for an unseen enter X, the logistic regression mannequin would give the likelihood of X belonging to the default class A:

$$ P(X) = P(A = 1 | X) $$

The enter X is classed as belonging to the default class A if its likelihood P(X) > 0.5. In any other case, it’s categorised as belonging to the non-default class B.

The logistic regression mannequin is represented by a set of parameters generally known as coefficients (or weights) realized from the coaching information. These coefficients are iteratively adjusted throughout coaching to attenuate the error between the mannequin predictions and the precise class labels.

The coefficient values could also be estimated throughout coaching utilizing gradient descent or most chance estimation (MLE) methods.

## Discovering Logistic Regression in OpenCV

Let’s begin with a easy binary classification job earlier than transferring on to extra advanced issues.

As now we have already finished in associated tutorials by means of which we familiarised ourselves with different machine studying algorithms in OpenCV (such because the SVM algorithm), we will be producing a dataset that contains 100 information factors (specified by n_samples), equally divided into 2 Gaussian clusters (specified by facilities) having an ordinary deviation set to five (specified by cluster_std). To have the ability to replicate the outcomes, we will once more exploit the random_state parameter, which we’re going to set to fifteen:

# Generate a dataset of 2D information factors and their floor reality labels

x, y_true = make_blobs(n_samples=100, facilities=2, cluster_std=5, random_state=15)

# Plot the dataset

scatter(x[:, 0], x[:, 1], c=y_true)

present()

# Generate a dataset of 2D information factors and their floor reality labels

x, y_true = make_blobs(n_samples=100, facilities=2, cluster_std=5, random_state=15)

# Plot the dataset

scatter(x[:, 0], x[:, 1], c=y_true)

present()

The code above ought to generate the next plot of information factors. It’s possible you’ll notice that we’re setting the colour values to the bottom reality labels to have the ability to distinguish between information factors belonging to the 2 completely different lessons:

The following step is to separate the dataset into coaching and testing units, the place the previous shall be used to coach the logistic regression mannequin and the latter to check it:

# Cut up the info into coaching and testing units

x_train, x_test, y_train, y_test = ms.train_test_split(x, y_true, test_size=0.2, random_state=10)

# Plot the coaching and testing datasets

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_train[:, 0], x_train[:, 1], c=y_train)

ax1.set_title(‘Coaching information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax2.set_title(‘Testing information’)

present()

# Cut up the info into coaching and testing units

x_train, x_test, y_train, y_test = ms.train_test_split(x, y_true, test_size=0.2, random_state=10)

# Plot the coaching and testing datasets

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_train[:, 0], x_train[:, 1], c=y_train)

ax1.set_title(‘Coaching information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax2.set_title(‘Testing information’)

present()

The picture above signifies that the 2 lessons seem clearly distinguishable within the coaching and testing information. Because of this, we count on that this binary classification downside needs to be a simple job for the educated linear regression mannequin. Let’s create and practice a logistic regression mannequin in OpenCV to finally see the way it performs on the testing a part of the dataset.

Step one is to create the logistic regression mannequin itself:

# Create an empty logistic regression mannequin

lr = ml.LogisticRegression_create()

# Create an empty logistic regression mannequin

lr = ml.LogisticRegression_create()

Within the subsequent step, we will select the coaching methodology by which we wish the mannequin’s coefficients to be up to date throughout coaching. The OpenCV implementation lets us select between two completely different strategies: the Batch Gradient Descent and the Mini-Batch Gradient Descent strategies.

If the Batch Gradient Descent methodology is chosen, the mannequin’s coefficients shall be up to date utilizing all the coaching dataset at every iteration of the gradient descent algorithm. If we’re working with very massive datasets, then this methodology of updating the mannequin’s coefficients can develop into very computationally costly.

A extra sensible method to updating the mannequin’s coefficients, particularly when working with massive datasets, is to go for a Mini-Batch Gradient Descent methodology, which relatively divides the coaching information into smaller batches (referred to as mini-batches, therefore the title of the strategy) and updates the mannequin’s coefficients by processing one mini-batch at a time.

We could verify what OpenCV implements as its default coaching methodology by making use of the next line of code:

# Verify the default coaching methodology

print(‘Coaching Technique:’, lr.getTrainMethod())

# Verify the default coaching methodology

print(‘Coaching Technique:’, lr.getTrainMethod())

The returned worth of 0 represents the Batch Gradient Descent methodology in OpenCV. If we wish to change this to the Mini-Batch Gradient Descent methodology, we are able to accomplish that by passing ml.LogisticRegression_MINI_BATCH to the setTrainMethod perform, after which proceed to set the scale of the mini-batch:

# Set the coaching methodology to Mini-Batch Gradient Descent and the scale of the mini-batch

lr.setTrainMethod(ml.LogisticRegression_MINI_BATCH)

lr.setMiniBatchSize(5)

# Set the coaching methodology to Mini-Batch Gradient Descent and the scale of the mini-batch

lr.setTrainMethod(ml.LogisticRegression_MINI_BATCH)

lr.setMiniBatchSize(5)

Setting the mini-batch measurement to five implies that the coaching information shall be divided into mini-batches containing 5 information factors every, and the mannequin’s coefficients shall be up to date iteratively after every of those mini-batches is processed in flip. If we needed to set the scale of the mini-batch to the entire variety of samples within the coaching dataset, this might successfully end in a Batch Gradient Descent operation for the reason that total batch of coaching information can be processed without delay, at every iteration.

Subsequent, we will outline the variety of iterations that we wish to run the chosen coaching algorithm for, earlier than it terminates:

# Set the variety of iterations

lr.setIterations(10)

# Set the variety of iterations

lr.setIterations(10)

We’re now set to coach the logistic regression mannequin on the coaching information:

# Prepare the logistic regressor on the set of coaching information

lr.practice(x_train.astype(float32), ml.ROW_SAMPLE, y_train.astype(float32))

# Prepare the logistic regressor on the set of coaching information

lr.practice(x_train.astype(float32), ml.ROW_SAMPLE, y_train.astype(float32))

As talked about earlier, the coaching course of goals to regulate the logistic regression mannequin’s coefficients iteratively to attenuate the error between the mannequin predictions and the precise class labels.

Every coaching pattern now we have fed into the mannequin contains two characteristic values, denoted by $x_1$ and $x_2$. Which means we should always count on the mannequin now we have generated to be outlined by two coefficients (one per enter characteristic) and a further coefficient that defines the bias (or intercept).

Then the likelihood worth, $hat{y}$, returned the mannequin might be outlined as follows:

$$ hat{y} = sigma( beta_0 + beta_1 ; x_1 + beta_2 ; x_2 ) $$

the place $beta_1$ and $beta_2$ denote the mannequin coefficients, $beta_0$ the bias, and $sigma$ the logistic (or sigmoid) perform that’s utilized to the real-valued output of the linear mixture of options.

Let’s print out the realized coefficient values to see whether or not we retrieve as many as we count on:

# Print the realized coefficients

print(lr.get_learnt_thetas())

# Print the realized coefficients

print(lr.get_learnt_thetas())

[[-0.02413555 -0.34612912 0.08475047]]

[[-0.02413555 -0.34612912 0.08475047]]

We discover that we retrieve three values as anticipated, which implies that we are able to outline the mannequin that finest separates between the two-class samples that we’re working with by:

$$ hat{y} = sigma( -0.0241 – ; 0.3461 ; x_1 + 0.0848 ; x_2 ) $$

We will assign a brand new, unseen information level to both of the 2 lessons by plugging in its characteristic values, $x_1$ and $x_2$, into the mannequin above. If the likelihood worth returned by the mannequin is > 0.5, we are able to take it as a prediction for sophistication 0 (the default class). In any other case, it’s a prediction for sophistication 1.

Let’s go forward to see how effectively this mannequin predicts the goal class labels by making an attempt it out on the testing a part of the dataset:

# Predict the goal labels of the testing information

_, y_pred = lr.predict(x_test.astype(float32))

# Compute and print the achieved accuracy

accuracy = (sum(y_pred[:, 0].astype(int) == y_test) / y_test.measurement) * 100

print(‘Accuracy:’, accuracy, ‘%’)

# Predict the goal labels of the testing information

_, y_pred = lr.predict(x_test.astype(float32))

# Compute and print the achieved accuracy

accuracy = (sum(y_pred[:, 0].astype(int) == y_test) / y_test.measurement) * 100

print(‘Accuracy:’, accuracy, ‘%’)

We will plot out the bottom reality in opposition to the expected lessons for the testing information, in addition to print out the bottom reality and predicted class labels, to research any misclassifications:

# Plot the bottom reality and predicted class labels

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax1.set_title(‘Floor Fact Testing information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_pred)

ax2.set_title(‘Predicted Testing information’)

present()

# Print the bottom reality and predicted class labels of the testing information

print(‘Floor reality class labels:’, y_test, ‘n’,

‘Predicted class labels:’, y_pred[:, 0].astype(int))

# Plot the bottom reality and predicted class labels

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax1.set_title(‘Floor Fact Testing information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_pred)

ax2.set_title(‘Predicted Testing information’)

present()

# Print the bottom reality and predicted class labels of the testing information

print(‘Floor reality class labels:’, y_test, ‘n’,

‘Predicted class labels:’, y_pred[:, 0].astype(int))

Floor reality class labels: [1 1 1 1 1 0 0 1 1 1 0 0 1 0 0 1 1 1 1 0]

Predicted class labels: [1 1 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 0]

Floor reality class labels: [1 1 1 1 1 0 0 1 1 1 0 0 1 0 0 1 1 1 1 0]

Predicted class labels: [1 1 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 1 1 0]

On this method, we are able to see that one pattern initially belonged to class 1 within the floor reality information however has been misclassified as belonging to class 0 within the mannequin’s prediction.

All the code itemizing is as follows:

from cv2 import ml

from sklearn.datasets import make_blobs

from sklearn import model_selection as ms

from numpy import float32

from matplotlib.pyplot import scatter, present, subplots

# Generate a dataset of 2D information factors and their floor reality labels

x, y_true = make_blobs(n_samples=100, facilities=2, cluster_std=5, random_state=15)

# Plot the dataset

scatter(x[:, 0], x[:, 1], c=y_true)

present()

# Cut up the info into coaching and testing units

x_train, x_test, y_train, y_test = ms.train_test_split(x, y_true, test_size=0.2, random_state=10)

# Plot the coaching and testing datasets

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_train[:, 0], x_train[:, 1], c=y_train)

ax1.set_title(‘Coaching information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax2.set_title(‘Testing information’)

present()

# Create an empty logistic regression mannequin

lr = ml.LogisticRegression_create()

# Verify the default coaching methodology

print(‘Coaching Technique:’, lr.getTrainMethod())

# Set the coaching methodology to Mini-Batch Gradient Descent and the scale of the mini-batch

lr.setTrainMethod(ml.LogisticRegression_MINI_BATCH)

lr.setMiniBatchSize(5)

# Set the variety of iterations

lr.setIterations(10)

# Prepare the logistic regressor on the set of coaching information

lr.practice(x_train.astype(float32), ml.ROW_SAMPLE, y_train.astype(float32))

# Print the realized coefficients

print(lr.get_learnt_thetas())

# Predict the goal labels of the testing information

_, y_pred = lr.predict(x_test.astype(float32))

# Compute and print the achieved accuracy

accuracy = (sum(y_pred[:, 0].astype(int) == y_test) / y_test.measurement) * 100

print(‘Accuracy:’, accuracy, ‘%’)

# Plot the bottom reality and predicted class labels

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax1.set_title(‘Floor reality testing information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_pred)

ax2.set_title(‘Predicted testing information’)

present()

# Print the bottom reality and predicted class labels of the testing information

print(‘Floor reality class labels:’, y_test, ‘n’,

‘Predicted class labels:’, y_pred[:, 0].astype(int))

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

from cv2 import ml

from sklearn.datasets import make_blobs

from sklearn import model_selection as ms

from numpy import float32

from matplotlib.pyplot import scatter, present, subplots

# Generate a dataset of 2D information factors and their floor reality labels

x, y_true = make_blobs(n_samples=100, facilities=2, cluster_std=5, random_state=15)

# Plot the dataset

scatter(x[:, 0], x[:, 1], c=y_true)

present()

# Cut up the info into coaching and testing units

x_train, x_test, y_train, y_test = ms.train_test_split(x, y_true, test_size=0.2, random_state=10)

# Plot the coaching and testing datasets

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_train[:, 0], x_train[:, 1], c=y_train)

ax1.set_title(‘Coaching information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax2.set_title(‘Testing information’)

present()

# Create an empty logistic regression mannequin

lr = ml.LogisticRegression_create()

# Verify the default coaching methodology

print(‘Coaching Technique:’, lr.getTrainMethod())

# Set the coaching methodology to Mini-Batch Gradient Descent and the scale of the mini-batch

lr.setTrainMethod(ml.LogisticRegression_MINI_BATCH)

lr.setMiniBatchSize(5)

# Set the variety of iterations

lr.setIterations(10)

# Prepare the logistic regressor on the set of coaching information

lr.practice(x_train.astype(float32), ml.ROW_SAMPLE, y_train.astype(float32))

# Print the realized coefficients

print(lr.get_learnt_thetas())

# Predict the goal labels of the testing information

_, y_pred = lr.predict(x_test.astype(float32))

# Compute and print the achieved accuracy

accuracy = (sum(y_pred[:, 0].astype(int) == y_test) / y_test.measurement) * 100

print(‘Accuracy:’, accuracy, ‘%’)

# Plot the bottom reality and predicted class labels

fig, (ax1, ax2) = subplots(1, 2)

ax1.scatter(x_test[:, 0], x_test[:, 1], c=y_test)

ax1.set_title(‘Floor reality testing information’)

ax2.scatter(x_test[:, 0], x_test[:, 1], c=y_pred)

ax2.set_title(‘Predicted testing information’)

present()

# Print the bottom reality and predicted class labels of the testing information

print(‘Floor reality class labels:’, y_test, ‘n’,

‘Predicted class labels:’, y_pred[:, 0].astype(int))

On this tutorial, now we have thought-about setting values for 2 particular coaching parameters of the logistic regression mannequin carried out in OpenCV. The parameters outlined the coaching methodology to make use of and the variety of iterations for which we wished to run the chosen coaching algorithm through the coaching course of.

Nevertheless, these will not be the one parameter values that may be set for the logistic regression methodology. Different parameters, similar to the training charge and the kind of regularization to carry out, may also be modified to attain higher coaching accuracy. Therefore, we recommend that you just discover these parameters and examine how completely different values can have an effect on the mannequin’s coaching and prediction accuracy.

## Additional Studying

This part gives extra assets on the subject if you wish to go deeper.

### Books

### Web sites

## Abstract

On this tutorial, you realized methods to apply OpenCV’s logistic regression algorithm, beginning with a customized two-class dataset we generated.

Particularly, you realized:

A number of of a very powerful traits of the logistic regression algorithm.

Find out how to use the logistic regression algorithm on a customized dataset in OpenCV.

Do you will have any questions?

Ask your questions within the feedback beneath, and I’ll do my finest to reply.

## Get Began on Machine Studying in OpenCV!

Learn to use machine studying methods in picture processing tasks

…utilizing OpenCV in superior methods and work past pixels

Uncover how in my new E book:

Machine Learing in OpenCV

It gives self-study tutorials with all working code in Python to show you from a novice to professional. It equips you with

logistic regression, random forest, SVM, k-means clustering, neural networks,

and rather more…all utilizing the machine studying module in OpenCV

Kick-start your deep studying journey with hands-on workout routines

See What’s Inside