IC Imaging Control ActiveX

Performing Advanced Image Processing

Performing Advanced Image Processing

This chapter illustrates how to perform sophisticated image processing.

The source code for this sample program can be found in the samples\VB6\Advanced Image Processing directory. Before looking at this chapter, we suggest that you first read chapter Displaying Buffers on the ImageAvailable Event.

Overview

This chapter illustrates how to:

  • display image buffers
  • render an overlay on the live image
  • process mouse events
  • process data contained in image buffers

The sample described in this chapter lets you draw a rectangle on the live image using the mouse. The region specified by the rectangle is used to check whether the live image has changed. If so, the changed image is displayed. The threshold for the difference between the images can be set by the user.

Setting up the Project

Create a new project and add IC Imaging Control to the form. Before you run the program, select the video device, input and video format as shown in the First Steps in Visual Basic 6 chapter. Alternatively, run the program without selecting a device. In this case, the program shows the device selection dialog provided by IC Imaging Control. If you close this dialog without making a selection, the displays error message and terminates.

Add two buttons to the form and label them Device and Settings. Name them cmdDevice and cmdSettings respectively. If you click the "Device" button, a device selection dialog is displayed. After you have selected a valid device, setupDevice is called. This is a helper procedure, which sets up a FrameHandlerSink that is used to grab images. The sink is initialized with a ring buffer size of 5 and Y800 as color format. ICImagingControl.LiveDisplay is disabled, because the buffers are drawn by hand in this sample. The Settings button shows a dialog that allows you to adjust the VCDProperties of the currently selected device. The code for both buttons looks as follows:

Private Sub cmdDevice_Click()
    With ICImagingControl1

        If .DeviceValid Then
            If .LiveVideoRunning Then
                .LiveStop
            End If
        End If

        .ShowDeviceSettingsDialog

        If .DeviceValid Then
            cmdStart.Enabled = True
            cmdSettings.Enabled = True
            setupDevice
        End If
    End With
End Sub
Private Sub cmdSettings_Click()
    ICImagingControl1.ShowPropertyDialog
End Sub

Declaring Global Variables

Create a user data type named RECT to hold the coordinates for a rectangle by inserting the following code at the beginning of your main frame.

Private Type RECT
    Left As Long
    Top As Long
    Right As Long
    Bottom As Long
End Type

Now, insert three global variables.

Private DisplayBuffer As ImageBuffer
Private UserROI As RECT
Private UserROICommited As Boolean
Private Sink As FrameHandlerSink
  • DisplayBuffer will contain a reference to the buffer that will be displayed on the ImageAvailable event. When deciding whether displaying a new buffer is necessary, the new buffer is compared to DisplayBuffer.
  • UserROI will contain the region of interest the user has set with the mouse.
  • UserROICommited will be used to toggle between drawing and testing.

Adding the Mouse Event Procedures

Add event procedures for ICImagingControl's MouseDown, MouseUp, and MouseMove events. Insert the following code into the event procedures:

MouseDown:

If Not UserROICommited And (Button And vbLeftButton) Then
    UserROI.Left = XPos
    UserROI.Top = YPos
End If

MouseUp:

If Not UserROICommited And Not (Button And vbLeftButton) Then
    UserROI.Right = XPos
    UserROI.Bottom = YPos
End If

MouseMove:

If Not UserROICommited And (Button And vbLeftButton) Then
    UserROI.Right = XPos
    UserROI.Bottom = YPos
End If

Because we use Y800 as sink color format, the mouse positions are identical to the pixel positions in the image buffers. Thus, we can set the UserROI.bottom and .top members to YPos.

When using a bottom-up sink color format such as RGB8, you had to transform the mouse positions to ICImagingControl1.Height - YPos.

Adding Functionality to the Application

As described in Displaying Buffers on the ImageAvailable Event, create two buttons ( Start / Stop ), the Form_Load event procedure and an ImageAvailable event handler. Add the line DisplayBuffer.ForceUnlock to the Stop_Click event handler. This is necessary as the DisplayBuffer will be locked in the ImageAvailable event procedure. When the control is stopped, the DisplayBuffer needs to be unlocked. Insert the following code into the ImageAvailable handler:

Private Sub ICImagingControl1_ImageAvailable(ByVal BufferIndex As Long)
    Dim Region As RECT
    Region = NormalizeRect(UserROI)

    ContinousMode Region
End Sub

This event handler uses the function NormalizeRect to make sure the rectangle's coordinates are not exchanged, e.g. left greater than right. It also calls the sub ContinousMode to paint that rectangle onto the image buffer. NormalizeRect is implemented as follows:

Private Function NormalizeRect(ByRef val As RECT) As RECT
    Dim Tmp As Long
    Dim r As RECT
    r = val
    If r.Top > r.Bottom Then
        Tmp = r.Top
        r.Top = r.Bottom
        r.Bottom = Tmp
    End If
    If r.Left > r.Right Then
        Tmp = r.Left
        r.Left = r.Right
        r.Right = Tmp
    End If
    If r.Top < 0 Then
        r.Top = 0
    End If
    If r.Left < 0 Then
        r.Left = 0
    End If
    If r.Bottom >= ICImagingControl1.ImageHeight Then
        r.Bottom = ICImagingControl1.ImageHeight - 1
    End If
    If r.Right >= ICImagingControl1.ImageWidth Then
        r.Right = ICImagingControl1.ImageWidth - 1
    End If
    NormalizeRect = r
End Function

The sub ContinousMode is implemented in the following way:

Private Sub ContinousMode(Region As RECT)
    Dim DisplayArr As Variant

    If Not DisplayBuffer Is Nothing Then DisplayBuffer.Unlock
    Set DisplayBuffer = Sink.ImageBuffers.Active
    DisplayBuffer.Lock

    DisplayArr = DisplayBuffer.GetImageData
    DrawRectangleY8 DisplayArr, Region
    DisplayBuffer.ReleaseImageData DisplayArr

    ICImagingControl1.DisplayImageBuffer DisplayBuffer
End Sub

First the DisplayBuffer is unlocked. It is generally good practice to release locks that are no longer required, otherwise the control can not write to that buffer. Then the image buffer specified by BufferIndex of the ImageBuffers collection is assigned to DisplayBuffer. This buffer is locked with DisplayBuffer.Lock to prevent DisplayBuffer from being overwritten by IC Imaging Control. Then the DrawRectangleY8 sub procedure draws a rectangle specified by Region into the image buffer. To display the buffer, ICImagingControl.DisplayImageBuffer is called with DisplayBuffer as its parameter. The buffer now contains a white rectangle that represents the current region of interest.

Implemented this way, IC Imaging Control enables the user to select a region by painting a rectangle on the control while a live video is running.

Inserting the "CompareMode"

Now, we add the functionality to only update the display, if something changes in the region of interest. Create a button, named cmdROICommit and add the following code to it's click event procedure:

Private Sub cmdROICommit_Click()
    If Not UserROICommited Then
        UserROICommited = True
        cmdROICommit.Caption = "Reset ROI"
    Else
        UserROICommited = False
        cmdROICommit.Caption = "Set current ROI"
    End If
End Sub

When this button is clicked, the region of interest is committed. The application then switches to the "CompareMode". In this mode, the display is only updated, if something within the region of interest has changed. When the "CompareMode" is already running, the button toggles to "ContinousMode".

Updating the Code in the Event Procedures

Add the the following code to the cmdStart_Click event procedure:

cmdROICommit.Enabled = True

This enables the painting of the ROI every time you start the live video. This is named "ContinousMode" in this example.

Add the following code to the cmdStop_Click event procedure:.

If UserROICommited Then
    cmdROICommit_Click  ' Change CompareMode to ContinousMode.
End If
cmdROICommit.Enabled = False

This will reset the mode from "CompareMode" to "ContinousMode" when the "Stop" button is clicked and the "CompareMode" is active. Further, add the line cmdROICommit.Enabled = False to the Form_Load event procedure.

Then alter the code in the ImageAvailable event procedure:

Dim Region As RECT
Region = NormalizeRect(UserROI)

If Not UserROICommited Then
    ContinousMode Region
Else
    CompareMode Region
End If

This calls the appropriate sub procedures, depending on UserROICommited.

The "CompareMode" Procedure

The new ImageAvailable event procedure calls the sub procedure CompareMode, which is implemented as follows:

Private Sub CompareMode(Region As RECT)
    Dim ArrOld, ArrNew As Variant
    Dim IBOld, IBNew As ImageBuffer
    Set IBOld = DisplayBuffer
    Set IBNew = Sink.ImageBuffers.Active

    ArrOld = IBOld.GetImageData
    ArrNew = IBNew.GetImageData

    If CompareRegion(ArrOld, ArrNew, Region, sldThresholdSlider.Value) Then

        DisplayBuffer.Unlock
        Set DisplayBuffer = IBNew
        DisplayBuffer.Lock

        DrawRectangleY8 ArrNew, Region

        ICImagingControl1.DisplayImageBuffer DisplayBuffer
    End If

    IBOld.ReleaseImageData ArrOld
    IBNew.ReleaseImageData ArrNew
End Sub

This sub calls the CompareRegion sub function, which compares the region of interest of the image buffers. If they differ, the old DisplayBuffer is unlocked and the new image buffer IBNew is assigned to DisplayBuffer. DisplayBuffer then contains the new image. The new DisplayBuffer is locked to prevent it from being overwritten by IC Imaging Control. The region of interest is drawn as a rectangle into the buffer, and the buffer is displayed, using ICImagingControl.DisplayImageBuffer.

The CompareRegion Function

The CompareRegion function compares the region of interest of the two image data arrays Arr and Arr2. The differences of the pixels in the specified Region are added. The result is saved in GreyscaleDifference. After adding, GreyscaleDifference is divided by the count of pixels in the region. GreyscaleDifference is compared to the threshold value passed to the function. If the threshold is lower or equal to GreyscaleDifference, the regions differ and the function returns True, False otherwise. The CompareRegion function is implemented as follows:

Private Function CompareRegion(Arr As Variant, Arr2 As Variant, Region As RECT, Threshold As Long) As Boolean
    Dim PixelCount As Long
    PixelCount = (Region.Bottom - Region.Top) * (Region.Right - Region.Left)

    If PixelCount > 0 Then

        Dim GreyscaleDifference As Long
        GreyscaleDifference = 0

        Dim x, y As Long
        For y = Region.Top To Region.Bottom
            For x = Region.Left To Region.Right
                GreyscaleDifference = GreyscaleDifference + Abs(Arr(x, y) - Arr2(x, y))
            Next x
        Next y

        GreyscaleDifference = GreyscaleDifference / PixelCount

        If GreyscaleDifference > Threshold Then
            CompareRegion = True
        Else
            CompareRegion = False
        End If
    Else
        CompareRegion = False
    End If

End Function

<< Programmer's Guide

Get in touch with us


About The Imaging Source

Established in 1990, The Imaging Source is one of the leading manufacturers of industrial cameras, frame grabbers and video converters for production automation, quality assurance, logistics, medicine, science and security.

Our comprehensive range of cameras with USB 3.1, USB 3.0, USB 2.0, GigE interfaces and other innovative machine vision products are renowned for their high quality and ability to meet the performance requirements of demanding applications.

Automated Imaging Association ISO 9001:2015 certified

Contact us