This chapter illustrates how to perform sophisticated image processing.
The source code for this sample program can be found in the samples\VB6\Advanced Image Processing directory. Before looking at this chapter, we suggest that you first read chapter Displaying Buffers on the ImageAvailable Event.
This chapter illustrates how to:
The sample described in this chapter lets you draw a rectangle on the live image using the mouse. The region specified by the rectangle is used to check whether the live image has changed. If so, the changed image is displayed. The threshold for the difference between the images can be set by the user.
Create a new project and add IC Imaging Control to the form. Before you run the program, select the video device, input and video format as shown in the First Steps in Visual Basic 6 chapter. Alternatively, run the program without selecting a device. In this case, the program shows the device selection dialog provided by IC Imaging Control. If you close this dialog without making a selection, the displays error message and terminates.
Add two buttons to the form and label them Device and Settings. Name them cmdDevice and cmdSettings respectively. If you click the "Device" button, a device selection dialog is displayed. After you have selected a valid device, setupDevice is called. This is a helper procedure, which sets up a FrameHandlerSink that is used to grab images. The sink is initialized with a ring buffer size of 5 and Y800 as color format. ICImagingControl.LiveDisplay is disabled, because the buffers are drawn by hand in this sample. The Settings button shows a dialog that allows you to adjust the VCDProperties of the currently selected device. The code for both buttons looks as follows:
Private Sub cmdDevice_Click() With ICImagingControl1 If .DeviceValid Then If .LiveVideoRunning Then .LiveStop End If End If .ShowDeviceSettingsDialog If .DeviceValid Then cmdStart.Enabled = True cmdSettings.Enabled = True setupDevice End If End With End Sub
Private Sub cmdSettings_Click() ICImagingControl1.ShowPropertyDialog End Sub
Create a user data type named RECT to hold the coordinates for a rectangle by inserting the following code at the beginning of your main frame.
Private Type RECT Left As Long Top As Long Right As Long Bottom As Long End Type
Now, insert three global variables.
Private DisplayBuffer As ImageBuffer Private UserROI As RECT Private UserROICommited As Boolean Private Sink As FrameHandlerSink
If Not UserROICommited And (Button And vbLeftButton) Then UserROI.Left = XPos UserROI.Top = YPos End If
If Not UserROICommited And Not (Button And vbLeftButton) Then UserROI.Right = XPos UserROI.Bottom = YPos End If
If Not UserROICommited And (Button And vbLeftButton) Then UserROI.Right = XPos UserROI.Bottom = YPos End If
Because we use Y800 as sink color format, the mouse positions are identical to the pixel positions in the image buffers. Thus, we can set the UserROI.bottom and .top members to YPos.
When using a bottom-up sink color format such as RGB8, you had to transform the mouse positions to ICImagingControl1.Height - YPos.
As described in Displaying Buffers on the ImageAvailable Event, create two buttons ( Start / Stop ), the Form_Load event procedure and an ImageAvailable event handler. Add the line DisplayBuffer.ForceUnlock to the Stop_Click event handler. This is necessary as the DisplayBuffer will be locked in the ImageAvailable event procedure. When the control is stopped, the DisplayBuffer needs to be unlocked. Insert the following code into the ImageAvailable handler:
Private Sub ICImagingControl1_ImageAvailable(ByVal BufferIndex As Long) Dim Region As RECT Region = NormalizeRect(UserROI) ContinousMode Region End Sub
This event handler uses the function NormalizeRect to make sure the rectangle's coordinates are not exchanged, e.g. left greater than right. It also calls the sub ContinousMode to paint that rectangle onto the image buffer. NormalizeRect is implemented as follows:
Private Function NormalizeRect(ByRef val As RECT) As RECT Dim Tmp As Long Dim r As RECT r = val If r.Top > r.Bottom Then Tmp = r.Top r.Top = r.Bottom r.Bottom = Tmp End If If r.Left > r.Right Then Tmp = r.Left r.Left = r.Right r.Right = Tmp End If If r.Top < 0 Then r.Top = 0 End If If r.Left < 0 Then r.Left = 0 End If If r.Bottom >= ICImagingControl1.ImageHeight Then r.Bottom = ICImagingControl1.ImageHeight - 1 End If If r.Right >= ICImagingControl1.ImageWidth Then r.Right = ICImagingControl1.ImageWidth - 1 End If NormalizeRect = r End Function
The sub ContinousMode is implemented in the following way:
Private Sub ContinousMode(Region As RECT) Dim DisplayArr As Variant If Not DisplayBuffer Is Nothing Then DisplayBuffer.Unlock Set DisplayBuffer = Sink.ImageBuffers.Active DisplayBuffer.Lock DisplayArr = DisplayBuffer.GetImageData DrawRectangleY8 DisplayArr, Region DisplayBuffer.ReleaseImageData DisplayArr ICImagingControl1.DisplayImageBuffer DisplayBuffer End Sub
First the DisplayBuffer is unlocked. It is generally good practice to release locks that are no longer required, otherwise the control can not write to that buffer. Then the image buffer specified by BufferIndex of the ImageBuffers collection is assigned to DisplayBuffer. This buffer is locked with DisplayBuffer.Lock to prevent DisplayBuffer from being overwritten by IC Imaging Control. Then the DrawRectangleY8 sub procedure draws a rectangle specified by Region into the image buffer. To display the buffer, ICImagingControl.DisplayImageBuffer is called with DisplayBuffer as its parameter. The buffer now contains a white rectangle that represents the current region of interest.
Implemented this way, IC Imaging Control enables the user to select a region by painting a rectangle on the control while a live video is running.
Now, we add the functionality to only update the display, if something changes in the region of interest. Create a button, named cmdROICommit and add the following code to it's click event procedure:
Private Sub cmdROICommit_Click() If Not UserROICommited Then UserROICommited = True cmdROICommit.Caption = "Reset ROI" Else UserROICommited = False cmdROICommit.Caption = "Set current ROI" End If End Sub
When this button is clicked, the region of interest is committed. The application then switches to the "CompareMode". In this mode, the display is only updated, if something within the region of interest has changed. When the "CompareMode" is already running, the button toggles to "ContinousMode".
Add the the following code to the cmdStart_Click event procedure:
cmdROICommit.Enabled = True
This enables the painting of the ROI every time you start the live video. This is named "ContinousMode" in this example.
Add the following code to the cmdStop_Click event procedure:.
If UserROICommited Then cmdROICommit_Click ' Change CompareMode to ContinousMode. End If cmdROICommit.Enabled = False
This will reset the mode from "CompareMode" to "ContinousMode" when the "Stop" button is clicked and the "CompareMode" is active. Further, add the line cmdROICommit.Enabled = False to the Form_Load event procedure.
Then alter the code in the ImageAvailable event procedure:
Dim Region As RECT Region = NormalizeRect(UserROI) If Not UserROICommited Then ContinousMode Region Else CompareMode Region End If
This calls the appropriate sub procedures, depending on UserROICommited.
The new ImageAvailable event procedure calls the sub procedure CompareMode, which is implemented as follows:
Private Sub CompareMode(Region As RECT) Dim ArrOld, ArrNew As Variant Dim IBOld, IBNew As ImageBuffer Set IBOld = DisplayBuffer Set IBNew = Sink.ImageBuffers.Active ArrOld = IBOld.GetImageData ArrNew = IBNew.GetImageData If CompareRegion(ArrOld, ArrNew, Region, sldThresholdSlider.Value) Then DisplayBuffer.Unlock Set DisplayBuffer = IBNew DisplayBuffer.Lock DrawRectangleY8 ArrNew, Region ICImagingControl1.DisplayImageBuffer DisplayBuffer End If IBOld.ReleaseImageData ArrOld IBNew.ReleaseImageData ArrNew End Sub
This sub calls the CompareRegion sub function, which compares the region of interest of the image buffers. If they differ, the old DisplayBuffer is unlocked and the new image buffer IBNew is assigned to DisplayBuffer. DisplayBuffer then contains the new image. The new DisplayBuffer is locked to prevent it from being overwritten by IC Imaging Control. The region of interest is drawn as a rectangle into the buffer, and the buffer is displayed, using ICImagingControl.DisplayImageBuffer.
The CompareRegion function compares the region of interest of the two image data arrays Arr and Arr2. The differences of the pixels in the specified Region are added. The result is saved in GreyscaleDifference. After adding, GreyscaleDifference is divided by the count of pixels in the region. GreyscaleDifference is compared to the threshold value passed to the function. If the threshold is lower or equal to GreyscaleDifference, the regions differ and the function returns True, False otherwise. The CompareRegion function is implemented as follows:
Private Function CompareRegion(Arr As Variant, Arr2 As Variant, Region As RECT, Threshold As Long) As Boolean Dim PixelCount As Long PixelCount = (Region.Bottom - Region.Top) * (Region.Right - Region.Left) If PixelCount > 0 Then Dim GreyscaleDifference As Long GreyscaleDifference = 0 Dim x, y As Long For y = Region.Top To Region.Bottom For x = Region.Left To Region.Right GreyscaleDifference = GreyscaleDifference + Abs(Arr(x, y) - Arr2(x, y)) Next x Next y GreyscaleDifference = GreyscaleDifference / PixelCount If GreyscaleDifference > Threshold Then CompareRegion = True Else CompareRegion = False End If Else CompareRegion = False End If End Function
Established in 1990, The Imaging Source is one of the leading manufacturers of industrial cameras, frame grabbers and video converters for production automation, quality assurance, logistics, medicine, science and security.
Our comprehensive range of cameras with USB 3.1, USB 3.0, USB 2.0, GigE interfaces and other innovative machine vision products are renowned for their high quality and ability to meet the performance requirements of demanding applications.