This chapter shows you how to write text and draw graphics on the live video images.
The source code for this sample program can be found in the samples\VB6\Creating an Overlay directory.
First of all, create a new project and add IC Imaging Control to the form as shown in the FirstSteps chapter.
Then, insert three new buttons to your form and name them cmdSelectDevice, cmdLiveVideo and cmdImageSetting. Label the buttons Device, Start and Settings respectively. These buttons will be used to select the video capture device, start and stop the live video and open the image settings dialog.
Now, implement the handler subs for the three buttons:
Private Sub cmdSelectDevice_Click() cmdLiveVideo.enabled = False cmdLiveVideo.Caption = "Start" cmdImageSettings.enabled = False ' Stop the live video, if it runs. If ICImagingControl1.DeviceValid = True Then If ICImagingControl1.LiveVideoRunning Then 'Stop the timer. Timer1.enabled = False ICImagingControl1.LiveStop cmdLiveVideo.Caption = "Start" End If End If ' Now, call the device settings dialog to select the device. ICImagingControl1.ShowDeviceSettingsDialog If ICImagingControl1.DeviceValid = True Then cmdLiveVideo.enabled = True cmdImageSettings.enabled = True End If End Sub
Private Sub cmdLiveVideo_Click() If ICImagingControl1.DeviceValid Then MousePointer = vbHourglass If ICImagingControl1.LiveVideoRunning Then Timer1.enabled = False ICImagingControl1.LiveStop cmdLiveVideo.Caption = "Start" Else StartLiveVideo cmdLiveVideo.Caption = "Stop" End If MousePointer = vbNormal End If End Sub
Private Sub cmdImageSettings_Click() If ICImagingControl1.DeviceValid Then ICImagingControl1.ShowPropertyDialog End If End Sub
Also, implement a sub named StartLiveVideo. It is called by cmdLiveVideo_Click and will be used later to do the initial drawings in the overlay.
Run the program: You should be able to select a video capture device and see the live video.
IC Imaging Control provides a special object for drawing and writing on the live video. The object is called OverlayBitmap.
The OverlayBitmap object contains a bitmap, which has the same size as the video format, or the size of a frame filter that transforms the image data before the overlay bitmap is applied. An application can draw text and graphics on this overlay bitmap. The overlay bitmap is copied to each sample of the video. A dropout color can be specified to determine, which pixels of the overlay bitmap are transparent and therefore not copied to the live video.
The overlay bitmap can only be used, if ICImagingControl.LiveStart has been called at least once after the video format has been changed. It is only visible, if the live video is running.
Since version 3.0, IC Imaging Control allows you to select whether the overlay is drawn on live video, on grabbed and recorded images, or both.
Enabling the overlay bitmap is a two-step process: The OverlayBitmapPosition has to be set before starting live mode and determines, at which path positions overlay operations are possible. OverlayBitmap.Enable enables or disables the application of a specific overlay bitmap while live mode is running.
A program can draw on any of the overlay bitmaps once live mode has been started. Live mode has to be started first, because the actual size and color format of the overlay bitmaps can depend on frame filters, thus it is impossible to determine the formats before.
The StartLiveVideo starts live video and then draws several graphics on the 3 overlay bitmaps:
Private Sub StartLiveVideo() Dim ob As OverlayBitmap FrameCount = 0 If ICImagingControl1.DeviceValid = True Then ICImagingControl1.LiveStart ' Initialize overlay bitmaps SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE) SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY) SetupOverlay ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_SINK) ' Display a coordinate system on the device overlay DrawCoordinatesystem ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE) ' Load a bitmap file and display it on the display overlay ShowBitmap ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY) ' Draw overlay info boxes DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE) DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DISPLAY) DrawOverlayInfo ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_SINK) ' Enable the timer. In the timer event handler sub, the current ' time is displayed. Timer1.enabled = True Timer1_Timer End If End Sub
The SetupOverlay sub is used to initialize each of the 3 overlays. It is called after live mode has been started.
Private Sub SetupOverlay(ob As OverlayBitmap) ' Enable the overlay bitmap for drawing. ob.Enable = True ' Set magenta as dropout color. ob.DropOutColor = RGB(255, 0, 255) ' Fill the overlay bitmap with the dropout color. ob.Fill ob.DropOutColor ' Print text in red. ob.FontTransparent = True ob.DrawText RGB(255, 0, 0), 10, 10, "IC Imaging Control 3.0" End Sub
The overlay is enabled by calling OverlayBitmap.Enable, and a text is displayed in the upper left hand corner of the bitmap. The sub also sets a dropout color.
The OverlayBitmap.DropOutColor property of the OverlayBitmap object determines, which pixel color is transparent on the live video. Each pixel that has the dropout color is not copied to the live video. The dropout color can be set with Visual Basic's RGB function. It defaults to black. For this reason, the background of the text written above is transparent. The FontBackColor property of the OverlayBitmap object is set to black as default too.
In order to be able to use black color in the overlay, we change OverlayBitmap.DropOutColor to another color, for example magenta (RGB(255,0,255)). If the application is started, only a black image with our text can be seen. The image is black, because the overlay bitmap is black and the dropout color has been changed to magenta. Therefore, the overlay bitmap must be filled with magenta. The method OverlayBitmap.Fill does this for us. When the dropout color is changed, the text background is visible as a solid black rectangle. To hide this, the property OverlayBitmap.FontTransparent can be set to True.
In the next step, we are going to draw a coordinate plane, consisting of lines and labels. To do this, we are going to use the DrawLine and DrawText methods of the OverlayBitmap object. Additionally, we are going to change the font used to write the labels.
We are going to implement a new sub called DrawCoordinatesystem. We want to use an 8 dpi Arial font for the labels. To achieve this, a new StdFont object Font must be created. This object has font properties like font name, font size and other font attributes. Before changing the font, we save the current font in order to be able to restore it after we have completed the coordinate plane.
The complete sub that draws the coordinate plane is implemented as follows:
Private Sub DrawCoordinatesystem(ob As OverlayBitmap) Dim Col As Integer Dim Row As Integer Dim i As Integer Dim Font As New StdFont Dim OldFont As StdFont Font.Name = "Arial" Font.Size = 8 With ob ' Calculate the center of the video image. Col = ICImagingControl1.Width / 2 Row = ICImagingControl1.Height / 2 Set OldFont = .Font .Font = Font .DrawLine RGB(255, 0, 0), Col, 0, Col, ICImagingControl1.Height .DrawLine RGB(255, 0, 0), 0, Row, ICImagingControl1.Width, Row For i = 0 To Row Step 20 .DrawLine RGB(255, 0, 0), Col - 5, Row - i, Col + 5, Row - i .DrawLine RGB(255, 0, 0), Col - 5, Row + i, Col + 5, Row + i If i > 0 Then .DrawText RGB(255, 0, 0), Col + 10, Row - i - 7, i / 10 .DrawText RGB(255, 0, 0), Col + 10, Row + i - 7, -i / 10 End If Next For i = 0 To Col Step 20 .DrawLine RGB(255, 0, 0), Col - i, Row - 5, Col - i, Row + 5 .DrawLine RGB(255, 0, 0), Col + i, Row - 5, Col + i, Row + 5 If i > 0 Then .DrawText RGB(255, 0, 0), Col + i - 5, Row - 17, i / 10 .DrawText RGB(255, 0, 0), Col - i - 10, Row - 17, -i / 10 End If Next .Font = OldFont End With End Sub
To display the current time on the live video, a timer is inserted in the form. It is called timer1 by default and its timer interval is set to 1000 milliseconds in the property window. The timer event handler sub is called Timer1_timer. The current time is displayed in the lower right hand corner. It has a white font with a black background. Furthermore, it has its own font. Therefore, a StdFont object must be created and used, as described above. The property OverlayBitmap.FontTransparent is set to False and OverlayBitmap.FontBackColor is set to black (RGB 0 0 0).
Private Sub Timer1_Timer() If ICImagingControl1.LiveVideoRunning = True Then Set ob = ICImagingControl1.OverlayBitmapAtPath(PATHPOSITION_DEVICE) Dim OldFont As StdFont Set OldFont = ob.Font Dim Font As New StdFont Font.Name = "Arial" Font.Bold = True Font.Size = 14 ob.Font = Font ' Draw the time in the lower left corner of the video window. Dim Col As Integer Dim Row As Integer Col = ICImagingControl1.Width - 81 Row = ICImagingControl1.Height - 20 ' Setup the background color and drawing mode. ob.FontTransparent = False ob.FontBackColor = RGB(0, 0, 0) ' Draw the text with white color. ob.DrawText RGB(255, 250, 250), Col, Row, Time ' Reset the font and drawing mode. ob.FontTransparent = True ob.Font = OldFont End If End Sub
IC Imaging Control provides an OverlayUpdate event. This event is called after a new frame has been delivered from the video capture device. Therefore, it is possible to insert individual information on to each frame (i.e. frame counter).
The OverlayUpdate sub in the example counts the frames in the variable FrameCount. The frame counter is used to draw a growing triangle. If the frame counter is greater than 25, it is reset to 0 and the triangle is deleted. Therefore, if a solid rectangle of the same size is drawn over the triangle. The color of the rectangle must be set to the DropOutColor to make it transparent. The method DrawSolidRect provides this functionality.
The OverlayUpdate event handler is implemented as follows:
Private Sub ICImagingControl1_OverlayUpdate(ByVal Overlay As ICImagingControl3Ctl.OverlayBitmap, ByVal SampleStartTime As Double, ByVal SampleEndTime As Double) Dim Font As New StdFont Dim OldFont As StdFont Font.Name = "Arial" Font.Size = 8 With Overlay If Overlay.PathPosition = PATHPOSITION_DISPLAY Then FrameCount = FrameCount + 1 If FrameCount > 25 Then FrameCount = 0 ' Delete the triangle. .DrawSolidRect .DropOutColor, 10, _ ICImagingControl1.Height - 70, _ 62, _ ICImagingControl1.Height - 9 End If ' Draw the new triangle line. .DrawLine RGB(200, 255, 0), FrameCount * 2 + 10, _ ICImagingControl1.Height - 10, _ FrameCount * 2 + 10, _ ICImagingControl1.Height - 10 - FrameCount ' Print the current frame number: ' Set the background color to the current dropout color ' and make the font opaque. .FontBackColor = .DropOutColor .FontTransparent = False ' Save the current font. Set OldFont = .Font ' Set the new created font. .Font = Font ' Draw the text. .DrawText RGB(200, 255, 0), 70, ICImagingControl1.Height - 19, Str(FrameCount) + " " ' Restore the previously used font. .Font = OldFont End If End With End Sub
The OverlayBitmap object does not provide all functionality of the graphic device interface (GDI). Therefore, the OverlayBitmap provides a method that returns a device context. The method is called GetDC. The bitmap of the OverlayBitmap object is already selected in the returned device context. It can be used for all GDI functions like BitBlt, Ellipse etc.
To use the GDI functions, they must be declared on top of the form's source code:
' The GDI (graphics device interface) functions ' must be declared before they can be used. Private Declare Function CreateCompatibleDC Lib "gdi32.dll" _ (ByVal handle As Long) As Long Private Declare Function DeleteDC Lib "gdi32.dll" _ (ByVal handle As Long) As Long Private Declare Function SelectObject Lib "gdi32.dll" _ (ByVal DC As Long, ByVal HGDIObj As Long) As Long Private Declare Function BitBlt Lib "gdi32.dll" ( _ ByVal hdcDest As Long, _ ByVal nXDest As Long, _ ByVal nYDest As Long, _ ByVal nWidth As Long, _ ByVal nHeight As Long, _ ByVal hdcSrc As Long, _ ByVal nXSrc As Long, _ ByVal nYSrc As Long, _ ByVal dwRop As Long _ ) As Boolean
A new sub ShowBitmap is implemented to demonstrate how to render a bitmap onto the overlay bitmap. It uses the GDI function BitBlt to copy one bitmap onto another. The sub contains a variable of Visual Basic's StdPicture type. It is used to load a bitmap from file. Also two long variables are declared, that receive the device contexts. BitBlt copies the bitmap data from one device context to another device context. Very important: The bitmap sizes of the StdPicture object must be converted in to pixels, because the overlay bitmap uses pixels as its units.
After the bitmap has been loaded and its size has been converted to pixels, the device context of the OverlayBitmap object is retrieved:
Set ob = ICImagingControl1.OverlayBitmap obDC = ob.GetDC
Once a valid device context has been retrieved, a second, compatible device context must be created. This is done by a call to the GDI function CreateCompatibleDC. With the GDI function SelectObject, the bitmap in the StdPicture object is selected in the new device context. As soon as these steps have been completed, the BitBlt function is called. The target coordinates are so calculated that the loaded bitmap appears in the upper right hand corner of the video image.
' ' ShowBitmap ' ' This sub demonstrates how to use OverlayBitmap.GetDC to blit a bitmap ' from a file on the live video. ' The bitmap will be blitted with transparency on the live video because ' it's background color is magenta (load the image "Hardware.png" ' with "Paint.exe" to verify this). Magenta is the currently set ' dropout color. The used GDI graphic functions are based on pixel units. ' Therefore, the scaling functions of Visual Basic are called to get the ' size of the loaded bitmap in pixels. ' Private Sub ShowBitmap(ob As OverlayBitmap) Dim Pic As New StdPicture Dim PicWidth As Integer Dim PicHeight As Integer Dim obDC As Long Dim SourceDC As Long Dim Col As Integer ' Load a BMP file. Set Pic = LoadPicture(App.Path & "\hardware.png") ' Transform the size of the picture from himetric to pixel units. PicWidth = ScaleX(Pic.Width, vbHimetric, vbPixels) PicHeight = ScaleX(Pic.Height, vbHimetric, vbPixels) ' Calculate a column to display the bitmap in the ' upper right corner of Imaging Control. Col = ICImagingControl1.Width - 5 - PicWidth ' Get the DC of the OverlayBitmap object. obDC = ob.GetDC If obDC <> 0 Then ' Create a compatible DC that is used to hold the loaded bitmap. SourceDC = CreateCompatibleDC(obDC) If SourceDC <> 0 Then ' Select the loaded bitmap in the source DC. SelectObject SourceDC, Pic.handle ' Now blit the source DC in the overlay bitmap DC. ' This copies the loaded bitmap to the overlay bitmap. BitBlt obDC, Col, 5, PicWidth, PicHeight, _ SourceDC, 0, 0, 13369376 ' 13369376 is SRCCOPY ' Delete the DC source that is no longer needed to avoid handle leaks. DeleteDC SourceDC End If ' Release the DC. ob.ReleaseDC obDC End If End Sub
Do not forget to delete the newly created device context by a call to the GDI function DeleteDC and to release the OverlayBitmap device context by a call to the method ReleaseDC of the OverlayBitmap object.
The end user of our example program can control which of the 3 overlay bitmaps are drawn on the live video. To change the PathPositions, the following helper sub is used:
Private Sub SetOverlayPathPosition(pos As PathPositions, enabled As Boolean) ' Remember whether live video has to be restarted Dim MustRestart As Boolean MustRestart = ICImagingControl1.LiveVideoRunning If MustRestart Then ICImagingControl1.LiveStop ' Adjust the overlay bitmap path position Dim newPP As PathPositions If enabled Then newPP = ICImagingControl1.OverlayBitmapPosition Or pos Else newPP = ICImagingControl1.OverlayBitmapPosition And Not pos End If ICImagingControl1.OverlayBitmapPosition = newPP ' Restart live video If MustRestart Then StartLiveVideo End Sub
This sub enables or disables the overlay at a specified path position.
An application can decide whether it wants a color or grayscale overlay. The OverlayBitmap.ColorMode property can be used to select one of the available color modes:
When the format of the image data stream does not match the overlay bitmap color mode, the image data is automatically converted.
Established in 1990, The Imaging Source is one of the leading manufacturers of industrial cameras, frame grabbers and video converters for production automation, quality assurance, logistics, medicine, science and security.
Our comprehensive range of cameras with USB 3.1, USB 3.0, USB 2.0, GigE interfaces and other innovative machine vision products are renowned for their high quality and ability to meet the performance requirements of demanding applications.