Overview

You can specify an image as the target UI using the features provided by the ImageRecognition library.
If a user captures an image, a library action is performed by recognizing the same UI factor as this image. You can perform actions, such as clicking a specific point in the image or converting the characters inside an image into text.
Also, using the Optical Character Recognition (OCR) server, you can convert the text and tables on images into text strings and array data. The Optical Character Recognition  features require the OCR server for operation.
Therefore, they can be used only when you are logged in to the Designer account.

The features provided by the ImageRecognition library are as follows:

Mouse action features: ImageClick, ImageHover, ImageRightClick, ImageDoubleClick
Image recognition features: IsExist, MatchImages, WaitImageAppear, WaitImageDisappear, GetImageBounds
Image capturing features: CaptureBounds, CaptureScreen, CaptureWindow, CaptureFromFile
Features related to image color: GetImageColor
Image text input features: ImageTextInput
Image using Lable Information: GetImageLabel, GetImageLabelOnFile

Some activity cards support the ability to select a target to use as the final result when there are multiple image-based search results.

Supports multi-image result index navigation : GetImageBounds, ImageClick, ImageRightClick, ImageHover, ImageDoubleClick, IsExist, WaitImageAppear, ImageTextInput, etc.

Use Designer or File Explorer to refer to the samples for the ImageRecognition library.

• On Designer: Menu > Help > Sample > Sample > ImageRecognition

• On File Explorer: C:\Users\user\AppData\Roaming\Brity RPA Designer\samples\ImageRcognition

[ImageRcognition sample project names and related activity cards]

• Capture Screen: CaptureBounds, CaptureFromFile, CaptureScreen, CaptureWindow

• Mouse Action: ImageClick, ImageDoubleClick, ImageHover, ImageRightClick

• Wait Image Target: WaitImageAppear, WaitImageDisappear

• Other Samples: GetImageBounds, GetImageColor, ImageTextInput, IsExist, MatchImages

• ImageLabel: GetImageLabel, GetImageLabelOnFile

Common Properties

IMAGE

Shows the information about the captured image (size and position).

Card properties
Properties	Type	Required	Unit	Auto-setting	Description
Image	Image	Y	-	Y	View captured images. If you double-click the image, you can change the image settings using ImageEditor.
Bounds	Number	N	-	Y	Shows the specified position and size of the target UI object as relative coordinates based on the screen. (Unit: Pixel) X: Horizontal, Y: Vertical, W: Width, H: Height Height
Image Index Method	Combo Box	N	-	N	Set how to sort the results when there are multiple results of image-based object search. - JustBest : Use only one with the highest comparison score- RowFirst : Sort by row-first order on the screen (Left->Right)- ColumnFirst : Sort by column priority order (top->bottom) - ScoreOrderRow : Sorts in order of highest comparison score , and in case of a tie, in the order of row priority (left->right). - ScoreOrderColumn : Sorts in the order of the highest comparison score, and in the case of a tie, the column priority (top->bottom) order.
Select Image Index	Number	N	-	N	When there are multiple results of image-based object search, set the index order of the last result to be used in the sorted list. It is an integer value starting from 0. Values outside the result count range are automatically converted and operated according to the result count.

The Image Index Method property and the Select Image Index property are only available for 8 cards that support multi-image result index navigation. See [Overview]

To set the image shape of the Image property for image-based navigation, double-click the picture part of the Image property to open the image property detail view window.

Image recognition Lib2

In the image detail view window, you can reduce the image comparison area candidate size to be used for actual image comparison. You can also specify the location to click with the mouse. While reducing the image comparison area candidate size, you can change the mouse click position with Apply after confirming the resizing with Crop.

For accurate image-based comparisons, set the image area to be compared as clearly as possible.

MORE OPTIONS

Card properties
Properties	Type	Required	Unit	Auto-setting	Description
Timeout(Sec)	Number	Y	sec	Y	Enter the search time for the target object in seconds. If the target object can be searched within the set time, “True” will be returned. If the target object cannot be searched within the set time, “False” will be returned.
Object Path	Toggle button	Y	-	Y	The Windows path information of the target object is automatically populated. Use this property to search for and identify an object with the path information. For more information about the Windows path information, see Tool > Inspector in the Designer menu. If “@” is entered, it will be regarded as an Xpath syntax.
SupportedPatterns	-	N	-	Y	The automation pattern available for the target object is automatically entered.
Focus Before	Toggle button	N	-	Y	You can set the focus on the target object before the activity card is operated. If the focus cannot be set on the target object, the focus is set to the main screen of the target app. If there are multiple target apps running, the app at the front is scanned first. If the target object cannot be found, the app in the back is brought to the front for a rescan.
Draw Bounds	Toggle button	N	-	Y	Set to display a box (border) to identify the object.
Raw	Toggle button	N	-	Y	Use a low level keyboard event to determine whether to enter a string. (You can enter only English, Korean, or special characters that can be typed with the keyboard.)
On Error	Combination box	N	-	N	Select the processing method for occurred errors. - --Ignore--: Ignore the error and proceed. - --Retry--: Try the task one more time. - --GoTo--: Try the scenario for the specified time if the activity fails. - --Event--: Select an event created within the project.

DESCRIPTION

Card properties
Properties	Type	Required	Unit	Auto-setting	Description
DESCRIPTION	Text	N	-	N	Enter the description for the activity card. The description entered in the DESCRIPTION field is used as the description of the activity. A representative value will be displayed in the absence of the DESCRIPTION input.

ImageClick

Overview

This activity card clicks a specific point on an image.

Application procedures

Double-click No Target on the activity card.
Specify the range of the target image to click.
On the range of the image, click the position for the mouse click.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Click Point	Text	N	-	Y	Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Opens the Amazon website (www.amazon.com) with Chrome, waits for the Amazon logo to be displayed, and verify if the "Returns & Orders” menu image exists. Click the menu if the menu exists. Otherwise, click the “Returns & Orders” menu image. Then, click the Amazon logo and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageClick_Test

Used variables

openBrowser = Variable to specify the launched browser
imgWait = Variable that stores the verification result of whether the Amazon logo and image exist
imgExist = Variable that stores the verification result of whether the "Returns & Orders" menu image exist

Used properties

Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True

Related/Similar activities

ImageHover, ImageRightClick, ImageDoubleClick

ImageTextInput

Overview

This activity card identifies a specified image and enters text. (TextInput feature)

Application procedures

Double-click No Target on the activity card.
Specify the range of the target image to click.
On the specified boundary of the image, click the position for the mouse click.
In the Input Text field, enter the text to enter in the specified area.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Input Text	Other	Y	-	N	Enter the string to input within single quotation marks.
N	Click Point	Text	N	-	Y	Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed.
N	Clear Before	Toggle button	N	-	Y	Select whether to delete all the text in text field before entering text. (Default: On)
N	Done(Enter)	Toggle button	N	-	Y	Set whether to press the Enter key after entering the text. (Default: False)
N	[OPTION] Before Delay(ms)	Text	N	Milliseconds	Y	Specify the delay time before the text input. (Default: 0)
N	[OPTION] After Delay(ms)	Text	N	Milliseconds	Y	Specify the delay time after the text input. (Default: 0)
N	[OPTION] Interval(ms)	Text	N	Milliseconds	Y	Set the interval between character inputs. (Default: 0)
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome, wait for three seconds until the Google logo is displayed. Then, enter “RPA” at the specified position, wait for three seconds, and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_ImageTextInput_Test

Used variables

Browser = Variable to specify the launched browser

Used properties

Input Text = 'RPA'
Done(Enter) = On

ImageHover

Overview

This activity card hovers the mouse pointer over a specific point on an image.

Application procedures

Double-click No Target on the activity card.
Specify the range of the target object over which to hover the mouse pointer.
On the specified range of the image, click the position to place the mouse pointer.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Click Point	Text	N	-	Y	Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed.
N	ActionDelay(ms)	Text	N	-	Y	Enter the amount of time to wait after finding the image before performing the actual action.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Amazon website (www.amazon.com) with Chrome and wait for one second. Place the mouse pointer over the three different menu images, wait for one second, then close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageHover_Test

Used variables

browser1 = Variable to specify the launched browser

Used properties

Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True

Related/Similar activities

ImageClick, ImageRightClick, ImageDoubleClick

ImageRightClick

Overview

This activity card right-clicks a specific point on the image.

Application procedures

Double-click No Target on the activity card.
Specify the range of the target object to right-click.
On the specified range of the image, click the position to right-click.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Click Point	Text	N	-	Y	Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome and click the Gmail logo. Right-click the Gmail logo image and wait for two seconds. Then, press the “ESC” key to close the context menu and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageRightClick_Test

Used variables

browser1 = Variable to specify the launched browser

Used properties

Threshold = 96
Timeout(Sec) = 10
Draw Bounds = True

Related/Similar activities

ImageClick, ImageHover, ImageDoubleClick

ImageDoubleClick

Overview

This activity card double-clicks a specific point on an image.

Application procedures

Double-click No Target on the activity card.
Specify the range of the target image to double-click.
On the specified range of the image, click the position for the double-click.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Click Point	Text	N	-	Y	Enter coordinates to click. After specifying the range, the click coordinate will be automatically filled in. You can change the click coordinate if needed.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Samsung SDS Brity RPA Help website and wait until the menu image is displayed.
Then, double-click the menu image three times, wait for two seconds, and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Mouse Action > P_ImageDoubleClick_Test

Used variables

None

Used properties

Threshold = 98
Timeout(Sec) = 5

Related/Similar activities

ImageClick, ImageRightClick, ImageHover

GetImageBounds

Overview

This activity card searches for a specific image on the screen and fetches the coordinates data.

Application procedures

Double-click No Target on the activity card.
On the specified range of the image, click the position to fetch the coordinates data.
Specify the range of the image to fetch the coordinates data.
In the [Output] Output field, enter the variable to store the boundary data of the searched image.
In the Properties window, specify other required properties.

Card output properties

Property	Type	Additional comments	Description	Example
Output	Rect	Variable type in class X : double Y : double Width : doubleHeight : double	Returns the area information of the image searched for.	X: 774 Y: 575 Width: 52 Height: 19

Property

Type

Additional comments

Description

Example

Output

Rect

Variable type in class

X : double

Y : double

Width : doubleHeight : double

Returns the area information of the image searched for.

X: 774

Y: 575

Width: 52

Height: 19

Card input properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Search Bounds	Text	Y	-	Y	Enter the boundary data (x, y, width, height) on the screen to search for the image. The boundary data will be automatically filled in when you specify the boundary by clicking No Target.
N	Threshold	Number	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome and search for the specified image and fetches the coordinates data. Then, output the coordinate data in a message box and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_GetImageBounds_Test

Used variables

point = Variable to store the boundary data of the searched image

Used properties

[Output] Output = this.point
Threshold = 96

Remarks

By specifying a position while pressing down the "Alt" key and specifying the image boundary, you can specify the relative coordinates of the image from a specific position (offset feature).

Related/Similar activities

GetImageColor

Overview

This activity card fetches the color data of the specified coordinates of an image.

Application procedures

Double-click No Target on the activity card.
Specify the range of the image to fetch the color.
On the specified range of the image, click the position to fetch the color.
In the [Output] Output field, enter the variable to store the fetched color.
In the Properties window, specify other required properties.

Card output properties

Property	Type	Additional comments	Description	Example
Output	String	-	Returns the color value of the specified coordinate in the searched image in RGB format.	'232,232,233'

Card input properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Search Bounds	Text	Y	-	Y	Enter the boundary data (x, y, width, height) on the screen to search for the image. The boundary data will be automatically filled in when you specify the boundary by clicking “No Target.”
N	Offset	Number	N	-	Y	Enter the relative coordinates to retrieve the color from. The coordinates will be automatically populated when you select the image.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome, search for the specified image, and fetch the color at the specified coordinates. Then, output the color in a message box and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_GetImageColor_Test

Used variables

openBrowsser = Variable to specify the launched browser
getImageColor = Variable to store the color of the specified coordinates of the image
waitImage = Variable to store the verification result of whether the image exists

Used properties

[Output] Output = this.getImageColor
Search Bounds = '1606,155,52,60'

Related/Similar activities

GetImageBounds

IsExist

Overview

This activity card verifies whether a specific image exists on the screen.

Application procedures

Double-click No Target on the activity card.
Specify the range of the target image to double-click.
In the [Output] Output field, enter the variable to store the execution result.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	[Output] Output	Variable	Y	-	Y	Specify the variable to save the OCR operation result. (Default: RESULT) “True” is returned if the image is found on the screen within the time entered in the “Timeout” field. Otherwise, “False” is returned.
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (www.google.com) with Chrome. When the search menu icon (magnifier) is displayed, hover the mouse pointer over the Google logo. Wait for two seconds and close the Chrome browser if the Google image is displayed. Otherwise, open the Google website again with Chrome.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_IsExist_Image_Test

Used variables

isExist = Variable that stores the verification result of whether the search menu icon (magnifier) exists

Used properties

[Output] Output = this.isExit
Threshold = 98
Draw Bounds = True
On Error = Ignore

Related/Similar activities

WaitImageAppear, WaitImageDisappear

MatchImages

Overview

This activity card returns a filename by finding an image file that matches the image inside a specific folder within a specified range.

The size of the search area to find a matching target must be larger than the maximum size (width. height) of each image file in the folder.

Application procedures

Double-click No Target on the activity card.
Specify the range of the image to compare with the image in the specified folder.
In the [Output] Output field, enter the variable to store the searched file name.
In the SourceImagePath field, enter the path and name of the image file to compare.
In the Properties window, specify other required properties.

Card output properties

Property	Type	Additional comments	Description	Example
Output	String	-	Returns the path to an image file that matches an image within a specific folder.※ If there is no matching image, an empty string is returned.	'C:/IPA_Temp_Imgage/image_sample.png'

Card input properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
N	Search Bounds	Text	N	-	N	Specify the range of the screen to search for an image. Without additional input, the search will be performed for the entire screen.
N	SourceImagePath	Text	Y	-	N	Enter the complete path and file name of the image file to compare.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open one image file with the Chrome browser and compare it with the specified image. Output the comparison result, “Matching file: compared_file_name,” in a message box and close the Chrome browser.
Repeat the procedure to compare another image with the specified image. Then, output the “I found a similar file in the given folder (C: \\ IPA_Temp_Image). Press OK, and the C: \\ IPA_Temp_Image folder will be deleted. ” message in a message box and delete the copied files and directories.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Other Samples > P_MatchImage_Test

Used variables

matchImages = The name of the searched file through the comparison
bid = Variable to specify the launched browser

Used properties

[Output] Output = this.matchImages
Threshold = 96
SourceImagePath = 'C:\\IPA_Temp_Image'
Draw Bounds = True
On Error = Ignore

WaitImageAppear

Overview

This activity card waits until a specific image appears on the screen.

Application procedures

Double-click No Target on the activity card.
Specify the boundary for the image to search for.
In the [Output] Output field, enter the variable to store the search result of the image.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	[Output] Output	Variable	Y	-	Y	Specify the variable to store the search result of the image. (Default: RESULT) “True” is returned if the image is found on the screen within the time entered in the “Timeout” field. Otherwise, “False” is returned.
N	Search Bounds	Text	N	-	N	Specify the range of the target screen to search for the image. Without additional input, the search will be performed for the entire screen.
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome and wait until the Google logo appears. Then, output the result of the image appears in a message box and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Wait Image Target > P_WaitImageAppear_Test

Used variables

appear = Variable to store the result of the image appearance

Used properties

[Output] Output = this.appear
Threshold = 96

Related/Similar activities

IsExist, WaitImageDisappear

WaitImageDisappear

Overview

This activity card waits until a specific image disappears from the screen.

Application procedures

Double-click No Target on the activity card.
Specify the boundary for the image to search for.
In the [Output] Output field, enter the variable to save the result of disappearing images.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	[Output] Output	Variable	Y	-	Y	Specifies a variable to store the result of the image disappearing (default: RESULT). ※ Returns True if the image disappears from the screen within the time entered in the Timeout field, and False if it does not disappear.
N	Search Bounds	Text	N	-	N	Specify the range of the target screen to search for the image. Without additional input, the search will be performed for the entire screen.
N	Threshold	Variable	Y	-	Y	Specify the similarity of the image. (0~100) The matching probability becomes higher as it gets closer to 100. (Default: 96)
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome and wait until the Google logo appears.
Open the Amazon website (http://www.amazon.com) with a new Chrome browser and wait until the Amazon logo appears.
Then, wait until the Google logo disappears, output the result in a message box, and close all Chrome browsers.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Wait Image Target > P_WaitImageDisappear_Test

Used variables

browser1, browser2 = Variables that specify the launched browsers
appear = Variable that stores the result of the image appearance
disAppear = Variable that stores the result of the image disappearance

Used properties

Threshold = 96

Related/Similar activities

IsExist, WaitImageAppear

CaptureBounds

Overview

This activity card captures a specified area on a PC screen.

Application procedures

Ensure that you know the coordinates and size of the captured area, which can be acquired with the GetImageBounds activity card.

In the CaptureBounds field, enter the boundary data of the captured area in the form of “x, y, width, and height.”
Set the SaveClipboard property to select whether to store the captured area on the clipboard.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	CaptureBounds	Text	Y	-	Y	Enter the boundary to capture in the order of Left, Top, Width, and Height. You can also specify the boundary fetched by the GetBounds or GetImageBounds activity cards as the variable. Without additional input, the “RESULT” variable is selected by default, under the assumption that the information acquired from the previous process will be used.
N	Capture Path	Text	Conditional	-	N	Enter the download path and file name of the captured image. This property becomes available when you disable the SaveClipboard property.
N	SaveClipboard	Toggle button	Y	-	Y	Set whether you want to save the captured area on the clipboard or as a file. (Default: On) - On: Save the captured area on the clipboard. - Off: Save the captured area as a file.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome. Capture the area with coordinates “500, 200, 600, 200” (x, y, width, height), store it on the clipboard, and then close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureBounds_Test

Used variables

openBrowser = Variable to specify the launched browser
waitImage = Variable that stores the result of image appearance

Used properties

CaptureBounds = '500,200,600,200'
SaveClipboard = true

Remarks

The GetBounds and GetImageBounds activity cards are often used together to verify the coordinates and size of the captured area.
The captured image will be saved on the clipboard when the SaveClipboard property is turned on and saved as a file when it is turned off.

Related/Similar activities

CaptureScreen, CaptureWindow, CaptureFromFile

CaptureScreen

Overview

This activity card captures the entire screen of a local PC.

Application procedures

In the Capture Path field, specify the path and name for the captured image file of the local PC screen.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Capture Path	Text	Conditional	-	N	Enter the download path and file name of the captured image. (File format: png)
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Open the Google website (http://www.google.com) with Chrome.
Then, capture the entire PC screen, save it as “C:\\CaptureScreenTest.png,” and close the Chrome browser.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureScreen_Test

Used variables

openBrowser = Variable to specify the launched browser
waitImage = Variable that stores the result of image appearance

Used properties

Capture Path = 'C:\\CaptureScreenTest.png'

Related/Similar activities

CaptureBounds, CaptureWindow, CaptureFromFile

CaptureWindow

Overview

This activity card captures a specified object or window on a PC screen.

Application procedures

Double-click No Target on the activity card.
Select a target object or a window to capture.
In the Path field, enter the path and file name to save the captured image.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Capture Path	Text	N	-	N	Enter download path and file name of the captured image. (File format: png)
N	IsOnlyTarget	Toggle button	Y	-	Y	Specify whether to capture the selected window only or to capture the top-most application of the object. (Default: On) - On: Save the specified object as an image file. - Off: Save the window that includes the specified object as an image file.
Y	Target	-	-	-	-	Set properties related to the target (UI object).
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Run Notepad, capture the object for the “File” menu, save it as “C:\\CaptureWindowTest.png,” and then close Notepad.

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureWindow_Test

Used variables

None

Used properties

Capture Path = 'C:\\CaptureWindowTest.png'
IsOnlyTarget = On

Related/Similar activities

CaptureBounds, CaptureScreen, CaptureFromFile

CaptureFromFile

Overview

This activity card captures a specific part of a specified file and saves the captured image on the clipboard or image file.

Application procedures

In the CaptureBounds field, enter the boundary data of the captured area in the form of “x, y, width, and height.”
In the SourceImagePath field, enter the path and name of the image file to capture.
Set the SaveClipboard property to select whether to store the captured area on the clipboard.
In the Properties window, specify other required properties.

Card properties

Card properties
Common	Properties	Type	Required	Unit	Auto-setting	Description
N	CaptureBounds	Text	Y	-	N	Enter the boundary to capture in the order of Left, Top, Width, and Height.
N	SourceImagePath	Text	Y	-	N	Enter the path and name for the image file to capture.
N	SaveClipboard	Toggle button	Y	-	Y	Set whether you want to save the captured area on the clipboard or as a file. (Default: On) - On: Save the captured area on the clipboard. - Off: Save the captured area as a file.
N	Capture Path	Text	Conditional	-	N	Enter download path and file name of the captured image. This property becomes available when you disable the SaveClipboard property.
Y	More Options	-	-	-	-	Common property of the ImageRecognition library.
Y	Description	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Capture the specified area of the image file, save it on the clipboard, and save the image on the clipboard as a separate file. Next, create a new Excel file, open it, and paste the image on the clipboard into cell “A1.”

Excel_NewExcel_1K

Sample file

Menu > Help > Sample > Sample > ImageRecognition > Capture Screen > P_CaptureFromFile_Test

Used variables

browser = Variable to assign the opened Excel document

Used properties

CaptureBounds = '0,100,200,200'
SourceImagePath = Global.ResourcesDir + 'IU.png'
SaveClipboard = On

Remarks

To save the captured image in the clipboard and as a file at the same time, you can enable the SaveClipboard property of the CaptureFromFile activity card and use the ClipboardToFile activity card to save the clipboard image.

Related/Similar activities

CaptureBounds, CaptureScreen, CaptureWindow

GetImageLabel

Overview

This activity card recognizes labels in specified images based on pre-made image files and labeling information.

Application procedures

In the [Output] Output field, enter a variable to store the execution result.
In the LabelNum field, enter the number of labels to read from the image.
Specify an XML file containing labeling information in ImageLabelPath.
In the Properties window, specify other required properties.

Card properties

Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Output	Variable	Y	-	Y	Specify a variable to store the execution result.
N	LabelNum	Number	Y	-	N	Enter the number of labels to read from the image.
N	ImageLabelPath	Text	Y	-	N	Specifies an XML file containing labeling information.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	MORE OPTIONS	-	-	-	-	Common property of the ImageRecognition library.
Y	DESCRIPTION	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

Executes the image file to be recognized, captures a specific area, identifies the image's labels in the area from the labeling information, and outputs it as a message.

Sample file

Menu > Help > Samples > Sample > ImageRecognition > ImageLabel_Sample > P_GetImageLabel_Test

Used variables

readBound = area from which to read the label

Used properties

LabelNum = 3
ImageLabelPath = Global.ResourcesDir + 'Training\\ImageLabel.xml'

Related/similar activities

GetImageLabelOnFile

How to create label information

Create a folder for each image to be recognized as a label, and write the XML format as shown in the figure below. By specifying the label values recognized in each folder, the final label information file is created and saved in the same location as the folder.

GetImageLabelOnFile

Overview

This activity card recognizes labels from pre-made and specified image files based on the labeling information.

Application procedure

In the [Output] Output field, enter a variable to store the execution result.
In the LabelNum field, enter the number of labels to read from the image.
Specify an XML file containing labeling information in ImageLabelPath.
Inthe SourceImagePath field, specify the image file whose label is to be recognized.
In the Properties window, specify other required properties.

Card properties

Common	Properties	Type	Required	Unit	Auto-setting	Description
N	Output	Variable	Y	-	Y	Specify a variable to store the execution result.
N	LabelNum	Number	Y	-	N	Enter the number of labels to read from the image
N	ImageLabelPath	Text	Y	-	N	Specifies an XML file containing labeling information.
N	SourceImagePath	Text	Y	-	N	Specifies the image file whose label is to be recognized.
Y	IMAGE	-	-	-	-	Common property of the ImageRecognition library.
Y	MORE OPTIONS	-	-	-	-	Common property of the ImageRecognition library.
Y	DESCRIPTION	-	-	-	-	Common property of the ImageRecognition library.

Example of utilization

It reads the image file to be recognized, identifies the image's labels in the labeling information, and outputs it as a message.

Sample file

Menu > Help > Samples > Sample > ImageRecognition > ImageLabel_Sample > P_GetImageLabelOnFile_Test

Variable used

None

Used properties

LabelNum = 3
ImageLabelPath = Global.ResourcesDir + 'Training\\ImageLabel.xml'
SourceImagePath = Global.ResourcesDir + 'Label_Test.png'

Related/similar activities

GetImageLabel