Picture Text Recognition

Some3C.com – A Professional Solution for iOS Development and Testing

1.Take a screenshot

Calling

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/screenshot	Request function name
id	String	Device ID	Multiple is not allowed
binary	Boolean	If true, returns binary data.	By default, json is used for return
jpg	Boolean	If true, returns the jpg format	Default bmp format
rect	Integer Array	Leave full screen blank	Left x, Up y, Right x, Down y
save_path	String	Save to local path	If left blank, return using memory stream or json

• httpGet request example

curl --request GET \
  --url 'http://192.168.9.9:9911/api/pic/screenshot?id=5C:F7:E6:CE:65:22&save_path=D%3A%5CiMouseXP%5CScreenshot%5C'

• httpPost request example

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'content-type: multipart/form-data' \
  --form fun=/pic/screenshot \
  --form id=5C:F7:E6:CE:65:22

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
    "fun": "/pic/screenshot",
    "data": {
        "id":"5C:F7:E6:CE:65:22",
        "binary":true
    }
}'

Response

When the binary parameter is true, this json will not be returned, and the binary image data will be returned directly

{
  "data": {
    "code": 0,
    "id": "6C:72:E7:EA:CD:95",
    "jpg": false,
    "binary": false,
    "rect": "",
    "image": "D:\\iMouseXP\\Screenshot\\1128935437.bmp",
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/screenshot"
}

• Response data field description

Fields	illustrate	Remark
jpg	-	-
binary	-	-
rect	-	-
image	Image data	If save_path is used, the image path is returned, otherwise the base64 string image data

2.General image search

Calling

This image search mode is exactly the same as DM

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/find-image	Request function name
id	String	Device ID	Multiple is not allowed
target_img	String	Big picture of goal	Generally, you don't need to fill it in. Our internal screen image will be used by default.
delta_color	String	Color Cast	Reference DM
all	Boolean	Have you found all?	false will only return one of the images in the list, true will find all the images in the list
direction	String	Find directions	Reference DM
img_list	String Array	Thumbnail	Base64 string image list or local path list
rect	Integer Array	Find Region	Left x, Up y, Right x, Down y
similarity	Floating point numbers	Similarity	Greater than 0 and less than 1

• httpGet request example

curl --request GET \
  --url 'http://192.168.9.9:9911/api/pic/find-image?id=6C:8D:C1:67:04:29&similarity=0.8&img_list=%5B%22D%3A%5C%5CiMouseXP%5C%5CScreenshot%5C%5C1.bmp%22%5D'

• httpPost request example

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header "content-type: multipart/form-data" \
  --form "fun=/pic/find-image" \
  --form "id=6C:8D:C1:67:04:29" \
  --form "similarity=0.8" \
  --form "img_list=[\"D:\\\iMouseXP\\\Screenshot\\\1.bmp\"]"

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/find-image",
     "data": {
          "id": "6C:8D:C1:67:04:29",
          "img_list": ["D:\\iMouseXP\\Screenshot\\1.bmp"],
          "similarity": 0.85
     }
}'

Response

{
  "data": {
    "code": 0,
    "id": "6C:8D:C1:67:04:29",
    "list": [
      {
        "index": 0,
        "centre": [
          457,
          1231
        ],
        "rect": [
          425,
          1206,
          489,
          1257
        ],
        "code": 0
      }
    ],
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/find-image"
}

• Response data field description

Fields	illustrate	Remark
index	index	The first picture is found, starting from 0
centre	Find the center point of the image	x and y
rect	Find the rectangular region of an image	Left x, Up y, Right x, Down y

3.OpenCV Image Search

Calling

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/find-image-cv	Request function name
id	String	Device ID	Multiple is not allowed
target_img	String	Big picture of goal	Generally, you don't need to fill it in. Our internal screen image will be used by default.
same	Boolean	Do you want to find the same image?	If true, all the same images in the image will be found
all	Boolean	Have you found all?	false will only return one of the images in the list, true will find all the images in the list
img_list	String Array	Thumbnail	Base64 string image list or local path list
rect	Integer Array	Find Region	Left x, Up y, Right x, Down y
similarity	Floating point numbers	Similarity	Greater than 0 and less than 1

• httpGet request example

curl --request GET \
  --url 'http://192.168.9.9:9911/api/pic/find-image-cv?id=6C:8D:C1:67:04:29&similarity=0.8&img_list=%5B%22D%3A%5C%5CiMouseXP%5C%5CScreenshot%5C%5C1.bmp%22%5D'

• httpPost request example

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header "content-type: multipart/form-data" \
  --form "fun=/pic/find-image-cv" \
  --form "id=6C:8D:C1:67:04:29" \
  --form "similarity=0.8" \
  --form "img_list=[\"D:\\\iMouseXP\\\Screenshot\\\1.bmp\"]"

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/find-image-cv",
     "data": {
          "id": "6C:8D:C1:67:04:29",
          "img_list": ["D:\\iMouseXP\\Screenshot\\1.bmp"],
          "similarity": 0.85
     }
}'

Response

{
  "data": {
    "code": 0,
    "id": "6C:8D:C1:67:04:29",
    "list": [
      {
        "index": 0,
        "centre": [
          457,
          1231
        ],
        "rect": [
          425,
          1206,
          489,
          1257
        ],
        "code": 0
      }
    ],
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/find-image-cv"
}

• Response data Data field description list returns a list of all found text

Fields	illustrate	Remark
index	index	The first picture is found, starting from 0
centre	Find the center point of the image	x and y
rect	Find the rectangular region of an image	Left x, Up y, Right x, Down y

4.OCR text recognition

Calling

There is also a function called /pic/ocr. The enhanced version of ocr recognition is more accurate, but it takes up more resources.

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/ocr	Request function name
id	String	Device ID	Multiple is not allowed
target_img	String	Big picture of goal	Generally, you don't need to fill it in. Our internal screen image will be used by default.
rect	Integer Array	Find Region	Left x, Up y, Right x, Down y

• httpGet request example

curl --request GET \
  --url 'http://192.168.9.9:9911/api/pic/ocr?id=6C:8D:C1:67:04:29'

• httpPost request example

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header "content-type: multipart/form-data" \
  --form "fun=/pic/ocr" \
  --form "id=6C:8D:C1:67:04:29"

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/ocr",
     "data": {
          "id": "6C:8D:C1:67:04:29"
     }
}'

Response

{
  "data": {
    "code": 0,
    "id": "6C:8D:C1:67:04:29",
    "list": [
      {
        "text": "Screen mirror output",
        "centre": [
          374,
          475
        ],
        "rect": [
          270,
          457,
          478,
          494
        ],
        "similarity": 0.82
      }
    ],
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/ocr"
}

• Response data Data field description list returns a list of all found text

Fields	illustrate	Remark
text	Recognized text
centre	The center point of the text	x and y
rect	Rectangular area of ​​text	Left x, Up y, Right x, Down y
similarity	Similarity

5.Find text

Calling

There is also a function called /pic/find-text-ex that uses an enhanced version of OCR that is more accurate but takes up more resources.

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/find-text	Request function name
id	String	Device ID	Multiple is not allowed
target_img	String	Big picture of goal	Generally, you don't need to fill it in. Our internal screen image will be used by default.
rect	Integer Array	Find Region	Left x, Up y, Right x, Down y
similarity	Floating point numbers	Similarity	It is best to use OCR to recognize it once to see how similar it is.
contain	Boolean	Contains
text	String Array	The text array to be searched

• httpGet request example

curl --request GET \
  --url 'http://192.168.9.9:9911/api/pic/find-text?id=6C:8D:C1:67:04:29&contain=true&similarity=0.8&text=%5B%22D2B6%22%2C%22DCA1%22%5D'

• httpPost request example

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header "content-type: multipart/form-data" \
  --form "fun=/pic/find-text" \
  --form "id=6C:8D:C1:67:04:29"

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/find-text",
     "data": {
          "id": "6C:8D:C1:67:04:29"
     }
}'

Response

{
  "data": {
    "code": 0,
    "id": "6C:8D:C1:67:04:29",
    "list": [
      {
        "text": "iMouseXP-D2B6",
        "centre": [
          283,
          581
        ],
        "rect": [
          153,
          565,
          413,
          598
        ],
        "similarity": 0.98
      },
      {
        "text": "iMouseXP-DCA1",
        "centre": [
          281,
          685
        ],
        "rect": [
          153,
          671,
          409,
          700
        ],
        "similarity": 0.91
      }
    ],
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/find-text"
}

• Response data Data field description list returns a list of all found text

Fields	illustrate	Remark
text	Recognized text
centre	The center point of the text	x and y
rect	Rectangular area of ​​text	Left x, Up y, Right x, Down y
similarity	Similarity

6.Find Multi Color

Calling

• Request Parameters

parameter	type	illustrate	Remark
fun	String	/pic/find-multi-color	Request function name
id	String	Device ID	Multiple is not allowed
target_img	String	Big picture of goal	Generally, you don't need to fill it in. Our internal screen image will be used by default.
same	Boolean	Do you want to find the same image?	If true, all the same color tables in the image will be found
all	Boolean	Have you found all?	false will return only one of the colors in the list, true will find all the colors in the list
list	json object array	-	-

Parameter description of json object array in list:

parameter	type	illustrate	Remark
first_color	String	The color table being searched	-
rect	Integer Array	Find Region	Left x, top y, right x, bottom y are empty and search in full screen
similarity	Floating point numbers	Similarity	-

• httpGet request example

curl --request GET \
--url 'http://192.168.9.9:9911/api/pic/find-multi-color?id=6C:8D:C1:67:04:29&list=%5B%7B%22first_color%22%3A%20%22284%7C1224%7C2AB9F3-000000%2C310%7C1235%7C29ADF1-000000%2C281%7C1225%7C29B8F2-000000%2C288%7C1236%7CEB2A31-000000%2C286%7C1258%7C2693EC-000000%2C262%7C1253%7C2699EE-000000%2C265%7C1228%7C2AB5F2-000000%2C305%7C1232%7CC25B78-000000%2C336%7C1253%7C2997EE-000000%2C301%7C1258%7C2693EC-000000%2C281%7C1245%7CFFFFFC-000000%22%2C%22similarity%22%3A%200.85%7D%5D'

• httpPost request example

curl --request POST \
--url http://192.168.9.9:9911/api \
--header "Content-Type: multipart/form-data" \
--form "fun=/pic/find-multi-color" \
--form "id=6C:8D:C1:67:04:29" \
--form "list=[{\"first_color\":\"284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000\",\"similarity\":0.85}]"

• httpPost json request example [websocket can also send the same json request]

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/find-multi-color",
     "data": {
          "id": "60:9A:C1:9A:22:A1",
          "list": [
               {
                    "first_color": "284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000",
                    "similarity": 0.85
               }
          ],
          "same": false,
          "all": false
     }
}'

curl --request POST \
  --url http://192.168.9.9:9911/api \
  --header 'Content-Type: application/json' \
  --data '{
     "fun": "/pic/find-multi-color",
     "data": {
          "id": "6C:8D:C1:67:04:29",
          "list": [
               {
                    "rect": [
                         0,
                         0,
                         0,
                         0
                    ],
                    "first_color": "284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000",
                    "similarity": 0.85
               }
          ],
          "same": false,
          "all": false
     }
}'

Response

{
  "data": {
    "code": 0,
    "id": "60:9A:C1:9A:22:A1",
    "list": [
      {
        "index": 0,
        "centre": [
          286,
          1222
        ]
      }
    ],
    "message": "成功"
  },
  "status": 200,
  "message": "成功",
  "msgid": 0,
  "fun": "/pic/find-multi-color"
}

• Response data Data field description list returns a list of all found text

Fields	illustrate	Remark
index	index	The first color search configuration is found, starting from 0
centre	Find the center point of the image	x and y