# Picture Text Recognition ## 1.Take a screenshot In special cases, you can use the shortcut (shortcut version 1.3 or higher) /shortcut/other/screenshot ### Calling * **Request Parameters** | parameter | type | illustrate | Remark | | ---------- | ------------- | ------------------------------- | ------------------------------------------------- | | fun | String | /pic/screenshot | Request function name | | id | String | Device ID | Multiple is not allowed | | binary | Boolean | If true, returns binary data. | By default, json is used for return | | jpg | Boolean | If true, returns the jpg format | Default bmp format | | rect | Integer Array | Leave full screen blank | Left x, Up y, Right x, Down y | | save\_path | String | Save to local path | If left blank, return using memory stream or json | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/screenshot?id=5C:F7:E6:CE:65:22&save_path=D%3A%5CiMouseXP%5CScreenshot%5C' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'content-type: multipart/form-data' \ --form fun=/pic/screenshot \ --form id=5C:F7:E6:CE:65:22 ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/screenshot", "data": { "id":"5C:F7:E6:CE:65:22", "binary":true } }' ``` ### Response When the binary parameter is true, this json will not be returned, and the binary image data will be returned directly ``` { "data": { "code": 0, "id": "6C:72:E7:EA:CD:95", "jpg": false, "binary": false, "rect": "", "image": "D:\\iMouseXP\\Screenshot\\1128935437.bmp", "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/screenshot" } ``` * **Response data field description** | Fields | illustrate | Remark | | ------ | ---------- | ----------------------------------------------------------------------------------------- | | jpg | - | - | | binary | - | - | | rect | - | - | | image | Image data | If save\_path is used, the image path is returned, otherwise the base64 string image data | ## 2.General image search In special cases, you can use the shortcut (shortcut version 1.3 or higher) /shortcut/other/find-image ### Calling This image search mode is exactly the same as DM * **Request Parameters** | parameter | type | illustrate | Remark | | ------------ | ---------------------- | ------------------- | ----------------------------------------------------------------------------------------------- | | fun | String | /pic/find-image | Request function name | | id | String | Device ID | Multiple is not allowed | | target\_img | String | Big picture of goal | Generally, you don't need to fill it in. Our internal screen image will be used by default. | | delta\_color | String | Color Cast | Reference DM | | all | Boolean | Have you found all? | false will only return one of the images in the list, true will find all the images in the list | | direction | String | Find directions | Reference DM | | img\_list | String Array | Thumbnail | Base64 string image list or local path list | | rect | Integer Array | Find Region | Left x, Up y, Right x, Down y | | similarity | Floating point numbers | Similarity | Greater than 0 and less than 1 | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/find-image?id=6C:8D:C1:67:04:29&similarity=0.8&img_list=%5B%22D%3A%5C%5CiMouseXP%5C%5CScreenshot%5C%5C1.bmp%22%5D' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header "content-type: multipart/form-data" \ --form "fun=/pic/find-image" \ --form "id=6C:8D:C1:67:04:29" \ --form "similarity=0.8" \ --form "img_list=[\"D:\\\iMouseXP\\\Screenshot\\\1.bmp\"]" ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/find-image", "data": { "id": "6C:8D:C1:67:04:29", "img_list": ["D:\\iMouseXP\\Screenshot\\1.bmp"], "similarity": 0.85 } }' ``` ### Response ``` { "data": { "code": 0, "id": "6C:8D:C1:67:04:29", "list": [ { "index": 0, "centre": [ 457, 1231 ], "rect": [ 425, 1206, 489, 1257 ], "code": 0 } ], "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/find-image" } ``` * **Response data field description** | Fields | illustrate | Remark | | ------ | --------------------------------------- | ------------------------------------------- | | index | index | The first picture is found, starting from 0 | | centre | Find the center point of the image | x and y | | rect | Find the rectangular region of an image | Left x, Up y, Right x, Down y | ## 3.OpenCV Image Search In special cases, you can use the shortcut (shortcut version 1.3 or higher): /shortcut/other/find-image-cv ### Calling * **Request Parameters** | parameter | type | illustrate | Remark | | ----------- | ---------------------- | ---------------------------------- | ----------------------------------------------------------------------------------------------- | | fun | String | /pic/find-image-cv | Request function name | | id | String | Device ID | Multiple is not allowed | | target\_img | String | Big picture of goal | Generally, you don't need to fill it in. Our internal screen image will be used by default. | | same | Boolean | Do you want to find the same image | If true, all the same images in the image will be found | | all | Boolean | Have you found all | false will only return one of the images in the list, true will find all the images in the list | | img\_list | String Array | Thumbnail | Base64 string image list or local path list | | rect | Integer Array | Find Region | Left x, Up y, Right x, Down y | | similarity | Floating point numbers | Similarity | Greater than 0 and less than 1 | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/find-image-cv?id=6C:8D:C1:67:04:29&similarity=0.8&img_list=%5B%22D%3A%5C%5CiMouseXP%5C%5CScreenshot%5C%5C1.bmp%22%5D' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header "content-type: multipart/form-data" \ --form "fun=/pic/find-image-cv" \ --form "id=6C:8D:C1:67:04:29" \ --form "similarity=0.8" \ --form "img_list=[\"D:\\\iMouseXP\\\Screenshot\\\1.bmp\"]" ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/find-image-cv", "data": { "id": "6C:8D:C1:67:04:29", "img_list": ["D:\\iMouseXP\\Screenshot\\1.bmp"], "similarity": 0.85 } }' ``` ### Response ``` { "data": { "code": 0, "id": "6C:8D:C1:67:04:29", "list": [ { "index": 0, "centre": [ 457, 1231 ], "rect": [ 425, 1206, 489, 1257 ], "code": 0 } ], "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/find-image-cv" } ``` * **Response data Data field description**\ list returns a list of all found text | Fields | illustrate | Remark | | ---------- | --------------------------------------- | ------------------------------------------- | | index | index | The first picture is found, starting from 0 | | centre | Find the center point of the image | x and y | | rect | Find the rectangular region of an image | Left x, Up y, Right x, Down y | | similarity | Similarity | | ## 4.OCR text recognition ### Calling In special cases, shortcuts can be used (shortcut version 1.3 or higher required): /shortcut/other/ocr /shortcut/other/ocr-ex\ There's another feature called /pic/ocr-ex, an enhanced OCR version with more accurate recognition, but it uses more resources. * **Request Parameters** | parameter | type | illustrate | Remark | | ----------- | ------------- | ------------------- | ------------------------------------------------------------------------------------------- | | fun | String | /pic/ocr | Request function name | | id | String | Device ID | Multiple is not allowed | | target\_img | String | Big picture of goal | Generally, you don't need to fill it in. Our internal screen image will be used by default. | | rect | Integer Array | Find Region | Left x, Up y, Right x, Down y | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/ocr?id=6C:8D:C1:67:04:29' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header "content-type: multipart/form-data" \ --form "fun=/pic/ocr" \ --form "id=6C:8D:C1:67:04:29" ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/ocr", "data": { "id": "6C:8D:C1:67:04:29" } }' ``` ### Response ``` { "data": { "code": 0, "id": "6C:8D:C1:67:04:29", "list": [ { "text": "Screen mirror output", "centre": [ 374, 475 ], "rect": [ 270, 457, 478, 494 ], "similarity": 0.82 } ], "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/ocr" } ``` * **Response data Data field description**\ list returns a list of all found text | Fields | illustrate | Remark | | ---------- | ---------------------------- | ----------------------------- | | text | Recognized text | | | centre | The center point of the text | x and y | | rect | Rectangular area of ​​text | Left x, Up y, Right x, Down y | | similarity | Similarity | | ## 5.Find text ### Calling In special cases, shortcuts can be used (shortcut version 1.3 or higher required): /shortcut/other/find-text /shortcut/other/find-text-ex There is also a function called /pic/find-text-ex that uses an enhanced version of OCR that is more accurate but takes up more resources. * **Request Parameters** | parameter | type | illustrate | Remark | | ----------- | ---------------------- | ----------------------------- | ------------------------------------------------------------------------------------------- | | fun | String | /pic/find-text | Request function name | | id | String | Device ID | Multiple is not allowed | | target\_img | String | Big picture of goal | Generally, you don't need to fill it in. Our internal screen image will be used by default. | | rect | Integer Array | Find Region | Left x, Up y, Right x, Down y | | similarity | Floating point numbers | Similarity | It is best to use OCR to recognize it once to see how similar it is. | | contain | Boolean | Contains | | | text | String Array | The text array to be searched | | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/find-text?id=6C:8D:C1:67:04:29&contain=true&similarity=0.8&text=%5B%22D2B6%22%2C%22DCA1%22%5D' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header "content-type: multipart/form-data" \ --form "fun=/pic/find-text" \ --form 'text=["aaa","bbb"]' \ --form "similarity=0.8" \ --form "id=FA:9E:10:3A:FE:E8" ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/find-text", "data": { "id": "FA:9E:10:3A:FE:E8", "text":["时钟","邮件"], "similarity":0.8 } }' ``` ### Response ``` { "data": { "code": 0, "id": "6C:8D:C1:67:04:29", "list": [ { "text": "iMouseXP-D2B6", "centre": [ 283, 581 ], "rect": [ 153, 565, 413, 598 ], "similarity": 0.98 }, { "text": "iMouseXP-DCA1", "centre": [ 281, 685 ], "rect": [ 153, 671, 409, 700 ], "similarity": 0.91 } ], "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/find-text" } ``` * **Response data Data field description**\ list returns a list of all found text | Fields | illustrate | Remark | | ---------- | ---------------------------- | ----------------------------- | | text | Recognized text | | | centre | The center point of the text | x and y | | rect | Rectangular area of ​​text | Left x, Up y, Right x, Down y | | similarity | Similarity | | ## 6.Find Multi Color In special cases, you can use the shortcut (shortcut version 1.3 or higher) /shortcut/other/find-multi-color ### Calling * **Request Parameters** | parameter | type | illustrate | Remark | | ----------- | ----------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------- | | fun | String | /pic/find-multi-color | Request function name | | id | String | Device ID | Multiple is not allowed | | target\_img | String | Big picture of goal | Generally, you don't need to fill it in. Our internal screen image will be used by default. | | same | Boolean | Do you want to find the same image? | If true, all the same color tables in the image will be found | | all | Boolean | Have you found all? | false will return only one of the colors in the list, true will find all the colors in the list | | list | json object array | - | - | Parameter description of json object array in list: | parameter | type | illustrate | Remark | | ------------ | ---------------------- | ------------------------------ | -------------------------------------------------------------------- | | first\_color | String | The color table being searched | - | | rect | Integer Array | Find Region | Left x, top y, right x, bottom y are empty and search in full screen | | similarity | Floating point numbers | Similarity | - | * **httpGet request example** ``` curl --request GET \ --url 'http://192.168.9.9:9911/api/pic/find-multi-color?id=6C:8D:C1:67:04:29&list=%5B%7B%22first_color%22%3A%20%22284%7C1224%7C2AB9F3-000000%2C310%7C1235%7C29ADF1-000000%2C281%7C1225%7C29B8F2-000000%2C288%7C1236%7CEB2A31-000000%2C286%7C1258%7C2693EC-000000%2C262%7C1253%7C2699EE-000000%2C265%7C1228%7C2AB5F2-000000%2C305%7C1232%7CC25B78-000000%2C336%7C1253%7C2997EE-000000%2C301%7C1258%7C2693EC-000000%2C281%7C1245%7CFFFFFC-000000%22%2C%22similarity%22%3A%200.85%7D%5D' ``` * **httpPost request example** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header "Content-Type: multipart/form-data" \ --form "fun=/pic/find-multi-color" \ --form "id=6C:8D:C1:67:04:29" \ --form "list=[{\"first_color\":\"284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000\",\"similarity\":0.85}]" ``` * **httpPost json request example \[websocket can also send the same json request]** ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/find-multi-color", "data": { "id": "60:9A:C1:9A:22:A1", "list": [ { "first_color": "284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000", "similarity": 0.85 } ], "same": false, "all": false } }' ``` ``` curl --request POST \ --url http://192.168.9.9:9911/api \ --header 'Content-Type: application/json' \ --data '{ "fun": "/pic/find-multi-color", "data": { "id": "6C:8D:C1:67:04:29", "list": [ { "rect": [ 0, 0, 0, 0 ], "first_color": "284|1224|2AB9F3-000000,310|1235|29ADF1-000000,281|1225|29B8F2-000000,288|1236|EB2A31-000000,286|1258|2693EC-000000,262|1253|2699EE-000000,265|1228|2AB5F2-000000,305|1232|C25B78-000000,336|1253|2997EE-000000,301|1258|2693EC-000000,281|1245|FFFFFC-000000", "similarity": 0.85 } ], "same": false, "all": false } }' ``` ### Response ``` { "data": { "code": 0, "id": "60:9A:C1:9A:22:A1", "list": [ { "index": 0, "centre": [ 286, 1222 ] } ], "message": "成功" }, "status": 200, "message": "成功", "msgid": 0, "fun": "/pic/find-multi-color" } ``` * **Response data Data field description**\ list returns a list of all found text | Fields | illustrate | Remark | | ------ | ---------------------------------- | -------------------------------------------------------------- | | index | index | The first color search configuration is found, starting from 0 | | centre | Find the center point of the image | x and y | ##
--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://doc.some3c.com/iphone-farm-setup/xp-api-documentation/picture-text-recognition.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.