{"id":5384,"date":"2025-09-07T22:33:58","date_gmt":"2025-09-07T14:33:58","guid":{"rendered":"https:\/\/blog.coderfan.org\/?p=5384"},"modified":"2025-09-24T11:48:02","modified_gmt":"2025-09-24T03:48:02","slug":"gui-agent-to-control-android-system-implemention","status":"publish","type":"post","link":"https:\/\/blog.coderfan.org\/en\/gui-agent-to-control-android-system-implemention.html","title":{"rendered":"Android\u7cfb\u7edf\u4e2d\u5b9e\u73b0GUI Agent\u8981\u70b9\u89e3\u6790"},"content":{"rendered":"<div class='booster-block booster-read-block'>\n                <div class=\"twp-read-time\">\n                \t<i class=\"booster-icon twp-clock\"><\/i> <span>Read Time:<\/span>3 Minute, 36 Second                <\/div>\n\n            <\/div>\n<p class=\"has-text-align-justify\">\u968f\u7740\u5927\u6a21\u578b\u80fd\u529b\u7684\u589e\u5f3a\uff0c\u901a\u8fc7\u5927\u6a21\u578b\u6765\u63a7\u5236\u6211\u4eec\u7684\u5b89\u5353\u7cfb\u7edf\u5df2\u7ecf\u9010\u6e10\u6210\u4e3a\u73b0\u5b9e\u3002\u57fa\u4e8e\u5927\u6a21\u578b\u5b9e\u73b0\u5bf9\u5b89\u5353\u8bbe\u5907\u7684\u63a7\u5236\uff0c\u53ef\u4ee5\u53d6\u4ee3\u4f20\u7edf\u7684\u57fa\u4e8e\u89c4\u5219\u811a\u672c\u6765\u63a7\u5236\u5b89\u5353\u8bbe\u5907\u7684\u65b9\u5f0f\uff0c\u5177\u6709\u66f4\u597d\u7684\u6027\u80fd\u8868\u73b0\u3001\u4efb\u52a1\u5b8c\u6210\u7387\u4ee5\u53ca\u66f4\u597d\u7684\u6cdb\u5316\u6027\uff0c\u6781\u5927\u5730\u8282\u7701\u5f00\u53d1\u8005\u4e0e\u6d4b\u8bd5\u4eba\u5458\u7684\u65f6\u95f4\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u5f53\u524d\u6bd4\u8f83\u6210\u719f\u5730\u57fa\u4e8e\u5927\u6a21\u578b\u6765\u63a7\u5236\u5b89\u5353\u8bbe\u5907\u7684\u9879\u76ee\u5305\u542b\u7531\u963f\u91cc\u5f00\u53d1\u7684MobileAgent\u9879\u76ee\u4ee5\u53ca\u7b2c\u4e09\u65b9\u4e2a\u4eba\u7684droidrun\u9879\u76ee\uff0c\u524d\u8005\u57fa\u4e8eMobileAgent\u7684\u4e13\u7528\u6a21\u578b\uff0c\u5177\u5907\u7aef\u5230\u7aef\u7684\u80fd\u529b\u3002\u800c\u540e\u8005\u5219\u662f\u5229\u7528OpenAI, Anthropic, Gemini\u7b49\u7b2c\u4e09\u65b9\u5927\u6a21\u578b\u7684\u80fd\u529b\uff0c\u9700\u8981\u989d\u5916\u7684\u524d\/\u540e\u5904\u7406\u3002\u5f53\u7136\uff0c\u622a\u6b62\u5230\u6211\u5199\u8fd9\u7bc7\u535a\u5ba2\u7684\u65f6\u95f4\uff0c\u9762\u58c1\u667a\u80fd\u63a8\u51fa\u7684AgentCPM-GUI\u4e5f\u5df2\u7ecf\u5f00\u6e90\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u5728\u5177\u4f53\u7684\u5b9e\u73b0\u4e0a\uff0c\u4e24\u8005\u4e5f\u5b58\u5728\u5dee\u8ddd\uff0c\u963f\u91ccMobileAgent\u4e3b\u8981\u662f\u57fa\u4e8eADB\u63a7\u5236\u5b89\u5353\u8bbe\u5907\uff0c\u5728\u4f7f\u7528\u65f6\u9700\u8981\u901a\u8fc7USB\u6216\u8005\u65e0\u7ebfADB\u7684\u65b9\u5f0f\u8fde\u63a5\u5b89\u5353\u8bbe\u5907\u4e0ePC\u4e3b\u673a\uff0c\u4ece\u800c\u8fdb\u884c\u8bbe\u5907\u63a7\u5236\uff1b\u800c\u540e\u8005\u4e3b\u8981\u662f\u57fa\u4e8e\u5b89\u5353\u7cfb\u7edf\u63d0\u4f9b\u7684Accessibility API\u8fdb\u884c\u63a7\u5236\uff0c\u9700\u8981\u5b89\u88c5\u7279\u5b9a\u7684APK\u5e76\u5f00\u542f\u76f8\u5e94\u7684\u6743\u9650\u65b9\u80fd\u63a7\u5236\u5b89\u5353\u8bbe\u5907\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u65e0\u8bba\u54ea\u79cd\u5b9e\u73b0\uff0c\u6211\u4eec\u90fd\u9700\u8981\u5177\u5907\u4ee5\u4e0b\u51e0\u79cd\u80fd\u529b\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5c4f\u5e55\u5143\u7d20\u83b7\u53d6\uff1a\u83b7\u53d6\u5f53\u524d\u5c4f\u5e55\u5185\u5bb9\u4fe1\u606f\uff0c\u4ee5\u5224\u65ad\u5f53\u524d\u754c\u9762\u72b6\u6001\uff0c\u4e3a\u4e0b\u4e00\u6b65\u51b3\u7b56\u63d0\u4f9b\u8f93\u5165<\/li>\n\n\n\n<li>\u6a21\u62df\u89e6\u63a7\u8f93\u5165\uff1a\u5f53\u6211\u4eec\u9700\u8981\u6267\u884c\u76f8\u5e94\u7684\u52a8\u4f5c\u65f6\uff0c\u9700\u8981\u8fdb\u884c\u83dc\u5355\u70b9\u51fb\u3001\u6ed1\u52a8\uff0c\u6216\u8005\u89e6\u53d1\u8f6f\u6309\u952e<\/li>\n\n\n\n<li>\u6587\u672c\u5185\u5bb9\u8f93\u5165\uff1a\u6587\u672c\u8f93\u5165\u5c5e\u4e8e\u57fa\u7840\u7684\u4ea4\u4e92,\u6d89\u53ca\u5230\u641c\u7d22\u3001\u804a\u5929\u7b49\u573a\u666f\u65f6\u9700\u8981\u652f\u6301<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-justify\">\u6bcf\u4e00\u79cd\u80fd\u529b\u90fd\u670d\u52a1\u4e8e\u7279\u5b9a\u7684\u76ee\u7684\uff0c\u5728\u5177\u4f53\u843d\u5730\u65f6\u91c7\u7528\u7684\u65b9\u6848\u4e5f\u53ef\u591a\u6837\u3002\u4eca\u5929\u8fd9\u7bc7\u6587\u7ae0\u5c06\u6839\u636e\u5de5\u4f5c\u65e5\u5e38\u4e2d\u63a5\u89e6\u5230\u7684\u5185\u5bb9\u8bb2\u8bb2\u5982\u4f55\u5b9e\u73b0\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u5c4f\u5e55\u5143\u7d20\u83b7\u53d6<\/h3>\n\n\n\n<p class=\"has-text-align-justify\">\u5c4f\u5e55\u5143\u7d20\u83b7\u53d6\u901a\u5e38\u4f5c\u4e3a\u7b2c\u4e00\u6b65\uff0c\u4e5f\u662f\u76f8\u5f53\u5173\u952e\u7684\u4e00\u6b65\u3002\u5728\u5177\u4f53\u843d\u5730\u5b9e\u65bd\u4e0a\uff0c\u4ee5MobileAgent\u4e3a\u4f8b\uff0c\u901a\u8fc7ADB\u547d\u4ee4screencap\u76f4\u63a5\u622a\u53d6\u5f53\u524d\u5c4f\u5e55\u7684\u5185\u5bb9\uff0c\u8fd9\u91cc\u7684\u5185\u5bb9\u5c31\u662f\u4eba\u773c\u53ef\u89c1\u7684\u5185\u5bb9\u3002\u5c3d\u7ba1Android\u7684\u56fe\u5f62\u7ed8\u5236\u7cfb\u7edf\u5b58\u5728\u4e0d\u540c\u7684\u7a97\u53e3\u5c42\u7ea7\uff0c\u4f46\u6700\u7ec8\u90fd\u4f1a\u7ecf\u8fc7SurfaceFlinger\u5230HWC\u8fdb\u884c\u5408\u6210\u9001\u663e\uff0c\u800cscreencap\u622a\u53d6\u5f97\u5185\u5bb9\u5b9e\u9645\u4e0a\u5c31\u662fSurfaceFlinger\u4e2d\u8fdb\u884c\u5408\u6210\u540e\u7684\u56fe\u50cf\u5185\u5bb9\u3002\u65e0\u8bba\u662f\u5e94\u7528\u7684Activity\u3001SystemUI\u7ec4\u4ef6\uff08\u72b6\u6001\u680f\u3001\u5bfc\u822a\u680f\u7b49\uff09\u8fd8\u662f\u4e00\u4e2a\u5355\u72ec\u7684dialog\uff0c\u90fd\u4f1a\u8fdb\u884c\u5408\u6210\u663e\u793a\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u9664\u4e86\u4f7f\u7528ADB\u547d\u4ee4screencap\uff0c\u6211\u4eec\u4e5f\u53ef\u4ee5\u4f7f\u7528Accessibility API\u6765\u5bf9\u5b89\u5353\u7cfb\u7edf\u7684\u5c4f\u5e55\u89c6\u56fe\u4fe1\u606f\u8fdb\u884c\u83b7\u53d6\uff0c\u8fd9\u91cc\u4e0escreencap\u4e0d\u540c\u7684\u662f\u6211\u4eec\u53ef\u4ee5\u83b7\u53d6\u5f53\u524d\u754c\u9762Layout\u4e0b\u7684XML\u5e03\u5c40\u4fe1\u606f\uff0c\u5f53\u7136\u4e5f\u53ef\u4ee5\u8fdb\u884c\u622a\u56fe\uff0c\u5177\u4f53\u53ef\u53c2\u8003\u5982\u4e0b\u4ee3\u7801\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"kotlin\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">\/\/\u5f53\u63a5\u6536\u5230\u754c\u9762\u53d8\u5316\u7684\u4e8b\u4ef6\u65f6dump AccessibilityNodeInfo\noverride fun onAccessibilityEvent(event: AccessibilityEvent?) {\n   Log.i(TAG, \"Event: ${event?.eventType}, ${event?.text}\")\n   \/\/ \u5f53\u7a97\u53e3\u5185\u5bb9\u53d8\u5316\u65f6\u5bfc\u51faView\u6811\n   if (event?.eventType == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED) {\n            val rootNode = rootInActiveWindow\n            if (rootNode != null) {\n                exportViewTreeToXml(rootNode)\n            }\n   }\n}\n\n\/\/\u5e8f\u5217\u5316\u4e3aJson\nprivate fun exportViewTreeToXml(root: AccessibilityNodeInfo) {\n        try {\n            val file = File(filesDir, \"view_tree.xml\")\n            val fos = FileOutputStream(file)\n            val serializer = Xml.newSerializer()\n            serializer.setOutput(fos, \"UTF-8\")\n            serializer.startDocument(\"UTF-8\", true)\n            serializer.startTag(null, \"ViewTree\")\n            dumpNodeToXml(root, serializer)\n            serializer.endTag(null, \"ViewTree\")\n            serializer.endDocument()\n            fos.close()\n            Log.i(TAG, \"View tree exported to: ${file.absolutePath}\")\n        } catch (e: Exception) {\n            Log.e(TAG, \"Failed to export view tree\", e)\n        }\n}\n\/\/\u6309\u7167\u4e0d\u540c\u5b57\u6bb5\u8fdb\u884cdump\nprivate fun dumpNodeToXml(node: AccessibilityNodeInfo, serializer: org.xmlpull.v1.XmlSerializer) {\n        serializer.startTag(null, \"Node\")\n        serializer.attribute(null, \"class\", node.className?.toString() ?: \"\")\n        serializer.attribute(null, \"text\", node.text?.toString() ?: \"\")\n        serializer.attribute(null, \"contentDescription\",    node.contentDescription?.toString() ?: \"\")\n        serializer.attribute(null, \"resourceId\", node.viewIdResourceName ?: \"\")\n        serializer.attribute(null, \"clickable\", node.isClickable.toString())\n        serializer.attribute(null, \"focusable\", node.isFocusable.toString())\n        serializer.attribute(null, \"enabled\", node.isEnabled.toString())\n        \/\/ \u9012\u5f52\u904d\u5386\u5b50\u8282\u70b9\n        for (i in 0 until node.childCount) {\n            val child = node.getChild(i)\n            if (child != null) {\n                dumpNodeToXml(child, serializer)\n                child.recycle()\n            }\n        }\n        serializer.endTag(null, \"Node\")\n}<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u4e0a\u9762\u7684\u4ee3\u7801\u793a\u4f8b\u4f7f\u7528\u65e0\u969c\u788d\u670d\u52a1\u63d0\u4f9b\u7684\u80fd\u529b\u5bf9\u754c\u9762\u7684Layout\u5e03\u5c40\u4fe1\u606f\u8fdb\u884c\u4e86dump\uff0c\u5927\u6a21\u578b\u53ef\u4ee5\u5206\u6790\u8fd9\u4e9b\u6570\u636e\u540e\u6765\u7406\u89e3\u5b89\u5353\u7cfb\u7edf\u7684\u754c\u9762\u5143\u7d20\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u9664\u4e86\u83b7\u53d6Layout\u5e03\u5c40\u4fe1\u606f\uff0c\u6211\u4eec\u4e5f\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528\u65e0\u969c\u788d\u670d\u52a1\u6765\u8fdb\u884c\u622a\u56fe\uff0c\u8fd9\u91cc\u6211\u4eec\u5176\u5b9e\u4f7f\u7528\u7684\u662f\u65e0\u969c\u788d\u670d\u52a1\u7684\u63a5\u53e3\u2014\u2014takeScreenshot\uff0c\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"kotlin\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">public void takeScreenshot (int displayId, \n                Executor executor, \n                AccessibilityService.TakeScreenshotCallback callback)\n\n\/\/\u622a\u56fe\u5b9e\u73b0\n@RequiresApi(Build.VERSION_CODES.R)\nfun takeScreenshotExample() {\n        \/\/ \u8c03\u7528 takeScreenshot\n        val screenshotCallback = object : AccessibilityService.ScreenshotCallback() {\n            override fun onSuccess(result: ScreenshotResult) {\n                val bitmap: Bitmap? = result.screenshot\n                bitmap?.let { saveBitmap(it) }\n                Log.d(TAG, \"Screenshot success!\")\n            }\n\n            override fun onFailure(errorCode: Int) {\n                Log.e(TAG, \"Screenshot failed, errorCode=$errorCode\")\n            }\n        }\n\n        \/\/ \u5ef6\u8fdf\u622a\u56fe\u6216\u8005\u89e6\u53d1\u622a\u56fe\u903b\u8f91\n        takeScreenshot(screenshotCallback, mainExecutor)\n}\n<\/code><\/pre>\n\n\n\n<p>\u8fd9\u91cc\u6709\u4e24\u4e2a\u6ce8\u610f\u70b9\uff1a<\/p>\n\n\n\n<p>1.takeScreenshot\u53ea\u652f\u6301Android12\u53ca\u4ee5\u4e0a\u7248\u672c;<\/p>\n\n\n\n<p class=\"has-text-align-justify\">2.\u4f7f\u7528\u65e0\u969c\u788d\u670d\u52a1\u9700\u8981\u58f0\u660eAccessibilityService_can<mark>TakeScreenshot<\/mark><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u6a21\u62df\u89e6\u63a7\u8f93\u5165<\/h3>\n\n\n\n<p class=\"has-text-align-justify\">\u6a21\u62df\u89e6\u63a7\u8f93\u5165\u5176\u5b9e\u5c31\u662f\u6a21\u62df\u4eba\u7c7b\u4f7f\u7528\u65f6\u7684\u70b9\u51fb\u3001\u6ed1\u52a8\u3001\u62d6\u52a8\u7b49\uff0c\u4e5f\u5305\u62ec\u8f6f\u6309\u952e\u89e6\u6478\u3002\u4ee5MobileAgent\u7684\u5b9e\u73b0\u4e3a\u4f8b\uff0c\u6a21\u62df\u4e8b\u4ef6\u8f93\u5165\u90fd\u91c7\u7528ADB\u7684\u65b9\u5f0f\u5b8c\u6210\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\"><strong>Click<\/strong>\uff1a\u70b9\u51fb\u662f\u6307\u4eba\u7c7b\u624b\u6307\u64cd\u4f5c\u5c4f\u5e55\u65f6\u77ed\u65f6\u95f4\u4ece\u6309\u4e0b\u5230\u62ac\u8d77\u7684\u8fd9\u4e48\u4e00\u4e2a\u8fc7\u7a0b\uff0c\u5728\u7cfb\u7edf\u5185\u6838\u5c42\u8fd9\u7c7b\u4e8b\u4ef6\u4ee5input event\u4e8b\u4ef6\u7684\u5f62\u5f0f\u8fdb\u884c\u4e0a\u62a5\uff0c\u6211\u4eec\u901a\u8fc7getevent -l\u547d\u4ee4\u8fdb\u884c\u67e5\u770b\uff0c\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">EV_ABS       ABS_MT_POSITION_X    00000320   # X = 800\nEV_ABS       ABS_MT_POSITION_Y    000005dc   # Y = 1500\nEV_KEY       BTN_TOUCH            DOWN\nEV_SYN       SYN_REPORT           00000000\n...\nEV_KEY       BTN_TOUCH            UP\nEV_SYN       SYN_REPORT           00000000\n<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u5176\u4e2d\u6bcf\u4e00\u884c\u7684\u5185\u5bb9\u90fd\u4f9d\u7167type\u3001code\u3001value\u7684\u5f62\u5f0f\u8fdb\u884c\u663e\u793a\u3002ABS_MT_POSITION_X\u4e0eABS_MT_POSITION_Y \u4ee3\u8868\u89e6\u70b9\u7684X,Y\u5750\u6807\uff0cvalue\u5373\u5bf9\u5e94\u7684\u53d6\u503c\u3002\u5728\u901a\u8fc7ADB\u8fdb\u884c\u6a21\u62df\u65f6\uff0c\u6211\u4eec\u540c\u6837\u9700\u8981\u8fd9\u4e24\u4e2a\u53c2\u6570\u6765\u786e\u5b9a\u6211\u4eec\u5177\u4f53\u70b9\u51fb\u7684\u4f4d\u7f6e\u3002\u8fd9\u91cc\u6211\u4eec\u901a\u8fc7input tap\u547d\u4ee4\u8fdb\u884c\u6a21\u62df\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">#\u6a21\u62df\u70b9\u51fb\uff1a\u70b9\u51fbx\u5750\u6807145\uff0cy\u5750\u6807\u4e3a569\u7684\u4f4d\u7f6e\ninput tap 145 569<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u9664\u4e86\u8fd9\u7c7b\u7b80\u5355\u7684\u70b9\u51fb\uff0c\u5728\u5b9e\u9645\u64cd\u4f5c\u8fc7\u7a0b\u4e2d\u8fd8\u5b58\u5728\u6ed1\u52a8\u4e0e\u62d6\u62fd\u8fd9\u4e24\u79cd\u6bd4\u8f83\u590d\u6742\u7684\u64cd\u4f5c\uff0c\u901a\u5e38\u53d1\u751f\u5728\u5904\u7406\u8fdb\u5ea6\u6761\u3001\u60ac\u6d6e\u7a97\u8fd9\u6837\u7684\u754c\u9762\u4e2d\u3002\u8fd9\u6837\u7c7b\u4f3c\u7684\u64cd\u4f5c\u4e5f\u53ef\u4ee5\u901a\u8fc7ADB\u547d\u4ee4\u5b8c\u6210\uff0c\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">#\u6a21\u62df\u6ed1\u52a8\uff1a\u4ecex\u5750\u6807325,y\u5750\u6807145\u7684\u4f4d\u7f6e\u5230x\u5750\u6807325,y\u5750\u6807\u4e3a298\u7684\u4f4d\u7f6e\ninput swipe 325 145 325  298\n#\u6a21\u62df\u62d6\u62fd\uff1a\u4ecex\u5750\u6807345,y\u5750\u6807124\u7684\u4f4d\u7f6e\u5230x\u5750\u6807348\u5230y\u5750\u6807459\u7684\u4f4d\u7f6e\ninput draganddrop 345 124 348 459<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u9664\u4e86ADB\uff0c\u6211\u4eec\u4e5f\u53ef\u4ee5\u901a\u8fc7\u65e0\u969c\u788d\u670d\u52a1\u6765\u5b9e\u73b0\u8fd9\u6837\u7684\u64cd\u4f5c\uff0c\u6b64\u65f6\u6211\u4eec\u9700\u8981\u501f\u52a9\u65e0\u969c\u788d\u670d\u52a1\u63d0\u4f9b\u7684\u63a5\u53e3\u2014\u2014dispatchGesture\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"java\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">public final boolean dispatchGesture (GestureDescription gesture, \n                AccessibilityService.GestureResultCallback callback, \n                Handler handler)\n\n\n\/\/click\u70b9\u51fb\nfun click(x: Float, y: Float) {\n        val path = Path().apply {\n            moveTo(x, y)\n        }\n\n        val gestureBuilder = GestureDescription.Builder()\n        gestureBuilder.addStroke(\n            GestureDescription.StrokeDescription(\n                path,\n                0,      \/\/ \u5f00\u59cb\u65f6\u95f4\uff08ms\uff09\n                100     \/\/ \u6301\u7eed\u65f6\u95f4\uff08ms\uff09\uff0c\u8d8a\u77ed\u8d8a\u63a5\u8fd1\u70b9\u51fb\n            )\n        )\n\n        val gesture = gestureBuilder.build()\n        dispatchGesture(gesture, null, null)\n}\n\n\/\/swipe\u6ed1\u52a8\n   fun swipe(x1: Float, y1: Float, x2: Float, y2: Float) {\n    val path = Path().apply {\n        moveTo(x1, y1)\n        lineTo(x2, y2)\n    }\n\n    val gesture = GestureDescription.Builder()\n        .addStroke(GestureDescription.StrokeDescription(path, 0, 500))\n        .build()\n\n    dispatchGesture(gesture, null, null)\n}\n<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u5728\u4f7f\u7528\u65e0\u969c\u788d\u670d\u52a1\u6765\u6267\u884c\u6a21\u62df\u70b9\u51fb\u65f6\uff0c\u5f53\u524d\u5c4f\u5e55\u4e0a\u6b63\u5728\u88ab\u5904\u7406\u7684\u70b9\u51fb\u884c\u4e3a\u90fd\u4f1a\u88ab\u53d6\u6d88\uff0c\u4e0d\u7ba1\u8be5\u884c\u4e3a\u662f\u6765\u81ea\u4e8e\u7528\u6237\u8fd8\u662f\u5176\u4ed6\u63a5\u5165\u65e0\u969c\u788d\u670d\u52a1\u7684\u670d\u52a1\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify has--font-size\">\u9700\u8981\u6ce8\u610f\u7684\u662f\uff0c\u4f7f\u7528\u65e0\u969c\u788d\u670d\u52a1\u6765\u6a21\u62df\u89e6\u63a7\u884c\u4e3a\u9700\u8981\u786e\u4fdd\u6211\u4eec\u7684\u670d\u52a1\u63a5\u5165\u65e0\u969c\u788d\u670d\u52a1\u4e14\u58f0\u660e\u4e86AccessibilityService_canPerformGestures\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u6587\u672c\u5185\u5bb9\u8f93\u5165<\/h3>\n\n\n\n<p class=\"has-text-align-justify\">\u5728\u6211\u4eec\u4e0e\u5b89\u5353\u8bbe\u5907\u4ea4\u4e92\u7684\u8fc7\u7a0b\u4e2d\uff0c\u6587\u672c\u8f93\u5165\u662f\u5fc5\u4e0d\u53ef\u5c11\u7684\u73af\u8282\u3002\u8981\u5b9e\u73b0GUI Agent\u4e5f\u5fc5\u7136\u8981\u5b9e\u73b0\u5bf9\u6587\u672c\u5185\u5bb9\u7684\u8f93\u5165\u3002\u5728\u5177\u4f53\u7684\u5b9e\u73b0\u4e0a\uff0c\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7ADB\u4e5f\u53ef\u4ee5\u901a\u8fc7\u65e0\u969c\u788d\u670d\u52a1\u5b9e\u73b0\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u901a\u8fc7ADB\u6765\u5b9e\u73b0\uff0c\u5b89\u88c5\u7cfb\u7edf\u672c\u8eab\u652f\u6301input text\u547d\u4ee4\uff0c\u5982\u679c\u6211\u4eec\u8f93\u5165\u82f1\u6587\u5185\u5bb9\u53ef\u4ee5\u76f4\u63a5\u4f7f\u7528input text\u547d\u4ee4\uff0c\u4f46\u8fd9\u79cd\u65b9\u5f0f\u5e76\u4e0d\u652f\u6301\u4e2d\u6587\u5b57\u7b26\uff08Unicode\u7f16\u7801\uff09\uff0c\u6b64\u65f6\u6211\u4eec\u9700\u8981\u501f\u52a9\u4e00\u4e2a\u7b2c\u4e09\u65b9\u7684\u8f93\u5165\u6cd5\uff1aADBKeyBoard\uff0c\u5176\u539f\u7406\u662f\u8ba9\u4f7f\u7528\u8005\u901a\u8fc7\u53d1\u9001\u5e7f\u64ad\u7684\u65b9\u5f0f\u6765\u8fdb\u884c\u6587\u672c\u8f93\u5165\u4e0e\u7f16\u8f91\uff0c\u4f7f\u7528\u65b9\u5f0f\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">adb shell am broadcast -a ADB_INPUT_TEXT --es msg '\u4f60\u597d'<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u5728\u4f7f\u7528\u4e4b\u524d\u6211\u4eec\u9700\u8981\u5b89\u88c5ADBKeyBoard\u5e76\u5207\u6362\u8f93\u5165\u6cd5\uff0c\u5177\u4f53\u64cd\u4f5c\u6b65\u9aa4\u5982\u4e0b\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">adb install ADBKeyboard.apk \nadb shell ime enable com.android.adbkeyboard\/.AdbIME\nadb shell ime set com.android.adbkeyboard\/.AdbIME   <\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u8fd9\u79cd\u65b9\u5f0f\u9700\u8981\u989d\u5916\u5b89\u88c5\u5e94\u7528\uff0c\u5982\u679c\u6211\u4eec\u66f4\u8fdb\u4e00\u6b65\uff0c\u53ef\u4ee5\u53c2\u8003ADBKeyBoard\u7684\u6e90\u7801\uff0c\u5728Agent\u670d\u52a1\u5185\u5b9e\u73b0\u4e00\u5957\u7c7b\u4f3c\u7684\uff0c\u5176\u539f\u7406\u4e5f\u5e76\u4e0d\u590d\u6742\uff0c\u7c7b\u4f3c\u4e8e\u5b9e\u73b0\u4e00\u4e2a\u7b80\u5355\u7684\u8f93\u5165\u6cd5\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u5982\u679c\u6211\u4eec\u4e0d\u901a\u8fc7ADB\u65b9\u5f0f\u6765\u5b9e\u73b0\uff0c\u4ea6\u53ef\u4ee5\u901a\u8fc7\u65e0\u969c\u788d\u670d\u52a1\u6765\u5b9e\u73b0\uff0c\u6b64\u65f6\u4e5f\u6709\u4e24\u79cd\u529e\u6cd5\uff1a<\/p>\n\n\n\n<p class=\"has-text-align-justify\">1.\u4f7f\u7528AccessibilityNodeInfo.ACTION_SET_TEXT\uff0c\u53ef\u76f4\u63a5\u4fee\u6539\u53ef\u7f16\u8f91\u63a7\u4ef6\u7684\u6587\u672c\uff0c\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"java\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">fun inputText(node: AccessibilityNodeInfo?, text: String) {\n    if (node == null) return\n\n    if (node.isEditable) {\n        val args = Bundle()\n        args.putCharSequence(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, text)\n        node.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args)\n    } else {\n        \/\/ \u904d\u5386\u5b50\u8282\u70b9\n        for (i in 0 until node.childCount) {\n            inputText(node.getChild(i), text)\n        }\n    }\n}<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">2.\u4f7f\u7528ACTION_FOCUS + ACTION_PASTE\uff0c\u5c06\u526a\u5207\u677f\u7684\u5185\u5bb9\u76f4\u63a5\u8fdb\u884c\u7c98\u8d34<\/p>\n\n\n\n<pre class=\"io-enlighter-pre\"><code class=\"gl\" data-enlighter-language=\"php\" data-enlighter-linenumbers=\"true\" data-enlighter-lineoffset=\"\" data-enlighter-highlight=\"\">fun pasteText(node: AccessibilityNodeInfo?, text: String) {\n    if (node == null) return\n\n    if (node.isEditable) {\n        \/\/ \u5148\u805a\u7126\n        node.performAction(AccessibilityNodeInfo.ACTION_FOCUS)\n        \n        \/\/ \u5c06\u6587\u5b57\u653e\u5165\u526a\u8d34\u677f\n        val clipboard = getSystemService(CLIPBOARD_SERVICE) as android.content.ClipboardManager\n        val clip = android.content.ClipData.newPlainText(\"text\", text)\n        clipboard.setPrimaryClip(clip)\n\n        \/\/ \u7c98\u8d34\n        node.performAction(AccessibilityNodeInfo.ACTION_PASTE)\n    } else {\n        for (i in 0 until node.childCount) {\n            pasteText(node.getChild(i), text)\n        }\n    }\n}<\/code><\/pre>\n\n\n\n<p class=\"has-text-align-justify\">\u5728\u5b9e\u8df5\u8fc7\u7a0b\u4e2d\uff0c\u6211\u53d1\u73b0\u4ec5\u4ec5\u662f\u5b9e\u73b0\u6587\u672c\u8f93\u5165\u53ef\u80fd\u8fd8\u8fbe\u4e0d\u5230\u76ee\u7684\uff0c\u5e94\u4e3a\u6709\u4e9b\u64cd\u4f5c\u8fd8\u662f\u4f9d\u8d56\u8f93\u5165\u6cd5\u7684\uff0c\u6bd4\u5982\u56de\u8f66\u6309\u952e\u3001\u641c\u7d22\u6309\u952e\u7b49\uff0c\u6b64\u65f6\u6211\u4eec\u53ef\u4ee5\u5148\u6267\u884c\u70b9\u51fb\u8f93\u5165\u6846\u6216\u8005\u4f7f\u7528ACTION_FOCUS\u6765\u8c03\u51fa\u952e\u76d8\uff0c\u5728\u6587\u672c\u8f93\u5165\u5b8c\u6210\u540e\u518d\u70b9\u51fb\u952e\u76d8\u5b8c\u6210\u5b8c\u6574\u64cd\u4f5c\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u8fd9\u91cc\u4e5f\u8bb8\u6709\u4eba\u4f1a\u7591\u60d1\uff0c\u4e3a\u4ec0\u4e48\u4e0d\u6309\u7167\u4eba\u7c7b\u7684\u8f93\u5165\u4e60\u60ef\u76f4\u63a5\u64cd\u4f5c\u952e\u76d8\u4ee5\u62fc\u97f3\u8f93\u5165\u6cd5\u7684\u65b9\u5f0f\u8fdb\u884c\u6587\u672c\u8f93\u5165\u5462\uff0c\u4e8b\u5b9e\u4e0a\u8fd9\u79cd\u65b9\u5f0f\u5f53\u7136\u53ef\u4ee5\uff0c\u4e0d\u8fc7\u5176\u6548\u7387\u8f83\u4f4e\uff0c\u5b9e\u8df5\u4e2d\u5e76\u4e0d\u63a8\u8350\u3002<\/p>\n\n\n\n<p class=\"has-text-align-justify\">\u4ee5\u4e0a\u5c31\u662f\u5728Android\u7cfb\u7edf\u4e2d\u5b9e\u73b0GUI Agent\u7684\u8981\u70b9\u89e3\u6790\uff0c\u5e0c\u671b\u80fd\u5bf9\u4f60\u6709\u6240\u5e2e\u52a9\u3002<\/p>\n        <div class=\"booster-block booster-reactions-block\">\n            <div class=\"twp-reactions-icons\">\n                \n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-1\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/happy.svg\" alt=\"Happy\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">\n                        Happy                    <\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">1<\/span>\n                        \n                                                <span class=\"twp-react-percent\"><span>100<\/span> %<\/span>\n                                            <\/div>\n                <\/div>\n\n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-2\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/sad.svg\" alt=\"Sad\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">\n                        Sad                    <\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">0<\/span>\n                                                                        <span class=\"twp-react-percent\"><span>0<\/span> %<\/span>\n                                            <\/div>\n                <\/div>\n\n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-3\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/excited.svg\" alt=\"Excited\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">\n                        Excited                    <\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">0<\/span>\n                                                                        <span class=\"twp-react-percent\"><span>0<\/span> %<\/span>\n                                            <\/div>\n                <\/div>\n\n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-6\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/sleepy.svg\" alt=\"Sleepy\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">\n                        Sleepy                    <\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">0<\/span>\n                        \n                                                <span class=\"twp-react-percent\"><span>0<\/span> %<\/span>\n                                            <\/div>\n                <\/div>\n\n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-4\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/angry.svg\" alt=\"Angry\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">Angry<\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">0<\/span>\n                                                                        <span class=\"twp-react-percent\"><span>0<\/span> %<\/span>\n                        \n                    <\/div>\n                <\/div>\n\n                <div class=\"twp-reacts-wrap\">\n                    <a react-data=\"be-react-5\" post-id=\"5384\" class=\"be-face-icons un-reacted\" href=\"javascript:void(0)\">\n                        <img decoding=\"async\" src=\"https:\/\/blog.coderfan.org\/wp-content\/plugins\/booster-extension\/\/assets\/icon\/surprise.svg\" alt=\"Surprise\">\n                    <\/a>\n                    <div class=\"twp-reaction-title\">Surprise<\/div>\n                    <div class=\"twp-count-percent\">\n                                                    <span style=\"display: none;\" class=\"twp-react-count\">0<\/span>\n                                                                        <span class=\"twp-react-percent\"><span>0<\/span> %<\/span>\n                                            <\/div>\n                <\/div>\n\n            <\/div>\n        <\/div>","protected":false},"excerpt":{"rendered":"<p>\u5b89\u5353\u7cfb\u7edf\u4e2d\u5b9e\u73b0GUI Agent\u7684\u5de5\u7a0b\u5b9e\u8df5\u5fc3\u5f97\u3002<\/p>","protected":false},"author":1,"featured_media":4483,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":0,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[18],"tags":[441,46,439],"class_list":["post-5384","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-operating-system","tag-agent","tag-android","tag-gui"],"blocksy_meta":[],"aioseo_notices":[],"featured_image_urls":{"full":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"thumbnail":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-150x150.jpg",150,150,true],"medium":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-300x158.jpg",300,158,true],"medium_large":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"large":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"1536x1536":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"2048x2048":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"trp-custom-language-flag":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-18x9.jpg",18,9,true]},"post_excerpt_stackable":"<p>\u5b89\u5353\u7cfb\u7edf\u4e2d\u5b9e\u73b0GUI Agent\u7684\u5de5\u7a0b\u5b9e\u8df5\u5fc3\u5f97\u3002<\/p>\n","category_list":"<a href=\"https:\/\/blog.coderfan.org\/en\/category\/operating-system\" rel=\"category tag\">\u64cd\u4f5c\u7cfb\u7edf<\/a>","author_info":{"name":"FranzKafka95","url":"https:\/\/blog.coderfan.org\/en\/author\/yushenglonely95"},"comments_num":"0 comments","featured_image_urls_v2":{"full":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"thumbnail":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-150x150.jpg",150,150,true],"medium":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-300x158.jpg",300,158,true],"medium_large":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"large":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"1536x1536":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"2048x2048":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android.jpg",512,269,false],"trp-custom-language-flag":["https:\/\/blog.coderfan.org\/wp-content\/uploads\/2023\/04\/android-18x9.jpg",18,9,true]},"post_excerpt_stackable_v2":"<p>\u5b89\u5353\u7cfb\u7edf\u4e2d\u5b9e\u73b0GUI Agent\u7684\u5de5\u7a0b\u5b9e\u8df5\u5fc3\u5f97\u3002<\/p>\n","category_list_v2":"<a href=\"https:\/\/blog.coderfan.org\/en\/category\/operating-system\" rel=\"category tag\">\u64cd\u4f5c\u7cfb\u7edf<\/a>","author_info_v2":{"name":"FranzKafka95","url":"https:\/\/blog.coderfan.org\/en\/author\/yushenglonely95"},"comments_num_v2":"0 comments","_links":{"self":[{"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/posts\/5384","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/comments?post=5384"}],"version-history":[{"count":24,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/posts\/5384\/revisions"}],"predecessor-version":[{"id":5501,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/posts\/5384\/revisions\/5501"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/media\/4483"}],"wp:attachment":[{"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/media?parent=5384"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/categories?post=5384"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.coderfan.org\/en\/wp-json\/wp\/v2\/tags?post=5384"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}