Instructions

The XR Computing Inc. XR & AI Era

Welcome to the future of personal computing. XR Workstation transforms your Android phone into a high-performance 3D spatial computer. The era of the traditional Desktop OS is over, replaced by a native, AI-powered spatial flow that spans every computing domain.

Device Compatibility: Optimized for the XR Computing Inc. ecosystem.

1. Daily Instructions

Follow these steps every time you start your workstation:

1. Open Termux app.
2. Enable Wireless Debugging in Developer Options.
3. Run adb connect command in Termux.
4. Open XR Workstation app and Split Screen with Termux.
5. Specify Environments using System File Picker.
6. Run ./XRServer.sh in Termux to launch Android Desktop.
7. Swipe away Termux to enter full spatial mode.

Inside Android Desktop:

• Right Fist (Away): Reset View / Recenter.
• Left M+R+P Bend (Away): Toggle 3D Screen Move Mode (Head & Hand tracking).
• Right M+R+P Bend (Away): Toggle Screen Zoom Mode (0.8m to 1.6m depth).
• Left Thumb Stretched (Away): Toggle 3D Screen Shape (FLAT / CURVED).
• Right Scissor / I+M Bend (Away): Toggle Hand Screen Move Mode (Hand Panning).
• Right Index Bend (Away): Adjust app layout (hold 4s).
• Right Palm Away R+P Bend (Away):Cycle through live 360/VR/2D environments.
• Right Mid+Rng Bend (Away): Voice Input (M+R bend).
• Right Rng+Pnk Bend (Away): Cycle Environment Video playlist (R+P bend).
• Right Ind+Mid+Rng Bend (Away): Adjust 3D Desktop Transparency (I+M+R bend).
• Left Ind+Mid+Rng Bend (Away): Pause/Resume Environment Video (I+M+R bend).
• Right Fist: Toggle type mode or spatial cursor.
• Move Right Hand: Move spatial cursor.
• Thumb-Index Scissor: Freeze / Unfreeze cursor or typing.
• Right Index Bend: Left Click.
• Swipe / Head Track: Panning in Mono / VR mode.
• Right/Left Signs: Input letters, numbers, and symbols (sign mode).
• Left Mid+Rng (Away): Switch input mode (hold 4s).
• Ring Bend: Press a key (keyboard mode).
• Voice & Suggestions: Right hand away (M+R bend) or twist and select.
• VR/Mono Toggle: Left hand index bend (hold 4s) with palm away.
• Help HUD: Left hand away stretched 3s.
• Other Signs: Check KeyMap manual.
• Physical Keyboard: Use anytime for long-form work.

2. System Setup (Mandatory)

To enable the Native Android Desktop and ensure high performance, you must configure your device's Developer Options.

Enable Desktop Mode

1. Go to Settings > About Phone and tap Build Number 7 times to enable Developer Options.
2. Go to Settings > System > Developer options > Window Management and enable "Force activities to be resizable", "Enable non-resizable in multi-window", "Enable freeform windows on secondary display".

Disable Phantom Process Killer (Android 12+)

1. In Developer options, locate "Disable child process restrictions" and turn it ON.
2. If the toggle is missing, you can use this ADB command:

   adb shell "/system/bin/device_config put activity_manager max_phantom_processes 2147483647"

3. Launching Native Android Desktop & 4K AI Workstation

This setup allows you to run Android apps in a high-performance windowed desktop environment floating in 3D space with zero lag. Fully integrated with Gemini AI for a superior spatial productivity flow.

Select Your Resolution

Before launching, you can select your preferred productivity baseline. High resolutions require more processing power but offer significantly sharper text.

• 1080p: Standard resolution, best thermal efficiency.
• 2K: Sharp text, ideal for coding and research.
• 4K: Extreme clarity, recommended for complex multi-window workflows.

Switching: Press CTRL + ALT + 1/2/3 and restart the app to apply the new hardware buffer size.

Prerequisites

1. Install Computer Launcher Pro for desktop mode and opening multiple apps.
2. Install Termux for initialization.

Step A: Link to System Shell (In Termux)

1. Install Termux from your app store.
2. Run: pkg install android-tools -y
3. Enable Wireless Debugging in your phone's Developer Options.
4. Open Wireless Debugging settings and tap "Pair device with pairing code".
5. Use Split Screen to keep the pairing code visible while you type in Termux:

   adb pair 127.0.0.1:[PAIR_PORT] [CODE]
   adb connect 127.0.0.1:[CONNECT_PORT]

6. Download and prepare the engine script:

   curl -O https://www.xrcomputing.tech/scripts/XRServer.sh && chmod +x XRServer.sh

7. Create a launcher shortcut script (DexLauncher.sh) to restart the desktop if it closes:

   echo 'adb shell am start --display 42 -n com.protheme.launcher.winx2.launcher/com.centsol.w10launcher.activity.SplashActivity' > DexLauncher.sh && chmod +x DexLauncher.sh

8. Run the initialization script:

   ./XRServer.sh

9. CRITICAL: After running the script, swipe away the Termux app from your recent apps list (while in split screen or after exiting) to ensure the Android Desktop takes full control of the virtual display.

4. Master the Sign Input System

Sign Input is the world's most advanced spatial typing system, allowing you to control your workstation with high-speed binary finger chords. No physical controllers or keyboards are required.

Core Concepts

• Binary Weights: Each finger has a value: Thumb (1), Index (2), Middle (4), Ring (8), Pinky (16).
• Typing: Combinations of these values form characters (A-Z) and symbols.
• Hand Roles: Right Hand handles Alphabet, Cursor, and Upper-layer Symbols; Left Hand handles Lower-layer Numbers, Symbols, and Global Commands.
• Global Commands: Triggered like sign inputs, but with your palms facing away (back of hand to camera).

Mode Toggles

• CURSOR MODE: Make a Right Hand Fist (Sign 31) to toggle the spatial cursor.
• LAYER TOGGLE: Make a Left Hand Fist (Sign 31) to switch between Alphabet and Numbers/Symbols.

Voice Input & Word Suggestions

• VOICE INPUT SIGN: Perform a Right Palm Away + Middle & Ring Bend (M+R bend, Sign 12 / Value 12) to activate voice input.
• SUGGESTIONS: Suggestion quads remain visible in Sign Mode. Twist your Right Hand back and forth to toggle selection.

Cursor Interaction

Directly control the spatial cursor using intuitive finger postures. Feedback colors indicate state: Green (Move), Blue (Freeze), Yellow (Scroll), Red (Drag), Purple (Select).

• FREEZE / UNFREEZE: Close the gap between your Thumb and Index finger (Scissor sign).
• LEFT CLICK: Perform a quick Index Finger Bend (Sign 2).
• RIGHT CLICK: Perform a quick Middle Finger Bend (Sign 4).
• SCROLL LOCK: Curl Index and Middle together (Sign 6).
• DRAG LOCK: Curl Index and Ring together (Sign 10).
• SELECT MODE: Curl Middle and Ring together (Sign 12).

5. Virtual Keyboard & Spatial Gestures

The virtual workstation supports robust typing and mouse interaction. Global Commands (Camera sees back of hand) function similarly to Sign Inputs but use distinct postures.

Keyboard Interaction (Camera sees Palm)

• TYPE / SELECT: Hover over a key and use a Ring Finger Curl to strike.
• MODE TOGGLE: Bend **Left Middle and Ring** together for 4 seconds to switch modes.
• TYPING FREEZE: Use the **Scissor sign** (Thumb-Index gap) to lock the keyboard hover.
• MODIFIER KEYS: Keys like SHIFT, CTRL, and ALT act as toggles. Strike them once to activate, and strike again to clear the state.

Global Commands (Camera sees Back of Hand / Palm Away)

• SCREEN MOVE MODE: Left Hand Middle, Ring & Pinky Bend (M+R+P bend, Sign 28). Toggles 3D Screen Move Mode.
• SCREEN ZOOM MODE: Right Hand Middle, Ring & Pinky Bend (M+R+P bend, Sign 28). Toggles Screen Zoom Mode (0.8m to 1.6m depth).
• 3D FLAT / CURVED SHAPE: Left Hand Only Thumb Stretched (Sign 30). Toggles 3D Screen Shape between FLAT and CURVED.
• 6 DOF REAL-SPACE ANCHOR: Left Hand Pinky Stretch (Sign 15). Toggles 6 DoF Real-Space Anchor Screen Mode.
• HAND SCREEN MOVE: Right Hand Index & Middle Bend (Sign 6). Toggles Hand Screen Move Mode (Hand Panning).
• VOICE INPUT: Right Hand Middle & Ring Bend (M+R bend, Sign 12).
• SYSTEM FILE PICKER: Left Hand Ring & Pinky Bend (R+P bend, hold 4s) with palm away. Launches the system file picker to select my_scenes.txt and all included local media files before running Android Desktop.
• CYCLE ENVIRONMENT VIDEO: Right Hand Ring & Pinky Bend (R+P bend, Sign 24). Cycles through background 360/VR/2D video playlist.
• DESKTOP TRANSPARENCY: Right Hand Index, Middle & Ring Bend (I+M+R bend, Sign 14). Adjusts 3D Android Desktop transparency.
• VIDEO PAUSE / RESUME: Left Hand Index, Middle & Ring Bend (I+M+R bend, Sign 14). Pauses or resumes environment video playback.
• RESET VR / TOGGLE MR: Right (Reset View) or Left (Toggle Passthrough) hand Fist (Sign 31).
• VR/MONO / ARRANGE: Left (VR/2D) or Right (Arrange Windows) hand Index Bend (hold 4s).
• HELP HUD: Left hand Stretched (hold 3s).

6. 3 Spatial Screen Modes (Curved, Flat & 6 DoF)

XR Workstation offers three flexible 3D screen modes to ensure crystal-clear text viewing and effortless interaction using head/hand tracking and ARCore 6DoF camera pose tracking:

• Curved Screen Mode: Curved spatial screen for comfortable multitasking. Toggle with Left Hand Thumb and Index Stretch.
• Flat Screen Mode: Planar floating screen for classic desktop ergonomics. Toggle with Left Hand Thumb Stretch.
• Curved & Flat Screen Move: Pan screen with head tracking. Toggle hand move with Right Hand Thumb-Index Scissor. Move stretched right hand back forth to zoom. Toggle Zoom off with Right Hand Thumb-Index stretch, then move stretched right hand left/right/up/down to pan screen.
• 6 DoF Real-Space Anchor Mode: Pins your 3D spatial workstation screen at your physical space using 6 DoF camera pose tracking—just like high-end camera-equipped XR glasses. Walk closer to enlarge screen details, walk around or inspect it from any angle, and even view the back of it. Toggle anytime using Left Hand Pinky Stretch.

7. Mono Mode (2D Workspace & Thermal Efficiency)

Mono Mode provides a zero-lag 2D desktop experience, ideal for multi-app multitasking and long-form desk work with multi-touch focal zoom and automatic power saving.

• Direct 2D Framebuffer: Renders directly to the display surface, bypassing offscreen FBO copies and barrel distortion passes for maximum speed and zero GPU lag.
• Automatic Thermal & Battery Savings: Entering Mono Mode automatically pauses camera frame acquisition, MediaPipe AI hand tracking, IMU sensor polling, and background video playback for cool operation.
• Focal Point Multi-Touch Zoom: Pinch with two fingers to zoom smoothly around the exact center of your touch fingers with zero accidental desktop clicks.
• Single-App Fullscreen Auto-Arrange: Signing "Arrange Windows" (Right Hand Index Bend) with 1 app open resizes the app to fill the entire display.
• Toggle Gesture: Hold your Left Hand Index Bend for 3 seconds to switch between 3D VR and Mono 2D modes.

8. Live Environment Maps & Custom Playlists

Immerse yourself in customizable 360, VR, and 2D background video maps while working in 3D space:

• Immersive Environments: Enjoy live environments such as four seasons, nature, cafés, cities, and planets. Some VR videos are specially designed to help restore vision, matching a stationary non-moving reality for maximum comfort.
• Downloading Sample Environments: Download the sample videos (360, VR, and 2D relaxing scenes) and configuration file from xrcomputing.tech/videos/environment/ and my_scenes.txt as my_scenes.txt.
• Custom Local Media Playlists: Select your my_scenes.txt file and all included local media files (MP4 videos or 360/VR/2D images/photos) together using the system file picker (Left Palm Away M+R Bend for 4s) before launching Android Desktop. The selected media files remain saved in persistent app storage for seamless playback across app launches without re-selecting until you update your playlist. Cycle through your custom playlist anytime using Right Palm Away R+P Bend (Sign 24).
• Video Format & Playback We strongly recommend encoding your custom background videos at 15 fps (MP4 format) for smooth playback and thermal/power efficiency. Pause or resume video playback using Left Palm Away I+M+R Bend (Sign 14) or IMR gesture.
• Desktop Transparency & Blending: Cycle 3D desktop transparency levels using Right Palm Away I+M+R Bend (Sign 14) to blend your Android workspace with live virtual environments. For seamless blending, set your phone to Dark Mode & Dark Theme.
• YouTube Integration: Direct YouTube integration comes next (in discussion with the YouTube team as public API lacks overlay support).

9. Keyboard Shortcuts

Use these shortcuts with a physical or Virtual Keyboard (4+ rows) when operating your spatial workstation. For long-form desktop computing (phone fixed at eye level on a desk/neck holder w/ or w/o mini VR glasses and w/ or w/o portable keyboard), these shortcuts provide instant viewport navigation and thermal/battery efficiency toggles: CTRL + ALT + K / H / E, CTRL + Arrow Keys, ALT + Up / Down Arrow.

10. Tips & Safety

• MR Blending Tip: To achieve the best transparency in Mixed Reality, set both your phone's wallpaper and your desktop launcher's theme/background to Pure Black (#000000).
• Health & Safety: Do not use while walking or driving. Take a 5-minute break every 30 minutes of spatial computing to prevent eye strain. [IMPORTANT] We strongly recommend turning on your phone's "Blue Light Filter" or "Eye Comfort Shield" to prevent eye strain during long spatial computing sessions.
• Pro Field Tip: No Wi-Fi? Simply connect to another phone's mobile hotspot to unlock the Wireless Debugging toggle. The XR Engine links via 127.0.0.1, consuming zero mobile data once started. You can use the other phone for mini keyboard too.
• Virtual & Physical Inputs: Use virtual spatial cursor controls and hand-typing for mobility or convenience. Use physical peripherals for mission-critical productivity.