Visual Context: Screenshot Annotations with Auto-Detect
By Edouard · February 16, 2026
The Challenge
Screenshots are invaluable for translators—they show exactly where and how text appears in your application. But annotating screenshots by manually drawing rectangles around each piece of text and searching for the corresponding segment? That’s tedious, especially for screens packed with UI elements.
The Solution: Auto-Detect Segments
Our new auto-detection feature uses OCR (Optical Character Recognition) to automatically find text in your screenshots and create draft annotations. Here’s how it works:
Full Screenshot Detection
Click the Auto-detect Segments button in the sidebar to scan the entire screenshot at once. The system identifies text elements and creates draft annotations for each one—with the detected text pre-filled in the search box.
Region-Based Detection
For more control, draw a selection on the image first. The button changes to Detect text in selection, scanning only the area you’ve selected. This is perfect for focusing on a specific part of a busy interface.
Smart Text Grouping
The detection intelligently groups related text together based on visual proximity. Side-by-side UI elements like “Settings · Profile” are recognized as separate items, each getting their own annotation.
The Workflow
- Upload your screenshot
- Click Auto-detect Segments or draw a selection first for targeted detection
- Review the draft annotations (shown with dashed purple borders)
- Click each draft to open a search dropdown pre-filled with the detected text
- Select the matching segment to convert the draft into a linked annotation
- Dismiss any drafts you don’t need with a single click
What used to take minutes of careful drawing and searching now takes seconds.
Best Practices
- Focus on translatable text: Skip user-generated content like usernames, dates, or dynamic data that isn’t in your translation files
- Use region selection: For complex screens, select specific areas to get more accurate results
- Review before linking: Auto-detection does the heavy lifting, but a quick review ensures accuracy
Try It Today
Visual Context with auto-detection is available now for all projects. Head to your project’s File Manager → Visual Context tab, upload a screenshot, and experience the faster way to provide visual context for your translators.
Learn more in our Visual Context documentation.









