# Computer use without screenshots

A local CoreML model finds every button and UI element on screen, on-device OCR reads the labels, and that text is all Jev gets.

Grade: X — Not a typed-decision job: needs written output, plain rules or lookups would do it, or needs raw image or audio. Also marks ideas that must not be built.. Score: 3.77.

Vertical: Cross-industry.

Function: Agents & dev.

Status: Seen in the wild.

Volume: routine.

Value: minor.

Risk: moderate.

Evidence: described plan.

Flags: check-fit.

- [Source](https://madewithjev.com/builds/computer-use-without-screenshots)
- [Vertical](/verticals/cross_industry/)
- [Function](/functions/agents_dev/)
- [Security alert categorization system](/use-cases/security-alert-categorization-system/)
- [Cross-domain: sensitive-data leak guard on LLM prompts](/use-cases/cross-domain-sensitive-data-leak-guard-on-llm-prompts/)
- [Phishing triage](/use-cases/phishing-triage/)
- [Keystroke-level sensitive-text guard](/use-cases/keystroke-level-sensitive-text-guard/)
- [Confidential message detector](/use-cases/confidential-message-detector/)
- [PII in free text](/use-cases/pii-in-free-text/)