robots.txt directive sanity check
Judges: A robots.txt directive line. Questions: noul:blocks_important_content - could this block money pages or assets. Action: Hits go to review before the crawl.
CA clean typed decision with low volume or low value, or too vague to act on.3.65
Key facts
- Vertical
- Cross-industry
- Function
- SEO & AI search
- Status
- Idea
- Volume
- occasional
- Value
- meaningful
- Risk
- moderate
- Evidence
- —
- Flags
- None
Build this with a classifier
Define a typed decision with a bounded answer, then evaluate it on examples.
{
"decision_type": "choice",
"question": "Does this input match the decision in “robots.txt directive sanity check”?",
"input": "<input to classify>",
"output": "one label from a fixed list"
}Related use cases
Cite this
Copy a link in your preferred format.