HTTPS surface reachable (robots ✓, sitemap ✗, title ✓)
Why it matters: Public files — robots.txt, sitemap.xml, head meta — are what attackers see first during reconnaissance. Misadvertised paths, stale sitemaps, and verbose generators leak more than intended (ISO 27001 A.8.9).
robots.txt
present
User-agent: *
Allow: */search/login
Allow: */login$
Allow: /login.mhtml$
Disallow: */login
Disallow: */base/logout
Allow: */search/account
Disallow: */account
Disallow: /subscribe_success
Allow: /download_history.mhtml$
Disallow: /download
Disallow: /checkout
Disallow: /account_purchase_history.mhtml
Disallow: /contests
Allow: */search/portfolio
Disallow: */portfolio
Disallow: */editor/template
Disallow: */editor/design
Disallow: */editor/search
Disallow: */video/cart
Disallow: */video/checkout
Disallow: */music/cart
Disallow: */music/user
Disallow: */music/orders
Disallow: /web/checkout
Disallow: /sstk/api/footage/wasabi/*
Disallow: /studioapi/wasabi/*
Disallow: /sstk/api/*
Disallow: /studioapi/*
Disallow: /napi/*
Disallow: /api/*
Disallow: /feed.mhtml
Disallow: /nuance-chat.html
Disallow: /sw/*
Disallow: /base/login
Allow: */search/collections
Allow: */editorial/collections
Disallow: */collections
Disallow: */generate/*
Allow: */generate/$
Disallow: /image-*chatId=*
Disallow: /image-*tool=*
Disallow: /image-*mode=*
Disallow: /blog/?s=*
Disallow: /cart
Disallow: */search/ris/*
Disallow: */search/models/*
Disallow: /search/personalized/*
Disallow: /search/similar/*
# Editor Images
Disallow: */editor/image/*
Allow: */editor/image/$
# Disallow User-Agents
User-agent: CCBot
Disallow: /
# Allow Open-AI crawler
User-agent: OAI-SearchBot
Disallow: /
# Allow blog and landing pages
Allow: */blog/*
Allow: */discover/*
Allow: */explore/*
Allow: */inspire/*
Allow: */featured-collections/*
Allow: */music/featured-collections/*
Allow: */category/*
Allow: */video/featured/*
Allow: */business*
Allow: */terms*
Allow: */pricing/*
Allow: */ai-image-generator/*
Allow: */unlimited/*
# Allow SRPs and ADPs.
Allow: */image-*
Allow: */search/*
# Disallow public gptbot
User-agent: GPTBot
Disallow: /
# Allow Perplexity
User-agent: PerplexityBot
Disallow: /
# Allow blog and landing pages
Allow: */blog/*
Allow: */discover/*
Allow: */explore/*
Allow: */inspire/*
Allow: */featured-collections/*
Allow: */music/featured-collections/*
Allow: */category/*
Allow: */video/featured/*
Allow: */business*
Allow: */terms*
Allow: */pricing/*
Allow: */ai-image-generator/*
Allow: */unlimited/*
# Allow SRPs and ADPs.
Allow: */image-*
Allow: */search/*
# Disallow meta externalagent
User-agent: meta-externalagent
Disallow: /
User-agent: adidxbot # Allow Bing ad bot to do it's thing
Allow: *
User-agent: bingbot # Disallow ADPs for Bing
Allow: */image-illustration/*.jpg # Allow Images that match Image ADP paths
Allow: */image-illustration/*.webp
Allow: */image-photo/*.jpg
Allow: */image-photo/*.webp
Allow: */image-vector/*.jpg
Allow: */image-vector/*.webp
Allow: */search/login
Allow: */login$
Allow: /login.mhtml$
Disallow: */login
Disallow: */base/logout
Allow: */search/account
Disallow: */account
Disallow: /subscribe_success
Allow: /download_history.mhtml$
Disallow: /download
Disallow: /checkout
Disallow: /account_purchase_history.mhtml
Disallow: /contests
Allow: */search/portfolio
Disallow: */portfolio
Disallow: */editor/template
Disallow: */editor/design
Disallow: */editor/search
Disallow: */video/cart
Disallow: */video/checkout
Disallow: */music/cart
Disallow: */music/user
Disallow: */music/orders
Disallow: /web/checkout
Disallow: /sstk/api/footage/wasabi/*
Disallow: /studioapi/wasabi/*
Disallow: /sstk/api/*
Disallow: /studioapi/*
Disallow: /napi/*
Disallow: /api/*
Disallow: /feed.mhtml
Disallow: /nuance-chat.html
Disallow: /sw/*
Disallow: /base/login
Allow: */search/collections
Allow: */editorial/collections
Disallow: */collections
Disallow: */generate/*
Allow: */generate/$
Disallow: */search/ris/*
Disallow: */search/models/*
Disallow: */editor/image/* # Editor Images
Allow: */editor/image/$
Disallow: /image-*chatId=*
Disallow: /image-*tool=*
Disallow: /image-*mode=*
Disallow: /blog/?s=*
Disallow: /cart
Disallow: /sitemaps
Disallow: /sitemaps/*
User-agent: Yandex # Disallow ADPs for Yandex (cannot combine with Bing)
Allow: */image-illustration/*.jpg # Allow Images that match Image ADP paths
Allow: */image-illustration/*.webp
Allow: */image-photo/*.jpg
head
- title
- shutterstock.com
- description
- —
social
no OpenGraph or Twitter meta tags found