Update: June 2024
Facebook appears to have recently changed its behaviour (without notice) to require an explicit entry for them in robots.txt. This is the file that tells bots and search engines whether or not they can access content. Additionally we have fixed the same functionality with Twitter/X
User-agent: LinkedInBot
Allow: /
User-agent: FacebookBot
Allow: /
User-agent: Twitterbot
Allow: /
User-agent: *
Disallow: /
We have added this to all clients' robots.txt file (which can be viewed by going to https://front-end-vanity-url/robots.txt)
This change will not affect existing rules telling other search engines such as Google not to index your content.
Update: December 2023
LinkedIn appears to have recently changed its behaviour (without notice) to require an explicit entry for them in robots.txt. This is the file that tells bots and search engines whether or not they can access content.
The required addition is:
User-agent: LinkedInBot
Allow: /
We have added this to all clients' robots.txt file (which can be viewed by going to https://front-end-vanity-url/robots.txt) and LinkedIn links should be working again.
This change will not affect existing rules telling other search engines such as Google not to index your content.
Will search engines index my email campaign content?
There is a robots.txt file at the root of the URL used for your email campaigns with rules that tell search engines not to index the content. You can view this file by going to {MyFrontEndURL}/robots.txt (e.g. https://sites-mycompany.vuturevx.com/robots.txt). The default is:
User-agent: *
Disallow: /
Well-behaved search engines should respect these rules and not index the content.
However, since these URLs are publicly available it is possible for anyone who knows the URL to view the content.
