When you click the "Markup" button, what should happen is that the page
sitecore/shell/Applications/Content Manager/Validator.aspx should pop up and all Rich-Text content stored in fields that live on (may be different from those that display on) the page item that you are previewing will be displayed and their markup validated.
This means that if you're previewing a page with the public URL
http://mysite.com/foo/bar and the Sitecore item path
/sitecore/content/sites/My Site/home/foo/bar, then only Rich-Text content that is stored in fields that exist directly on the item
/sitecore/content/sites/My Site/home/foo/bar will be validated.
In other words, rich-text content that is stored in fields on items that are used as rendering/sublayout data sources will not be validated, unless that data source item also happens to be the page item (in such cases, the field will not be validated twice). If you do not have any rich-text content stored in Rich Text fields on the item that you are previewing then the
/sitecore/shell/Applications/Content Manager/Validator.aspx page will be blank.
Opinion on the Markup Button in 'Preview' and 'Experience Editor' Mode
IMHO, I think that the "Markup" button - as it has been designed - is a bit out of place in both the "Preview" and "Experience Editor" modes. This is because the "Markup" button only executes in the scope of a single item, the context item, whereas the rendering of an item's presentation in the "Preview" and "Experience Editor" modes means that you may be rendering and possibly editing content from multiple items.
Since only the Content Editor limits the scope of the item fields visible to those of the "context item" (quoted because in the Content Editor the item is technically the selected item but not the context item), I believe that this is - from a UX perspective - the only location where the "Markup" button's purpose and functionality is truly intuitive.
Potential Improvement 1: Remove the Markup Button from the Toolbars
You could remove the markup button from the Experience Editor and Preview toolbars entirely, but I recommend instead removing it from display via Access Rights. You can make it so that the button only displays for Developers and Administrators, whom you would then train on its functionality. You can do this by assigning Access Rights in the "core" database for the item
/sitecore/content/Applications/WebEdit/Ribbons/WebEdit/Page Editor/Proofing/Markup to restrict access to only members of the developer and administrator roles.
Potential Improvement 2: Replace the Markup Button with an Extended Markup Button
You could replace the code executed by the markup button with your own implementation that validates all rich text content from items that are displayed on the page (limiting the fields may be a bit of a challenge, so I would just do the full items). Improving on this idea, you could create your own Markup button, and only replace the ones that display in the Experience Editor and in "Preview" mode with it. You can find the Markup button to replace it or update its functionality at the path
Potential Improvement 3: Add Custom Experience Editor Markup Buttons
This doesn't really do much to help you if you are in "Preview" mode, but it is still an improvement. You could add your own Custom Experience Editor Button for Markup so so that it displays when you click on a field and can execute for the item that the field comes from. This way, if an author is editing content from a field that is stored on a data source item and clicks the "Markup" button then the author will be validating the item on which the content they were just editing is stored. You can add Custom Experience Editor buttons at the path
/sitecore/content/Applications/WebEdit/Custom Experience Buttons. This article may be a good starting point.