For your question, you can find the values for the FormatBlock under the item
/sitecore/system/Settings/Html Editor Profiles/Rich Text Full/Paragraphs. Out of the box, Sitecore's formatting is pretty boring. But you can add some styling to the selections to make them look better. Similar to the class drop down.
See this link to find out more about styling the formatblock values.
Using the style drop down
I have found it a lot easier to define explicitly the classes that show up in the RTE. It first starts with your custom RTE css file which you set with the setting WebStylesheet.
<setting name="WebStylesheet" value="/styles/rte.min.css" />
Then in the ToolsFile.xml, located in the folder
\sitecore\shell\Controls\Rich Text Editor\ToolsFile.xml, you can define exactly which classes you want in your RTE classes dropdown and what they are called. I find this a lot better than having to name the class to help the editor.
<class name="Red Button" value="a.btn-danger" />
<class name="Blue Button" value="a.btn-blue" />
<class name="Sitecore Light Blue" value=".light-blue" />