This PowerShell query breaks, if the item name contains query operators such as "and" or "child". How should I escape it, so that they don't break it?
$SourceItemPath = $SourceItem.Paths.path
$SourceAncestorItem = Get-Item -Path "master:" -Query "$SourceItemPath/ancestor-or-self::*[@@templatekey='my template key']"
# #
. This would likely be an issue with the Sitecore API rather than SPE, consider verifying with documentation supporting the query format.