The Sitecore XML configuration has some fairly handy things in it, and
ref is one of them. I spent a bit of time playing around with it some time ago; I think that I can provide an answer, but it's quite a long one so buckle in.
With 'how it works'... For fine detail, you can investigate the
Factory.CreateObject(XmlNode, string, bool, IFactoryHelper) and the
Factory.CreateFromReference(XmlNode, string, bool) methods, but in a nutshell Sitecore will attempt to construct the object in a few different ways. One of which is using a given
path attribute. Sitecore does not, at any point, actually fiddle with the underlying XML (e.g. by merging or copying it).
As you'll likely be aware, Sitecore assigns the properties of the object based on the child nodes present in the XML that identifies the type name. There are some exceptions to this case, but by and large this holds so long as the
hint attribute is not used. To provide an example, the following XML (when requested through
Factory) will create an object of type
MyType and then proceed to set the value of the
MyProperty property on that object to "foo":
<someNode type="MyType, MyAssembly">
The population process described above occurs on two separate occasions: first with properties matching the child nodes present in the "referred" XML, and second with the properties matching child nodes present in the "original" XML. This is what allows for configuration such as with the Content Search indexes where index configuration can override a "default".
The interesting caveat here, is that because the XML is never merged, the "original" property values take precedence over the "referred" property values when used normally (i.e. set property X to value Y). This is why you often have to duplicate swathes of the default index configuration if you just want to add a value.
A further caveat to the above is: if you're using
hint attributes to call methods then my understanding is that the method will be called for the "referred" XML and then the "original" XML. I haven't looked too deeply into this - but it makes sense given that it would be tricky to somehow "undo" the unknown effect of a method call. This is your best bet if you want to "merge" your values, and is probably why it only "sometimes" works if you just add in your changes rather than the full set of default values.
In terms of where it can be used, you can only use
ref for XML that governs the instantiation of an object. Typically this means when you're creating an object for an object's property, or if you're making some top-level XML that you yourself intend to use the
Factory class to instantiate.
To clarify the above: you - unfortunately - cannot use it if you're simply constructing an XML-only configuration node and want to reduce repetition, or to copy in commonly used values for properties (e.g. cache sizes for databases). You must be creating an object. I spent quite a bit of time attempting to do this and scratching my head when it didn't work, before digging a bit deeper.
A rather trivial point is that the
ref will only be followed once per object. You cannot use a
ref attribute to point to an element which is in turn another
ref. Although you can obviously use
ref on properties inside of those objects.
Hopefully that wasn't a whole lot of TL;DR and it answers your questions enough; if there's something I've missed or any mistakes please let me know.