Did you create rendering parameters for your custom component?
The rendering parameter template for your component should inherit from /sitecore/templates/Foundation/Experience Accelerator/Rendering Variants/Rendering Parameters/IComponentVariant
(don't forget to also inherit /sitecore/templates/System/Layout/Rendering Parameters/Standard Rendering Parameters
instead of the Standard Template).

Using this base template will give you the variants dropdown in the rendering properties. By the way, SXA has a similar base template for Styling (/sitecore/templates/Foundation/Experience Accelerator/Presentation/Rendering Parameters/IStyling
).
https://ggullentops.blogspot.com/2017/03/variants-sxa-sitecore-rendering.html