No, there is no any restrictions on how many times a visitor can be mapped to a pattern card based on their interactions with various pages and resources on a Sitecore website. Note, that a visitor can only be mapped to one pattern card per profile at a time and as the visitor navigates through a Sitecore website, the pattern card that they are mapped to can be changed on the fly based on their interactions on the website.
How pattern cards work
When a visitor navigates through a Sitecore website, they accumulate the profile values of all the pages and resources that they browse. Sitecore calculates the average score that the visitor has accumulated for each profile and maps the visitor to the pattern card that is the closest match using a measurement called N-Dimensional Euclidean Distance. So, it is important to assign appropriate profile values to each pattern card to ensure that profile values are applied uniformly across your website.
I suspect that in your case you either have too close or too distant profile values between your pattern cards and that is why the mapping engine doesn't respond as per your expectations. Suggest to check on your profile values setup for your pattern cards and make sure that they have the different journey pattern and don't score identically for the same interaction.