You can tag taxonomy A to the 500 assets and taxonomy B to the other remaining 4500 assets(default assets). 

Thereafter create two user groups one with taxonomy A access and other with taxonomy B access. Then using policy combination feature we can grant access to the users whom require access on both the taxonomies or either of the taxonomies. 

Policy can be configured inside the rules section on the user group polices while creating a user group as below:

 1. Provide Access to Portal.Pages for Asset, Asset Details and Mass Edit page.

[![enter image description here][1]][1]

 2. Provide access to the taxonomy values through the taxonomy relation.In the below example, access is provided to those assets which are in approved state and contains the taxonomy 'Brochure' in the AssetType relation.

[![enter image description here][2]][2]
 
**Note**: Using a policy to grant permissions based on a specific taxonomy value also affects entities that have no value for that taxonomy. This implies when the user has permission to the assets containing taxonomy value as brochure, then user will also have access to assets where the taxonomy value is empty.

Hope this helps!!!


  [1]: https://i.sstatic.net/DVjGb.png
  [2]: https://i.sstatic.net/bHjSf.png