How can I apply limits to a specific group of customers(eg, wholesale)

You can accomplish this by using customer tags. Tags you put on registered customer records, not products.

You can use Shopify customer tagging functionality for this task or you can take a look at this option in our "Preferences" settings section. Please see the screenshot below:

No results found