How can I apply limits to a specific group of customers(eg, wholesale)

You can accomplish this by using customer tags. Tags you put on registered customer records, not products. 

You can use Shopify customer tagging functionality for this task or you can take a look at this option in our "Preferences" settings section. Please see the screenshot below:  

Also more information please check the Customer Tags FAQ.