Prices of fertilisers up yet we did not let farmers suffer: PM Modi