There is at least one supermarket in Hikkaduwa, Jaffna, Kandy and most major towns. Large hotels may also carry some items as well. Cargills, Keels, Arpico, Star United and Laugfs are supermarket chains to look for.
Even a small town will have a convenience store to purchase cow’s milk. In Colombo you can get the familiar plastic bottles from the fridge. Once you travel into the countryside, UHT milk is generally sold in boxed cartons, which you don’t need to refrigerate until open. If there’s no fridge at the hotel/guest-house, you can leave the carton of milk overnight in the sink with cold water. Not all places have a kettle, so if you need warm milk, pack a thermos flask for boiling water (provided by accommodation) to then add to the milk to heat.
Nappies are available from convenience stores. However, you may have to make do with the local brand. Don’t bother with Pampers, it’s not at all the same quality as the brand you’re perhaps used to, instead go for Soft Love nappies (or Drypers). Stock up when you see them and strap a pack to the side of your backpack. Also, get a size bigger than usual as they tend to come up small in Sri Lanka.