The compost shouldn't be washing away as you water. Try watering a bit less vigorously, using a rose on the watering can. You can always top dress with some more compost if necessary.
My experience has been that pot size doesn't make much difference to the size of tomato plants, it's all down to variety and growing conditions. You do need to water and feed more with small pots though. In this hot weather I'm having to water my outdoor potted toms daily. Plenty of flowers and little toms forming though
Paul