Infants are much more instinctive than us adults. They eat when their body needs food (although it takes time for them to learn to consciously interpret their body's signals). As opposed to us who mostly eat when it is time to eat. And, as others noted, they differ wildly in their eating habits. Some are just not into eating that much.
If your son is otherwise healthy and has more or less normal weight, I would say it is OK. As a wise Hungarian expert wrote: no child is ever going to starve if there is food available. Just make sure that the little he eats is mostly healthy, diverse food, not sweets or junk food snacked in between regular meals.
Also it is important to note that the growth of children is not smooth and continuous: they grow in spurts. During such a growth spurt they may eat 2-3 times more than average for a few weeks or months. However, between spurts, they may eat much less.