In brain scans of almost 1500 people throughout Europe, researchers found that people with 'older-looking' brains had both lower birth weight and genes for smaller brains, compared with those with normal aging brains. Both these factors are present early in life, indicating that your 'brain age' is mostly related to early life influences and not so much on events that happen later in life.