I grew up as youngest kid in a family with very strong personalities. I was always too young, too unexperienced, too immature etc, so the others would do everything for me.... resulting in being blamed for being too lazy, not willing to take responsibility etc.. Even nowadays this attitude still persist amongst my siblings. I'm kind of glad that they live on another continent.
Keeping friendships is indeed a problem for me. I have the tendency to withdraw from social life for weeks, months in a row. I often doubt myself whether I can do something both in professional and academic life - I never finished any formal education. I have lost jobs because my job performance nose dived due to depressions. I used to smoke anything that could be smoked (gave it up 15 years ago) and if it wasn't for the meds I would be drinking heavily (I have woken up on park benches in the past).
I never knew whether the low self-esteem was caused by what I did, or the low self-esteem was the cause of what I did. I still don't really know.