Early life adversity (ELA) has been associated with an increased risk for diseases in which the immune system plays a critical role. The ELA immune phenotype is characterized by inflammation, impaired cellular immunity, and immunosenescence. However, data on cell-specific immune effects are largely absent. Additionally, stress systems and health behaviors are altered in ELA, which may contribute to the generation of the ELA immune phenotype. The present investigation tested cell-specific immune differences in relationship to the ELA immune phenotype, altered stress parameters, and health behaviors in individuals with ELA (n = 42) and those without a history of ELA (control, n = 73). Relative number and activation status (CD25, CD69, HLA-DR, CD11a, CD11b) of monocytes, NK cells, B cells, T cells, and their main subsets were assessed by flow cytometry. ELAwas associated with significantly reduced numbers of CD69+CD8+ T cells (p = 0.022), increased numbers of HLA-DR+ CD4 and HLA-DR+ CD8 T cells (p < 0.001), as well as increased numbers of CD25+CD8+ T cells (p = 0.036). ELA also showed a trend toward higher numbers of CCR4+ CXCR32CCR6+ CD4 T cells. Taken together, our data suggest an elevated state of immune activation in ELA, in which particularly T cells are affected. Although several aspects of the ELA immune phenotype were related to increased activation markers, neither stress nor health-risk behaviors explained the observed group differences. Thus, the state of immune activation in ELA does not seem to be secondary to alterations in the stress system or health-risk behaviors, but rather a primary effect of early life programming on immune cells.