Longitudinal data are often segmented by unobserved time-varying factors, which introduce latent heterogeneity at the observation level, in addition to heterogeneity across subjects. We account for this latent structure by a linear mixed hidden Markov model. It integrates subject-specific random effects and Markovian sequences of time-varying effects in the linear predictor. We propose an expectationŰ-maximization algorithm for maximum likelihood estimation, based on data augmentation. It reduces to the iterative maximization of the expected value of a complete likelihood function, derived from an augmented dataset with case weights, alternated with weights updating. In a case study of the Survey on Stress Aging and Health in Russia, the model is exploited to estimate the influence of the observed covariates under unobserved time-varying factors, which affect the cardiovascular activity of each subject during the observation period.